diff options
972 files changed, 45997 insertions, 20098 deletions
diff --git a/.gitignore b/.gitignore index b391ecbe57..d40f49b56f 100644 --- a/.gitignore +++ b/.gitignore @@ -98,6 +98,8 @@ lib/wx/c_src/win32/ lib/wx/priv/win32/ lib/wx/win32/ make/win32/ +make/otp_built +make/otp_doc_built # OSE *.d diff --git a/HOWTO/INSTALL-CROSS.md b/HOWTO/INSTALL-CROSS.md index cb9d12e753..7f852860c9 100644 --- a/HOWTO/INSTALL-CROSS.md +++ b/HOWTO/INSTALL-CROSS.md @@ -537,7 +537,11 @@ When a variable has been set, no warning will be issued. * `erl_xcomp_ose_LM_LCF` - OSE load module linker configuration file -* `erl_xcomp_ose_LM_CONF` - OSE load module default configuration file +* `erl_xcomp_ose_BEAM_LM_CONF` - Beam OSE load module configuration file + +* `erl_xcomp_ose_EPMD_LM_CONF` - EPMD OSE load module configuration file + +* `erl_xcomp_ose_RUN_ERL_LM_CONF` - run_erl_lm OSE load module configuration file Copyright and License --------------------- @@ -559,19 +563,11 @@ under the License. %CopyrightEnd% -Modifying This Document ------------------------ - -Before modifying this document you need to have a look at the -[$ERL_TOP/HOWTO/MARKDOWN.md][] document. - [$ERL_TOP/HOWTO/INSTALL.md]: INSTALL.md - [Building in Git]: INSTALL.md#How-to-Build-and-Install-ErlangOTP_Building-in-Git - [How to Build the Documentation]: INSTALL.md#The-ErlangOTP-Documentation_How-to-Build-the-Documentation + [Building in Git]: INSTALL.md#How-to-Build-and-Install-ErlangOTP + [How to Build the Documentation]: INSTALL.md#How-to-Build-and-Install-ErlangOTP_How-to-Build-the-Documentation [cross configuration variables]: #Currently-Used-Configuration-Variables [DESTDIR]: http://www.gnu.org/prep/standards/html_node/DESTDIR.html - [$ERL_TOP/HOWTO/MARKDOWN.md]: MARKDOWN.md - [?TOC]: true diff --git a/HOWTO/INSTALL-WIN32.md b/HOWTO/INSTALL-WIN32.md index 94d3688f23..0387572dd3 100644 --- a/HOWTO/INSTALL-WIN32.md +++ b/HOWTO/INSTALL-WIN32.md @@ -1018,7 +1018,7 @@ Copyright and License %CopyrightBegin% -Copyright Ericsson AB 2003-2012. All Rights Reserved. +Copyright Ericsson AB 2003-2014. All Rights Reserved. The contents of this file are subject to the Erlang Public License, Version 1.1, (the "License"); you may not use this file except in @@ -1033,15 +1033,7 @@ under the License. %CopyrightEnd% -Modifying This Document ------------------------ - -Before modifying this document you need to have a look at the -[$ERL_TOP/HOWTO/MARKDOWN.md][] document. - - [1]: http://www.erlang.org/faq.html "mailing lists" - [$ERL_TOP/HOWTO/MARKDOWN.md]: MARKDOWN.md [?TOC]: true diff --git a/HOWTO/INSTALL.md b/HOWTO/INSTALL.md index 368947b36c..7a7e63164c 100644 --- a/HOWTO/INSTALL.md +++ b/HOWTO/INSTALL.md @@ -4,174 +4,109 @@ Building and Installing Erlang/OTP Introduction ------------ -This document describes how to build and install Erlang/OTP-%OTP-REL%. You -are advised to read the whole document before attempting to build and install -Erlang/OTP. You can find more information about Open Source Erlang/OTP at: +This document describes how to build and install Erlang/OTP-%OTP-REL%. +Erlang/OTP should be possible to build from source on any Unix/Linux system, +including OS X. You are advised to read the whole document +before attempting to build and install Erlang/OTP. - <http://www.erlang.org/> +The source code can be downloaded from the official site of Erlang/OTP or GitHub. +* <http://www.erlang.org> +* <https://github.com/erlang/otp> -The source code for Erlang/OTP can also be found in a Git repository: - - <http://github.com/erlang/otp> - -Erlang/OTP should be possible to build from source on any Unix system, -including Mac OS X. This document describes how to native compile Erlang/OTP -on Unix. For detailed instructions on how to - -* cross compile Erlang/OTP, see the [$ERL_TOP/HOWTO/INSTALL-CROSS.md][] - document. - -* build Erlang/OTP on Windows, see the [$ERL_TOP/HOWTO/INSTALL-WIN32.md][] - document. +Required Utilities +------------------ - Binary releases for Windows can be found at - <http://www.erlang.org/download.html>. +These are the tools you need in order to unpack and build Erlang/OTP. -Before reading the above mentioned documents you are in any case advised to -read this document first, since it covers building Erlang/OTP in general as -well as other important information. +> *WARNING*: Please have a look at the [Known platform issues][] chapter +> before you start. -Daily Build and Test --------------------- -At Ericsson we have a "Daily Build and Test" that runs on: - -* Solaris 8, 9 - * Sparc32 - * Sparc64 -* Solaris 10 - * Sparc32 - * Sparc64 - * x86 -* SuSE Linux/GNU 9.4, 10.1 - * x86 -* SuSE Linux/GNU 10.0, 10.1, 11.0 - * x86 - * x86\_64 -* openSuSE 11.4 (Celadon) - * x86\_64 (valgrind) -* Fedora 7 - * PowerPC -* Fedora 14 - * x86\_64 -* Gentoo Linux/GNU 1.12.11.1 - * x86 -* Ubuntu Linux/GNU 7.04, 10.04, 10.10, 11.0 - * x86\_64 -* MontaVista Linux/GNU 4.0.1 - * PowerPC -* FreeBSD 8.2 - * x86 -* OpenBSD 5.0 - * x86\_64 -* Mac OS X 10.5.8 (Leopard), 10.7.3 (Lion), 10.9 (Mavericks) - * x86 -* Windows XP SP3, 2003, Vista, 7 - * x86 -* Windows 7 - * x86\_64 +### Unpacking ### -We also have the following "Daily Cross Builds": +* GNU unzip, or a modern uncompress. +* A TAR program that understands the GNU TAR format for long filenames. -* SuSE Linux/GNU 10.1 x86 -> SuSE Linux/GNU 10.1 x86\_64 -* SuSE Linux/GNU 10.1 x86\_64 -> Linux/GNU TILEPro64 +### Building ### -and the following "Daily Cross Build Tests": +* GNU `make` +* Compiler -- GNU C Compiler, `gcc` or the C compiler frontend for LLVM, `clang`. +* Perl 5 +* GNU `m4` -- If HiPE (native code) support is enabled. HiPE can be + disabled using `--disable-hipe` +* `ncurses`, `termcap`, or `termlib` -- The development headers and + libraries are needed, often known as `ncurses-devel`. Use + `--without-termcap` to build without any of these libraries. Note that + in this case only the old shell (without any line editing) can be used. +* `sed` -- Stream Editor for basic text transformation. -* SuSE Linux/GNU 10.1 x86\_64 +#### Building in Git #### -Versions Known NOT to Work --------------------------- +* GNU `autoconf` of at least version 2.59. Note that `autoconf` is not + needed when building an unmodified version of the released source. -* Suse linux 9.1 is shipped with a patched GCC version 3.3.3, having the - rpm named `gcc-3.3.3-41`. That version has a serious optimization bug - that makes it unusable for building the Erlang emulator. Please - upgrade GCC to a newer version before building on Suse 9.1. Suse Linux - Enterprise edition 9 (SLES9) has `gcc-3.3.3-43` and is not affected. +#### Building on OS X #### -* `gcc-4.3.0` has a serious optimizer bug. It produces an Erlang emulator - that will crash immediately. The bug is supposed to be fixed in - `gcc-4.3.1`. +* Xcode -- Download and install via the Mac App Store. + Read about [Building on a Mac][] before proceeding. -* FreeBSD had a bug which caused `kqueue`/`poll`/`select` to fail to detect - that a `writev()` on a pipe has been made. This bug should have been fixed - in FreeBSD 6.3 and FreeBSD 7.0. NetBSD and DragonFlyBSD probably have or - have had the same bug. More information can be found at: +### Installing ### - * <http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/kern/sys_pipe.c> - * <http://lists.freebsd.org/pipermail/freebsd-arch/2007-September/006790.html> +* An `install` program that can take multiple file names. -* `getcwd()` on Solaris 9 can cause an emulator crash. If you have - async-threads enabled you can increase the stack size of the - async-threads as a temporary workaround. See the `+a` command-line - argument in the documentation of `erl(1)`. Without async-threads the - emulator is not as vulnerable to this bug, but if you hit it without - async-threads the only workaround available is to enable async-threads - and increase the stack size of the async-threads. Sun has however - released patches that fixes the issue: - > Problem Description: 6448300 large mnttab can cause stack overrun - > during Solaris 9 getcwd +Optional Utilities +------------------ - More information can be found at: +Some applications are automatically skipped if the dependencies aren't met. +Here is a list of utilities needed for those applications. You will +also find the utilities needed for building the documentation. - * <http://sunsolve.sun.com/search/document.do?assetkey=1-21-112874-40-1&searchclause=6448300> - * <http://sunsolve.sun.com/search/document.do?assetkey=1-21-114432-29-1&searchclause=6448300> +### Building ### -Required Utilities ------------------- +* OpenSSL -- The opensource toolkit for Secure Socket Layer + and Transport Layer Security. + Required for building the application `crypto`. + Further, `ssl` and `ssh` require a working crypto application and + will also be skipped if OpenSSL is missing. The `public_key` + application will available without `crypto`, but the functionality + will be very limited. + + The development package of OpenSSL including the header files are needed as well + as the binary command program `openssl`. At least version 0.9.8 of OpenSSL is required. + Read more and download from <http://www.openssl.org>. +* Oracle Java SE JDK -- The Java Development Kit (Standard Edition). + Required for building the application `jinterface` and parts of `ic` and `orber`. + At least version 1.5.0 of the JDK is required. + + Download from <http://www.oracle.com/technetwork/java/javase/downloads>. + We have also tested with IBM's JDK 1.5.0. +* X Windows -- Development headers and libraries are needed + to build the Erlang/OTP application `gs` on Unix/Linux. +* `flex` -- Headers and libraries are needed to build the flex + scanner for the `megaco` application on Unix/Linux. +* wxWidgets -- Toolkit for GUI applications. + Required for building the `wx` application. At least + version 3.0 of wxWidgets is required. -These are the tools you will need in order to unpack and build Erlang/OTP. + Download from <http://sourceforge.net/projects/wxwindows/files/3.0.0/> + or get it from GitHub: <https://github.com/wxWidgets/wxWidgets> -### Unpacking ### + Further instructions on wxWidgets, read [Building with wxErlang][]. -* GNU unzip, or a modern uncompress. -* A TAR program that understands the GNU TAR format for long filenames - (such as GNU TAR). -### Building ### -* GNU `make` -* `gcc` -- GNU C compiler -* Perl 5 -* GNU `m4` -- If HiPE (native code) support is enabled. HiPE can be - disabled using `--disable-hipe` -* `ncurses`, `termcap`, or `termlib` -- The development headers and - libraries are needed, often known as `ncurses-devel`. Use - `--without-termcap` to build without any of these libraries. Note that - in this case only the old shell (without any line editing) can be used. -* OpenSSL -- Optional, but needed for building the Erlang/OTP applications - `ssl` and `crypto`. You need the "development package" of OpenSSL, i.e. - including the header files. For building the application `ssl` the OpenSSL - binary command program `openssl` is also needed. At least version 0.9.8 - of OpenSSL is required. Can be downloaded from <http://www.openssl.org>. -* Sun Java jdk-1.5.0 or higher -- Optional but needed for building the - Erlang/OTP application `jinterface` and parts of `ic` and `orber`. Can - be downloaded from <http://java.sun.com>. We have also tested IBM's - JDK 1.5.0. -* X Windows -- Optional, but development headers and libraries are needed - to build the Erlang/OTP application `gs` on Unix/Linux. -* `sed` -- There seem to be some problems with some of the `sed` version on - Solaris. Make sure `/bin/sed` or `/usr/bin/sed` is used on the Solaris - platform. -* `flex` -- Optional, headers and libraries are needed to build the `flex` - scanner for the `megaco` application on Unix/Linux. +### Building Documentation ### -#### Building Documentation #### +* `xsltproc` -- A command line XSLT processor. -* `xsltproc` -- XSLT processor. A tool for applying XSLT stylesheets - to XML documents. Can be downloaded from + A tool for applying XSLT stylesheets + to XML documents. Download xsltproc from <http://xmlsoft.org/XSLT/xsltproc2.html>. + * `fop` -- Apache FOP print formatter (requires Java). Can be downloaded from <http://xmlgraphics.apache.org/fop>. -#### Building in Git #### -* GNU `autoconf` of at least version 2.59. Note that `autoconf` is not - needed when building an unmodified version of the released source. - -### Installing ### - -* An `install` program that can take multiple file names. How to Build and Install Erlang/OTP ----------------------------------- @@ -186,65 +121,205 @@ section below before proceeding. ### Unpacking ### -Step 1: Start by unpacking the Erlang/OTP distribution file with your GNU +Start by unpacking the Erlang/OTP distribution file with your GNU compatible TAR program. - $ gunzip -c otp_src_%OTP-VSN%.tar.gz | tar xf - + $ tar -zxf otp_src_%OTP-VSN%.tar.gz # Assuming bash/sh -alternatively: - - $ zcat otp_src_%OTP-VSN%.tar.gz | tar xf - - - -Step 2: Now cd into the base directory (`$ERL_TOP`). +Now change directory into the base directory and set the `$ERL_TOP` variable. $ cd otp_src_%OTP-VSN% + $ export ERL_TOP=`pwd` # Assuming bash/sh ### Configuring ### -Step 3: On some platforms Perl may behave strangely if certain locales are -set, so optionally you may need to set the LANG variable: - - # Bourne shell - $ LANG=C; export LANG +Run the following commands to configure the build: -or + $ ./configure [ options ] - # C-Shell - $ setenv LANG C +> *NOTE*: If you are building Erlang/OTP from git you will need to run `./otp_build autoconf` to generate +> the configure scripts. -Step 4: Run the following commands to configure the build: +By default, Erlang/OTP release will be installed in `/usr/local/{bin,lib/erlang}`. +If you for instance don't have the permission to install in the standard location, + you can install Erlang/OTP somewhere else. For example, to install in +`/opt/erlang/%OTP-VSN%/{bin,lib/erlang}`, use the `--prefix=/opt/erlang/%OTP-VSN%` option. - $ ./configure [ options ] +On some platforms Perl may behave strangely if certain locales are +set. If you get errors when building, try setting the LANG variable: -If you are building it from git you will need to run `./otp_build autoconf` to generate configure file. -By default, Erlang/OTP will be installed in `/usr/local/{bin,lib/erlang}`. -To instead install in `<BaseDir>/{bin,lib/erlang}`, use the -`--prefix=<BaseDir>` option. + $ export LANG=C # Assuming bash/sh -If you upgraded the source with some patch you may need to clean up -from previous builds before the new build. Before doing a `make clean`, -be sure to read the [Pre-built Source Release][] section below. ### Building ### -Step 5: Build the Erlang/OTP package. +Build the Erlang/OTP release. $ make + +### Testing ### + +Before installation you should test whether your build is working properly +by running our smoke test. The smoke test is a subset of the complete Erlang/OTP test suites. +First you will need to build and release the test suites. + + $ make release_tests + +This creates an additional folder in `$ERL_TOP/release` called `tests`. +Now, it's time to start the smoke test. + + $ cd release/tests/test_server + $ $ERL_TOP/bin/erl -s ts install -s ts smoke_test batch -s init stop + +To verify that everything is ok you should open `$ERL_TOP/release/tests/test_server/index.html` +in your web browser and make sure that there are zero failed test cases. + +> *NOTE*: On builds without `crypto`, `ssl` and `ssh` there is a failed test case +> for undefined functions. Verify that the failed test case log only shows calls +> to skipped applications. + ### Installing ### -Step 6: Install then Erlang/OTP package +You are now ready to install the Erlang/OTP release! +The following command will install the release on your system. $ make install -### A Closer Look at the individual Steps ### -Let us go through them in some detail. +### Running ### + +You should now have a working release of Erlang/OTP! +Jump to [System Principles][] for instructions on running Erlang/OTP. + + +### How to Build the Documentation ### + +Make sure you're in the top directory in the source tree. + + $ cd $ERL_TOP + +If you have just built Erlang/OTP in the current source tree, you have +already ran `configure` and do not need to do this again; otherwise, run +`configure`. + + $ ./configure [Configure Args] + +When building the documentation you need a full Erlang/OTP-%OTP-VSN% system in +the `$PATH`. + + $ export PATH=$ERL_TOP/bin:$PATH # Assuming bash/sh + +Build the documentation. -#### Configuring #### + $ make docs + +#### Build Issues #### + +We have sometimes experienced problems with Oracle's `java` running out of +memory when running `fop`. Increasing the amount of memory available +as follows has in our case solved the problem. + + $ export FOP_OPTS="-Xmx<Installed amount of RAM in MB>m" + +More information can be found at +* <http://xmlgraphics.apache.org/fop/0.95/running.html#memory>. + + +### How to Install the Documentation ### + +The documentation can be installed either using the `install-docs` target, +or using the `release_docs` target. + +* If you have installed Erlang/OTP using the `install` target, install + the documentation using the `install-docs` target. Install locations + determined by `configure` will be used. `$DESTDIR` can be used the + same way as when doing `make install`. + + $ make install-docs + +* If you have installed Erlang/OTP using the `release` target, install + the documentation using the `release_docs` target. You typically want + to use the same `RELEASE_ROOT` as when invoking `make release`. + + $ make release_docs RELEASE_ROOT=<release dir> + + +### Accessing the Documentation ### + +After installation you can access the documentation by + +* Reading man pages. Make sure that `erl` is referring to the + installed version. For example `/usr/local/bin/erl`. + Try viewing at the man page for Mnesia + + $ erl -man mnesia + +* Browsing the html pages by loading the page `/usr/local/lib/erlang/doc/erlang/index.html` + or `<BaseDir>/lib/erlang/doc/erlang/index.html` if the prefix option has been used. + + +### How to Install the Pre-formatted Documentation ### + +Pre-formatted [html documentation][] and [man pages][] can be downloaded from +* <http://www.erlang.org/download.html>. + +Extract the html archive in the installation directory. + + $ cd <ReleaseDir> + $ tar -zxf otp_html_%OTP-VSN%.tar.gz + +For `erl -man <page>` to work the Unix manual pages have to be +installed in the same way, i.e. + + $ cd <ReleaseDir> + $ tar -zxf otp_man_%OTP-VSN%.tar.gz + +Where `<ReleaseDir>` is + +* `<PrefixDir>/lib/erlang` if you have installed Erlang/OTP using + `make install`. +* `$DESTDIR<PrefixDir>/lib/erlang` if you have installed Erlang/OTP + using `make install DESTDIR=<TmpInstallDir>`. +* `RELEASE_ROOT` if you have installed using + `make release RELEASE_ROOT=<ReleaseDir>`. + + +Advanced configuration and build of Erlang/OTP +---------------------------------------------- + +If you want to tailor your Erlang/OTP build and installation, please read +on for detailed information about the individual steps. + +### make and $ERL\_TOP ### + +All the makefiles in the entire directory tree use the environment +variable `ERL_TOP` to find the absolute path of the installation. The +`configure` script will figure this out and set it in the top level +Makefile (which, when building, it will pass on). However, when +developing it is sometimes convenient to be able to run make in a +subdirectory. To do this you must set the `ERL_TOP` variable +before you run make. -Step 4 runs a configuration script created by the GNU autoconf utility, which +For example, assume your GNU make program is called `make` and you +want to rebuild the application `STDLIB`, then you could do: + + $ cd lib/stdlib; env ERL_TOP=<Dir> make + +where `<Dir>` would be what you find `ERL_TOP` is set to in the top level +Makefile. + +### otp\_build vs configure/make ### + +Building Erlang/OTP can be done either by using the `$ERL_TOP/otp_build` +script, or by invoking `$ERL_TOP/configure` and `make` directly. Building using +`otp_build` is easier since it involves fewer steps, but the `otp_build` build +procedure is not as flexible as the `configure`/`make` build procedure. The binary +releases for Windows that we deliver are built using `otp_build`. + +### Configuring ### + +The configure script is created by the GNU autoconf utility, which checks for system specific features and then creates a number of makefiles. The configure script allows you to customize a number of parameters; @@ -260,10 +335,10 @@ use the `--prefix` argument like this: `./configure --prefix=<Dir>`. Some of the available `configure` options are: * `--prefix=PATH` - Specify installation prefix. -* `--{enable,disable}-threads` - Thread support (enabled by default if - possible) + +* `--{enable,disable}-threads` - Thread support. This is enabled by default if possible. * `--{enable,disable}-smp-support` - SMP support (enabled by default if - possible) + a usable POSIX thread library or native Windows threads is found) * `--{enable,disable}-kernel-poll` - Kernel poll support (enabled by default if possible) * `--{enable,disable}-hipe` - HiPE support (enabled by default on supported @@ -290,7 +365,7 @@ Some of the available `configure` options are: * `--{with,without}-javac` - Java compiler (without implies that the `jinterface` application won't be built) * `--{enable,disable}-dynamic-ssl-lib` - Dynamic OpenSSL libraries -* `--{enable,disable}-shared-zlib` - Shared zlib library +* `--{enable,disable}-builtin-zlib` - Use the built-in source for zlib. * `--with-ssl=PATH` - Specify location of OpenSSL include and lib * `--{with,without}-ssl` - OpenSSL (without implies that the `crypto`, `ssh`, and `ssl` won't be built) @@ -310,7 +385,7 @@ Some of the available `configure` options are: * `--enable-static-{nifs,drivers}` - To allow usage of nifs and drivers on OSs that do not support dynamic linking of libraries it is possible to statically link nifs and drivers with the main Erlang VM binary. This is done by passing - a comma seperated list to the archives that you want to statically link. e.g. + a comma separated list to the archives that you want to statically link. e.g. `--enable-static-nifs=/home/$USER/my_nif.a`. The path has to be absolute and the name of the archive has to be the same as the module, i.e. `my_nif` in the example above. This is also true for drivers, but then it is the driver name @@ -321,9 +396,9 @@ Some of the available `configure` options are: to configure. * `--without-$app` - By default all applications in Erlang/OTP will be included in a release. If this is not wanted it is possible to specify that Erlang/OTP - should be compiled without that applications, i.e. `--without-wx`. There is - no automatic dependency handling inbetween applications. So if you disable - an application that another depends on, you also have to disable the + should be compiled without one or more applications, i.e. `--without-wx`. There is + no automatic dependency handling between applications. If you disable + an application that another application depends on, you also have to disable the dependant application. * `--enable-dirty-schedulers` - Enable the **experimental** dirty schedulers functionality. Note that the dirty schedulers functionality is experimental, @@ -334,26 +409,138 @@ Some of the available `configure` options are: If you or your system has special requirements please read the `Makefile` for additional configuration information. -#### Building #### -Step 5 builds the Erlang/OTP system. On a fast computer, this will take about -5 minutes. After completion of this step, you should have a working -Erlang/OTP system which you can try by typing `bin/erl`. This should start -up Erlang/OTP and give you a prompt: +### Building ### + +Building Erlang/OTP on a relatively fast computer takes approximately +5 minutes. To speed it up, you can utilize parallel make with the `-j<num_jobs>` option. + + $ export MAKEFLAGS=-j8 # Assuming bash/sh + $ make + +If you've upgraded the source with a patch you may need to clean up from previous +builds before the new build. +Make sure to read the [Pre-built Source Release][] section below before doing a `make clean`. + +#### Within Git #### + +When building in a Git working directory you also have to have a GNU `autoconf` +of at least version 2.59 on your system, because you need to generate the +`configure` scripts before you can start building. + +The `configure` scripts are generated by invoking `./otp_build autoconf` in +the `$ERL_TOP` directory. The `configure` scripts also have to be regenerated +when a `configure.in` or `aclocal.m4` file has been modified. Note that when +checking out a branch a `configure.in` or `aclocal.m4` file may change +content, and you may therefore have to regenerate the `configure` scripts +when checking out a branch. Regenerated `configure` scripts imply that you +have to run `configure` and build again. + +> *NOTE*: Running `./otp_build autoconf` is **not** needed when building +> an unmodified version of the released source. + +Other useful information can be found at our GitHub wiki: +* <http://wiki.github.com/erlang/otp> + +#### OS X (Darwin) #### + +Make sure that the command `hostname` returns a valid fully qualified host +name (this is configured in `/etc/hostconfig`). Otherwise you might experience +problems when running distributed systems. + +If you develop linked-in drivers (shared library) you need to link using +`gcc` and the flags `-bundle -flat_namespace -undefined suppress`. You also +include `-fno-common` in `CFLAGS` when compiling. Use `.so` as the library +suffix. + +If you have Xcode 4.3, or later, you will also need to download +"Command Line Tools" via the Downloads preference pane in Xcode. + +#### Building with wxErlang #### + +If you want to build the `wx` application, you will need to get wxWidgets-3.0 +(`wxWidgets-3.0.0.tar.bz2` from <http://sourceforge.net/projects/wxwindows/files/3.0.0/>) or get it from github with bug fixes: + + $ git clone --branch WX_3_0_branch [email protected]:wxWidgets/wxWidgets.git + +Be aware that the wxWidgets-3.0 is a new release of wxWidgets, it is not as +mature as the old releases and the OS X port still lags behind the other ports. + +Configure and build wxWidgets (on Mavericks - 10.9): - $ bin/erl - Erlang/OTP %OTP-REL% [erts-%ERTS-VSN%] [source] [smp:4:4] [async-threads:0] [kernel-poll:false] + $ ./configure --with-cocoa --prefix=/usr/local + or without support for old versions and with static libs + $ ./configure --with-cocoa --prefix=/usr/local --with-macosx-version-min=10.9 --disable-shared + $ make + $ sudo make install + $ export PATH=/usr/local/bin:$PATH - Eshell V%ERTS-VSN% (abort with ^G) - 1> _ +Check that you got the correct wx-config -#### Installing #### + $ which wx-config && wx-config --version-full -Step 6 is optional. It installs Erlang/OTP at a standardized location (if you -change your mind about where you wish to install you can rerun step 4, -without having to do step 5 again). +Build Erlang/OTP + + $ export PATH=/usr/local/bin:$PATH + $ cd $ERL_TOP + $ ./configure + $ make + $ sudo make install -##### Alternative Installation Procedures ##### + +#### Pre-built Source Release #### + +The source release is delivered with a lot of platform independent +build results already pre-built. If you want to remove these pre-built +files, invoke `./otp_build remove_prebuilt_files` from the `$ERL_TOP` +directory. After you have done this, you can build exactly the same way +as before, but the build process will take a much longer time. + +> *WARNING*: Doing `make clean` in an arbitrary directory of the source +> tree, may remove files needed for bootstrapping the build. +> +> Doing `./otp_build save_bootstrap` from the `$ERL_TOP` directory before +> doing `make clean` will ensure that it will be possible to build after +> doing `make clean`. `./otp_build save_bootstrap` will be invoked +> automatically when `make` is invoked from `$ERL_TOP` with either the +> `clean` target, or the default target. It is also automatically invoked +> if `./otp_build remove_prebuilt_files` is invoked. + +#### How to Build a Debug Enabled Erlang RunTime System #### + +After completing all the normal building steps described above a debug +enabled runtime system can be built. To do this you have to change +directory to `$ERL_TOP/erts/emulator`. + +In this directory execute: + + $ make debug FLAVOR=$FLAVOR + +where `$FLAVOR` is either `plain` or `smp`. The flavor options will +produce a beam.debug and beam.smp.debug executable respectively. The +files are installed along side with the normal (opt) versions `beam.smp` +and `beam`. + +To start the debug enabled runtime system execute: + + $ $ERL_TOP/bin/cerl -debug + +The debug enabled runtime system features lock violation checking, +assert checking and various sanity checks to help a developer ensure +correctness. Some of these features can be enabled on a normal beam +using appropriate configure options. + +There are other types of runtime systems that can be built as well +using the similar steps just described. + + $ make $TYPE FLAVOR=$FLAVOR + +where `$TYPE` is `opt`, `gcov`, `gprof`, `debug`, `valgrind`, or `lcnt`. +These different beam types are useful for debugging and profiling +purposes. + + +### Installing ### * Staged install using [DESTDIR][]. You can perform the install phase in a temporary directory and later move the installation into @@ -435,7 +622,7 @@ without having to do step 5 again). if you want to try the system out, running test suites, etc, before doing the real install without `EXTRA_PREFIX`. -### Symbolic Links in --bindir ### +#### Symbolic Links in --bindir #### When doing `make install` and the default installation prefix is used, relative symbolic links will be created from `/usr/local/bin` to all public @@ -449,156 +636,10 @@ passed to `configure`. One can force relative, or absolute links by passing phase. Note that such a request might cause a failure if the request cannot be satisfied. -### Pre-built Source Release ### - -The source release is delivered with a lot of platform independent -build results already pre-built. If you want to remove these pre-built -files, invoke `./otp_build remove_prebuilt_files` from the `$ERL_TOP` -directory. After you have done this, you can build exactly the same way -as before, but the build process will take a much longer time. - -> *WARNING*: Doing `make clean` in an arbitrary directory of the source -> tree, may remove files needed for bootstrapping the build. -> -> Doing `./otp_build save_bootstrap` from the `$ERL_TOP` directory before -> doing `make clean` will ensure that it will be possible to build after -> doing `make clean`. `./otp_build save_bootstrap` will be invoked -> automatically when `make` is invoked from `$ERL_TOP` with either the -> `clean` target, or the default target. It is also automatically invoked -> if `./otp_build remove_prebuilt_files` is invoked. - -### Building in Git ### - -When building in a Git working directory you also have to have a GNU `autoconf` -of at least version 2.59 on your system, because you need to generate the -`configure` scripts before you can start building. - -The `configure` scripts are generated by invoking `./otp_build autoconf` in -the `$ERL_TOP` directory. The `configure` scripts also have to be regenerated -when a `configure.in` or `aclocal.m4` file has been modified. Note that when -checking out a branch a `configure.in` or `aclocal.m4` file may change -content, and you may therefore have to regenerate the `configure` scripts -when checking out a branch. Regenerated `configure` scripts imply that you -have to run `configure` and build again. - -> *NOTE*: Running `./otp_build autoconf` is **not** needed when building -> an unmodified version of the released source. - -Other useful information can be found at our github wiki: -<http://wiki.github.com/erlang/otp> - -### make and $ERL\_TOP ### - -All the makefiles in the entire directory tree use the environment -variable `ERL_TOP` to find the absolute path of the installation. The -`configure` script will figure this out and set it in the top level -Makefile (which, when building, it will pass on). However, when -developing it is sometimes convenient to be able to run make in a -subdirectory. To do this you must set the `ERL_TOP` variable -before you run make. - -For example, assume your GNU make program is called `make` and you -want to rebuild the application `STDLIB`, then you could do: - - $ cd lib/stdlib; env ERL_TOP=<Dir> make - -where `<Dir>` would be what you find `ERL_TOP` is set to in the top level -Makefile. - -The Erlang/OTP Documentation ----------------------------- - -### How to Build the Documentation ### - -Before you can build the documentation you need to either [native build][] -or [cross build][] the Erlang/OTP system. After this you can build the -documentation as follows. - - $ cd $ERL_TOP - $ make docs - -The documentation can be installed either using the `install-docs` target, -or using the `release_docs` target. - -* If you have installed Erlang/OTP using the `install` target, install - the documentation using the `install-docs` target. Install locations - determined by `configure` will be used. `$DESTDIR` can be used the - same way as when doing `make install`. - - $ make install-docs - -* If you have installed Erlang/OTP using the `release` target, install - the documentation using the `release_docs` target. You typically want - to use the same `RELEASE_ROOT` as when invoking `make release`. - - $ make release_docs RELEASE_ROOT=<release dir> -#### Build Issues #### +### Running ### -We have sometimes experienced problems with Sun's `java` running out of -memory when running `fop`. Increasing the amount of memory available -as follows has in our case solved the problem. - - $ export FOP_OPTS="-Xmx<Installed amount of RAM in MB>m" - -More information can be found at -<http://xmlgraphics.apache.org/fop/0.95/running.html#memory>. - -### How to Install the Pre-formatted Documentation ### - -Pre-formatted [html documentation][] and [man pages][] can be downloaded at -<http://www.erlang.org/download.html>. - -For some graphical tools to find the on-line help you have to install -the HTML documentation on top of the installed OTP applications, i.e. - - $ cd <ReleaseDir> - $ gunzip -c otp_html_%OTP-VSN%.tar.gz | tar xf - - -For `erl -man <page>` to work the Unix manual pages have to be -installed in the same way, i.e. - - $ cd <ReleaseDir> - $ gunzip -c otp_man_%OTP-VSN%.tar.gz | tar xf - - -Where `<ReleaseDir>` is - -* `<PrefixDir>/lib/erlang` if you have installed Erlang/OTP using - `make install`. -* `$DESTDIR<PrefixDir>/lib/erlang` if you have installed Erlang/OTP - using `make install DESTDIR=<TmpInstallDir>`. -* `RELEASE_ROOT` if you have installed using - `make release RELEASE_ROOT=<ReleaseDir>`. - -Support for SMP (Symmetric Multi Processing) --------------------------------------------- - -An emulator with SMP support will be built by default on most platforms -if a usable POSIX thread library or native Windows threads is found. - -You can force building of an SMP emulator, by using -`./configure --enable-smp-support`. However, if configure does not -automatically enable SMP support, the build is very likely to fail. - -Use `./configure --disable-smp-support` if you for some reason do not -want to have the emulator with SMP support built. - -If SMP support is enabled, support for threaded I/O will also be turned on -(also in the emulator without SMP support). - -The `erl` command will automatically start the SMP emulator if the -computer has more than one logical processor. You can force a start -of the emulator with SMP support by passing `-smp enable` as -command line arguments to erl, and you can force a start of the -emulator without SMP support by passing `-smp disable`. - -GS (Graphic System) -------------------- - -GS now Tcl/Tk 8.4. It will be searched for when starting GS. - -Using HiPE ----------- +#### Using HiPE #### HiPE supports the following system configurations: @@ -619,12 +660,12 @@ HiPE supports the following system configurations: * FreeBSD: FreeBSD 6.1 and 6.2 in 32-bit and 64-bit modes should work. - * MacOSX/Darwin: Darwin 9.8.0 in 32-bit mode should work. + * OS X/Darwin: Darwin 9.8.0 in 32-bit mode should work. * PowerPC: All 32-bit 6xx/7xx(G3)/74xx(G4) processors should work. 32-bit mode on 970 (G5) and POWER5 processors should work. - * Linux (Yellow Dog) and Mac OSX 10.4 are supported. + * Linux (Yellow Dog) and OS X 10.4 are supported. * SPARC: All UltraSPARC processors running 32-bit user code should work. @@ -643,11 +684,11 @@ HiPE is automatically enabled on the following systems: * x86 in 32-bit mode: Linux, Solaris, FreeBSD * x86 in 64-bit mode: Linux, Solaris, FreeBSD -* PowerPC: Linux, MacOSX +* PowerPC: Linux, Mac OSX * SPARC: Linux * ARM: Linux -On other supported systems you need to `./configure --enable-hipe`. +On other supported systems, see [Advanced Configure][] on how to enable HiPE. If you are running on a platform supporting HiPE and if you have not disabled HiPE, you can compile a module into native code like this from the Erlang @@ -659,7 +700,7 @@ or 1> c(Module, [native|OtherOptions]). -Using the erlc program, write like this: +Using the erlc program, write like this $ erlc +native Module.erl @@ -674,88 +715,102 @@ Use `hipe:help_options/0` to print out the available options. 1> hipe:help_options(). -Mac OS X (Darwin) ------------------ - -Make sure that the command `hostname` returns a valid fully qualified host -name (this is configured in `/etc/hostconfig`). - -If you develop linked-in drivers (shared library) you need to link using -`gcc` and the flags `-bundle -flat_namespace -undefined suppress`. You also -include `-fno-common` in `CFLAGS` when compiling. Use `.so` as the library -suffix. - -Install `Xcode` from the `AppStore` if it is not already installed. - -If you have Xcode 4.3, or later, you will also need to download -"Command Line Tools" via the Downloads preference pane in Xcode. - -### Building with wxErlang ### +#### Running with GS #### -If you want to build the `wx` application, you will need to get wxWidgets-3.0 (or later) -(`wxWidgets-3.0.0.tar.bz2` from <http://sourceforge.net/projects/wxwindows/files/3.0.0/>) -or get it from github: - $ git clone [email protected]:wxWidgets/wxWidgets.git +The `gs` application requires the GUI toolkit Tcl/Tk to run. At least +version 8.4 is required. -Be aware that the wxWidgets-3.0 is a new release of wxWidgets, it is not as matured -as the old releases and the MacOsX port still lags behind the other ports. - -Configure and build wxWidgets (on Mavericks - 10.9): - - $ ./configure --with-cocoa --prefix=/usr/local - or without support for old versions and with static libs - $ ./configure --with-cocoa --prefix=/usr/local --with-macosx-version-min=10.9 --disable-shared - $ make - $ sudo make install - $ export PATH=/usr/local/bin:$PATH +Known platform issues +--------------------- -Check that you got the correct wx-config +* Suse linux 9.1 is shipped with a patched GCC version 3.3.3, having the + rpm named `gcc-3.3.3-41`. That version has a serious optimization bug + that makes it unusable for building the Erlang emulator. Please + upgrade GCC to a newer version before building on Suse 9.1. Suse Linux + Enterprise edition 9 (SLES9) has `gcc-3.3.3-43` and is not affected. - $ which wx-config +* `gcc-4.3.0` has a serious optimizer bug. It produces an Erlang emulator + that will crash immediately. The bug is supposed to be fixed in + `gcc-4.3.1`. -### Finish up ### +* FreeBSD had a bug which caused `kqueue`/`poll`/`select` to fail to detect + that a `writev()` on a pipe has been made. This bug should have been fixed + in FreeBSD 6.3 and FreeBSD 7.0. NetBSD and DragonFlyBSD probably have or + have had the same bug. More information can be found at: -Build Erlang + * <http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/kern/sys_pipe.c> + * <http://lists.freebsd.org/pipermail/freebsd-arch/2007-September/006790.html> - $ export PATH=/usr/local/bin:$PATH - $ cd $ERL_TOP - $ ./configure --enable-shared-zlib - $ make - $ sudo make install +* `getcwd()` on Solaris 9 can cause an emulator crash. If you have + async-threads enabled you can increase the stack size of the + async-threads as a temporary workaround. See the `+a` command-line + argument in the documentation of `erl(1)`. Without async-threads the + emulator is not as vulnerable to this bug, but if you hit it without + async-threads the only workaround available is to enable async-threads + and increase the stack size of the async-threads. Oracle has however + released patches that fixes the issue: -How to Build a Debug Enabled Erlang RunTime System --------------------------------------------------- + > Problem Description: 6448300 large mnttab can cause stack overrun + > during Solaris 9 getcwd -After completing all the normal building steps described above a debug -enabled runtime system can be built. To do this you have to change -directory to `$ERL_TOP/erts/emulator`. + More information can be found at: + * <https://getupdates.oracle.com/readme/112874-40> + * <https://getupdates.oracle.com/readme/114432-29> -In this directory execute: +* `sed` on Solaris seem to have some problems. For example on + Solaris 8, the BSD `sed` and XPG4 `sed` should be avoided. + Make sure `/bin/sed` or `/usr/bin/sed` is used on the Solaris + platform. - $ make debug FLAVOR=$FLAVOR -where `$FLAVOR` is either `plain` or `smp`. The flavor options will -produce a beam.debug and beam.smp.debug executable respectively. The -files are installed along side with the normal (opt) versions `beam.smp` -and `beam`. +Daily Build and Test +-------------------- +At Ericsson we have a "Daily Build and Test" that runs on: -To start the debug enabled runtime system execute: +* Solaris 8, 9 + * Sparc32 + * Sparc64 +* Solaris 10 + * Sparc32 + * Sparc64 + * x86 +* SuSE Linux/GNU 9.4, 10.1 + * x86 +* SuSE Linux/GNU 10.0, 10.1, 11.0 + * x86 + * x86\_64 +* openSuSE 11.4 (Celadon) + * x86\_64 (valgrind) +* Fedora 7 + * PowerPC +* Fedora 16 + * x86\_64 +* Gentoo Linux/GNU 1.12.11.1 + * x86 +* Ubuntu Linux/GNU 7.04, 10.04, 10.10, 11.04, 12.04 + * x86\_64 +* MontaVista Linux/GNU 4.0.1 + * PowerPC +* FreeBSD 10.0 + * x86 +* OpenBSD 5.4 + * x86\_64 +* OS X 10.5.8 (Leopard), 10.7.5 (Lion), 10.9.1 (Mavericks) + * x86 +* Windows XP SP3, 2003, Vista, 7 + * x86 +* Windows 7 + * x86\_64 - $ $ERL_TOP/bin/cerl -debug +We also have the following "Daily Cross Builds": -The debug enabled runtime system features lock violation checking, -assert checking and various sanity checks to help a developer ensure -correctness. Some of these features can be enabled on a normal beam -using appropriate configure options. +* SuSE Linux/GNU 10.1 x86 -> SuSE Linux/GNU 10.1 x86\_64 +* SuSE Linux/GNU 10.1 x86\_64 -> Linux/GNU TILEPro64 -There are other types of runtime systems that can be built as well -using the similar steps just described. +and the following "Daily Cross Build Tests": - $ make $TYPE FLAVOR=$FLAVOR +* SuSE Linux/GNU 10.1 x86\_64 -where `$TYPE` is `opt`, `gcov`, `gprof`, `debug`, `valgrind`, or `lcnt`. -These different beam types are useful for debugging and profiling -purposes. Authors ------- @@ -764,6 +819,7 @@ Authors are mostly listed in the application's `AUTHORS` files, that is `$ERL_TOP/lib/*/AUTHORS` and `$ERL_TOP/erts/AUTHORS`, not in the individual source files. + Copyright and License --------------------- @@ -784,30 +840,25 @@ under the License. %CopyrightEnd% -More Information ----------------- - -More information can be found at <http://www.erlang.org>. -Modifying This Document ------------------------ - -Before modifying this document you need to have a look at the -[$ERL_TOP/HOWTO/MARKDOWN.md][] document. [$ERL_TOP/HOWTO/INSTALL-CROSS.md]: INSTALL-CROSS.md [$ERL_TOP/HOWTO/INSTALL-WIN32.md]: INSTALL-WIN32.md [DESTDIR]: http://www.gnu.org/prep/standards/html_node/DESTDIR.html - [Building in Git]: #How-to-Build-and-Install-ErlangOTP_Building-in-Git - [Pre-built Source Release]: #How-to-Build-and-Install-ErlangOTP_Prebuilt-Source-Release - [make and $ERL_TOP]: #How-to-Build-and-Install-ErlangOTP_make-and-ERLTOP + [Building in Git]: #Advanced-configuration-and-build-of-ErlangOTP_Building_Within-Git + [Advanced Configure]: #Advanced-configuration-and-build-of-ErlangOTP_Configuring + [Pre-built Source Release]: #Advanced-configuration-and-build-of-ErlangOTP_Building_Prebuilt-Source-Release + [make and $ERL_TOP]: #Advanced-configuration-and-build-of-ErlangOTP_make-and-ERLTOP [html documentation]: http://www.erlang.org/download/otp_doc_html_%OTP-VSN%.tar.gz [man pages]: http://www.erlang.org/download/otp_doc_man_%OTP-VSN%.tar.gz [the released source tar ball]: http://www.erlang.org/download/otp_src_%OTP-VSN%.tar.gz + [System Principles]: ../system_principles/system_principles + [Known platform issues]: #Known-platform-issues [native build]: #How-to-Build-and-Install-ErlangOTP [cross build]: INSTALL-CROSS.md - [$ERL_TOP/HOWTO/MARKDOWN.md]: MARKDOWN.md - - [?TOC]: true + [Required Utilities]: #Required-Utilities + [Optional Utilities]: #Optional-Utilities + [Building on a Mac]: #Advanced-configuration-and-build-of-ErlangOTP_Building_OS-X-Darwin + [Building with wxErlang]: #Advanced-configuration-and-build-of-ErlangOTP_Building_Building-with-wxErlang diff --git a/Makefile.in b/Makefile.in index c667eb5f79..5c96c789dc 100644 --- a/Makefile.in +++ b/Makefile.in @@ -422,6 +422,10 @@ endif cd $(ERL_TOP)/system/doc && \ PATH=$(BOOT_PREFIX)"$${PATH}" \ ERL_TOP=$(ERL_TOP) $(MAKE) TESTROOT="$(RELEASE_ROOT)" DOCGEN=$(BOOTSTRAP_ROOT)/bootstrap/lib/erl_docgen $@ +ifneq ($(OTP_SMALL_BUILD),true) + echo "OTP doc built" > $(ERL_TOP)/make/otp_doc_built +endif + mod2app: PATH=$(BOOT_PREFIX)"$${PATH}" escript $(BOOTSTRAP_ROOT)/bootstrap/lib/erl_docgen/priv/bin/xref_mod_app.escript -topdir $(ERL_TOP) -outfile $(ERL_TOP)/make/$(TARGET)/mod2app.xml @@ -447,6 +451,7 @@ else $(make_verbose)cd lib && \ ERL_TOP=$(ERL_TOP) PATH=$(BOOT_PREFIX)"$${PATH}" \ $(MAKE) opt BUILD_ALL=true + $(V_at)echo "OTP built" > $(ERL_TOP)/make/otp_built endif kernel: $(make_verbose)cd lib/kernel && \ diff --git a/OTP_VERSION b/OTP_VERSION index cfc4bcdf57..9f09b58414 100644 --- a/OTP_VERSION +++ b/OTP_VERSION @@ -1 +1 @@ -17.0-rc2 +17.4-rc0 @@ -12,6 +12,15 @@ middle-ware to develop these systems. It includes its own distributed database, applications to interface towards other languages, debugging and release handling tools. +ERTS and BEAM +------------- +**BEAM** is the name of the virtual machine where all Erlang code is executed. +Every compiled Erlang file has the suffix .beam. The virtual machine +is sometimes referred to as the emulator. + +**ERTS** is the Erlang Runtime System where the BEAM, kernel and +standard libraries amongst others are included. + More information can be found at [erlang.org] [1]. Building and Installing @@ -57,7 +66,7 @@ Copyright and License > %CopyrightBegin% > -> Copyright Ericsson AB 2010-2012. All Rights Reserved. +> Copyright Ericsson AB 2010-2014. All Rights Reserved. > > The contents of this file are subject to the Erlang Public License, > Version 1.1, (the "License"); you may not use this file except in diff --git a/aclocal.m4 b/aclocal.m4 index c51c26794a..ed492d55ff 100644 --- a/aclocal.m4 +++ b/aclocal.m4 @@ -83,8 +83,10 @@ AC_ARG_VAR(erl_xcomp_ose_LM_POST_LINK, [OSE postlink tool (only used when cross AC_ARG_VAR(erl_xcomp_ose_LM_SET_CONF, [Sets the configuration for an OSE load module (only used when cross compiling for OSE)]) AC_ARG_VAR(erl_xcomp_ose_LM_ELF_SIZE, [Prints the section size information for an OSE load module (only used when cross compiling for OSE)]) AC_ARG_VAR(erl_xcomp_ose_LM_LCF, [OSE load module linker configuration file (only used when cross compiling for OSE)]) -AC_ARG_VAR(erl_xcomp_ose_LM_CONF, [OSE load module default configuration file (only used when cross compiling for OSE)]) -AC_ARG_VAR(erl_xcomp_ose_CONFD, [OSE OSE confd source file]) +AC_ARG_VAR(erl_xcomp_ose_BEAM_LM_CONF, [BEAM OSE load module default configuration file (only used when cross compiling for OSE)]) +AC_ARG_VAR(erl_xcomp_ose_EPMD_LM_CONF, [EPMD OSE load module default configuration file (only used when cross compiling for OSE)]) +AC_ARG_VAR(erl_xcomp_ose_RUN_ERL_LM_CONF, [run_erl_lm OSE load module default configuration file (only used when cross compiling for OSE)]) +AC_ARG_VAR(erl_xcomp_ose_CONFD, [OSE confd source file]) AC_ARG_VAR(erl_xcomp_ose_CRT0_LM, [OSE crt0 lm source file]) ]) @@ -1116,7 +1118,7 @@ case "$THR_LIB_NAME" in [Define if you have the "ose_spi/ose_spi.h" header file.])) ;; esac - if test "x$THR_LIB_NAME" == "xpthread"; then + if test "x$THR_LIB_NAME" = "xpthread"; then case $host_os in openbsd*) # The default stack size is insufficient for our needs @@ -1220,7 +1222,7 @@ case "$THR_LIB_NAME" in dnl dnl Check for functions dnl - if test "x$THR_LIB_NAME" == "xpthread"; then + if test "x$THR_LIB_NAME" = "xpthread"; then AC_CHECK_FUNC(pthread_spin_lock, \ [ethr_have_native_spinlock=yes \ AC_DEFINE(ETHR_HAVE_PTHREAD_SPIN_LOCK, 1, \ diff --git a/bootstrap/bin/start.boot b/bootstrap/bin/start.boot Binary files differindex b57eef57ab..c497fe37fc 100644 --- a/bootstrap/bin/start.boot +++ b/bootstrap/bin/start.boot diff --git a/bootstrap/bin/start_clean.boot b/bootstrap/bin/start_clean.boot Binary files differindex b57eef57ab..c497fe37fc 100644 --- a/bootstrap/bin/start_clean.boot +++ b/bootstrap/bin/start_clean.boot diff --git a/bootstrap/lib/compiler/ebin/beam_asm.beam b/bootstrap/lib/compiler/ebin/beam_asm.beam Binary files differindex a3b9f3df8a..f58fdd7171 100644 --- a/bootstrap/lib/compiler/ebin/beam_asm.beam +++ b/bootstrap/lib/compiler/ebin/beam_asm.beam diff --git a/bootstrap/lib/compiler/ebin/beam_disasm.beam b/bootstrap/lib/compiler/ebin/beam_disasm.beam Binary files differindex c129898523..f050f39ca0 100644 --- a/bootstrap/lib/compiler/ebin/beam_disasm.beam +++ b/bootstrap/lib/compiler/ebin/beam_disasm.beam diff --git a/bootstrap/lib/compiler/ebin/cerl.beam b/bootstrap/lib/compiler/ebin/cerl.beam Binary files differindex d5e97e4f68..0f0670c9ae 100644 --- a/bootstrap/lib/compiler/ebin/cerl.beam +++ b/bootstrap/lib/compiler/ebin/cerl.beam diff --git a/bootstrap/lib/compiler/ebin/compiler.app b/bootstrap/lib/compiler/ebin/compiler.app index 83554a68a5..f12713417f 100644 --- a/bootstrap/lib/compiler/ebin/compiler.app +++ b/bootstrap/lib/compiler/ebin/compiler.app @@ -18,7 +18,7 @@ {application, compiler, [{description, "ERTS CXC 138 10"}, - {vsn, "4.9.4"}, + {vsn, "5.0.1"}, {modules, [ beam_a, beam_asm, @@ -67,4 +67,6 @@ ]}, {registered, []}, {applications, [kernel, stdlib]}, - {env, []}]}. + {env, []}, + {runtime_dependencies, ["stdlib-2.0","kernel-3.0","hipe-3.10.3","erts-6.0", + "crypto-3.3"]}]}. diff --git a/bootstrap/lib/compiler/ebin/compiler.appup b/bootstrap/lib/compiler/ebin/compiler.appup index 13c2644004..e73b9aa60c 100644 --- a/bootstrap/lib/compiler/ebin/compiler.appup +++ b/bootstrap/lib/compiler/ebin/compiler.appup @@ -15,7 +15,7 @@ %% under the License. %% %% %CopyrightEnd% -{"4.9.4", +{"5.0", [{<<".*">>,[{restart_application, compiler}]}], [{<<".*">>,[{restart_application, compiler}]}] }. diff --git a/bootstrap/lib/compiler/ebin/core_lib.beam b/bootstrap/lib/compiler/ebin/core_lib.beam Binary files differindex 4957fbabb7..bb38837049 100644 --- a/bootstrap/lib/compiler/ebin/core_lib.beam +++ b/bootstrap/lib/compiler/ebin/core_lib.beam diff --git a/bootstrap/lib/compiler/ebin/core_pp.beam b/bootstrap/lib/compiler/ebin/core_pp.beam Binary files differindex fbfd68a93b..e6e60fb658 100644 --- a/bootstrap/lib/compiler/ebin/core_pp.beam +++ b/bootstrap/lib/compiler/ebin/core_pp.beam diff --git a/bootstrap/lib/compiler/ebin/sys_core_fold.beam b/bootstrap/lib/compiler/ebin/sys_core_fold.beam Binary files differindex 9bdab52015..cce0ac7832 100644 --- a/bootstrap/lib/compiler/ebin/sys_core_fold.beam +++ b/bootstrap/lib/compiler/ebin/sys_core_fold.beam diff --git a/bootstrap/lib/compiler/ebin/sys_pre_expand.beam b/bootstrap/lib/compiler/ebin/sys_pre_expand.beam Binary files differindex 5c14528eff..c0c537f623 100644 --- a/bootstrap/lib/compiler/ebin/sys_pre_expand.beam +++ b/bootstrap/lib/compiler/ebin/sys_pre_expand.beam diff --git a/bootstrap/lib/compiler/ebin/v3_codegen.beam b/bootstrap/lib/compiler/ebin/v3_codegen.beam Binary files differindex 6bf28f441e..3670fd206c 100644 --- a/bootstrap/lib/compiler/ebin/v3_codegen.beam +++ b/bootstrap/lib/compiler/ebin/v3_codegen.beam diff --git a/bootstrap/lib/compiler/ebin/v3_core.beam b/bootstrap/lib/compiler/ebin/v3_core.beam Binary files differindex b876bd9970..5da654686a 100644 --- a/bootstrap/lib/compiler/ebin/v3_core.beam +++ b/bootstrap/lib/compiler/ebin/v3_core.beam diff --git a/bootstrap/lib/compiler/ebin/v3_kernel.beam b/bootstrap/lib/compiler/ebin/v3_kernel.beam Binary files differindex f5dcecb807..1e549a41c7 100644 --- a/bootstrap/lib/compiler/ebin/v3_kernel.beam +++ b/bootstrap/lib/compiler/ebin/v3_kernel.beam diff --git a/bootstrap/lib/compiler/ebin/v3_life.beam b/bootstrap/lib/compiler/ebin/v3_life.beam Binary files differindex c2363fc62d..0eb0cb6caf 100644 --- a/bootstrap/lib/compiler/ebin/v3_life.beam +++ b/bootstrap/lib/compiler/ebin/v3_life.beam diff --git a/bootstrap/lib/kernel/ebin/application.beam b/bootstrap/lib/kernel/ebin/application.beam Binary files differindex 2c8444fb95..6946d17545 100644 --- a/bootstrap/lib/kernel/ebin/application.beam +++ b/bootstrap/lib/kernel/ebin/application.beam diff --git a/bootstrap/lib/kernel/ebin/application_controller.beam b/bootstrap/lib/kernel/ebin/application_controller.beam Binary files differindex 659f29e35e..2a4525c27f 100644 --- a/bootstrap/lib/kernel/ebin/application_controller.beam +++ b/bootstrap/lib/kernel/ebin/application_controller.beam diff --git a/bootstrap/lib/kernel/ebin/code_server.beam b/bootstrap/lib/kernel/ebin/code_server.beam Binary files differindex 430c2e0733..340c5feed9 100644 --- a/bootstrap/lib/kernel/ebin/code_server.beam +++ b/bootstrap/lib/kernel/ebin/code_server.beam diff --git a/bootstrap/lib/kernel/ebin/disk_log_server.beam b/bootstrap/lib/kernel/ebin/disk_log_server.beam Binary files differindex ec9422d41b..8c1622c7d8 100644 --- a/bootstrap/lib/kernel/ebin/disk_log_server.beam +++ b/bootstrap/lib/kernel/ebin/disk_log_server.beam diff --git a/bootstrap/lib/kernel/ebin/erl_boot_server.beam b/bootstrap/lib/kernel/ebin/erl_boot_server.beam Binary files differindex 7e43b0fb36..6e56a1061d 100644 --- a/bootstrap/lib/kernel/ebin/erl_boot_server.beam +++ b/bootstrap/lib/kernel/ebin/erl_boot_server.beam diff --git a/bootstrap/lib/kernel/ebin/erl_epmd.beam b/bootstrap/lib/kernel/ebin/erl_epmd.beam Binary files differindex 1a74da7f1f..bf171450e9 100644 --- a/bootstrap/lib/kernel/ebin/erl_epmd.beam +++ b/bootstrap/lib/kernel/ebin/erl_epmd.beam diff --git a/bootstrap/lib/kernel/ebin/erts_debug.beam b/bootstrap/lib/kernel/ebin/erts_debug.beam Binary files differindex 7d059d10cf..17320342bc 100644 --- a/bootstrap/lib/kernel/ebin/erts_debug.beam +++ b/bootstrap/lib/kernel/ebin/erts_debug.beam diff --git a/bootstrap/lib/kernel/ebin/file.beam b/bootstrap/lib/kernel/ebin/file.beam Binary files differindex 4444b79bf7..25f2506078 100644 --- a/bootstrap/lib/kernel/ebin/file.beam +++ b/bootstrap/lib/kernel/ebin/file.beam diff --git a/bootstrap/lib/kernel/ebin/global.beam b/bootstrap/lib/kernel/ebin/global.beam Binary files differindex 1629089858..d25be3994c 100644 --- a/bootstrap/lib/kernel/ebin/global.beam +++ b/bootstrap/lib/kernel/ebin/global.beam diff --git a/bootstrap/lib/kernel/ebin/hipe_unified_loader.beam b/bootstrap/lib/kernel/ebin/hipe_unified_loader.beam Binary files differindex d24c06f112..b24c850af3 100644 --- a/bootstrap/lib/kernel/ebin/hipe_unified_loader.beam +++ b/bootstrap/lib/kernel/ebin/hipe_unified_loader.beam diff --git a/bootstrap/lib/kernel/ebin/inet.beam b/bootstrap/lib/kernel/ebin/inet.beam Binary files differindex fe203f816f..4d156b726c 100644 --- a/bootstrap/lib/kernel/ebin/inet.beam +++ b/bootstrap/lib/kernel/ebin/inet.beam diff --git a/bootstrap/lib/kernel/ebin/inet_config.beam b/bootstrap/lib/kernel/ebin/inet_config.beam Binary files differindex 03b42209e6..e858ac20f1 100644 --- a/bootstrap/lib/kernel/ebin/inet_config.beam +++ b/bootstrap/lib/kernel/ebin/inet_config.beam diff --git a/bootstrap/lib/kernel/ebin/kernel.app b/bootstrap/lib/kernel/ebin/kernel.app index f491239a45..e60b36e1e7 100644 --- a/bootstrap/lib/kernel/ebin/kernel.app +++ b/bootstrap/lib/kernel/ebin/kernel.app @@ -21,7 +21,7 @@ {application, kernel, [ {description, "ERTS CXC 138 10"}, - {vsn, "3.0"}, + {vsn, "3.0.2"}, {modules, [application, application_controller, application_master, @@ -114,6 +114,7 @@ pg2]}, {applications, []}, {env, [{error_logger, tty}]}, - {mod, {kernel, []}} + {mod, {kernel, []}}, + {runtime_dependencies, ["erts-6.1.2", "stdlib-2.0", "sasl-2.4"]} ] }. diff --git a/bootstrap/lib/kernel/ebin/net_adm.beam b/bootstrap/lib/kernel/ebin/net_adm.beam Binary files differindex fb722b3b4e..4b09aae8a5 100644 --- a/bootstrap/lib/kernel/ebin/net_adm.beam +++ b/bootstrap/lib/kernel/ebin/net_adm.beam diff --git a/bootstrap/lib/kernel/ebin/os.beam b/bootstrap/lib/kernel/ebin/os.beam Binary files differindex b8b01b31ab..b1b92ca36b 100644 --- a/bootstrap/lib/kernel/ebin/os.beam +++ b/bootstrap/lib/kernel/ebin/os.beam diff --git a/bootstrap/lib/stdlib/ebin/epp.beam b/bootstrap/lib/stdlib/ebin/epp.beam Binary files differindex ab22f9971b..83469557cd 100644 --- a/bootstrap/lib/stdlib/ebin/epp.beam +++ b/bootstrap/lib/stdlib/ebin/epp.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_compile.beam b/bootstrap/lib/stdlib/ebin/erl_compile.beam Binary files differindex 88da2b246f..e650afbf6d 100644 --- a/bootstrap/lib/stdlib/ebin/erl_compile.beam +++ b/bootstrap/lib/stdlib/ebin/erl_compile.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_eval.beam b/bootstrap/lib/stdlib/ebin/erl_eval.beam Binary files differindex aab2f38b91..95454c885a 100644 --- a/bootstrap/lib/stdlib/ebin/erl_eval.beam +++ b/bootstrap/lib/stdlib/ebin/erl_eval.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_expand_records.beam b/bootstrap/lib/stdlib/ebin/erl_expand_records.beam Binary files differindex 0e72af0e35..ff84000ac0 100644 --- a/bootstrap/lib/stdlib/ebin/erl_expand_records.beam +++ b/bootstrap/lib/stdlib/ebin/erl_expand_records.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_lint.beam b/bootstrap/lib/stdlib/ebin/erl_lint.beam Binary files differindex 9660509270..8ae35899d0 100644 --- a/bootstrap/lib/stdlib/ebin/erl_lint.beam +++ b/bootstrap/lib/stdlib/ebin/erl_lint.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_parse.beam b/bootstrap/lib/stdlib/ebin/erl_parse.beam Binary files differindex c1c114dd3c..998bb0b209 100644 --- a/bootstrap/lib/stdlib/ebin/erl_parse.beam +++ b/bootstrap/lib/stdlib/ebin/erl_parse.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_pp.beam b/bootstrap/lib/stdlib/ebin/erl_pp.beam Binary files differindex 683a07fbd5..ff196f3a45 100644 --- a/bootstrap/lib/stdlib/ebin/erl_pp.beam +++ b/bootstrap/lib/stdlib/ebin/erl_pp.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_scan.beam b/bootstrap/lib/stdlib/ebin/erl_scan.beam Binary files differindex fe45755343..b2b97bda8a 100644 --- a/bootstrap/lib/stdlib/ebin/erl_scan.beam +++ b/bootstrap/lib/stdlib/ebin/erl_scan.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_tar.beam b/bootstrap/lib/stdlib/ebin/erl_tar.beam Binary files differindex 6401818109..76f5f95270 100644 --- a/bootstrap/lib/stdlib/ebin/erl_tar.beam +++ b/bootstrap/lib/stdlib/ebin/erl_tar.beam diff --git a/bootstrap/lib/stdlib/ebin/filelib.beam b/bootstrap/lib/stdlib/ebin/filelib.beam Binary files differindex b70aa54301..9fc7dd79cb 100644 --- a/bootstrap/lib/stdlib/ebin/filelib.beam +++ b/bootstrap/lib/stdlib/ebin/filelib.beam diff --git a/bootstrap/lib/stdlib/ebin/gen_event.beam b/bootstrap/lib/stdlib/ebin/gen_event.beam Binary files differindex 46f1512a24..d45e508d22 100644 --- a/bootstrap/lib/stdlib/ebin/gen_event.beam +++ b/bootstrap/lib/stdlib/ebin/gen_event.beam diff --git a/bootstrap/lib/stdlib/ebin/gen_fsm.beam b/bootstrap/lib/stdlib/ebin/gen_fsm.beam Binary files differindex d6719e68d9..1eee773838 100644 --- a/bootstrap/lib/stdlib/ebin/gen_fsm.beam +++ b/bootstrap/lib/stdlib/ebin/gen_fsm.beam diff --git a/bootstrap/lib/stdlib/ebin/gen_server.beam b/bootstrap/lib/stdlib/ebin/gen_server.beam Binary files differindex 806c4d58d7..cb9421b7c5 100644 --- a/bootstrap/lib/stdlib/ebin/gen_server.beam +++ b/bootstrap/lib/stdlib/ebin/gen_server.beam diff --git a/bootstrap/lib/stdlib/ebin/io_lib_format.beam b/bootstrap/lib/stdlib/ebin/io_lib_format.beam Binary files differindex e579d3cb2e..fcfce54d0e 100644 --- a/bootstrap/lib/stdlib/ebin/io_lib_format.beam +++ b/bootstrap/lib/stdlib/ebin/io_lib_format.beam diff --git a/bootstrap/lib/stdlib/ebin/maps.beam b/bootstrap/lib/stdlib/ebin/maps.beam Binary files differindex f524bc516c..8473da95d7 100644 --- a/bootstrap/lib/stdlib/ebin/maps.beam +++ b/bootstrap/lib/stdlib/ebin/maps.beam diff --git a/bootstrap/lib/stdlib/ebin/otp_internal.beam b/bootstrap/lib/stdlib/ebin/otp_internal.beam Binary files differindex 1146ef9ab8..23b140aeb5 100644 --- a/bootstrap/lib/stdlib/ebin/otp_internal.beam +++ b/bootstrap/lib/stdlib/ebin/otp_internal.beam diff --git a/bootstrap/lib/stdlib/ebin/pg.beam b/bootstrap/lib/stdlib/ebin/pg.beam Binary files differindex 65195defbf..31f30461d8 100644 --- a/bootstrap/lib/stdlib/ebin/pg.beam +++ b/bootstrap/lib/stdlib/ebin/pg.beam diff --git a/bootstrap/lib/stdlib/ebin/proc_lib.beam b/bootstrap/lib/stdlib/ebin/proc_lib.beam Binary files differindex ee23fef6f0..368d3e39a0 100644 --- a/bootstrap/lib/stdlib/ebin/proc_lib.beam +++ b/bootstrap/lib/stdlib/ebin/proc_lib.beam diff --git a/bootstrap/lib/stdlib/ebin/shell.beam b/bootstrap/lib/stdlib/ebin/shell.beam Binary files differindex 3756c0d46a..8c4734f208 100644 --- a/bootstrap/lib/stdlib/ebin/shell.beam +++ b/bootstrap/lib/stdlib/ebin/shell.beam diff --git a/bootstrap/lib/stdlib/ebin/slave.beam b/bootstrap/lib/stdlib/ebin/slave.beam Binary files differindex ba741284ac..532c46cd38 100644 --- a/bootstrap/lib/stdlib/ebin/slave.beam +++ b/bootstrap/lib/stdlib/ebin/slave.beam diff --git a/bootstrap/lib/stdlib/ebin/stdlib.app b/bootstrap/lib/stdlib/ebin/stdlib.app index 31c7222100..8cbf23e958 100644 --- a/bootstrap/lib/stdlib/ebin/stdlib.app +++ b/bootstrap/lib/stdlib/ebin/stdlib.app @@ -19,7 +19,7 @@ %% {application, stdlib, [{description, "ERTS CXC 138 10"}, - {vsn, "2.0"}, + {vsn, "2.1.1"}, {modules, [array, base64, beam_lib, @@ -102,5 +102,8 @@ {registered,[timer_server,rsh_starter,take_over_monitor,pool_master, dets]}, {applications, [kernel]}, - {env, []}]}. + {env, []}, + {runtime_dependencies, ["sasl-2.4","kernel-3.0.2","erts-6.2","crypto-3.3", + "compiler-5.0"]} +]}. diff --git a/bootstrap/lib/stdlib/ebin/sys.beam b/bootstrap/lib/stdlib/ebin/sys.beam Binary files differindex 2aed63dc88..1413b822f2 100644 --- a/bootstrap/lib/stdlib/ebin/sys.beam +++ b/bootstrap/lib/stdlib/ebin/sys.beam diff --git a/configure.in b/configure.in index be906dcb4e..780e660f9d 100644 --- a/configure.in +++ b/configure.in @@ -366,6 +366,12 @@ elif test X"$TMPSYS" '=' X"Darwin-i386"; then export LDFLAGS fi +m4_define(DEFAULT_SANITIZERS, [address,undefined]) +AC_ARG_ENABLE(sanitizers, + AS_HELP_STRING( + [--enable-sanitizers@<:@=comma-separated list of sanitizers@:>@], + [Default=DEFAULT_SANITIZERS])) + AC_ARG_ENABLE([silent-rules], [dnl AS_HELP_STRING( [--enable-silent-rules], diff --git a/erts/aclocal.m4 b/erts/aclocal.m4 index c51c26794a..ed492d55ff 100644 --- a/erts/aclocal.m4 +++ b/erts/aclocal.m4 @@ -83,8 +83,10 @@ AC_ARG_VAR(erl_xcomp_ose_LM_POST_LINK, [OSE postlink tool (only used when cross AC_ARG_VAR(erl_xcomp_ose_LM_SET_CONF, [Sets the configuration for an OSE load module (only used when cross compiling for OSE)]) AC_ARG_VAR(erl_xcomp_ose_LM_ELF_SIZE, [Prints the section size information for an OSE load module (only used when cross compiling for OSE)]) AC_ARG_VAR(erl_xcomp_ose_LM_LCF, [OSE load module linker configuration file (only used when cross compiling for OSE)]) -AC_ARG_VAR(erl_xcomp_ose_LM_CONF, [OSE load module default configuration file (only used when cross compiling for OSE)]) -AC_ARG_VAR(erl_xcomp_ose_CONFD, [OSE OSE confd source file]) +AC_ARG_VAR(erl_xcomp_ose_BEAM_LM_CONF, [BEAM OSE load module default configuration file (only used when cross compiling for OSE)]) +AC_ARG_VAR(erl_xcomp_ose_EPMD_LM_CONF, [EPMD OSE load module default configuration file (only used when cross compiling for OSE)]) +AC_ARG_VAR(erl_xcomp_ose_RUN_ERL_LM_CONF, [run_erl_lm OSE load module default configuration file (only used when cross compiling for OSE)]) +AC_ARG_VAR(erl_xcomp_ose_CONFD, [OSE confd source file]) AC_ARG_VAR(erl_xcomp_ose_CRT0_LM, [OSE crt0 lm source file]) ]) @@ -1116,7 +1118,7 @@ case "$THR_LIB_NAME" in [Define if you have the "ose_spi/ose_spi.h" header file.])) ;; esac - if test "x$THR_LIB_NAME" == "xpthread"; then + if test "x$THR_LIB_NAME" = "xpthread"; then case $host_os in openbsd*) # The default stack size is insufficient for our needs @@ -1220,7 +1222,7 @@ case "$THR_LIB_NAME" in dnl dnl Check for functions dnl - if test "x$THR_LIB_NAME" == "xpthread"; then + if test "x$THR_LIB_NAME" = "xpthread"; then AC_CHECK_FUNC(pthread_spin_lock, \ [ethr_have_native_spinlock=yes \ AC_DEFINE(ETHR_HAVE_PTHREAD_SPIN_LOCK, 1, \ diff --git a/erts/autoconf/vxworks/sed.general b/erts/autoconf/vxworks/sed.general index dbb9420b67..efa4e99054 100644 --- a/erts/autoconf/vxworks/sed.general +++ b/erts/autoconf/vxworks/sed.general @@ -57,6 +57,7 @@ s|@ETHR_LIB_NAME@|| s|@ETHR_DEFS@|| s|@ETHR_THR_LIB_BASE@|| s|@ETHR_THR_LIB_BASE_DIR@|| +s|@SYSTEMD_DAEMON_LIBS@|| s|@EMU_THR_DEFS@|| s|@EMU_THR_LIBS@|| s|@EMU_THR_LIB_NAME@|ethread| diff --git a/erts/configure.in b/erts/configure.in index 208c294106..c8b96c50f0 100644 --- a/erts/configure.in +++ b/erts/configure.in @@ -214,9 +214,6 @@ AC_ARG_ENABLE(native-libs, AS_HELP_STRING([--enable-native-libs], [compile Erlang libraries to native code])) -AC_ARG_ENABLE(tsp, -AS_HELP_STRING([--enable-tsp], [compile tsp app])) - AC_ARG_ENABLE(fp-exceptions, AS_HELP_STRING([--enable-fp-exceptions], [use hardware floating point exceptions (default if hipe enabled)]), @@ -366,6 +363,11 @@ AC_DEFINE_UNQUOTED(ASSUMED_CACHE_LINE_SIZE, $with_assumed_cache_line_size, [Assumed cache-line size (in bytes)]) +AC_ARG_ENABLE(systemd, +AS_HELP_STRING([--enable-systemd], [enable systemd support in epmd]), +[], +[enable_systemd=no]) + dnl Magic test for clearcase. OTP_RELEASE= if test "${ERLANG_COMMERCIAL_BUILD}" != ""; then @@ -444,22 +446,22 @@ else fi AC_ARG_ENABLE(static-nifs, -AS_HELP_STRING([--enable-static-nifs], [link nifs statically. If yes then all nifs in all Erlang/OTP applications will be statically linked into the main binary. It is also possible to give a list of nifs that should be linked statically. The list should be a comma seperated and contain the absolute path to a .a archive for each nif that is to be statically linked. The name of the .a archive has to be the same as the name of the nif. Note that you have to link any external dependencies that the nifs have to the main binary, so for the crypto nif you want to pass LIBS=-lcrypto to configure.]), +AS_HELP_STRING([--enable-static-nifs], [link nifs statically. If yes then all nifs in all Erlang/OTP applications will be statically linked into the main binary. It is also possible to give a list of nifs that should be linked statically. The list should be a comma separated and contain the absolute path to a .a archive for each nif that is to be statically linked. The name of the .a archive has to be the same as the name of the nif. Note that you have to link any external dependencies that the nifs have to the main binary, so for the crypto nif you want to pass LIBS=-lcrypto to configure.]), STATIC_NIFS="$enableval", STATIC_NIFS=no) AC_SUBST(STATIC_NIFS) AC_ARG_ENABLE(static-drivers, -AS_HELP_STRING([--enable-static-drivers], [comma seperated list of linked-in drivers to link statically with the main binary. The list should contain the absolute path to a .a archive for each driver that is to be statically linked. The name of the .a archive has to be the same as the name of the driver.]), +AS_HELP_STRING([--enable-static-drivers], [comma separated list of linked-in drivers to link statically with the main binary. The list should contain the absolute path to a .a archive for each driver that is to be statically linked. The name of the .a archive has to be the same as the name of the driver.]), STATIC_DRIVERS="$enableval", STATIC_DRIVERS=no) AC_SUBST(STATIC_DRIVERS) AC_ARG_WITH(ets-write-concurrency-locks, - AS_HELP_STRING([--with-ets-write-concurrency-locks={8,16,32,64,128,256}], - [specify how many locks the write_concurrency option for ets should use.]) - AS_HELP_STRING([--without-ets-write-concurrency-locks], - [use the default number of write_concurrency locks (default)])) +AS_HELP_STRING([--with-ets-write-concurrency-locks={8|16|32|64|128|256}], + [specify how many locks the write_concurrency option for ets should use.]) +AS_HELP_STRING([--without-ets-write-concurrency-locks], + [use the default number of write_concurrency locks (default)])) if test X"$with_ets_write_concurrency_locks" != X""; then AC_DEFINE_UNQUOTED(ERTS_DB_HASH_LOCK_CNT,$with_ets_write_concurrency_locks, @@ -671,6 +673,7 @@ case $chk_arch_ in x86_64) ARCH=amd64;; amd64) ARCH=amd64;; macppc) ARCH=ppc;; + powerpc) ARCH=ppc;; ppc) ARCH=ppc;; ppc64) ARCH=ppc64;; "Power Macintosh") ARCH=ppc;; @@ -678,8 +681,10 @@ case $chk_arch_ in armv5teb) ARCH=arm;; armv5tel) ARCH=arm;; armv5tejl) ARCH=arm;; - armv6l) ARCH=arm;; + armv6l) ARCH=arm;; + armv6hl) ARCH=arm;; armv7l) ARCH=arm;; + armv7hl) ARCH=arm;; tile) ARCH=tile;; *) ARCH=noarch;; esac @@ -1678,6 +1683,30 @@ AC_CHECK_MEMBERS([struct ifreq.ifr_enaddr], [], [], ]) dnl ---------------------------------------------------------------------- +dnl Check the availability of systemd +dnl ---------------------------------------------------------------------- +if test x"$enable_systemd" != x"no"; then + +systemd_daemon_save_LIBS=$LIBS +LIBS= +AC_SEARCH_LIBS(sd_listen_fds,[systemd systemd-daemon], + [have_sd_listen_fds=yes],[have_sd_listen_fds=no],$systemd_daemon_save_LIBS) +AC_CHECK_HEADERS(systemd/sd-daemon.h, + [have_systemd_sd_daemon_h=yes],[have_systemd_sd_daemon_h=no]) + +if test x"$have_sd_listen_fds" = x"yes" && \ + test x"$have_systemd_sd_daemon_h" = x"yes"; then + AC_DEFINE([HAVE_SYSTEMD_DAEMON],[1],[Define if you have systemd daemon]) + SYSTEMD_DAEMON_LIBS=$LIBS +elif test x"$enable_systemd" = x"yes"; then + AC_MSG_FAILURE([--enable-systemd was given, but test for systemd failed]) +fi +LIBS=$systemd_daemon_save_LIBS +fi +AC_SUBST(SYSTEMD_DAEMON_LIBS) + + +dnl ---------------------------------------------------------------------- dnl Check the availability for libdlpi dnl ---------------------------------------------------------------------- AC_CHECK_LIB(dlpi, dlpi_open) @@ -3428,16 +3457,6 @@ dnl (using the --disable, --enable and --with switches). dnl ---------------------------------------------------------------------- # -# Check if we should compile TSP app -# - -TSP_APP= -if test X${enable_tsp} = Xyes; then - TSP_APP=tsp -fi -AC_SUBST(TSP_APP) - -# # Check if we should enable HiPE. # @@ -4802,6 +4821,26 @@ if test "x$GCC" = xyes; then fi dnl ---------------------------------------------------------------------- +dnl Enable -fsanitize= flags. +dnl ---------------------------------------------------------------------- + +m4_define(DEFAULT_SANITIZERS, [address,undefined]) +AC_ARG_ENABLE( + sanitizers, + AS_HELP_STRING( + [--enable-sanitizers@<:@=comma-separated list of sanitizers@:>@], + [Default=DEFAULT_SANITIZERS]), +[ +case "$enableval" in + no) sanitizers= ;; + yes) sanitizers="-fsanitize=DEFAULT_SANITIZERS" ;; + *) sanitizers="-fsanitize=$enableval" ;; +esac +CFLAGS="$CFLAGS $sanitizers" +LDFLAGS="$LDFLAGS $sanitizers" +]) + +dnl ---------------------------------------------------------------------- dnl Output the result. dnl ---------------------------------------------------------------------- diff --git a/erts/doc/src/crash_dump.xml b/erts/doc/src/crash_dump.xml index c59741f250..d3de29b876 100644 --- a/erts/doc/src/crash_dump.xml +++ b/erts/doc/src/crash_dump.xml @@ -85,20 +85,22 @@ operating system.</p> <list type="bulleted"> <item>"<em><A></em>: Cannot allocate <em><N></em> - bytes of memory (of type "<em><T></em>")." - The system - has run out of memory. <A> is the allocator that failed - to allocate memory, <N> is the number of bytes that - <A> tried to allocate, and <T> is the memory block - type that the memory was needed for. The most common case is - that a process stores huge amounts of data. In this case - <T> is most often <c><![CDATA[heap]]></c>, <c><![CDATA[old_heap]]></c>, - <c><![CDATA[heap_frag]]></c>, or <c><![CDATA[binary]]></c>. For more information on - allocators see - <seealso marker="erts_alloc">erts_alloc(3)</seealso>.</item> + bytes of memory (of type "<em><T></em>", thread + <em><I></em>em>)." - The system has run out of memory. <A> + is the allocator that failed to allocate memory, <N> is the + number of bytes that <A> tried to allocate, <T> is the + memory block type that the memory was needed for, and <I> is the + thread identifier. The most common case is that a process stores huge + amounts of data. In this case <T> is most often + <c><![CDATA[heap]]></c>, <c><![CDATA[old_heap]]></c>, + <c><![CDATA[heap_frag]]></c>, or <c><![CDATA[binary]]></c>. + For more information on allocators see + <seealso marker="erts_alloc">erts_alloc(3)</seealso>.</item> <item>"<em><A></em>: Cannot reallocate <em><N></em> - bytes of memory (of type "<em><T></em>")." - Same as - above with the exception that memory was being reallocated - instead of being allocated when the system ran out of memory.</item> + bytes of memory (of type "<em><T></em>", thread + <em><I></em>em>)." - Same as above with the exception that memory + was being reallocated instead of being allocated when the system ran + out of memory.</item> <item>"Unexpected op code <em>N</em>" - Error in compiled code, <c><![CDATA[beam]]></c> file damaged or error in the compiler.</item> <item>"Module <em>Name</em> undefined" <c><![CDATA[|]]></c> "Function @@ -246,6 +248,9 @@ <tag><em>Last scheduled in for | Current call</em></tag> <item>The current function of the process. These fields will not always exist.</item> + <tag><em>Run queue</em></tag> + <item>The identifier of the scheduler run queue in which the process is + running.</item> <tag><em>Spawned by</em></tag> <item>The parent of the process, i.e. the process which executed <c><![CDATA[spawn]]></c> or <c><![CDATA[spawn_link]]></c>.</item> diff --git a/erts/doc/src/epmd.xml b/erts/doc/src/epmd.xml index 963d35c3c8..25f819ab50 100644 --- a/erts/doc/src/epmd.xml +++ b/erts/doc/src/epmd.xml @@ -58,12 +58,12 @@ of the IP address and a port number. The name of the node is an atom on the form of <c><![CDATA[Name@Node]]></c>. The job of the <c><![CDATA[epmd]]></c> daemon is to keep track of which - node name listens on which address. Hence, <c><![CDATA[epmd]]></c> map + node name listens on which address. Hence, <c><![CDATA[epmd]]></c> maps symbolic node names to machine addresses.</p> <p>The TCP/IP <c>epmd</c> daemon actually only keeps track of - the <c>Name</c> (first) part of an Erlang node name, the <c>Host</c> - part (whatever is after the <c><![CDATA[@]]></c> is implicit in the + the <c>Name</c> (first) part of an Erlang node name. The <c>Host</c> + part (whatever is after the <c><![CDATA[@]]></c>) is implicit in the node name where the <c>epmd</c> daemon was actually contacted, as is the IP address where the Erlang node can be reached. Consistent and correct TCP naming services are @@ -77,12 +77,12 @@ <p>The daemon is started automatically by the <c>erl</c> command if the node is to be distributed and there is no running instance present. If automatically launched, - environment variables has to be used to alter the behavior of + environment variables have to be used to alter the behavior of the daemon. See the <seealso marker="#environment_variables">Environment variables</seealso> section below.</p> - <p>If the -daemon argument is not given, the + <p>If the -daemon argument is not given, <c><![CDATA[epmd]]></c> runs as a normal program with the controlling terminal of the shell in which it is started. Normally, it should run as a daemon.</p> @@ -122,7 +122,7 @@ comma-separated list of IP addresses and on the loopback address (which is implicitly added to the list if it has not been specified). This can also be set using the - <c><![CDATA[ERL_EPMD_ADDRESS]]></c> environment variable, see the + <c><![CDATA[ERL_EPMD_ADDRESS]]></c> environment variable. See the section <seealso marker="#environment_variables">Environment variables</seealso> below.</p> </item> @@ -130,7 +130,7 @@ <item> <p>Let this instance of epmd listen to another TCP port than default 4369. This can also be set using the - <c><![CDATA[ERL_EPMD_PORT]]></c> environment variable, see the + <c><![CDATA[ERL_EPMD_PORT]]></c> environment variable. See the section <seealso marker="#environment_variables">Environment variables</seealso> below</p> </item> @@ -153,7 +153,7 @@ <p>With relaxed command checking, the <c>epmd</c> daemon can be killed from the localhost with i.e. <c>epmd -kill</c> even if there are active nodes registered. Normally only daemons with an empty node database can be killed with the <c>epmd -kill</c> command.</p> </item> <item> - <p>The <c>epmd -stop</c> command (and the corresponding messages to epmd, as can be given using <c>erl_interface/ei</c>) is normally always ignored, as it opens up for strange situation when two nodes of the same name can be alive at the same time. A node unregisters itself by just closing the connection to epmd, why the <c>stop</c> command was only intended for use in debugging situations.</p> + <p>The <c>epmd -stop</c> command (and the corresponding messages to epmd, as can be given using <c>erl_interface/ei</c>) is normally always ignored, as it opens up the possibility of a strange situation where two nodes of the same name can be alive at the same time. A node unregisters itself by just closing the connection to epmd, which is why the <c>stop</c> command was only intended for use in debugging situations.</p> <p>With relaxed command checking enabled, you can forcibly unregister live nodes.</p> </item> </list> @@ -166,7 +166,7 @@ <section> <marker id="debug_flags"></marker> <title>DbgExtra options</title> - <p>These options are purely for debugging and testing epmd clients, they should not be used in normal operation.</p> + <p>These options are purely for debugging and testing epmd clients. They should not be used in normal operation.</p> <taglist> <tag><c><![CDATA[-packet_timeout Seconds]]></c></tag> @@ -177,9 +177,9 @@ </item> <tag><c><![CDATA[-delay_accept Seconds]]></c></tag> <item> - <p>To simulate a busy server you can insert a delay between epmd - gets notified about that a new connection is requested and - when the connections gets accepted.</p> + <p>To simulate a busy server you can insert a delay between when epmd + gets notified that a new connection is requested and + when the connection gets accepted.</p> </item> <tag><c><![CDATA[-delay_write Seconds]]></c></tag> <item> @@ -191,15 +191,15 @@ <section> <marker id="interactive_flags"></marker> <title>Interactive options</title> - <p>These options make <c>epmd</c> run as an interactive command displaying the results of sending queries ta an already running instance of <c>epmd</c>. The epmd contacted is always on the local node, but the <c>-port</c> option can be used to select between instances if several are running using different port on the host.</p> + <p>These options make <c>epmd</c> run as an interactive command, displaying the results of sending queries to an already running instance of <c>epmd</c>. The epmd contacted is always on the local node, but the <c>-port</c> option can be used to select between instances if several are running using different ports on the host.</p> <taglist> <tag><c><![CDATA[-port No]]></c></tag> <item> <p>Contacts the <c>epmd</c> listening on the given TCP port number (default 4369). This can also be set using the - <c><![CDATA[ERL_EPMD_PORT]]></c> environment variable, see the + <c><![CDATA[ERL_EPMD_PORT]]></c> environment variable. See the section <seealso marker="#environment_variables">Environment - variables</seealso> below</p> + variables</seealso> below.</p> </item> <tag><c><![CDATA[-names]]></c></tag> <item> @@ -210,7 +210,7 @@ <p>Kill the currently running <c>epmd</c>.</p> <p>Killing the running <c>epmd</c> is only allowed if <c>epmd - -names</c> show an empty database or + -names</c> shows an empty database or <c>-relaxed_command_check</c> was given when the running instance of <c>epmd</c> was started. Note that <c>-relaxed_command_check</c> is given when starting the @@ -228,7 +228,7 @@ <p>This command can only be used when contacting <c>epmd</c> instances started with the <c>-relaxed_command_check</c> flag. Note that relaxed command checking has to be enabled for - the <c>epmd</c> daemon contacted, When running epmd + the <c>epmd</c> daemon contacted. When running epmd interactively, <c>-relaxed_command_check</c> has no effect.</p> </item> @@ -259,7 +259,7 @@ <item> <p>If set prior to start, the <c>epmd</c> daemon will behave as if the <c>-relaxed_command_check</c> option was given at - start-up. If consequently setting this option before starting + start-up. Consequently, if this option is set before starting the Erlang virtual machine, the automatically started <c>epmd</c> will accept the <c>-kill</c> and <c>-stop</c> commands without restrictions.</p> @@ -287,8 +287,8 @@ remote hosts. However, only the query commands are answered (and acted upon) if the query comes from a remote host. It is always an error to try to register a nodename if the client is not a process - located on the same host as the <c>epmd</c> instance is running on, - why such requests are considered hostile and the connection is + located on the same host as the <c>epmd</c> instance is running on- + such requests are considered hostile and the connection is immediately closed.</p> <p>The queries accepted from remote nodes are:</p> @@ -307,3 +307,4 @@ </comref> + diff --git a/erts/doc/src/erl.xml b/erts/doc/src/erl.xml index 9724a1345a..f856b9ab86 100644 --- a/erts/doc/src/erl.xml +++ b/erts/doc/src/erl.xml @@ -495,7 +495,7 @@ <c><![CDATA[werl]]></c>, not <c><![CDATA[erl]]></c> (<c><![CDATA[oldshell]]></c>). Note also that <c><![CDATA[Ctrl-Break]]></c> is used instead of <c><![CDATA[Ctrl-C]]></c> on Windows.</p> </item> - <tag><c><![CDATA[+c]]></c></tag> + <tag><marker id="+c"><c><![CDATA[+c]]></c></marker></tag> <item> <p>Disable compensation for sudden changes of system time.</p> <p>Normally, <c><![CDATA[erlang:now/0]]></c> will not immediately reflect @@ -510,6 +510,9 @@ reflect the current system time. Note that timers are based on <c><![CDATA[erlang:now/0]]></c>. If the system time jumps, timers then time out at the wrong time.</p> + <p><em>NOTE</em>: You can check whether the adjustment is enabled or + disabled by calling + <seealso marker="erlang#system_info_tolerant_timeofday">erlang:system_info(tolerant_timeofday)</seealso>.</p> </item> <tag><c><![CDATA[+d]]></c></tag> <item> @@ -848,6 +851,19 @@ </p> </item> <tag><marker id="+SDio"><c><![CDATA[+SDio IOSchedulers]]></c></marker></tag> + <item> + <p>Sets the number of dirty I/O scheduler threads to create when threading + support has been enabled. The valid range is 0-1024. By default, the number + of dirty I/O scheduler threads created is 10, same as the default number of + threads in the <seealso marker="#async_thread_pool_size">async thread pool + </seealso>. + </p> + <p>This option is ignored if the emulator doesn't have threading support + enabled. Currently, <em>this option is experimental</em> and is supported only + if the emulator was configured and built with support for dirty schedulers + enabled (it's disabled by default). + </p> + </item> <tag><c><![CDATA[+sFlag Value]]></c></tag> <item> <p>Scheduling specific flags.</p> @@ -1170,7 +1186,7 @@ utilization. </p> </item> - <tag><marker id="+swct"><c>+sws very_eager|eager|medium|lazy|very_lazy</c></marker></tag> + <tag><marker id="+swct"><c>+swct very_eager|eager|medium|lazy|very_lazy</c></marker></tag> <item> <p> Set scheduler wake cleanup threshold. Default is <c>medium</c>. diff --git a/erts/doc/src/erl_driver.xml b/erts/doc/src/erl_driver.xml index ad37813ac0..4a1aab75c7 100644 --- a/erts/doc/src/erl_driver.xml +++ b/erts/doc/src/erl_driver.xml @@ -546,6 +546,7 @@ typedef struct ErlDrvSysInfo { int scheduler_threads; int nif_major_version; int nif_minor_version; + int dirty_scheduler_support; } ErlDrvSysInfo; </code> @@ -610,6 +611,10 @@ typedef struct ErlDrvSysInfo { <tag><c>nif_minor_version</c></tag> <item>The value of <c>ERL_NIF_MINOR_VERSION</c> when the runtime system was compiled. </item> + <tag><c>dirty_scheduler_support</c></tag> + <item>A value <c>!= 0</c> if the runtime system has support for dirty scheduler threads; + otherwise <c>0</c>. + </item> </taglist> </item> <tag><marker id="ErlDrvBinary"/>ErlDrvBinary</tag> diff --git a/erts/doc/src/erl_nif.xml b/erts/doc/src/erl_nif.xml index 6b1f4cccf8..3de94be9ff 100644 --- a/erts/doc/src/erl_nif.xml +++ b/erts/doc/src/erl_nif.xml @@ -168,16 +168,18 @@ ok <p><marker id="lengthy_work"/> As mentioned in the <seealso marker="#WARNING">warning</seealso> text at the beginning of this document it is of vital importance that a native function - does return relatively fast. It is hard to give an exact maximum amount + return relatively quickly. It is hard to give an exact maximum amount of time that a native function is allowed to work, but as a rule of thumb - a well behaving native function should return to its caller before a + a well-behaving native function should return to its caller before a millisecond has passed. This can be achieved using different approaches. - If you have full control over the code that are to execute in the native + If you have full control over the code to execute in the native function, the best approach is to divide the work into multiple chunks of - work and call the native function multiple times. Function + work and call the native function multiple times, either directly from Erlang code + or by having a native function schedule a future NIF call via the + <seealso marker="#enif_schedule_nif"> enif_schedule_nif</seealso> function. Function <seealso marker="#enif_consume_timeslice">enif_consume_timeslice</seealso> can be - used this facilitate such work division. In some cases, however, this might not - be possible, e.g. when calling third party libraries. Then you typically want + used to help with such work division. In some cases, however, this might not + be possible, e.g. when calling third-party libraries. Then you typically want to dispatch the work to another thread, return from the native function, and wait for the result. The thread can send the result back to the calling thread using message passing. Information @@ -342,37 +344,38 @@ ok libraries might however fail if deprecated features are used. </p></item> - <tag>Dirty NIFs</tag> - <item><p><marker id="dirty_nifs"/><em>Note that the dirty NIF functionality - is experimental</em> and that you have to enable support for dirty - schedulers when building OTP in order to try the functionality out. Native functions + <tag>Long-running NIFs</tag> + <item><p><marker id="dirty_nifs"/>Native functions <seealso marker="#lengthy_work"> must normally run quickly</seealso>, as explained earlier in this document. They generally should execute for no more than a millisecond. But not all native functions can execute so quickly; for example, functions that encrypt large blocks of data or perform lengthy file system operations can often run for tens of seconds or more.</p> - <p>A NIF that cannot execute in a millisecond or less is called a "dirty NIF" since - it performs work that the Erlang runtime cannot handle cleanly. Applications - that make use of such functions must indicate to the runtime that the functions are + <p>If the functionality of a long-running NIF can be split so that its work can be + achieved through a series of shorter NIF calls, the application can either make that series + of NIF calls from the Erlang level, or it can call a NIF that first performs a chunk of the + work, then invokes the <seealso marker="#enif_schedule_nif">enif_schedule_nif</seealso> + function to schedule another NIF call to perform the next chunk. The final call scheduled + in this manner can then return the overall result. Breaking up a long-running function in + this manner enables the VM to regain control between calls to the NIFs, thereby avoiding + degraded responsiveness, scheduler load balancing problems, and other strange behaviours.</p> + <p>A NIF that cannot be split and cannot execute in a millisecond or less is called a "dirty NIF" + because it performs work that the Erlang runtime cannot handle cleanly. + <em>Note that the dirty NIF functionality described here is experimental</em> and that you have to + enable support for dirty schedulers when building OTP in order to try the functionality out. + Applications that make use of such functions must indicate to the runtime that the functions are dirty so they can be handled specially. To schedule a dirty NIF for execution, the - application calls <seealso marker="#enif_schedule_dirty_nif">enif_schedule_dirty_nif</seealso>, - passing to it a pointer to the dirty NIF to be executed and indicating with a flag + appropriate flags value can be set for the NIF in its <seealso marker="#ErlNifFunc">ErlNifFunc</seealso> + entry, or the application can call <seealso marker="#enif_schedule_nif">enif_schedule_nif</seealso>, + passing to it a pointer to the dirty NIF to be executed and indicating with the <c>flags</c> argument whether it expects the operation to be CPU-bound or I/O-bound.</p> - <p>All dirty NIFs must ultimately invoke the <seealso marker="#enif_schedule_dirty_nif_finalizer"> - enif_schedule_dirty_nif_finalizer</seealso> as their final action, passing to it the - result they wish to return to the original caller. A finalizer function can either - receive the result and return it directly, or it can return a different value instead. - For convenience, the NIF API provides the <seealso marker="#enif_dirty_nif_finalizer"> - enif_dirty_nif_finalizer</seealso> function that applications can use as a finalizer; - it simply returns its result argument.</p> <note><p>Dirty NIF support is available only when the emulator is configured with dirty schedulers enabled. This feature is currently disabled by default. To determine whether the dirty NIF API is available, native code can check to see if the C preprocessor macro <c>ERL_NIF_DIRTY_SCHEDULER_SUPPORT</c> is defined. Also, if the Erlang runtime was built without threading support, dirty schedulers are disabled. To check at runtime for the presence - of dirty scheduler threads, code can call the <seealso marker="#enif_have_dirty_schedulers"><c> - enif_have_dirty_schedulers()</c></seealso> API function, which returns true if dirty - scheduler threads are present, false otherwise.</p></note> + of dirty scheduler threads, code can use the <seealso marker="#enif_system_info"><c> + enif_system_info()</c></seealso> API function.</p></note> </item> </taglist> </section> @@ -498,6 +501,7 @@ typedef struct { const char* <em>name</em>; unsigned <em>arity</em>; ERL_NIF_TERM (*<em>fptr</em>)(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); + unsigned flags; } ErlNifFunc; </code> <p>Describes a NIF by its name, arity and implementation. @@ -508,7 +512,17 @@ typedef struct { will thus denote the Nth argument to the NIF. Note that the <c>argc</c> argument allows for the same C function to implement several Erlang functions with different arity (but - same name probably).</p> + same name probably). For a regular NIF, <c>flags</c> is 0 (and + so its value can be omitted for statically initialized <c>ErlNifFunc</c> + instances), or it can be used to indicate that the NIF is a <seealso + marker="#dirty_nifs">dirty NIF</seealso> that should be executed + on a dirty scheduler thread (<em>note that the dirty NIF functionality + described here is experimental</em> and that you have to enable + support for dirty schedulers when building OTP in order to try the + functionality out). If the dirty NIF is expected to be + CPU-bound, its <c>flags</c> field should be set to + <c>ERL_NIF_DIRTY_JOB_CPU_BOUND</c>, or for I/O-bound jobs, + <c>ERL_NIF_DIRTY_JOB_IO_BOUND</c>.</p> </item> <tag><marker id="ErlNifBinary"/>ErlNifBinary</tag> <item> @@ -672,18 +686,6 @@ typedef enum { See also the <seealso marker="#WARNING">warning</seealso> text at the beginning of this document.</p> </desc> </func> - <func><name><ret>ERL_NIF_TERM</ret><nametext>enif_dirty_nif_finalizer(ErlNifEnv* env, ERL_NIF_TERM result)</nametext></name> - <fsummary>Simple dirty NIF result finalizer</fsummary> - <desc> - <p>A convenience function that a dirty NIF can use as a finalizer that simply - return its <c>result</c> argument as its return value. This function is provided - for dirty NIFs with results that should be returned directly to the original caller.</p> - <note><p>This function is available only when the emulator is configured with dirty - schedulers enabled. This feature is currently disabled by default. To determine whether - the dirty NIF API is available, native code can check to see if the C preprocessor macro - <c>ERL_NIF_DIRTY_SCHEDULER_SUPPORT</c> is defined.</p></note> - </desc> - </func> <func><name><ret>int</ret><nametext>enif_equal_tids(ErlNifTid tid1, ErlNifTid tid2)</nametext></name> <fsummary></fsummary> <desc><p>Same as <seealso marker="erl_driver#erl_drv_equal_tids">erl_drv_equal_tids</seealso>. @@ -804,22 +806,6 @@ typedef enum { and return true, or return false if <c>term</c> is not an unsigned integer or is outside the bounds of type <c>unsigned long</c>.</p></desc> </func> - <func><name><ret>int</ret><nametext>enif_have_dirty_schedulers()</nametext></name> - <fsummary>Runtime check for the presence of dirty scheduler threads</fsummary> - <desc> - <p>Check at runtime for the presence of dirty scheduler threads. If the emulator is - built with threading support, dirty scheduler threads are available and - <c>enif_have_dirty_schedulers()</c> returns true. If the emulator was built without - threading support, <c>enif_have_dirty_schedulers()</c> returns false.</p> - <p>If dirty scheduler threads are not available in the emulator, calls to - <c>enif_schedule_dirty_nif</c> and <c>enif_schedule_dirty_nif_finalizer</c> result in - the NIF and finalizer functions being called directly within the calling thread.</p> - <note><p>This function is available only when the emulator is configured with dirty - schedulers enabled. This feature is currently disabled by default. To determine whether - the dirty NIF API is available, native code can check to see if the C preprocessor macro - <c>ERL_NIF_DIRTY_SCHEDULER_SUPPORT</c> is defined.</p></note> - </desc> - </func> <func><name><ret>int</ret><nametext>enif_inspect_binary(ErlNifEnv* env, ERL_NIF_TERM bin_term, ErlNifBinary* bin)</nametext></name> <fsummary>Inspect the content of a binary</fsummary> <desc><p>Initialize the structure pointed to by <c>bin</c> with @@ -873,8 +859,8 @@ typedef enum { <p>Check to see if the current NIF is executing on a dirty scheduler thread. If the emulator is built with threading support, calling <c>enif_is_on_dirty_scheduler</c> from within a dirty NIF returns true. It returns false when the calling NIF is a regular - NIF or a NIF finalizer, both of which run on normal scheduler threads, or when the emulator - is built without threading support.</p> + NIF running on a normal scheduler thread, or when the emulator is built without threading + support.</p> <note><p>This function is available only when the emulator is configured with dirty schedulers enabled. This feature is currently disabled by default. To determine whether the dirty NIF API is available, native code can check to see if the C preprocessor macro @@ -1245,46 +1231,29 @@ typedef enum { <desc><p>Same as <seealso marker="erl_driver#erl_drv_rwlock_tryrwlock">erl_drv_rwlock_tryrwlock</seealso>. </p></desc> </func> - <func><name><ret>ERL_NIF_TERM</ret><nametext>enif_schedule_dirty_nif(ErlNifEnv* env, int flags, ERL_NIF_TERM (*fp)(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]), int argc, const ERL_NIF_TERM argv[])</nametext></name> - <fsummary>Schedule a dirty NIF for execution</fsummary> + <func><name><ret>ERL_NIF_TERM</ret><nametext>enif_schedule_nif(ErlNifEnv* env, const char* fun_name, int flags, ERL_NIF_TERM (*fp)(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]), int argc, const ERL_NIF_TERM argv[])</nametext></name> + <fsummary>Schedule a NIF for execution</fsummary> <desc> - <p>Schedule dirty NIF <c>fp</c> to execute a long-running operation. The <c>flags</c> - argument must be set to either <c>ERL_NIF_DIRTY_JOB_CPU_BOUND</c> if the job is expected to - be primarily CPU-bound, or <c>ERL_NIF_DIRTY_JOB_IO_BOUND</c> for jobs that will be - I/O-bound. The <c>argc</c> and <c>argv</c> arguments can either be the originals passed - into the calling NIF, or they can be values created by the calling NIF. The calling - NIF must use the return value of <c>enif_schedule_dirty_nif</c> as its own return value.</p> - <p>Be aware that <c>enif_schedule_dirty_nif</c>, as its name implies, only schedules the - dirty NIF for future execution. The calling NIF does not block waiting for the dirty NIF to - execute and return, which means that the calling NIF can't expect to receive the dirty NIF + <p>Schedule NIF <c>fp</c> to execute. This function allows an application to break up long-running + work into multiple regular NIF calls or to schedule a <seealso marker="#dirty_nifs">dirty NIF</seealso> + to execute on a dirty scheduler thread (<em>note that the dirty NIF functionality described here is + experimental</em> and that you have to enable support for dirty schedulers when building OTP in + order to try the functionality out).</p> + <p>The <c>fun_name</c> argument provides a name for the NIF being scheduled for execution. If it cannot + be converted to an atom, <c>enif_schedule_nif</c> returns a <c>badarg</c> exception.</p> + <p>The <c>flags</c> argument must be set to 0 for a regular NIF, or if the emulator was built the + experimental dirty scheduler support enabled, <c>flags</c> can be set to either <c>ERL_NIF_DIRTY_JOB_CPU_BOUND</c> + if the job is expected to be primarily CPU-bound, or <c>ERL_NIF_DIRTY_JOB_IO_BOUND</c> for jobs that will + be I/O-bound. If dirty scheduler threads are not available in the emulator, a try to schedule such a job + will result in a <c>badarg</c> exception.</p> + + <p>The <c>argc</c> and <c>argv</c> arguments can either be the originals passed into the calling NIF, or + they can be values created by the calling NIF.</p> + <p>The calling NIF must use the return value of <c>enif_schedule_nif</c> as its own return value.</p> + <p>Be aware that <c>enif_schedule_nif</c>, as its name implies, only schedules the + NIF for future execution. The calling NIF does not block waiting for the scheduled NIF to + execute and return, which means that the calling NIF can't expect to receive the scheduled NIF return value and use it for further operations.</p> - <p>A dirty NIF may not invoke the <seealso marker="#enif_make_badarg">enif_make_badarg</seealso> - to raise an exception. If it wishes to return an exception, the dirty NIF should pass a - regular result indicating the exception details to its finalizer, and allow the finalizer - to raise the exception on its behalf.</p> - <note><p>This function is available only when the emulator is configured with dirty schedulers - enabled. This feature is currently disabled by default. To determine whether the dirty NIF API - is available, native code can check to see if the C preprocessor macro - <c>ERL_NIF_DIRTY_SCHEDULER_SUPPORT</c> is defined.</p></note> - </desc> - </func> - <func><name><ret>ERL_NIF_TERM</ret><nametext>enif_schedule_dirty_nif_finalizer(ErlNifEnv* env, ERL_NIF_TERM result, ERL_NIF_TERM (*fp)(ErlNifEnv* env, ERL_NIF_TERM result))</nametext></name> - <fsummary>Schedule a dirty NIF finalizer</fsummary> - <desc> - <p>When a dirty NIF finishes executing, it must schedule a finalizer function to return - its result to the original NIF caller. The dirty NIF passes <c>result</c> as the value it - wants the finalizer to use as the return value. The <c>fp</c> argument is a pointer to the - finalizer function. The NIF API provides the <seealso marker="#enif_dirty_nif_finalizer"> - enif_dirty_nif_finalizer</seealso> function that can be used as a finalizer that simply - returns its <c>result</c> argument. You are also free to write your own custom finalizer - that uses <c>result</c> to derive a different return value, or ignores <c>result</c> - entirely and returns a completely different value.</p> - <p>Without exception, all dirty NIFs must invoke <c>enif_schedule_dirty_nif_finalizer</c> - to complete their execution.</p> - <note><p>This function is available only when the emulator is configured with dirty - schedulers enabled. This feature is currently disabled by default. To determine whether - the dirty NIF API is available, native code can check to see if the C preprocessor macro - <c>ERL_NIF_DIRTY_SCHEDULER_SUPPORT</c> is defined.</p></note> </desc> </func> <func><name><ret>ErlNifPid *</ret><nametext>enif_self(ErlNifEnv* caller_env, ErlNifPid* pid)</nametext></name> @@ -1384,4 +1353,3 @@ typedef enum { <p><seealso marker="erlang#load_nif-2">erlang:load_nif/2</seealso></p> </section> </cref> - diff --git a/erts/doc/src/erl_prim_loader.xml b/erts/doc/src/erl_prim_loader.xml index 6751deda4d..171f84decc 100644 --- a/erts/doc/src/erl_prim_loader.xml +++ b/erts/doc/src/erl_prim_loader.xml @@ -148,6 +148,22 @@ </desc> </func> <func> + <name name="read_link_info" arity="1"/> + <fsummary>Get information about a link or file</fsummary> + <desc> + <p>This function works like + <seealso marker="#read_file_info/1">read_file_info/1</seealso> + except that if <c><anno>Filename</anno></c> is a symbolic link, + information about the link will be returned in the <c>file_info</c> + record and the <c>type</c> field of the record will be set to + <c>symlink</c>.</p> + <p>If <c><anno>Filename</anno></c> is not a symbolic link, this function + returns exactly the same result as <c>read_file_info/1</c>. + On platforms that do not support symbolic links, this function + is always equivalent to <c>read_file_info/1</c>.</p> + </desc> + </func> + <func> <name name="set_path" arity="1"/> <fsummary>Set the path of the loader</fsummary> <desc> diff --git a/erts/doc/src/erlang.xml b/erts/doc/src/erlang.xml index b06d5aeb12..84168397f6 100644 --- a/erts/doc/src/erlang.xml +++ b/erts/doc/src/erlang.xml @@ -887,7 +887,7 @@ <fsummary>Print a term on standard output</fsummary> <desc> <p>Prints a text representation of <c><anno>Term</anno></c> on the standard - output.</p> + output. On OSE the term is printed to the ramlog.</p> <warning> <p>This BIF is intended for debugging only.</p> </warning> @@ -1784,6 +1784,15 @@ os_prompt% </pre> </desc> </func> <func> + <name name="is_map" arity="1"/> + <fsummary>Check whether a term is a map</fsummary> + <desc> + <p>Returns <c>true</c> if <c><anno>Term</anno></c> is a map; + otherwise returns <c>false</c>.</p> + <p>Allowed in guard tests.</p> + </desc> + </func> + <func> <name name="is_number" arity="1"/> <fsummary>Check whether a term is a number</fsummary> <desc> @@ -2220,6 +2229,17 @@ os_prompt% </pre> </desc> </func> <func> + <name name="map_size" arity="1"/> + <fsummary>Return the size of a map</fsummary> + <desc> + <p>Returns an integer which is the number of key-value pairs in <c><anno>Map</anno></c>.</p> + <pre> +> <input>map_size(#{a=>1, b=>2, c=>3}).</input> +3</pre> + <p>Allowed in guard tests.</p> + </desc> + </func> + <func> <name name="max" arity="2"/> <fsummary>Return the largest of two term</fsummary> <desc> @@ -4822,6 +4842,8 @@ true</pre> <c><anno>Node</anno></c> does not exist, a useless pid is returned. Otherwise works like <seealso marker="#spawn_opt/4">spawn_opt/4</seealso>.</p> + <note><p>The <c>monitor</c> option is currently not supported by + <c>spawn_opt/5</c>.</p></note> </desc> </func> <func> @@ -4946,7 +4968,7 @@ true</pre> <desc> <p>Note that the run-time is the sum of the run-time for all threads in the Erlang run-time system and may therefore be greater - than the wall-clock time.</p> + than the wall-clock time. The time is returned in milliseconds.</p> <pre> > <input>statistics(runtime).</input> {1690,1620} @@ -6273,6 +6295,13 @@ ok (<seealso marker="erts:erl_driver#driver_async">driver_async()</seealso>) as an integer.</p> </item> + <tag><marker id="system_info_tolerant_timeofday"><c>tolerant_timeofday</c></marker></tag> + <item> + <p>Returns whether compensation for sudden changes of system + time is <c>enabled</c> or <c>disabled</c>.</p> + <p>See also <seealso marker="erts:erl#+c">+c</seealso> + command line flag.</p> + </item> <tag><c>trace_control_word</c></tag> <item> <p>Returns the value of the node's trace control word. diff --git a/erts/doc/src/erts_alloc.xml b/erts/doc/src/erts_alloc.xml index c9eca39a99..1ade41f1aa 100644 --- a/erts/doc/src/erts_alloc.xml +++ b/erts/doc/src/erts_alloc.xml @@ -4,7 +4,7 @@ <cref> <header> <copyright> - <year>2002</year><year>2013</year> + <year>2002</year><year>2014</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -531,15 +531,9 @@ <p>Multiple, thread specific instances of the allocator. This option will only have any effect on the runtime system with SMP support. Default behaviour on the runtime system with - SMP support:</p> - <taglist> - <tag><c>ll_alloc</c></tag> - <item><c>1</c> instance.</item> - <tag>Other allocators</tag> - <item><c>NoSchedulers+1</c> instances. Each scheduler will use - a lock-free instance of its own and other threads will use - a common instance.</item> - </taglist> + SMP support is <c>NoSchedulers+1</c> instances. Each scheduler will use + a lock-free instance of its own and other threads will use + a common instance.</p> <p>It was previously (before ERTS version 5.9) possible to configure a smaller amount of thread specific instances than schedulers. This is, however, not possible any more.</p> diff --git a/erts/doc/src/escript.xml b/erts/doc/src/escript.xml index 1abbdb2180..9159d68f60 100644 --- a/erts/doc/src/escript.xml +++ b/erts/doc/src/escript.xml @@ -236,8 +236,13 @@ factorial 5 = 120 <v>EmuArgs = string() | 'undefined'</v> <v>Body = {source, SourceCode} | {beam, BeamCode} - | {archive, ZipArchive}</v> - <v>SourceCode = BeamCode = ZipArchive = binary()</v> + | {archive, ZipArchive} + | {archive, ZipFiles, ZipOptions}</v> + <v>SourceCode = BeamCode = file:filename() | binary()</v> + <v>ZipArchive = <seealso marker="stdlib:zip#type-filename">zip:filename()</seealso> | binary()</v> + <v>ZipFiles = [ZipFile]</v> + <v>ZipFile = file:filename() | {file:filename(), binary()} | {file:filename(), binary(), file:file_info()}</v> + <v>ZipOptions = [<seealso marker="stdlib:zip#type-create_option">zip:create_option()</seealso>]</v> </type> <desc> <p>The <marker id="create_2"></marker> <c>create/2</c> diff --git a/erts/doc/src/notes.xml b/erts/doc/src/notes.xml index b4ebef72f4..743369951f 100644 --- a/erts/doc/src/notes.xml +++ b/erts/doc/src/notes.xml @@ -30,6 +30,1113 @@ </header> <p>This document describes the changes made to the ERTS application.</p> +<section><title>Erts 6.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + General documentation updates.</p> + <p> + Own Id: OTP-12052</p> + </item> + <item> + <p>A bug in the VM code implementing sending of signals + to ports could cause the receiving port queue to remain + in a busy state forever. When this state had been + reached, processes sending command signals to the port + either got suspended forever, or, if the <c>nosuspend</c> + feature was used, always failed to send to the port. This + bug was introduced in ERTS version 5.10.</p> + <p>In order for this bug to be triggered on a port, one + had to at least once utilize the <c>nosuspend</c> + functionality when passing a signal to the port. This by + either calling</p> <list> <item> <seealso + marker="erlang#port_command/3"><c>port_command(Port, + Data, [nosuspend | Options])</c></seealso>, </item> + <item> <seealso + marker="erlang#send/3"><c>erlang:send(Port, {PortOwner, + {command, Data}}, [nosuspend | Options])</c></seealso>, + </item> <item> <seealso + marker="erlang#send_nosuspend/2"><c>erlang:send_nosuspend(Port, + {PortOwner, {command, Data}})</c></seealso>, or </item> + <item> <seealso + marker="erlang#send_nosuspend/3"><c>erlang:send_nosuspend(Port, + {PortOwner, {command, Data}}, Options)</c></seealso>. + </item> </list> + <p>Thanks Vasily Demidenok for reporting the issue, and + Sergey Kudryashov for providing a testcase.</p> + <p> + Own Id: OTP-12082 Aux Id: OTP-10336 </p> + </item> + <item> + <p> + Fix size overflow bug at memory allocation. A memory + allocation call, with an insane size close to the entire + address space, could return successfully as if it had + allocated just a few bytes. (Thanks to Don A. Bailey for + reporting)</p> + <p> + Own Id: OTP-12091</p> + </item> + <item> + <p> + Fix various issues where negating a signed integer would + trigger undefined behaviour. This fixes issues in the + enif_make_int64 interface and some edge cases inside the + erlang runtime system.</p> + <p> + Own Id: OTP-12097</p> + </item> + <item> + <p> + The documentation erroneously listed the <seealso + marker="erl#+swct"><c>+swct</c></seealso> command line + argument under <c>+sws</c>.</p> + <p> + Own Id: OTP-12102 Aux Id: OTP-10994 </p> + </item> + <item> + <p> + Profiling messages could be delivered out of order when + profiling on <c>runnable_procs</c> and/or + <c>runnable_ports</c> using <seealso + marker="erlang#system_profile/2"><c>erlang:system_profile/2</c></seealso>. + This bug was introduced in ERTS version 5.10.</p> + <p> + Own Id: OTP-12105 Aux Id: OTP-10336 </p> + </item> + <item> + <p> + Various logging fixes, including: Add run queue index to + the process dump in crash dumps.<br/> Add thread index to + enomem slogan when crashing.<br/> Remove error logger + message for sending messages to old instances of the same + node.</p> + <p> + Own Id: OTP-12115</p> + </item> + <item> + <p> + Fix compiler warnings reported by LLVM</p> + <p> + Own Id: OTP-12138</p> + </item> + <item> + <p> + Correct conversion of <c>MIN_SMALL</c> by + <c>list_to_integer/1</c> and <c>binary_to_integer/1</c>. + The bug produced an unnormalized bignum which can cause + strange behavior such as comparing different to a correct + <c>MIN_SMALL</c> integer. The value <c>MIN_SMALL</c> is + <c>-(1 bsl 27) = -134217728</c> on a 32-bit VM and <c>-(1 + bsl 59) = -576460752303423488</c> on a 64-bit VM. (Thanks + to Jesper Louis Andersen, Mikael Pettersson and Anthony + Ramine for report, patch and optimization suggestion)</p> + <p> + Own Id: OTP-12140</p> + </item> + <item> + <p> + Fix bug in <c>term_to_binary</c> that reallocates binary + with inconsistent size information. Bug has never been + confirmed to be the cause of any faulty behavior.</p> + <p> + Own Id: OTP-12141</p> + </item> + <item> + <p> + Real_path method used while prim loading archive files + was not taking into account the fact that windows + directory symlinks can be across different drives.</p> + <p> + Own Id: OTP-12155</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Add log2 histogram to lcnt for lock wait time</p> + <p> + Own Id: OTP-12059</p> + </item> + <item> + <p> + Introduced <seealso + marker="erl_nif#enif_schedule_nif"><c>enif_schedule_nif()</c></seealso> + to the NIF API.</p> + <p> + The <c>enif_schedule_nif()</c> function allows a + long-running NIF to be broken into separate NIF + invocations without the help of a wrapper function + written in Erlang. The NIF first executes part of the + long-running task, then calls <c>enif_schedule_nif()</c> + to schedule a NIF for later execution to continue the + task. Any number of NIFs can be scheduled in this manner, + one after another. Since the emulator regains control + between invocations, this helps avoid problems caused by + native code tying up scheduler threads for too long.</p> + <p> + The <c>enif_schedule_nif()</c> function also replaces the + <c>enif_schedule_dirty_nif()</c> in the experimental + dirty NIF API. Note that the only incompatible changes + made are in the experimental dirty NIF API.</p> + <p> + See the <seealso marker="erl_nif">NIF + documentation</seealso> for more information.</p> + <p> + Thanks to Steve Vinoski.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-12128</p> + </item> + </list> + </section> + +</section> + +<section><title>Erts 6.1.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + OTP-11850 fixed filelib:wildcard/1 to work with broken + symlinks. This correction, however, introduced problems + since symlinks were no longer followed for functions like + filelib:ensure_dir/1, filelib:is_dir/1, + filelib:file_size/1, etc. This is now corrected.</p> + <p> + Own Id: OTP-12054 Aux Id: seq12660 </p> + </item> + </list> + </section> + +</section> + +<section><title>Erts 6.1.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fixed ETHR_FORCE_INLINE which caused the build to break + on some platforms without adequate thread support + (VxWorks).</p> + <p> + Own Id: OTP-12010</p> + </item> + </list> + </section> + +</section> + +<section><title>Erts 6.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p>The documentation for <c>spawn_opt/5</c> now has a + note mentioning that the <c>monitor</c> option is not + supported.</p> + <p> + Own Id: OTP-11849</p> + </item> + <item> + <p> + Fix broken system monitoring of <c>large_heap</c> for + non-smp VM. No message for <c>large_heap</c> was ever + sent on non-smp VM. Bug exist since R16B.</p> + <p> + Own Id: OTP-11852</p> + </item> + <item> + <p> + The emulator without SMP support crashed when passing a + message to a process without enough heap space for the + message. This bug was introduced in <c>erts-6.0</c>.</p> + <p> + Own Id: OTP-11887 Aux Id: OTP-11388 </p> + </item> + <item> + <p> + Fix race between ETS table deletion and unfixation that + could cause VM crash. The race could happen between a + terminating process that does not own the table but has a + fixation on it and another process that deletes the table + (maybe the owner terminating) at the same time. Bug + existed since R15B02.</p> + <p> + Own Id: OTP-11892</p> + </item> + <item> + <p>The string following the <c>-eval</c> option when + invoking <c>erl</c> would not be properly translated from + UTF-8 to a list of Unicode characters (as would the + arguments for <c>-run</c>).</p> + <p>That bug would cause the build of Erlang/OTP to fail + when building in a directory whose pathname contained + non-US ASCII characters encoded in UTF-8. (Thanks to Eric + Pailleau for reporting this bug.)</p> + <p> + Own Id: OTP-11916</p> + </item> + <item> + <p> + Fix erts_debug:size/1 to handle Map sizes</p> + <p> + Own Id: OTP-11923</p> + </item> + <item> + <p> + Removed <c>erlang:bitstr_to_list/1</c> and + <c>erlang:list_to_bitstr/1</c>. They were added by + mistake, and have always raised an <c>undefined</c> + exception when called.</p> + <p> + Own Id: OTP-11942</p> + </item> + <item> + <p> + Fixed compilation using mingw-w64 on Windows.</p> + <p> + Thanks to Jani Hakala.</p> + <p> + Own Id: OTP-11945</p> + </item> + <item> + <p> + The git sha is no longer printed in the shell start + header when erlang is built from a tagged git release.</p> + <p> + Own Id: OTP-11961</p> + </item> + <item> + <p> + Fixed a bug where <c>send</c> trace events were + erroneously dropped when the send was done to a + registered process. This bug was introduced in R16B.</p> + <p> + Own Id: OTP-11968</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p>The following native functions now bump an appropriate + amount of reductions and yield when out of + reductions:</p> <list> + <item><c>erlang:binary_to_list/1</c></item> + <item><c>erlang:binary_to_list/3</c></item> + <item><c>erlang:bitstring_to_list/1</c></item> + <item><c>erlang:list_to_binary/1</c></item> + <item><c>erlang:iolist_to_binary/1</c></item> + <item><c>erlang:list_to_bitstring/1</c></item> + <item><c>binary:list_to_bin/1</c></item> </list> + <p>Characteristics impact:</p> <taglist> + <tag>Performance</tag> <item>The functions converting + from lists got a performance loss for very small lists, + and a performance gain for very large lists.</item> + <tag>Priority</tag> <item>Previously a process executing + one of these functions effectively got an unfair priority + boost. This priority boost depended on the input size. + The larger the input was, the larger the priority boost + got. This unfair priority boost is now lost. </item> + </taglist> + <p> + Own Id: OTP-11888</p> + </item> + <item> + <p> + The systemd features of epmd have been removed from epmd + by default. To enable them you have to build erlang with + the configure option --enable-systemd.</p> + <p> + Own Id: OTP-11921</p> + </item> + <item> + <p> + Removed Erlang wrapper code used when calling + <c>binary_to_term/1</c>, and <c>binary_to_term/2</c>. + This improves the performance of these BIFs especially + when they are called with small binaries as input.</p> + <p> + Own Id: OTP-11931</p> + </item> + <item> + <p> + Add erlang:system_info(tolerant_timeofday), an API to + check whether compensation for sudden changes of system + time is enabled or not.</p> + <p> + Own Id: OTP-11970</p> + </item> + </list> + </section> + +</section> + +<section><title>Erts 6.0.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fix broken system monitoring of <c>large_heap</c> for + non-smp VM. No message for <c>large_heap</c> was ever + sent on non-smp VM. Bug exist since R16B.</p> + <p> + Own Id: OTP-11852</p> + </item> + <item> + <p> + Fixed type spec of <c>erlang:system_info/1</c>.</p> + <p> + Own Id: OTP-11859 Aux Id: OTP-11615 </p> + </item> + </list> + </section> + +</section> + +<section><title>Erts 6.0</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + The option dupnames did not work as intended in re. When + looking for names with {capture, [Name, ...]}, re:run + returned a random instance of the match for that name, + instead of the leftmost matching instance, which was what + the documentation stated. This is now corrected to adhere + to the documentation. The option {capture,all_names} + along with a re:inspect/2 function is also added to + further help in using named subpatterns.</p> + <p> + Own Id: OTP-11205</p> + </item> + <item> + <p> + Allow loading of NIF library with unicode path name</p> + <p> + Own Id: OTP-11408</p> + </item> + <item> + <p> + Allow loading of driver with unicode path name</p> + <p> + Own Id: OTP-11549</p> + </item> + <item> + <p> + Fixed a bug where starting Erlang without having an open + stdin on fd 0 would sometimes deadlock the emulator when + terminating.</p> + <p> + Own Id: OTP-11558</p> + </item> + <item> + <p> + The option '-names' in epmd now works on Windows (Thanks + to Johannes Weißl)</p> + <p> + Own Id: OTP-11565</p> + </item> + <item> + <p> + Correction of the examples in escript documentation. + (Thanks to Pierre Fenoll).</p> + <p> + Own Id: OTP-11577</p> + </item> + <item> + <p> + Fix bs_get_integer instruction</p> + <p> + The instruction bs_get_integer could unnecessarily + trigger a garbage collection in failure cases which is + unwanted or outright dangerous.</p> + <p> + Ex:</p> + <p> + <<X:Sz,_/bits>> = <<"some + binary">></p> + <p> + Previously, if Sz induced X to a bignum it would reserved + memory size this on the heap via a garbage collection + before checking if the size could actually match.</p> + <p> + It will now check the binary size before triggering a + collection.</p> + <p> + Own Id: OTP-11581</p> + </item> + <item> + <p> + Remove heap space overestimation in <c>binary_to_term</c> + (and remote message reception) for integers in the + intervals [-2147483648,-1] and [256,2147483647] on 64-bit + emulators.</p> + <p> + Own Id: OTP-11585</p> + </item> + <item> + <p> + Add support for detecting the separate tinfo library from + ncurses (Thanks to Dirkjan Ochtman)</p> + <p> + Own Id: OTP-11590</p> + </item> + <item> + <p> + Deprecation warning for system_flag(cpu_topology) has + been extended for removal in OTP 18 (Thanks to Steve + Vinoski for the update)</p> + <p> + Own Id: OTP-11602</p> + </item> + <item> + <p> + Documentation improvement regarding some awkward wording + around the +spp flag. (Thanks to Brian L. Troutwine )</p> + <p> + Own Id: OTP-11607</p> + </item> + <item> + <p> + Fixed bug where sendfile would return the wrong error + code for a remotely closed socket if the socket was in + passive mode. (Thanks to Vincent Siliakus for reporting + the bug.)</p> + <p> + Own Id: OTP-11614</p> + </item> + <item> + <p> + Increase garbage collection tenure rate</p> + <p>The garbage collector tries to maintain the previous + heap block size during a minor gc, i.e. 'need' is not + utilized in determining the size of the new heap, instead + it relies on tenure and garbage to be sufficiently + large.</p> + <p>In instances during intense growing with exclusively + live data on the heap coupled with delayed tenure, + fullsweeps would be triggered directly after a minor gc + to make room for 'need' since the new heap would be + full.</p> + <p>To remedy this, the tenure of terms on the minor heap + will always happen (if it is below the high watermark) + instead of every other minor gc.</p> + <p>Characteristics Impact: Reduced CPU-time spent in + garbage collection but may infer delays in collecting + garbage from the heap. Tweak 'fullsweep_after' options to + increase gc pressure if needed.</p> + <p> + Own Id: OTP-11617</p> + </item> + <item> + <p> + Fix bug when comparing integers with floats larger than + 2^992. The bug could potentially cause memory corruption + on 32-bit emulators.</p> + <p> + Own Id: OTP-11618</p> + </item> + <item> + <p> + Cross-compilation fixes for TileraMDE-3.0.1.125620</p> + <p> + Own Id: OTP-11635</p> + </item> + <item> + <p> + sendfile no longer uses async threads by default</p> + <p> + This has been done because a slow client attack is + possible if the async thread pool is used. The scenario + is:</p> + <p> + Client does a request for a file and then slowly receives + the file one byte at a time. This will eventually fill + the async thread pool with blocking sendfile operations + and thus starving the vm of all file operations.</p> + <p> + If you still want to use the async threads pool for + sendfile an option to enable it has been introduced.</p> + <p> + Thanks to Christopher Faulet for identifying this + vulnerability.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-11639</p> + </item> + <item> + <p> + Do proper rollback of calls to + <c>enif_open_resource_type</c> when load/upgrade + callbacks of NIF library return failure.</p> + <p> + Own Id: OTP-11722</p> + </item> + <item> + <p> + Changed the default configuration when configuring with + <c>$ERL_TOP/configure</c> to be the same as when + configuring with <c>$ERL_TOP/otp_build configure</c>.</p> + <p> + Previously floating point exceptions got enabled by + default on Linux when HiPE was enabled when configuring + with <c>$ERL_TOP/configure</c>, but not when configuring + with <c>$ERL_TOP/otp_build configure</c>. The default is + now in both cases not to use floating point exceptions + since there still exist unresolved issues with floating + point exceptions on Linux.</p> + <p> + For more information see <seealso + marker="doc/installation_guide:INSTALL"><c>$ERL_TOP/HOWTO/INSTALL.md</c></seealso>.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-11723</p> + </item> + <item> + <p> + A comment in erl_db_tree.c no longer differ from the + code. (Thanks to Cobus Carstens)</p> + <p> + Own Id: OTP-11793</p> + </item> + <item> + <p> + Fix epmd debug functionality for VxWorks (Thanks to Jay + True)</p> + <p> + Own Id: OTP-11808</p> + </item> + <item> + <p> + Use closefrom/2 when available in child_setup (Thanks to + Rick Reed and Anthony Ramine)</p> + <p> + Own Id: OTP-11809</p> + </item> + <item> + <p> + Fix dtrace/systemtap bug where the probe arguments would + be concatenated due to faulty length calculation. </p> + <p> + Thanks to Michal Ptaszek and Scott Lystig Fritchie</p> + <p> + Own Id: OTP-11816</p> + </item> + <item> + <p> + It is now better documented that the <c>+fn*</c> flags to + <c>erl</c> also affect how command line parameters and + environment variables are read. (Thanks to Vlad + Dumitrescu)</p> + <p> + Own Id: OTP-11818</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Options to set match_limit and match_limit_recursion are + added to re:run. The option report_errors is also added + to get more information when re:run fails due to limits + or compilation errors.</p> + <p> + Own Id: OTP-10285</p> + </item> + <item> + <p> Dialyzer's <c>unmatched_return</c> warnings have been + corrected. </p> + <p> + Own Id: OTP-10908</p> + </item> + <item> + <p> + A common case is to wrap an argument to + <c>list_to_binary/1</c> in a list to ensure conversion + can happen even though the argument may already be a + binary. Take special care of this case and do not copy + binary.</p> + <p> + Impact: May cause incompatibility since a single binary + is no longer copied. Use <c>binary:copy/1,2</c> instead.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-11082</p> + </item> + <item> + <p> + Make erlang:open_port/2 spawn and spawn_executable handle + unicode.</p> + <p> + Own Id: OTP-11105</p> + </item> + <item> + <p> + Handle unicode (widestring) in erl, erlc, heart, etc on + windows.</p> + <p> + Own Id: OTP-11135</p> + </item> + <item> + <p> + The version of the PCRE library Used by Erlang's re + module is raised to 8.33 from 7.6. This means, among + other things, better Unicode and Unicode Character + Properties support. New options connected to PCRE 8.33 + are also added to the re module (ucd, notempty_atstart, + no_start_optimize). PCRE has extended the regular + expression syntax between 7.6 and 8.33, why this imposes + a potential incompatibility. Only very complicated + regular expressions may be affected, but if you know you + are using obscure features, please test run your regular + expressions and verify that their behavior has not + changed.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-11204</p> + </item> + <item> + <p>Filenames containing UTF-8 encoded characters can now + be handled by erlc.</p> + <p>If you have set the <c>ERLC_EMULATOR</c> environment + variable, note that <c>erlc</c> in OTP 17 will only work + with <c>erl</c> in OTP 17 since the protocol between the + <c>erlc</c> program and the <c>erl_compile</c> module has + changed.</p> + <p> + Own Id: OTP-11248</p> + </item> + <item> + <p> + By giving --enable-static-{nifs,drivers} to configure it + is now possible to statically linking of nifs and drivers + to the main Erlang VM binary. At the moment only the asn1 + and crypto nifs of the Erlang/OTP nifs and drivers have + been prepared to be statically linked. For more details + see the Installation Guide in the System documentation.</p> + <p> + Own Id: OTP-11258</p> + </item> + <item> + <p> + Erlang/OTP has been ported to the realtime operating + system OSE. The port supports both smp and non-smp + emulator. For details around the port and how to started + see the User's Guide in the <seealso + marker="ose:ose_intro">ose</seealso> application. </p> + <p> + Note that not all parts of Erlang/OTP has been ported. </p> + <p> + Notable things that work are: non-smp and smp emulators, + OSE signal interaction, crypto, asn1, run_erl/to_erl, + tcp, epmd, distribution and most if not all non-os + specific functionality of Erlang.</p> + <p> + Notable things that does not work are: udp/sctp, os_mon, + erl_interface, binding of schedulers.</p> + <p> + Own Id: OTP-11334</p> + </item> + <item> + <p> + Add the {active,N} socket option for TCP, UDP, and SCTP, + where N is an integer in the range -32768..32767, to + allow a caller to specify the number of data messages to + be delivered to the controlling process. Once the + socket's delivered message count either reaches 0 or is + explicitly set to 0 with inet:setopts/2 or by including + {active,0} as an option when the socket is created, the + socket transitions to passive ({active, false}) mode and + the socket's controlling process receives a message to + inform it of the transition. TCP sockets receive + {tcp_passive,Socket}, UDP sockets receive + {udp_passive,Socket} and SCTP sockets receive + {sctp_passive,Socket}. </p> + <p> + The socket's delivered message counter defaults to 0, but + it can be set using {active,N} via any gen_tcp, gen_udp, + or gen_sctp function that takes socket options as + arguments, or via inet:setopts/2. New N values are added + to the socket's current counter value, and negative + numbers can be used to reduce the counter value. + Specifying a number that would cause the socket's counter + value to go above 32767 causes an einval error. If a + negative number is specified such that the counter value + would become negative, the socket's counter value is set + to 0 and the socket transitions to passive mode. If the + counter value is already 0 and inet:setopts(Socket, + [{active,0}]) is specified, the counter value remains at + 0 but the appropriate passive mode transition message is + generated for the socket.</p> + <p> + Thanks to Steve Vinoski</p> + <p> + Own Id: OTP-11368</p> + </item> + <item> + <p> + A new optional scheduler utilization balancing mechanism + has been introduced. For more information see the + <seealso marker="erl#+sub"><c>+sub</c></seealso> command + line argument.</p> + <p> + Characteristics impact: None, when not enabled. When + enabled, changed timing in the system, normally a small + overhead due to measuring of utilization and calculating + balancing information. On some systems, such as old + Windows systems, the overhead can be quite substantial. + This time measurement overhead highly depend on the + underlying primitives provided by the OS.</p> + <p> + Own Id: OTP-11385</p> + </item> + <item> + <p> + A call to either the <c>garbage_collect/1</c> BIF or the + <c>check_process_code/2</c> BIF may trigger garbage + collection of another processes than the process calling + the BIF. The previous implementations performed these + kinds of garbage collections without considering the + internal state of the process being garbage collected. In + order to be able to more easily and more efficiently + implement yielding native code, these types of garbage + collections have been rewritten. A garbage collection + like this is now triggered by an asynchronous request + signal, the actual garbage collection is performed by the + process being garbage collected itself, and finalized by + a reply signal to the process issuing the request. Using + this approach processes can disable garbage collection + and yield without having to set up the heap in a state + that can be garbage collected.</p> + <p> + The <seealso + marker="erts:erlang#garbage_collect/2"><c>garbage_collect/2</c></seealso>, + and <seealso + marker="erts:erlang#check_process_code/3"><c>check_process_code/3</c></seealso> + BIFs have been introduced. Both taking an option list as + last argument. Using these, one can issue asynchronous + requests.</p> + <p> + <c>code:purge/1</c> and <c>code:soft_purge/1</c> have + been rewritten to utilize asynchronous + <c>check_process_code</c> requests in order to + parallelize work.</p> + <p> + Characteristics impact: A call to the + <c>garbage_collect/1</c> BIF or the + <c>check_process_code/2</c> BIF will normally take longer + time to complete while the system as a whole wont be as + much negatively effected by the operation as before. A + call to <c>code:purge/1</c> and <c>code:soft_purge/1</c> + may complete faster or slower depending on the state of + the system while the system as a whole wont be as much + negatively effected by the operation as before.</p> + <p> + Own Id: OTP-11388 Aux Id: OTP-11535, OTP-11648 </p> + </item> + <item> + <p> + Cleanup 'Buckets' and 'Time left' fields in crashdump to + ease parsing.</p> + <p> + Own Id: OTP-11419</p> + </item> + <item> + <p> + Add sync option to file:open/2.</p> + <p> + The sync option adds the POSIX O_SYNC flag to the open + system call on platforms that support the flag or its + equivalent, e.g., FILE_FLAG_WRITE_THROUGH on Windows. For + platforms that don't support it, file:open/2 returns + {error, enotsup} if the sync option is passed in. Thank + to Steve Vinoski and Joseph Blomstedt</p> + <p> + Own Id: OTP-11498</p> + </item> + <item> + <p> + erlang:binary_to_term will now cost an appropriate amount + of reductions and will interrupt (yield) for reschedule + if the term is big. This avoids too long schedules when + binary_to_term is used. (Thanks to Svante Karlsson for + the original patch)</p> + <p> + Impact: Programs running binary_to_term on large binaries + will run more smoothly, but rescheduling will impact the + single process performance of the BIF. Single threaded + benchmarks might show degraded performance of the BIF, + while general system behaviour will be improved.</p> + <p> + Own Id: OTP-11535 Aux Id: OTP-11388 </p> + </item> + <item> + <p> + Added high resolution icon for windows. (Thanks to Daniel + Goertz for the inspiration.)</p> + <p> + Own Id: OTP-11560</p> + </item> + <item> + <p> + Migration of memory carriers has been enabled by default + on all ERTS internal memory allocators based on the + <seealso + marker="erts_alloc#alloc_util"><c>alloc_util</c></seealso> + framework except for <c>temp_alloc</c>. That is, <seealso + marker="erts_alloc#M_acul"><c>+M<S>acul + de</c></seealso> is default for these allocators. Note + that this also implies changed allocation strategies for + all of these allocators. They will all now use the + "address order first fit carrier best fit" strategy.</p> + <p> + By passing <c>+Muacul 0</c> on the command line, all + configuration changes made by this change will be + reverted.</p> + <p> + Characteristics impact: Improved memory characteristics + with a smaller memory footprint at the expense of a quite + small performance cost.</p> + <p> + Own Id: OTP-11604 Aux Id: OTP-10279 </p> + </item> + <item> + <p>A clarification has been added to the documentation of + <c>-on_load()</c> in the Reference Manual that it is only + recommended for loading NIF libraries.</p> + <p> + Own Id: OTP-11611</p> + </item> + <item> + <p><c>+fnaw</c> is now default when starting the + emulator; it used to be <c>+fnl</c>.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-11612</p> + </item> + <item> + <p> + EEP43: New data type - Maps</p> + <p> + With Maps you may for instance: <taglist> <item><c>M0 = + #{ a => 1, b => 2}, % create + associations</c></item> <item><c>M1 = M0#{ a := 10 }, % + update values</c></item> <item><c>M2 = M1#{ "hi" => + "hello"}, % add new associations</c></item> <item><c>#{ + "hi" := V1, a := V2, b := V3} = M2. % match keys with + values</c></item> </taglist></p> + <p> + For information on how to use Maps please see Map Expressions in the + <seealso marker="doc/reference_manual:expressions#map_expressions"> + Reference Manual</seealso>.</p> + <p> + The current implementation is without the following + features: <taglist> <item>No variable keys</item> + <item>No single value access</item> <item>No map + comprehensions</item> </taglist></p> + <p> + Note that Maps is <em>experimental</em> during OTP 17.0.</p> + <p> + Own Id: OTP-11616</p> + </item> + <item> + <p> + The previously deprecated driver API function + <c>driver_async_cancel()</c> has been removed. Due to + this, the driver API version has been bumped to 3.0.</p> + <p> + Thanks to Steve Vinoski.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-11628</p> + </item> + <item> + <p> + Experimental "dirty scheduler" functionality has been + introduced. In order to try the functionality out, you + need to pass the command line argument + <c>--enable-dirty-schedulers</c> to <c>configure</c> when + building the system.</p> + <p> + Dirty schedulers can currently only be used by NIFs on a + system with SMP support. More information can be found in + the <seealso + marker="erl_nif#dirty_nifs"><c>erl_nif(3)</c></seealso> + documentation, the <seealso + marker="erl"><c>erl(1)</c></seealso> documentation, and + in the git commit comment of commit + 'c1c03ae4ee50e58b7669ea88ec4d29c6b2b67c7b'.</p> + <p> + Note that the functionality is <em>experimental</em>, and + <em>not supported</em>. This functionality <em>will</em> + be subject to backward incompatible changes. You should + <em>not</em> enable the dirty scheduler functionality on + production systems. It is only provided for testing.</p> + <p> + Thanks to Steve Vinoski.</p> + <p> + Own Id: OTP-11629</p> + </item> + <item> + <p> + Improve reduction cost and yielding of + <c>term_to_binary</c>. The reduction cost is increased + and garbage collection is disabled during yield.</p> + <p> + Impact: Improves system responsiveness when + <c>term_to_binary</c> is called with large terms without + significant degradation of single threaded performance.</p> + <p> + Own Id: OTP-11648 Aux Id: OTP-11388 </p> + </item> + <item> + <p> + By default, the system's version of zlib will be used, + provided its version is 1.2.4 or higher; otherwise the + built-in zlib will be used. The built-in version of zlib + has been bumped to 1.2.8. (Use the + <c>--enable-builtin-zlib</c> option to <c>configure</c> + to force the use of the built-in zlib.)</p> + <p> + Own Id: OTP-11669</p> + </item> + <item> + <p> + The default float encoding in binary_to_term and + external_size has been changed to use minor_mode 1 + instead of 0.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-11738</p> + </item> + <item> + <p> + Introduced the <c>configure</c> option + <c>--with-assumed-cache-line-size=SIZE</c>. For more + information see <seealso + marker="doc/installation_guide:INSTALL"><c>$ERL_TOP/HOWTO/INSTALL.md</c></seealso>.</p> + <p> + Own Id: OTP-11742</p> + </item> + <item> + <p> + Halfword emulator is marked as deprecated. It still works + as before but is planned to be removed in a future major + release.</p> + <p> + Own Id: OTP-11777</p> + </item> + <item> + <p> + The external format for Maps has changed in a way that is + not compatible with the format used in OTP 17.0-rc1 and + OTP 17.0-rc2.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-11782</p> + </item> + <item> + <p> + Fixed faulty make dependency that would make some make + versions fail while building gen_git_version.mk.</p> + <p> + Own Id: OTP-11784</p> + </item> + <item> + <p> + Introduced functionality for allowing old drivers and NIF + libraries to be loaded during a transition period. For + more information see <seealso + marker="erts:erl_driver#version_management">the version + management section in the <c>erl_driver(3)</c> + documentation</seealso> and <seealso + marker="erts:erl_nif#version_management">the version + management section in the <c>erl_nif(3)</c> + documentation</seealso>.</p> + <p> + Own Id: OTP-11799</p> + </item> + <item> + <p> + Support file paths longer than 259 characters on Windows. + Long absolute paths are automatically converted to UNC + format with a <c>\\?\</c> prefix which is the only way to + represent long paths. The 259 character limit still + applies for individual file names, relative paths and the + current working directory.</p> + <p> + Own Id: OTP-11813</p> + </item> + <item> + <p> + Document that escript:create/2 also accepts a 3-elements + tuple containing files and zip:create/3 options to build + a zip file.</p> + <p> + Thanks to Pierre Fenoll</p> + <p> + Own Id: OTP-11827</p> + </item> + <item> + <p> + Add systemd socket activation for epmd.</p> + <p> + Thanks to Matwey V. Kornilov</p> + <p> + Own Id: OTP-11829</p> + </item> + </list> + </section> + +</section> + +<section><title>Erts 5.10.4.1</title> + + <section><title>Known Bugs and Problems</title> + <list> + <item> + <p> + When using gen_tcp:connect and the <c>fd</c> option with + <c>port</c> and/or <c>ip</c>, the <c>port</c> and + <c>ip</c> options were ignored. This has been fixed so + that if <c>port</c> and/or <c>ip</c> is specified + together with <c>fd</c> a bind is requested for that + <c>fd</c>. If <c>port</c> and/or <c>ip</c> is not + specified bind will not be called.</p> + <p> + Own Id: OTP-12061</p> + </item> + </list> + </section> + +</section> + <section><title>Erts 5.10.4</title> <section><title>Fixed Bugs and Malfunctions</title> @@ -4122,7 +5229,7 @@ <item> <p> The <c>configure</c> command line argument <seealso - marker="doc/installation_guide:INSTALL#How-to-Build-and-Install-ErlangOTP_A-Closer-Look-at-the-individual-Steps_Configuring">--enable-ethread-pre-pentium4-compatibility</seealso> + marker="doc/installation_guide:INSTALL#Advanced-configuration-and-build-of-ErlangOTP">--enable-ethread-pre-pentium4-compatibility</seealso> had no effect. This option is now also automatically enabled if required on the build machine.</p> <p> @@ -4701,7 +5808,7 @@ platforms than before. If <c>configure</c> warns about no atomic implementation available, try using the <c>libatomic_ops</c> library. Use the <seealso - marker="doc/installation_guide:INSTALL#How-to-Build-and-Install-ErlangOTP_A-Closer-Look-at-the-individual-Steps_Configuring">--with-libatomic_ops=PATH</seealso> + marker="doc/installation_guide:INSTALL#Advanced-configuration-and-build-of-ErlangOTP">--with-libatomic_ops=PATH</seealso> <c>configure</c> command line argument when specifying where the <c>libatomic_ops</c> installation is located. The <c>libatomic_ops</c> library can be downloaded from: @@ -4719,7 +5826,7 @@ the pentium 4 processor. If you want the runtime system to be compatible with older processors (back to 486) you need to pass the <seealso - marker="doc/installation_guide:INSTALL#How-to-Build-and-Install-ErlangOTP_A-Closer-Look-at-the-individual-Steps_Configuring">--enable-ethread-pre-pentium4-compatibility</seealso> + marker="doc/installation_guide:INSTALL#Advanced-configuration-and-build-of-ErlangOTP">--enable-ethread-pre-pentium4-compatibility</seealso> <c>configure</c> command line argument when configuring the system.</p> <p> diff --git a/erts/emulator/Makefile.in b/erts/emulator/Makefile.in index 58e77ed1fa..7145824f91 100644 --- a/erts/emulator/Makefile.in +++ b/erts/emulator/Makefile.in @@ -1035,11 +1035,10 @@ else ifeq ($(findstring ose,$(TARGET)),ose) $(BINDIR)/$(EMULATOR_EXECUTABLE): $(INIT_OBJS) $(OBJS) $(DEPLIBS) $(LCF) $(call build-ose-load-module, $@, $(INIT_OBJS) $(OBJS), $(STATIC_NIF_LIBS) \ - $(STATIC_DRIVER_LIBS) $(LIBS), $(LMCONF)) + $(STATIC_DRIVER_LIBS) $(LIBS), $(BEAM_LMCONF)) else $(BINDIR)/$(EMULATOR_EXECUTABLE): $(INIT_OBJS) $(OBJS) $(DEPLIBS) - echo $(DEPLIBS) $(ld_verbose)$(PURIFY) $(LD) -o $(BINDIR)/$(EMULATOR_EXECUTABLE) \ $(HIPEBEAMLDFLAGS) $(LDFLAGS) $(DEXPORT) $(INIT_OBJS) $(OBJS) \ $(STATIC_NIF_LIBS) $(STATIC_DRIVER_LIBS) $(LIBS) diff --git a/erts/emulator/beam/atom.names b/erts/emulator/beam/atom.names index d28e519ae1..5d06a32941 100644 --- a/erts/emulator/beam/atom.names +++ b/erts/emulator/beam/atom.names @@ -116,6 +116,7 @@ atom binary_longest_prefix_trap atom binary_longest_suffix_trap atom binary_match_trap atom binary_matches_trap +atom binary_to_list_continue atom binary_to_term_trap atom block atom blocked @@ -315,6 +316,7 @@ atom line_length atom linked_in_driver atom links atom list +atom list_to_binary_continue atom little atom loaded atom load_cancelled diff --git a/erts/emulator/beam/beam_emu.c b/erts/emulator/beam/beam_emu.c index 9634faff1d..8bfb7d2ad2 100644 --- a/erts/emulator/beam/beam_emu.c +++ b/erts/emulator/beam/beam_emu.c @@ -3503,6 +3503,7 @@ get_map_elements_fail: * I[0]: &&call_nif * I[1]: Function pointer to NIF function * I[2]: Pointer to erl_module_nif + * I[3]: Function pointer to dirty NIF */ BifFunction vbf; @@ -3523,13 +3524,6 @@ get_map_elements_fail: reg[0] = r(0); nif_bif_result = (*fp)(&env, bif_nif_arity, reg); erts_post_nif(&env); -#ifdef ERTS_DIRTY_SCHEDULERS - if (is_non_value(nif_bif_result) && c_p->freason == TRAP) { - Export* ep = (Export*) c_p->psd->data[ERTS_PSD_DIRTY_SCHED_TRAP_EXPORT]; - ep->code[0] = I[-3]; - ep->code[1] = I[-2]; - } -#endif } ASSERT(!ERTS_PROC_IS_EXITING(c_p) || is_non_value(nif_bif_result)); PROCESS_MAIN_CHK_LOCKS(c_p); diff --git a/erts/emulator/beam/beam_load.c b/erts/emulator/beam/beam_load.c index e96177cfd9..cfc6146b0a 100644 --- a/erts/emulator/beam/beam_load.c +++ b/erts/emulator/beam/beam_load.c @@ -2363,7 +2363,11 @@ load_code(LoaderState* stp) if (stp->may_load_nif) { const int finfo_ix = ci - FUNC_INFO_SZ; - enum { MIN_FUNC_SZ = 3 }; +#ifdef ERTS_DIRTY_SCHEDULERS + enum { MIN_FUNC_SZ = 4 }; +#else + enum { MIN_FUNC_SZ = 3 }; +#endif if (finfo_ix - last_func_start < MIN_FUNC_SZ && last_func_start) { /* Must make room for call_nif op */ int pad = MIN_FUNC_SZ - (finfo_ix - last_func_start); diff --git a/erts/emulator/beam/bif.c b/erts/emulator/beam/bif.c index 06a1230ca0..a5be8e1529 100644 --- a/erts/emulator/beam/bif.c +++ b/erts/emulator/beam/bif.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1996-2013. All Rights Reserved. + * Copyright Ericsson AB 1996-2014. All Rights Reserved. * * The contents of this file are subject to the Erlang Public License, * Version 1.1, (the "License"); you may not use this file except in @@ -1869,6 +1869,7 @@ do_send(Process *p, Eterm to, Eterm msg, int suspend, Eterm *refp) { } else if (is_external_pid(to)) { dep = external_pid_dist_entry(to); if(dep == erts_this_dist_entry) { +#if DEBUG erts_dsprintf_buf_t *dsbufp = erts_create_logger_dsbuf(); erts_dsprintf(dsbufp, "Discarding message %T from %T to %T in an old " @@ -1879,6 +1880,7 @@ do_send(Process *p, Eterm to, Eterm msg, int suspend, Eterm *refp) { external_pid_creation(to), erts_this_node->creation); erts_send_error_to_logger(p->group_leader, dsbufp); +#endif return 0; } return remote_send(p, dep, to, to, msg, suspend); @@ -1886,8 +1888,13 @@ do_send(Process *p, Eterm to, Eterm msg, int suspend, Eterm *refp) { Eterm id = erts_whereis_name_to_id(p, to); rp = erts_proc_lookup(id); - if (rp) + if (rp) { + if (IS_TRACED(p)) + trace_send(p, to, msg); + if (ERTS_PROC_GET_SAVED_CALLS_BUF(p)) + save_calls(p, &exp_send); goto send_message; + } pt = erts_port_lookup(id, (erts_port_synchronous_ops @@ -1907,6 +1914,7 @@ do_send(Process *p, Eterm to, Eterm msg, int suspend, Eterm *refp) { } else if (is_external_port(to) && (external_port_dist_entry(to) == erts_this_dist_entry)) { +#if DEBUG erts_dsprintf_buf_t *dsbufp = erts_create_logger_dsbuf(); erts_dsprintf(dsbufp, "Discarding message %T from %T to %T in an old " @@ -1917,6 +1925,7 @@ do_send(Process *p, Eterm to, Eterm msg, int suspend, Eterm *refp) { external_port_creation(to), erts_this_node->creation); erts_send_error_to_logger(p->group_leader, dsbufp); +#endif return 0; } else if (is_internal_port(to)) { int ret_val; @@ -2882,9 +2891,6 @@ static int do_list_to_integer(Process *p, Eterm orig_list, res = big_plus_small(res, m, hp); } - if (is_big(res)) /* check if small */ - res = big_plus_small(res, 0, hp); /* includes conversion to small */ - if (neg) { if (is_small(res)) res = make_small(-signed_val(res)); @@ -2894,8 +2900,12 @@ static int do_list_to_integer(Process *p, Eterm orig_list, } } - if (is_big(res)) { - hp += (big_arity(res)+1); + if (is_not_small(res)) { + res = big_plus_small(res, 0, hp); /* includes conversion to small */ + + if (is_not_small(res)) { + hp += (big_arity(res)+1); + } } HRelease(p,hp_end,hp); } diff --git a/erts/emulator/beam/bif.h b/erts/emulator/beam/bif.h index 51b77a95ed..72c55ccb55 100644 --- a/erts/emulator/beam/bif.h +++ b/erts/emulator/beam/bif.h @@ -124,12 +124,85 @@ do { \ return THE_NON_VALUE; \ } while(0) +#define ERTS_BIF_ERROR_TRAPPED0(Proc, Reason, Bif) \ +do { \ + (Proc)->freason = (Reason); \ + (Proc)->current = (Bif)->code; \ + return THE_NON_VALUE; \ +} while (0) + +#define ERTS_BIF_ERROR_TRAPPED1(Proc, Reason, Bif, A0) \ +do { \ + Eterm* reg = ERTS_PROC_GET_SCHDATA((Proc))->x_reg_array; \ + (Proc)->freason = (Reason); \ + (Proc)->current = (Bif)->code; \ + reg[0] = (Eterm) (A0); \ + return THE_NON_VALUE; \ +} while (0) + +#define ERTS_BIF_ERROR_TRAPPED2(Proc, Reason, Bif, A0, A1) \ +do { \ + Eterm* reg = ERTS_PROC_GET_SCHDATA((Proc))->x_reg_array; \ + (Proc)->freason = (Reason); \ + (Proc)->current = (Bif)->code; \ + reg[0] = (Eterm) (A0); \ + reg[1] = (Eterm) (A1); \ + return THE_NON_VALUE; \ +} while (0) + +#define ERTS_BIF_ERROR_TRAPPED3(Proc, Reason, Bif, A0, A1, A2) \ +do { \ + Eterm* reg = ERTS_PROC_GET_SCHDATA((Proc))->x_reg_array; \ + (Proc)->freason = (Reason); \ + (Proc)->current = (Bif)->code; \ + reg[0] = (Eterm) (A0); \ + reg[1] = (Eterm) (A1); \ + reg[2] = (Eterm) (A2); \ + return THE_NON_VALUE; \ +} while (0) + #define ERTS_BIF_PREP_ERROR(Ret, Proc, Reason) \ do { \ (Proc)->freason = (Reason); \ (Ret) = THE_NON_VALUE; \ } while (0) +#define ERTS_BIF_PREP_ERROR_TRAPPED0(Ret, Proc, Reason, Bif) \ +do { \ + (Proc)->freason = (Reason); \ + (Proc)->current = (Bif)->code; \ + (Ret) = THE_NON_VALUE; \ +} while (0) + +#define ERTS_BIF_PREP_ERROR_TRAPPED1(Ret, Proc, Reason, Bif, A0) \ +do { \ + Eterm* reg = ERTS_PROC_GET_SCHDATA((Proc))->x_reg_array; \ + (Proc)->freason = (Reason); \ + (Proc)->current = (Bif)->code; \ + reg[0] = (Eterm) (A0); \ + (Ret) = THE_NON_VALUE; \ +} while (0) + +#define ERTS_BIF_PREP_ERROR_TRAPPED2(Ret, Proc, Reason, Bif, A0, A1) \ +do { \ + Eterm* reg = ERTS_PROC_GET_SCHDATA((Proc))->x_reg_array; \ + (Proc)->freason = (Reason); \ + (Proc)->current = (Bif)->code; \ + reg[0] = (Eterm) (A0); \ + reg[1] = (Eterm) (A1); \ + (Ret) = THE_NON_VALUE; \ +} while (0) + +#define ERTS_BIF_PREP_ERROR_TRAPPED3(Ret, Proc, Reason, Bif, A0, A1, A2) \ +do { \ + Eterm* reg = ERTS_PROC_GET_SCHDATA((Proc))->x_reg_array; \ + (Proc)->freason = (Reason); \ + (Proc)->current = (Bif)->code; \ + reg[0] = (Eterm) (A0); \ + reg[1] = (Eterm) (A1); \ + reg[2] = (Eterm) (A2); \ + (Ret) = THE_NON_VALUE; \ +} while (0) #define ERTS_BIF_PREP_TRAP0(Ret, Trap, Proc) \ do { \ @@ -392,6 +465,51 @@ erts_bif_prep_await_proc_exit_apply_trap(Process *c_p, Eterm args[], int nargs); +#ifndef HIPE + +#define HIPE_WRAPPER_BIF_DISABLE_GC(BIF_NAME, ARITY) + +#else + +#include "erl_fun.h" +#include "hipe_mode_switch.h" + +/* + * Hipe wrappers used by native code for BIFs that disable GC while trapping. + * Also add usage of the wrapper in ../hipe/hipe_bif_list.m4 + * + * Problem: + * When native code calls a BIF that traps, hipe_mode_switch will push a + * "trap frame" on the Erlang stack in order to find its way back from beam_emu + * back to native caller when finally done. If GC is disabled and stack/heap + * is full there is no place to push the "trap frame". + * + * Solution: + * We reserve space on stack for the "trap frame" here before the BIF is called. + * If the BIF does not trap, the space is reclaimed here before returning. + * If the BIF traps, hipe_push_beam_trap_frame() will detect that a "trap frame" + * already is reserved and use it. + */ + + +#define HIPE_WRAPPER_BIF_DISABLE_GC(BIF_NAME, ARITY) \ +BIF_RETTYPE hipe_wrapper_ ## BIF_NAME ## _ ## ARITY (Process* c_p, \ + Eterm* args); \ +BIF_RETTYPE hipe_wrapper_ ## BIF_NAME ## _ ## ARITY (Process* c_p, \ + Eterm* args) \ +{ \ + BIF_RETTYPE res; \ + hipe_reserve_beam_trap_frame(c_p, args, ARITY); \ + res = BIF_NAME ## _ ## ARITY (c_p, args); \ + if (is_value(res) || c_p->freason != TRAP) { \ + hipe_unreserve_beam_trap_frame(c_p); \ + } \ + return res; \ +} + +#endif + + #include "erl_bif_table.h" #endif diff --git a/erts/emulator/beam/bif.tab b/erts/emulator/beam/bif.tab index 2d888862bf..e68b8e6274 100644 --- a/erts/emulator/beam/bif.tab +++ b/erts/emulator/beam/bif.tab @@ -45,6 +45,7 @@ bif erlang:apply/3 bif erlang:atom_to_list/1 bif erlang:binary_to_list/1 bif erlang:binary_to_list/3 +bif erlang:binary_to_term/1 bif erlang:crc32/1 bif erlang:crc32/2 bif erlang:crc32_combine/3 @@ -151,12 +152,11 @@ bif erts_internal:port_command/3 bif erts_internal:port_control/3 bif erts_internal:port_close/1 bif erts_internal:port_connect/2 -bif erts_internal:binary_to_term/1 -bif erts_internal:binary_to_term/2 bif erts_internal:request_system_task/3 bif erts_internal:check_process_code/2 +bif erts_internal:map_to_tuple_keys/1 # inet_db support bif erlang:port_set_data/2 @@ -480,6 +480,11 @@ bif erlang:call_on_load_function/1 bif erlang:finish_after_on_load/2 # +# New Bifs in R13B04 +# +bif erlang:binary_to_term/2 + +# # The binary match bifs (New in R14A - EEP9) # @@ -596,6 +601,10 @@ bif maps:values/1 bif erts_internal:cmp_term/2 # +# New in 17.1. +# +bif erlang:fun_info_mfa/1 +# # Obsolete # diff --git a/erts/emulator/beam/big.c b/erts/emulator/beam/big.c index 41a041eba6..a8710dd910 100644 --- a/erts/emulator/beam/big.c +++ b/erts/emulator/beam/big.c @@ -274,6 +274,9 @@ _b = _b << _s; \ _vn1 = _b >> H_EXP; \ _vn0 = _b & LO_MASK; \ + /* Sometimes _s is 0 which triggers undefined behaviour for the \ + (_a0>>(D_EXP-_s)) shift, but this is ok because the \ + & -s will make it all to 0 later anyways. */ \ _un32 = (_a1 << _s) | ((_a0>>(D_EXP-_s)) & (-_s >> (D_EXP-1))); \ _un10 = _a0 << _s; \ _un1 = _un10 >> H_EXP; \ @@ -1506,13 +1509,15 @@ Eterm uword_to_big(UWord x, Eterm *y) */ Eterm small_to_big(Sint x, Eterm *y) { + Uint xu; if (x >= 0) { + xu = x; *y = make_pos_bignum_header(1); } else { - x = -x; + xu = -(Uint)x; *y = make_neg_bignum_header(1); } - BIG_DIGIT(y, 0) = x; + BIG_DIGIT(y, 0) = xu; return make_big(y); } @@ -1540,21 +1545,24 @@ Eterm erts_uint64_to_big(Uint64 x, Eterm **hpp) Eterm erts_sint64_to_big(Sint64 x, Eterm **hpp) { Eterm *hp = *hpp; + Uint64 ux; int neg; - if (x >= 0) + if (x >= 0) { neg = 0; + ux = x; + } else { neg = 1; - x = -x; + ux = -(Uint64)x; } #if defined(ARCH_32) || HALFWORD_HEAP - if (x >= (((Uint64) 1) << 32)) { + if (ux >= (((Uint64) 1) << 32)) { if (neg) *hp = make_neg_bignum_header(2); else *hp = make_pos_bignum_header(2); - BIG_DIGIT(hp, 0) = (Uint) (x & ((Uint) 0xffffffff)); - BIG_DIGIT(hp, 1) = (Uint) ((x >> 32) & ((Uint) 0xffffffff)); + BIG_DIGIT(hp, 0) = (Uint) (ux & ((Uint) 0xffffffff)); + BIG_DIGIT(hp, 1) = (Uint) ((ux >> 32) & ((Uint) 0xffffffff)); *hpp += 3; } else @@ -1564,7 +1572,7 @@ Eterm erts_sint64_to_big(Sint64 x, Eterm **hpp) *hp = make_neg_bignum_header(1); else *hp = make_pos_bignum_header(1); - BIG_DIGIT(hp, 0) = (Uint) x; + BIG_DIGIT(hp, 0) = (Uint) ux; *hpp += 2; } return make_big(hp); @@ -2667,9 +2675,6 @@ Eterm erts_chars_to_integer(Process *BIF_P, char *bytes, res = big_plus_small(res, m, hp); } - if (is_big(res)) /* check if small */ - res = big_plus_small(res, 0, hp); /* includes conversion to small */ - if (neg) { if (is_small(res)) res = make_small(-signed_val(res)); @@ -2679,8 +2684,12 @@ Eterm erts_chars_to_integer(Process *BIF_P, char *bytes, } } - if (is_big(res)) { - hp += (big_arity(res) + 1); + if (is_not_small(res)) { + res = big_plus_small(res, 0, hp); /* includes conversion to small */ + + if (is_not_small(res)) { + hp += (big_arity(res) + 1); + } } HRelease(BIF_P, hp_end, hp); goto bytebuf_to_integer_1_done; diff --git a/erts/emulator/beam/big.h b/erts/emulator/beam/big.h index d80111822e..da31876d75 100644 --- a/erts/emulator/beam/big.h +++ b/erts/emulator/beam/big.h @@ -101,7 +101,7 @@ typedef Uint dsize_t; /* Vector size type */ #define ERTS_SINT64_HEAP_SIZE(X) \ (IS_SSMALL((X)) \ ? 0 \ - : ERTS_UINT64_BIG_HEAP_SIZE__((X) >= 0 ? (X) : -(X))) + : ERTS_UINT64_BIG_HEAP_SIZE__((X) >= 0 ? (X) : -(Uint64)(X))) #define ERTS_UINT64_HEAP_SIZE(X) \ (IS_USMALL(0, (X)) ? 0 : ERTS_UINT64_BIG_HEAP_SIZE__((X))) diff --git a/erts/emulator/beam/binary.c b/erts/emulator/beam/binary.c index c7926f18af..f50d484576 100644 --- a/erts/emulator/beam/binary.c +++ b/erts/emulator/beam/binary.c @@ -31,12 +31,11 @@ #include "erl_binary.h" #include "erl_bits.h" -#ifdef DEBUG -static int list_to_bitstr_buf(Eterm obj, char* buf, Uint len); -#else -static int list_to_bitstr_buf(Eterm obj, char* buf); -#endif -static int bitstr_list_len(Eterm obj, Uint* num_bytes); +static Export binary_to_list_continue_export; +static Export list_to_binary_continue_export; + +static BIF_RETTYPE binary_to_list_continue(BIF_ALIST_1); +static BIF_RETTYPE list_to_binary_continue(BIF_ALIST_1); void erts_init_binary(void) @@ -49,6 +48,15 @@ erts_init_binary(void) "Internal error: Address of orig_bytes[0] of a Binary" " is *not* 8-byte aligned\n"); } + + erts_init_trap_export(&binary_to_list_continue_export, + am_erts_internal, am_binary_to_list_continue, 1, + &binary_to_list_continue); + + erts_init_trap_export(&list_to_binary_continue_export, + am_erts_internal, am_list_to_binary_continue, 1, + &list_to_binary_continue); + } /* @@ -333,6 +341,132 @@ BIF_RETTYPE integer_to_binary_1(BIF_ALIST_1) BIF_RET(res); } +#define ERTS_B2L_BYTES_PER_REDUCTION 256 + +typedef struct { + Eterm res; + Eterm *hp; +#ifdef DEBUG + Eterm *hp_end; +#endif + byte *bytes; + Uint size; + Uint bitoffs; +} ErtsB2LState; + +static void b2l_state_destructor(Binary *mbp) +{ + ASSERT(ERTS_MAGIC_BIN_DESTRUCTOR(mbp) == b2l_state_destructor); +} + +static BIF_RETTYPE +binary_to_list_chunk(Process *c_p, + Eterm mb_eterm, + ErtsB2LState* sp, + int reds_left, + int gc_disabled) +{ + BIF_RETTYPE ret; + int bump_reds; + Uint size; + byte *bytes; + + size = (reds_left + 1)*ERTS_B2L_BYTES_PER_REDUCTION; + if (size > sp->size) + size = sp->size; + bytes = sp->bytes + (sp->size - size); + + bump_reds = (size - 1)/ERTS_B2L_BYTES_PER_REDUCTION + 1; + BUMP_REDS(c_p, bump_reds); + + ASSERT(is_list(sp->res) || is_nil(sp->res)); + + sp->res = erts_bin_bytes_to_list(sp->res, + sp->hp, + bytes, + size, + sp->bitoffs); + sp->size -= size; + sp->hp += 2*size; + + if (sp->size > 0) { + + if (!gc_disabled) + erts_set_gc_state(c_p, 0); + + ASSERT(c_p->flags & F_DISABLE_GC); + ASSERT(is_value(mb_eterm)); + ERTS_BIF_PREP_TRAP1(ret, + &binary_to_list_continue_export, + c_p, + mb_eterm); + } + else { + + ASSERT(sp->hp == sp->hp_end); + ASSERT(sp->size == 0); + + if (!gc_disabled || !erts_set_gc_state(c_p, 1)) + ERTS_BIF_PREP_RET(ret, sp->res); + else + ERTS_BIF_PREP_YIELD_RETURN(ret, c_p, sp->res); + ASSERT(!(c_p->flags & F_DISABLE_GC)); + } + + return ret; +} + +static ERTS_INLINE BIF_RETTYPE +binary_to_list(Process *c_p, Eterm *hp, Eterm tail, byte *bytes, Uint size, Uint bitoffs) +{ + int reds_left = ERTS_BIF_REDS_LEFT(c_p); + if (size < reds_left*ERTS_B2L_BYTES_PER_REDUCTION) { + Eterm res; + BIF_RETTYPE ret; + int bump_reds = (size - 1)/ERTS_B2L_BYTES_PER_REDUCTION + 1; + BUMP_REDS(c_p, bump_reds); + res = erts_bin_bytes_to_list(tail, hp, bytes, size, bitoffs); + ERTS_BIF_PREP_RET(ret, res); + return ret; + } + else { + Binary *mbp = erts_create_magic_binary(sizeof(ErtsB2LState), + b2l_state_destructor); + ErtsB2LState *sp = ERTS_MAGIC_BIN_DATA(mbp); + Eterm mb; + + sp->res = tail; + sp->hp = hp; +#ifdef DEBUG + sp->hp_end = sp->hp + 2*size; +#endif + sp->bytes = bytes; + sp->size = size; + sp->bitoffs = bitoffs; + + hp = HAlloc(c_p, PROC_BIN_SIZE); + mb = erts_mk_magic_binary_term(&hp, &MSO(c_p), mbp); + return binary_to_list_chunk(c_p, mb, sp, reds_left, 0); + } +} + +static BIF_RETTYPE binary_to_list_continue(BIF_ALIST_1) +{ + Binary *mbp = ((ProcBin *) binary_val(BIF_ARG_1))->val; + + ASSERT(ERTS_MAGIC_BIN_DESTRUCTOR(mbp) == b2l_state_destructor); + + ASSERT(BIF_P->flags & F_DISABLE_GC); + + return binary_to_list_chunk(BIF_P, + BIF_ARG_1, + (ErtsB2LState*) ERTS_MAGIC_BIN_DATA(mbp), + ERTS_BIF_REDS_LEFT(BIF_P), + 1); +} + +HIPE_WRAPPER_BIF_DISABLE_GC(binary_to_list, 1) + BIF_RETTYPE binary_to_list_1(BIF_ALIST_1) { Eterm real_bin; @@ -354,14 +488,15 @@ BIF_RETTYPE binary_to_list_1(BIF_ALIST_1) } else { Eterm* hp = HAlloc(BIF_P, 2 * size); byte* bytes = binary_bytes(real_bin)+offset; - - BIF_RET(erts_bin_bytes_to_list(NIL, hp, bytes, size, bitoffs)); + return binary_to_list(BIF_P, hp, NIL, bytes, size, bitoffs); } error: BIF_ERROR(BIF_P, BADARG); } +HIPE_WRAPPER_BIF_DISABLE_GC(binary_to_list, 3) + BIF_RETTYPE binary_to_list_3(BIF_ALIST_3) { byte* bytes; @@ -387,12 +522,13 @@ BIF_RETTYPE binary_to_list_3(BIF_ALIST_3) } i = stop-start+1; hp = HAlloc(BIF_P, 2*i); - BIF_RET(erts_bin_bytes_to_list(NIL, hp, bytes+start-1, i, bitoffs)); - + return binary_to_list(BIF_P, hp, NIL, bytes+start-1, i, bitoffs); error: BIF_ERROR(BIF_P, BADARG); } +HIPE_WRAPPER_BIF_DISABLE_GC(bitstring_to_list, 1) + BIF_RETTYPE bitstring_to_list_1(BIF_ALIST_1) { Eterm real_bin; @@ -431,124 +567,441 @@ BIF_RETTYPE bitstring_to_list_1(BIF_ALIST_1) previous = CONS(hp, make_binary(last), previous); hp += 2; } - BIF_RET(erts_bin_bytes_to_list(previous, hp, bytes, size, bitoffs)); + + return binary_to_list(BIF_P, hp, previous, bytes, size, bitoffs); } /* Turn a possibly deep list of ints (and binaries) into */ /* One large binary object */ -/* - * This bif also exists in the binary module, under the name - * binary:list_to_bin/1, why it's divided into interface and - * implementation. Also the backend for iolist_to_binary_1. - */ +typedef enum { + ERTS_L2B_OK, + ERTS_L2B_YIELD, + ERTS_L2B_TYPE_ERROR, + ERTS_L2B_OVERFLOW_ERROR +} ErtsL2BResult; -BIF_RETTYPE erts_list_to_binary_bif(Process *p, Eterm arg) -{ +#define ERTS_L2B_STATE_INITER(C_P, ARG, BIF, SZFunc, TBufFunc) \ + {ERTS_IOLIST2BUF_STATE_INITER((C_P), (ARG)), \ + (ARG), THE_NON_VALUE, (BIF), (SZFunc), (TBufFunc)} + +#define ERTS_L2B_STATE_MOVE(TO, FROM) \ + sys_memcpy((void *) (TO), (void *) (FROM), sizeof(ErtsL2BState)) + +typedef struct ErtsL2BState_ ErtsL2BState; + +struct ErtsL2BState_ { + ErtsIOList2BufState buf; + Eterm arg; Eterm bin; - Eterm h,t; - ErlDrvSizeT size; - byte* bytes; -#ifdef DEBUG - ErlDrvSizeT offset; -#endif + Export *bif; + int (*iolist_to_buf_size)(ErtsIOListState *); + ErlDrvSizeT (*iolist_to_buf)(ErtsIOList2BufState *); +}; + +static ERTS_INLINE ErtsL2BResult +list_to_binary_engine(ErtsL2BState *sp) +{ + ErlDrvSizeT res; + Process *c_p = sp->buf.iolist.c_p; + + /* + * have_size == 0 while sp->iolist_to_buf_size() + * has not finished the calculation. + */ + + if (!sp->buf.iolist.have_size) { + switch (sp->iolist_to_buf_size(&sp->buf.iolist)) { + case ERTS_IOLIST_YIELD: + return ERTS_L2B_YIELD; + case ERTS_IOLIST_OVERFLOW: + return ERTS_L2B_OVERFLOW_ERROR; + case ERTS_IOLIST_TYPE: + return ERTS_L2B_TYPE_ERROR; + case ERTS_IOLIST_OK: + break; + default: + ASSERT(0); + break; + } + + ASSERT(sp->buf.iolist.have_size); + + /* + * Size calculated... Setup state for + * sp->iolist_to_buf_*() + */ + + sp->bin = new_binary(c_p, + (byte *) NULL, + sp->buf.iolist.size); + + if (sp->buf.iolist.size == 0) + return ERTS_L2B_OK; + + sp->buf.buf = (char *) binary_bytes(sp->bin); + sp->buf.len = sp->buf.iolist.size; + sp->buf.iolist.obj = sp->arg; - if (is_nil(arg)) { - BIF_RET(new_binary(p,(byte*)"",0)); + if (sp->buf.iolist.reds_left <= 0) { + BUMP_ALL_REDS(c_p); + return ERTS_L2B_YIELD; + } } - if (is_not_list(arg)) { - goto error; + + ASSERT(sp->buf.iolist.size != 0); + ASSERT(is_value(sp->bin)); + ASSERT(sp->buf.buf); + + res = sp->iolist_to_buf(&sp->buf); + + if (!ERTS_IOLIST_TO_BUF_FAILED(res)) { + ASSERT(res == 0); + return ERTS_L2B_OK; } - /* check for [binary()] case */ - h = CAR(list_val(arg)); - t = CDR(list_val(arg)); - if (is_binary(h) && is_nil(t) && !( - HEADER_SUB_BIN == *(binary_val(h)) && ( - ((ErlSubBin *)binary_val(h))->bitoffs != 0 || - ((ErlSubBin *)binary_val(h))->bitsize != 0 - ))) { - return h; - } - switch (erts_iolist_size(arg, &size)) { - case ERTS_IOLIST_OVERFLOW: BIF_ERROR(p, SYSTEM_LIMIT); - case ERTS_IOLIST_TYPE: goto error; - default: ; - } - bin = new_binary(p, (byte *)NULL, size); - bytes = binary_bytes(bin); -#ifdef DEBUG - offset = -#endif - erts_iolist_to_buf(arg, (char*) bytes, size); - ASSERT(offset == 0); - BIF_RET(bin); + switch (res) { + case ERTS_IOLIST_TO_BUF_YIELD: + return ERTS_L2B_YIELD; + case ERTS_IOLIST_TO_BUF_OVERFLOW: + return ERTS_L2B_OVERFLOW_ERROR; + case ERTS_IOLIST_TO_BUF_TYPE_ERROR: + return ERTS_L2B_TYPE_ERROR; + default: + ERTS_INTERNAL_ERROR("Invalid return value from iolist_to_buf_yielding()"); + return ERTS_L2B_TYPE_ERROR; + } +} + +static void +l2b_state_destructor(Binary *mbp) +{ + ErtsL2BState *sp = ERTS_MAGIC_BIN_DATA(mbp); + ASSERT(ERTS_MAGIC_BIN_DESTRUCTOR(mbp) == l2b_state_destructor); + DESTROY_SAVED_ESTACK(&sp->buf.iolist.estack); +} + +static ERTS_INLINE Eterm +l2b_final_touch(Process *c_p, ErtsL2BState *sp) +{ + Eterm *hp; + ErlSubBin* sbin; + if (sp->buf.offset == 0) + return sp->bin; + + hp = HAlloc(c_p, ERL_SUB_BIN_SIZE); + ASSERT(sp->buf.offset > 0); + sbin = (ErlSubBin *) hp; + sbin->thing_word = HEADER_SUB_BIN; + sbin->size = sp->buf.iolist.size-1; + sbin->offs = 0; + sbin->orig = sp->bin; + sbin->bitoffs = 0; + sbin->bitsize = sp->buf.offset; + sbin->is_writable = 0; + return make_binary(sbin); +} + +static BIF_RETTYPE +list_to_binary_chunk(Eterm mb_eterm, + ErtsL2BState* sp, + int reds_left, + int gc_disabled) +{ + Eterm err = BADARG; + BIF_RETTYPE ret; + Process *c_p = sp->buf.iolist.c_p; + + sp->buf.iolist.reds_left = reds_left; - error: - BIF_ERROR(p, BADARG); + switch (list_to_binary_engine(sp)) { + + case ERTS_L2B_OK: { + Eterm result = l2b_final_touch(c_p, sp); + if (!gc_disabled || !erts_set_gc_state(c_p, 1)) + ERTS_BIF_PREP_RET(ret, result); + else + ERTS_BIF_PREP_YIELD_RETURN(ret, c_p, result); + ASSERT(!(c_p->flags & F_DISABLE_GC)); + break; + } + case ERTS_L2B_YIELD: + if (!gc_disabled) { + /* first yield... */ + Eterm *hp; + Binary *mbp = erts_create_magic_binary(sizeof(ErtsL2BState), + l2b_state_destructor); + ErtsL2BState *new_sp = ERTS_MAGIC_BIN_DATA(mbp); + + ERTS_L2B_STATE_MOVE(new_sp, sp); + sp = new_sp; + + hp = HAlloc(c_p, PROC_BIN_SIZE); + mb_eterm = erts_mk_magic_binary_term(&hp, &MSO(c_p), mbp); + + ASSERT(is_value(mb_eterm)); + + erts_set_gc_state(c_p, 0); + } + + ASSERT(c_p->flags & F_DISABLE_GC); + + ERTS_BIF_PREP_TRAP1(ret, + &list_to_binary_continue_export, + c_p, + mb_eterm); + break; + + case ERTS_L2B_OVERFLOW_ERROR: + err = SYSTEM_LIMIT; + /* fall through */ + + case ERTS_L2B_TYPE_ERROR: + if (!gc_disabled) + ERTS_BIF_PREP_ERROR(ret, c_p, err); + else { + if (erts_set_gc_state(c_p, 1)) + ERTS_VBUMP_ALL_REDS(c_p); + + ERTS_BIF_PREP_ERROR_TRAPPED1(ret, + c_p, + err, + sp->bif, + sp->arg); + } + + ASSERT(!(c_p->flags & F_DISABLE_GC)); + break; + + default: + ERTS_INTERNAL_ERROR("Invalid return value from list_to_binary_engine()"); + ERTS_BIF_PREP_ERROR(ret,c_p, EXC_INTERNAL_ERROR); + break; + } + return ret; } +static BIF_RETTYPE list_to_binary_continue(BIF_ALIST_1) +{ + Binary *mbp = ((ProcBin *) binary_val(BIF_ARG_1))->val; + ASSERT(ERTS_MAGIC_BIN_DESTRUCTOR(mbp) == l2b_state_destructor); + + ASSERT(BIF_P->flags & F_DISABLE_GC); + + return list_to_binary_chunk(BIF_ARG_1, + ERTS_MAGIC_BIN_DATA(mbp), + ERTS_BIF_REDS_LEFT(BIF_P), + 1); +} + +BIF_RETTYPE erts_list_to_binary_bif(Process *c_p, Eterm arg, Export *bif) +{ + BIF_RETTYPE ret; + + if (is_nil(arg)) + ERTS_BIF_PREP_RET(ret, new_binary(c_p, (byte *) "", 0)); + else if (is_not_list(arg)) + ERTS_BIF_PREP_ERROR(ret, c_p, BADARG); + else { + /* check for [binary()] case */ + Eterm h = CAR(list_val(arg)); + Eterm t = CDR(list_val(arg)); + if (is_binary(h) + && is_nil(t) + && !(HEADER_SUB_BIN == *(binary_val(h)) + && (((ErlSubBin *)binary_val(h))->bitoffs != 0 + || ((ErlSubBin *)binary_val(h))->bitsize != 0))) { + ERTS_BIF_PREP_RET(ret, h); + } + else { + ErtsL2BState state = ERTS_L2B_STATE_INITER(c_p, + arg, + bif, + erts_iolist_size_yielding, + erts_iolist_to_buf_yielding); + int orig_reds_left = ERTS_BIF_REDS_LEFT(c_p); + + /* + * First try to do it all at once without having to use + * yielding iolist_to_buf(). + */ + state.buf.iolist.reds_left = orig_reds_left; + switch (erts_iolist_size_yielding(&state.buf.iolist)) { + case ERTS_IOLIST_OK: { + ErlDrvSizeT size = state.buf.iolist.size; + Eterm bin; + char *buf; + + if (size == 0) { + ERTS_BIF_PREP_RET(ret, new_binary(c_p, (byte *) NULL, 0)); + break; /* done */ + } + + bin = new_binary(c_p, (byte *) NULL, size); + buf = (char *) binary_bytes(bin); + + if (size < ERTS_IOLIST_TO_BUF_BYTES_PER_RED*CONTEXT_REDS) { + /* An (over) estimation of reductions needed */ + int reds_left = state.buf.iolist.reds_left; + int to_buf_reds = orig_reds_left - reds_left; + to_buf_reds += size/ERTS_IOLIST_TO_BUF_BYTES_PER_RED; + if (to_buf_reds <= reds_left) { + ErlDrvSizeT res; + + res = erts_iolist_to_buf(arg, buf, size); + if (res == 0) { + BUMP_REDS(c_p, to_buf_reds); + ERTS_BIF_PREP_RET(ret, bin); + break; /* done */ + } + if (!ERTS_IOLIST_TO_BUF_FAILED(res)) + ERTS_INTERNAL_ERROR("iolist_size/iolist_to_buf missmatch"); + if (res == ERTS_IOLIST_TO_BUF_OVERFLOW) + goto overflow; + goto type_error; + } + } + /* + * Since size has been computed list_to_binary_chunk() expects + * state prepared for iolist_to_buf. + */ + state.bin = bin; + state.buf.buf = buf; + state.buf.len = size; + state.buf.iolist.obj = arg; + /* Fall through... */ + } + case ERTS_IOLIST_YIELD: + ret = list_to_binary_chunk(THE_NON_VALUE, + &state, + state.buf.iolist.reds_left, + 0); + break; + case ERTS_IOLIST_OVERFLOW: + overflow: + ERTS_BIF_PREP_ERROR(ret, c_p, SYSTEM_LIMIT); + break; + case ERTS_IOLIST_TYPE: + type_error: + default: + ERTS_BIF_PREP_ERROR(ret, c_p, BADARG); + break; + } + } + } + return ret; +} + +HIPE_WRAPPER_BIF_DISABLE_GC(list_to_binary, 1) + BIF_RETTYPE list_to_binary_1(BIF_ALIST_1) { - return erts_list_to_binary_bif(BIF_P, BIF_ARG_1); + return erts_list_to_binary_bif(BIF_P, BIF_ARG_1, bif_export[BIF_list_to_binary_1]); } -/* Turn a possibly deep list of ints (and binaries) into */ -/* One large binary object */ +HIPE_WRAPPER_BIF_DISABLE_GC(iolist_to_binary, 1) BIF_RETTYPE iolist_to_binary_1(BIF_ALIST_1) { if (is_binary(BIF_ARG_1)) { BIF_RET(BIF_ARG_1); } - return erts_list_to_binary_bif(BIF_P, BIF_ARG_1); + return erts_list_to_binary_bif(BIF_P, BIF_ARG_1, bif_export[BIF_iolist_to_binary_1]); } +static int bitstr_list_len(ErtsIOListState *); +static ErlDrvSizeT list_to_bitstr_buf_yielding(ErtsIOList2BufState *); +static ErlDrvSizeT list_to_bitstr_buf_not_yielding(ErtsIOList2BufState *); + +HIPE_WRAPPER_BIF_DISABLE_GC(list_to_bitstring, 1) + BIF_RETTYPE list_to_bitstring_1(BIF_ALIST_1) { - Eterm bin; - Uint sz; - int offset; - byte* bytes; - ErlSubBin* sb1; - Eterm* hp; - - if (is_nil(BIF_ARG_1)) { - BIF_RET(new_binary(BIF_P,(byte*)"",0)); - } - if (is_not_list(BIF_ARG_1)) { - error: - BIF_ERROR(BIF_P, BADARG); - } - switch (bitstr_list_len(BIF_ARG_1, &sz)) { - case ERTS_IOLIST_TYPE: - goto error; - case ERTS_IOLIST_OVERFLOW: - BIF_ERROR(BIF_P, SYSTEM_LIMIT); - } - bin = new_binary(BIF_P, (byte *)NULL, sz); - bytes = binary_bytes(bin); -#ifdef DEBUG - offset = list_to_bitstr_buf(BIF_ARG_1, (char*) bytes, sz); -#else - offset = list_to_bitstr_buf(BIF_ARG_1, (char*) bytes); -#endif - ASSERT(offset >= 0); - if (offset > 0) { - hp = HAlloc(BIF_P, ERL_SUB_BIN_SIZE); - sb1 = (ErlSubBin *) hp; - sb1->thing_word = HEADER_SUB_BIN; - sb1->size = sz-1; - sb1->offs = 0; - sb1->orig = bin; - sb1->bitoffs = 0; - sb1->bitsize = offset; - sb1->is_writable = 0; - bin = make_binary(sb1); + BIF_RETTYPE ret; + + if (is_nil(BIF_ARG_1)) + ERTS_BIF_PREP_RET(ret, new_binary(BIF_P, (byte *) "", 0)); + else if (is_not_list(BIF_ARG_1)) + ERTS_BIF_PREP_ERROR(ret, BIF_P, BADARG); + else { + /* check for [bitstring()] case */ + Eterm h = CAR(list_val(BIF_ARG_1)); + Eterm t = CDR(list_val(BIF_ARG_1)); + if (is_binary(h) && is_nil(t)) { + ERTS_BIF_PREP_RET(ret, h); + } + else { + ErtsL2BState state = ERTS_L2B_STATE_INITER(BIF_P, + BIF_ARG_1, + bif_export[BIF_list_to_bitstring_1], + bitstr_list_len, + list_to_bitstr_buf_yielding); + int orig_reds_left = ERTS_BIF_REDS_LEFT(BIF_P); + + /* + * First try to do it all at once without having to use + * yielding list_to_bitstr_buf(). + */ + state.buf.iolist.reds_left = orig_reds_left; + switch (bitstr_list_len(&state.buf.iolist)) { + case ERTS_IOLIST_OK: { + ErlDrvSizeT size = state.buf.iolist.size; + + state.bin = new_binary(BIF_P, (byte *) NULL, size); + state.buf.buf = (char *) binary_bytes(state.bin); + state.buf.len = size; + state.buf.iolist.obj = BIF_ARG_1; + + if (size < ERTS_IOLIST_TO_BUF_BYTES_PER_RED*CONTEXT_REDS) { + /* An (over) estimation of reductions needed */ + int reds_left = state.buf.iolist.reds_left; + int to_buf_reds = orig_reds_left - reds_left; + to_buf_reds += size/ERTS_IOLIST_TO_BUF_BYTES_PER_RED; + if (to_buf_reds <= reds_left) { + ErlDrvSizeT res; + + res = list_to_bitstr_buf_not_yielding(&state.buf); + if (res == 0) { + Eterm res_bin = l2b_final_touch(BIF_P, &state); + BUMP_REDS(BIF_P, to_buf_reds); + ERTS_BIF_PREP_RET(ret, res_bin); + break; /* done */ + } + if (!ERTS_IOLIST_TO_BUF_FAILED(res)) + ERTS_INTERNAL_ERROR("iolist_size/iolist_to_buf missmatch"); + if (res == ERTS_IOLIST_TO_BUF_OVERFLOW) + goto overflow; + goto type_error; + } + } + /* + * Since size has been computed list_to_binary_chunk() expects + * the state prepared for list_to_bitstr_buf. + */ + + /* Fall through... */ + } + case ERTS_IOLIST_YIELD: + ret = list_to_binary_chunk(THE_NON_VALUE, + &state, + state.buf.iolist.reds_left, + 0); + break; + case ERTS_IOLIST_OVERFLOW: + overflow: + ERTS_BIF_PREP_ERROR(ret, BIF_P, SYSTEM_LIMIT); + break; + case ERTS_IOLIST_TYPE: + type_error: + default: + ERTS_BIF_PREP_ERROR(ret, BIF_P, BADARG); + break; + } + } } - - BIF_RET(bin); + + return ret; } BIF_RETTYPE split_binary_2(BIF_ALIST_2) @@ -605,123 +1058,353 @@ BIF_RETTYPE split_binary_2(BIF_ALIST_2) * Local functions. */ +static int +list_to_bitstr_buf_bcopy(ErtsIOList2BufState *state, Eterm obj, int *yield_countp); + /* * The input list is assumed to be type-correct and the buffer is * assumed to be of sufficient size. Those assumptions are verified in * the DEBUG-built emulator. */ -static int +static ErlDrvSizeT +list_to_bitstr_buf(int yield_support, ErtsIOList2BufState *state) +{ + +#undef LIST_TO_BITSTR_BUF_BCOPY_DBG +#undef LIST_TO_BITSTR_BUF_BCOPY #ifdef DEBUG -list_to_bitstr_buf(Eterm obj, char* buf, Uint len) +#define LIST_TO_BITSTR_BUF_BCOPY_DBG \ + len -= size + (offset>7); #else -list_to_bitstr_buf(Eterm obj, char* buf) +#define LIST_TO_BITSTR_BUF_BCOPY_DBG #endif -{ - Eterm* objp; - int offset = 0; +#define LIST_TO_BITSTR_BUF_BCOPY(CONSP) \ + do { \ + byte* bptr; \ + Uint bitsize; \ + Uint bitoffs; \ + Uint num_bits; \ + size_t size = binary_size(obj); \ + if (yield_support) { \ + size_t max_size = ERTS_IOLIST_TO_BUF_BYTES_PER_YIELD_COUNT; \ + if (yield_count > 0) \ + max_size *= yield_count+1; \ + if (size > max_size) { \ + state->objp = CONSP; \ + goto L_bcopy_yield; \ + } \ + if (size >= ERTS_IOLIST_TO_BUF_BYTES_PER_YIELD_COUNT) { \ + int cost = (int) size; \ + cost /= ERTS_IOLIST_TO_BUF_BYTES_PER_YIELD_COUNT; \ + yield_count -= cost; \ + } \ + } \ + ASSERT(size <= len); \ + ERTS_GET_BINARY_BYTES(obj, bptr, bitoffs, bitsize); \ + num_bits = 8*size+bitsize; \ + copy_binary_to_buffer(buf, offset, bptr, bitoffs, num_bits); \ + offset += bitsize; \ + buf += size + (offset>7); \ + LIST_TO_BITSTR_BUF_BCOPY_DBG; \ + offset = offset & 7; \ + } while(0) + +#ifdef DEBUG + ErlDrvSizeT len; +#endif + Eterm obj; + char *buf; + Eterm *objp = NULL; + int offset; + int init_yield_count = 0, yield_count; DECLARE_ESTACK(s); - goto L_again; - - while (!ESTACK_ISEMPTY(s)) { - obj = ESTACK_POP(s); - L_again: - if (is_list(obj)) { - L_iter_list: - objp = list_val(obj); - obj = CAR(objp); - if (is_byte(obj)) { - ASSERT(len > 0); - if (offset == 0) { - *buf++ = unsigned_val(obj); - } else { - *buf = (char)((unsigned_val(obj) >> offset) | - ((*buf >> (8-offset)) << (8-offset))); - buf++; - *buf = (unsigned_val(obj) << (8-offset)); - } + + obj = state->iolist.obj; + buf = state->buf; + offset = state->offset; #ifdef DEBUG - len--; + len = state->len; #endif - } else if (is_binary(obj)) { - byte* bptr; - size_t size = binary_size(obj); - Uint bitsize; - Uint bitoffs; - Uint num_bits; - - ASSERT(size <= len); - ERTS_GET_BINARY_BYTES(obj, bptr, bitoffs, bitsize); - num_bits = 8*size+bitsize; - copy_binary_to_buffer(buf, offset, bptr, bitoffs, num_bits); - offset += bitsize; - buf += size + (offset>7); + + if (!yield_support) { + yield_count = init_yield_count = 0; /* Shut up faulty warning... >:-( */ + goto L_again; + } + else { + + if (state->iolist.reds_left <= 0) + return ERTS_IOLIST_TO_BUF_YIELD; + + ESTACK_CHANGE_ALLOCATOR(s, ERTS_ALC_T_SAVED_ESTACK); + init_yield_count = (ERTS_IOLIST_TO_BUF_YIELD_COUNT_PER_RED + * state->iolist.reds_left); + yield_count = init_yield_count; + + if (!state->iolist.estack.start) + goto L_again; + else { + int chk_stack; + /* Restart; restore state... */ + ESTACK_RESTORE(s, &state->iolist.estack); + + if (!state->bcopy.bptr) + chk_stack = 0; + else { + chk_stack = 1; + if (list_to_bitstr_buf_bcopy(state, THE_NON_VALUE, &yield_count)) { + /* Yield again... */ + BUMP_ALL_REDS(state->iolist.c_p); + state->iolist.reds_left = 0; + ESTACK_SAVE(s, &state->iolist.estack); + return ERTS_IOLIST_TO_BUF_YIELD; + } + buf = state->buf; + offset = state->offset; #ifdef DEBUG - len -= size + (offset>7); + len = state->len; #endif - offset = offset & 7; - } else if (is_list(obj)) { - ESTACK_PUSH(s, CDR(objp)); - goto L_iter_list; /* on head */ - } else { - ASSERT(is_nil(obj)); } - obj = CDR(objp); - if (is_list(obj)) { - goto L_iter_list; /* on tail */ - } else if (is_binary(obj)) { - byte* bptr; - size_t size = binary_size(obj); - Uint bitsize; - Uint bitoffs; - Uint num_bits; - - ASSERT(size <= len); - ERTS_GET_BINARY_BYTES(obj, bptr, bitoffs, bitsize); - num_bits = 8*size+bitsize; - copy_binary_to_buffer(buf, offset, bptr, bitoffs, num_bits); - offset += bitsize; - buf += size+(offset>7); + objp = state->objp; + state->objp = NULL; + + if (objp) + goto L_tail; + if (!chk_stack) + goto L_again; + /* check stack */ + } + } + + while (!ESTACK_ISEMPTY(s)) { + obj = ESTACK_POP(s); + L_again: + if (is_list(obj)) { + while (1) { /* Tail loop */ + while (1) { /* Head loop */ + if (yield_support && --yield_count <= 0) + goto L_yield; + objp = list_val(obj); + obj = CAR(objp); + if (is_byte(obj)) { + ASSERT(len > 0); + if (offset == 0) { + *buf++ = unsigned_val(obj); + } else { + *buf = (char)((unsigned_val(obj) >> offset) | + ((*buf >> (8-offset)) << (8-offset))); + buf++; + *buf = (unsigned_val(obj) << (8-offset)); + } #ifdef DEBUG - len -= size+(offset>7); + len--; #endif - offset = offset & 7; - } else { - ASSERT(is_nil(obj)); + } else if (is_binary(obj)) { + LIST_TO_BITSTR_BUF_BCOPY(objp); + } else if (is_list(obj)) { + ESTACK_PUSH(s, CDR(objp)); + continue; /* Head loop */ + } else { + ASSERT(is_nil(obj)); + } + break; + } + + L_tail: + + obj = CDR(objp); + if (is_list(obj)) { + continue; /* Tail loop */ + } else if (is_binary(obj)) { + LIST_TO_BITSTR_BUF_BCOPY(NULL); + } else { + ASSERT(is_nil(obj)); + } + break; } } else if (is_binary(obj)) { - byte* bptr; - size_t size = binary_size(obj); - Uint bitsize; - Uint bitoffs; - Uint num_bits; - - ASSERT(size <= len); - ERTS_GET_BINARY_BYTES(obj, bptr, bitoffs, bitsize); - num_bits = 8*size+bitsize; - copy_binary_to_buffer(buf, offset, bptr, bitoffs, num_bits); - offset += bitsize; - buf += size + (offset>7); -#ifdef DEBUG - len -= size + (offset>7); -#endif - offset = offset & 7; + LIST_TO_BITSTR_BUF_BCOPY(NULL); } else { + if (yield_support && --yield_count <= 0) + goto L_yield; ASSERT(is_nil(obj)); } } DESTROY_ESTACK(s); - return offset; + + if (yield_support) { + int reds; + CLEAR_SAVED_ESTACK(&state->iolist.estack); + reds = ((init_yield_count - yield_count - 1) + / ERTS_IOLIST_TO_BUF_YIELD_COUNT_PER_RED) + 1; + BUMP_REDS(state->iolist.c_p, reds); + state->iolist.reds_left -= reds; + if (state->iolist.reds_left < 0) + state->iolist.reds_left = 0; + } + state->buf = buf; + state->offset = offset; + return 0; + +L_bcopy_yield: + + state->buf = buf; + state->offset = offset; +#ifdef DEBUG + state->len = len; +#endif + + if (list_to_bitstr_buf_bcopy(state, obj, &yield_count) == 0) + ERTS_INTERNAL_ERROR("Missing yield"); + + BUMP_ALL_REDS(state->iolist.c_p); + state->iolist.reds_left = 0; + ESTACK_SAVE(s, &state->iolist.estack); + return ERTS_IOLIST_TO_BUF_YIELD; + +L_yield: + + BUMP_ALL_REDS(state->iolist.c_p); + state->iolist.reds_left = 0; + state->iolist.obj = obj; + state->buf = buf; + state->offset = offset; + ESTACK_SAVE(s, &state->iolist.estack); +#ifdef DEBUG + state->len = len; +#endif + return ERTS_IOLIST_TO_BUF_YIELD; + + +#undef LIST_TO_BITSTR_BUF_BCOPY_DBG +#undef LIST_TO_BITSTR_BUF_BCOPY + +} + +static ErlDrvSizeT +list_to_bitstr_buf_yielding(ErtsIOList2BufState *state) +{ + return list_to_bitstr_buf(1, state); +} + +static ErlDrvSizeT +list_to_bitstr_buf_not_yielding(ErtsIOList2BufState *state) +{ + return list_to_bitstr_buf(0, state); } static int -bitstr_list_len(Eterm obj, Uint* num_bytes) +list_to_bitstr_buf_bcopy(ErtsIOList2BufState *state, Eterm obj, int *yield_countp) +{ + int res; + char *buf = state->buf; + char *next_buf; + int offset = state->offset; + int next_offset; +#ifdef DEBUG + ErlDrvSizeT len = state->len; + ErlDrvSizeT next_len; +#endif + byte* bptr; + size_t size; + size_t max_size; + Uint bitoffs; + Uint num_bits; + Uint bitsize; + int yield_count = *yield_countp; + + if (state->bcopy.bptr) { + bptr = state->bcopy.bptr; + size = state->bcopy.size; + bitoffs = state->bcopy.bitoffs; + bitsize = state->bcopy.bitsize; + state->bcopy.bptr = NULL; + } + else { + + ASSERT(is_binary(obj)); + + size = binary_size(obj); + + ASSERT(size <= len); + + ERTS_GET_BINARY_BYTES(obj, bptr, bitoffs, bitsize); + } + + max_size = (size_t) ERTS_IOLIST_TO_BUF_BYTES_PER_YIELD_COUNT; + if (yield_count > 0) + max_size *= (size_t) (yield_count+1); + + if (size <= max_size) { + if (size >= ERTS_IOLIST_TO_BUF_BYTES_PER_YIELD_COUNT) { + int cost = (int) size; + cost /= ERTS_IOLIST_TO_BUF_BYTES_PER_YIELD_COUNT; + yield_count -= cost; + } + next_offset = offset + bitsize; + next_buf = buf + size+(next_offset>7); +#ifdef DEBUG + next_len = len - size+(next_offset>7); +#endif + next_offset &= 7; + num_bits = 8*size+bitsize; + res = 0; + } + else { + ASSERT(0 < max_size && max_size < size); + yield_count = 0; + state->bcopy.bptr = bptr + max_size; + state->bcopy.bitoffs = bitoffs; + state->bcopy.bitsize = bitsize; + state->bcopy.size = size - max_size; + next_buf = buf + max_size; +#ifdef DEBUG + next_len = len - max_size; +#endif + next_offset = offset; + num_bits = 8*max_size; + size = max_size; + res = 1; + } + + copy_binary_to_buffer(buf, offset, bptr, bitoffs, num_bits); + + state->offset = next_offset; + state->buf = next_buf; +#ifdef DEBUG + state->len = next_len; +#endif + *yield_countp = yield_count; + + return res; +} + +static int +bitstr_list_len(ErtsIOListState *state) { Eterm* objp; - Uint len = 0; - Uint offs = 0; + Eterm obj; + Uint len, offs; + int res, init_yield_count, yield_count; DECLARE_ESTACK(s); + + if (state->reds_left <= 0) + return ERTS_IOLIST_YIELD; + + len = (Uint) state->size; + offs = state->offs; + obj = state->obj; + + ESTACK_CHANGE_ALLOCATOR(s, ERTS_ALC_T_SAVED_ESTACK); + init_yield_count = ERTS_IOLIST_SIZE_YIELDS_COUNT_PER_RED; + init_yield_count *= state->reds_left; + yield_count = init_yield_count; + if (state->estack.start) { + /* Restart; restore estack... */ + ESTACK_RESTORE(s, &state->estack); + } + goto L_again; #define SAFE_ADD(Var, Val) \ @@ -748,46 +1431,55 @@ bitstr_list_len(Eterm obj, Uint* num_bytes) obj = ESTACK_POP(s); L_again: if (is_list(obj)) { - L_iter_list: - objp = list_val(obj); - /* Head */ - obj = CAR(objp); - if (is_byte(obj)) { - len++; - if (len == 0) { - goto L_overflow_error; + while (1) { /* Tail loop */ + while (1) { /* Head loop */ + if (--yield_count <= 0) + goto L_yield; + objp = list_val(obj); + /* Head */ + obj = CAR(objp); + if (is_byte(obj)) { + len++; + if (len == 0) { + goto L_overflow_error; + } + } else if (is_binary(obj)) { + SAFE_ADD(len, binary_size(obj)); + SAFE_ADD_BITSIZE(offs, obj); + } else if (is_list(obj)) { + ESTACK_PUSH(s, CDR(objp)); + continue; /* Head loop */ + } else if (is_not_nil(obj)) { + goto L_type_error; + } + break; } - } else if (is_binary(obj)) { - SAFE_ADD(len, binary_size(obj)); - SAFE_ADD_BITSIZE(offs, obj); - } else if (is_list(obj)) { - ESTACK_PUSH(s, CDR(objp)); - goto L_iter_list; /* on head */ - } else if (is_not_nil(obj)) { - goto L_type_error; + /* Tail */ + obj = CDR(objp); + if (is_list(obj)) + continue; /* Tail loop */ + else if (is_binary(obj)) { + SAFE_ADD(len, binary_size(obj)); + SAFE_ADD_BITSIZE(offs, obj); + } else if (is_not_nil(obj)) { + goto L_type_error; + } + break; } - /* Tail */ - obj = CDR(objp); - if (is_list(obj)) - goto L_iter_list; /* on tail */ - else if (is_binary(obj)) { + } else { + if (--yield_count <= 0) + goto L_yield; + if (is_binary(obj)) { SAFE_ADD(len, binary_size(obj)); SAFE_ADD_BITSIZE(offs, obj); } else if (is_not_nil(obj)) { goto L_type_error; } - } else if (is_binary(obj)) { - SAFE_ADD(len, binary_size(obj)); - SAFE_ADD_BITSIZE(offs, obj); - } else if (is_not_nil(obj)) { - goto L_type_error; } } #undef SAFE_ADD #undef SAFE_ADD_BITSIZE - DESTROY_ESTACK(s); - /* * Make sure that the number of bits in the bitstring will fit * in an Uint to ensure that the binary can be matched using @@ -800,15 +1492,42 @@ bitstr_list_len(Eterm obj, Uint* num_bytes) if (len << 3 < len) { goto L_overflow_error; } - *num_bytes = len; - return ERTS_IOLIST_OK; + state->size = len; - L_type_error: - DESTROY_ESTACK(s); - return ERTS_IOLIST_TYPE; + res = ERTS_IOLIST_OK; + + L_return: { + int yc = init_yield_count - yield_count; + int reds; + + DESTROY_ESTACK(s); + CLEAR_SAVED_ESTACK(&state->estack); + + reds = (yc - 1)/ERTS_IOLIST_SIZE_YIELDS_COUNT_PER_RED + 1; + BUMP_REDS(state->c_p, reds); + state->reds_left -= reds; + state->size = (ErlDrvSizeT) len; + state->have_size = 1; + return res; + } L_overflow_error: - DESTROY_ESTACK(s); - return ERTS_IOLIST_OVERFLOW; + res = ERTS_IOLIST_OVERFLOW; + len = 0; + goto L_return; + + L_type_error: + res = ERTS_IOLIST_TYPE; + len = 0; + goto L_return; + + L_yield: + BUMP_ALL_REDS(state->c_p); + state->reds_left = 0; + state->size = len; + state->offs = offs; + state->obj = obj; + ESTACK_SAVE(s, &state->estack); + return ERTS_IOLIST_YIELD; } diff --git a/erts/emulator/beam/break.c b/erts/emulator/beam/break.c index 7d4f52ee23..08265b590d 100644 --- a/erts/emulator/beam/break.c +++ b/erts/emulator/beam/break.c @@ -256,6 +256,7 @@ print_process_info(int to, void *to_arg, Process *p) p->current[1], p->current[2]); } + erts_print(to, to_arg, "Run queue: %d\n", erts_get_runq_proc(p)->ix); erts_print(to, to_arg, "Spawned by: %T\n", p->parent); approx_started = (time_t) p->approx_started; diff --git a/erts/emulator/beam/erl_alloc.c b/erts/emulator/beam/erl_alloc.c index 05ac24e04d..90cd227fae 100644 --- a/erts/emulator/beam/erl_alloc.c +++ b/erts/emulator/beam/erl_alloc.c @@ -1873,8 +1873,8 @@ erts_alc_fatal_error(int error, int func, ErtsAlcType_t n, ...) size = va_arg(argp, Uint); va_end(argp); erl_exit(1, - "%s: Cannot %s %lu bytes of memory (of type \"%s\").\n", - allctr_str, op, size, t_str); + "%s: Cannot %s %lu bytes of memory (of type \"%s\", thread %d).\n", + allctr_str, op, size, t_str, ERTS_ALC_GET_THR_IX()); break; } case ERTS_ALC_E_NOALLCTR: diff --git a/erts/emulator/beam/erl_alloc.types b/erts/emulator/beam/erl_alloc.types index 17ac6316b7..37354b7f8d 100644 --- a/erts/emulator/beam/erl_alloc.types +++ b/erts/emulator/beam/erl_alloc.types @@ -357,6 +357,7 @@ type DB_MS_PSDO_PROC LONG_LIVED_LOW ETS db_match_pseudo_proc type SCHDLR_DATA LONG_LIVED_LOW SYSTEM scheduler_data type LL_TEMP_TERM LONG_LIVED_LOW SYSTEM ll_temp_term +type NIF_TRAP_EXPORT STANDARD_LOW CODE nif_trap_export_entry type EXPORT LONG_LIVED_LOW CODE export_entry type MONITOR_SH STANDARD_LOW PROCESSES monitor_sh type NLINK_SH STANDARD_LOW PROCESSES nlink_sh @@ -375,6 +376,7 @@ type DB_MS_PSDO_PROC LONG_LIVED ETS db_match_pseudo_proc type SCHDLR_DATA LONG_LIVED SYSTEM scheduler_data type LL_TEMP_TERM LONG_LIVED SYSTEM ll_temp_term +type NIF_TRAP_EXPORT STANDARD CODE nif_trap_export_entry type EXPORT LONG_LIVED CODE export_entry type MONITOR_SH FIXED_SIZE PROCESSES monitor_sh type NLINK_SH FIXED_SIZE PROCESSES nlink_sh diff --git a/erts/emulator/beam/erl_alloc_util.c b/erts/emulator/beam/erl_alloc_util.c index 45f0cc4312..a4e164bf51 100644 --- a/erts/emulator/beam/erl_alloc_util.c +++ b/erts/emulator/beam/erl_alloc_util.c @@ -3274,6 +3274,15 @@ create_carrier(Allctr_t *allctr, Uint umem_sz, UWord flags) ASSERT(!(flags & CFLG_FORCE_MSEG && flags & CFLG_FORCE_SYS_ALLOC)); + if (umem_sz > (ERTS_UINT_MAX - ERTS_UINT_MAX/100)) { + /* Do an overly conservative _overflow_ check here so we don't + * have to deal with it from here on. I guess we could be more accurate + * but I don't think the need to allocate over 99% of the address space + * will ever arise on any machine, neither 32 nor 64 bit. + */ + return NULL; + } + blk_sz = UMEMSZ2BLKSZ(allctr, umem_sz); #ifdef ERTS_SMP diff --git a/erts/emulator/beam/erl_bif_binary.c b/erts/emulator/beam/erl_bif_binary.c index ff775691b3..3bf78adce7 100644 --- a/erts/emulator/beam/erl_bif_binary.c +++ b/erts/emulator/beam/erl_bif_binary.c @@ -1324,9 +1324,9 @@ static int parse_match_opts_list(Eterm l, Eterm bin, Uint *posp, Uint *endp) goto badarg; } if (len < 0) { - Sint lentmp = -len; + Uint lentmp = -(Uint)len; /* overflow */ - if (lentmp == len || lentmp < 0 || -lentmp != len) { + if ((Sint)lentmp < 0) { goto badarg; } len = lentmp; @@ -1555,9 +1555,9 @@ BIF_RETTYPE erts_binary_part(Process *p, Eterm binary, Eterm epos, Eterm elen) goto badarg; } if (len < 0) { - Sint lentmp = -len; + Uint lentmp = -(Uint)len; /* overflow */ - if (lentmp == len || lentmp < 0 || -lentmp != len) { + if ((Sint)lentmp < 0) { goto badarg; } len = lentmp; @@ -1644,9 +1644,9 @@ BIF_RETTYPE erts_gc_binary_part(Process *p, Eterm *reg, Eterm live, int range_is goto badarg; } if (len < 0) { - Sint lentmp = -len; + Uint lentmp = -(Uint)len; /* overflow */ - if (lentmp == len || lentmp < 0 || -lentmp != len) { + if ((Sint)lentmp < 0) { goto badarg; } len = lentmp; @@ -2213,9 +2213,9 @@ static BIF_RETTYPE binary_bin_to_list_common(Process *p, goto badarg; } if (len < 0) { - Sint lentmp = -len; + Uint lentmp = -(Uint)len; /* overflow */ - if (lentmp == len || lentmp < 0 || -lentmp != len) { + if ((Sint)lentmp < 0) { goto badarg; } len = lentmp; @@ -2294,18 +2294,11 @@ BIF_RETTYPE binary_bin_to_list_1(BIF_ALIST_1) BIF_ERROR(BIF_P,BADARG); } -/* - * Ok, erlang:list_to_binary does not interrupt, and we really don't want - * an alternative implementation for the exact same thing, why we - * have descided to use the old non-restarting implementation for now. - * In reality, there are seldom many iterations involved in doing this, so the - * problem of long-running bifs is not really that big in this case. - * So, for now we use the old implementation also in the module binary. - */ +HIPE_WRAPPER_BIF_DISABLE_GC(binary_list_to_bin, 1) BIF_RETTYPE binary_list_to_bin_1(BIF_ALIST_1) { - return erts_list_to_binary_bif(BIF_P, BIF_ARG_1); + return erts_list_to_binary_bif(BIF_P, BIF_ARG_1, bif_export[BIF_binary_list_to_bin_1]); } typedef struct { diff --git a/erts/emulator/beam/erl_bif_info.c b/erts/emulator/beam/erl_bif_info.c index 2adba9b240..6efe9d9550 100755..100644 --- a/erts/emulator/beam/erl_bif_info.c +++ b/erts/emulator/beam/erl_bif_info.c @@ -2691,6 +2691,11 @@ BIF_RETTYPE system_info_1(BIF_ALIST_1) else if (ERTS_IS_ATOM_STR("ets_limit",BIF_ARG_1)) { BIF_RET(make_small(erts_db_get_max_tabs())); } + else if (ERTS_IS_ATOM_STR("tolerant_timeofday",BIF_ARG_1)) { + BIF_RET(erts_disable_tolerant_timeofday + ? am_disabled + : am_enabled); + } BIF_ERROR(BIF_P, BADARG); } @@ -3050,6 +3055,25 @@ fun_info_2(BIF_ALIST_2) return TUPLE2(hp, what, val); } +BIF_RETTYPE +fun_info_mfa_1(BIF_ALIST_1) +{ + Process* p = BIF_P; + Eterm fun = BIF_ARG_1; + Eterm* hp; + + if (is_fun(fun)) { + ErlFunThing* funp = (ErlFunThing *) fun_val(fun); + hp = HAlloc(p, 4); + BIF_RET(TUPLE3(hp,funp->fe->module,funp->fe->address[-2],make_small(funp->arity))); + } else if (is_export(fun)) { + Export* exp = (Export *) ((UWord) (export_val(fun))[1]); + hp = HAlloc(p, 4); + BIF_RET(TUPLE3(hp,exp->code[0],exp->code[1],make_small(exp->code[2]))); + } + BIF_ERROR(p, BADARG); +} + BIF_RETTYPE is_process_alive_1(BIF_ALIST_1) { if(is_internal_pid(BIF_ARG_1)) { @@ -3851,16 +3875,19 @@ static Eterm lcnt_build_lock_stats_term(Eterm **hpp, Uint *szp, erts_lcnt_lock_s Uint tries = 0, colls = 0; unsigned long timer_s = 0, timer_ns = 0, timer_n = 0; unsigned int line = 0; + unsigned int i; Eterm af, uil; Eterm uit, uic; Eterm uits, uitns, uitn; Eterm tt, tstat, tloc, t; + Eterm thist, vhist[ERTS_LCNT_HISTOGRAM_SLOT_SIZE]; /* term: - * [{{file, line}, {tries, colls, {seconds, nanoseconds, n_blocks}}}] + * [{{file, line}, {tries, colls, {seconds, nanoseconds, n_blocks}}, + * { .. histogram .. }] */ - + tries = (Uint) ethr_atomic_read(&stats->tries); colls = (Uint) ethr_atomic_read(&stats->colls); @@ -3869,23 +3896,27 @@ static Eterm lcnt_build_lock_stats_term(Eterm **hpp, Uint *szp, erts_lcnt_lock_s timer_ns = stats->timer.ns; timer_n = stats->timer_n; - af = erts_atom_put(stats->file, strlen(stats->file), ERTS_ATOM_ENC_LATIN1, 1); + af = erts_atom_put((byte *)stats->file, strlen(stats->file), ERTS_ATOM_ENC_LATIN1, 1); uil = erts_bld_uint( hpp, szp, line); tloc = erts_bld_tuple(hpp, szp, 2, af, uil); - uit = erts_bld_uint( hpp, szp, tries); - uic = erts_bld_uint( hpp, szp, colls); - + uit = erts_bld_uint( hpp, szp, tries); + uic = erts_bld_uint( hpp, szp, colls); + uits = erts_bld_uint( hpp, szp, timer_s); uitns = erts_bld_uint( hpp, szp, timer_ns); uitn = erts_bld_uint( hpp, szp, timer_n); tt = erts_bld_tuple(hpp, szp, 3, uits, uitns, uitn); tstat = erts_bld_tuple(hpp, szp, 3, uit, uic, tt); - - t = erts_bld_tuple(hpp, szp, 2, tloc, tstat); - - res = erts_bld_cons( hpp, szp, t, res); + + for(i = 0; i < ERTS_LCNT_HISTOGRAM_SLOT_SIZE; i++) { + vhist[i] = erts_bld_uint(hpp, szp, stats->hist.ns[i]); + } + thist = erts_bld_tuplev(hpp, szp, ERTS_LCNT_HISTOGRAM_SLOT_SIZE, vhist); + + t = erts_bld_tuple(hpp, szp, 3, tloc, tstat, thist); + res = erts_bld_cons( hpp, szp, t, res); return res; } @@ -3906,13 +3937,13 @@ static Eterm lcnt_build_lock_term(Eterm **hpp, Uint *szp, erts_lcnt_lock_t *lock ASSERT(ltype); - type = erts_atom_put(ltype, strlen(ltype), ERTS_ATOM_ENC_LATIN1, 1); - name = erts_atom_put(lock->name, strlen(lock->name), ERTS_ATOM_ENC_LATIN1, 1); + type = erts_atom_put((byte *)ltype, strlen(ltype), ERTS_ATOM_ENC_LATIN1, 1); + name = erts_atom_put((byte *)lock->name, strlen(lock->name), ERTS_ATOM_ENC_LATIN1, 1); if (lock->flag & ERTS_LCNT_LT_ALLOC) { /* use allocator types names as id's for allocator locks */ ltype = (char *) ERTS_ALC_A2AD(signed_val(lock->id)); - id = erts_atom_put(ltype, strlen(ltype), ERTS_ATOM_ENC_LATIN1, 1); + id = erts_atom_put((byte *)ltype, strlen(ltype), ERTS_ATOM_ENC_LATIN1, 1); } else if (lock->flag & ERTS_LCNT_LT_PROCLOCK) { /* use registered names as id's for process locks if available */ proc = erts_proc_lookup(lock->id); @@ -3923,16 +3954,15 @@ static Eterm lcnt_build_lock_term(Eterm **hpp, Uint *szp, erts_lcnt_lock_t *lock id = lock->id; } } else { - id = lock->id; + id = lock->id; } - + for (i = 0; i < lock->n_stats; i++) { stats = lcnt_build_lock_stats_term(hpp, szp, &(lock->stats[i]), stats); } - - t = erts_bld_tuple(hpp, szp, 4, name, id, type, stats); - - res = erts_bld_cons( hpp, szp, t, res); + + t = erts_bld_tuple(hpp, szp, 4, name, id, type, stats); + res = erts_bld_cons( hpp, szp, t, res); return res; } @@ -3952,12 +3982,12 @@ static Eterm lcnt_build_result_term(Eterm **hpp, Uint *szp, erts_lcnt_data_t *da dtns = erts_bld_uint( hpp, szp, data->duration.ns); tdt = erts_bld_tuple(hpp, szp, 2, dts, dtns); - adur = erts_atom_put(str_duration, strlen(str_duration), ERTS_ATOM_ENC_LATIN1, 1); + adur = erts_atom_put((byte *)str_duration, strlen(str_duration), ERTS_ATOM_ENC_LATIN1, 1); tdur = erts_bld_tuple(hpp, szp, 2, adur, tdt); /* lock tuple */ - aloc = erts_atom_put(str_locks, strlen(str_locks), ERTS_ATOM_ENC_LATIN1, 1); + aloc = erts_atom_put((byte *)str_locks, strlen(str_locks), ERTS_ATOM_ENC_LATIN1, 1); for (lock = data->current_locks->head; lock != NULL ; lock = lock->next ) { lloc = lcnt_build_lock_term(hpp, szp, lock, lloc); diff --git a/erts/emulator/beam/erl_binary.h b/erts/emulator/beam/erl_binary.h index 819b19e566..06dfeb1260 100644 --- a/erts/emulator/beam/erl_binary.h +++ b/erts/emulator/beam/erl_binary.h @@ -166,7 +166,7 @@ Eterm erts_bin_bytes_to_list(Eterm previous, Eterm* hp, byte* bytes, Uint size, * Common implementation for erlang:list_to_binary/1 and binary:list_to_bin/1 */ -BIF_RETTYPE erts_list_to_binary_bif(Process *p, Eterm arg); +BIF_RETTYPE erts_list_to_binary_bif(Process *p, Eterm arg, Export *bif); BIF_RETTYPE erts_gc_binary_part(Process *p, Eterm *reg, Eterm live, int range_is_tuple); BIF_RETTYPE erts_binary_part(Process *p, Eterm binary, Eterm epos, Eterm elen); @@ -236,6 +236,8 @@ erts_bin_drv_alloc_fnf(Uint size) { Uint bsize = ERTS_SIZEOF_Binary(size) + CHICKEN_PAD; void *res; + if (bsize < size) /* overflow */ + return NULL; res = erts_alloc_fnf(ERTS_ALC_T_DRV_BINARY, bsize); ERTS_CHK_BIN_ALIGNMENT(res); return (Binary *) res; @@ -246,6 +248,8 @@ erts_bin_drv_alloc(Uint size) { Uint bsize = ERTS_SIZEOF_Binary(size) + CHICKEN_PAD; void *res; + if (bsize < size) /* overflow */ + erts_alloc_enomem(ERTS_ALC_T_DRV_BINARY, size); res = erts_alloc(ERTS_ALC_T_DRV_BINARY, bsize); ERTS_CHK_BIN_ALIGNMENT(res); return (Binary *) res; @@ -257,6 +261,8 @@ erts_bin_nrml_alloc(Uint size) { Uint bsize = ERTS_SIZEOF_Binary(size) + CHICKEN_PAD; void *res; + if (bsize < size) /* overflow */ + erts_alloc_enomem(ERTS_ALC_T_BINARY, size); res = erts_alloc(ERTS_ALC_T_BINARY, bsize); ERTS_CHK_BIN_ALIGNMENT(res); return (Binary *) res; @@ -267,11 +273,12 @@ erts_bin_realloc_fnf(Binary *bp, Uint size) { Binary *nbp; Uint bsize = ERTS_SIZEOF_Binary(size) + CHICKEN_PAD; + ErtsAlcType_t type = (bp->flags & BIN_FLAG_DRV) ? ERTS_ALC_T_DRV_BINARY + : ERTS_ALC_T_BINARY; ASSERT((bp->flags & BIN_FLAG_MAGIC) == 0); - if (bp->flags & BIN_FLAG_DRV) - nbp = erts_realloc_fnf(ERTS_ALC_T_DRV_BINARY, (void *) bp, bsize); - else - nbp = erts_realloc_fnf(ERTS_ALC_T_BINARY, (void *) bp, bsize); + if (bsize < size) /* overflow */ + return NULL; + nbp = erts_realloc_fnf(type, (void *) bp, bsize); ERTS_CHK_BIN_ALIGNMENT(nbp); return nbp; } @@ -281,17 +288,14 @@ erts_bin_realloc(Binary *bp, Uint size) { Binary *nbp; Uint bsize = ERTS_SIZEOF_Binary(size) + CHICKEN_PAD; + ErtsAlcType_t type = (bp->flags & BIN_FLAG_DRV) ? ERTS_ALC_T_DRV_BINARY + : ERTS_ALC_T_BINARY; ASSERT((bp->flags & BIN_FLAG_MAGIC) == 0); - if (bp->flags & BIN_FLAG_DRV) - nbp = erts_realloc_fnf(ERTS_ALC_T_DRV_BINARY, (void *) bp, bsize); - else - nbp = erts_realloc_fnf(ERTS_ALC_T_BINARY, (void *) bp, bsize); + if (bsize < size) /* overflow */ + erts_realloc_enomem(type, bp, size); + nbp = erts_realloc_fnf(type, (void *) bp, bsize); if (!nbp) - erts_realloc_n_enomem(ERTS_ALC_T2N(bp->flags & BIN_FLAG_DRV - ? ERTS_ALC_T_DRV_BINARY - : ERTS_ALC_T_BINARY), - bp, - bsize); + erts_realloc_enomem(type, bp, bsize); ERTS_CHK_BIN_ALIGNMENT(nbp); return nbp; } @@ -312,6 +316,7 @@ erts_create_magic_binary(Uint size, void (*destructor)(Binary *)) { Uint bsize = ERTS_MAGIC_BIN_SIZE(size); Binary* bptr = erts_alloc_fnf(ERTS_ALC_T_BINARY, bsize); + ASSERT(bsize > size); if (!bptr) erts_alloc_n_enomem(ERTS_ALC_T2N(ERTS_ALC_T_BINARY), bsize); ERTS_CHK_BIN_ALIGNMENT(bptr); diff --git a/erts/emulator/beam/erl_db.c b/erts/emulator/beam/erl_db.c index a5d67571e2..8f246ffa07 100644 --- a/erts/emulator/beam/erl_db.c +++ b/erts/emulator/beam/erl_db.c @@ -259,10 +259,11 @@ static void schedule_free_dbtable(DbTable* tb) /* * NON-SMP case: Caller is *not* allowed to access the *tb * structure after this function has returned! - * SMP case: Caller is allowed to access the *tb structure - * until the bif has returned (we typically - * need to unlock the table lock after this - * function has returned). + * SMP case: Caller is allowed to access the *common* part of the *tb + * structure until the bif has returned (we typically need to + * unlock the table lock after this function has returned). + * Caller is *not* allowed to access the specialized part + * (hash or tree) of *tb after this function has returned. */ ASSERT(erts_refc_read(&tb->common.ref, 0) == 0); erts_schedule_thr_prgr_later_cleanup_op(free_dbtable, @@ -3279,34 +3280,37 @@ erts_db_process_exiting(Process *c_p, ErtsProcLocks c_p_locks) } erts_smp_rwmtx_runlock(mmtl); if (tb) { - int reds; - DbFixation** pp; + int reds = 0; db_lock(tb, LCK_WRITE_REC); - #ifdef ERTS_SMP - erts_smp_mtx_lock(&tb->common.fixlock); - #endif - reds = 10; - - for (pp = &tb->common.fixations; *pp != NULL; - pp = &(*pp)->next) { - if ((*pp)->pid == pid) { - DbFixation* fix = *pp; - erts_aint_t diff = -((erts_aint_t) fix->counter); - erts_refc_add(&tb->common.ref,diff,0); - *pp = fix->next; - erts_db_free(ERTS_ALC_T_DB_FIXATION, - tb, fix, sizeof(DbFixation)); - ERTS_ETS_MISC_MEM_ADD(-sizeof(DbFixation)); - break; + if (!(tb->common.status & DB_DELETE)) { + DbFixation** pp; + + #ifdef ERTS_SMP + erts_smp_mtx_lock(&tb->common.fixlock); + #endif + reds = 10; + + for (pp = &tb->common.fixations; *pp != NULL; + pp = &(*pp)->next) { + if ((*pp)->pid == pid) { + DbFixation* fix = *pp; + erts_aint_t diff = -((erts_aint_t) fix->counter); + erts_refc_add(&tb->common.ref,diff,0); + *pp = fix->next; + erts_db_free(ERTS_ALC_T_DB_FIXATION, + tb, fix, sizeof(DbFixation)); + ERTS_ETS_MISC_MEM_ADD(-sizeof(DbFixation)); + break; + } + } + #ifdef ERTS_SMP + erts_smp_mtx_unlock(&tb->common.fixlock); + #endif + if (!IS_FIXED(tb) && IS_HASH_TABLE(tb->common.status)) { + db_unfix_table_hash(&(tb->hash)); + reds += 40; } - } - #ifdef ERTS_SMP - erts_smp_mtx_unlock(&tb->common.fixlock); - #endif - if (!IS_FIXED(tb) && IS_HASH_TABLE(tb->common.status)) { - db_unfix_table_hash(&(tb->hash)); - reds += 40; } db_unlock(tb, LCK_WRITE_REC); BUMP_REDS(c_p, reds); diff --git a/erts/emulator/beam/erl_driver.h b/erts/emulator/beam/erl_driver.h index 3ecb379326..f9938fc66c 100644 --- a/erts/emulator/beam/erl_driver.h +++ b/erts/emulator/beam/erl_driver.h @@ -133,7 +133,7 @@ typedef struct { #define ERL_DRV_EXTENDED_MARKER (0xfeeeeeed) #define ERL_DRV_EXTENDED_MAJOR_VERSION 3 -#define ERL_DRV_EXTENDED_MINOR_VERSION 0 +#define ERL_DRV_EXTENDED_MINOR_VERSION 1 /* * The emulator will refuse to load a driver with a major version @@ -198,7 +198,7 @@ typedef long long ErlDrvSInt64; #error No 64-bit integer type #endif -#if defined(__WIN32__) +#if defined(__WIN32__) || defined(_WIN32) typedef ErlDrvUInt ErlDrvSizeT; typedef ErlDrvSInt ErlDrvSSizeT; #else diff --git a/erts/emulator/beam/erl_drv_nif.h b/erts/emulator/beam/erl_drv_nif.h index 3f829ea7ea..4e8c6dc68b 100644 --- a/erts/emulator/beam/erl_drv_nif.h +++ b/erts/emulator/beam/erl_drv_nif.h @@ -35,6 +35,7 @@ typedef struct { int scheduler_threads; int nif_major_version; int nif_minor_version; + int dirty_scheduler_support; } ErlDrvSysInfo; typedef struct { diff --git a/erts/emulator/beam/erl_gc.c b/erts/emulator/beam/erl_gc.c index aa15d2cc57..0db42d4325 100644 --- a/erts/emulator/beam/erl_gc.c +++ b/erts/emulator/beam/erl_gc.c @@ -2018,6 +2018,20 @@ setup_rootset(Process *p, Eterm *objv, int nobj, Rootset *rootset) roots[n].sz = 1; n++; } + + /* + * If a NIF has saved arguments, they need to be added + */ + if (ERTS_PROC_GET_NIF_TRAP_EXPORT(p)) { + Eterm* argv; + int argc; + if (erts_setup_nif_gc(p, &argv, &argc)) { + roots[n].v = argv; + roots[n].sz = argc; + n++; + } + } + ASSERT(n <= rootset->size); mp = p->msg.first; diff --git a/erts/emulator/beam/erl_init.c b/erts/emulator/beam/erl_init.c index d54658f1ea..88c4006934 100644 --- a/erts/emulator/beam/erl_init.c +++ b/erts/emulator/beam/erl_init.c @@ -324,7 +324,6 @@ erl_init(int ncpu, BIN_VH_MIN_SIZE = erts_next_heap_size(BIN_VH_MIN_SIZE, 0); erts_init_trace(); - erts_init_binary(); erts_init_bits(); erts_code_ix_init(); erts_init_fun_table(); @@ -337,6 +336,7 @@ erl_init(int ncpu, erts_ddll_init(); init_emulator(); erts_ptab_init(); /* Must be after init_emulator() */ + erts_init_binary(); /* Must be after init_emulator() */ erts_bp_init(); init_db(); /* Must be after init_emulator */ erts_bif_timer_init(); @@ -2066,8 +2066,10 @@ erl_exit_vv(int n, int flush_async, char *fmt, va_list args1, va_list args2) system_cleanup(flush_async); save_statistics(); - - an = abs(n); + if (n < 0) + an = -(unsigned int)n; + else + an = n; if (erts_mtrace_enabled) erts_mtrace_exit((Uint32) an); diff --git a/erts/emulator/beam/erl_lock_check.c b/erts/emulator/beam/erl_lock_check.c index 7e3a90779d..b105ece6f1 100644 --- a/erts/emulator/beam/erl_lock_check.c +++ b/erts/emulator/beam/erl_lock_check.c @@ -139,7 +139,6 @@ static erts_lc_lock_order_t erts_lock_order[] = { { "drv_tsd", NULL }, { "async_enq_mtx", NULL }, #ifdef ERTS_SMP - { "sys_msg_q", NULL }, { "atom_tab", NULL }, { "make_ref", NULL }, { "misc_op_list_pre_alloc_lock", "address" }, @@ -148,6 +147,7 @@ static erts_lc_lock_order_t erts_lock_order[] = { { "btm_pre_alloc_lock", NULL, }, { "dist_entry_out_queue", "address" }, { "port_sched_lock", "port_id" }, + { "sys_msg_q", NULL }, { "port_table", NULL }, #endif { "mtrace_op", NULL }, @@ -227,8 +227,7 @@ rw_op_str(Uint16 flags) case ERTS_LC_FLG_LO_READ: return " (r)"; case ERTS_LC_FLG_LO_WRITE: - erts_fprintf(stderr, "\nInternal error\n"); - lc_abort(); + ERTS_INTERNAL_ERROR("Only write flag present"); default: break; } @@ -270,9 +269,9 @@ union erts_lc_free_block_t_ { static ethr_tsd_key locks_key; -static erts_lc_locked_locks_t *erts_locked_locks; +static erts_lc_locked_locks_t *erts_locked_locks = NULL; -static erts_lc_free_block_t *free_blocks; +static erts_lc_free_block_t *free_blocks = NULL; #ifdef ERTS_LC_STATIC_ALLOC #define ERTS_LC_FB_CHUNK_SIZE 10000 @@ -311,8 +310,7 @@ static ERTS_INLINE void lc_free(void *p) static void *lc_core_alloc(void) { lc_unlock(); - erts_fprintf(stderr, "Lock checker out of memory!\n"); - lc_abort(); + ERTS_INTERNAL_ERROR("Lock checker out of memory!\n"); } #else @@ -325,8 +323,7 @@ static void *lc_core_alloc(void) fbs = (erts_lc_free_block_t *) malloc(sizeof(erts_lc_free_block_t) * ERTS_LC_FB_CHUNK_SIZE); if (!fbs) { - erts_fprintf(stderr, "Lock checker failed to allocate memory!\n"); - lc_abort(); + ERTS_INTERNAL_ERROR("Lock checker failed to allocate memory!"); } for (i = 1; i < ERTS_LC_FB_CHUNK_SIZE - 1; i++) { #ifdef DEBUG @@ -366,11 +363,11 @@ create_locked_locks(char *thread_name) { erts_lc_locked_locks_t *l_lcks = malloc(sizeof(erts_lc_locked_locks_t)); if (!l_lcks) - lc_abort(); + ERTS_INTERNAL_ERROR("Lock checker failed to allocate memory!"); l_lcks->thread_name = strdup(thread_name ? thread_name : "unknown"); if (!l_lcks->thread_name) - lc_abort(); + ERTS_INTERNAL_ERROR("Lock checker failed to allocate memory!"); l_lcks->emu_thread = 0; l_lcks->tid = erts_thr_self(); @@ -691,7 +688,7 @@ erts_lc_set_thread_name(char *thread_name) free((void *) l_lcks->thread_name); l_lcks->thread_name = strdup(thread_name ? thread_name : "unknown"); if (!l_lcks->thread_name) - lc_abort(); + ERTS_INTERNAL_ERROR("strdup failed"); } l_lcks->emu_thread = 1; } @@ -1330,7 +1327,7 @@ erts_lc_init(void) #endif /* #ifdef ERTS_LC_STATIC_ALLOC */ if (ethr_spinlock_init(&free_blocks_lock) != 0) - lc_abort(); + ERTS_INTERNAL_ERROR("spinlock_init failed"); erts_tsd_key_create(&locks_key,"erts_lock_check_key"); } diff --git a/erts/emulator/beam/erl_lock_count.c b/erts/emulator/beam/erl_lock_count.c index 6f44bf097b..cf6996ea06 100644 --- a/erts/emulator/beam/erl_lock_count.c +++ b/erts/emulator/beam/erl_lock_count.c @@ -61,6 +61,25 @@ static ERTS_INLINE void lcnt_unlock(void) { ethr_mutex_unlock(&lcnt_data_lock); } +const int log2_tab64[64] = { + 63, 0, 58, 1, 59, 47, 53, 2, + 60, 39, 48, 27, 54, 33, 42, 3, + 61, 51, 37, 40, 49, 18, 28, 20, + 55, 30, 34, 11, 43, 14, 22, 4, + 62, 57, 46, 52, 38, 26, 32, 41, + 50, 36, 17, 19, 29, 10, 13, 21, + 56, 45, 25, 31, 35, 16, 9, 12, + 44, 24, 15, 8, 23, 7, 6, 5}; + +static ERTS_INLINE int lcnt_log2(Uint64 v) { + v |= v >> 1; + v |= v >> 2; + v |= v >> 4; + v |= v >> 8; + v |= v >> 16; + v |= v >> 32; + return log2_tab64[((Uint64)((v - (v >> 1))*0x07EDD5E59A4E28C2)) >> 58]; +} static char* lcnt_lock_type(Uint16 flag) { switch(flag & ERTS_LCNT_LT_ALL) { @@ -81,19 +100,20 @@ static void lcnt_clear_stats(erts_lcnt_lock_stats_t *stats) { stats->timer_n = 0; stats->file = (char *)str_undefined; stats->line = 0; + sys_memzero(stats->hist.ns, sizeof(stats->hist.ns)); } static void lcnt_time(erts_lcnt_time_t *time) { -#ifdef HAVE_GETHRTIME +#if 0 || defined(HAVE_GETHRTIME) SysHrTime hr_time; hr_time = sys_gethrtime(); time->s = (unsigned long)(hr_time / 1000000000LL); time->ns = (unsigned long)(hr_time - 1000000000LL*time->s); -#else - SysTimeval tv; - sys_gettimeofday(&tv); - time->s = tv.tv_sec; - time->ns = tv.tv_usec*1000LL; +#else + SysTimeval tv; + sys_gettimeofday(&tv); + time->s = tv.tv_sec; + time->ns = tv.tv_usec*1000LL; #endif } @@ -111,28 +131,29 @@ static void lcnt_time_diff(erts_lcnt_time_t *d, erts_lcnt_time_t *t1, erts_lcnt_ dns += 1000000000LL; } + ASSERT(ds >= 0); + d->s = ds; d->ns = dns; } -/* difference d must be positive */ +/* difference d must be non-negative */ static void lcnt_time_add(erts_lcnt_time_t *t, erts_lcnt_time_t *d) { - unsigned long ngns = 0; - t->s += d->s; t->ns += d->ns; - ngns = t->ns / 1000000000LL; + t->s += t->ns / 1000000000LL; t->ns = t->ns % 1000000000LL; - - t->s += ngns; } static erts_lcnt_thread_data_t *lcnt_thread_data_alloc(void) { erts_lcnt_thread_data_t *eltd; eltd = (erts_lcnt_thread_data_t*)malloc(sizeof(erts_lcnt_thread_data_t)); + if (!eltd) { + ERTS_INTERNAL_ERROR("Lock counter failed to allocate memory!"); + } eltd->timer_set = 0; eltd->lock_in_conflict = 0; @@ -158,59 +179,64 @@ static char* lock_opt(Uint16 flag) { return "--"; } -static void print_lock_x(erts_lcnt_lock_t *lock, Uint16 flag, char *action, char *extra) { - erts_aint_t colls, tries, w_state, r_state; - erts_lcnt_lock_stats_t *stats = NULL; - +static void print_lock_x(erts_lcnt_lock_t *lock, Uint16 flag, char *action) { + erts_aint_t w_state, r_state; char *type; - int i; - + + if (strcmp(lock->name, "run_queue") != 0) return; type = lcnt_lock_type(lock->flag); r_state = ethr_atomic_read(&lock->r_state); w_state = ethr_atomic_read(&lock->w_state); - if (lock->flag & flag) { - erts_printf("%20s [%30s] [r/w state %4ld/%4ld] id %T %s\r\n", - action, - lock->name, - r_state, - w_state, - lock->id, - extra); + erts_fprintf(stderr,"%10s [%24s] [r/w state %4ld/%4ld] %2s id %T\r\n", + action, + lock->name, + r_state, + w_state, + type, + lock->id); } } - -static void print_lock(erts_lcnt_lock_t *lock, char *action) { - if (strcmp(lock->name, "proc_main") == 0) { - print_lock_x(lock, ERTS_LCNT_LT_ALL, action, ""); - } -} - #endif static erts_lcnt_lock_stats_t *lcnt_get_lock_stats(erts_lcnt_lock_t *lock, char *file, unsigned int line) { unsigned int i; erts_lcnt_lock_stats_t *stats = NULL; - - for (i = 0; i < lock->n_stats; i++) { - if ((lock->stats[i].file == file) && (lock->stats[i].line == line)) { - return &(lock->stats[i]); - } - } - if (lock->n_stats < ERTS_LCNT_MAX_LOCK_LOCATIONS) { - stats = &lock->stats[lock->n_stats]; - lock->n_stats++; - stats->file = file; - stats->line = line; - return stats; + if (erts_lcnt_rt_options & ERTS_LCNT_OPT_LOCATION) { + for (i = 0; i < lock->n_stats; i++) { + if ((lock->stats[i].file == file) && (lock->stats[i].line == line)) { + return &(lock->stats[i]); + } + } + if (lock->n_stats < ERTS_LCNT_MAX_LOCK_LOCATIONS) { + stats = &lock->stats[lock->n_stats]; + lock->n_stats++; + stats->file = file; + stats->line = line; + return stats; + } } return &lock->stats[0]; +} +static void lcnt_update_stats_hist(erts_lcnt_hist_t *hist, erts_lcnt_time_t *time_wait) { + int idx; + unsigned long r; + + if (time_wait->s > 0 || time_wait->ns > ERTS_LCNT_HISTOGRAM_MAX_NS) { + idx = ERTS_LCNT_HISTOGRAM_SLOT_SIZE - 1; + } else { + r = time_wait->ns >> ERTS_LCNT_HISTOGRAM_RSHIFT; + if (r) idx = lcnt_log2(r); + else idx = 0; + } + hist->ns[idx]++; } -static void lcnt_update_stats(erts_lcnt_lock_stats_t *stats, int lock_in_conflict, erts_lcnt_time_t *time_wait) { +static void lcnt_update_stats(erts_lcnt_lock_stats_t *stats, int lock_in_conflict, + erts_lcnt_time_t *time_wait) { ethr_atomic_inc(&stats->tries); @@ -220,6 +246,7 @@ static void lcnt_update_stats(erts_lcnt_lock_stats_t *stats, int lock_in_conflic if (time_wait) { lcnt_time_add(&(stats->timer), time_wait); stats->timer_n++; + lcnt_update_stats_hist(&stats->hist,time_wait); } } @@ -248,6 +275,9 @@ void erts_lcnt_init() { /* init lcnt structure */ erts_lcnt_data = (erts_lcnt_data_t*)malloc(sizeof(erts_lcnt_data_t)); + if (!erts_lcnt_data) { + ERTS_INTERNAL_ERROR("Lock counter failed to allocate memory!"); + } erts_lcnt_data->current_locks = erts_lcnt_list_init(); erts_lcnt_data->deleted_locks = erts_lcnt_list_init(); @@ -269,6 +299,9 @@ erts_lcnt_lock_list_t *erts_lcnt_list_init(void) { erts_lcnt_lock_list_t *list; list = (erts_lcnt_lock_list_t*)malloc(sizeof(erts_lcnt_lock_list_t)); + if (!list) { + ERTS_INTERNAL_ERROR("Lock counter failed to allocate memory!"); + } list->head = NULL; list->tail = NULL; list->n = 0; @@ -330,8 +363,9 @@ void erts_lcnt_list_delete(erts_lcnt_lock_list_t *list, erts_lcnt_lock_t *lock) /* interface to erl_threads.h */ /* only lock on init and destroy, all others should use atomics */ void erts_lcnt_init_lock(erts_lcnt_lock_t *lock, char *name, Uint16 flag ) { - erts_lcnt_init_lock_x(lock, name, flag, am_undefined); + erts_lcnt_init_lock_x(lock, name, flag, NIL); } + void erts_lcnt_init_lock_x(erts_lcnt_lock_t *lock, char *name, Uint16 flag, Eterm id) { int i; if (!name) { @@ -360,7 +394,6 @@ void erts_lcnt_init_lock_x(erts_lcnt_lock_t *lock, char *name, Uint16 flag, Eter } erts_lcnt_list_insert(erts_lcnt_data->current_locks, lock); - lcnt_unlock(); } @@ -375,6 +408,9 @@ void erts_lcnt_destroy_lock(erts_lcnt_lock_t *lock) { /* copy structure and insert the copy */ deleted_lock = (erts_lcnt_lock_t*)malloc(sizeof(erts_lcnt_lock_t)); + if (!deleted_lock) { + ERTS_INTERNAL_ERROR("Lock counter failed to allocate memory!"); + } memcpy(deleted_lock, lock, sizeof(erts_lcnt_lock_t)); deleted_lock->next = NULL; @@ -417,8 +453,9 @@ void erts_lcnt_lock_opt(erts_lcnt_lock_t *lock, Uint16 option) { if ((w_state > 0) || (r_state > 0)) { eltd->lock_in_conflict = 1; - if (eltd->timer_set == 0) + if (eltd->timer_set == 0) { lcnt_time(&eltd->timer); + } eltd->timer_set++; } else { eltd->lock_in_conflict = 0; @@ -433,7 +470,7 @@ void erts_lcnt_lock(erts_lcnt_lock_t *lock) { if (!ERTS_LCNT_LOCK_TYPE(lock)) return; w_state = ethr_atomic_read(&lock->w_state); - ethr_atomic_inc( &lock->w_state); + ethr_atomic_inc(&lock->w_state); eltd = lcnt_get_thread_data(); @@ -446,10 +483,10 @@ void erts_lcnt_lock(erts_lcnt_lock_t *lock) { * 'atomicly'. All other locks will block the thread if w_state > 0 * i.e. locked. */ - if (eltd->timer_set == 0) + if (eltd->timer_set == 0) { lcnt_time(&eltd->timer); + } eltd->timer_set++; - } else { eltd->lock_in_conflict = 0; } @@ -459,11 +496,10 @@ void erts_lcnt_lock(erts_lcnt_lock_t *lock) { void erts_lcnt_lock_unaquire(erts_lcnt_lock_t *lock) { /* should check if this thread was "waiting" */ - if (erts_lcnt_rt_options & ERTS_LCNT_OPT_SUSPEND) return; if (!ERTS_LCNT_LOCK_TYPE(lock)) return; - ethr_atomic_dec( &lock->w_state); + ethr_atomic_dec(&lock->w_state); } /* erts_lcnt_lock_post @@ -491,7 +527,7 @@ void erts_lcnt_lock_post_x(erts_lcnt_lock_t *lock, char *file, unsigned int line if (!(lock->flag & (ERTS_LCNT_LT_RWMUTEX | ERTS_LCNT_LT_RWSPINLOCK))) { flowstate = ethr_atomic_read(&lock->flowstate); ASSERT(flowstate == 0); - ethr_atomic_inc( &lock->flowstate); + ethr_atomic_inc(&lock->flowstate); } #endif @@ -500,19 +536,12 @@ void erts_lcnt_lock_post_x(erts_lcnt_lock_t *lock, char *file, unsigned int line ASSERT(eltd); /* if lock was in conflict, time it */ - - if (erts_lcnt_rt_options & ERTS_LCNT_OPT_LOCATION) { - stats = lcnt_get_lock_stats(lock, file, line); - } else { - stats = &lock->stats[0]; - } - + stats = lcnt_get_lock_stats(lock, file, line); if (eltd->timer_set) { lcnt_time(&timer); lcnt_time_diff(&time_wait, &timer, &(eltd->timer)); lcnt_update_stats(stats, eltd->lock_in_conflict, &time_wait); - eltd->timer_set--; ASSERT(eltd->timer_set >= 0); } else { @@ -541,11 +570,11 @@ void erts_lcnt_unlock(erts_lcnt_lock_t *lock) { /* flowstate */ flowstate = ethr_atomic_read(&lock->flowstate); ASSERT(flowstate == 1); - ethr_atomic_dec( &lock->flowstate); + ethr_atomic_dec(&lock->flowstate); /* write state */ w_state = ethr_atomic_read(&lock->w_state); - ASSERT(w_state > 0) + ASSERT(w_state > 0); #endif ethr_atomic_dec(&lock->w_state); } @@ -582,9 +611,7 @@ void erts_lcnt_trylock(erts_lcnt_lock_t *lock, int res) { ethr_atomic_inc( &lock->flowstate); #endif ethr_atomic_inc(&lock->w_state); - lcnt_update_stats(&(lock->stats[0]), 0, NULL); - } else { ethr_atomic_inc(&lock->stats[0].tries); ethr_atomic_inc(&lock->stats[0].colls); diff --git a/erts/emulator/beam/erl_lock_count.h b/erts/emulator/beam/erl_lock_count.h index 75f7cd028b..ffbb93da1b 100644 --- a/erts/emulator/beam/erl_lock_count.h +++ b/erts/emulator/beam/erl_lock_count.h @@ -35,6 +35,10 @@ * | | | - collisions (including trylock busy) * | | | - timer (time spent in waiting for lock) * | | | - n_timer (collisions excluding trylock busy) + * | | | - histogram + * | | | | - # 0 = log2(lock wait_time ns) + * | | | | - ... + * | | | | - # n = log2(lock wait_time ns) * * Each instance of a lock is the unique lock, i.e. set and id in that set. * For each lock there is a set of statistics with where and what impact @@ -68,8 +72,17 @@ #include "ethread.h" +#define ERTS_LCNT_MAX_LOCK_LOCATIONS (10) -#define ERTS_LCNT_MAX_LOCK_LOCATIONS (10) +/* histogram */ +#define ERTS_LCNT_HISTOGRAM_MAX_NS (((unsigned long)1LL << 28) - 1) +#if 0 || defined(HAVE_GETHRTIME) +#define ERTS_LCNT_HISTOGRAM_SLOT_SIZE (30) +#define ERTS_LCNT_HISTOGRAM_RSHIFT (0) +#else +#define ERTS_LCNT_HISTOGRAM_SLOT_SIZE (20) +#define ERTS_LCNT_HISTOGRAM_RSHIFT (10) +#endif #define ERTS_LCNT_LT_SPINLOCK (((Uint16) 1) << 0) #define ERTS_LCNT_LT_RWSPINLOCK (((Uint16) 1) << 1) @@ -104,6 +117,10 @@ typedef struct { extern erts_lcnt_time_t timer_start; +typedef struct { + Uint32 ns[ERTS_LCNT_HISTOGRAM_SLOT_SIZE]; /* log2 array of nano seconds occurences */ +} erts_lcnt_hist_t; + typedef struct erts_lcnt_lock_stats_s { /* "tries" and "colls" needs to be atomic since * trylock busy does not aquire a lock and there @@ -118,6 +135,7 @@ typedef struct erts_lcnt_lock_stats_s { unsigned long timer_n; /* #times waited for lock */ erts_lcnt_time_t timer; /* total wait time for lock */ + erts_lcnt_hist_t hist; } erts_lcnt_lock_stats_t; /* rw locks uses both states, other locks only uses w_state */ diff --git a/erts/emulator/beam/erl_map.c b/erts/emulator/beam/erl_map.c index fdd2d0c0f6..5e740aacdd 100644 --- a/erts/emulator/beam/erl_map.c +++ b/erts/emulator/beam/erl_map.c @@ -58,6 +58,8 @@ * - maps:size/1 * - maps:without/2 * + * DEBUG: for sharing calculation + * - erts_internal:map_to_tuple_keys/1 */ /* erlang:map_size/1 @@ -819,3 +821,17 @@ int erts_validate_and_sort_map(map_t* mp) } return 1; } + +/* + * erts_internal:map_to_tuple_keys/1 + * + * Used in erts_debug:size/1 + */ + +BIF_RETTYPE erts_internal_map_to_tuple_keys_1(BIF_ALIST_1) { + if (is_map(BIF_ARG_1)) { + map_t *mp = (map_t*)map_val(BIF_ARG_1); + BIF_RET(mp->keys); + } + BIF_ERROR(BIF_P, BADARG); +} diff --git a/erts/emulator/beam/erl_message.c b/erts/emulator/beam/erl_message.c index 0eb8117980..8870fac7d9 100644 --- a/erts/emulator/beam/erl_message.c +++ b/erts/emulator/beam/erl_message.c @@ -415,7 +415,13 @@ erts_queue_dist_message(Process *rcvr, if (!(*rcvr_locks & ERTS_PROC_LOCK_MSGQ)) erts_smp_proc_unlock(rcvr, ERTS_PROC_LOCK_MSGQ); - erts_proc_notify_new_message(rcvr); + erts_proc_notify_new_message(rcvr, +#ifdef ERTS_SMP + *rcvr_locks +#else + 0 +#endif + ); } } @@ -542,7 +548,13 @@ queue_message(Process *c_p, if (locked_msgq) erts_smp_proc_unlock(receiver, ERTS_PROC_LOCK_MSGQ); - erts_proc_notify_new_message(receiver); + erts_proc_notify_new_message(receiver, +#ifdef ERTS_SMP + *receiver_locks +#else + 0 +#endif + ); #ifndef ERTS_SMP ERTS_HOLE_CHECK(receiver); @@ -1032,7 +1044,6 @@ erts_send_message(Process* sender, } BM_SWAP_TIMER(send,system); } else { -#ifdef ERTS_SMP ErlOffHeap *ohp; Eterm *hp; erts_aint32_t state; @@ -1064,42 +1075,6 @@ erts_send_message(Process* sender, #endif ); BM_SWAP_TIMER(send,system); -#else - ErlMessage* mp = message_alloc(); - Eterm *hp; - BM_SWAP_TIMER(send,size); - msize = size_object(message); - BM_SWAP_TIMER(size,send); - - if (receiver->stop - receiver->htop <= msize) { - BM_SWAP_TIMER(send,system); - erts_garbage_collect(receiver, msize, receiver->arg_reg, receiver->arity); - BM_SWAP_TIMER(system,send); - } - hp = receiver->htop; - receiver->htop = hp + msize; - BM_SWAP_TIMER(send,copy); - message = copy_struct(message, msize, &hp, &receiver->off_heap); - BM_MESSAGE_COPIED(msize); - BM_SWAP_TIMER(copy,send); - DTRACE6(message_send, sender_name, receiver_name, - (uint32_t)msize, tok_label, tok_lastcnt, tok_serial); - ERL_MESSAGE_TERM(mp) = message; - ERL_MESSAGE_TOKEN(mp) = NIL; -#ifdef USE_VM_PROBES - ERL_MESSAGE_DT_UTAG(mp) = NIL; -#endif - mp->next = NULL; - mp->data.attached = NULL; - LINK_MESSAGE(receiver, mp); - res = receiver->msg.len; - erts_proc_notify_new_message(receiver); - - if (IS_TRACED_FL(receiver, F_TRACE_RECEIVE)) { - trace_receive(receiver, message); - } - BM_SWAP_TIMER(send,system); -#endif /* #ifndef ERTS_SMP */ } return res; } diff --git a/erts/emulator/beam/erl_nif.c b/erts/emulator/beam/erl_nif.c index 063dba056e..ede5f335dc 100644 --- a/erts/emulator/beam/erl_nif.c +++ b/erts/emulator/beam/erl_nif.c @@ -472,6 +472,18 @@ int enif_inspect_binary(ErlNifEnv* env, Eterm bin_term, ErlNifBinary* bin) struct enif_tmp_obj_t* tmp; byte* raw_ptr; }u; + + if (is_boxed(bin_term) && *binary_val(bin_term) == HEADER_SUB_BIN) { + ErlSubBin* sb = (ErlSubBin*) binary_val(bin_term); + if (sb->is_writable) { + ProcBin* pb = (ProcBin*) binary_val(sb->orig); + ASSERT(pb->thing_word == HEADER_PROC_BIN); + if (pb->flags) { + erts_emasculate_writable_binary(pb); + sb->is_writable = 0; + } + } + } u.tmp = NULL; bin->data = erts_get_aligned_binary_bytes_extra(bin_term, &u.raw_ptr, allocator, sizeof(struct enif_tmp_obj_t)); @@ -1513,59 +1525,263 @@ int enif_consume_timeslice(ErlNifEnv* env, int percent) return ERTS_BIF_REDS_LEFT(env->proc) == 0; } -#ifdef ERTS_DIRTY_SCHEDULERS +/* + * NIF exports need a few more items than the Export struct provides, + * including the erl_module_nif* and a NIF function pointer, so the + * NifExport below adds those. The Export member must be first in the + * struct. The saved_mfa, saved_argc, nif_level, alloced_argv_sz and argv + * members are used to track the MFA and arguments of the top NIF in case a + * chain of one or more enif_schedule_nif() calls results in an exception, + * since in that case the original MFA and registers have to be restored + * before returning to Erlang to ensure stacktrace information associated + * with the exception is correct. + */ +typedef ERL_NIF_TERM (*NativeFunPtr)(ErlNifEnv*, int, const ERL_NIF_TERM[]); + +typedef struct { + Export exp; + struct erl_module_nif* m; + NativeFunPtr fp; + Eterm saved_mfa[3]; + int saved_argc; + int alloced_argv_sz; + Eterm argv[1]; +} NifExport; -static void -alloc_proc_psd(Process* proc, Export **ep) +/* + * If a process has saved arguments, they need to be part of the GC + * rootset. The function below is called from setup_rootset() in + * erl_gc.c. This function is declared in erl_process.h. + */ +int +erts_setup_nif_gc(Process* proc, Eterm** objv, int* nobj) { + NifExport* ep = (NifExport*) ERTS_PROC_GET_NIF_TRAP_EXPORT(proc); + int gc = (ep && ep->saved_argc > 0); + + if (gc) { + *objv = ep->argv; + *nobj = ep->saved_argc; + } + return gc; +} + +/* + * Allocate a NifExport and set it in proc specific data + */ +static NifExport* +allocate_nif_sched_data(Process* proc, int argc) +{ + NifExport* ep; + size_t argv_extra, total; int i; - if (!*ep) { - *ep = erts_alloc(ERTS_ALC_T_PSD, sizeof(Export)); - sys_memset((void*) *ep, 0, sizeof(Export)); - for (i=0; i<ERTS_NUM_CODE_IX; i++) { - (*ep)->addressv[i] = &(*ep)->code[3]; - } - (*ep)->code[3] = (BeamInstr) em_call_nif; + + argv_extra = argc > 1 ? sizeof(Eterm)*(argc-1) : 0; + total = sizeof(NifExport) + argv_extra; + ep = erts_alloc(ERTS_ALC_T_NIF_TRAP_EXPORT, total); + sys_memset((void*) ep, 0, total); + ep->alloced_argv_sz = argc; + for (i=0; i<ERTS_NUM_CODE_IX; i++) { + ep->exp.addressv[i] = &ep->exp.code[3]; } - (void) ERTS_PROC_SET_DIRTY_SCHED_TRAP_EXPORT(proc, ERTS_PROC_LOCK_MAIN, *ep); + ep->exp.code[3] = (BeamInstr) em_call_nif; + (void) ERTS_PROC_SET_NIF_TRAP_EXPORT(proc, ERTS_PROC_LOCK_MAIN, ep); + return ep; +} + +static ERTS_INLINE void +destroy_nif_export(NifExport *nif_export) +{ + erts_free(ERTS_ALC_T_NIF_TRAP_EXPORT, (void *) nif_export); } +void +erts_destroy_nif_export(void *nif_export) +{ + destroy_nif_export((NifExport *) nif_export); +} + +/* + * Initialize a NifExport struct. Create it if needed and store it in the + * proc. The direct_fp function is what will be invoked by op_call_nif, and + * the indirect_fp function, if not NULL, is what the direct_fp function + * will call. If the allocated NifExport isn't enough to hold all of argv, + * allocate a larger one. Save MFA and registers only if the need_save + * parameter is true. + */ static ERL_NIF_TERM -execute_dirty_nif_finalizer(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) +init_nif_sched_data(ErlNifEnv* env, NativeFunPtr direct_fp, NativeFunPtr indirect_fp, + int need_save, int argc, const ERL_NIF_TERM argv[]) { - Eterm* reg = ERTS_PROC_GET_SCHDATA(env->proc)->x_reg_array; - ERL_NIF_TERM result = (ERL_NIF_TERM) reg[0]; - typedef ERL_NIF_TERM (*FinalizerFP)(ErlNifEnv*, ERL_NIF_TERM); - FinalizerFP fp; -#if HAVE_INT64 && SIZEOF_LONG != 8 - ASSERT(sizeof(fp) <= sizeof(ErlNifUInt64)); - enif_get_uint64(env, reg[1], (ErlNifUInt64 *) &fp); -#else - ASSERT(sizeof(fp) <= sizeof(unsigned long)); - enif_get_ulong(env, reg[1], (unsigned long *) &fp); -#endif - return (*fp)(env, result); + Process* proc = env->proc; + Eterm* reg = ERTS_PROC_GET_SCHDATA(proc)->x_reg_array; + NifExport* ep; + int i; + + ep = (NifExport*) ERTS_PROC_GET_NIF_TRAP_EXPORT(proc); + if (!ep) + ep = allocate_nif_sched_data(proc, argc); + else if (need_save && ep->alloced_argv_sz < argc) { + NifExport* new_ep = allocate_nif_sched_data(proc, argc); + destroy_nif_export(ep); + ep = new_ep; + } + ERTS_VBUMP_ALL_REDS(proc); + for (i = 0; i < argc; i++) { + if (need_save) + ep->argv[i] = reg[i]; + reg[i] = (Eterm) argv[i]; + } + if (need_save) { + ep->saved_mfa[0] = proc->current[0]; + ep->saved_mfa[1] = proc->current[1]; + ep->saved_mfa[2] = proc->current[2]; + ep->saved_argc = argc; + } + proc->i = (BeamInstr*) ep->exp.addressv[0]; + ep->exp.code[0] = (BeamInstr) proc->current[0]; + ep->exp.code[1] = (BeamInstr) proc->current[1]; + ep->exp.code[2] = argc; + ep->exp.code[4] = (BeamInstr) direct_fp; + ep->m = env->mod_nif; + ep->fp = indirect_fp; + proc->freason = TRAP; + return THE_NON_VALUE; } -#endif /* ERTS_DIRTY_SCHEDULERS */ +/* + * Restore saved MFA and registers. Registers are restored only when the + * exception flag is true. + */ +static void +restore_nif_mfa(Process* proc, NifExport* ep, int exception) +{ + int i; + Eterm* reg = ERTS_PROC_GET_SCHDATA(proc)->x_reg_array; -#ifdef ERL_NIF_DIRTY_SCHEDULER_SUPPORT + proc->current[0] = ep->saved_mfa[0]; + proc->current[1] = ep->saved_mfa[1]; + proc->current[2] = ep->saved_mfa[2]; + if (exception) + for (i = 0; i < ep->saved_argc; i++) + reg[i] = ep->argv[i]; + ep->saved_argc = 0; + ep->saved_mfa[0] = THE_NON_VALUE; +} -ERL_NIF_TERM -enif_schedule_dirty_nif(ErlNifEnv* env, int flags, - ERL_NIF_TERM (*fp)(ErlNifEnv*, int, const ERL_NIF_TERM[]), - int argc, const ERL_NIF_TERM argv[]) +#ifdef ERTS_DIRTY_SCHEDULERS + +/* + * Finalize a dirty NIF call. This function is scheduled to cause the VM to + * switch the process off a dirty scheduler thread and back onto a regular + * scheduler thread, and then return the result from the dirty NIF. It also + * restores the original NIF MFA when necessary based on the value of + * ep->fp set by execute_dirty_nif via init_nif_sched_data -- non-NULL + * means restore, NULL means do not restore. + */ +static ERL_NIF_TERM +dirty_nif_finalizer(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) +{ + Process* proc = env->proc; + NifExport* ep; + + ASSERT(argc == 1); + ASSERT(!ERTS_SCHEDULER_IS_DIRTY(env->proc->scheduler_data)); + ep = (NifExport*) ERTS_PROC_GET_NIF_TRAP_EXPORT(proc); + ASSERT(ep); + if (ep->fp) + restore_nif_mfa(proc, ep, 0); + return argv[0]; +} + +/* Finalize a dirty NIF call that raised an exception. Otherwise same as + * the dirty_nif_finalizer() function. + */ +static ERL_NIF_TERM +dirty_nif_exception(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) +{ + Process* proc = env->proc; + NifExport* ep; + + ASSERT(!ERTS_SCHEDULER_IS_DIRTY(env->proc->scheduler_data)); + ep = (NifExport*) ERTS_PROC_GET_NIF_TRAP_EXPORT(proc); + ASSERT(ep); + if (ep->fp) + restore_nif_mfa(proc, ep, 1); + return enif_make_badarg(env); +} + +/* + * Dirty NIF execution wrapper function. Invoke an application's dirty NIF, + * then check the result and schedule the appropriate finalizer function + * where needed. Also restore the original NIF MFA when appropriate. + */ +static ERL_NIF_TERM +execute_dirty_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) +{ + Process* proc = env->proc; + NativeFunPtr fp = (NativeFunPtr) proc->current[6]; + NifExport* ep; + ERL_NIF_TERM result; + + ASSERT(ERTS_SCHEDULER_IS_DIRTY(env->proc->scheduler_data)); + + /* + * Set ep->fp to NULL before the native call so we know later whether it scheduled another NIF for execution + */ + ep = (NifExport*) ERTS_PROC_GET_NIF_TRAP_EXPORT(proc); + ASSERT(ep); + ep->fp = NULL; + result = (*fp)(env, argc, argv); + erts_smp_atomic32_read_band_mb(&proc->state, + ~(ERTS_PSFLG_DIRTY_CPU_PROC + |ERTS_PSFLG_DIRTY_IO_PROC + |ERTS_PSFLG_DIRTY_CPU_PROC_IN_Q + |ERTS_PSFLG_DIRTY_IO_PROC_IN_Q)); + if (erts_refc_dectest(&env->mod_nif->rt_dtor_cnt, 0) == 0 && env->mod_nif->mod == NULL) + close_lib(env->mod_nif); + /* + * If no more NIFs were scheduled by the native call via + * enif_schedule_nif(), then ep->fp will still be NULL as set above, in + * which case we need to restore the original NIF calling + * context. Reuse fp essentially as a boolean for this, passing it to + * init_nif_sched_data below. Both dirty_nif_exception and + * dirty_nif_finalizer then check ep->fp to decide whether or not to + * restore the original calling context. + */ + ep = (NifExport*) ERTS_PROC_GET_NIF_TRAP_EXPORT(proc); + ASSERT(ep); + if (ep->fp) + fp = NULL; + if (is_non_value(result)) { + if (proc->freason != TRAP) { + ASSERT(proc->freason == BADARG); + return init_nif_sched_data(env, dirty_nif_exception, fp, 0, argc, argv); + } else { + if (ep->fp == NULL) + restore_nif_mfa(proc, ep, 1); + return result; + } + } + else + return init_nif_sched_data(env, dirty_nif_finalizer, fp, 0, 1, &result); +} + +/* + * Dirty NIF scheduling wrapper function. Schedule a dirty NIF to execute + * via the execute_dirty_nif() wrapper function. The dirty scheduler thread + * type (CPU or I/O) is indicated in flags parameter. + */ +static ERTS_INLINE ERL_NIF_TERM +schedule_dirty_nif(ErlNifEnv* env, int flags, int argc, const ERL_NIF_TERM argv[]) { -#ifdef USE_THREADS erts_aint32_t state, n, a; Process* proc = env->proc; - Eterm* reg = ERTS_PROC_GET_SCHDATA(proc)->x_reg_array; - Export* ep = NULL; - int i; + NativeFunPtr fp = (NativeFunPtr) proc->current[6]; + NifExport* ep; + int need_save; - int chkflgs = (flags & (ERL_NIF_DIRTY_JOB_IO_BOUND|ERL_NIF_DIRTY_JOB_CPU_BOUND)); - if (chkflgs != ERL_NIF_DIRTY_JOB_IO_BOUND && chkflgs != ERL_NIF_DIRTY_JOB_CPU_BOUND) - return enif_make_badarg(env); + ASSERT(flags==ERL_NIF_DIRTY_JOB_IO_BOUND || flags==ERL_NIF_DIRTY_JOB_CPU_BOUND); a = erts_smp_atomic32_read_acqb(&proc->state); while (1) { @@ -1577,7 +1793,7 @@ enif_schedule_dirty_nif(ErlNifEnv* env, int flags, */ n &= ~(ERTS_PSFLG_DIRTY_CPU_PROC|ERTS_PSFLG_DIRTY_IO_PROC |ERTS_PSFLG_DIRTY_CPU_PROC_IN_Q|ERTS_PSFLG_DIRTY_IO_PROC_IN_Q); - if (chkflgs == ERL_NIF_DIRTY_JOB_CPU_BOUND) + if (flags == ERL_NIF_DIRTY_JOB_CPU_BOUND) n |= ERTS_PSFLG_DIRTY_CPU_PROC; else n |= ERTS_PSFLG_DIRTY_IO_PROC; @@ -1585,82 +1801,106 @@ enif_schedule_dirty_nif(ErlNifEnv* env, int flags, if (a == state) break; } - if (!(ep = ERTS_PROC_GET_DIRTY_SCHED_TRAP_EXPORT(proc))) - alloc_proc_psd(proc, &ep); - ERTS_VBUMP_ALL_REDS(proc); - ep->code[2] = argc; - for (i = 0; i < argc; i++) { - reg[i] = (Eterm) argv[i]; - } - proc->i = (BeamInstr*) ep->addressv[0]; - ep->code[4] = (BeamInstr) fp; - proc->freason = TRAP; + erts_refc_inc(&env->mod_nif->rt_dtor_cnt, 1); + ep = (NifExport*) ERTS_PROC_GET_NIF_TRAP_EXPORT(proc); + need_save = (ep == NULL || is_non_value(ep->saved_mfa[0])); + return init_nif_sched_data(env, execute_dirty_nif, fp, need_save, argc, argv); +} - return THE_NON_VALUE; -#else - return (*fp)(env, argc, argv); -#endif +static ERL_NIF_TERM +schedule_dirty_io_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) +{ + return schedule_dirty_nif(env, ERL_NIF_DIRTY_JOB_IO_BOUND, argc, argv); +} + +static ERL_NIF_TERM +schedule_dirty_cpu_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) +{ + return schedule_dirty_nif(env, ERL_NIF_DIRTY_JOB_CPU_BOUND, argc, argv); +} + +#endif /* ERTS_DIRTY_SCHEDULERS */ + +/* + * NIF execution wrapper used by enif_schedule_nif() for regular NIFs. It + * calls the actual NIF, restores original NIF MFA if necessary, and + * then returns the NIF result. + */ +static ERL_NIF_TERM +execute_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) +{ + Process* proc = env->proc; + NativeFunPtr fp = (NativeFunPtr) proc->current[6]; + NifExport* ep; + ERL_NIF_TERM result; + + ep = (NifExport*) ERTS_PROC_GET_NIF_TRAP_EXPORT(proc); + ASSERT(ep); + ep->fp = NULL; + result = (*fp)(env, argc, argv); + ep = (NifExport*) ERTS_PROC_GET_NIF_TRAP_EXPORT(proc); + ASSERT(ep); + /* + * If no NIFs were scheduled by the native call via + * enif_schedule_nif(), then ep->fp will still be NULL as set above, in + * which case we need to restore the original NIF MFA. + */ + if (ep->fp == NULL) + restore_nif_mfa(proc, ep, is_non_value(result) && proc->freason != TRAP); + return result; } ERL_NIF_TERM -enif_schedule_dirty_nif_finalizer(ErlNifEnv* env, ERL_NIF_TERM result, - ERL_NIF_TERM (*fp)(ErlNifEnv*, ERL_NIF_TERM)) +enif_schedule_nif(ErlNifEnv* env, const char* fun_name, int flags, + ERL_NIF_TERM (*fp)(ErlNifEnv*, int, const ERL_NIF_TERM[]), + int argc, const ERL_NIF_TERM argv[]) { -#ifdef USE_THREADS Process* proc = env->proc; - Eterm* reg = ERTS_PROC_GET_SCHDATA(proc)->x_reg_array; - Export* ep; + NifExport* ep; + ERL_NIF_TERM fun_name_atom, result; + int need_save; - erts_smp_atomic32_read_band_mb(&proc->state, - ~(ERTS_PSFLG_DIRTY_CPU_PROC - |ERTS_PSFLG_DIRTY_IO_PROC - |ERTS_PSFLG_DIRTY_CPU_PROC_IN_Q - |ERTS_PSFLG_DIRTY_IO_PROC_IN_Q)); - if (!(ep = ERTS_PROC_GET_DIRTY_SCHED_TRAP_EXPORT(proc))) - alloc_proc_psd(proc, &ep); - ERTS_VBUMP_ALL_REDS(proc); - ep->code[2] = 2; - reg[0] = (Eterm) result; -#if HAVE_INT64 && SIZEOF_LONG != 8 - ASSERT(sizeof(fp) <= sizeof(ErlNifUInt64)); - reg[1] = (Eterm) enif_make_uint64(env, (ErlNifUInt64) fp); -#else - ASSERT(sizeof(fp) <= sizeof(unsigned long)); - reg[1] = (Eterm) enif_make_ulong(env, (unsigned long) fp); -#endif - proc->i = (BeamInstr*) ep->addressv[0]; - ep->code[4] = (BeamInstr) execute_dirty_nif_finalizer; - proc->freason = TRAP; + if (argc > MAX_ARG) + return enif_make_badarg(env); + fun_name_atom = enif_make_atom(env, fun_name); + if (enif_is_exception(env, fun_name_atom)) + return fun_name_atom; - return THE_NON_VALUE; + ep = (NifExport*) ERTS_PROC_GET_NIF_TRAP_EXPORT(proc); + need_save = (ep == NULL || is_non_value(ep->saved_mfa[0])); + + if (flags) { +#ifdef ERTS_DIRTY_SCHEDULERS + NativeFunPtr sched_fun; + int chkflgs = (flags & (ERL_NIF_DIRTY_JOB_IO_BOUND|ERL_NIF_DIRTY_JOB_CPU_BOUND)); + if (chkflgs == ERL_NIF_DIRTY_JOB_IO_BOUND) + sched_fun = schedule_dirty_io_nif; + else if (chkflgs == ERL_NIF_DIRTY_JOB_CPU_BOUND) + sched_fun = schedule_dirty_cpu_nif; + else + return enif_make_badarg(env); + result = init_nif_sched_data(env, sched_fun, fp, need_save, argc, argv); #else - return (*fp)(env, result); + return enif_make_badarg(env); #endif -} + } + else + result = init_nif_sched_data(env, execute_nif, fp, need_save, argc, argv); -/* A simple finalizer that just returns its result argument */ -ERL_NIF_TERM -enif_dirty_nif_finalizer(ErlNifEnv* env, ERL_NIF_TERM result) -{ + ep = (NifExport*) ERTS_PROC_GET_NIF_TRAP_EXPORT(proc); + ASSERT(ep); + ep->exp.code[1] = (BeamInstr) fun_name_atom; return result; } +#ifdef ERL_NIF_DIRTY_SCHEDULER_SUPPORT + int enif_is_on_dirty_scheduler(ErlNifEnv* env) { return ERTS_SCHEDULER_IS_DIRTY(env->proc->scheduler_data); } -int -enif_have_dirty_schedulers() -{ -#ifdef USE_THREADS - return 1; -#else - return 0; -#endif -} - #endif /* ERL_NIF_DIRTY_SCHEDULER_SUPPORT */ /* Maps */ @@ -1961,6 +2201,35 @@ static Eterm load_nif_error(Process* p, const char* atom, const char* format, .. return ret; } +/* + * The function below is for looping through ErlNifFunc arrays, helping + * provide backwards compatibility across the version 2.7 change that added + * the "flags" field to ErlNifFunc. + */ +static ErlNifFunc* next_func(ErlNifEntry* entry, int* incrp, ErlNifFunc* func) +{ + ASSERT(incrp); + if (!*incrp) { + if (entry->major > 2 || (entry->major == 2 && entry->minor >= 7)) + *incrp = sizeof(ErlNifFunc); + else { + /* + * ErlNifFuncV1 below is what ErlNifFunc was before the + * addition of the flags field for 2.7, and is needed to handle + * backward compatibility. + */ + typedef struct { + const char* name; + unsigned arity; + ERL_NIF_TERM (*fptr)(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); + }ErlNifFuncV1; + *incrp = sizeof(ErlNifFuncV1); + } + } + return (ErlNifFunc*) ((char*)func + *incrp); +} + + BIF_RETTYPE load_nif_2(BIF_ALIST_2) { static const char bad_lib[] = "bad_lib"; @@ -2070,22 +2339,48 @@ BIF_RETTYPE load_nif_2(BIF_ALIST_2) } else { /*erts_fprintf(stderr, "Found module %T\r\n", mod_atom);*/ - + + int maybe_dirty_nifs = ((entry->major > 2 || (entry->major == 2 && entry->minor >= 7)) + && (entry->options & ERL_NIF_DIRTY_NIF_OPTION)); + int incr = 0; + ErlNifFunc* f = entry->funcs; for (i=0; i < entry->num_of_funcs && ret==am_ok; i++) { BeamInstr** code_pp; - ErlNifFunc* f = &entry->funcs[i]; if (!erts_atom_get(f->name, sys_strlen(f->name), &f_atom, ERTS_ATOM_ENC_LATIN1) || (code_pp = get_func_pp(mod->curr.code, f_atom, f->arity))==NULL) { ret = load_nif_error(BIF_P,bad_lib,"Function not found %T:%s/%u", mod_atom, f->name, f->arity); - } - else if (code_pp[1] - code_pp[0] < (5+3)) { + } + else if (maybe_dirty_nifs && f->flags) { + /* + * If the flags field is non-zero and this emulator was + * built with dirty scheduler support, check that the flags + * value is legal. But if this emulator was built without + * dirty scheduler support, treat a non-zero flags field as + * a load error. + */ +#ifdef ERTS_DIRTY_SCHEDULERS + if (f->flags != ERL_NIF_DIRTY_JOB_IO_BOUND && f->flags != ERL_NIF_DIRTY_JOB_CPU_BOUND) + ret = load_nif_error(BIF_P, bad_lib, "Illegal flags field value %d for NIF %T:%s/%u", + f->flags, mod_atom, f->name, f->arity); +#else + ret = load_nif_error(BIF_P, bad_lib, "NIF %T:%s/%u requires a runtime with dirty scheduler support.", + mod_atom, f->name, f->arity); +#endif + } +#ifdef ERTS_DIRTY_SCHEDULERS + else if (code_pp[1] - code_pp[0] < (5+4)) +#else + else if (code_pp[1] - code_pp[0] < (5+3)) +#endif + { ret = load_nif_error(BIF_P,bad_lib,"No explicit call to load_nif" - " in module (%T:%s/%u to small)", - mod_atom, entry->funcs[i].name, entry->funcs[i].arity); + " in module (%T:%s/%u too small)", + mod_atom, f->name, f->arity); } /*erts_fprintf(stderr, "Found NIF %T:%s/%u\r\n", - mod_atom, entry->funcs[i].name, entry->funcs[i].arity);*/ + mod_atom, f->name, f->arity);*/ + f = next_func(entry, &incr, f); } } @@ -2111,7 +2406,8 @@ BIF_RETTYPE load_nif_2(BIF_ALIST_2) * is deprecated and was only ment as a development feature not to * be used in production systems. (See warning below) */ - int k; + int k, old_incr = 0; + ErlNifFunc* old_func; lib->priv_data = mod->curr.nif->priv_data; ASSERT(mod->curr.nif->entry != NULL); @@ -2120,13 +2416,16 @@ BIF_RETTYPE load_nif_2(BIF_ALIST_2) goto error; } /* Check that no NIF is removed */ + old_func = mod->curr.nif->entry->funcs; for (k=0; k < mod->curr.nif->entry->num_of_funcs; k++) { - ErlNifFunc* old_func = &mod->curr.nif->entry->funcs[k]; + int incr = 0; + ErlNifFunc* f = entry->funcs; for (i=0; i < entry->num_of_funcs; i++) { - if (old_func->arity == entry->funcs[i].arity - && sys_strcmp(old_func->name, entry->funcs[i].name) == 0) { + if (old_func->arity == f->arity + && sys_strcmp(old_func->name, f->name) == 0) { break; } + f = next_func(entry, &incr, f); } if (i == entry->num_of_funcs) { ret = load_nif_error(BIF_P,reload,"Reloaded library missing " @@ -2134,7 +2433,8 @@ BIF_RETTYPE load_nif_2(BIF_ALIST_2) old_func->name, old_func->arity); goto error; } - } + old_func = next_func(mod->curr.nif->entry, &old_incr, old_func); + } erts_pre_nif(&env, BIF_P, lib); veto = entry->reload(&env, &lib->priv_data, BIF_ARG_2); erts_post_nif(&env); @@ -2181,13 +2481,17 @@ BIF_RETTYPE load_nif_2(BIF_ALIST_2) /* ** Everything ok, patch the beam code with op_call_nif */ - mod->curr.nif = lib; + + int incr = 0; + ErlNifFunc* f = entry->funcs; + + mod->curr.nif = lib; for (i=0; i < entry->num_of_funcs; i++) { BeamInstr* code_ptr; - erts_atom_get(entry->funcs[i].name, sys_strlen(entry->funcs[i].name), &f_atom, ERTS_ATOM_ENC_LATIN1); - code_ptr = *get_func_pp(mod->curr.code, f_atom, entry->funcs[i].arity); - + erts_atom_get(f->name, sys_strlen(f->name), &f_atom, ERTS_ATOM_ENC_LATIN1); + code_ptr = *get_func_pp(mod->curr.code, f_atom, f->arity); + if (code_ptr[1] == 0) { code_ptr[5+0] = (BeamInstr) BeamOp(op_call_nif); } @@ -2195,10 +2499,21 @@ BIF_RETTYPE load_nif_2(BIF_ALIST_2) GenericBp* g = (GenericBp *) code_ptr[1]; ASSERT(code_ptr[5+0] == (BeamInstr) BeamOp(op_i_generic_breakpoint)); - g->orig_instr = (BeamInstr) BeamOp(op_call_nif); - } - code_ptr[5+1] = (BeamInstr) entry->funcs[i].fptr; + g->orig_instr = (BeamInstr) BeamOp(op_call_nif); + } + if ((entry->major > 2 || (entry->major == 2 && entry->minor >= 7)) + && (entry->options & ERL_NIF_DIRTY_NIF_OPTION) && f->flags) { +#ifdef ERL_NIF_DIRTY_SCHEDULER_SUPPORT + code_ptr[5+3] = (BeamInstr) f->fptr; + code_ptr[5+1] = (f->flags == ERL_NIF_DIRTY_JOB_IO_BOUND) ? + (BeamInstr) schedule_dirty_io_nif : + (BeamInstr) schedule_dirty_cpu_nif; +#endif + } + else + code_ptr[5+1] = (BeamInstr) f->fptr; code_ptr[5+2] = (BeamInstr) lib; + f = next_func(entry, &incr, f); } } else { diff --git a/erts/emulator/beam/erl_nif.h b/erts/emulator/beam/erl_nif.h index 5b93c2398e..226fc199a1 100644 --- a/erts/emulator/beam/erl_nif.h +++ b/erts/emulator/beam/erl_nif.h @@ -42,9 +42,13 @@ ** 2.5: R17 Maps API additions ** 2.6: R17 with maps ** R17 dirty schedulers +** 2.7: 17.3 add enif_schedule_nif +** remove enif_schedule_dirty_nif, enif_schedule_dirty_nif_finalizer, enif_dirty_nif_finalizer +** add ErlNifEntry options +** add ErlNifFunc flags */ #define ERL_NIF_MAJOR_VERSION 2 -#define ERL_NIF_MINOR_VERSION 6 +#define ERL_NIF_MINOR_VERSION 7 /* * The emulator will refuse to load a nif-lib with a major version @@ -125,8 +129,10 @@ typedef struct const char* name; unsigned arity; ERL_NIF_TERM (*fptr)(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); + unsigned flags; }ErlNifFunc; + typedef struct enif_entry_t { int major; @@ -139,8 +145,11 @@ typedef struct enif_entry_t int (*upgrade)(ErlNifEnv*, void** priv_data, void** old_priv_data, ERL_NIF_TERM load_info); void (*unload) (ErlNifEnv*, void* priv_data); const char* vm_variant; + unsigned options; }ErlNifEntry; +/* Field bits for ErlNifEntry options */ +#define ERL_NIF_DIRTY_NIF_OPTION 1 typedef struct @@ -232,10 +241,21 @@ extern TWinDynNifCallbacks WinDynNifCallbacks; # else # define ERL_NIF_INIT_DECL(MODNAME) __declspec(dllexport) ErlNifEntry* nif_init(TWinDynNifCallbacks* callbacks) # endif -# define ERL_NIF_INIT_BODY memcpy(&WinDynNifCallbacks,callbacks,sizeof(TWinDynNifCallbacks)) +# ifdef ERL_NIF_DIRTY_SCHEDULER_SUPPORT +# define ERL_NIF_INIT_BODY do { \ + memcpy(&WinDynNifCallbacks,callbacks,sizeof(TWinDynNifCallbacks)); \ + entry.options = ERL_NIF_DIRTY_NIF_OPTION; \ + } while(0) +# else +# define ERL_NIF_INIT_BODY memcpy(&WinDynNifCallbacks,callbacks,sizeof(TWinDynNifCallbacks)) +# endif #else # define ERL_NIF_INIT_GLOB -# define ERL_NIF_INIT_BODY +# ifdef ERL_NIF_DIRTY_SCHEDULER_SUPPORT +# define ERL_NIF_INIT_BODY entry.options = ERL_NIF_DIRTY_NIF_OPTION +# else +# define ERL_NIF_INIT_BODY +# endif # ifdef STATIC_ERLANG_NIF # define ERL_NIF_INIT_DECL(MODNAME) ErlNifEntry* MODNAME ## _nif_init(void) # else diff --git a/erts/emulator/beam/erl_nif_api_funcs.h b/erts/emulator/beam/erl_nif_api_funcs.h index d7c554e60b..630cefae93 100644 --- a/erts/emulator/beam/erl_nif_api_funcs.h +++ b/erts/emulator/beam/erl_nif_api_funcs.h @@ -22,7 +22,7 @@ #endif /* -** WARNING: add new ERL_NIF_API_FUNC_DECL entries at the bottom of the list +** WARNING: Add new ERL_NIF_API_FUNC_DECL entries at the bottom of the list ** to keep compatibility on Windows!!! ** ** And don't forget to increase ERL_NIF_MINOR_VERSION in erl_nif.h @@ -141,14 +141,6 @@ ERL_NIF_API_FUNC_DECL(int,enif_is_number,(ErlNifEnv*, ERL_NIF_TERM term)); ERL_NIF_API_FUNC_DECL(void*,enif_dlopen,(const char* lib, void (*err_handler)(void*,const char*), void* err_arg)); ERL_NIF_API_FUNC_DECL(void*,enif_dlsym,(void* handle, const char* symbol, void (*err_handler)(void*,const char*), void* err_arg)); ERL_NIF_API_FUNC_DECL(int,enif_consume_timeslice,(ErlNifEnv*, int percent)); -#ifdef ERL_NIF_DIRTY_SCHEDULER_SUPPORT -ERL_NIF_API_FUNC_DECL(ERL_NIF_TERM,enif_schedule_dirty_nif,(ErlNifEnv*,int,ERL_NIF_TERM (*)(ErlNifEnv*,int,const ERL_NIF_TERM[]),int,const ERL_NIF_TERM[])); -ERL_NIF_API_FUNC_DECL(ERL_NIF_TERM,enif_schedule_dirty_nif_finalizer,(ErlNifEnv*,ERL_NIF_TERM,ERL_NIF_TERM (*)(ErlNifEnv*,ERL_NIF_TERM))); -ERL_NIF_API_FUNC_DECL(ERL_NIF_TERM,enif_dirty_nif_finalizer,(ErlNifEnv*,ERL_NIF_TERM)); -ERL_NIF_API_FUNC_DECL(int,enif_is_on_dirty_scheduler,(ErlNifEnv*)); -ERL_NIF_API_FUNC_DECL(int,enif_have_dirty_schedulers,(void)); -#endif - ERL_NIF_API_FUNC_DECL(int, enif_is_map, (ErlNifEnv* env, ERL_NIF_TERM term)); ERL_NIF_API_FUNC_DECL(int, enif_get_map_size, (ErlNifEnv* env, ERL_NIF_TERM term, size_t *size)); ERL_NIF_API_FUNC_DECL(ERL_NIF_TERM, enif_make_new_map, (ErlNifEnv* env)); @@ -163,12 +155,22 @@ ERL_NIF_API_FUNC_DECL(int, enif_map_iterator_is_tail, (ErlNifEnv *env, ErlNifMap ERL_NIF_API_FUNC_DECL(int, enif_map_iterator_next, (ErlNifEnv *env, ErlNifMapIterator *iter)); ERL_NIF_API_FUNC_DECL(int, enif_map_iterator_prev, (ErlNifEnv *env, ErlNifMapIterator *iter)); ERL_NIF_API_FUNC_DECL(int, enif_map_iterator_get_pair, (ErlNifEnv *env, ErlNifMapIterator *iter, ERL_NIF_TERM *key, ERL_NIF_TERM *value)); - +ERL_NIF_API_FUNC_DECL(ERL_NIF_TERM,enif_schedule_nif,(ErlNifEnv*,const char*,int,ERL_NIF_TERM (*)(ErlNifEnv*,int,const ERL_NIF_TERM[]),int,const ERL_NIF_TERM[])); /* -** Add new entries here to keep compatibility on Windows!!! +** ADD NEW ENTRIES HERE (before this comment) !!! */ + + +/* + * Conditional EXPERIMENTAL stuff always last. + * Must be moved up and made unconditional to support binary backward + * compatibility on Windows. + */ +#ifdef ERL_NIF_DIRTY_SCHEDULER_SUPPORT +ERL_NIF_API_FUNC_DECL(int,enif_is_on_dirty_scheduler,(ErlNifEnv*)); #endif +#endif /* ERL_NIF_API_FUNC_DECL */ /* ** Please keep the ERL_NIF_API_FUNC_MACRO list below in the same order @@ -282,21 +284,12 @@ ERL_NIF_API_FUNC_DECL(int, enif_map_iterator_get_pair, (ErlNifEnv *env, ErlNifMa # define enif_make_int64 ERL_NIF_API_FUNC_MACRO(enif_make_int64) # define enif_make_uint64 ERL_NIF_API_FUNC_MACRO(enif_make_uint64) #endif - # define enif_is_exception ERL_NIF_API_FUNC_MACRO(enif_is_exception) # define enif_make_reverse_list ERL_NIF_API_FUNC_MACRO(enif_make_reverse_list) # define enif_is_number ERL_NIF_API_FUNC_MACRO(enif_is_number) # define enif_dlopen ERL_NIF_API_FUNC_MACRO(enif_dlopen) # define enif_dlsym ERL_NIF_API_FUNC_MACRO(enif_dlsym) # define enif_consume_timeslice ERL_NIF_API_FUNC_MACRO(enif_consume_timeslice) -#ifdef ERL_NIF_DIRTY_SCHEDULER_SUPPORT -# define enif_schedule_dirty_nif ERL_NIF_API_FUNC_MACRO(enif_schedule_dirty_nif) -# define enif_schedule_dirty_nif_finalizer ERL_NIF_API_FUNC_MACRO(enif_schedule_dirty_nif_finalizer) -# define enif_dirty_nif_finalizer ERL_NIF_API_FUNC_MACRO(enif_dirty_nif_finalizer) -# define enif_is_on_dirty_scheduler ERL_NIF_API_FUNC_MACRO(enif_is_on_dirty_scheduler) -# define enif_have_dirty_schedulers ERL_NIF_API_FUNC_MACRO(enif_have_dirty_schedulers) -#endif - # define enif_is_map ERL_NIF_API_FUNC_MACRO(enif_is_map) # define enif_get_map_size ERL_NIF_API_FUNC_MACRO(enif_get_map_size) # define enif_make_new_map ERL_NIF_API_FUNC_MACRO(enif_make_new_map) @@ -311,11 +304,21 @@ ERL_NIF_API_FUNC_DECL(int, enif_map_iterator_get_pair, (ErlNifEnv *env, ErlNifMa # define enif_map_iterator_next ERL_NIF_API_FUNC_MACRO(enif_map_iterator_next) # define enif_map_iterator_prev ERL_NIF_API_FUNC_MACRO(enif_map_iterator_prev) # define enif_map_iterator_get_pair ERL_NIF_API_FUNC_MACRO(enif_map_iterator_get_pair) +# define enif_schedule_nif ERL_NIF_API_FUNC_MACRO(enif_schedule_nif) /* -** Add new entries here +** ADD NEW ENTRIES HERE (before this comment) */ + +/* + * Conditional EXPERIMENTAL stuff always last + * Must be moved up and made unconditional to support binary backward + * compatibility on Windows. + */ +#ifdef ERL_NIF_DIRTY_SCHEDULER_SUPPORT +# define enif_is_on_dirty_scheduler ERL_NIF_API_FUNC_MACRO(enif_is_on_dirty_scheduler) #endif +#endif /* ERL_NIF_API_FUNC_MACRO */ #if defined(__GNUC__) && !(defined(__WIN32__) || defined(_WIN32) || defined(_WIN32_)) diff --git a/erts/emulator/beam/erl_port_task.c b/erts/emulator/beam/erl_port_task.c index 31d9a1e26e..682f6f8f4b 100644 --- a/erts/emulator/beam/erl_port_task.c +++ b/erts/emulator/beam/erl_port_task.c @@ -68,6 +68,13 @@ static void chk_task_queues(Port *pp, ErtsPortTask *execq, int processing_busy_q #define DTRACE_DRIVER(PROBE_NAME, PP) do {} while(0) #endif +#define ERTS_SMP_LC_VERIFY_RQ(RQ, PP) \ + do { \ + ERTS_SMP_LC_ASSERT(erts_smp_lc_runq_is_locked(runq)); \ + ERTS_SMP_LC_ASSERT((RQ) == ((ErtsRunQueue *) \ + erts_smp_atomic_read_nob(&(PP)->run_queue))); \ + } while (0) + erts_smp_atomic_t erts_port_task_outstanding_io_tasks; #define ERTS_PT_STATE_SCHEDULED 0 @@ -798,12 +805,13 @@ schedule_port_task_handle_list_free(ErtsPortTaskHandleList *pthlp) static ERTS_INLINE void abort_nosuspend_task(Port *pp, ErtsPortTaskType type, - ErtsPortTaskTypeData *tdp) + ErtsPortTaskTypeData *tdp, + int bpq_data) { ASSERT(type == ERTS_PORT_TASK_PROC_SIG); - if (!pp->sched.taskq.bpq) + if (!bpq_data) tdp->psig.callback(NULL, ERTS_PORT_SFLG_INVALID, ERTS_PROC2PORT_SIG_ABORT_NOSUSPEND, @@ -991,6 +999,7 @@ static ERTS_INLINE int finalize_exec(Port *pp, ErtsPortTask **execq, int processing_busy_q) { erts_aint32_t act; + unsigned int prof_runnable_ports; if (!processing_busy_q) pp->sched.taskq.local.first = *execq; @@ -1007,6 +1016,10 @@ finalize_exec(Port *pp, ErtsPortTask **execq, int processing_busy_q) if (act & ERTS_PTS_FLG_CHK_UNSET_BUSY_PORT_Q) act = check_unset_busy_port_q(pp, act, pp->sched.taskq.bpq); + prof_runnable_ports = erts_system_profile_flags.runnable_ports; + if (prof_runnable_ports) + erts_port_task_sched_lock(&pp->sched); + while (1) { erts_aint32_t new, exp; @@ -1018,12 +1031,24 @@ finalize_exec(Port *pp, ErtsPortTask **execq, int processing_busy_q) act = erts_smp_atomic32_cmpxchg_relb(&pp->sched.flags, new, exp); - ASSERT(!(act & ERTS_PTS_FLG_IN_RUNQ)); + ERTS_LC_ASSERT(!(act & ERTS_PTS_FLG_IN_RUNQ)); + ERTS_LC_ASSERT(!(act & ERTS_PTS_FLG_EXEC_IMM)); if (exp == act) break; } + if (prof_runnable_ports | IS_TRACED_FL(pp, F_TRACE_SCHED_PORTS)) { + /* trace port scheduling, out */ + if (IS_TRACED_FL(pp, F_TRACE_SCHED_PORTS)) + trace_sched_ports(pp, am_out); + if (prof_runnable_ports) { + if (!(act & (ERTS_PTS_FLG_EXEC_IMM|ERTS_PTS_FLG_HAVE_TASKS))) + profile_runnable_port(pp, am_inactive); + erts_port_task_sched_unlock(&pp->sched); + } + } + return (act & ERTS_PTS_FLG_HAVE_TASKS) != 0; } @@ -1345,7 +1370,7 @@ erts_port_task_abort_nosuspend_tasks(Port *pp) #endif schedule_port_task_handle_list_free(pthlp); - abort_nosuspend_task(pp, type, &td); + abort_nosuspend_task(pp, type, &td, pp->sched.taskq.bpq != NULL); } } @@ -1369,6 +1394,7 @@ erts_port_task_schedule(Eterm id, Port *pp; ErtsPortTask *ptp = NULL; erts_aint32_t act, add_flags; + unsigned int prof_runnable_ports; if (pthp && erts_port_task_is_scheduled(pthp)) { ASSERT(0); @@ -1457,6 +1483,10 @@ erts_port_task_schedule(Eterm id, if (ns_pthlp) add_flags |= ERTS_PTS_FLG_HAVE_NS_TASKS; + prof_runnable_ports = erts_system_profile_flags.runnable_ports; + if (prof_runnable_ports) + erts_port_task_sched_lock(&pp->sched); + while (1) { erts_aint32_t new, exp; @@ -1481,6 +1511,13 @@ erts_port_task_schedule(Eterm id, goto done; /* Died after our task insert... */ } + if (prof_runnable_ports) { + if (!(act & ERTS_PTS_FLG_EXEC_IMM)) + profile_runnable_port(pp, am_active); + erts_port_task_sched_unlock(&pp->sched); + prof_runnable_ports = 0; + } + /* Enqueue port on run-queue */ runq = erts_port_runq(pp); @@ -1489,8 +1526,10 @@ erts_port_task_schedule(Eterm id, #ifdef ERTS_SMP xrunq = erts_check_emigration_need(runq, ERTS_PORT_PRIO_LEVEL); + ERTS_SMP_LC_ASSERT(runq != xrunq); + ERTS_SMP_LC_VERIFY_RQ(runq, pp); if (xrunq) { - /* Port emigrated ... */ + /* Emigrate port ... */ erts_smp_atomic_set_nob(&pp->run_queue, (erts_aint_t) xrunq); erts_smp_runq_unlock(runq); runq = erts_port_runq(pp); @@ -1500,10 +1539,6 @@ erts_port_task_schedule(Eterm id, #endif enqueue_port(runq, pp); - - if (erts_system_profile_flags.runnable_ports) { - profile_runnable_port(pp, am_active); - } erts_smp_runq_unlock(runq); @@ -1511,6 +1546,9 @@ erts_port_task_schedule(Eterm id, done: + if (prof_runnable_ports) + erts_port_task_sched_unlock(&pp->sched); + #ifdef ERTS_SMP if (dhndl != ERTS_THR_PRGR_DHANDLE_MANAGED) erts_port_dec_refc(pp); @@ -1525,7 +1563,7 @@ abort_nosuspend: erts_port_dec_refc(pp); #endif - abort_nosuspend_task(pp, ptp->type, &ptp->u.alive.td); + abort_nosuspend_task(pp, ptp->type, &ptp->u.alive.td, 0); ASSERT(ns_pthlp); erts_free(ERTS_ALC_T_PT_HNDL_LIST, ns_pthlp); @@ -1609,6 +1647,8 @@ erts_port_task_execute(ErtsRunQueue *runq, Port **curr_port_pp) goto done; } + ERTS_SMP_LC_VERIFY_RQ(runq, pp); + erts_smp_runq_unlock(runq); *curr_port_pp = pp; @@ -1765,10 +1805,6 @@ erts_port_task_execute(ErtsRunQueue *runq, Port **curr_port_pp) erts_unblock_fpe(fpe_was_unmasked); - /* trace port scheduling, out */ - if (IS_TRACED_FL(pp, F_TRACE_SCHED_PORTS)) { - trace_sched_ports(pp, am_out); - } if (io_tasks_executed) { ASSERT(erts_smp_atomic_read_nob(&erts_port_task_outstanding_io_tasks) @@ -1791,11 +1827,7 @@ erts_port_task_execute(ErtsRunQueue *runq, Port **curr_port_pp) erts_smp_runq_lock(runq); - if (!active) { - if (erts_system_profile_flags.runnable_ports) - profile_runnable_port(pp, am_inactive); - } - else { + if (active) { #ifdef ERTS_SMP ErtsRunQueue *xrunq; #endif @@ -1804,6 +1836,8 @@ erts_port_task_execute(ErtsRunQueue *runq, Port **curr_port_pp) #ifdef ERTS_SMP xrunq = erts_check_emigration_need(runq, ERTS_PORT_PRIO_LEVEL); + ERTS_SMP_LC_ASSERT(runq != xrunq); + ERTS_SMP_LC_VERIFY_RQ(runq, pp); if (!xrunq) { #endif enqueue_port(runq, pp); @@ -1811,7 +1845,7 @@ erts_port_task_execute(ErtsRunQueue *runq, Port **curr_port_pp) #ifdef ERTS_SMP } else { - /* Port emigrated ... */ + /* Emigrate port... */ erts_smp_atomic_set_nob(&pp->run_queue, (erts_aint_t) xrunq); erts_smp_runq_unlock(runq); diff --git a/erts/emulator/beam/erl_port_task.h b/erts/emulator/beam/erl_port_task.h index 1d30465ec9..9ef0cfcedc 100644 --- a/erts/emulator/beam/erl_port_task.h +++ b/erts/emulator/beam/erl_port_task.h @@ -78,6 +78,7 @@ extern erts_smp_atomic_t erts_port_task_outstanding_io_tasks; #define ERTS_PTS_FLG_PARALLELISM (((erts_aint32_t) 1) << 9) #define ERTS_PTS_FLG_FORCE_SCHED (((erts_aint32_t) 1) << 10) #define ERTS_PTS_FLG_EXITING (((erts_aint32_t) 1) << 11) +#define ERTS_PTS_FLG_EXEC_IMM (((erts_aint32_t) 1) << 12) #define ERTS_PTS_FLGS_BUSY \ (ERTS_PTS_FLG_BUSY_PORT | ERTS_PTS_FLG_BUSY_PORT_Q) @@ -87,6 +88,7 @@ extern erts_smp_atomic_t erts_port_task_outstanding_io_tasks; | ERTS_PTS_FLG_HAVE_BUSY_TASKS \ | ERTS_PTS_FLG_HAVE_TASKS \ | ERTS_PTS_FLG_EXEC \ + | ERTS_PTS_FLG_EXEC_IMM \ | ERTS_PTS_FLG_FORCE_SCHED \ | ERTS_PTS_FLG_EXITING) diff --git a/erts/emulator/beam/erl_process.c b/erts/emulator/beam/erl_process.c index 8bf0cc9491..20a88ec581 100644 --- a/erts/emulator/beam/erl_process.c +++ b/erts/emulator/beam/erl_process.c @@ -590,12 +590,10 @@ erts_pre_init_process(void) erts_psd_required_locks[ERTS_PSD_DELAYED_GC_TASK_QS].set_locks = ERTS_PSD_DELAYED_GC_TASK_QS_SET_LOCKS; -#ifdef ERTS_DIRTY_SCHEDULERS - erts_psd_required_locks[ERTS_PSD_DIRTY_SCHED_TRAP_EXPORT].get_locks - = ERTS_PSD_DIRTY_SCHED_TRAP_EXPORT_GET_LOCKS; - erts_psd_required_locks[ERTS_PSD_DIRTY_SCHED_TRAP_EXPORT].set_locks - = ERTS_PSD_DIRTY_SCHED_TRAP_EXPORT_SET_LOCKS; -#endif + erts_psd_required_locks[ERTS_PSD_NIF_TRAP_EXPORT].get_locks + = ERTS_PSD_NIF_TRAP_EXPORT_GET_LOCKS; + erts_psd_required_locks[ERTS_PSD_NIF_TRAP_EXPORT].set_locks + = ERTS_PSD_NIF_TRAP_EXPORT_SET_LOCKS; /* Check that we have locks for all entries */ for (ix = 0; ix < ERTS_PSD_SIZE; ix++) { @@ -2211,6 +2209,9 @@ aux_work_timeout_early_init(int no_schedulers) p = (UWord) malloc((sizeof(ErtsAuxWorkTmo) + sizeof(erts_atomic32_t)*(no_schedulers+1)) + ERTS_CACHE_LINE_SIZE-1); + if (!p) { + ERTS_INTERNAL_ERROR("malloc failed to allocate memory!"); + } if (p & ERTS_CACHE_LINE_MASK) p = (p & ~ERTS_CACHE_LINE_MASK) + ERTS_CACHE_LINE_SIZE; ASSERT((p & ERTS_CACHE_LINE_MASK) == 0); @@ -2672,6 +2673,13 @@ aux_thread(void *unused) ErtsThrPrgrCallbacks callbacks; int thr_prgr_active = 1; +#ifdef ERTS_ENABLE_LOCK_CHECK + { + char buf[] = "aux_thread"; + erts_lc_set_thread_name(buf); + } +#endif + ssi->event = erts_tse_fetch(); callbacks.arg = (void *) ssi; @@ -3748,17 +3756,25 @@ evacuate_run_queue(ErtsRunQueue *rq, } #ifdef ERTS_DIRTY_SCHEDULERS else if (state & ERTS_PSFLG_DIRTY_CPU_PROC_IN_Q) { - erts_aint32_t old; - old = erts_smp_atomic32_read_band_nob(&proc->state, - ~(ERTS_PSFLG_DIRTY_CPU_PROC - | ERTS_PSFLG_DIRTY_CPU_PROC_IN_Q)); +#ifdef DEBUG + erts_aint32_t old = +#else + (void) +#endif + erts_smp_atomic32_read_band_nob(&proc->state, + ~(ERTS_PSFLG_DIRTY_CPU_PROC + | ERTS_PSFLG_DIRTY_CPU_PROC_IN_Q)); /* assert that no other dirty flags are set */ ASSERT(!(old & (ERTS_PSFLG_DIRTY_IO_PROC|ERTS_PSFLG_DIRTY_IO_PROC_IN_Q))); } else if (state & ERTS_PSFLG_DIRTY_IO_PROC_IN_Q) { - erts_aint32_t old; - old = erts_smp_atomic32_read_band_nob(&proc->state, - ~(ERTS_PSFLG_DIRTY_IO_PROC - | ERTS_PSFLG_DIRTY_IO_PROC_IN_Q)); +#ifdef DEBUG + erts_aint32_t old = +#else + (void) +#endif + erts_smp_atomic32_read_band_nob(&proc->state, + ~(ERTS_PSFLG_DIRTY_IO_PROC + | ERTS_PSFLG_DIRTY_IO_PROC_IN_Q)); /* assert that no other dirty flags are set */ ASSERT(!(old & (ERTS_PSFLG_DIRTY_CPU_PROC|ERTS_PSFLG_DIRTY_CPU_PROC_IN_Q))); } @@ -5867,6 +5883,9 @@ schedule_out_process(ErtsRunQueue *c_rq, erts_aint32_t state, Process *p, Proces case ERTS_ENQUEUE_NOT: if (erts_system_profile_flags.runnable_procs) { + /* Status lock prevents out of order "runnable proc" trace msgs */ + ERTS_SMP_LC_ASSERT(ERTS_PROC_LOCK_STATUS & erts_proc_lc_my_proc_locks(p)); + if (!(a & ERTS_PSFLG_ACTIVE_SYS) && (!(a & ERTS_PSFLG_ACTIVE) || (a & ERTS_PSFLG_SUSPENDED))) { @@ -5980,7 +5999,8 @@ change_proc_schedule_state(Process *p, erts_aint32_t clear_state_flags, erts_aint32_t set_state_flags, erts_aint32_t *statep, - erts_aint32_t *enq_prio_p) + erts_aint32_t *enq_prio_p, + ErtsProcLocks locks) { /* * NOTE: ERTS_PSFLG_RUNNING, ERTS_PSFLG_RUNNING_SYS and @@ -5989,6 +6009,11 @@ change_proc_schedule_state(Process *p, */ erts_aint32_t a = *statep, n; int enqueue; /* < 0 -> use proxy */ + unsigned int prof_runnable_procs = erts_system_profile_flags.runnable_procs; + unsigned int lock_status = (prof_runnable_procs + && !(locks & ERTS_PROC_LOCK_STATUS)); + + ERTS_SMP_LC_ASSERT(locks == erts_proc_lc_my_proc_locks(p)); ASSERT(!(a & ERTS_PSFLG_PROXY)); ASSERT((clear_state_flags & (ERTS_PSFLG_RUNNING @@ -5998,6 +6023,9 @@ change_proc_schedule_state(Process *p, | ERTS_PSFLG_RUNNING_SYS | ERTS_PSFLG_ACTIVE_SYS)) == 0); + if (lock_status) + erts_smp_proc_lock(p, ERTS_PROC_LOCK_STATUS); + while (1) { erts_aint32_t e; n = e = a; @@ -6033,7 +6061,9 @@ change_proc_schedule_state(Process *p, break; } - if (erts_system_profile_flags.runnable_procs) { + if (prof_runnable_procs) { + + /* Status lock prevents out of order "runnable proc" trace msgs */ if (((n & (ERTS_PSFLG_SUSPENDED | ERTS_PSFLG_ACTIVE)) == ERTS_PSFLG_ACTIVE) @@ -6046,15 +6076,18 @@ change_proc_schedule_state(Process *p, profile_runnable_proc(p, am_active); } + if (lock_status) + erts_smp_proc_unlock(p, ERTS_PROC_LOCK_STATUS); } + *statep = a; return enqueue; } static ERTS_INLINE void -schedule_process(Process *p, erts_aint32_t in_state) +schedule_process(Process *p, erts_aint32_t in_state, ErtsProcLocks locks) { erts_aint32_t enq_prio = -1; erts_aint32_t state = in_state; @@ -6062,7 +6095,8 @@ schedule_process(Process *p, erts_aint32_t in_state) 0, ERTS_PSFLG_ACTIVE, &state, - &enq_prio); + &enq_prio, + locks); if (enqueue != ERTS_ENQUEUE_NOT) add2runq(enqueue > 0 ? p : make_proxy_proc(NULL, p, enq_prio), state, @@ -6070,16 +6104,27 @@ schedule_process(Process *p, erts_aint32_t in_state) } void -erts_schedule_process(Process *p, erts_aint32_t state) +erts_schedule_process(Process *p, erts_aint32_t state, ErtsProcLocks locks) { - schedule_process(p, state); + schedule_process(p, state, locks); } static void schedule_process_sys_task(Process *p, erts_aint32_t state, Process *proxy) { + /* + * Expects status lock to be locked when called, and + * returns with status lock unlocked... + */ erts_aint32_t a = state, n, enq_prio = -1; int enqueue; /* < 0 -> use proxy */ + unsigned int prof_runnable_procs = erts_system_profile_flags.runnable_procs; + + /* Status lock prevents out of order "runnable proc" trace msgs */ + ERTS_SMP_LC_ASSERT(ERTS_PROC_LOCK_STATUS & erts_proc_lc_my_proc_locks(p)); + + if (!prof_runnable_procs) + erts_smp_proc_unlock(p, ERTS_PROC_LOCK_STATUS); ASSERT(!(state & ERTS_PSFLG_PROXY)); @@ -6088,7 +6133,7 @@ schedule_process_sys_task(Process *p, erts_aint32_t state, Process *proxy) n = e = a; if (a & ERTS_PSFLG_FREE) - return; /* We don't want to schedule free processes... */ + goto cleanup; /* We don't want to schedule free processes... */ enqueue = ERTS_ENQUEUE_NOT; n |= ERTS_PSFLG_ACTIVE_SYS; @@ -6101,7 +6146,7 @@ schedule_process_sys_task(Process *p, erts_aint32_t state, Process *proxy) goto cleanup; } - if (erts_system_profile_flags.runnable_procs) { + if (prof_runnable_procs) { if (!(a & (ERTS_PSFLG_ACTIVE_SYS | ERTS_PSFLG_RUNNING @@ -6111,6 +6156,8 @@ schedule_process_sys_task(Process *p, erts_aint32_t state, Process *proxy) profile_runnable_proc(p, am_active); } + erts_smp_proc_unlock(p, ERTS_PROC_LOCK_STATUS); + prof_runnable_procs = 0; } if (enqueue != ERTS_ENQUEUE_NOT) { @@ -6125,8 +6172,14 @@ schedule_process_sys_task(Process *p, erts_aint32_t state, Process *proxy) } cleanup: + + if (prof_runnable_procs) + erts_smp_proc_unlock(p, ERTS_PROC_LOCK_STATUS); + if (proxy) free_proxy_proc(proxy); + + ERTS_SMP_LC_ASSERT(!(ERTS_PROC_LOCK_STATUS & erts_proc_lc_my_proc_locks(p))); } static ERTS_INLINE int @@ -6144,7 +6197,7 @@ suspend_process(Process *c_p, Process *p) if (c_p == p) { state = erts_smp_atomic32_read_bor_relb(&p->state, ERTS_PSFLG_SUSPENDED); - ASSERT(state & ERTS_PSFLG_RUNNING); + ASSERT(state & (ERTS_PSFLG_RUNNING|ERTS_PSFLG_RUNNING_SYS)); suspended = (state & ERTS_PSFLG_SUSPENDED) ? -1: 1; } else { @@ -6193,7 +6246,7 @@ suspend_process(Process *c_p, Process *p) } static ERTS_INLINE void -resume_process(Process *p) +resume_process(Process *p, ErtsProcLocks locks) { erts_aint32_t state, enq_prio = -1; int enqueue; @@ -6210,7 +6263,8 @@ resume_process(Process *p) ERTS_PSFLG_SUSPENDED, 0, &state, - &enq_prio); + &enq_prio, + locks); if (enqueue) add2runq(enqueue > 0 ? p : make_proxy_proc(NULL, p, enq_prio), state, @@ -7811,6 +7865,9 @@ erts_start_schedulers(void) #ifdef ETHR_HAVE_THREAD_NAMES opts.name = malloc(80); + if (!opts.name) { + ERTS_INTERNAL_ERROR("malloc failed to allocate memory!"); + } #endif #ifdef ERTS_SMP @@ -8023,7 +8080,8 @@ handle_pend_sync_suspend(Process *suspendee, } /* suspender is suspended waiting for suspendee to suspend; resume suspender */ - resume_process(suspender); + ASSERT(suspendee != suspender); + resume_process(suspender, ERTS_PROC_LOCK_STATUS); erts_smp_proc_unlock(suspender, ERTS_PROC_LOCK_STATUS); } } @@ -8058,7 +8116,7 @@ pid2proc_not_running(Process *c_p, ErtsProcLocks c_p_locks, ASSERT(c_p->flags & F_P2PNR_RESCHED); c_p->flags &= ~F_P2PNR_RESCHED; if (!suspend && rp) - resume_process(rp); + resume_process(rp, rp_locks); } else { @@ -8216,7 +8274,8 @@ handle_pend_bif_sync_suspend(Process *suspendee, } /* suspender is suspended waiting for suspendee to suspend; resume suspender */ - resume_process(suspender); + ASSERT(suspender != suspendee); + resume_process(suspender, ERTS_PROC_LOCK_LINK|ERTS_PROC_LOCK_STATUS); erts_smp_proc_unlock(suspender, ERTS_PROC_LOCK_LINK|ERTS_PROC_LOCK_STATUS); } @@ -8576,7 +8635,8 @@ resume_process_1(BIF_ALIST_1) ASSERT(ERTS_PSFLG_SUSPENDED & erts_smp_atomic32_read_nob(&suspendee->state)); - resume_process(suspendee); + ASSERT(BIF_P != suspendee); + resume_process(suspendee, ERTS_PROC_LOCK_STATUS); erts_smp_proc_unlock(suspendee, ERTS_PROC_LOCK_STATUS); } @@ -8706,7 +8766,7 @@ erts_resume(Process* process, ErtsProcLocks process_locks) ERTS_SMP_LC_ASSERT(process_locks == erts_proc_lc_my_proc_locks(process)); if (!(process_locks & ERTS_PROC_LOCK_STATUS)) erts_smp_proc_lock(process, ERTS_PROC_LOCK_STATUS); - resume_process(process); + resume_process(process, process_locks|ERTS_PROC_LOCK_STATUS); if (!(process_locks & ERTS_PROC_LOCK_STATUS)) erts_smp_proc_unlock(process, ERTS_PROC_LOCK_STATUS); } @@ -8725,7 +8785,7 @@ erts_resume_processes(ErtsProcList *list) proc = erts_pid2proc(NULL, 0, plp->pid, ERTS_PROC_LOCK_STATUS); if (proc) { if (erts_proclist_same(plp, proc)) { - resume_process(proc); + resume_process(proc, ERTS_PROC_LOCK_STATUS); nresumed++; } erts_smp_proc_unlock(proc, ERTS_PROC_LOCK_STATUS); @@ -9207,7 +9267,6 @@ Process *schedule(Process *p, int calls) */ pick_next_process: { erts_aint32_t psflg_band_mask; - erts_aint32_t running_flag; int prio_q; int qmask; @@ -9269,12 +9328,6 @@ Process *schedule(Process *p, int calls) state = erts_smp_atomic32_read_nob(&p->state); } - - if (state & ERTS_PSFLG_ACTIVE_SYS) - running_flag = ERTS_PSFLG_RUNNING_SYS; - else - running_flag = ERTS_PSFLG_RUNNING; - while (1) { erts_aint32_t exp, new, tmp; tmp = new = exp = state; @@ -9284,8 +9337,12 @@ Process *schedule(Process *p, int calls) tmp = state & (ERTS_PSFLG_SUSPENDED | ERTS_PSFLG_PENDING_EXIT | ERTS_PSFLG_ACTIVE_SYS); - if (tmp != ERTS_PSFLG_SUSPENDED) - new |= running_flag; + if (tmp != ERTS_PSFLG_SUSPENDED) { + if (state & ERTS_PSFLG_ACTIVE_SYS) + new |= ERTS_PSFLG_RUNNING_SYS; + else + new |= ERTS_PSFLG_RUNNING; + } } state = erts_smp_atomic32_cmpxchg_relb(&p->state, new, exp); if (state == exp) { @@ -9964,8 +10021,10 @@ erts_internal_request_system_task_3(BIF_ALIST_3) rp_state = n; } - erts_smp_proc_unlock(rp, ERTS_PROC_LOCK_STATUS); - + /* + * schedule_process_sys_task() unlocks status + * lock on process. + */ schedule_process_sys_task(rp, rp_state, NULL); if (free_stqs) @@ -10710,7 +10769,7 @@ erl_create_process(Process* parent, /* Parent of process (default group leader). * Schedule process for execution. */ - schedule_process(p, state); + schedule_process(p, state, 0); VERBOSE(DEBUG_PROCESSES, ("Created a new process: %T\n",p->common.id)); @@ -11031,7 +11090,8 @@ set_proc_exiting(Process *p, ERTS_PSFLG_SUSPENDED|ERTS_PSFLG_PENDING_EXIT, ERTS_PSFLG_EXITING|ERTS_PSFLG_ACTIVE, &state, - &enq_prio); + &enq_prio, + ERTS_PROC_LOCKS_ALL); p->fvalue = reason; if (bp) @@ -11072,7 +11132,8 @@ set_proc_self_exiting(Process *c_p) ERTS_PSFLG_SUSPENDED|ERTS_PSFLG_PENDING_EXIT, ERTS_PSFLG_EXITING|ERTS_PSFLG_ACTIVE, &state, - &enq_prio); + &enq_prio, + ERTS_PROC_LOCKS_ALL); ASSERT(!enqueue); return state; @@ -11717,8 +11778,9 @@ resume_suspend_monitor(ErtsSuspendMonitor *smon, void *vc_p) Process *suspendee = erts_pid2proc((Process *) vc_p, ERTS_PROC_LOCK_MAIN, smon->pid, ERTS_PROC_LOCK_STATUS); if (suspendee) { + ASSERT(suspendee != vc_p); if (smon->active) - resume_process(suspendee); + resume_process(suspendee, ERTS_PROC_LOCK_STATUS); erts_smp_proc_unlock(suspendee, ERTS_PROC_LOCK_STATUS); } erts_destroy_suspend_monitor(smon); @@ -11810,6 +11872,7 @@ erts_continue_exit_process(Process *p) struct saved_calls *scb; process_breakpoint_time_t *pbt; erts_aint32_t state; + void *nif_export; #ifdef DEBUG int yield_allowed = 1; @@ -11960,6 +12023,7 @@ erts_continue_exit_process(Process *p) : NULL); scb = ERTS_PROC_SET_SAVED_CALLS_BUF(p, ERTS_PROC_LOCKS_ALL, NULL); pbt = ERTS_PROC_SET_CALL_TIME(p, ERTS_PROC_LOCKS_ALL, NULL); + nif_export = ERTS_PROC_SET_NIF_TRAP_EXPORT(p, ERTS_PROC_LOCKS_ALL, NULL); erts_smp_proc_unlock(p, ERTS_PROC_LOCKS_ALL); #ifdef BM_COUNTERS @@ -12007,6 +12071,9 @@ erts_continue_exit_process(Process *p) if (pbt) erts_free(ERTS_ALC_T_BPD, (void *) pbt); + if (nif_export) + erts_destroy_nif_export(nif_export); + delete_process(p); #ifdef ERTS_SMP @@ -12051,7 +12118,7 @@ timeout_proc(Process* p) state = erts_smp_atomic32_read_acqb(&p->state); if (!(state & ERTS_PSFLG_ACTIVE)) - schedule_process(p, state); + schedule_process(p, state, ERTS_PROC_LOCK_MAIN|ERTS_PROC_LOCK_STATUS); } diff --git a/erts/emulator/beam/erl_process.h b/erts/emulator/beam/erl_process.h index ed6dadbffa..3b0798207e 100644 --- a/erts/emulator/beam/erl_process.h +++ b/erts/emulator/beam/erl_process.h @@ -734,13 +734,9 @@ erts_smp_reset_max_len(ErtsRunQueue *rq, ErtsRunQueueInfo *rqi) #define ERTS_PSD_DIST_ENTRY 3 #define ERTS_PSD_CALL_TIME_BP 4 #define ERTS_PSD_DELAYED_GC_TASK_QS 5 -#ifdef ERTS_DIRTY_SCHEDULERS -#define ERTS_PSD_DIRTY_SCHED_TRAP_EXPORT 6 +#define ERTS_PSD_NIF_TRAP_EXPORT 6 #define ERTS_PSD_SIZE 7 -#else -#define ERTS_PSD_SIZE 6 -#endif typedef struct { void *data[ERTS_PSD_SIZE]; @@ -767,10 +763,8 @@ typedef struct { #define ERTS_PSD_DELAYED_GC_TASK_QS_GET_LOCKS ERTS_PROC_LOCK_MAIN #define ERTS_PSD_DELAYED_GC_TASK_QS_SET_LOCKS ERTS_PROC_LOCK_MAIN -#ifdef ERTS_DIRTY_SCHEDULERS -#define ERTS_PSD_DIRTY_SCHED_TRAP_EXPORT_GET_LOCKS ERTS_PROC_LOCK_MAIN -#define ERTS_PSD_DIRTY_SCHED_TRAP_EXPORT_SET_LOCKS ERTS_PROC_LOCK_MAIN -#endif +#define ERTS_PSD_NIF_TRAP_EXPORT_GET_LOCKS ERTS_PROC_LOCK_MAIN +#define ERTS_PSD_NIF_TRAP_EXPORT_SET_LOCKS ERTS_PROC_LOCK_MAIN typedef struct { ErtsProcLocks get_locks; @@ -1367,6 +1361,9 @@ Uint64 erts_get_proc_interval(void); Uint64 erts_ensure_later_proc_interval(Uint64); Uint64 erts_step_proc_interval(void); +int erts_setup_nif_gc(Process* proc, Eterm** objv, int* nobj); /* see erl_nif.c */ +void erts_destroy_nif_export(void *); /* see erl_nif.c */ + ErtsProcList *erts_proclist_create(Process *); void erts_proclist_destroy(ErtsProcList *); @@ -1704,17 +1701,17 @@ ErtsSchedulerData *erts_get_scheduler_data(void) #endif #endif -void erts_schedule_process(Process *, erts_aint32_t); +void erts_schedule_process(Process *, erts_aint32_t, ErtsProcLocks); -ERTS_GLB_INLINE void erts_proc_notify_new_message(Process *p); +ERTS_GLB_INLINE void erts_proc_notify_new_message(Process *p, ErtsProcLocks locks); #if ERTS_GLB_INLINE_INCL_FUNC_DEF ERTS_GLB_INLINE void -erts_proc_notify_new_message(Process *p) +erts_proc_notify_new_message(Process *p, ErtsProcLocks locks) { /* No barrier needed, due to msg lock */ erts_aint32_t state = erts_smp_atomic32_read_nob(&p->state); if (!(state & ERTS_PSFLG_ACTIVE)) - erts_schedule_process(p, state); + erts_schedule_process(p, state, locks); } #endif @@ -1817,12 +1814,10 @@ erts_psd_set(Process *p, ErtsProcLocks plocks, int ix, void *data) #define ERTS_PROC_SET_DELAYED_GC_TASK_QS(P, L, PBT) \ ((ErtsProcSysTaskQs *) erts_psd_set((P), (L), ERTS_PSD_DELAYED_GC_TASK_QS, (void *) (PBT))) -#ifdef ERTS_DIRTY_SCHEDULERS -#define ERTS_PROC_GET_DIRTY_SCHED_TRAP_EXPORT(P) \ - ((Export *) erts_psd_get((P), ERTS_PSD_DIRTY_SCHED_TRAP_EXPORT)) -#define ERTS_PROC_SET_DIRTY_SCHED_TRAP_EXPORT(P, L, DSTE) \ - ((Export *) erts_psd_set((P), (L), ERTS_PSD_DIRTY_SCHED_TRAP_EXPORT, (void *) (DSTE))) -#endif +#define ERTS_PROC_GET_NIF_TRAP_EXPORT(P) \ + erts_psd_get((P), ERTS_PSD_NIF_TRAP_EXPORT) +#define ERTS_PROC_SET_NIF_TRAP_EXPORT(P, L, NTE) \ + erts_psd_set((P), (L), ERTS_PSD_NIF_TRAP_EXPORT, (void *) (NTE)) ERTS_GLB_INLINE Eterm erts_proc_get_error_handler(Process *p); diff --git a/erts/emulator/beam/erl_term.h b/erts/emulator/beam/erl_term.h index f10a3a9d38..37014ccf94 100644 --- a/erts/emulator/beam/erl_term.h +++ b/erts/emulator/beam/erl_term.h @@ -112,11 +112,11 @@ struct erl_node_; /* Declared in erl_node_tables.h */ * 1000 REFC_BINARY | | * 1001 HEAP_BINARY | BINARIES | * 1010 SUB_BINARY | | - * 1011 Not used + * 1011 Not used; see comment below * 1100 EXTERNAL_PID | | * 1101 EXTERNAL_PORT | EXTERNAL THINGS | * 1110 EXTERNAL_REF | | - * 1111 Not used + * 1111 MAP * * COMMENTS: * @@ -140,10 +140,11 @@ struct erl_node_; /* Declared in erl_node_tables.h */ #define REFC_BINARY_SUBTAG (0x8 << _TAG_PRIMARY_SIZE) /* BINARY */ #define HEAP_BINARY_SUBTAG (0x9 << _TAG_PRIMARY_SIZE) /* BINARY */ #define SUB_BINARY_SUBTAG (0xA << _TAG_PRIMARY_SIZE) /* BINARY */ -#define MAP_SUBTAG (0xB << _TAG_PRIMARY_SIZE) /* MAP */ +/* _BINARY_XXX_MASK depends on 0xB being unused */ #define EXTERNAL_PID_SUBTAG (0xC << _TAG_PRIMARY_SIZE) /* EXTERNAL_PID */ #define EXTERNAL_PORT_SUBTAG (0xD << _TAG_PRIMARY_SIZE) /* EXTERNAL_PORT */ #define EXTERNAL_REF_SUBTAG (0xE << _TAG_PRIMARY_SIZE) /* EXTERNAL_REF */ +#define MAP_SUBTAG (0xF << _TAG_PRIMARY_SIZE) /* MAP */ #define _TAG_HEADER_ARITYVAL (TAG_PRIMARY_HEADER|ARITYVAL_SUBTAG) @@ -156,11 +157,11 @@ struct erl_node_; /* Declared in erl_node_tables.h */ #define _TAG_HEADER_REFC_BIN (TAG_PRIMARY_HEADER|REFC_BINARY_SUBTAG) #define _TAG_HEADER_HEAP_BIN (TAG_PRIMARY_HEADER|HEAP_BINARY_SUBTAG) #define _TAG_HEADER_SUB_BIN (TAG_PRIMARY_HEADER|SUB_BINARY_SUBTAG) -#define _TAG_HEADER_MAP (TAG_PRIMARY_HEADER|MAP_SUBTAG) #define _TAG_HEADER_EXTERNAL_PID (TAG_PRIMARY_HEADER|EXTERNAL_PID_SUBTAG) #define _TAG_HEADER_EXTERNAL_PORT (TAG_PRIMARY_HEADER|EXTERNAL_PORT_SUBTAG) #define _TAG_HEADER_EXTERNAL_REF (TAG_PRIMARY_HEADER|EXTERNAL_REF_SUBTAG) #define _TAG_HEADER_BIN_MATCHSTATE (TAG_PRIMARY_HEADER|BIN_MATCHSTATE_SUBTAG) +#define _TAG_HEADER_MAP (TAG_PRIMARY_HEADER|MAP_SUBTAG) #define _TAG_HEADER_MASK 0x3F @@ -892,7 +893,8 @@ typedef struct external_thing_ { (((x) & _TAG_HEADER_MASK) == _TAG_HEADER_EXTERNAL_REF) #define is_external_header(x) \ - (((x) & (_TAG_HEADER_MASK-_BINARY_XXX_MASK)) == _TAG_HEADER_EXTERNAL_PID) + (((x) & (_TAG_HEADER_MASK-_BINARY_XXX_MASK)) == _TAG_HEADER_EXTERNAL_PID \ + && ((x) & _TAG_HEADER_MASK) != _TAG_HEADER_MAP) #define is_external(x) (is_boxed((x)) && is_external_header(*boxed_val((x)))) diff --git a/erts/emulator/beam/erl_trace.c b/erts/emulator/beam/erl_trace.c index 305058ceff..ea5c850a30 100644 --- a/erts/emulator/beam/erl_trace.c +++ b/erts/emulator/beam/erl_trace.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1999-2013. All Rights Reserved. + * Copyright Ericsson AB 1999-2014. All Rights Reserved. * * The contents of this file are subject to the Erlang Public License, * Version 1.1, (the "License"); you may not use this file except in @@ -2524,7 +2524,7 @@ monitor_large_heap(Process *p) { #ifndef ERTS_SMP ASSERT(is_internal_pid(system_monitor)); monitor_p = erts_proc_lookup(system_monitor); - if (monitor_p || p == monitor_p) { + if (!monitor_p || p == monitor_p) { return; } #endif diff --git a/erts/emulator/beam/erl_unicode.c b/erts/emulator/beam/erl_unicode.c index 3a968594f3..f8e1431a53 100644 --- a/erts/emulator/beam/erl_unicode.c +++ b/erts/emulator/beam/erl_unicode.c @@ -2126,6 +2126,8 @@ Eterm erts_convert_native_to_filename(Process *p, byte *bytes) mac = 1; case ERL_FILENAME_UTF8: size = strlen((char *) bytes); + if (size == 0) + return NIL; if (erts_analyze_utf8(bytes,size,&err_pos,&num_chars,NULL) != ERTS_UTF8_OK) { goto noconvert; } diff --git a/erts/emulator/beam/external.c b/erts/emulator/beam/external.c index 656de7c49a..9b9b4b2a62 100644 --- a/erts/emulator/beam/external.c +++ b/erts/emulator/beam/external.c @@ -44,9 +44,6 @@ #include "erl_zlib.h" #include "erl_map.h" -#ifdef HIPE -#include "hipe_mode_switch.h" -#endif #define in_area(ptr,start,nbytes) ((UWord)((char*)(ptr) - (char*)(start)) < (nbytes)) #define MAX_STRING_LEN 0xffff @@ -111,26 +108,17 @@ static int encode_size_struct_int(struct TTBSizeContext_*, ErtsAtomCacheMap *acm static Export binary_to_term_trap_export; static BIF_RETTYPE binary_to_term_trap_1(BIF_ALIST_1); -static Eterm binary_to_term_int(Process* p, Uint32 flags, Eterm bin, Binary* context_b); +static BIF_RETTYPE binary_to_term_int(Process* p, Uint32 flags, Eterm bin, Binary* context_b, + Export *bif, Eterm arg0, Eterm arg1); void erts_init_external(void) { -#if 1 /* In R16 */ erts_init_trap_export(&term_to_binary_trap_export, - am_erlang, am_term_to_binary_trap, 1, + am_erts_internal, am_term_to_binary_trap, 1, &term_to_binary_trap_1); erts_init_trap_export(&binary_to_term_trap_export, - am_erlang, am_binary_to_term_trap, 1, + am_erts_internal, am_binary_to_term_trap, 1, &binary_to_term_trap_1); -#else - sys_memset((void *) &term_to_binary_trap_export, 0, sizeof(Export)); - term_to_binary_trap_export.address = &term_to_binary_trap_export.code[3]; - term_to_binary_trap_export.code[0] = am_erlang; - term_to_binary_trap_export.code[1] = am_term_to_binary_trap; - term_to_binary_trap_export.code[2] = 1; - term_to_binary_trap_export.code[3] = (BeamInstr) em_apply_bif; - term_to_binary_trap_export.code[4] = (BeamInstr) &term_to_binary_trap_1; -#endif return; } @@ -1069,6 +1057,8 @@ static BIF_RETTYPE term_to_binary_trap_1(BIF_ALIST_1) } } +HIPE_WRAPPER_BIF_DISABLE_GC(term_to_binary, 1) + BIF_RETTYPE term_to_binary_1(BIF_ALIST_1) { Eterm res = erts_term_to_binary_int(BIF_P, BIF_ARG_1, 0, TERM_TO_BINARY_DFLAGS, NULL); @@ -1081,6 +1071,8 @@ BIF_RETTYPE term_to_binary_1(BIF_ALIST_1) } } +HIPE_WRAPPER_BIF_DISABLE_GC(term_to_binary, 2) + BIF_RETTYPE term_to_binary_2(BIF_ALIST_2) { Process* p = BIF_P; @@ -1185,6 +1177,8 @@ typedef struct B2TContext_t { Uint32 flags; SWord reds; Eterm trap_bin; + Export *bif; + Eterm arg[2]; enum B2TState state; union { B2TSizeContext sc; @@ -1356,7 +1350,8 @@ static BIF_RETTYPE binary_to_term_trap_1(BIF_ALIST_1) Binary *context_bin = ((ProcBin *) binary_val(BIF_ARG_1))->val; ASSERT(ERTS_MAGIC_BIN_DESTRUCTOR(context_bin) == b2t_context_destructor); - return binary_to_term_int(BIF_P, 0, THE_NON_VALUE, context_bin); + return binary_to_term_int(BIF_P, 0, THE_NON_VALUE, context_bin, NULL, + THE_NON_VALUE, THE_NON_VALUE); } @@ -1391,8 +1386,10 @@ static B2TContext* b2t_export_context(Process* p, B2TContext* src) return ctx; } -static Eterm binary_to_term_int(Process* p, Uint32 flags, Eterm bin, Binary* context_b) +static BIF_RETTYPE binary_to_term_int(Process* p, Uint32 flags, Eterm bin, Binary* context_b, + Export *bif_init, Eterm arg0, Eterm arg1) { + BIF_RETTYPE ret_val; #ifdef EXTREME_B2T_TRAPPING SWord initial_reds = 1 + b2t_rand() % 4; #else @@ -1409,6 +1406,9 @@ static Eterm binary_to_term_int(Process* p, Uint32 flags, Eterm bin, Binary* con ctx->state = B2TPrepare; ctx->aligned_alloc = NULL; ctx->flags = flags; + ctx->bif = bif_init; + ctx->arg[0] = arg0; + ctx->arg[1] = arg1; IF_DEBUG(ctx->trap_bin = THE_NON_VALUE;) } else { is_first_call = 0; @@ -1504,12 +1504,24 @@ static Eterm binary_to_term_int(Process* p, Uint32 flags, Eterm bin, Binary* con HRelease(p, ctx->u.dc.hp_end, ctx->u.dc.hp_start); /*fall through*/ case B2TBadArg: - b2t_destroy_context(ctx); - if (!is_first_call) { - erts_set_gc_state(p, 1); - } BUMP_REDS(p, (initial_reds - ctx->reds) / B2T_BYTES_PER_REDUCTION); - BIF_ERROR(p, BADARG & ~EXF_SAVETRACE); + + ASSERT(ctx->bif == bif_export[BIF_binary_to_term_1] + || ctx->bif == bif_export[BIF_binary_to_term_2]); + + if (is_first_call) + ERTS_BIF_PREP_ERROR(ret_val, p, BADARG); + else { + erts_set_gc_state(p, 1); + if (is_non_value(ctx->arg[1])) + ERTS_BIF_PREP_ERROR_TRAPPED1(ret_val, p, BADARG, ctx->bif, + ctx->arg[0]); + else + ERTS_BIF_PREP_ERROR_TRAPPED2(ret_val, p, BADARG, ctx->bif, + ctx->arg[0], ctx->arg[1]); + } + b2t_destroy_context(ctx); + return ret_val; case B2TDone: b2t_destroy_context(ctx); @@ -1524,7 +1536,8 @@ static Eterm binary_to_term_int(Process* p, Uint32 flags, Eterm bin, Binary* con erts_set_gc_state(p, 1); } BUMP_REDS(p, (initial_reds - ctx->reds) / B2T_BYTES_PER_REDUCTION); - return ctx->u.dc.res; + ERTS_BIF_PREP_RET(ret_val, ctx->u.dc.res); + return ret_val; default: ASSERT(!"Unknown state in binary_to_term"); @@ -1541,15 +1554,24 @@ static Eterm binary_to_term_int(Process* p, Uint32 flags, Eterm bin, Binary* con erts_set_gc_state(p, 0); } BUMP_ALL_REDS(p); - BIF_TRAP1(&binary_to_term_trap_export, p, ctx->trap_bin); + + ERTS_BIF_PREP_TRAP1(ret_val, &binary_to_term_trap_export, + p, ctx->trap_bin); + + return ret_val; } -BIF_RETTYPE erts_internal_binary_to_term_1(BIF_ALIST_1) +HIPE_WRAPPER_BIF_DISABLE_GC(binary_to_term, 1) + +BIF_RETTYPE binary_to_term_1(BIF_ALIST_1) { - return binary_to_term_int(BIF_P, 0, BIF_ARG_1, NULL); + return binary_to_term_int(BIF_P, 0, BIF_ARG_1, NULL, bif_export[BIF_binary_to_term_1], + BIF_ARG_1, THE_NON_VALUE); } -BIF_RETTYPE erts_internal_binary_to_term_2(BIF_ALIST_2) +HIPE_WRAPPER_BIF_DISABLE_GC(binary_to_term, 2) + +BIF_RETTYPE binary_to_term_2(BIF_ALIST_2) { Eterm opts; Eterm opt; @@ -1570,7 +1592,8 @@ BIF_RETTYPE erts_internal_binary_to_term_2(BIF_ALIST_2) if (is_not_nil(opts)) goto error; - return binary_to_term_int(BIF_P, flags, BIF_ARG_1, NULL); + return binary_to_term_int(BIF_P, flags, BIF_ARG_1, NULL, bif_export[BIF_binary_to_term_2], + BIF_ARG_1, BIF_ARG_2); error: BIF_ERROR(BIF_P, BADARG); @@ -1902,6 +1925,7 @@ static Eterm erts_term_to_binary_int(Process* p, Eterm Term, int level, Uint fla } real_size = endp - bytes; result_bin = erts_bin_realloc(context->s.ec.result_bin,real_size); + result_bin->orig_size = real_size; level = context->s.ec.level; BUMP_REDS(p, (initial_reds - reds) / TERM_TO_BINARY_LOOP_FACTOR); if (level == 0 || real_size < 6) { /* We are done */ @@ -1981,6 +2005,7 @@ static Eterm erts_term_to_binary_int(Process* p, Eterm Term, int level, Uint fla erl_zlib_deflate_finish(&(context->s.cc.stream)); result_bin = erts_bin_realloc(context->s.cc.destination_bin, context->s.cc.dest_len+6); + result_bin->orig_size = context->s.cc.dest_len+6; context->s.cc.destination_bin = NULL; pb = (ProcBin *) HAlloc(p, PROC_BIN_SIZE); pb->thing_word = HEADER_PROC_BIN; @@ -4440,66 +4465,3 @@ error: #undef SKIP2 #undef CHKSIZE } - - -#ifdef HIPE -BIF_RETTYPE hipe_wrapper_term_to_binary_1(BIF_ALIST_1); -BIF_RETTYPE hipe_wrapper_term_to_binary_2(BIF_ALIST_2); -BIF_RETTYPE hipe_wrapper_erts_internal_binary_to_term_1(BIF_ALIST_1); -BIF_RETTYPE hipe_wrapper_erts_internal_binary_to_term_2(BIF_ALIST_2); - -/* Hipe wrappers used by native code for BIFs that disable GC while trapping. - * - * Problem: - * When native code calls a BIF that traps, hipe_mode_switch will push a - * "trap frame" on the Erlang stack in order to find its way back from beam_emu - * back to native caller when finally done. If GC is disabled and stack/heap - * is full there is no place to push the "trap frame". - * - * Solution: - * We reserve space on stack for the "trap frame" here before the BIF is called. - * If the BIF does not trap, the space is reclaimed here before returning. - * If the BIF traps, hipe_push_beam_trap_frame() will detect that a "trap frame" - * already is reserved and use it. - */ -BIF_RETTYPE hipe_wrapper_term_to_binary_1(BIF_ALIST_1) -{ - Eterm res; - hipe_reserve_beam_trap_frame(BIF_P, BIF__ARGS, 1); - res = term_to_binary_1(BIF_P, BIF__ARGS); - if (is_value(res) || BIF_P->freason != TRAP) { - hipe_unreserve_beam_trap_frame(BIF_P); - } - return res; -} -BIF_RETTYPE hipe_wrapper_term_to_binary_2(BIF_ALIST_2) -{ - Eterm res; - hipe_reserve_beam_trap_frame(BIF_P, BIF__ARGS, 2); - res = term_to_binary_2(BIF_P, BIF__ARGS); - if (is_value(res) || BIF_P->freason != TRAP) { - hipe_unreserve_beam_trap_frame(BIF_P); - } - return res; -} -BIF_RETTYPE hipe_wrapper_erts_internal_binary_to_term_1(BIF_ALIST_1) -{ - Eterm res; - hipe_reserve_beam_trap_frame(BIF_P, BIF__ARGS, 1); - res = erts_internal_binary_to_term_1(BIF_P, BIF__ARGS); - if (is_value(res) || BIF_P->freason != TRAP) { - hipe_unreserve_beam_trap_frame(BIF_P); - } - return res; -} -BIF_RETTYPE hipe_wrapper_erts_internal_binary_to_term_2(BIF_ALIST_2) -{ - Eterm res; - hipe_reserve_beam_trap_frame(BIF_P, BIF__ARGS, 2); - res = erts_internal_binary_to_term_2(BIF_P, BIF__ARGS); - if (is_value(res) || BIF_P->freason != TRAP) { - hipe_unreserve_beam_trap_frame(BIF_P); - } - return res; -} -#endif /*HIPE*/ diff --git a/erts/emulator/beam/global.h b/erts/emulator/beam/global.h index 8fcb95d0e2..891046a8b5 100755..100644 --- a/erts/emulator/beam/global.h +++ b/erts/emulator/beam/global.h @@ -435,6 +435,8 @@ do {\ }\ } while(0) +#define CLEAR_SAVED_ESTACK(estack) ((void) ((estack)->start = NULL)) + /* * Use on empty stack, only the allocator can be changed before this. * The src stack is reset to NULL. @@ -551,6 +553,8 @@ do {\ }\ } while(0) +#define CLEAR_SAVED_WSTACK(wstack) ((void) ((wstack)->wstart = NULL)) + /* * Use on empty stack, only the allocator can be changed before this. * The src stack is reset to NULL. @@ -951,20 +955,67 @@ struct Sint_buf { }; char* Sint_to_buf(Sint, struct Sint_buf*); +#define ERTS_IOLIST_STATE_INITER(C_P, OBJ) \ + {(C_P), 0, 0, (OBJ), {NULL, NULL, NULL, ERTS_ALC_T_INVALID}, 0, 0} + +#define ERTS_IOLIST_STATE_MOVE(TO, FROM) \ + sys_memcpy((void *) (TO), (void *) (FROM), sizeof(ErtsIOListState)) + +#define ERTS_IOLIST_SIZE_YIELDS_COUNT_PER_RED 8 + +typedef struct { + Process *c_p; + ErlDrvSizeT size; + Uint offs; + Eterm obj; + ErtsEStack estack; + int reds_left; + int have_size; +} ErtsIOListState; + +#define ERTS_IOLIST2BUF_STATE_INITER(C_P, OBJ) \ + {ERTS_IOLIST_STATE_INITER((C_P), (OBJ)), {NULL, 0, 0, 0}, NULL, 0, NULL, 0} + +#define ERTS_IOLIST2BUF_STATE_MOVE(TO, FROM) \ + sys_memcpy((void *) (TO), (void *) (FROM), sizeof(ErtsIOList2BufState)) + +#define ERTS_IOLIST_TO_BUF_BYTES_PER_YIELD_COUNT 32 +#define ERTS_IOLIST_TO_BUF_YIELD_COUNT_PER_RED 8 +#define ERTS_IOLIST_TO_BUF_BYTES_PER_RED \ + (ERTS_IOLIST_TO_BUF_YIELD_COUNT_PER_RED*ERTS_IOLIST_TO_BUF_BYTES_PER_YIELD_COUNT) + +typedef struct { + ErtsIOListState iolist; + struct { + byte *bptr; + size_t size; + Uint bitoffs; + Uint bitsize; + } bcopy; + char *buf; + ErlDrvSizeT len; + Eterm *objp; + int offset; +} ErtsIOList2BufState; + #define ERTS_IOLIST_OK 0 #define ERTS_IOLIST_OVERFLOW 1 #define ERTS_IOLIST_TYPE 2 +#define ERTS_IOLIST_YIELD 3 Eterm buf_to_intlist(Eterm**, const char*, size_t, Eterm); /* most callers pass plain char*'s */ #define ERTS_IOLIST_TO_BUF_OVERFLOW (~((ErlDrvSizeT) 0)) #define ERTS_IOLIST_TO_BUF_TYPE_ERROR (~((ErlDrvSizeT) 1)) +#define ERTS_IOLIST_TO_BUF_YIELD (~((ErlDrvSizeT) 2)) #define ERTS_IOLIST_TO_BUF_FAILED(R) \ - (((R) & (~((ErlDrvSizeT) 1))) == (~((ErlDrvSizeT) 1))) + (((R) & (~((ErlDrvSizeT) 3))) == (~((ErlDrvSizeT) 3))) #define ERTS_IOLIST_TO_BUF_SUCCEEDED(R) \ (!ERTS_IOLIST_TO_BUF_FAILED((R))) ErlDrvSizeT erts_iolist_to_buf(Eterm, char*, ErlDrvSizeT); +ErlDrvSizeT erts_iolist_to_buf_yielding(ErtsIOList2BufState *); +int erts_iolist_size_yielding(ErtsIOListState *state); int erts_iolist_size(Eterm, ErlDrvSizeT *); int is_string(Eterm); void erl_at_exit(void (*) (void*), void*); diff --git a/erts/emulator/beam/io.c b/erts/emulator/beam/io.c index edf4a28784..9ae973e108 100644 --- a/erts/emulator/beam/io.c +++ b/erts/emulator/beam/io.c @@ -1218,9 +1218,10 @@ typedef struct { static ERTS_INLINE ErtsTryImmDrvCallResult try_imm_drv_call(ErtsTryImmDrvCallState *sp) { + unsigned int prof_runnable_ports; ErtsTryImmDrvCallResult res; int reds_left_in; - erts_aint32_t invalid_state, invalid_sched_flags; + erts_aint32_t act, exp, invalid_state, invalid_sched_flags; Port *prt = sp->port; Process *c_p = sp->c_p; @@ -1247,18 +1248,39 @@ try_imm_drv_call(ErtsTryImmDrvCallState *sp) goto locked_fail; } - sp->sched_flags = erts_smp_atomic32_read_nob(&prt->sched.flags); - if (sp->sched_flags & invalid_sched_flags) { - res = ERTS_TRY_IMM_DRV_CALL_INVALID_SCHED_FLAGS; - goto locked_fail; - } + prof_runnable_ports = erts_system_profile_flags.runnable_ports; + if (prof_runnable_ports) + erts_port_task_sched_lock(&prt->sched); + + act = erts_smp_atomic32_read_nob(&prt->sched.flags); + do { + erts_aint32_t new; + + if (act & invalid_sched_flags) { + res = ERTS_TRY_IMM_DRV_CALL_INVALID_SCHED_FLAGS; + sp->sched_flags = act; + goto locked_fail; + } + exp = act; + new = act | ERTS_PTS_FLG_EXEC_IMM; + act = erts_smp_atomic32_cmpxchg_mb(&prt->sched.flags, new, exp); + } while (act != exp); + + sp->sched_flags = act; if (!c_p) reds_left_in = CONTEXT_REDS/10; else { if (IS_TRACED_FL(c_p, F_TRACE_SCHED_PROCS)) trace_virtual_sched(c_p, am_out); + /* + * No status lock held while sending runnable + * proc trace messages. It is however not needed + * in this case, since only this thread can send + * such messages for this process until the process + * has been scheduled out. + */ if (erts_system_profile_flags.runnable_procs && erts_system_profile_flags.exclusive) profile_runnable_proc(c_p, am_inactive); @@ -1273,11 +1295,14 @@ try_imm_drv_call(ErtsTryImmDrvCallState *sp) ERTS_SMP_CHK_NO_PROC_LOCKS; - if (IS_TRACED_FL(prt, F_TRACE_SCHED_PORTS)) - trace_sched_ports_where(prt, am_in, sp->port_op); - if (erts_system_profile_flags.runnable_ports - && !erts_port_is_scheduled(prt)) - profile_runnable_port(prt, am_active); + if (prof_runnable_ports | IS_TRACED_FL(prt, F_TRACE_SCHED_PORTS)) { + if (prof_runnable_ports && !(act & (ERTS_PTS_FLG_IN_RUNQ|ERTS_PTS_FLG_EXEC))) + profile_runnable_port(prt, am_active); + if (IS_TRACED_FL(prt, F_TRACE_SCHED_PORTS)) + trace_sched_ports_where(prt, am_in, sp->port_op); + if (prof_runnable_ports) + erts_port_task_sched_unlock(&prt->sched); + } sp->fpe_was_unmasked = erts_block_fpe(); @@ -1294,17 +1319,31 @@ finalize_imm_drv_call(ErtsTryImmDrvCallState *sp) int reds; Port *prt = sp->port; Process *c_p = sp->c_p; + erts_aint32_t act; + unsigned int prof_runnable_ports; reds = prt->reds; reds += erts_port_driver_callback_epilogue(prt, NULL); erts_unblock_fpe(sp->fpe_was_unmasked); - if (IS_TRACED_FL(prt, F_TRACE_SCHED_PORTS)) - trace_sched_ports_where(prt, am_out, sp->port_op); - if (erts_system_profile_flags.runnable_ports - && !erts_port_is_scheduled(prt)) - profile_runnable_port(prt, am_inactive); + prof_runnable_ports = erts_system_profile_flags.runnable_ports; + if (prof_runnable_ports) + erts_port_task_sched_lock(&prt->sched); + + act = erts_smp_atomic32_read_band_mb(&prt->sched.flags, + ~ERTS_PTS_FLG_EXEC_IMM); + ERTS_SMP_LC_ASSERT(act & ERTS_PTS_FLG_EXEC_IMM); + + if (prof_runnable_ports | IS_TRACED_FL(prt, F_TRACE_SCHED_PORTS)) { + if (IS_TRACED_FL(prt, F_TRACE_SCHED_PORTS)) + trace_sched_ports_where(prt, am_out, sp->port_op); + if (prof_runnable_ports) { + if (!(act & (ERTS_PTS_FLG_IN_RUNQ|ERTS_PTS_FLG_EXEC))) + profile_runnable_port(prt, am_inactive); + erts_port_task_sched_unlock(&prt->sched); + } + } erts_port_release(prt); @@ -1319,6 +1358,13 @@ finalize_imm_drv_call(ErtsTryImmDrvCallState *sp) if (IS_TRACED_FL(c_p, F_TRACE_SCHED_PROCS)) trace_virtual_sched(c_p, am_in); + /* + * No status lock held while sending runnable + * proc trace messages. It is however not needed + * in this case, since only this thread can send + * such messages for this process until the process + * has been scheduled out. + */ if (erts_system_profile_flags.runnable_procs && erts_system_profile_flags.exclusive) profile_runnable_proc(c_p, am_active); @@ -6129,7 +6175,7 @@ driver_pdl_create(ErlDrvPort dp) return NULL; pdl = erts_alloc(ERTS_ALC_T_PORT_DATA_LOCK, sizeof(struct erl_drv_port_data_lock)); - erts_mtx_init(&pdl->mtx, "port_data_lock"); + erts_mtx_init_x(&pdl->mtx, "port_data_lock", pp->common.id, 1); pdl_init_refc(pdl); erts_port_inc_refc(pp); pdl->prt = pp; @@ -7166,7 +7212,7 @@ char *driver_dl_error(void) #define ERL_DRV_SYS_INFO_SIZE(LAST_FIELD) \ - (((size_t) &((ErlDrvSysInfo *) 0)->LAST_FIELD) \ + (offsetof(ErlDrvSysInfo, LAST_FIELD) \ + sizeof(((ErlDrvSysInfo *) 0)->LAST_FIELD)) void @@ -7228,6 +7274,18 @@ driver_system_info(ErlDrvSysInfo *sip, size_t si_size) sip->nif_major_version = ERL_NIF_MAJOR_VERSION; sip->nif_minor_version = ERL_NIF_MINOR_VERSION; } + /* + * 'dirty_scheduler_support' is the last field in the 4th version + * (driver version 3.1, NIF version 2.7) + */ + if (si_size >= ERL_DRV_SYS_INFO_SIZE(dirty_scheduler_support)) { +#if defined(ERL_NIF_DIRTY_SCHEDULER_SUPPORT) && defined(USE_THREADS) + sip->dirty_scheduler_support = 1; +#else + sip->dirty_scheduler_support = 0; +#endif + } + } diff --git a/erts/emulator/beam/sys.h b/erts/emulator/beam/sys.h index 05f07e57b2..3d8dd9c6d0 100644 --- a/erts/emulator/beam/sys.h +++ b/erts/emulator/beam/sys.h @@ -274,6 +274,7 @@ __decl_noreturn void __noreturn erl_assert_error(const char* expr, const char *f typedef unsigned int Eterm; typedef unsigned int Uint; typedef int Sint; +#define ERTS_UINT_MAX UINT_MAX #define ERTS_SIZEOF_ETERM SIZEOF_INT #define ErtsStrToSint strtol #else @@ -347,6 +348,7 @@ typedef long long Sint; typedef Uint UWord; typedef Sint SWord; +#define ERTS_UINT_MAX ERTS_UWORD_MAX #endif /* HALFWORD_HEAP */ diff --git a/erts/emulator/beam/utils.c b/erts/emulator/beam/utils.c index 738f793020..55f9e68e78 100644 --- a/erts/emulator/beam/utils.c +++ b/erts/emulator/beam/utils.c @@ -3197,106 +3197,303 @@ buf_to_intlist(Eterm** hpp, const char *buf, size_t len, Eterm tail) ** */ -ErlDrvSizeT erts_iolist_to_buf(Eterm obj, char* buf, ErlDrvSizeT alloced_len) +typedef enum { + ERTS_IL2B_BCOPY_OK, + ERTS_IL2B_BCOPY_YIELD, + ERTS_IL2B_BCOPY_OVERFLOW, + ERTS_IL2B_BCOPY_TYPE_ERROR +} ErtsIL2BBCopyRes; + +static ErtsIL2BBCopyRes +iolist_to_buf_bcopy(ErtsIOList2BufState *state, Eterm obj, int *yield_countp); + +static ERTS_INLINE ErlDrvSizeT +iolist_to_buf(const int yield_support, + ErtsIOList2BufState *state, + Eterm obj, + char* buf, + ErlDrvSizeT alloced_len) { - ErlDrvSizeT len = (ErlDrvSizeT) alloced_len; - Eterm* objp; +#undef IOLIST_TO_BUF_BCOPY +#define IOLIST_TO_BUF_BCOPY(CONSP) \ +do { \ + size_t size = binary_size(obj); \ + if (size > 0) { \ + Uint bitsize; \ + byte* bptr; \ + Uint bitoffs; \ + Uint num_bits; \ + if (yield_support) { \ + size_t max_size = ERTS_IOLIST_TO_BUF_BYTES_PER_YIELD_COUNT; \ + if (yield_count > 0) \ + max_size *= yield_count+1; \ + if (size > max_size) { \ + state->objp = CONSP; \ + goto L_bcopy_yield; \ + } \ + if (size >= ERTS_IOLIST_TO_BUF_BYTES_PER_YIELD_COUNT) { \ + int cost = (int) size; \ + cost /= ERTS_IOLIST_TO_BUF_BYTES_PER_YIELD_COUNT; \ + yield_count -= cost; \ + } \ + } \ + if (len < size) \ + goto L_overflow; \ + ERTS_GET_BINARY_BYTES(obj, bptr, bitoffs, bitsize); \ + if (bitsize != 0) \ + goto L_type_error; \ + num_bits = 8*size; \ + copy_binary_to_buffer(buf, 0, bptr, bitoffs, num_bits); \ + buf += size; \ + len -= size; \ + } \ +} while (0) + + ErlDrvSizeT res, len; + Eterm* objp = NULL; + int init_yield_count; + int yield_count; DECLARE_ESTACK(s); - goto L_again; - - while (!ESTACK_ISEMPTY(s)) { - obj = ESTACK_POP(s); - L_again: - if (is_list(obj)) { - L_iter_list: - objp = list_val(obj); - obj = CAR(objp); - if (is_byte(obj)) { - if (len == 0) { - goto L_overflow; - } - *buf++ = unsigned_val(obj); - len--; - } else if (is_binary(obj)) { - byte* bptr; - size_t size = binary_size(obj); - Uint bitsize; - Uint bitoffs; - Uint num_bits; - - if (len < size) { + + len = (ErlDrvSizeT) alloced_len; + + if (!yield_support) { + yield_count = init_yield_count = 0; /* Shut up faulty warning... >:-( */ + goto L_again; + } + else { + + if (state->iolist.reds_left <= 0) + return ERTS_IOLIST_TO_BUF_YIELD; + + ESTACK_CHANGE_ALLOCATOR(s, ERTS_ALC_T_SAVED_ESTACK); + init_yield_count = (ERTS_IOLIST_TO_BUF_YIELD_COUNT_PER_RED + * state->iolist.reds_left); + yield_count = init_yield_count; + + if (!state->iolist.estack.start) + goto L_again; + else { + int chk_stack; + /* Restart; restore state... */ + ESTACK_RESTORE(s, &state->iolist.estack); + + if (!state->bcopy.bptr) + chk_stack = 0; + else { + chk_stack = 1; + switch (iolist_to_buf_bcopy(state, THE_NON_VALUE, &yield_count)) { + case ERTS_IL2B_BCOPY_OK: + break; + case ERTS_IL2B_BCOPY_YIELD: + BUMP_ALL_REDS(state->iolist.c_p); + state->iolist.reds_left = 0; + ESTACK_SAVE(s, &state->iolist.estack); + return ERTS_IOLIST_TO_BUF_YIELD; + case ERTS_IL2B_BCOPY_OVERFLOW: goto L_overflow; - } - ERTS_GET_BINARY_BYTES(obj, bptr, bitoffs, bitsize); - if (bitsize != 0) { + case ERTS_IL2B_BCOPY_TYPE_ERROR: goto L_type_error; } - num_bits = 8*size; - copy_binary_to_buffer(buf, 0, bptr, bitoffs, num_bits); - buf += size; - len -= size; - } else if (is_list(obj)) { - ESTACK_PUSH(s, CDR(objp)); - goto L_iter_list; /* on head */ - } else if (is_not_nil(obj)) { - goto L_type_error; } - obj = CDR(objp); - if (is_list(obj)) { - goto L_iter_list; /* on tail */ - } else if (is_binary(obj)) { - byte* bptr; - size_t size = binary_size(obj); - Uint bitsize; - Uint bitoffs; - Uint num_bits; - if (len < size) { - goto L_overflow; + obj = state->iolist.obj; + buf = state->buf; + len = state->len; + objp = state->objp; + state->objp = NULL; + if (objp) + goto L_tail; + if (!chk_stack) + goto L_again; + /* check stack */ + } + } + + while (!ESTACK_ISEMPTY(s)) { + obj = ESTACK_POP(s); + L_again: + if (is_list(obj)) { + while (1) { /* Tail loop */ + while (1) { /* Head loop */ + if (yield_support && --yield_count <= 0) + goto L_yield; + objp = list_val(obj); + obj = CAR(objp); + if (is_byte(obj)) { + if (len == 0) { + goto L_overflow; + } + *buf++ = unsigned_val(obj); + len--; + } else if (is_binary(obj)) { + IOLIST_TO_BUF_BCOPY(objp); + } else if (is_list(obj)) { + ESTACK_PUSH(s, CDR(objp)); + continue; /* Head loop */ + } else if (is_not_nil(obj)) { + goto L_type_error; + } + break; } - ERTS_GET_BINARY_BYTES(obj, bptr, bitoffs, bitsize); - if (bitsize != 0) { + + L_tail: + + obj = CDR(objp); + + if (is_list(obj)) { + continue; /* Tail loop */ + } else if (is_binary(obj)) { + IOLIST_TO_BUF_BCOPY(NULL); + } else if (is_not_nil(obj)) { goto L_type_error; } - num_bits = 8*size; - copy_binary_to_buffer(buf, 0, bptr, bitoffs, num_bits); - buf += size; - len -= size; - } else if (is_not_nil(obj)) { - goto L_type_error; + break; } } else if (is_binary(obj)) { - byte* bptr; - size_t size = binary_size(obj); - Uint bitsize; - Uint bitoffs; - Uint num_bits; - if (len < size) { - goto L_overflow; - } - ERTS_GET_BINARY_BYTES(obj, bptr, bitoffs, bitsize); - if (bitsize != 0) { - goto L_type_error; - } - num_bits = 8*size; - copy_binary_to_buffer(buf, 0, bptr, bitoffs, num_bits); - buf += size; - len -= size; + IOLIST_TO_BUF_BCOPY(NULL); } else if (is_not_nil(obj)) { goto L_type_error; - } + } else if (yield_support && --yield_count <= 0) + goto L_yield; } + res = len; + + L_return: + DESTROY_ESTACK(s); - return len; + + if (yield_support) { + int reds; + CLEAR_SAVED_ESTACK(&state->iolist.estack); + reds = ((init_yield_count - yield_count - 1) + / ERTS_IOLIST_TO_BUF_YIELD_COUNT_PER_RED) + 1; + BUMP_REDS(state->iolist.c_p, reds); + state->iolist.reds_left -= reds; + if (state->iolist.reds_left < 0) + state->iolist.reds_left = 0; + } + + + return res; L_type_error: - DESTROY_ESTACK(s); - return ERTS_IOLIST_TO_BUF_TYPE_ERROR; + res = ERTS_IOLIST_TO_BUF_TYPE_ERROR; + goto L_return; L_overflow: - DESTROY_ESTACK(s); - return ERTS_IOLIST_TO_BUF_OVERFLOW; + res = ERTS_IOLIST_TO_BUF_OVERFLOW; + goto L_return; + + L_bcopy_yield: + + state->buf = buf; + state->len = len; + + switch (iolist_to_buf_bcopy(state, obj, &yield_count)) { + case ERTS_IL2B_BCOPY_OK: + ERTS_INTERNAL_ERROR("Missing yield"); + case ERTS_IL2B_BCOPY_YIELD: + BUMP_ALL_REDS(state->iolist.c_p); + state->iolist.reds_left = 0; + ESTACK_SAVE(s, &state->iolist.estack); + return ERTS_IOLIST_TO_BUF_YIELD; + case ERTS_IL2B_BCOPY_OVERFLOW: + goto L_overflow; + case ERTS_IL2B_BCOPY_TYPE_ERROR: + goto L_type_error; + } + + L_yield: + + BUMP_ALL_REDS(state->iolist.c_p); + state->iolist.reds_left = 0; + state->iolist.obj = obj; + state->buf = buf; + state->len = len; + ESTACK_SAVE(s, &state->iolist.estack); + return ERTS_IOLIST_TO_BUF_YIELD; + +#undef IOLIST_TO_BUF_BCOPY +} + +static ErtsIL2BBCopyRes +iolist_to_buf_bcopy(ErtsIOList2BufState *state, Eterm obj, int *yield_countp) +{ + ErtsIL2BBCopyRes res; + char *buf = state->buf; + ErlDrvSizeT len = state->len; + byte* bptr; + size_t size; + size_t max_size; + Uint bitoffs; + Uint num_bits; + int yield_count = *yield_countp; + + if (state->bcopy.bptr) { + bptr = state->bcopy.bptr; + size = state->bcopy.size; + bitoffs = state->bcopy.bitoffs; + state->bcopy.bptr = NULL; + } + else { + Uint bitsize; + + ASSERT(is_binary(obj)); + + size = binary_size(obj); + if (size <= 0) + return ERTS_IL2B_BCOPY_OK; + + if (len < size) + return ERTS_IL2B_BCOPY_OVERFLOW; + + ERTS_GET_BINARY_BYTES(obj, bptr, bitoffs, bitsize); + if (bitsize != 0) + return ERTS_IL2B_BCOPY_TYPE_ERROR; + } + + ASSERT(size > 0); + max_size = (size_t) ERTS_IOLIST_TO_BUF_BYTES_PER_YIELD_COUNT; + if (yield_count > 0) + max_size *= (size_t) (yield_count+1); + + if (size <= max_size) { + if (size >= ERTS_IOLIST_TO_BUF_BYTES_PER_YIELD_COUNT) { + int cost = (int) size; + cost /= ERTS_IOLIST_TO_BUF_BYTES_PER_YIELD_COUNT; + yield_count -= cost; + } + res = ERTS_IL2B_BCOPY_OK; + } + else { + ASSERT(0 < max_size && max_size < size); + yield_count = 0; + state->bcopy.bptr = bptr + max_size; + state->bcopy.bitoffs = bitoffs; + state->bcopy.size = size - max_size; + size = max_size; + res = ERTS_IL2B_BCOPY_YIELD; + } + + num_bits = 8*size; + copy_binary_to_buffer(buf, 0, bptr, bitoffs, num_bits); + state->buf += size; + state->len -= size; + *yield_countp = yield_count; + + return res; +} + +ErlDrvSizeT erts_iolist_to_buf_yielding(ErtsIOList2BufState *state) +{ + return iolist_to_buf(1, state, state->iolist.obj, state->buf, state->len); +} + +ErlDrvSizeT erts_iolist_to_buf(Eterm obj, char* buf, ErlDrvSizeT alloced_len) +{ + return iolist_to_buf(0, NULL, obj, buf, alloced_len); } /* @@ -3307,11 +3504,32 @@ ErlDrvSizeT erts_iolist_to_buf(Eterm obj, char* buf, ErlDrvSizeT alloced_len) * Any input term error detected in erts_iolist_to_buf should also * be detected in this function! */ -int erts_iolist_size(Eterm obj, ErlDrvSizeT* sizep) + +static ERTS_INLINE int +iolist_size(const int yield_support, ErtsIOListState *state, Eterm obj, ErlDrvSizeT* sizep) { + int res, init_yield_count, yield_count; Eterm* objp; - Uint size = 0; /* Intentionally Uint due to halfword heap */ + Uint size = (Uint) *sizep; /* Intentionally Uint due to halfword heap */ DECLARE_ESTACK(s); + + if (!yield_support) + yield_count = init_yield_count = 0; /* Shut up faulty warning... >:-( */ + else { + if (state->reds_left <= 0) + return ERTS_IOLIST_YIELD; + ESTACK_CHANGE_ALLOCATOR(s, ERTS_ALC_T_SAVED_ESTACK); + init_yield_count = ERTS_IOLIST_SIZE_YIELDS_COUNT_PER_RED; + init_yield_count *= state->reds_left; + yield_count = init_yield_count; + if (state->estack.start) { + /* Restart; restore state... */ + ESTACK_RESTORE(s, &state->estack); + size = (Uint) state->size; + obj = state->obj; + } + } + goto L_again; #define SAFE_ADD(Var, Val) \ @@ -3327,51 +3545,101 @@ int erts_iolist_size(Eterm obj, ErlDrvSizeT* sizep) obj = ESTACK_POP(s); L_again: if (is_list(obj)) { - L_iter_list: - objp = list_val(obj); - /* Head */ - obj = CAR(objp); - if (is_byte(obj)) { - size++; - if (size == 0) { - goto L_overflow_error; + while (1) { /* Tail loop */ + while (1) { /* Head loop */ + if (yield_support && --yield_count <= 0) + goto L_yield; + objp = list_val(obj); + /* Head */ + obj = CAR(objp); + if (is_byte(obj)) { + size++; + if (size == 0) { + goto L_overflow_error; + } + } else if (is_binary(obj) && binary_bitsize(obj) == 0) { + SAFE_ADD(size, binary_size(obj)); + } else if (is_list(obj)) { + ESTACK_PUSH(s, CDR(objp)); + continue; /* Head loop */ + } else if (is_not_nil(obj)) { + goto L_type_error; + } + break; } - } else if (is_binary(obj) && binary_bitsize(obj) == 0) { - SAFE_ADD(size, binary_size(obj)); - } else if (is_list(obj)) { - ESTACK_PUSH(s, CDR(objp)); - goto L_iter_list; /* on head */ - } else if (is_not_nil(obj)) { - goto L_type_error; + /* Tail */ + obj = CDR(objp); + if (is_list(obj)) + continue; /* Tail loop */ + else if (is_binary(obj) && binary_bitsize(obj) == 0) { + SAFE_ADD(size, binary_size(obj)); + } else if (is_not_nil(obj)) { + goto L_type_error; + } + break; } - /* Tail */ - obj = CDR(objp); - if (is_list(obj)) - goto L_iter_list; /* on tail */ - else if (is_binary(obj) && binary_bitsize(obj) == 0) { + } else { + if (yield_support && --yield_count <= 0) + goto L_yield; + if (is_binary(obj) && binary_bitsize(obj) == 0) { /* Tail was binary */ SAFE_ADD(size, binary_size(obj)); } else if (is_not_nil(obj)) { goto L_type_error; } - } else if (is_binary(obj) && binary_bitsize(obj) == 0) { /* Tail was binary */ - SAFE_ADD(size, binary_size(obj)); - } else if (is_not_nil(obj)) { - goto L_type_error; } } #undef SAFE_ADD - DESTROY_ESTACK(s); *sizep = (ErlDrvSizeT) size; - return ERTS_IOLIST_OK; - L_overflow_error: + res = ERTS_IOLIST_OK; + + L_return: + DESTROY_ESTACK(s); - return ERTS_IOLIST_OVERFLOW; + + if (yield_support) { + int yc, reds; + CLEAR_SAVED_ESTACK(&state->estack); + yc = init_yield_count - yield_count; + reds = ((yc - 1) / ERTS_IOLIST_SIZE_YIELDS_COUNT_PER_RED) + 1; + BUMP_REDS(state->c_p, reds); + state->reds_left -= reds; + state->size = (ErlDrvSizeT) size; + state->have_size = 1; + } + + return res; + + L_overflow_error: + res = ERTS_IOLIST_OVERFLOW; + size = 0; + goto L_return; L_type_error: - DESTROY_ESTACK(s); - return ERTS_IOLIST_TYPE; + res = ERTS_IOLIST_TYPE; + size = 0; + goto L_return; + + L_yield: + BUMP_ALL_REDS(state->c_p); + state->reds_left = 0; + state->size = size; + state->obj = obj; + ESTACK_SAVE(s, &state->estack); + return ERTS_IOLIST_YIELD; +} + +int erts_iolist_size_yielding(ErtsIOListState *state) +{ + ErlDrvSizeT size = state->size; + return iolist_size(1, state, state->obj, &size); +} + +int erts_iolist_size(Eterm obj, ErlDrvSizeT* sizep) +{ + *sizep = 0; + return iolist_size(0, NULL, obj, sizep); } /* return 0 if item is not a non-empty flat list of bytes */ @@ -3680,6 +3948,9 @@ erts_save_emu_args(int argc, char **argv) size += sz+1; } ptr = (char *) malloc(size); + if (!ptr) { + ERTS_INTERNAL_ERROR("malloc failed to allocate memory!"); + } #ifdef DEBUG end_ptr = ptr + size; #endif diff --git a/erts/emulator/drivers/common/inet_drv.c b/erts/emulator/drivers/common/inet_drv.c index 357a4b7bcb..891589d1c5 100644 --- a/erts/emulator/drivers/common/inet_drv.c +++ b/erts/emulator/drivers/common/inet_drv.c @@ -93,6 +93,10 @@ typedef unsigned long long llu_t; #define INT16_MAX (32767) #endif +#ifdef __OSE__ +#include "inet.h" +#endif + #ifdef __WIN32__ #define STRNCASECMP strncasecmp @@ -121,10 +125,6 @@ typedef unsigned long long llu_t; #undef WANT_NONBLOCKING #include "sys.h" -#ifdef __OSE__ -#include "inet.h" -#endif - #undef EWOULDBLOCK #undef ETIMEDOUT @@ -200,6 +200,7 @@ typedef unsigned long long llu_t; #define HAVE_MULTICAST_SUPPORT +#define HAVE_UDP #define ERRNO_BLOCK WSAEWOULDBLOCK @@ -294,29 +295,55 @@ static unsigned long zero_value = 0; static unsigned long one_value = 1; #elif defined (__OSE__) + +/* + * Some notes about how inet (currently only tcp) works on OSE. + * The driver uses OSE signals to communicate with the one_inet + * process. Because of the difference in how signals and file descriptors + * work the whole select/deselect mechanic is very different. + * In ose when a sock_select is done a function is called. That function + * notes the changes that the driver want to do, but does not act on it. + * later when the function returns the new desired state is compared + * to the previous state and the apprioriate actions are taken. The action + * is usually to either request more data from the stack or stop requesting + * data. + * + * One thing to note is that the driver never does select/deselect. It always + * listens for the signals. Flow of data is regulated by sending or not sending + * signals to the ose inet process. + * + * The interesting functions to look at are: + * * inet_driver_select : called when sock_select is called + * * tcp_inet_ose_dispatch_signal : checks state changes and sends new signals + * * tcp_inet_drv_output_ose : ready output callback, reads signals and calls + * dispatch_signal + * * tcp_inet_drv_input_ose : ready input callback. + */ + +#include "efs.h" #include "sys/socket.h" #include "sys/uio.h" #include "sfk/sys/sfk_uio.h" #include "netinet/in.h" #include "netinet/tcp.h" #include "netdb.h" +#include "ose_spi/socket.sig" -ssize_t writev(int fd, const struct iovec *iov, int iovcnt) -{ - return 0; -} + +static ssize_t writev_fallback(int fd, const struct iovec *iov, int iovcnt, int max_sz); #define INVALID_SOCKET -1 #define INVALID_EVENT -1 #define SOCKET_ERROR -1 #define SOCKET int -#define HANDLE long int -#define FD_READ ERL_DRV_READ -#define FD_WRITE ERL_DRV_WRITE -#define FD_CLOSE 0 -#define FD_CONNECT ERL_DRV_WRITE -#define FD_ACCEPT ERL_DRV_READ +#define HANDLE int +#define FD_READ ERL_DRV_READ +#define FD_WRITE ERL_DRV_WRITE +#define FD_CLOSE 0 +#define FD_CONNECT (1<<4) +#define FD_ACCEPT (1<<5) +#define SOCK_FD_ERROR (1<<6) #define sock_connect(s, addr, len) connect((s), (addr), (len)) #define sock_listen(s, b) listen((s), (b)) @@ -335,11 +362,12 @@ ssize_t writev(int fd, const struct iovec *iov, int iovcnt) #define sock_sendto(s,buf,blen,flag,addr,alen) \ sendto((s),(buf),(blen),(flag),(addr),(alen)) #define sock_sendv(s, vec, size, np, flag) \ - (*(np) = writev((s), (struct iovec*)(vec), (size))) + (*(np) = writev_fallback((s), (struct iovec*)(vec), (size), (*(np)))) #define sock_sendmsg(s,msghdr,flag) sendmsg((s),(msghdr),(flag)) #define sock_open(af, type, proto) socket((af), (type), (proto)) #define sock_close(s) close((s)) +#define sock_dup(s) dup((s)) #define sock_shutdown(s, how) shutdown((s), (how)) #define sock_hostname(buf, len) gethostname((buf), (len)) @@ -355,19 +383,6 @@ ssize_t writev(int fd, const struct iovec *iov, int iovcnt) #define sock_create_event(d) ((d)->s) /* return file descriptor */ #define sock_close_event(e) /* do nothing */ -#define inet_driver_select(port, e, mode, on) \ - driver_select(port, e, mode | (on?ERL_DRV_USE:0), on) - -#define sock_select(d, flags, onoff) do { \ - ASSERT(!(d)->is_ignored); \ - (d)->event_mask = (onoff) ? \ - ((d)->event_mask | (flags)) : \ - ((d)->event_mask & ~(flags)); \ - DEBUGF(("sock_select(%ld): flags=%02X, onoff=%d, event_mask=%02lX\r\n", \ - (long) (d)->port, (flags), (onoff), (unsigned long) (d)->event_mask)); \ - inet_driver_select((d)->port, (ErlDrvEvent)(long)(d)->event, (flags), (onoff)); \ - } while(0) - #ifndef WANT_NONBLOCKING #define WANT_NONBLOCKING #endif @@ -397,6 +412,16 @@ typedef unsigned long u_long; #define IN_EXPERIMENTAL(a) ((((in_addr_t)(a)) & 0xe0000000) == 0xe0000000) #define IN_BADCLASS(a) ((((in_addr_t)(a)) & 0xf0000000) == 0xf0000000) +#define sock_select(d, flags, onoff) do { \ + ASSERT(!(d)->is_ignored); \ + (d)->event_mask = (onoff) ? \ + ((d)->event_mask | (flags)) : \ + ((d)->event_mask & ~(flags)); \ + DEBUGF(("(%s / %d) sock_select(%ld): flags=%02X, onoff=%d, event_mask=%02lX, s=%d\r\n", \ + __FILE__, __LINE__, (long) (d)->port, (flags), (onoff), (unsigned long) (d)->event_mask, (d)->s)); \ + inet_driver_select((d), (flags), (onoff)); \ + } while(0) + #else /* !__OSE__ && !__WIN32__ */ #include <sys/time.h> @@ -438,6 +463,8 @@ typedef unsigned long u_long; #include <setns.h> #endif +#define HAVE_UDP + /* SCTP support -- currently for UNIX platforms only: */ #undef HAVE_SCTP #if defined(HAVE_SCTP_H) @@ -660,13 +687,13 @@ static int my_strncasecmp(const char *s1, const char *s2, size_t n) (d)->event_mask = (onoff) ? \ ((d)->event_mask | (flags)) : \ ((d)->event_mask & ~(flags)); \ - DEBUGF(("sock_select(%ld): flags=%02X, onoff=%d, event_mask=%02lX\r\n", \ - (long) (d)->port, (flags), (onoff), (unsigned long) (d)->event_mask)); \ + DEBUGF(("(%s / %d) sock_select(%ld): flags=%02X, onoff=%d, event_mask=%02lX\r\n", \ + __FILE__, __LINE__, (long) (d)->port, (flags), (onoff), (unsigned long) (d)->event_mask)); \ inet_driver_select((d)->port, (ErlDrvEvent)(long)(d)->event, (flags), (onoff)); \ } while(0) -#endif /* #ifdef __WIN32__ #else */ +#endif /* !__WIN32__ && !__OSE__ */ #ifdef HAVE_SOCKLEN_T # define SOCKLEN_T socklen_t @@ -1119,6 +1146,13 @@ typedef struct { char *netns; /* Socket network namespace name as full file path */ #endif +#ifdef __OSE__ + int select_state; /* state to keep track of whether we + should trigger another read/write + request at end of ready_input/output */ + ErlDrvEvent events[6]; +#endif + } inet_descriptor; @@ -1134,8 +1168,10 @@ static void tcp_inet_stop(ErlDrvData); static void tcp_inet_command(ErlDrvData, char*, ErlDrvSizeT); static void tcp_inet_commandv(ErlDrvData, ErlIOVec*); static void tcp_inet_flush(ErlDrvData drv_data); +#ifndef __OSE__ static void tcp_inet_drv_input(ErlDrvData, ErlDrvEvent); static void tcp_inet_drv_output(ErlDrvData data, ErlDrvEvent event); +#endif static ErlDrvData tcp_inet_start(ErlDrvPort, char* command); static ErlDrvSSizeT tcp_inet_ctl(ErlDrvData, unsigned int, char*, ErlDrvSizeT, char**, ErlDrvSizeT); @@ -1147,7 +1183,72 @@ static void tcp_inet_event(ErlDrvData, ErlDrvEvent); static void find_dynamic_functions(void); #endif -static struct erl_drv_entry tcp_inet_driver_entry = +#ifdef __OSE__ +/* The structure of the signal used for requesting asynchronous + * notification from the stack. Under normal circumstances the network stack + * shouldn't overwrite the value set in the fd field by the sender + * of the request */ +struct OseAsyncSig { + struct FmEvent event; + int fd; +}; + +union SIGNAL { + SIGSELECT signo; + struct OseAsyncSig async; +}; + +static ErlDrvSSizeT tcp_inet_ctl_ose(ErlDrvData e, unsigned int cmd, + char* buf, ErlDrvSizeT len, + char** rbuf, ErlDrvSizeT rsize); +static void tcp_inet_commandv_ose(ErlDrvData e, ErlIOVec* ev); +static void tcp_inet_drv_output_ose(ErlDrvData data, ErlDrvEvent event); +static void tcp_inet_drv_input_ose(ErlDrvData data, ErlDrvEvent event); +static ErlDrvOseEventId inet_resolve_signal(union SIGNAL *sig); + +#ifdef INET_DRV_DEBUG + +static char *read_req = "SO_EVENT_READ_REQUEST"; +static char *read_rep = "SO_EVENT_READ_REPLY"; +static char *write_req = "SO_EVENT_WRITE_REQUEST"; +static char *write_rep = "SO_EVENT_WRITE_REPLY"; +static char *eof_req = "SO_EVENT_EOF_REQUEST"; +static char *eof_rep = "SO_EVENT_EOF_REPLY"; +static char *accept_req = "SO_EVENT_ACCEPT_REQUEST"; +static char *accept_rep = "SO_EVENT_ACCEPT_REPLY"; +static char *connect_req = "SO_EVENT_CONNECT_REQUEST"; +static char *connect_rep = "SO_EVENT_CONNECT_REPLY"; +static char *error_req = "SO_EVENT_ERROR_REQUEST"; +static char *error_rep = "SO_EVENT_ERROR_REPLY"; +static char signo_tmp[32]; + +static char *signo_to_string(SIGSELECT signo) { + switch (signo) { + case SO_EVENT_READ_REQUEST: { return read_req; } + case SO_EVENT_READ_REPLY: { return read_rep; } + case SO_EVENT_WRITE_REQUEST: { return write_req; } + case SO_EVENT_WRITE_REPLY: { return write_rep; } + case SO_EVENT_EOF_REQUEST: { return eof_req; } + case SO_EVENT_EOF_REPLY: { return eof_rep; } + case SO_EVENT_ACCEPT_REQUEST: { return accept_req; } + case SO_EVENT_ACCEPT_REPLY: { return accept_rep; } + case SO_EVENT_CONNECT_REQUEST: { return connect_req; } + case SO_EVENT_CONNECT_REPLY: { return connect_rep; } + case SO_EVENT_ERROR_REQUEST: { return error_req; } + case SO_EVENT_ERROR_REPLY: { return error_rep; } + } + + snprintf(signo_tmp,32,"0x%x",signo); + + return signo_tmp; +} + +#endif + +#endif /* __OSE__ */ + + +static struct erl_drv_entry tcp_inet_driver_entry = { tcp_inet_init, /* inet_init will add this driver !! */ tcp_inet_start, @@ -1156,6 +1257,9 @@ static struct erl_drv_entry tcp_inet_driver_entry = #ifdef __WIN32__ tcp_inet_event, NULL, +#elif defined(__OSE__) + tcp_inet_drv_input_ose, /*ready_input*/ + tcp_inet_drv_output_ose, /*ready_output*/ #else tcp_inet_drv_input, tcp_inet_drv_output, @@ -1163,9 +1267,17 @@ static struct erl_drv_entry tcp_inet_driver_entry = "tcp_inet", NULL, NULL, +#ifdef __OSE__ + tcp_inet_ctl_ose, +#else tcp_inet_ctl, +#endif tcp_inet_timeout, +#ifdef __OSE__ + tcp_inet_commandv_ose, +#else tcp_inet_commandv, +#endif NULL, tcp_inet_flush, NULL, @@ -1181,6 +1293,7 @@ static struct erl_drv_entry tcp_inet_driver_entry = +#ifdef HAVE_UDP static int packet_inet_init(void); static void packet_inet_stop(ErlDrvData); static void packet_inet_command(ErlDrvData, char*, ErlDrvSizeT); @@ -1230,6 +1343,7 @@ static struct erl_drv_entry udp_inet_driver_entry = NULL, inet_stop_select }; +#endif #ifdef HAVE_SCTP static struct erl_drv_entry sctp_inet_driver_entry = @@ -1293,6 +1407,7 @@ static int tcp_deliver(tcp_descriptor* desc, int len); static int tcp_inet_output(tcp_descriptor* desc, HANDLE event); static int tcp_inet_input(tcp_descriptor* desc, HANDLE event); +#ifdef HAVE_UDP typedef struct { inet_descriptor inet; /* common data structure (DON'T MOVE) */ int read_packets; /* Number of packets to read per invocation */ @@ -1304,10 +1419,19 @@ typedef struct { static int packet_inet_input(udp_descriptor* udesc, HANDLE event); static int packet_inet_output(udp_descriptor* udesc, HANDLE event); +#endif /* convert descriptor poiner to inet_descriptor pointer */ #define INETP(d) (&(d)->inet) +#ifdef __OSE__ +static void inet_driver_select(inet_descriptor* desc, + int flags, int onoff); +static void tcp_inet_ose_dispatch_signals(tcp_descriptor *desc, + int prev_select_state, + union SIGNAL *sig); +#endif + static int async_ref = 0; /* async reference id generator */ #define NEW_ASYNC_ID() ((async_ref++) & 0xffff) @@ -1324,7 +1448,6 @@ static int async_ref = 0; /* async reference id generator */ static ErlDrvTermData am_ok; static ErlDrvTermData am_tcp; -static ErlDrvTermData am_udp; static ErlDrvTermData am_error; static ErlDrvTermData am_einval; static ErlDrvTermData am_inet_async; @@ -1334,10 +1457,13 @@ static ErlDrvTermData am_closed; static ErlDrvTermData am_tcp_passive; static ErlDrvTermData am_tcp_closed; static ErlDrvTermData am_tcp_error; -static ErlDrvTermData am_udp_passive; -static ErlDrvTermData am_udp_error; static ErlDrvTermData am_empty_out_q; static ErlDrvTermData am_ssl_tls; +#ifdef HAVE_UDP +static ErlDrvTermData am_udp; +static ErlDrvTermData am_udp_passive; +static ErlDrvTermData am_udp_error; +#endif #ifdef HAVE_SCTP static ErlDrvTermData am_sctp; static ErlDrvTermData am_sctp_passive; @@ -1545,6 +1671,7 @@ static void *realloc_wrapper(void *current, ErlDrvSizeT size){ # define SCTP_ANC_BUFF_SIZE INET_DEF_BUFFER/2 /* XXX: not very good... */ #endif +#ifdef HAVE_UDP static int load_ip_port(ErlDrvTermData* spec, int i, char* buf) { spec[i++] = ERL_DRV_INT; @@ -1579,6 +1706,7 @@ static int load_ip_address(ErlDrvTermData* spec, int i, int family, char* buf) } return i; } +#endif #ifdef HAVE_SCTP @@ -1745,10 +1873,12 @@ static void release_buffer(ErlDrvBinary* buf) } } +#ifdef HAVE_UDP static ErlDrvBinary* realloc_buffer(ErlDrvBinary* buf, ErlDrvSizeT newsz) { return driver_realloc_binary(buf, newsz); } +#endif /* use a TRICK, access the refc field to see if any one else has * a ref to this buffer then call driver_free_binary else @@ -3409,6 +3539,7 @@ static int tcp_error_message(tcp_descriptor* desc, int err) return erl_drv_output_term(desc->inet.dport, spec, i); } +#ifdef HAVE_UDP /* ** active mode message: ** {udp, S, IP, Port, [H1,...Hsz | Data]} or @@ -3499,6 +3630,7 @@ static int packet_binary_message ASSERT(i <= PACKET_ERL_DRV_TERM_DATA_LEN); return erl_drv_output_term(desc->dport, spec, i); } +#endif /* ** active mode message: send active-to-passive transition message @@ -3513,6 +3645,9 @@ static int packet_binary_message DEBUGF(("packet_passive_message(%ld):\r\n", (long)desc->port)); +#if !defined(HAVE_UDP) && !defined(HAVE_SCTP) + i = LOAD_ATOM(spec, i, am_tcp_passive); +#else if (desc->sprotocol == IPPROTO_TCP) i = LOAD_ATOM(spec, i, am_tcp_passive); else { @@ -3522,12 +3657,14 @@ static int packet_binary_message i = LOAD_ATOM(spec, i, am_udp_passive); #endif } +#endif i = LOAD_PORT(spec, i, desc->dport); i = LOAD_TUPLE(spec, i, 2); ASSERT(i <= 6); return erl_drv_output_term(desc->dport, spec, i); } +#ifdef HAVE_UDP /* ** send active message {udp_error|sctp_error, S, Error} */ @@ -3554,7 +3691,7 @@ static int packet_error_message(udp_descriptor* udesc, int err) ASSERT(i == sizeof(spec)/sizeof(*spec)); return erl_drv_output_term(desc->dport, spec, i); } - +#endif /* ** active=TRUE: @@ -3619,7 +3756,7 @@ tcp_reply_binary_data(tcp_descriptor* desc, ErlDrvBinary* bin, int offs, int len return code; } - +#ifdef HAVE_UDP static int packet_reply_binary_data(inet_descriptor* desc, unsigned int hsz, ErlDrvBinary * bin, int offs, int len, @@ -3642,6 +3779,7 @@ packet_reply_binary_data(inet_descriptor* desc, unsigned int hsz, return code; } } +#endif /* ---------------------------------------------------------------------------- @@ -3817,7 +3955,9 @@ static int inet_init() INIT_ATOM(ok); INIT_ATOM(tcp); +#ifdef HAVE_UDP INIT_ATOM(udp); +#endif INIT_ATOM(error); INIT_ATOM(einval); INIT_ATOM(inet_async); @@ -3827,8 +3967,10 @@ static int inet_init() INIT_ATOM(tcp_passive); INIT_ATOM(tcp_closed); INIT_ATOM(tcp_error); +#ifdef HAVE_UDP INIT_ATOM(udp_passive); INIT_ATOM(udp_error); +#endif INIT_ATOM(empty_out_q); INIT_ATOM(ssl_tls); @@ -3847,7 +3989,10 @@ static int inet_init() /* add TCP, UDP and SCTP drivers */ add_driver_entry(&tcp_inet_driver_entry); +#ifdef HAVE_UDP add_driver_entry(&udp_inet_driver_entry); +#endif + #ifdef HAVE_SCTP /* Check the size of SCTP AssocID -- currently both this driver and the Erlang part require 32 bit: */ @@ -4176,6 +4321,16 @@ static void desc_close(inet_descriptor* desc) desc->forced_events = 0; desc->send_would_block = 0; #endif +#ifdef __OSE__ + if (desc->events[0]) { + driver_select(desc->port,desc->events[0],FD_READ|FD_WRITE|ERL_DRV_USE,0); + driver_select(desc->port,desc->events[1],FD_READ|FD_WRITE|ERL_DRV_USE,0); + driver_select(desc->port,desc->events[2],FD_READ|FD_WRITE|ERL_DRV_USE,0); + driver_select(desc->port,desc->events[3],FD_READ|FD_WRITE|ERL_DRV_USE,0); + driver_select(desc->port,desc->events[4],FD_READ|FD_WRITE|ERL_DRV_USE,0); + driver_select(desc->port,desc->events[5],FD_READ|FD_WRITE|ERL_DRV_USE,0); + } +#else /* * We should close the fd here, but the other driver might still * be selecting on it. @@ -4185,6 +4340,7 @@ static void desc_close(inet_descriptor* desc) ERL_DRV_USE, 0); else inet_stop_select((ErlDrvEvent)(long)desc->event,NULL); +#endif desc->event = INVALID_EVENT; /* closed by stop_select callback */ desc->s = INVALID_SOCKET; desc->event_mask = 0; @@ -4216,7 +4372,7 @@ static int erl_inet_close(inet_descriptor* desc) desc_close(desc); desc->state = INET_STATE_CLOSED; } else if (desc->prebound && (desc->s != INVALID_SOCKET)) { - sock_select(desc, FD_READ | FD_WRITE | FD_CLOSE, 0); + sock_select(desc, FD_READ | FD_WRITE | FD_CLOSE | ERL_DRV_USE_NO_CALLBACK, 0); desc->event_mask = 0; #ifdef __WIN32__ desc->forced_events = 0; @@ -4226,6 +4382,64 @@ static int erl_inet_close(inet_descriptor* desc) return 0; } +#ifdef __OSE__ +static void inet_select_init(inet_descriptor* desc) +{ + desc->events[0] = + erl_drv_ose_event_alloc(SO_EVENT_READ_REPLY, + desc->s, + inet_resolve_signal, + NULL); + driver_select(desc->port, desc->events[0], + ERL_DRV_READ|ERL_DRV_USE, 1); + + desc->events[1] = + erl_drv_ose_event_alloc(SO_EVENT_EOF_REPLY, + desc->s, + inet_resolve_signal, + NULL); + driver_select(desc->port, desc->events[1], + ERL_DRV_READ|ERL_DRV_USE, 1); + + desc->events[2] = + erl_drv_ose_event_alloc(SO_EVENT_ACCEPT_REPLY, + desc->s, + inet_resolve_signal, + NULL); + driver_select(desc->port, desc->events[2], + ERL_DRV_READ|ERL_DRV_USE, 1); + + /* trigger tcp_inet_input */ + desc->events[3] = + erl_drv_ose_event_alloc(SO_EVENT_WRITE_REPLY, + desc->s, + inet_resolve_signal, + NULL); + driver_select(desc->port, desc->events[3], + ERL_DRV_WRITE|ERL_DRV_USE, 1); + + desc->events[4] = + erl_drv_ose_event_alloc(SO_EVENT_CONNECT_REPLY, + desc->s, + inet_resolve_signal, + NULL); + driver_select(desc->port, desc->events[4], + ERL_DRV_WRITE|ERL_DRV_USE, 1); + + desc->events[5] = + erl_drv_ose_event_alloc(SO_EVENT_ERROR_REPLY, + desc->s, + inet_resolve_signal, + NULL); + driver_select(desc->port, desc->events[5], + ERL_DRV_WRITE|ERL_DRV_USE, 1); + + /* Issue a select on error event before any other select to be sure we are + prepared to receive error notifications from the stack, even in the + situations when select isn't issued */ + sock_select(desc, SOCK_FD_ERROR, 1); +} +#endif static ErlDrvSSizeT inet_ctl_open(inet_descriptor* desc, int domain, int type, char** rbuf, ErlDrvSizeT rsize) @@ -4309,6 +4523,10 @@ static ErlDrvSSizeT inet_ctl_open(inet_descriptor* desc, int domain, int type, #ifdef __WIN32__ driver_select(desc->port, desc->event, ERL_DRV_READ, 1); #endif +#ifdef __OSE__ + inet_select_init(desc); +#endif + desc->state = INET_STATE_OPEN; desc->stype = type; desc->sfamily = domain; @@ -4318,7 +4536,8 @@ static ErlDrvSSizeT inet_ctl_open(inet_descriptor* desc, int domain, int type, /* as inet_open but pass in an open socket (MUST BE OF RIGHT TYPE) */ static ErlDrvSSizeT inet_ctl_fdopen(inet_descriptor* desc, int domain, int type, - SOCKET s, char** rbuf, ErlDrvSizeT rsize) + SOCKET s, Uint32 bound, + char** rbuf, ErlDrvSizeT rsize) { inet_address name; unsigned int sz = sizeof(name); @@ -4328,18 +4547,37 @@ static ErlDrvSSizeT inet_ctl_fdopen(inet_descriptor* desc, int domain, int type, return ctl_error(sock_errno(), rbuf, rsize); if (name.sa.sa_family != domain) return ctl_error(EINVAL, rbuf, rsize); +#ifdef __OSE__ + /* for fdopen duplicating the sd will allow to uniquely identify + the signal from OSE with erlang port */ + desc->s = sock_dup(s); +#else desc->s = s; +#endif + if ((desc->event = sock_create_event(desc)) == INVALID_EVENT) return ctl_error(sock_errno(), rbuf, rsize); SET_NONBLOCKING(desc->s); #ifdef __WIN32__ driver_select(desc->port, desc->event, ERL_DRV_READ, 1); #endif - desc->state = INET_STATE_BOUND; /* assume bound */ + + if (bound) + desc->state = INET_STATE_BOUND; + else + desc->state = INET_STATE_OPEN; + if (type == SOCK_STREAM) { /* check if connected */ sz = sizeof(name); - if (!IS_SOCKET_ERROR(sock_peer(s, (struct sockaddr*) &name, &sz))) + if (!IS_SOCKET_ERROR(sock_peer(s, (struct sockaddr*) &name, &sz))) { desc->state = INET_STATE_CONNECTED; +#ifdef __OSE__ + /* since we are dealing with different descriptors (i.e. inet and + socket) the select part should be initialized with the right + values */ + inet_select_init(desc); +#endif + } } desc->prebound = 1; /* used to prevent a real close since @@ -4365,8 +4603,7 @@ struct addr_if { #ifndef SIOCGIFNETMASK -static struct in_addr net_mask(in) -struct in_addr in; +static struct in_addr net_mask(struct in_addr in) { register u_long i = sock_ntohl(in.s_addr); @@ -5541,7 +5778,7 @@ done: ia_p->Ipv6IfIndex && ia_p->Ipv6IfIndex != index) { - /* Oops, there was an other interface for IPv6. Possible? XXX */ + /* Oops, there was another interface for IPv6. Possible? XXX */ index = ia_p->Ipv6IfIndex; goto index; } @@ -5929,6 +6166,7 @@ static int inet_set_opts(inet_descriptor* desc, char* ptr, int len) } continue; +#ifdef HAVE_UDP case INET_LOPT_UDP_READ_PACKETS: if (desc->stype == SOCK_DGRAM) { udp_descriptor* udesc = (udp_descriptor*) desc; @@ -5936,6 +6174,7 @@ static int inet_set_opts(inet_descriptor* desc, char* ptr, int len) udesc->read_packets = ival; } continue; +#endif #ifdef HAVE_SETNS case INET_LOPT_NETNS: @@ -6902,6 +7141,7 @@ static ErlDrvSSizeT inet_fill_opts(inet_descriptor* desc, } continue; +#ifdef HAVE_UDP case INET_LOPT_UDP_READ_PACKETS: if (desc->stype == SOCK_DGRAM) { *ptr++ = opt; @@ -6911,6 +7151,7 @@ static ErlDrvSSizeT inet_fill_opts(inet_descriptor* desc, TRUNCATE_TO(0,ptr); } continue; +#endif #ifdef HAVE_SETNS case INET_LOPT_NETNS: @@ -8041,6 +8282,15 @@ static ErlDrvData inet_start(ErlDrvPort port, int size, int protocol) #ifdef HAVE_SETNS desc->netns = NULL; #endif +#ifdef __OSE__ + desc->select_state = 0; + desc->events[0] = NULL; + desc->events[1] = NULL; + desc->events[2] = NULL; + desc->events[3] = NULL; + desc->events[4] = NULL; + desc->events[5] = NULL; +#endif return (ErlDrvData)desc; } @@ -8761,6 +9011,11 @@ static tcp_descriptor* tcp_inet_copy(tcp_descriptor* desc,SOCKET s, copy_desc->inet.port = port; copy_desc->inet.dport = driver_mk_port(port); + +#ifdef __OSE__ + inet_select_init(©_desc->inet); +#endif + *err = 0; return copy_desc; } @@ -8822,8 +9077,22 @@ static void tcp_inet_stop(ErlDrvData e) inet_stop(INETP(desc)); } +#ifdef __OSE__ - +static ErlDrvSSizeT tcp_inet_ctl_ose(ErlDrvData e, unsigned int cmd, + char* buf, ErlDrvSizeT len, + char** rbuf, ErlDrvSizeT rsize) { + + tcp_descriptor* desc = (tcp_descriptor*)e; + int prev_select_state = INETP(desc)->select_state; + + ErlDrvSSizeT res = tcp_inet_ctl(e,cmd,buf,len,rbuf,rsize); + + tcp_inet_ose_dispatch_signals((tcp_descriptor*)e,prev_select_state,NULL); + + return res; +} +#endif /* TCP requests from Erlang */ static ErlDrvSSizeT tcp_inet_ctl(ErlDrvData e, unsigned int cmd, @@ -8858,10 +9127,11 @@ static ErlDrvSSizeT tcp_inet_ctl(ErlDrvData e, unsigned int cmd, break; } - case INET_REQ_FDOPEN: { /* pass in an open socket */ + case INET_REQ_FDOPEN: { /* pass in an open (and optionally bound) socket */ int domain; + int bound; DEBUGF(("tcp_inet_ctl(%ld): FDOPEN\r\n", (long)desc->inet.port)); - if (len != 6) return ctl_error(EINVAL, rbuf, rsize); + if (len != 6 && len != 10) return ctl_error(EINVAL, rbuf, rsize); switch(buf[0]) { case INET_AF_INET: domain = AF_INET; @@ -8879,8 +9149,13 @@ static ErlDrvSSizeT tcp_inet_ctl(ErlDrvData e, unsigned int cmd, return ctl_error(EINVAL, rbuf, rsize); } if (buf[1] != INET_TYPE_STREAM) return ctl_error(EINVAL, rbuf, rsize); + + if (len == 6) bound = 1; + else bound = get_int32(buf+2+4); + return inet_ctl_fdopen(INETP(desc), domain, SOCK_STREAM, - (SOCKET) get_int32(buf+2), rbuf, rsize); + (SOCKET) get_int32(buf+2), + bound, rbuf, rsize); break; } @@ -9039,7 +9314,7 @@ static ErlDrvSSizeT tcp_inet_ctl(ErlDrvData e, unsigned int cmd, ErlDrvTermData caller = driver_caller(desc->inet.port); tcp_descriptor* accept_desc; int err; - + if ((accept_desc = tcp_inet_copy(desc,s,caller,&err)) == NULL) { sock_close(s); return ctl_error(err, rbuf, rsize); @@ -9073,7 +9348,8 @@ static ErlDrvSSizeT tcp_inet_ctl(ErlDrvData e, unsigned int cmd, char tbuf[2]; int n; - DEBUGF(("tcp_inet_ctl(%ld): RECV\r\n", (long)desc->inet.port)); + DEBUGF(("tcp_inet_ctl(%ld): RECV (s=%d)\r\n", + (long)desc->inet.port, desc->inet.s)); /* INPUT: Timeout(4), Length(4) */ if (!IS_CONNECTED(INETP(desc))) { if (desc->tcp_add_flags & TCP_ADDF_DELAYED_CLOSE_RECV) { @@ -9245,6 +9521,16 @@ static void tcp_inet_command(ErlDrvData e, char *buf, ErlDrvSizeT len) DEBUGF(("tcp_inet_command(%ld) }\r\n", (long)desc->inet.port)); } +#ifdef __OSE__ + +static void tcp_inet_commandv_ose(ErlDrvData e, ErlIOVec* ev) { + int prev_select_state = INETP((tcp_descriptor*)e)->select_state; + tcp_inet_commandv(e, ev); + tcp_inet_ose_dispatch_signals((tcp_descriptor*)e,prev_select_state,NULL); +} + +#endif + static void tcp_inet_commandv(ErlDrvData e, ErlIOVec* ev) { @@ -9308,6 +9594,22 @@ static void inet_stop_select(ErlDrvEvent event, void* _) { #ifdef __WIN32__ WSACloseEvent((HANDLE)event); +#elif defined(__OSE__) + ErlDrvOseEventId id; + union SIGNAL *sig; + erl_drv_ose_event_fetch(event, NULL, &id,NULL); + DEBUGF(("inet_stop_select(?#?) {s=%d\n",id)); + sock_close((int)id); + /* On socket close all the signals waiting to be processed as part of the + select should be deallocated */ + while((sig = erl_drv_ose_get_signal(event))) { + DEBUGF(("inet_stop_select(?#?): Freeing signal %s\n", + signo_to_string(sig->signo))); + free_buf(&sig); + } + erl_drv_ose_event_free(event); + DEBUGF(("inet_stop_select(?#?) }\n")); + #else sock_close((SOCKET)(long)event); #endif @@ -9427,12 +9729,13 @@ static int tcp_remain(tcp_descriptor* desc, int* len) int n = desc->i_ptr - ptr; /* number of bytes read */ int tlen; - DEBUGF(("tcp_remain(%ld): s=%d, n=%d, nfill=%d nsz=%d\r\n", - (long)desc->inet.port, desc->inet.s, n, nfill, nsz)); - tlen = packet_get_length(desc->inet.htype, ptr, n, desc->inet.psize, desc->i_bufsz, &desc->http_state); + + DEBUGF(("tcp_remain(%ld): s=%d, n=%d, nfill=%d nsz=%d, tlen %d\r\n", + (long)desc->inet.port, desc->inet.s, n, nfill, nsz, tlen)); + if (tlen > 0) { if (tlen <= n) { /* got a packet */ *len = tlen; @@ -9840,7 +10143,146 @@ static void tcp_inet_event(ErlDrvData e, ErlDrvEvent event) return; } -#endif /* WIN32 */ +#elif defined(__OSE__) /* !__WIN32__ */ +/* The specific resolve signal function. It will return the socket descriptor + for which the select was issued */ +static ErlDrvOseEventId inet_resolve_signal(union SIGNAL *sig) { + DEBUGF(("%s(?#?): s=%d got signal %s, status = %d, extra = %d, sender = 0x%x\n", + __FUNCTION__,sig->async.fd,signo_to_string(sig->signo), + sig->async.event.status, + sig->async.event.extra,sender(&sig))); + if (sig->signo == SO_EVENT_READ_REPLY || + sig->signo == SO_EVENT_ACCEPT_REPLY || + sig->signo == SO_EVENT_EOF_REPLY || + sig->signo == SO_EVENT_WRITE_REPLY || + sig->signo == SO_EVENT_ERROR_REPLY || + sig->signo == SO_EVENT_CONNECT_REPLY ) { + return sig->async.fd; + } + + return -1; +} + +static void inet_driver_select(inet_descriptor* desc, + int flags, int onoff) { + ASSERT(!desc->is_ignored); + + if(onoff) { + desc->select_state |= flags; + } else { + desc->select_state &= ~flags; + } +} + +static ssize_t writev_fallback(int fd, const struct iovec *iov, int iovcnt, int max_sz) +{ + size_t data_len = 0; + size_t sent = 0; + ssize_t n; + int i; + + for(i = 0; i < iovcnt; i++) + { + data_len = iov[i].iov_len; +tryagain: + n = sock_send(fd, iov[i].iov_base, data_len, 0); + if (IS_SOCKET_ERROR(n)) { + /* If buffer length is greater than the amount stack is able to + * send out then try to send at least max_sz (this comes with + * SO_EVENT_WRITE_REPLY signal*/ + if ((errno == EMSGSIZE) && (max_sz > 0) && (data_len > max_sz)) { + data_len = max_sz; + goto tryagain; + } + break; + } + sent += n; + } + return sent; +} + +#define OSE_EVENT_REQ(TCP_DESC,EVENT) do { \ + union SIGNAL *sig = alloc(sizeof(struct OseAsyncSig), EVENT); \ + sig->async.fd = INETP(TCP_DESC)->s; \ + ose_request_event(INETP(TCP_DESC)->s, &sig, 1); \ + DEBUGF(("%s(%ld): s=%d sent %s\r\n",__FUNCTION__, \ + INETP(TCP_DESC)->port,INETP(TCP_DESC)->s,signo_to_string(EVENT))); \ + } while(0) + +static void tcp_inet_ose_dispatch_signals(tcp_descriptor *desc, + int prev_select_state, + union SIGNAL *sig) { + if (sig) { + DEBUGF(("tcp_inet_ose_dispatch_signals(%ld) {s=%d resend\r\n", + (long)INETP(desc)->port,INETP(desc)->s)); + /* We are reacting to a signal, which means that if + the select_state for that signal is still activated + we should send a new signal */ + switch (sig->signo) { + case SO_EVENT_READ_REPLY: { + if (INETP(desc)->select_state & FD_READ) + OSE_EVENT_REQ(desc,SO_EVENT_READ_REQUEST); + break; + } + case SO_EVENT_WRITE_REPLY: { + if (INETP(desc)->select_state & FD_WRITE) + OSE_EVENT_REQ(desc,SO_EVENT_WRITE_REQUEST); + break; + } + case SO_EVENT_CONNECT_REPLY: { + if (INETP(desc)->select_state & FD_CONNECT) + OSE_EVENT_REQ(desc,SO_EVENT_CONNECT_REQUEST); + break; + } + case SO_EVENT_ACCEPT_REPLY: { + if (INETP(desc)->select_state & FD_ACCEPT) + OSE_EVENT_REQ(desc,SO_EVENT_ACCEPT_REQUEST); + break; + } + case SO_EVENT_ERROR_REPLY: { + if (INETP(desc)->select_state & SOCK_FD_ERROR) + OSE_EVENT_REQ(desc,SO_EVENT_ERROR_REQUEST); + break; + } + + } + DEBUGF(("tcp_inet_ose_dispatch_signals(%ld) }\r\n", + (long)INETP(desc)->port)); + } + + if (INETP(desc)->select_state != prev_select_state) { + /* If the select state has changed we have to issue signals for + the state parts that have changed. */ + int xor_select_state = INETP(desc)->select_state ^ prev_select_state; + DEBUGF(("tcp_inet_ose_dispatch_signals(%ld) {s=%d select change\r\n", + (long)INETP(desc)->port,INETP(desc)->s)); + if ((xor_select_state & FD_READ) && + (INETP(desc)->select_state & FD_READ)) { + OSE_EVENT_REQ(desc,SO_EVENT_READ_REQUEST); + } + if ((xor_select_state & FD_WRITE) && + (INETP(desc)->select_state & FD_WRITE)) { + OSE_EVENT_REQ(desc,SO_EVENT_WRITE_REQUEST); + } + if ((xor_select_state & FD_CONNECT) && + (INETP(desc)->select_state & FD_CONNECT)) { + OSE_EVENT_REQ(desc,SO_EVENT_CONNECT_REQUEST); + } + if ((xor_select_state & FD_ACCEPT) && + (INETP(desc)->select_state & FD_ACCEPT)) { + OSE_EVENT_REQ(desc,SO_EVENT_ACCEPT_REQUEST); + } + if ((xor_select_state & SOCK_FD_ERROR) && + (INETP(desc)->select_state & SOCK_FD_ERROR)) { + OSE_EVENT_REQ(desc,SO_EVENT_ERROR_REQUEST); + } + + DEBUGF(("tcp_inet_ose_dispatch_signals(%ld) }\r\n", + (long)INETP(desc)->port)); + } +} + +#endif /* __OSE__ */ /* socket has input: @@ -9861,7 +10303,7 @@ static int tcp_inet_input(tcp_descriptor* desc, HANDLE event) unsigned int len; inet_address remote; inet_async_op *this_op = desc->inet.opt; - + len = sizeof(desc->inet.remote); s = sock_accept(desc->inet.s, (struct sockaddr*) &remote, &len); if (s == INVALID_SOCKET && sock_errno() == ERRNO_BLOCK) { @@ -9930,7 +10372,6 @@ static int tcp_inet_input(tcp_descriptor* desc, HANDLE event) while (desc->inet.state == INET_STATE_MULTI_ACCEPTING) { len = sizeof(desc->inet.remote); s = sock_accept(desc->inet.s, (struct sockaddr*) &remote, &len); - if (s == INVALID_SOCKET && sock_errno() == ERRNO_BLOCK) { /* Just try again, no real error, keep the last return code */ goto done; @@ -10256,6 +10697,49 @@ static int tcp_send(tcp_descriptor* desc, char* ptr, ErlDrvSizeT len) return 0; } +#ifdef __OSE__ + +static void tcp_inet_drv_output_ose(ErlDrvData data, ErlDrvEvent event) +{ + union SIGNAL *event_sig = erl_drv_ose_get_signal(event); + + while (event_sig) { + int prev_select_state = INETP((tcp_descriptor*)data)->select_state; + int res = tcp_inet_output((tcp_descriptor*)data, (HANDLE)event_sig); + if (res != -1) { + tcp_inet_ose_dispatch_signals((tcp_descriptor*)data, + prev_select_state,event_sig); + free_buf(&event_sig); + event_sig = erl_drv_ose_get_signal(event); + } else { + /* NOTE: here the event object could have been deallocated!!!! + inet_stop_select is called when doing driver_select(ERL_DRV_USE,0) + */ + free_buf(&event_sig); + return; + } + } +} + +static void tcp_inet_drv_input_ose(ErlDrvData data, ErlDrvEvent event) +{ + union SIGNAL *event_sig = erl_drv_ose_get_signal(event); + + while (event_sig) { + int prev_select_state = INETP((tcp_descriptor*)data)->select_state; + int res = tcp_inet_input((tcp_descriptor*)data, (HANDLE)event); + if (res != -1) { + tcp_inet_ose_dispatch_signals((tcp_descriptor*)data, prev_select_state, + event_sig); + free_buf(&event_sig); + event_sig = erl_drv_ose_get_signal(event); + } else { + free_buf(&event_sig); + return; + } + } +} +#else static void tcp_inet_drv_output(ErlDrvData data, ErlDrvEvent event) { (void)tcp_inet_output((tcp_descriptor*)data, (HANDLE)event); @@ -10265,6 +10749,7 @@ static void tcp_inet_drv_input(ErlDrvData data, ErlDrvEvent event) { (void)tcp_inet_input((tcp_descriptor*)data, (HANDLE)event); } +#endif /* socket ready for ouput: ** 1. INET_STATE_CONNECTING => non block connect ? @@ -10330,6 +10815,13 @@ static int tcp_inet_output(tcp_descriptor* desc, HANDLE event) ssize_t n; SysIOVec* iov; +#ifdef __OSE__ + /* For large size buffers case the amount of data that the stack is + able to send out (received in the .extra field) should be passed + down to writev_fallback */ + n = event ? ((union SIGNAL*)event)->async.event.extra : 0; +#endif + if ((iov = driver_peekq(ix, &vsize)) == NULL) { sock_select(INETP(desc), FD_WRITE, 0); send_empty_out_q_msgs(INETP(desc)); @@ -10341,8 +10833,8 @@ static int tcp_inet_output(tcp_descriptor* desc, HANDLE event) if (IS_SOCKET_ERROR(sock_sendv(desc->inet.s, iov, vsize, &n, 0))) { write_error: if ((sock_errno() != ERRNO_BLOCK) && (sock_errno() != EINTR)) { - DEBUGF(("tcp_inet_output(%ld): sock_sendv(%d) errno = %d\r\n", - (long)desc->inet.port, vsize, sock_errno())); + DEBUGF(("tcp_inet_output(%ld): sock_sendv(%d) errno = %d (errno %d)\r\n", + (long)desc->inet.port, vsize, sock_errno(), errno)); ret = tcp_send_error(desc, sock_errno()); goto done; } @@ -10355,6 +10847,12 @@ static int tcp_inet_output(tcp_descriptor* desc, HANDLE event) sizes > (max 32 bit signed int) */ size_t howmuch = 0x7FFFFFFF; /* max signed 32 bit */ int x; +#ifdef __OSE__ + /* For EWOULDBLOCK sock_sendv returns 0 so we have to be sure it + wasn't the case */ + if(sock_errno() == ERRNO_BLOCK) + goto done; +#endif for(x = 0; x < vsize && iov[x].iov_len == 0; ++x) ; if (x < vsize) { @@ -10499,6 +10997,7 @@ static udp_descriptor* sctp_inet_copy(udp_descriptor* desc, SOCKET s, int* err) +#ifdef HAVE_UDP static int packet_inet_init() { return 0; @@ -10529,6 +11028,7 @@ static ErlDrvData udp_inet_start(ErlDrvPort port, char *args) set_default_msgq_limits(port); return data; } +#endif #ifdef HAVE_SCTP static ErlDrvData sctp_inet_start(ErlDrvPort port, char *args) @@ -10539,6 +11039,7 @@ static ErlDrvData sctp_inet_start(ErlDrvPort port, char *args) } #endif +#ifdef HAVE_UDP static void packet_inet_stop(ErlDrvData e) { /* There should *never* be any "empty out q" subscribers on @@ -10627,10 +11128,11 @@ static ErlDrvSSizeT packet_inet_ctl(ErlDrvData e, unsigned int cmd, char* buf, return replen; - case INET_REQ_FDOPEN: { /* pass in an open (and bound) socket */ + case INET_REQ_FDOPEN: { /* pass in an open (and optionally bound) socket */ SOCKET s; + int bound; DEBUGF(("packet inet_ctl(%ld): FDOPEN\r\n", (long)desc->port)); - if (len != 6) { + if (len != 6 && len != 10) { return ctl_error(EINVAL, rbuf, rsize); } switch (buf[0]) { @@ -10655,7 +11157,11 @@ static ErlDrvSSizeT packet_inet_ctl(ErlDrvData e, unsigned int cmd, char* buf, return ctl_error(EINVAL, rbuf, rsize); } s = (SOCKET)get_int32(buf+2); - replen = inet_ctl_fdopen(desc, af, type, s, rbuf, rsize); + + if (len == 6) bound = 1; + else bound = get_int32(buf+2+4); + + replen = inet_ctl_fdopen(desc, af, type, s, bound, rbuf, rsize); if ((*rbuf)[0] != INET_REP_ERROR) { if (desc->active) @@ -11049,7 +11555,7 @@ static void packet_inet_command(ErlDrvData e, char* buf, ErlDrvSizeT len) else inet_reply_ok(desc); } - +#endif #ifdef __WIN32__ static void packet_inet_event(ErlDrvData e, ErlDrvEvent event) @@ -11071,6 +11577,7 @@ static void packet_inet_event(ErlDrvData e, ErlDrvEvent event) #endif +#ifdef HAVE_UDP static void packet_inet_drv_input(ErlDrvData e, ErlDrvEvent event) { (void) packet_inet_input((udp_descriptor*)e, (HANDLE)event); @@ -11327,6 +11834,7 @@ static int packet_inet_output(udp_descriptor* udesc, HANDLE event) DEBUGF(("packet_inet_output(%ld) }\r\n", (long)desc->port)); return ret; } +#endif /*---------------------------------------------------------------------------*/ diff --git a/erts/emulator/drivers/unix/multi_drv.c b/erts/emulator/drivers/unix/multi_drv.c index 822c96730c..724d325ed5 100644 --- a/erts/emulator/drivers/unix/multi_drv.c +++ b/erts/emulator/drivers/unix/multi_drv.c @@ -20,7 +20,7 @@ /* Purpose: Multidriver interface This is an example of a driver which allows multiple instances of itself. I.e have one erlang process execute open_port(multi......) and - at the same time have an other erlang process open an other port + at the same time have another erlang process open another port running multi there as well. */ diff --git a/erts/emulator/drivers/unix/unix_efile.c b/erts/emulator/drivers/unix/unix_efile.c index 42f41c5f3d..878beb055b 100644 --- a/erts/emulator/drivers/unix/unix_efile.c +++ b/erts/emulator/drivers/unix/unix_efile.c @@ -360,7 +360,12 @@ efile_openfile(Efile_error* errInfo, /* Where to return error codes. */ int fd; int mode; /* Open mode. */ - if (stat(name, &statbuf) >= 0 && !ISREG(statbuf)) { + if (stat(name, &statbuf) < 0) { + /* statbuf is undefined: if the caller depends on it, + i.e. invoke_read_file(), fail the call immediately */ + if (pSize && flags == EFILE_MODE_READ) + return check_error(-1, errInfo); + } else if (!ISREG(statbuf)) { /* * For UNIX only, here is some ugly code to allow * /dev/null to be opened as a file. diff --git a/erts/emulator/hipe/hipe_amd64_bifs.m4 b/erts/emulator/hipe/hipe_amd64_bifs.m4 index 0de69a617f..a3219c7586 100644 --- a/erts/emulator/hipe/hipe_amd64_bifs.m4 +++ b/erts/emulator/hipe/hipe_amd64_bifs.m4 @@ -39,7 +39,10 @@ define(HANDLE_GOT_MBUF,` jmp 2b') `#if defined(ERTS_ENABLE_LOCK_CHECK) && defined(ERTS_SMP) -# define CALL_BIF(F) movq $CSYM(F), P_BIF_CALLEE(P); call CSYM(hipe_debug_bif_wrapper) +# define CALL_BIF(F) \ + movq CSYM(F)@GOTPCREL(%rip), %r11; \ + movq %r11, P_BIF_CALLEE(P); \ + call CSYM(hipe_debug_bif_wrapper) #else # define CALL_BIF(F) call CSYM(F) #endif' diff --git a/erts/emulator/hipe/hipe_bif_list.m4 b/erts/emulator/hipe/hipe_bif_list.m4 index 0997d81b2f..5f92b6bac4 100644 --- a/erts/emulator/hipe/hipe_bif_list.m4 +++ b/erts/emulator/hipe/hipe_bif_list.m4 @@ -268,9 +268,16 @@ noproc_primop_interface_1(nbif_atomic_inc, hipe_atomic_inc) */ define(CFUN,`ifelse($1,term_to_binary_1,hipe_wrapper_term_to_binary_1, ifelse($1,term_to_binary_2,hipe_wrapper_term_to_binary_2, -ifelse($1,erts_internal_binary_to_term_1,hipe_wrapper_erts_internal_binary_to_term_1, -ifelse($1,erts_internal_binary_to_term_2,hipe_wrapper_erts_internal_binary_to_term_2, -$1))))') +ifelse($1,binary_to_term_1,hipe_wrapper_binary_to_term_1, +ifelse($1,binary_to_term_2,hipe_wrapper_binary_to_term_2, +ifelse($1,binary_to_list_1,hipe_wrapper_binary_to_list_1, +ifelse($1,binary_to_list_3,hipe_wrapper_binary_to_list_3, +ifelse($1,bitstring_to_list_1,hipe_wrapper_bitstring_to_list_1, +ifelse($1,list_to_binary_1,hipe_wrapper_list_to_binary_1, +ifelse($1,iolist_to_binary_1,hipe_wrapper_iolist_to_binary_1, +ifelse($1,binary_list_to_bin_1,hipe_wrapper_binary_list_to_bin_1, +ifelse($1,list_to_bitstring_1,hipe_wrapper_list_to_bitstring_1, +$1)))))))))))') define(BIF_LIST,`standard_bif_interface_$3(nbif_$4, CFUN($4))') include(TARGET/`erl_bif_list.h') diff --git a/erts/emulator/sys/common/erl_poll.c b/erts/emulator/sys/common/erl_poll.c index 0a58a625b2..aa412a20c8 100644 --- a/erts/emulator/sys/common/erl_poll.c +++ b/erts/emulator/sys/common/erl_poll.c @@ -2157,7 +2157,7 @@ ERTS_POLL_EXPORT(erts_poll_wait)(ErtsPollSet ps, #ifdef ERTS_POLL_DEBUG_PRINT erts_printf("Entering erts_poll_wait(), timeout=%d\n", - (int) tv->tv_sec*1000 + tv->tv_usec/1000); + (int) tvp->tv_sec*1000 + tvp->tv_usec/1000); #endif if (ERTS_POLLSET_SET_POLLED_CHK(ps)) { diff --git a/erts/emulator/sys/ose/default.lmconf b/erts/emulator/sys/ose/beam.lmconf index a66b0ece56..4ad46b01d9 100644 --- a/erts/emulator/sys/ose/default.lmconf +++ b/erts/emulator/sys/ose/beam.lmconf @@ -4,12 +4,13 @@ OSE_LM_POOL_SIZE=0x200000 OSE_LM_MAIN_NAME=main OSE_LM_MAIN_STACK_SIZE=0xF000 OSE_LM_MAIN_PRIORITY=20 +## Has to be of a type that allows MAM OSE_LM_PROGRAM_TYPE=APP_RAM OSE_LM_DATA_INIT=YES OSE_LM_BSS_INIT=YES OSE_LM_EXEC_MODEL=SHARED HEAP_MAX_SIZE=1000000000 -HEAP_SMALL_BUF_INIT_SIZE=64000000 +HEAP_SMALL_BUF_INIT_SIZE=20971520 HEAP_LARGE_BUF_THRESHOLD=16000000 HEAP_LOCK_TYPE=2 diff --git a/erts/emulator/sys/ose/erl_main.c b/erts/emulator/sys/ose/erl_main.c index 03119c3fec..23a9bc93a4 100644 --- a/erts/emulator/sys/ose/erl_main.c +++ b/erts/emulator/sys/ose/erl_main.c @@ -30,6 +30,8 @@ int main(int argc, char **argv) { + (void)stdin;(void)stdout;(void)stderr; + /* When starting using pm_create -c ARGV="-- -root ..", argv[0] is the first part of ARGV and not the name of the executable. So we shuffle some pointers here to make erl_start happy. */ diff --git a/erts/emulator/sys/ose/erl_poll.c b/erts/emulator/sys/ose/erl_poll.c index ca1ed6e53a..7d2a3d1e0b 100644 --- a/erts/emulator/sys/ose/erl_poll.c +++ b/erts/emulator/sys/ose/erl_poll.c @@ -551,7 +551,12 @@ int erts_poll_wait(ErtsPollSet ps, fd.id, fd.signo, current_process()); erts_send_error_to_logger_nogl(dsbufp); timeout = 0; - ASSERT(0); + /* Under normal circumstances the signal is deallocated by the + * driver that issued the select operation. But in this case + * there's no driver waiting for such signal so we have to + * deallocate it here */ + if (sig) + free_buf(&sig); } else { int i; struct erts_sys_fd_type *fd = NULL; @@ -737,6 +742,7 @@ union SIGNAL *erl_drv_ose_get_signal(ErlDrvEvent drv_ev) { ev->msgs = msg->next; ethr_mutex_unlock(&ev->mtx); erts_free(ERTS_ALC_T_FD_SIG_LIST,msg); + restore(sig); return sig; } } diff --git a/erts/emulator/sys/ose/sys.c b/erts/emulator/sys/ose/sys.c index c892cc69c7..5b950a7dae 100644 --- a/erts/emulator/sys/ose/sys.c +++ b/erts/emulator/sys/ose/sys.c @@ -195,7 +195,9 @@ static volatile int children_died; write_buff += sizeof(struct aiocb *); \ memcpy(write_buff,BUFF,SIZE+1); \ SET_AIO(*write_req,FD,SIZE,write_buff); \ - aio_write(write_req); \ + if (aio_write(write_req)) \ + ramlog_printf("%s:%d: write failed with %d\n", \ + __FILE__,__LINE__,errno); \ } \ } while(0) @@ -210,13 +212,13 @@ static volatile int children_died; driver_free(buffer_ptr); \ } while(0) -/* When we have several schedulers, we need to make sure - * that scheduler issuing aio_dispatch() is the owner on the signal */ #define DISPATCH_AIO(sig) do { \ - restore(sig); \ - aio_dispatch(sig); \ + if (aio_dispatch(sig)) \ + ramlog_printf("%s:%d: dispatch failed with %d\n", \ + __FILE__,__LINE__,errno); \ } while(0) +#define AIO_PIPE_SIZE 1024 /* debug print macros */ #define DEBUG_RES 0 @@ -371,6 +373,63 @@ thr_create_prepare_child(void *vtcdp) #endif /* #ifdef USE_THREADS */ +/* The two functions below are stolen from win_con.c + They have to use malloc/free/realloc directly becasue + we want to do able to do erts_printf very early on. + */ +#define VPRINTF_BUF_INC_SIZE 128 +static erts_dsprintf_buf_t * +grow_vprintf_buf(erts_dsprintf_buf_t *dsbufp, size_t need) +{ + char *buf; + size_t size; + + ASSERT(dsbufp); + + if (!dsbufp->str) { + size = (((need + VPRINTF_BUF_INC_SIZE - 1) + / VPRINTF_BUF_INC_SIZE) + * VPRINTF_BUF_INC_SIZE); + buf = (char *) malloc(size * sizeof(char)); + } + else { + size_t free_size = dsbufp->size - dsbufp->str_len; + + if (need <= free_size) + return dsbufp; + + size = need - free_size + VPRINTF_BUF_INC_SIZE; + size = (((size + VPRINTF_BUF_INC_SIZE - 1) + / VPRINTF_BUF_INC_SIZE) + * VPRINTF_BUF_INC_SIZE); + size += dsbufp->size; + buf = (char *) realloc((void *) dsbufp->str, + size * sizeof(char)); + } + if (!buf) + return NULL; + if (buf != dsbufp->str) + dsbufp->str = buf; + dsbufp->size = size; + return dsbufp; +} + +static int erts_sys_ramlog_printf(char *format, va_list arg_list) +{ + int res,i; + erts_dsprintf_buf_t dsbuf = ERTS_DSPRINTF_BUF_INITER(grow_vprintf_buf); + res = erts_vdsprintf(&dsbuf, format, arg_list); + if (res >= 0) { + for (i = 0; i < dsbuf.str_len; i+= 50) + /* We print 50 characters at a time because otherwise + the ramlog looks broken */ + ramlog_printf("%.*s",dsbuf.str_len-50 < 0?dsbuf.str_len:50,dsbuf.str+i); + } + if (dsbuf.str) + free((void *) dsbuf.str); + return res; +} + void erts_sys_pre_init(void) { @@ -409,6 +468,9 @@ erts_sys_pre_init(void) children_died = 0; #endif #endif /* USE_THREADS */ + + erts_printf_stdout_func = erts_sys_ramlog_printf; + erts_smp_atomic_init_nob(&sys_misc_mem_sz, 0); } @@ -650,7 +712,7 @@ static void stop_select(ErlDrvEvent, void*); static PROCESS get_signal_proxy_pid(void) { union SIGNAL *sig; - SIGSELECT any_sig[] = {0}; + SIGSELECT any_sig[] = {1,ERTS_SIGNAL_OSE_DRV_ATTACH}; if (!sig_proxy_pid) { sig = alloc(sizeof(union SIGNAL), ERTS_SIGNAL_OSE_DRV_ATTACH); @@ -685,7 +747,7 @@ resolve_signal(union SIGNAL* sig) { struct erl_drv_entry spawn_driver_entry = { spawn_init, spawn_start, - erl_stop, + NULL, /* erl_stop, */ output, ready_input, ready_output, @@ -784,7 +846,11 @@ set_driver_data(ErlDrvPort port_num, /* READ */ if (read_write & DO_READ) { - efs_examine_fd(ifd, FLIB_FD_HANDLE, &driver_data[ifd].handle, 0); + EfsStatus res = efs_examine_fd(ifd, FLIB_FD_HANDLE, + &driver_data[ifd].handle, 0); + if (res != EFS_SUCCESS) + ramlog_printf("%s:%d: efs_examine_fd(%d) failed with %d\n", + __FILE__,__LINE__,ifd,errno); driver_data[ifd].ifd = ifd; driver_data[ifd].packet_bytes = packet_bytes; driver_data[ifd].port_num = port_num; @@ -792,10 +858,9 @@ set_driver_data(ErlDrvPort port_num, /* async read struct */ memset(&driver_data[ifd].aiocb, 0, sizeof(struct aiocb)); - driver_data[ifd].aiocb.aio_buf = driver_alloc(255); + driver_data[ifd].aiocb.aio_buf = driver_alloc(AIO_PIPE_SIZE); driver_data[ifd].aiocb.aio_fildes = ifd; - driver_data[ifd].aiocb.aio_nbytes = 255; - + driver_data[ifd].aiocb.aio_nbytes = (packet_bytes?packet_bytes:AIO_PIPE_SIZE); driver_data[ifd].alive = 1; driver_data[ifd].status = 0; driver_data[ifd].input_event = @@ -826,7 +891,9 @@ set_driver_data(ErlDrvPort port_num, (void) driver_select(port_num, driver_data[ifd].input_event, (ERL_DRV_READ | ERL_DRV_USE), 1); - aio_read(&driver_data[ifd].aiocb); + if (aio_read(&driver_data[ifd].aiocb)) + ramlog_printf("%s:%d: aio_read(%d) failed with %d\n", + __FILE__,__LINE__,ifd,errno); } else { /* WRITE ONLY */ efs_examine_fd(ofd, FLIB_FD_HANDLE, &driver_data[ofd].handle, 0); @@ -926,7 +993,7 @@ spawn_start(ErlDrvPort port_num, char* name, SysDriverOpts* opts) { int ifd[2]; int ofd[2]; - static uint32_t ticker = 0; + static uint32_t ticker = 1; PmStatus pm_status; OSDOMAIN domain = PM_NEW_DOMAIN; PROCESS progpid, mainbid, mainpid; @@ -938,39 +1005,53 @@ spawn_start(ErlDrvPort port_num, char* name, SysDriverOpts* opts) int handle_size; char *ptr; - /* handle arguments */ - ptr = strchr(name, ' '); - if (ptr != NULL) { - *ptr ='\0'; - ptr++; - args = ptr; + + args = driver_alloc(strlen(name)+1); + strcpy(args, name); + /* We need to handle name in three parts + * - install handle (must be unique) + * - install binary (needed for ose_pm_install_load_module()) + * - full path (as argument to the spawned applications env.var + */ + + /* full path including arguments */ + args = driver_alloc(strlen(name)+1); + strcpy(args, name); + + /* handle path */ + tmp_handle = strrchr(name, '/'); + if (tmp_handle == NULL) { + tmp_handle = name; } else { - args = NULL; + tmp_handle++; } - /* create an install handle */ - ptr = strrchr(name, '/'); + /* handle args */ + ptr = strchr(tmp_handle, ' '); if (ptr != NULL) { - ptr++; - tmp_handle = ptr; + *ptr = '\0'; + handle_size = ptr - tmp_handle; } else { - tmp_handle = name; + handle_size = strlen(name)+1; } - handle_size = strlen(tmp_handle)+1; - handle_size += (ticker<10)?3:((ticker<100)?4:5); + /* make room for ticker */ + handle_size += (ticker<10)?3:((ticker<100)?4:5); handle = driver_alloc(handle_size); - snprintf(handle, handle_size, "%s_%d", tmp_handle, ticker); - + do { - snprintf(handle, handle_size, "%s_%d", tmp_handle, ticker++); + snprintf(handle, handle_size, "%s_%d", tmp_handle, ticker); pm_status = ose_pm_install_load_module(0, "ELF", name, handle, 0, 0, NULL); - + ticker++; } while (pm_status == PM_EINSTALL_HANDLE_ALREADY_INSTALLED); - DEBUG_CHECK_RES(pm_status, PM_SUCCESS); + + if (pm_status != PM_SUCCESS) { + errno = ENOSYS; /* FIXME add comment */ + return ERL_DRV_ERROR_ERRNO; + } /* Create Program */ pm_status = ose_pm_create_program(&domain, handle, 0, 0, @@ -1143,17 +1224,13 @@ static void erl_stop(ErlDrvData drv_data) if (data->ifd != data->ofd) { /* read and write */ nbio_stop_fd(data->port_num, data->input_event); nbio_stop_fd(data->port_num, data->output_event); - driver_select(data->port_num, data->input_event, ERL_DRV_USE, 0); - driver_select(data->port_num, data->output_event, ERL_DRV_USE, 0); } else { /* write only */ nbio_stop_fd(data->port_num, data->output_event); - driver_select(data->port_num, data->output_event, ERL_DRV_USE, 0); } } else { /* read only */ nbio_stop_fd(data->port_num, data->input_event); - driver_select(data->port_num, data->input_event, ERL_DRV_USE, 0); } close(data->ifd); close(data->ofd); @@ -1177,18 +1254,31 @@ static void output(ErlDrvData drv_data, char* buf, ErlDrvSizeT len) lbp = lb + (4-(data->packet_bytes)); if ((sz = driver_sizeq(data->port_num)) > 0) { - driver_enq(data->port_num, lbp, data->packet_bytes); - driver_enq(data->port_num, buf, len); - if (sz + len + data->packet_bytes >= (1 << 13)) + if (data->packet_bytes != 0) { + driver_enq(data->port_num, lbp, data->packet_bytes); + } + driver_enq(data->port_num, buf, len); + + if (sz + len + data->packet_bytes >= (1 << 13)) set_busy_port(data->port_num, 1); } else { - driver_enq(data->port_num, buf, len); /* n is the skip value */ - + char *pbbuf; + if (data->packet_bytes != 0) { + pbbuf = malloc(len + data->packet_bytes); + int i; + for (i = 0; i < data->packet_bytes; i++) { + *pbbuf++ = *lbp++; + } + strncpy(pbbuf, buf, len); + pbbuf -= data->packet_bytes; + } driver_select(data->port_num, data->output_event, ERL_DRV_WRITE|ERL_DRV_USE, 1); - - WRITE_AIO(data->ofd, len, buf); + WRITE_AIO(data->ofd, + (data->packet_bytes ? len+data->packet_bytes : len), + (data->packet_bytes ? pbbuf : buf)); + if (data->packet_bytes != 0) free(pbbuf); } return; /* 0; */ } @@ -1204,12 +1294,12 @@ static int port_inp_failure(ErlDrvPort port_num, ErlDrvEvent ready_fd, int res) ASSERT(res <= 0); erl_drv_ose_event_fetch(ready_fd,&sig_no, NULL, (void **)&fd); - /* As we need to handle two signals, we do this in two steps */ if (driver_data[*fd].alive) { report_exit_status(driver_data[*fd].report_exit, 0); /* status? */ } else { + driver_select(port_num,ready_fd,DO_READ|DO_WRITE,0); clear_fd_data(*fd); driver_report_exit(driver_data[*fd].port_num, driver_data[*fd].status); /* As we do not really know if the spawn has crashed or exited nicely @@ -1248,6 +1338,10 @@ static void ready_input(ErlDrvData drv_data, ErlDrvEvent ready_fd) } else { res = sig->fm_read_reply.actual; + if (res == 0) { + port_inp_failure(data->port_num, ready_fd, res); + break; + } if (data->packet_bytes == 0) { if (res < 0) { @@ -1258,6 +1352,7 @@ static void ready_input(ErlDrvData drv_data, ErlDrvEvent ready_fd) else if (res == 0) { /* read of 0 bytes, eof, otherside of pipe is assumed dead */ port_inp_failure(data->port_num, ready_fd, res); + break; } else { buf = driver_alloc(res); @@ -1267,100 +1362,91 @@ static void ready_input(ErlDrvData drv_data, ErlDrvEvent ready_fd) driver_output(data->port_num, (char*) buf, res); driver_free(buf); } + /* clear the previous read */ + memset(data->aiocb.aio_buf, 0, res); + + /* issue a new read */ + DISPATCH_AIO(sig); + aio_read(&data->aiocb); } - /* We try to read the remainder */ - else if (fd_data[data->ifd].remain > 0) { - if (res < 0) { - if ((errno != EINTR) && (errno != ERRNO_BLOCK)) { - port_inp_failure(data->port_num, ready_fd, res); + else if (data->packet_bytes && fd_data[data->ifd].remain > 0) { + /* we've read a partial package, or a header */ + + if (res == fd_data[data->ifd].remain) { /* we are done! */ + char *buf = data->aiocb.aio_buf; + int i; + + /* do we have anything buffered? */ + if (fd_data[data->ifd].buf != NULL) { + memcpy(fd_data[data->ifd].buf + fd_data[data->ifd].sz, + buf, res); + buf = fd_data[data->ifd].buf; } - } - else if (res == 0) { - port_inp_failure(data->port_num, ready_fd, res); - } - else if (res == fd_data[data->ifd].remain) { /* we're done */ - driver_output(data->port_num, - fd_data[data->ifd].buf, - fd_data[data->ifd].sz); + + fd_data[data->ifd].sz += res; + driver_output(data->port_num, buf, (fd_data[data->ifd].sz>0?fd_data[data->ifd].sz:res)); clear_fd_data(data->ifd); - } - else { /* if (res < fd_data[fd].remain) */ - fd_data[data->ifd].cpos += res; - fd_data[data->ifd].remain -= res; - } - } - else if (fd_data[data->ifd].remain == 0) { /* clean fd */ - if (res < 0) { - if ((errno != EINTR) && (errno != ERRNO_BLOCK)) { - port_inp_failure(data->port_num, ready_fd, res); + + /* clear the previous read */ + memset(data->aiocb.aio_buf, 0, res); + + /* issue a new read */ + DISPATCH_AIO(sig); + data->aiocb.aio_nbytes = data->packet_bytes; + + if (data->aiocb.aio_buf == NULL) { + port_inp_failure(data->port_num, ready_fd, -1); } + aio_read(&data->aiocb); } - else if (res == 0) { /* eof */ - port_inp_failure(data->port_num, ready_fd, res); - } - else if (res < data->packet_bytes - fd_data[data->ifd].psz) { - memcpy(fd_data[data->ifd].pbuf+fd_data[data->ifd].psz, - (void *)data->aiocb.aio_buf, res); - fd_data[data->ifd].psz += res; - } - else { /* if (res >= packet_bytes) */ - unsigned char* cpos = (unsigned char*)data->aiocb.aio_buf; - int bytes_left = res; - - while (1) { - int psz = fd_data[data->ifd].psz; - char* pbp = fd_data[data->ifd].pbuf + psz; - - while (bytes_left && (psz < data->packet_bytes)) { - *pbp++ = *cpos++; - bytes_left--; - psz++; - } - - if (psz < data->packet_bytes) { - fd_data[data->ifd].psz = psz; - break; - } - fd_data[data->ifd].psz = 0; - - switch (data->packet_bytes) { - case 1: h = get_int8(fd_data[data->ifd].pbuf); break; - case 2: h = get_int16(fd_data[data->ifd].pbuf); break; - case 4: h = get_int32(fd_data[data->ifd].pbuf); break; - default: ASSERT(0); return; /* -1; */ - } - - if (h <= (bytes_left)) { - driver_output(data->port_num, (char*) cpos, h); - cpos += h; - bytes_left -= h; - continue; - } - else { /* The last message we got was split */ - char *buf = erts_alloc_fnf(ERTS_ALC_T_FD_ENTRY_BUF, h); - if (!buf) { - errno = ENOMEM; - port_inp_failure(data->port_num, ready_fd, -1); - } - else { - erts_smp_atomic_add_nob(&sys_misc_mem_sz, h); - sys_memcpy(buf, cpos, bytes_left); - fd_data[data->ifd].buf = buf; - fd_data[data->ifd].sz = h; - fd_data[data->ifd].remain = h - bytes_left; - fd_data[data->ifd].cpos = buf + bytes_left; - } - break; - } + else if(res < fd_data[data->ifd].remain) { /* received part of a package */ + if (fd_data[data->ifd].sz == 0) { + + fd_data[data->ifd].sz += res; + memcpy(fd_data[data->ifd].buf, data->aiocb.aio_buf, res); + fd_data[data->ifd].remain -= res; + } + else { + memcpy(fd_data[data->ifd].buf + fd_data[data->ifd].sz, + data->aiocb.aio_buf, res); + fd_data[data->ifd].sz += res; + fd_data[data->ifd].remain -= res; + } + /* clear the previous read */ + memset(data->aiocb.aio_buf, 0, res); + + /* issue a new read */ + DISPATCH_AIO(sig); + data->aiocb.aio_nbytes = fd_data[data->ifd].remain; + + if (data->aiocb.aio_buf == NULL) { + port_inp_failure(data->port_num, ready_fd, -1); } + aio_read(&data->aiocb); } } + else if (data->packet_bytes && fd_data[data->ifd].remain == 0) { /* we've recieved a header */ + + /* analyze the header FIXME */ + switch (data->packet_bytes) { + case 1: h = get_int8(data->aiocb.aio_buf); break; + case 2: h = get_int16(data->aiocb.aio_buf); break; + case 4: h = get_int32(data->aiocb.aio_buf); break; + } - /* reset the read buffer and init next asynch read */ - DISPATCH_AIO(sig); - memset((void *)data->aiocb.aio_buf, 0, 255); + fd_data[data->ifd].buf = erts_alloc_fnf(ERTS_ALC_T_FD_ENTRY_BUF, h + data->packet_bytes); + fd_data[data->ifd].remain = ((h + data->packet_bytes) - res); - if (res > 0) { + /* clear the previous read */ + memset(data->aiocb.aio_buf, 0, data->packet_bytes); + + /* issue a new read */ + DISPATCH_AIO(sig); + data->aiocb.aio_nbytes = h; + + if (data->aiocb.aio_buf == NULL) { + port_inp_failure(data->port_num, ready_fd, -1); + } aio_read(&data->aiocb); } } @@ -1400,11 +1486,16 @@ static void ready_output(ErlDrvData drv_data, ErlDrvEvent ready_fd) DISPATCH_AIO(sig); FREE_AIO(sig->fm_write_reply.buffer); res = driver_deq(data->port_num, iov[0].iov_len); - if (res > 0) { + if (res > 0) { iov = driver_peekq(data->port_num, &vlen); WRITE_AIO(data->ofd, iov[0].iov_len, iov[0].iov_base); } } + else if (vlen == 0) { + DISPATCH_AIO(sig); + FREE_AIO(sig->fm_write_reply.buffer); + } + } sig = erl_drv_ose_get_signal(ready_fd); } @@ -1650,10 +1741,10 @@ erl_assert_error(const char* expr, const char* func, { fflush(stdout); fprintf(stderr, "%s:%d:%s() Assertion failed: %s\n", - file, func, line, expr); + file, line, func, expr); fflush(stderr); ramlog_printf("%s:%d:%s() Assertion failed: %s\n", - file, func, line, expr); + file, line, func, expr); abort(); } diff --git a/erts/emulator/sys/win32/sys.c b/erts/emulator/sys/win32/sys.c index 0ded6b274e..0ded6b274e 100755..100644 --- a/erts/emulator/sys/win32/sys.c +++ b/erts/emulator/sys/win32/sys.c diff --git a/erts/emulator/test/Makefile b/erts/emulator/test/Makefile index 0b0568c31a..dfbe47786a 100644 --- a/erts/emulator/test/Makefile +++ b/erts/emulator/test/Makefile @@ -31,6 +31,7 @@ MODULES= \ a_SUITE \ after_SUITE \ alloc_SUITE \ + async_ports_SUITE \ beam_SUITE \ beam_literals_SUITE \ bif_SUITE \ diff --git a/erts/emulator/test/async_ports_SUITE.erl b/erts/emulator/test/async_ports_SUITE.erl new file mode 100644 index 0000000000..c89b3655ff --- /dev/null +++ b/erts/emulator/test/async_ports_SUITE.erl @@ -0,0 +1,118 @@ +-module(async_ports_SUITE). + +-include_lib("common_test/include/ct.hrl"). + +-compile(export_all). + +-define(PACKET_SIZE, (10 * 1024 * 8)). +-define(CPORT_DELAY, 100). +-define(TEST_LOOPS_COUNT, 100000). +-define(SLEEP_BEFORE_CHECK, 1000). +-define(TEST_PROCS_COUNT, 2). +-define(TC_TIMETRAP_SECONDS, 10). + +suite() -> [{ct_hooks,[ts_install_cth]}]. + +all() -> + [ + permanent_busy_test + ]. + +permanent_busy_test(Config) -> + ct:timetrap({seconds, ?TC_TIMETRAP_SECONDS}), + ExePath = filename:join(?config(data_dir, Config), "cport"), + + Self = self(), + spawn_link( + fun() -> + Block = <<0:?PACKET_SIZE>>, + + Port = open_port(ExePath), + + Testers = + lists:map( + fun(_) -> + erlang:spawn_link(?MODULE, run_loop, + [Self, + Port, + Block, + ?TEST_LOOPS_COUNT, + 0]) + end, + lists:seq(1, ?TEST_PROCS_COUNT)), + Self ! {test_info, Port, Testers}, + endless_flush(Port) + end), + + receive + {test_info, Port, Testers} -> + MaxWaitTime = round(0.7 * ?TC_TIMETRAP_SECONDS * 1000), + ct:log("wait testers, maximum ~w mcsec~n", [MaxWaitTime]), + ok = wait_testers(MaxWaitTime, Testers), + timer:sleep(?SLEEP_BEFORE_CHECK), + case erlang:port_command(Port, <<"test">>, [nosuspend]) of + false -> + exit(port_dead); + true -> + ok + end + end. + +wait_testers(Timeout, Testers) -> + lists:foldl( + fun(Pid, AccIn) -> + StartWait = os:timestamp(), + receive + {Pid, port_dead} -> + recalc_timeout(AccIn, StartWait) + after AccIn -> + Pid ! stop, + recalc_timeout(AccIn, StartWait) + end + end, Timeout, Testers), + ok. + +recalc_timeout(TimeoutIn, WaitStart) -> + erlang:max(0, TimeoutIn - round(timer:now_diff(os:timestamp(), WaitStart)) div 1000). + +open_port(ExePath) -> + erlang:open_port({spawn, ExePath ++ " 100"}, [{packet, 4}, eof, exit_status, use_stdio, binary]). + +run_loop(RootProc, Port, Block, CheckLimit, BusyCnt) -> + receive + stop -> + ok + after 0 -> + case erlang:port_command(Port, Block, [nosuspend]) of + true -> + run_loop(RootProc, Port, Block, CheckLimit, 0); + false -> + if + BusyCnt + 1 > CheckLimit -> + check_dead(RootProc, Port, Block, CheckLimit); + true -> + run_loop(RootProc, Port, Block, CheckLimit, BusyCnt + 1) + end + end + end. + +check_dead(RootProc, Port, Block, CheckLimit) -> + ct:log("~p: check port dead~n", [self()]), + timer:sleep(?SLEEP_BEFORE_CHECK), + case erlang:port_command(Port, Block, [nosuspend]) of + true -> + ct:log("not dead~n"), + run_loop(RootProc, Port, Block, CheckLimit, 0); + false -> + ct:log("port dead: ~p~n", [Port]), + RootProc ! {self(), port_dead}, + ok + end. + +endless_flush(Port) -> + receive + {Port, {data, _}} -> + endless_flush(Port); + {Port, SomethingWrong} -> + erlang:error({someting_wrong, SomethingWrong}) + end. diff --git a/erts/emulator/test/async_ports_SUITE_data/Makefile.src b/erts/emulator/test/async_ports_SUITE_data/Makefile.src new file mode 100644 index 0000000000..56da3fbe12 --- /dev/null +++ b/erts/emulator/test/async_ports_SUITE_data/Makefile.src @@ -0,0 +1,15 @@ +CC = @CC@ +LD = @LD@ +CFLAGS = @CFLAGS@ @DEFS@ +CROSSLDFLAGS = @CROSSLDFLAGS@ + +PROGS = cport@exe@ + + +all: $(PROGS) + +cport@exe@: cport@obj@ + $(LD) $(CROSSLDFLAGS) -o cport cport@obj@ @LIBS@ + +cport@obj@: cport.c + $(CC) -c -o cport@obj@ $(CFLAGS) cport.c diff --git a/erts/emulator/test/async_ports_SUITE_data/cport.c b/erts/emulator/test/async_ports_SUITE_data/cport.c new file mode 100644 index 0000000000..033aff382a --- /dev/null +++ b/erts/emulator/test/async_ports_SUITE_data/cport.c @@ -0,0 +1,81 @@ +#include <stdlib.h> +#include <stdio.h> +#include <errno.h> +#include <string.h> +#ifdef __WIN32__ +# include "windows.h" +# include "winbase.h" +#else +# include <unistd.h> +#endif + +typedef unsigned char byte; + +int read_cmd(byte *buf) +{ + int len; + if (read_exact(buf, 4) != 4) + return(-1); + + len = (buf[0] << 24) | (buf[1] << 16) | (buf[2] << 8) | buf[3]; + return read_exact(buf, len); +} + +int write_cmd(byte *buf, int len) +{ + byte li[4]; + li[0] = (len >> 24) & 0xff; + li[1] = (len >> 16) & 0xff; + li[2] = (len >> 8) & 0xff; + li[3] = len & 0xff; + write_exact(&li, 4); + + return write_exact(buf, len); +} + +int read_exact(byte *buf, int len) +{ + int i, got=0; + do { + if ((i = read(0, buf+got, len-got)) <= 0) + { + return(i); + } + got += i; + } while (got<len); + return len; +} + +int write_exact(byte *buf, int len) +{ + int i, wrote = 0; + do { + if ((i = write(1, buf+wrote, len-wrote)) < 0) + return (i); + wrote += i; + } while (wrote<len); + return len; +} + +byte static_buf[31457280]; // 30 mb + +int main(int argc, char **argv) { + int sleep_time = atoi(argv[1]); + int fn, arg, res; + byte *buf = &static_buf[0]; + int len = 0; + if (sleep_time <= 0) + sleep_time = 0; +#ifdef __WIN32__ + else + sleep_time = ((sleep_time - 1) / 1000) + 1; /* Milli seconds */ +#endif + while ((len = read_cmd(buf)) > 0) { +#ifdef __WIN32__ + Sleep((DWORD) sleep_time); +#else + usleep(sleep_time); +#endif + write_cmd(buf, len); + } +} diff --git a/erts/emulator/test/binary_SUITE.erl b/erts/emulator/test/binary_SUITE.erl index 938aac6a0e..44e9e4f243 100644 --- a/erts/emulator/test/binary_SUITE.erl +++ b/erts/emulator/test/binary_SUITE.erl @@ -58,13 +58,14 @@ ordering/1,unaligned_order/1,gc_test/1, bit_sized_binary_sizes/1, otp_6817/1,deep/1,obsolete_funs/1,robustness/1,otp_8117/1, - otp_8180/1, trapping/1]). + otp_8180/1, trapping/1, large/1, + error_after_yield/1, cmp_old_impl/1]). %% Internal exports. -export([sleeper/0,trapping_loop/4]). suite() -> [{ct_hooks,[ts_install_cth]}, - {timetrap,{minutes,2}}]. + {timetrap,{minutes,4}}]. all() -> [copy_terms, conversions, deep_lists, deep_bitstr_lists, @@ -76,7 +77,8 @@ all() -> bad_term_to_binary, more_bad_terms, otp_5484, otp_5933, ordering, unaligned_order, gc_test, bit_sized_binary_sizes, otp_6817, otp_8117, deep, - obsolete_funs, robustness, otp_8180, trapping]. + obsolete_funs, robustness, otp_8180, trapping, large, + error_after_yield, cmp_old_impl]. groups() -> []. @@ -1266,7 +1268,7 @@ deep(Config) when is_list(Config) -> deep_roundtrip(T) -> B = term_to_binary(T), - T = binary_to_term_stress(B). + T = binary_to_term(B). obsolete_funs(Config) when is_list(Config) -> erts_debug:set_internal_state(available_internal_state, true), @@ -1351,7 +1353,16 @@ trapping(Config) when is_list(Config)-> do_trapping(5, term_to_binary, fun() -> [lists:duplicate(2000000,2000000)] end), do_trapping(5, binary_to_term, - fun() -> [term_to_binary(lists:duplicate(2000000,2000000))] end). + fun() -> [term_to_binary(lists:duplicate(2000000,2000000))] end), + do_trapping(5, binary_to_list, + fun() -> [list_to_binary(lists:duplicate(2000000,$x))] end), + do_trapping(5, list_to_binary, + fun() -> [lists:duplicate(2000000,$x)] end), + do_trapping(5, bitstring_to_list, + fun() -> [list_to_bitstring([lists:duplicate(2000000,$x),<<7:4>>])] end), + do_trapping(5, list_to_bitstring, + fun() -> [[lists:duplicate(2000000,$x),<<7:4>>]] end) + . do_trapping(0, _, _) -> ok; @@ -1384,9 +1395,189 @@ trapping_loop2(Bif,Args,N) -> apply(erlang,Bif,Args), trapping_loop2(Bif, Args, N-1). +large(Config) when is_list(Config) -> + List = lists:flatten(lists:map(fun (_) -> + [0,1,2,3,4,5,6,7,8] + end, + lists:seq(1, 131072))), + Bin = list_to_binary(List), + List = binary_to_list(Bin), + PartList = lists:reverse(tl(tl(lists:reverse(tl(tl(List)))))), + PartList = binary_to_list(Bin, 3, length(List)-2), + ListBS = List ++ [<<7:4>>], + ListBS = bitstring_to_list(list_to_bitstring(ListBS)), + BitStr1 = list_to_bitstring(lists:duplicate(1024*1024, [<<1,5:3>>])), + BitStr1 = list_to_bitstring(bitstring_to_list(BitStr1)), + BitStr2 = list_to_bitstring([lists:duplicate(512*1024, [<<1,5:3>>]), + Bin]), + BitStr2 = list_to_bitstring(bitstring_to_list(BitStr2)), + ok. + +error_after_yield(Config) when is_list(Config) -> + L2BTrap = {erts_internal, list_to_binary_continue, 1}, + error_after_yield(badarg, erlang, list_to_binary, 1, fun () -> [[mk_list(1000000), oops]] end, L2BTrap), + error_after_yield(badarg, erlang, iolist_to_binary, 1, fun () -> [[list2iolist(mk_list(1000000)), oops]] end, L2BTrap), + error_after_yield(badarg, erlang, list_to_bitstring, 1, fun () -> [[list2bitstrlist(mk_list(1000000)), oops]] end, L2BTrap), + error_after_yield(badarg, binary, list_to_bin, 1, fun () -> [[mk_list(1000000), oops]] end, L2BTrap), + + B2TTrap = {erts_internal, binary_to_term_trap, 1}, + + error_after_yield(badarg, erlang, binary_to_term, 1, fun () -> [error_after_yield_bad_ext_term()] end, B2TTrap), + error_after_yield(badarg, erlang, binary_to_term, 2, fun () -> [error_after_yield_bad_ext_term(), [safe]] end, B2TTrap), + + case erlang:system_info(wordsize) of + 4 -> + SysLimitSz = 1 bsl 32, + error_after_yield(system_limit, erlang, list_to_binary, 1, fun () -> [[huge_iolist(SysLimitSz), $x]] end, L2BTrap), + error_after_yield(system_limit, erlang, iolist_to_binary, 1, fun () -> [[huge_iolist(SysLimitSz), $x]] end, L2BTrap), + error_after_yield(system_limit, erlang, list_to_bitstring, 1, fun () -> [[huge_iolist(SysLimitSz), $x]] end, L2BTrap), + error_after_yield(system_limit, binary, list_to_bin, 1, fun () -> [[huge_iolist(SysLimitSz), $x]] end, L2BTrap); + 8 -> + % Takes waaaay to long time to test system_limit on 64-bit archs... + ok + end, + ok. + +error_after_yield(Type, M, F, AN, AFun, TrapFunc) -> + io:format("Testing ~p for ~p:~p/~p~n", [Type, M, F, AN]), + Tracer = self(), + {Pid, Mon} = spawn_monitor(fun () -> + A = AFun(), + try + erlang:yield(), + erlang:trace(self(),true,[running,{tracer,Tracer}]), + apply(M, F, A), + exit({unexpected_success, {M, F, A}}) + catch + error:Type -> + erlang:trace(self(),false,[running,{tracer,Tracer}]), + %% We threw the exception from the native + %% function we trapped to, but we want + %% the BIF that originally was called + %% to appear in the stack trace. + [{M, F, A, _} | _] = erlang:get_stacktrace() + end + end), + receive + {'DOWN', Mon, process, Pid, Reason} -> + normal = Reason + end, + TD = erlang:trace_delivered(Pid), + receive + {trace_delivered, Pid, TD} -> + NoYields = error_after_yield_sched(Pid, TrapFunc, 0), + io:format("No of yields: ~p~n", [NoYields]), + true = NoYields > 2 + end, + ok. + +error_after_yield_sched(P, TrapFunc, N) -> + receive + {trace, P, out, TrapFunc} -> + receive + {trace, P, in, TrapFunc} -> + error_after_yield_sched(P, TrapFunc, N+1) + after 0 -> + exit(trap_sched_mismatch) + end; + {trace, P, out, Func} -> + receive + {trace, P, in, Func} -> + error_after_yield_sched(P, TrapFunc, N) + after 0 -> + exit(other_sched_mismatch) + end + after 0 -> + N + end. + +error_after_yield_bad_ext_term() -> + TupleSz = 2000000, + <<131, % Version magic + AtomExt/binary>> = term_to_binary(an_atom_we_use_for_this), + BadAtomExt = [100, %% ATOM_EXT + 255, 255, % Invalid size of 65535 bytes + "oops"], + + %% Produce a large tuple where the last element is invalid + list_to_binary([131, %% Version magic + 105, %% LARGE_TUPLE_EXT + <<TupleSz:32/big>>, %% Tuple size + lists:duplicate(TupleSz-1, AtomExt), %% Valid atoms + BadAtomExt]). %% Invalid atom at the end + +cmp_old_impl(Config) when is_list(Config) -> + %% Compare results from new yielding implementations with + %% old non yielding implementations + Cookie = atom_to_list(erlang:get_cookie()), + Rel = "r16b_latest", + case test_server:is_release_available(Rel) of + false -> + {skipped, "No "++Rel++" available"}; + true -> + {ok, Node} = ?t:start_node(list_to_atom(atom_to_list(?MODULE)++"_"++Rel), + peer, + [{args, " -setcookie "++Cookie}, + {erl, [{release, Rel}]}]), + + cmp_node(Node, {erlang, list_to_binary, [list2iolist(mk_list(1))]}), + cmp_node(Node, {erlang, list_to_binary, [list2iolist(mk_list(10))]}), + cmp_node(Node, {erlang, list_to_binary, [list2iolist(mk_list(100))]}), + cmp_node(Node, {erlang, list_to_binary, [list2iolist(mk_list(1000))]}), + cmp_node(Node, {erlang, list_to_binary, [list2iolist(mk_list(10000))]}), + cmp_node(Node, {erlang, list_to_binary, [list2iolist(mk_list(100000))]}), + cmp_node(Node, {erlang, list_to_binary, [list2iolist(mk_list(1000000))]}), + cmp_node(Node, {erlang, list_to_binary, [list2iolist(mk_list(10000000))]}), + cmp_node(Node, {erlang, list_to_binary, [list2iolist(mk_list_lb(10000000))]}), + + cmp_node(Node, {erlang, binary_to_list, [list_to_binary(mk_list(1))]}), + cmp_node(Node, {erlang, binary_to_list, [list_to_binary(mk_list(10))]}), + cmp_node(Node, {erlang, binary_to_list, [list_to_binary(mk_list(100))]}), + cmp_node(Node, {erlang, binary_to_list, [list_to_binary(mk_list(1000))]}), + cmp_node(Node, {erlang, binary_to_list, [list_to_binary(mk_list(10000))]}), + cmp_node(Node, {erlang, binary_to_list, [list_to_binary(mk_list(100000))]}), + cmp_node(Node, {erlang, binary_to_list, [list_to_binary(mk_list(1000000))]}), + cmp_node(Node, {erlang, binary_to_list, [list_to_binary(mk_list(10000000))]}), + + cmp_node(Node, {erlang, list_to_bitstring, [list2bitstrlist(mk_list(1))]}), + cmp_node(Node, {erlang, list_to_bitstring, [list2bitstrlist(mk_list(10))]}), + cmp_node(Node, {erlang, list_to_bitstring, [list2bitstrlist(mk_list(100))]}), + cmp_node(Node, {erlang, list_to_bitstring, [list2bitstrlist(mk_list(1000))]}), + cmp_node(Node, {erlang, list_to_bitstring, [list2bitstrlist(mk_list(10000))]}), + cmp_node(Node, {erlang, list_to_bitstring, [list2bitstrlist(mk_list(100000))]}), + cmp_node(Node, {erlang, list_to_bitstring, [list2bitstrlist(mk_list(1000000))]}), + cmp_node(Node, {erlang, list_to_bitstring, [list2bitstrlist(mk_list(10000000))]}), + + cmp_node(Node, {erlang, bitstring_to_list, [list_to_bitstring(list2bitstrlist(mk_list(1)))]}), + cmp_node(Node, {erlang, bitstring_to_list, [list_to_bitstring(list2bitstrlist(mk_list(10)))]}), + cmp_node(Node, {erlang, bitstring_to_list, [list_to_bitstring(list2bitstrlist(mk_list(100)))]}), + cmp_node(Node, {erlang, bitstring_to_list, [list_to_bitstring(list2bitstrlist(mk_list(1000)))]}), + cmp_node(Node, {erlang, bitstring_to_list, [list_to_bitstring(list2bitstrlist(mk_list(10000)))]}), + cmp_node(Node, {erlang, bitstring_to_list, [list_to_bitstring(list2bitstrlist(mk_list(100000)))]}), + cmp_node(Node, {erlang, bitstring_to_list, [list_to_bitstring(list2bitstrlist(mk_list(1000000)))]}), + cmp_node(Node, {erlang, bitstring_to_list, [list_to_bitstring(list2bitstrlist(mk_list(10000000)))]}), + + ?t:stop_node(Node), + + ok + end. %% Utilities. +huge_iolist(Lim) -> + Sz = 1024, + huge_iolist(list_to_binary(mk_list(Sz)), Sz, Lim). + +huge_iolist(X, Sz, Lim) when Sz >= Lim -> + X; +huge_iolist(X, Sz, Lim) -> + huge_iolist([X, X], Sz*2, Lim). + +cmp_node(Node, {M, F, A}) -> + Res = rpc:call(Node, M, F, A), + Res = apply(M, F, A), + ok. + make_sub_binary(Bin) when is_binary(Bin) -> {_,B} = split_binary(list_to_binary([0,1,3,Bin]), 3), B; @@ -1467,3 +1658,78 @@ get_reds() -> erts_debug:set_internal_state(available_internal_state, true), get_reds() end. + +-define(LARGE_BIN, (512*1024+10)). +-define(LARGE_BIN_LIM, (1024*1024)). + +mk_list(0, Acc) -> + Acc; +mk_list(Sz, Acc) -> + mk_list(Sz-1, [$A+(Sz band 63) | Acc]). + +mk_list(Sz) when Sz >= ?LARGE_BIN_LIM -> + SzLeft = Sz - ?LARGE_BIN, + SzHd = SzLeft div 2, + SzTl = SzLeft - SzHd, + [mk_list(SzHd, []), erlang:list_to_binary(mk_list(?LARGE_BIN, [])), mk_list(SzTl, [])]; +mk_list(Sz) -> + mk_list(Sz, []). + +mk_list_lb(Sz) when Sz >= ?LARGE_BIN_LIM -> + SzLeft = Sz - ?LARGE_BIN, + SzHd = SzLeft div 2, + SzTl = SzLeft - SzHd, + [mk_list(SzHd, []), erlang:list_to_binary(mk_list(?LARGE_BIN, [])), mk_list(SzTl, [])]; +mk_list_lb(Sz) -> + mk_list(Sz, []). + + +list2iolist(List) -> + list2iolist(List, []). + +list2iolist([], Acc) -> + Acc; +list2iolist([X0, X1, X2, X3, X4, X5 | Xs], Acc) when is_integer(X0), 0 =< X0, X0 < 256, + is_integer(X1), 0 =< X1, X1 < 256, + is_integer(X2), 0 =< X2, X2 < 256, + is_integer(X3), 0 =< X3, X3 < 256, + is_integer(X4), 0 =< X4, X4 < 256, + is_integer(X5), 0 =< X5, X5 < 256 -> + NewAcc = case (X0+X1+X2+X3+X4+X5) band 3 of + 0 -> + [Acc, [[[[[[[[[[[[X0,[],<<"">>,X1]]]]]]]]],[X2,X3]],[],[],[],[],X4],X5]]; + 1 -> + [Acc, [], erlang:list_to_binary([X0, X1, X2, X3, X4, X5])]; + 2 -> + [Acc, [[[[X0|erlang:list_to_binary([X1])],[X2|erlang:list_to_binary([X3])],[X4|erlang:list_to_binary([X5])]]]|<<"">>]]; + 3 -> + [Acc, X0, X1, X2, <<"">>, [], X3, X4 | erlang:list_to_binary([X5])] + end, + list2iolist(Xs, NewAcc); +list2iolist([X | Xs], Acc) -> + list2iolist(Xs, [Acc,X]). + +list2bitstrlist(List) -> + [list2bitstrlist(List, []), <<4:7>>]. + +list2bitstrlist([], Acc) -> + Acc; +list2bitstrlist([X0, X1, X2, X3, X4, X5 | Xs], Acc) when is_integer(X0), 0 =< X0, X0 < 256, + is_integer(X1), 0 =< X1, X1 < 256, + is_integer(X2), 0 =< X2, X2 < 256, + is_integer(X3), 0 =< X3, X3 < 256, + is_integer(X4), 0 =< X4, X4 < 256, + is_integer(X5), 0 =< X5, X5 < 256 -> + NewAcc = case (X0+X1+X2+X3+X4+X5) band 3 of + 0 -> + [Acc, [[[[[[[[[[[[X0,[],<<"">>,X1]]]]]]]]],[X2,X3]],[],[],[],[],X4],X5]]; + 1 -> + [Acc, [], <<X0:X1>>, <<X2:X3>>, <<X4:X5>>]; + 2 -> + [Acc, [[[[X0|<<X1:X2>>],X3]],[X4|erlang:list_to_binary([X5])]|<<"">>]]; + 3 -> + [Acc, X0, X1, X2, <<"">>, [], X3, X4 | erlang:list_to_binary([X5])] + end, + list2bitstrlist(Xs, NewAcc); +list2bitstrlist([X | Xs], Acc) -> + list2bitstrlist(Xs, [Acc,X]). diff --git a/erts/emulator/test/busy_port_SUITE.erl b/erts/emulator/test/busy_port_SUITE.erl index 4b4af0babe..2ed5aaa0d0 100644 --- a/erts/emulator/test/busy_port_SUITE.erl +++ b/erts/emulator/test/busy_port_SUITE.erl @@ -98,8 +98,10 @@ generator(0, Writer, _Data) -> %% Calling process_info(Pid, current_function) on a suspended process %% used to crash Beam. - {current_function, {erlang, send, 2}} = - process_info(Writer, current_function), + case process_info(Writer, [status,current_function]) of + [{status,suspended},{current_function,{erlang,send,2}}] -> ok; + [{status,suspended},{current_function,{erlang,bif_return_trap,_}}] -> ok + end, unlock_slave(); generator(N, Writer, Data) -> Writer ! {exec, Data}, diff --git a/erts/emulator/test/driver_SUITE.erl b/erts/emulator/test/driver_SUITE.erl index c62bc0c454..336b6188f6 100644 --- a/erts/emulator/test/driver_SUITE.erl +++ b/erts/emulator/test/driver_SUITE.erl @@ -1062,10 +1062,9 @@ otp_6602(Config) when is_list(Config) -> %% Inet driver use port locking... {ok, S} = gen_udp:open(0), {ok, Fd} = inet:getfd(S), - {ok, Port} = inet:port(S), %% Steal fd (lock checker used to %% trigger here). - {ok, _S2} = gen_udp:open(Port,[{fd,Fd}]), + {ok, _S2} = gen_udp:open(0,[{fd,Fd}]), Parent ! Done end), ?line receive Done -> ok end, @@ -1085,7 +1084,15 @@ otp_6602(Config) when is_list(Config) -> ["async_thrs", "sched_thrs"])). --define(EXPECTED_SYSTEM_INFO_NAMES, ?EXPECTED_SYSTEM_INFO_NAMES2). +-define(EXPECTED_SYSTEM_INFO_NAMES3, + (?EXPECTED_SYSTEM_INFO_NAMES2 ++ + ["emu_nif_vsn"])). + +-define(EXPECTED_SYSTEM_INFO_NAMES4, + (?EXPECTED_SYSTEM_INFO_NAMES3 ++ + ["dirty_sched"])). + +-define(EXPECTED_SYSTEM_INFO_NAMES, ?EXPECTED_SYSTEM_INFO_NAMES4). 'driver_system_info_base_ver'(doc) -> []; @@ -1133,16 +1140,18 @@ check_driver_system_info_result(Result) -> drv_vsn_str2tup(erlang:system_info(driver_version))} of {DDVSN, DDVSN} -> ?line [] = Ns; - {{1, 0}, _} -> + %% {{1, 0}, _} -> + %% ?line ExpNs = lists:sort(?EXPECTED_SYSTEM_INFO_NAMES + %% -- ?EXPECTED_SYSTEM_INFO_NAMES1), + %% ?line ExpNs = lists:sort(Ns); + %% {{1, 1}, _} -> + %% ?line ExpNs = lists:sort(?EXPECTED_SYSTEM_INFO_NAMES + %% -- ?EXPECTED_SYSTEM_INFO_NAMES2), + %% ?line ExpNs = lists:sort(Ns); + {{3, 0}, _} -> ?line ExpNs = lists:sort(?EXPECTED_SYSTEM_INFO_NAMES - -- ?EXPECTED_SYSTEM_INFO_NAMES1), - ?line ExpNs = lists:sort(Ns); - {{1, 1}, _} -> - ?line ExpNs = lists:sort(?EXPECTED_SYSTEM_INFO_NAMES - -- ?EXPECTED_SYSTEM_INFO_NAMES2), - ?line ExpNs = lists:sort(Ns); - {{2, 0}, _} -> - ?line [] = Ns + -- ?EXPECTED_SYSTEM_INFO_NAMES3), + ?line ExpNs = lists:sort(Ns) end. chk_sis(SIs, Ns) -> @@ -1189,6 +1198,14 @@ check_si_res(["async_thrs", Value]) -> check_si_res(["sched_thrs", Value]) -> ?line Value = integer_to_list(erlang:system_info(schedulers)); +%% Data added in 3rd version of driver_system_info() (driver version 1.5) +check_si_res(["emu_nif_vsn", _Value]) -> + true; + +%% Data added in 4th version of driver_system_info() (driver version 3.1) +check_si_res(["dirty_sched", _Value]) -> + true; + check_si_res(Unexpected) -> ?line ?t:fail({unexpected_result, Unexpected}). diff --git a/erts/emulator/test/driver_SUITE_data/sys_info_base_drv.c b/erts/emulator/test/driver_SUITE_data/sys_info_base_drv.c index e44c7dbd5e..964034f5a6 100644 --- a/erts/emulator/test/driver_SUITE_data/sys_info_base_drv.c +++ b/erts/emulator/test/driver_SUITE_data/sys_info_base_drv.c @@ -41,7 +41,9 @@ "thread=%s " \ "smp=%s " \ "async_thrs=%d " \ - "sched_thrs=%d" + "sched_thrs=%d " \ + "emu_nif_vsn=%d.%d" + static size_t sys_info_drv_max_res_len(ErlDrvSysInfo *sip) @@ -55,6 +57,7 @@ sys_info_drv_max_res_len(ErlDrvSysInfo *sip) slen += 5; /* smp */ slen += 20; /* async_thrs */ slen += 20; /* sched_thrs */ + slen += 2*20; /* emu_nif_vsn */ return slen; } @@ -72,7 +75,9 @@ sys_info_drv_sprintf_sys_info(ErlDrvSysInfo *sip, char *str) sip->thread_support ? "true" : "false", sip->smp_support ? "true" : "false", sip->async_threads, - sip->scheduler_threads); + sip->scheduler_threads, + sip->nif_major_version, + sip->nif_minor_version); } #include "sys_info_drv_impl.c" diff --git a/erts/emulator/test/driver_SUITE_data/sys_info_curr_drv.c b/erts/emulator/test/driver_SUITE_data/sys_info_curr_drv.c index 5bbc966932..6d2c47fdaf 100644 --- a/erts/emulator/test/driver_SUITE_data/sys_info_curr_drv.c +++ b/erts/emulator/test/driver_SUITE_data/sys_info_curr_drv.c @@ -40,7 +40,9 @@ "thread=%s " \ "smp=%s " \ "async_thrs=%d " \ - "sched_thrs=%d" + "sched_thrs=%d " \ + "emu_nif_vsn=%d.%d " \ + "dirty_sched=%s" static size_t sys_info_drv_max_res_len(ErlDrvSysInfo *sip) @@ -54,6 +56,8 @@ sys_info_drv_max_res_len(ErlDrvSysInfo *sip) slen += 5; /* smp */ slen += 20; /* async_thrs */ slen += 20; /* sched_thrs */ + slen += 2*20; /* emu_nif_vsn */ + slen += 5; /* dirty_sched */ return slen; } @@ -71,7 +75,10 @@ sys_info_drv_sprintf_sys_info(ErlDrvSysInfo *sip, char *str) sip->thread_support ? "true" : "false", sip->smp_support ? "true" : "false", sip->async_threads, - sip->scheduler_threads); + sip->scheduler_threads, + sip->nif_major_version, + sip->nif_minor_version, + sip->dirty_scheduler_support ? "true" : "false"); } #include "sys_info_drv_impl.c" diff --git a/erts/emulator/test/driver_SUITE_data/sys_info_prev_drv.c b/erts/emulator/test/driver_SUITE_data/sys_info_prev_drv.c index 63c69f751c..2271d7027b 100644 --- a/erts/emulator/test/driver_SUITE_data/sys_info_prev_drv.c +++ b/erts/emulator/test/driver_SUITE_data/sys_info_prev_drv.c @@ -41,7 +41,9 @@ "thread=%s " \ "smp=%s " \ "async_thrs=%d " \ - "sched_thrs=%d" + "sched_thrs=%d " \ + "emu_nif_vsn=%d.%d" + static size_t sys_info_drv_max_res_len(ErlDrvSysInfo *sip) @@ -55,6 +57,7 @@ sys_info_drv_max_res_len(ErlDrvSysInfo *sip) slen += 5; /* smp */ slen += 20; /* async_thrs */ slen += 20; /* sched_thrs */ + slen += 2*20; /* emu_nif_vsn */ return slen; } @@ -72,7 +75,9 @@ sys_info_drv_sprintf_sys_info(ErlDrvSysInfo *sip, char *str) sip->thread_support ? "true" : "false", sip->smp_support ? "true" : "false", sip->async_threads, - sip->scheduler_threads); + sip->scheduler_threads, + sip->nif_major_version, + sip->nif_minor_version); } #include "sys_info_drv_impl.c" diff --git a/erts/emulator/test/erts_debug_SUITE.erl b/erts/emulator/test/erts_debug_SUITE.erl index 87778dd0c2..e5c904cfb9 100644 --- a/erts/emulator/test/erts_debug_SUITE.erl +++ b/erts/emulator/test/erts_debug_SUITE.erl @@ -67,6 +67,9 @@ test_size(Config) when is_list(Config) -> 2 = do_test_size({[]}), 3 = do_test_size({a,b}), 7 = do_test_size({a,[b,c]}), + 8 = do_test_size(#{b => 2,c => 3}), + 4 = do_test_size(#{}), + 32 = do_test_size(#{b => 2,c => 3,txt => "hello world"}), %% Test internal consistency of sizes, but without testing %% exact sizes. @@ -97,6 +100,9 @@ test_size(Config) when is_list(Config) -> do_test_size({SimplestFun,SimplestFun}, 2*FunSz0+do_test_size({a,b}), FunSz0+do_test_size({a,b})), + + M = id(#{ "atom" => first, i => 0}), + do_test_size([M,M#{ "atom" := other },M#{i := 42}],54,32), ok. do_test_size(Term) -> diff --git a/erts/emulator/test/fun_SUITE.erl b/erts/emulator/test/fun_SUITE.erl index 8ad5f290ed..2968f5bebb 100644 --- a/erts/emulator/test/fun_SUITE.erl +++ b/erts/emulator/test/fun_SUITE.erl @@ -30,7 +30,7 @@ fun_to_port/1,t_hash/1,t_phash/1,t_phash2/1,md5/1, refc/1,refc_ets/1,refc_dist/1, const_propagation/1,t_arity/1,t_is_function2/1, - t_fun_info/1]). + t_fun_info/1,t_fun_info_mfa/1]). -export([nothing/0]). @@ -42,7 +42,8 @@ all() -> [bad_apply, bad_fun_call, badarity, ext_badarity, equality, ordering, fun_to_port, t_hash, t_phash, t_phash2, md5, refc, refc_ets, refc_dist, - const_propagation, t_arity, t_is_function2, t_fun_info]. + const_propagation, t_arity, t_is_function2, t_fun_info, + t_fun_info_mfa]. groups() -> []. @@ -824,6 +825,24 @@ t_fun_info(Config) when is_list(Config) -> ?line bad_info(<<1,2>>), ok. +t_fun_info_mfa(Config) when is_list(Config) -> + Fun1 = fun spawn_call/2, + {module,M1} = erlang:fun_info(Fun1, module), + {name,F1} = erlang:fun_info(Fun1, name), + {arity,A1} = erlang:fun_info(Fun1, arity), + {M1,F1,A1=2} = erlang:fun_info_mfa(Fun1), + %% Module fun. + Fun2 = fun ?MODULE:t_fun_info/1, + {module,M2} = erlang:fun_info(Fun2, module), + {name,F2} = erlang:fun_info(Fun2, name), + {arity,A2} = erlang:fun_info(Fun2, arity), + {M2,F2,A2=1} = erlang:fun_info_mfa(Fun2), + + %% Not fun. + {'EXIT',_} = (catch erlang:fun_info_mfa(id(d))), + ok. + + bad_info(Term) -> try erlang:fun_info(Term, module) of Any -> diff --git a/erts/emulator/test/match_spec_SUITE.erl b/erts/emulator/test/match_spec_SUITE.erl index 330bef7104..fdce157abc 100644 --- a/erts/emulator/test/match_spec_SUITE.erl +++ b/erts/emulator/test/match_spec_SUITE.erl @@ -42,7 +42,7 @@ -export([init_per_testcase/2, end_per_testcase/2]). init_per_testcase(Func, Config) when is_atom(Func), is_list(Config) -> - Dog=?t:timetrap(?t:seconds(10)), + Dog=?t:timetrap(?t:seconds(30)), [{watchdog, Dog}|Config]. end_per_testcase(_Func, Config) -> @@ -1009,12 +1009,14 @@ loop_runner(Collector, Fun, Laps) -> end, loop_runner_cont(Collector, Fun, 0, Laps). -loop_runner_cont(_Collector, _Fun, Laps, Laps) -> +loop_runner_cont(Collector, _Fun, Laps, Laps) -> receive - {done, Collector} -> - io:format("loop_runner ~p exit after ~p laps\n", [self(), Laps]), - Collector ! {gone, self()} - end; + {done, Collector} -> ok; + {abort, Collector} -> ok + end, + io:format("loop_runner ~p exit after ~p laps\n", [self(), Laps]), + Collector ! {gone, self()}; + loop_runner_cont(Collector, Fun, N, Laps) -> Fun(), receive diff --git a/erts/emulator/test/nif_SUITE.erl b/erts/emulator/test/nif_SUITE.erl index b2da6f58af..14e6585220 100644 --- a/erts/emulator/test/nif_SUITE.erl +++ b/erts/emulator/test/nif_SUITE.erl @@ -37,7 +37,9 @@ threading/1, send/1, send2/1, send3/1, send_threaded/1, neg/1, is_checks/1, get_length/1, make_atom/1, make_string/1, reverse_list_test/1, - otp_9668/1, consume_timeslice/1, dirty_nif/1, dirty_nif_send/1 + otp_9828/1, + otp_9668/1, consume_timeslice/1, dirty_nif/1, dirty_nif_send/1, + dirty_nif_exception/1, nif_schedule/1 ]). -export([many_args_100/100]). @@ -64,7 +66,9 @@ all() -> resource_takeover, threading, send, send2, send3, send_threaded, neg, is_checks, get_length, make_atom, make_string,reverse_list_test, - otp_9668, consume_timeslice, dirty_nif, dirty_nif_send + otp_9828, + otp_9668, consume_timeslice, + nif_schedule, dirty_nif, dirty_nif_send, dirty_nif_exception ]. groups() -> @@ -1440,6 +1444,20 @@ otp_9668(Config) -> ?line verify_tmpmem(TmpMem), ok. +otp_9828(doc) -> ["Copy of writable binary"]; +otp_9828(Config) -> + ensure_lib_loaded(Config, 1), + + otp_9828_loop(<<"I'm alive!">>, 1000). + +otp_9828_loop(Bin, 0) -> + ok; +otp_9828_loop(Bin, Val) -> + WrtBin = <<Bin/binary, Val:32>>, + ok = otp_9828_nif(WrtBin), + otp_9828_loop(WrtBin, Val-1). + + consume_timeslice(Config) when is_list(Config) -> CONTEXT_REDS = 2000, Me = self(), @@ -1524,6 +1542,20 @@ consume_timeslice(Config) when is_list(Config) -> ok. +nif_schedule(Config) when is_list(Config) -> + ensure_lib_loaded(Config), + A = "this is a string", + B = {this,is,a,tuple}, + {B,A} = call_nif_schedule(A, B), + ok = try call_nif_schedule(1, 2) + catch + error:badarg -> + [{?MODULE,call_nif_schedule,[1,2],_}|_] = + erlang:get_stacktrace(), + ok + end, + ok. + dirty_nif(Config) when is_list(Config) -> try erlang:system_info(dirty_cpu_schedulers) of N when is_integer(N) -> @@ -1556,6 +1588,24 @@ dirty_nif_send(Config) when is_list(Config) -> {skipped,"No dirty scheduler support"} end. +dirty_nif_exception(Config) when is_list(Config) -> + try erlang:system_info(dirty_cpu_schedulers) of + N when is_integer(N) -> + ensure_lib_loaded(Config), + try + call_dirty_nif_exception(), + ?t:fail(expected_badarg) + catch + error:badarg -> + [{?MODULE,call_dirty_nif_exception,[],_}|_] = + erlang:get_stacktrace(), + ok + end + catch + error:badarg -> + {skipped,"No dirty scheduler support"} + end. + next_msg(_Pid) -> receive M -> M @@ -1684,9 +1734,12 @@ reverse_list(_) -> ?nif_stub. echo_int(_) -> ?nif_stub. type_sizes() -> ?nif_stub. otp_9668_nif(_) -> ?nif_stub. +otp_9828_nif(_) -> ?nif_stub. consume_timeslice_nif(_,_) -> ?nif_stub. +call_nif_schedule(_,_) -> ?nif_stub. call_dirty_nif(_,_,_) -> ?nif_stub. send_from_dirty_nif(_) -> ?nif_stub. +call_dirty_nif_exception() -> ?nif_stub. %% maps is_map_nif(_) -> ?nif_stub. diff --git a/erts/emulator/test/nif_SUITE_data/nif_SUITE.c b/erts/emulator/test/nif_SUITE_data/nif_SUITE.c index 955dc64189..291c903947 100644 --- a/erts/emulator/test/nif_SUITE_data/nif_SUITE.c +++ b/erts/emulator/test/nif_SUITE_data/nif_SUITE.c @@ -1473,6 +1473,26 @@ static ERL_NIF_TERM otp_9668_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM ar return atom_ok; } +static ERL_NIF_TERM otp_9828_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) +{ + /* copy a writable binary could reallocate it due to "emasculation" + and thereby render a previous inspection invalid. + */ + ErlNifBinary bin1; + ErlNifEnv* myenv; + + if (!enif_inspect_binary(env, argv[0], &bin1)) { + return enif_make_badarg(env); + } + + myenv = enif_alloc_env(); + enif_make_copy(myenv, argv[0]); + enif_free_env(myenv); + + return memcmp(bin1.data, "I'm alive!", 10)==0 ? atom_ok : atom_false; +} + + static ERL_NIF_TERM consume_timeslice_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) { int percent; @@ -1493,25 +1513,57 @@ static ERL_NIF_TERM consume_timeslice_nif(ErlNifEnv* env, int argc, const ERL_NI } } +static ERL_NIF_TERM nif_sched2(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) +{ + char s[64]; + if (!enif_get_string(env, argv[2], s, sizeof s, ERL_NIF_LATIN1)) + return enif_make_badarg(env); + return enif_make_tuple2(env, argv[3], argv[2]); +} + +static ERL_NIF_TERM nif_sched1(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) +{ + ERL_NIF_TERM new_argv[4]; + new_argv[0] = enif_make_atom(env, "garbage0"); + new_argv[1] = enif_make_atom(env, "garbage1"); + new_argv[2] = argv[0]; + new_argv[3] = argv[1]; + return enif_schedule_nif(env, "nif_sched2", 0, nif_sched2, 4, new_argv); +} + +static ERL_NIF_TERM call_nif_schedule(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) +{ + if (argc != 2) + return enif_make_atom(env, "false"); + return enif_schedule_nif(env, "nif_sched1", 0, nif_sched1, argc, argv); +} + #ifdef ERL_NIF_DIRTY_SCHEDULER_SUPPORT + +static int have_dirty_schedulers(void) +{ + ErlNifSysInfo si; + enif_system_info(&si, sizeof(si)); + return si.dirty_scheduler_support; +} + static ERL_NIF_TERM dirty_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) { int n; char s[10]; ErlNifBinary b; ERL_NIF_TERM result; - if (enif_have_dirty_schedulers()) { + if (have_dirty_schedulers()) { assert(enif_is_on_dirty_scheduler(env)); } assert(argc == 3); enif_get_int(env, argv[0], &n); enif_get_string(env, argv[1], s, sizeof s, ERL_NIF_LATIN1); enif_inspect_binary(env, argv[2], &b); - result = enif_make_tuple3(env, - enif_make_int(env, n), - enif_make_string(env, s, ERL_NIF_LATIN1), - enif_make_binary(env, &b)); - return enif_schedule_dirty_nif_finalizer(env, result, enif_dirty_nif_finalizer); + return enif_make_tuple3(env, + enif_make_int(env, n), + enif_make_string(env, s, ERL_NIF_LATIN1), + enif_make_binary(env, &b)); } static ERL_NIF_TERM call_dirty_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) @@ -1522,11 +1574,11 @@ static ERL_NIF_TERM call_dirty_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM assert(!enif_is_on_dirty_scheduler(env)); if (argc != 3) return enif_make_badarg(env); - if (enif_have_dirty_schedulers()) { + if (have_dirty_schedulers()) { if (enif_get_int(env, argv[0], &n) && enif_get_string(env, argv[1], s, sizeof s, ERL_NIF_LATIN1) && enif_inspect_binary(env, argv[2], &b)) - return enif_schedule_dirty_nif(env, ERL_NIF_DIRTY_JOB_CPU_BOUND, dirty_nif, argc, argv); + return enif_schedule_nif(env, "call_dirty_nif", ERL_NIF_DIRTY_JOB_CPU_BOUND, dirty_nif, argc, argv); else return enif_make_badarg(env); } else { @@ -1534,35 +1586,42 @@ static ERL_NIF_TERM call_dirty_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM } } -static ERL_NIF_TERM dirty_sender(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) +static ERL_NIF_TERM send_from_dirty_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) { ERL_NIF_TERM result; ErlNifPid pid; ErlNifEnv* menv; int res; - enif_get_local_pid(env, argv[0], &pid); + if (!enif_get_local_pid(env, argv[0], &pid)) + return enif_make_badarg(env); result = enif_make_tuple2(env, enif_make_atom(env, "ok"), enif_make_pid(env, &pid)); menv = enif_alloc_env(); res = enif_send(env, &pid, menv, result); enif_free_env(menv); if (!res) - /* Note the next line will crash, since dirty nifs can't return exceptions. - * This is intentional, since enif_send should not fail if the test succeeds. - */ - return enif_schedule_dirty_nif_finalizer(env, enif_make_badarg(env), enif_dirty_nif_finalizer); + return enif_make_badarg(env); else - return enif_schedule_dirty_nif_finalizer(env, result, enif_dirty_nif_finalizer); + return result; } -static ERL_NIF_TERM send_from_dirty_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) +static ERL_NIF_TERM call_dirty_nif_exception(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) { - ERL_NIF_TERM result; - ErlNifPid pid; - - if (!enif_get_local_pid(env, argv[0], &pid)) + switch (argc) { + case 0: { + ERL_NIF_TERM args[255]; + int i; + for (i = 0; i < 255; i++) + args[i] = enif_make_int(env, i); + return enif_schedule_nif(env, "call_dirty_nif_exception", ERL_NIF_DIRTY_JOB_CPU_BOUND, + call_dirty_nif_exception, 255, argv); + } + case 1: return enif_make_badarg(env); - return enif_schedule_dirty_nif(env, ERL_NIF_DIRTY_JOB_CPU_BOUND, dirty_sender, argc, argv); + default: + return enif_schedule_nif(env, "call_dirty_nif_exception", ERL_NIF_DIRTY_JOB_CPU_BOUND, + call_dirty_nif_exception, argc-1, argv); + } } #endif @@ -1741,10 +1800,13 @@ static ErlNifFunc nif_funcs[] = {"echo_int", 1, echo_int}, {"type_sizes", 0, type_sizes}, {"otp_9668_nif", 1, otp_9668_nif}, + {"otp_9828_nif", 1, otp_9828_nif}, {"consume_timeslice_nif", 2, consume_timeslice_nif}, + {"call_nif_schedule", 2, call_nif_schedule}, #ifdef ERL_NIF_DIRTY_SCHEDULER_SUPPORT {"call_dirty_nif", 3, call_dirty_nif}, - {"send_from_dirty_nif", 1, send_from_dirty_nif}, + {"send_from_dirty_nif", 1, send_from_dirty_nif, ERL_NIF_DIRTY_JOB_CPU_BOUND}, + {"call_dirty_nif_exception", 0, call_dirty_nif_exception, ERL_NIF_DIRTY_JOB_IO_BOUND}, #endif {"is_map_nif", 1, is_map_nif}, {"get_map_size_nif", 1, get_map_size_nif}, diff --git a/erts/emulator/test/num_bif_SUITE.erl b/erts/emulator/test/num_bif_SUITE.erl index ff8d18eef8..8cf8377c30 100644 --- a/erts/emulator/test/num_bif_SUITE.erl +++ b/erts/emulator/test/num_bif_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2013. All Rights Reserved. +%% Copyright Ericsson AB 1997-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -394,18 +394,15 @@ t_string_to_integer(Config) when is_list(Config) -> test_sti(268435455), test_sti(-268435455), - %% 1 bsl 28 - 1, just before 32 bit bignum - test_sti(1 bsl 28 - 1), - %% 1 bsl 28, just beyond 32 bit small - test_sti(1 bsl 28), - %% 1 bsl 33, just beyond 32 bit - test_sti(1 bsl 33), - %% 1 bsl 60 - 1, just before 64 bit bignum - test_sti(1 bsl 60 - 1), - %% 1 bsl 60, just beyond 64 bit small - test_sti(1 bsl 60), - %% 1 bsl 65, just beyond 64 bit - test_sti(1 bsl 65), + % Interesting values around 2-pows, such as MIN_SMALL and MAX_SMALL. + lists:foreach(fun(Bits) -> + N = 1 bsl Bits, + test_sti(N - 1), + test_sti(N), + test_sti(N + 1) + end, + lists:seq(16, 130)), + %% Bignums. test_sti(123456932798748738738,16), test_sti(list_to_integer(lists:duplicate(2000, $1))), @@ -454,10 +451,11 @@ test_sti(Num) -> end|| Base <- lists:seq(2,36)]. test_sti(Num,Base) -> - Num = list_to_integer(int2list(Num,Base),Base), - Num = -1*list_to_integer(int2list(Num*-1,Base),Base), - Num = binary_to_integer(int2bin(Num,Base),Base), - Num = -1*binary_to_integer(int2bin(Num*-1,Base),Base). + Neg = -Num, + Num = list_to_integer(int2list(Num,Base),Base), + Neg = list_to_integer(int2list(Num*-1,Base),Base), + Num = binary_to_integer(int2bin(Num,Base),Base), + Neg = binary_to_integer(int2bin(Num*-1,Base),Base). % Calling this function (which is not supposed to be inlined) prevents % the compiler from calculating the answer, so we don't test the compiler diff --git a/erts/emulator/test/port_SUITE.erl b/erts/emulator/test/port_SUITE.erl index 202a8b7537..e01b2f253b 100644 --- a/erts/emulator/test/port_SUITE.erl +++ b/erts/emulator/test/port_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2013. All Rights Reserved. +%% Copyright Ericsson AB 1997-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -1698,12 +1698,13 @@ otp_5119(Config) when is_list(Config) -> Path = ?config(data_dir, Config), ok = load_driver(Path, "exit_drv"), PI1 = port_ix(otp_5119_fill_empty_port_tab([])), - PI2 = port_ix(erlang:open_port({spawn, "exit_drv"}, [])), + Port2 = erlang:open_port({spawn, "exit_drv"}, []), + PI2 = port_ix(Port2), {PortIx1, PortIx2} = case PI2 > PI1 of true -> {PI1, PI2}; false -> - {port_ix(otp_5119_fill_empty_port_tab([PI2])), + {port_ix(otp_5119_fill_empty_port_tab([Port2])), port_ix(erlang:open_port({spawn, "exit_drv"}, []))} end, MaxPorts = max_ports(), @@ -2318,7 +2319,7 @@ close_deaf_port(Config) when is_list(Config) -> test_server:timetrap_cancel(Dog), Res. -close_deaf_port_1(1000, _) -> +close_deaf_port_1(200, _) -> ok; close_deaf_port_1(N, Cmd) -> Timeout = integer_to_list(random:uniform(5*1000)), diff --git a/erts/emulator/test/system_info_SUITE.erl b/erts/emulator/test/system_info_SUITE.erl index ceb4afb5cf..f959714be7 100644 --- a/erts/emulator/test/system_info_SUITE.erl +++ b/erts/emulator/test/system_info_SUITE.erl @@ -155,6 +155,7 @@ misc_smoke_tests(Config) when is_list(Config) -> ?line true = is_binary(erlang:system_info(loaded)), ?line true = is_binary(erlang:system_info(dist)), ?line ok = try erlang:system_info({cpu_topology,erts_get_cpu_topology_error_case}), fail catch error:badarg -> ok end, + true = lists:member(erlang:system_info(tolerant_timeofday), [enabled, disabled]), ?line ok. diff --git a/erts/emulator/test/trace_SUITE.erl b/erts/emulator/test/trace_SUITE.erl index 2251575e5a..4d7598cf1f 100644 --- a/erts/emulator/test/trace_SUITE.erl +++ b/erts/emulator/test/trace_SUITE.erl @@ -181,6 +181,13 @@ send_trace(Config) when is_list(Config) -> ?line {trace, Sender, send, to_receiver, Receiver} = receive_first(), ?line receive_nothing(), + %% Check that a message sent to another registered process is traced. + register(?MODULE,Receiver), + Sender ! {send_please, ?MODULE, to_receiver}, + {trace, Sender, send, to_receiver, ?MODULE} = receive_first(), + receive_nothing(), + unregister(?MODULE), + %% Check that a message sent to this process is traced. ?line Sender ! {send_please, self(), to_myself}, ?line receive to_myself -> ok end, @@ -188,6 +195,21 @@ send_trace(Config) when is_list(Config) -> ?line {trace, Sender, send, to_myself, Self} = receive_first(), ?line receive_nothing(), + %% Check that a message sent to dead process is traced. + {Pid,Ref} = spawn_monitor(fun() -> ok end), + receive {'DOWN',Ref,_,_,_} -> ok end, + Sender ! {send_please, Pid, to_dead}, + {trace, Sender, send_to_non_existing_process, to_dead, Pid} = receive_first(), + receive_nothing(), + + %% Check that a message sent to unknown registrated process is traced. + BadargSender = fun_spawn(fun sender/0), + 1 = erlang:trace(BadargSender, true, [send]), + unlink(BadargSender), + BadargSender ! {send_please, not_registered, to_unknown}, + {trace, BadargSender, send, to_unknown, not_registered} = receive_first(), + receive_nothing(), + %% Another process should not be able to trace Sender. ?line Intruder = fun_spawn(fun() -> erlang:trace(Sender, true, [send]) end), ?line {'EXIT', Intruder, {badarg, _}} = receive_first(), diff --git a/erts/emulator/test/tuple_SUITE.erl b/erts/emulator/test/tuple_SUITE.erl index 46ece41096..f627eea07f 100644 --- a/erts/emulator/test/tuple_SUITE.erl +++ b/erts/emulator/test/tuple_SUITE.erl @@ -21,8 +21,9 @@ init_per_group/2,end_per_group/2, t_size/1, t_tuple_size/1, t_element/1, t_setelement/1, t_insert_element/1, t_delete_element/1, - t_list_to_tuple/1, t_tuple_to_list/1, - t_make_tuple_2/1, t_make_tuple_3/1, t_append_element/1, + t_list_to_tuple/1, t_list_to_upper_boundry_tuple/1, t_tuple_to_list/1, + t_make_tuple_2/1, t_make_upper_boundry_tuple_2/1, t_make_tuple_3/1, + t_append_element/1, t_append_element_upper_boundry/1, build_and_match/1, tuple_with_case/1, tuple_in_guard/1]). -include_lib("test_server/include/test_server.hrl"). @@ -40,8 +41,10 @@ suite() -> [{ct_hooks,[ts_install_cth]}]. all() -> [build_and_match, t_size, t_tuple_size, t_list_to_tuple, + t_list_to_upper_boundry_tuple, t_tuple_to_list, t_element, t_setelement, - t_make_tuple_2, t_make_tuple_3, t_append_element, + t_make_tuple_2, t_make_upper_boundry_tuple_2, t_make_tuple_3, + t_append_element, t_append_element_upper_boundry, t_insert_element, t_delete_element, tuple_with_case, tuple_in_guard]. @@ -49,11 +52,21 @@ groups() -> []. init_per_suite(Config) -> + A0 = case application:start(sasl) of + ok -> [sasl]; + _ -> [] + end, + A = case application:start(os_mon) of + ok -> [os_mon|A0]; + _ -> A0 + end, + [{started_apps, A}|Config]. + +end_per_suite(Config) -> + As = ?config(started_apps, Config), + lists:foreach(fun (A) -> application:stop(A) end, As), Config. -end_per_suite(_Config) -> - ok. - init_per_group(_GroupName, Config) -> Config. @@ -176,14 +189,19 @@ t_list_to_tuple(Config) when is_list(Config) -> {'EXIT', {badarg, _}} = (catch list_to_tuple(id([a|b]))), {'EXIT', {badarg, _}} = (catch list_to_tuple(id([a|b]))), - % test upper boundry, 16777215 elements - MaxSize = 1 bsl 24 - 1, - MaxTuple = list_to_tuple(lists:seq(1, MaxSize)), - MaxSize = size(MaxTuple), - {'EXIT', {badarg,_}} = (catch list_to_tuple(lists:seq(1, 1 bsl 24))), ok. +t_list_to_upper_boundry_tuple(Config) when is_list(Config) -> + sys_mem_cond_run(2048, + fun () -> + %% test upper boundry, 16777215 elements + MaxSize = 1 bsl 24 - 1, + MaxTuple = list_to_tuple(lists:seq(1, MaxSize)), + MaxSize = size(MaxTuple), + ok + end). + %% Tests tuple_to_list/1. t_tuple_to_list(Config) when is_list(Config) -> @@ -214,8 +232,6 @@ t_make_tuple_2(Config) when is_list(Config) -> t_make_tuple1({a}), t_make_tuple1(erlang:make_tuple(400, [])), - % test upper boundry, 16777215 elements - t_make_tuple(1 bsl 24 - 1, a), {'EXIT', {badarg,_}} = (catch erlang:make_tuple(1 bsl 24, a)), {'EXIT', {badarg,_}} = (catch erlang:make_tuple(-1, a)), @@ -225,6 +241,13 @@ t_make_tuple_2(Config) when is_list(Config) -> {'EXIT', {badarg,_}} = (catch erlang:make_tuple(1 bsl 65 + 3, a)), ok. +t_make_upper_boundry_tuple_2(Config) when is_list(Config) -> + sys_mem_cond_run(2048, + fun () -> + %% test upper boundry, 16777215 elements + t_make_tuple(1 bsl 24 - 1, a) + end). + t_make_tuple1(Element) -> lists:foreach(fun(Size) -> t_make_tuple(Size, Element) end, [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 255, 256, 511, 512, 999, @@ -309,13 +332,17 @@ t_delete_element(Config) when is_list(Config) -> %% Tests the append_element/2 BIF. t_append_element(Config) when is_list(Config) -> - ok = t_append_element({}, 2048, 2048), - - % test upper boundry, 16777215 elements - MaxSize = 1 bsl 24 - 1, - MaxTuple = list_to_tuple(lists:seq(1, MaxSize)), - {'EXIT',{badarg,_}} = (catch erlang:append_element(MaxTuple, a)), - ok. + ok = t_append_element({}, 2048, 2048). + +t_append_element_upper_boundry(Config) when is_list(Config) -> + sys_mem_cond_run(2048, + fun () -> + %% test upper boundry, 16777215 elements + MaxSize = 1 bsl 24 - 1, + MaxTuple = list_to_tuple(lists:seq(1, MaxSize)), + {'EXIT',{badarg,_}} = (catch erlang:append_element(MaxTuple, a)), + ok + end). t_append_element(_Tuple, 0, _High) -> ok; t_append_element(Tuple, N, High) -> @@ -371,3 +398,31 @@ tuple_in_guard(Config) when is_list(Config) -> %% Use this function to avoid compile-time evaluation of an expression. id(I) -> I. + +sys_mem_cond_run(ReqSizeMB, TestFun) when is_integer(ReqSizeMB) -> + case total_memory() of + TotMem when is_integer(TotMem), TotMem >= ReqSizeMB -> + TestFun(); + TotMem when is_integer(TotMem) -> + {skipped, "Not enough memory ("++integer_to_list(TotMem)++" MB)"}; + undefined -> + {skipped, "Could not retrieve memory information"} + end. + + +total_memory() -> + %% Totat memory in MB. + try + MemoryData = memsup:get_system_memory_data(), + case lists:keysearch(total_memory, 1, MemoryData) of + {value, {total_memory, TM}} -> + TM div (1024*1024); + false -> + {value, {system_total_memory, STM}} = + lists:keysearch(system_total_memory, 1, MemoryData), + STM div (1024*1024) + end + catch + _ : _ -> + undefined + end. diff --git a/erts/emulator/utils/gen_git_version b/erts/emulator/utils/gen_git_version index ef06a4b8e2..9faf015b62 100755 --- a/erts/emulator/utils/gen_git_version +++ b/erts/emulator/utils/gen_git_version @@ -5,9 +5,9 @@ OUTPUT_FILE=$1 if command -v git 2>&1 >/dev/null && test -d $ERL_TOP/.git -o -f $ERL_TOP/.git then - VSN=`git describe --match "OTP_R[0-9][0-9][A-B]*" HEAD` + VSN=`git describe --match "OTP-[0-9]*" HEAD` case "$VSN" in - OTP_R*-g*) + OTP-*-g*) VSN=`echo $VSN | sed -e 's/.*-g\\(.*\\)/\\1/g'` ;; *) VSN="na" ;; esac @@ -36,4 +36,4 @@ then fi exit 0 fi -exit 1
\ No newline at end of file +exit 1 diff --git a/erts/epmd/src/Makefile.in b/erts/epmd/src/Makefile.in index 2ea8630491..0c7787a3b1 100644 --- a/erts/epmd/src/Makefile.in +++ b/erts/epmd/src/Makefile.in @@ -84,7 +84,7 @@ LD = @LD@ ifeq ($(findstring ose,$(TARGET)),ose) LIBS = $(ERTS_INTERNAL_LIBS) @LIBS@ else -LIBS = @LIBS@ $(ERTS_INTERNAL_LIBS) +LIBS = @LIBS@ @SYSTEMD_DAEMON_LIBS@ $(ERTS_INTERNAL_LIBS) endif LDFLAGS = @LDFLAGS@ @@ -148,7 +148,7 @@ endif ifeq ($(findstring ose,$(TARGET)),ose) $(BINDIR)/$(EPMD): $(EPMD_OBJS) $(ERTS_LIB) $(OSE_LM_OBJS) - $(call build-ose-load-module, $@, $(EPMD_OBJS) $(OSE_LM_OBJS), $(LIBS), $(LMCONF)) + $(call build-ose-load-module, $@, $(EPMD_OBJS) $(OSE_LM_OBJS), $(LIBS), $(EPMD_LMCONF)) else $(BINDIR)/$(EPMD): $(EPMD_OBJS) $(ERTS_LIB) $(ld_verbose)$(PURIFY) $(LD) $(LDFLAGS) -o $@ $(EPMD_OBJS) $(LIBS) diff --git a/erts/epmd/src/epmd.c b/erts/epmd/src/epmd.c index 1678d537d1..9630e0cdf0 100644 --- a/erts/epmd/src/epmd.c +++ b/erts/epmd/src/epmd.c @@ -175,6 +175,9 @@ int main(int argc, char** argv) g->nodes.reg = g->nodes.unreg = g->nodes.unreg_tail = NULL; g->nodes.unreg_count = 0; g->active_conn = 0; +#ifdef HAVE_SYSTEMD_SD_DAEMON_H + g->is_systemd = 0; +#endif for (i = 0; i < MAX_LISTEN_SOCKETS; i++) g->listenfd[i] = -1; @@ -248,8 +251,12 @@ int main(int argc, char** argv) else usage(g); epmd_cleanup_exit(g,0); - } - else +#ifdef HAVE_SYSTEMD_SD_DAEMON_H + } else if (strcmp(argv[0], "-systemd") == 0) { + g->is_systemd = 1; + argv++; argc--; +#endif + } else usage(g); } dbg_printf(g,1,"epmd running - daemon = %d",g->is_daemon); @@ -454,6 +461,11 @@ static void usage(EpmdVars *g) fprintf(stderr, " Forcibly unregisters a name with epmd\n"); fprintf(stderr, " (only allowed if -relaxed_command_check was given when \n"); fprintf(stderr, " epmd was started).\n"); +#ifdef HAVE_SYSTEMD_SD_DAEMON_H + fprintf(stderr, " -systemd\n"); + fprintf(stderr, " Wait for socket from systemd. The option makes sense\n"); + fprintf(stderr, " when started from .socket unit.\n"); +#endif epmd_cleanup_exit(g,1); } @@ -486,7 +498,11 @@ static void dbg_gen_printf(int onsyslog,int perr,int from_level, #ifdef HAVE_SYSLOG_H if (onsyslog) { - erts_vsnprintf(buf, DEBUG_BUFFER_SIZE, format, args); + int len; + len = erts_vsnprintf(buf, DEBUG_BUFFER_SIZE, format, args); + if (perr != 0 && len < sizeof(buf)) { + erts_snprintf(buf+len, sizeof(buf)-len, ": %s", strerror(perr)); + } syslog(LOG_ERR,"epmd: %s",buf); } #endif diff --git a/erts/epmd/src/epmd_int.h b/erts/epmd/src/epmd_int.h index d4597be30c..c8f2192f7f 100644 --- a/erts/epmd/src/epmd_int.h +++ b/erts/epmd/src/epmd_int.h @@ -125,6 +125,9 @@ # include "sys/select.h" #endif +#ifdef HAVE_SYSTEMD_SD_DAEMON_H +# include <systemd/sd-daemon.h> +#endif /* ************************************************************************ */ /* Replace some functions by others by making the function name a macro */ @@ -337,6 +340,9 @@ typedef struct { int listenfd[MAX_LISTEN_SOCKETS]; char *addresses; char **argv; +#ifdef HAVE_SYSTEMD_SD_DAEMON_H + int is_systemd; +#endif } EpmdVars; void dbg_printf(EpmdVars*,int,const char*,...); diff --git a/erts/epmd/src/epmd_srv.c b/erts/epmd/src/epmd_srv.c index 247fd34d5a..48fd7a5f9c 100644 --- a/erts/epmd/src/epmd_srv.c +++ b/erts/epmd/src/epmd_srv.c @@ -213,6 +213,39 @@ void run(EpmdVars *g) node_init(g); g->conn = conn_init(g); +#ifdef HAVE_SYSTEMD_DAEMON + if (g->is_systemd) + { + int n; + + dbg_printf(g,2,"try to obtain sockets from systemd"); + + n = sd_listen_fds(0); + if (n < 0) + { + dbg_perror(g,"cannot obtain sockets from systemd"); + epmd_cleanup_exit(g,1); + } + else if (n == 0) + { + dbg_tty_printf(g,0,"systemd provides no sockets"); + epmd_cleanup_exit(g,1); + } + else if (n > MAX_LISTEN_SOCKETS) + { + dbg_tty_printf(g,0,"cannot listen on more than %d IP addresses", MAX_LISTEN_SOCKETS); + epmd_cleanup_exit(g,1); + } + num_sockets = n; + for (i = 0; i < num_sockets; i++) + { + g->listenfd[i] = listensock[i] = SD_LISTEN_FDS_START + i; + } + } + else + { +#endif + dbg_printf(g,2,"try to initiate listening port %d", g->port); if (g->addresses != NULL && /* String contains non-separator characters if: */ @@ -277,6 +310,9 @@ void run(EpmdVars *g) SET_ADDR(iserv_addr[0],EPMD_ADDR_ANY,sport); num_sockets = 1; } +#ifdef HAVE_SYSTEMD_DAEMON + } +#endif #if !defined(__WIN32__) && !defined(__OSE__) /* We ignore the SIGPIPE signal that is raised when we call write @@ -294,6 +330,13 @@ void run(EpmdVars *g) FD_ZERO(&g->orig_read_mask); g->select_fd_top = 0; +#ifdef HAVE_SYSTEMD_SD_DAEMON_H + if (g->is_systemd) + for (i = 0; i < num_sockets; i++) + select_fd_set(g, listensock[i]); + else + { +#endif for (i = 0; i < num_sockets; i++) { if ((listensock[i] = socket(FAMILY,SOCK_STREAM,0)) < 0) @@ -356,6 +399,9 @@ void run(EpmdVars *g) } select_fd_set(g, listensock[i]); } +#ifdef HAVE_SYSTEMD_SD_DAEMON_H + } +#endif dbg_tty_printf(g,2,"entering the main select() loop"); diff --git a/erts/etc/common/Makefile.in b/erts/etc/common/Makefile.in index cfd36af962..0cf965f915 100644 --- a/erts/etc/common/Makefile.in +++ b/erts/etc/common/Makefile.in @@ -509,7 +509,7 @@ $(OBJDIR)/crt0_lm.o: $(CRT0_LM) OSE_LM_OBJS += $(OBJDIR)/ose_confd.o $(OBJDIR)/crt0_lm.o $(BINDIR)/run_erl_lm: $(OBJDIR)/run_erl_main.o $(OBJDIR)/safe_string.o $(OBJDIR)/run_erl.o $(OBJDIR)/run_erl_common.o $(OBJDIR)/to_erl_common.o $(OSE_LM_OBJS) - $(call build-ose-load-module, $@, $^, $(LIBS), $(LMCONF)) + $(call build-ose-load-module, $@, $^, $(LIBS), $(RUN_ERL_LMCONF)) $(OBJDIR)/run_erl_main.o: $(OSEETC)/run_erl_main.c $(OSEETC)/run_erl.h ../common/to_erl_common.h $(RC_GENERATED) diff --git a/erts/etc/common/run_erl_common.c b/erts/etc/common/run_erl_common.c index dc55c2bea4..580b6cc3c5 100644 --- a/erts/etc/common/run_erl_common.c +++ b/erts/etc/common/run_erl_common.c @@ -74,15 +74,6 @@ * run_erl multiple times with different global variables without them * effecting eachother. */ -typedef struct run_erl_ run_erl; - -#ifdef __OSE__ -static OSPPDKEY run_erl_pp_key; -#define RE_DATA (*(run_erl**)ose_get_ppdata(run_erl_pp_key)) -#else -static run_erl re; -#define RE_DATA (&re) -#endif #define STATUSFILE (RE_DATA->statusfile) #define LOG_DIR (RE_DATA->log_dir) @@ -116,6 +107,16 @@ struct run_erl_ { unsigned protocol_ver; }; +typedef struct run_erl_ run_erl; + +#ifdef __OSE__ +static OSPPDKEY run_erl_pp_key; +#define RE_DATA (*(run_erl**)ose_get_ppdata(run_erl_pp_key)) +#else +static run_erl re; +#define RE_DATA (&re) +#endif + /* prototypes */ static int next_log(int log_num); diff --git a/erts/etc/common/to_erl_common.c b/erts/etc/common/to_erl_common.c index a49be44b6c..ab706fffe0 100644 --- a/erts/etc/common/to_erl_common.c +++ b/erts/etc/common/to_erl_common.c @@ -126,7 +126,8 @@ static int version_handshake(char* buf, int len, int wfd); #define READ_AIO(REQ,FD,SIZE,BUFF) \ SET_AIO(REQ,FD,SIZE,BUFF); \ if (aio_read(&(REQ)) != 0) \ - fprintf(stderr,"aio_read of child_read_req(%d) failed\n",FD) + fprintf(stderr,"aio_read of child_read_req(%d) failed" \ + "with error %d\n",FD,errno) union SIGNAL { SIGSELECT signo; diff --git a/erts/etc/ose/etc.lmconf b/erts/etc/ose/etc.lmconf new file mode 100644 index 0000000000..b402b325b1 --- /dev/null +++ b/erts/etc/ose/etc.lmconf @@ -0,0 +1,20 @@ +OSE_LM_STACK_SIZES=256,512,1024,2048,4096,8192,16384,65536 +OSE_LM_SIGNAL_SIZES=31,63,127,255,1023,4095,16383,65535 +OSE_LM_POOL_SIZE=0x200000 +OSE_LM_MAIN_NAME=main +OSE_LM_MAIN_STACK_SIZE=0xF000 +OSE_LM_MAIN_PRIORITY=20 +## Has to be of a type that allows MAM +OSE_LM_PROGRAM_TYPE=APP_RAM +OSE_LM_DATA_INIT=YES +OSE_LM_BSS_INIT=YES +OSE_LM_EXEC_MODEL=SHARED +HEAP_MAX_SIZE=1000000000 +HEAP_SMALL_BUF_INIT_SIZE=64000000 +HEAP_LARGE_BUF_THRESHOLD=16000000 +HEAP_LOCK_TYPE=2 + +# Setting the environment variable EFS_RESOLVE_TMO on the block to 0. +# This will eliminiate delays when trying to open files on not mounted +# volumes. +EFS_RESOLVE_TMO=0 diff --git a/erts/etc/ose/run_erl_main.c b/erts/etc/ose/run_erl_main.c index d396ebe93b..2d92924ff2 100644 --- a/erts/etc/ose/run_erl_main.c +++ b/erts/etc/ose/run_erl_main.c @@ -45,6 +45,8 @@ int main(int argc, char **argv) char run_erl_usage[320], to_erl_usage[120]; + (void)stdin;(void)stdout;(void)stderr; + sprintf(run_erl_usage,RUN_ERL_USAGE,"run_erl [-daemon] [-block blockname]"); sprintf(to_erl_usage,TO_ERL_USAGE,"pipename"); diff --git a/erts/etc/unix/etp-commands.in b/erts/etc/unix/etp-commands.in index 9e39764195..bf6eb00314 100644 --- a/erts/etc/unix/etp-commands.in +++ b/erts/etc/unix/etp-commands.in @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 2005-2012. All Rights Reserved. +# Copyright Ericsson AB 2005-2014. All Rights Reserved. # # The contents of this file are subject to the Erlang Public License, # Version 1.1, (the "License"); you may not use this file except in @@ -54,13 +54,23 @@ document etp-help % etp-mfa, etp-cp, % etp-msgq, etpf-msgq, % etp-stacktrace, etp-stackdump, etpf-stackdump, etp-dictdump -% etp-offheapdump, etpf-offheapdump, -% etp-print-procs, etp-search-heaps, etp-search-alloc, +% etp-process-info, etp-process-memory-info +% etp-port-info, etp-port-state, etp-port-sched-flags +% etp-heapdump, etp-offheapdump, etpf-offheapdump, +% etp-search-heaps, etp-search-alloc, % etp-ets-tables, etp-ets-tabledump % % Complex commands that use the Erlang support module. % etp-overlapped-heaps, etp-chart, etp-chart-start, etp-chart-end -% +% +% System inspection +% etp-system-info, etp-schedulers, etp-process, etp-ports, etp-lc-dump, +% etp-migration-info, etp-processes-memory, +% etp-compile-info, etp-config-h-info +% +% Platform specific (when gdb fails you) +% etp-ppc-stacktrace +% % Erlang support module handling commands: % etp-run % @@ -1055,7 +1065,9 @@ define etp-cp-1 set $etp_cp_mid = $etp_cp_low + ($etp_cp_high-$etp_cp_low)/2 end if $etp_cp_p - set $etp_cp_low = (Eterm**)($etp_cp_p->start + 8) + # 12 = MI_FUNCTIONS + set $etp_cp_low = (Eterm**)($etp_cp_p->start + 12) + # 0 = MI_NUM_FUNCTIONS set $etp_cp_high = $etp_cp_low +$etp_cp_p->start[0] set $etp_cp_p = 0 while $etp_cp_low < $etp_cp_high @@ -2830,6 +2842,152 @@ document etp-search-alloc end +define etp-alloc-stats + printf "\nIx Name Inst. Blocks Bytes Carriers Crr.bytes Util\n" + set $etp_tot_block_no = 0 + set $etp_tot_block_sz = 0 + set $etp_tot_crr_no = 0 + set $etp_tot_crr_sz = 0 + set $etp_ERTS_ALC_A_MIN = 1 + set $etp_ERTS_ALC_A_MAX = (sizeof(erts_allctrs) / sizeof(*erts_allctrs)) - 1 + + set $etp_ix = $etp_ERTS_ALC_A_MIN + while $etp_ix <= $etp_ERTS_ALC_A_MAX + set $etp_allctr = 0 + set $etp_alloc = erts_allctrs[$etp_ix].alloc + if $etp_alloc != erts_sys_alloc + if $etp_alloc == erts_alcu_alloc_thr_spec || \ + $etp_alloc == erts_alcu_alloc_thr_pref + set $etp_instance = 0 + set $etp_block_no = 0 + set $etp_block_sz = 0 + set $etp_crr_no = 0 + set $etp_crr_sz = 0 + set $etp_tspec = (ErtsAllocatorThrSpec_t *) erts_allctrs[$etp_ix].extra + if $etp_tspec->enabled + while $etp_instance < $etp_tspec->size + set $etp_allctr = $etp_tspec->allctr[$etp_instance] + set $etp_block_no = $etp_block_no + $etp_allctr->mbcs.blocks.curr.no \ + + $etp_allctr->sbcs.blocks.curr.no + set $etp_block_sz = $etp_block_sz + $etp_allctr->mbcs.blocks.curr.size \ + + $etp_allctr->sbcs.blocks.curr.size + set $etp_crr_no = $etp_crr_no + $etp_allctr->mbcs.curr.norm.mseg.no \ + + $etp_allctr->sbcs.curr.norm.mseg.no \ + + $etp_allctr->mbcs.curr.norm.sys_alloc.no \ + + $etp_allctr->sbcs.curr.norm.sys_alloc.no + set $etp_crr_sz = $etp_crr_sz + $etp_allctr->mbcs.curr.norm.mseg.size \ + + $etp_allctr->sbcs.curr.norm.mseg.size \ + + $etp_allctr->mbcs.curr.norm.sys_alloc.size \ + + $etp_allctr->sbcs.curr.norm.sys_alloc.size + set $etp_instance = $etp_instance + 1 + end + else + printf "erts_allctr[%d]: Disabled (thread specific)\n", $etp_ix + end + else + if $etp_alloc == erts_alcu_alloc_ts || $etp_alloc == erts_alcu_alloc + set $etp_allctr = (Allctr_t*) erts_allctrs[$etp_ix].extra + set $etp_block_no = $etp_allctr->mbcs.blocks.curr.no \ + + $etp_allctr->sbcs.blocks.curr.no + set $etp_block_sz = $etp_allctr->mbcs.blocks.curr.size \ + + $etp_allctr->sbcs.blocks.curr.size + set $etp_crr_no = $etp_allctr->mbcs.curr.norm.mseg.no \ + + $etp_allctr->sbcs.curr.norm.mseg.no \ + + $etp_allctr->mbcs.curr.norm.sys_alloc.no \ + + $etp_allctr->sbcs.curr.norm.sys_alloc.no + set $etp_crr_sz = $etp_allctr->mbcs.curr.norm.mseg.size \ + + $etp_allctr->sbcs.curr.norm.mseg.size \ + + $etp_allctr->mbcs.curr.norm.sys_alloc.size \ + + $etp_allctr->sbcs.curr.norm.sys_alloc.size + set $etp_instance = 1 + else + printf "erts_allctr[%d]: Unknown allocation function: ", $etp_ix + p $etp_alloc + end + end + end + if $etp_allctr != 0 + printf "%2d %-8s%2d%12lu%13lu%12lu%13lu", $etp_ix, $etp_allctr->name_prefix, \ + $etp_instance, \ + $etp_block_no, $etp_block_sz, $etp_crr_no, $etp_crr_sz + if $etp_crr_sz != 0 + printf "%5lu%%", ($etp_block_sz * 100) / $etp_crr_sz + end + printf "\n" + set $etp_tot_block_no = $etp_tot_block_no + $etp_block_no + set $etp_tot_block_sz = $etp_tot_block_sz + $etp_block_sz + set $etp_tot_crr_no = $etp_tot_crr_no + $etp_crr_no + set $etp_tot_crr_sz = $etp_tot_crr_sz + $etp_crr_sz + end + set $etp_ix = $etp_ix + 1 + end + printf "\nTotal: %12lu%13lu%12lu%13lu", $etp_tot_block_no, $etp_tot_block_sz, \ + $etp_tot_crr_no, $etp_tot_crr_sz + if $etp_tot_crr_sz != 0 + printf "%5lu%%", ($etp_tot_block_sz * 100) / $etp_tot_crr_sz + end + printf "\n" +end + +document etp-alloc-stats +%--------------------------------------------------------------------------- +% etp-alloc-stats +% +% Combine and print allocator statistics +%--------------------------------------------------------------------------- +end + + +define etp-alloc-instances + set $etp_ERTS_ALC_A_MIN = 1 + set $etp_ERTS_ALC_A_MAX = (sizeof(erts_allctrs) / sizeof(*erts_allctrs)) - 1 + + set $etp_ix = $arg0 + if $etp_ix >= $etp_ERTS_ALC_A_MIN && $etp_ix <= $etp_ERTS_ALC_A_MAX + set $etp_allctr = 0 + set $etp_alloc = erts_allctrs[$etp_ix].alloc + if $etp_alloc == erts_sys_alloc + printf "Allocator %d is sys_alloc\n", $etp_ix + else + if $etp_alloc == erts_alcu_alloc_thr_spec || \ + $etp_alloc == erts_alcu_alloc_thr_pref + set $etp_instance = 0 + set $etp_tspec = (ErtsAllocatorThrSpec_t *) erts_allctrs[$etp_ix].extra + if $etp_tspec->enabled + printf "All instances for allocator '%s'\n", $etp_tspec->allctr[0]->name_prefix + while $etp_instance < $etp_tspec->size + p $etp_tspec->allctr[$etp_instance] + set $etp_instance = $etp_instance + 1 + end + else + printf "erts_allctr[%d]: Disabled (thread specific)\n", $etp_ix + end + else + if $etp_alloc == erts_alcu_alloc_ts || $etp_alloc == erts_alcu_alloc + set $etp_allctr = (Allctr_t*) erts_allctrs[$etp_ix].extra + printf "Single instances for allocator '%s'\n", $etp_allctr->name_prefix + p $etp_allctr + else + printf "erts_allctr[%d]: Unknown allocation function: ", $etp_ix + p $etp_alloc + end + end + end + else + printf "Allocator type not between %d and %d\n", $etp_ERTS_ALC_A_MIN, $etp_ERTS_ALC_A_MAX + end +end + +document etp-alloc-instances +%--------------------------------------------------------------------------- +% etp-alloc-instances +% +% Print pointers to all allocator instances for a specific type (Ix) +%--------------------------------------------------------------------------- +end + + + define etp-overlapped-heaps # Args: @@ -3133,6 +3291,77 @@ document etp-ets-tabledump %--------------------------------------------------------------------------- end +define etp-lc-dump +# Non-reentrant + set $etp_lc_dump_thread = erts_locked_locks + while $etp_lc_dump_thread + printf "Thread %s\n", $etp_lc_dump_thread->thread_name + set $etp_lc_dump_thread_locked = $etp_lc_dump_thread->locked.first + while $etp_lc_dump_thread_locked + if 0 <= $etp_lc_dump_thread_locked->id && $etp_lc_dump_thread_locked->id < sizeof(erts_lock_order)/sizeof(erts_lc_lock_order_t) + printf " %s:", erts_lock_order[$etp_lc_dump_thread_locked->id].name + else + printf " unkown:" + end + if ($etp_lc_dump_thread_locked->extra & 0x3) == 0x3 + etp-1 $etp_lc_dump_thread_locked->extra + else + printf "%p", $etp_lc_dump_thread_locked->extra + end + if ($etp_lc_dump_thread_locked->flags & (0x1f)) == (1 << 0) + printf "[spinlock]" + end + if ($etp_lc_dump_thread_locked->flags & (0x1f)) == (1 << 1) + printf "[rw(spin)lock]" + end + if ($etp_lc_dump_thread_locked->flags & (0x1f)) == (1 << 2) + printf "[mutex]" + end + if ($etp_lc_dump_thread_locked->flags & (0x1f)) == (1 << 3) + printf "[rwmutex]" + end + if ($etp_lc_dump_thread_locked->flags & (0x1f)) == (1 << 4) + printf "[proclock]" + end + printf "(%s:%d)", $etp_lc_dump_thread_locked->file, $etp_lc_dump_thread_locked->line + if ($etp_lc_dump_thread_locked->flags & (0x60)) == (1 << 5) + printf "(r)" + end + if ($etp_lc_dump_thread_locked->flags & (0x60)) == ((1 << 5) | (1 << 6)) + printf "(rw)" + end + printf "\n" + set $etp_lc_dump_thread_locked = $etp_lc_dump_thread_locked->next + end + set $etp_lc_dump_thread = $etp_lc_dump_thread->next + end +end + +document etp-lc-dump +%--------------------------------------------------------------------------- +% etp-lc-dump +% +% Dump all info about locks in the lock checker +%--------------------------------------------------------------------------- +end + +define etp-ppc-stacktrace +# Args: R1 +# Non-reentrant + set $etp_ppc_st_fp = ($arg0) + while $etp_ppc_st_fp + info symbol ((void**)$etp_ppc_st_fp)[1] + set $etp_ppc_st_fp = ((void**)$etp_ppc_st_fp)[0] + end +end + +document etp-ppc-stacktrace +%--------------------------------------------------------------------------- +% etp-ppc-stacktrace R1 +% +% Dump stacktrace from given $r1 frame pointer +%--------------------------------------------------------------------------- +end ############################################################################ # OSE support @@ -3177,6 +3406,154 @@ define etp-thr end ############################################################################ +# erl_alloc_util (blocks and carriers) +# + +define etp-block-size-1 +# +# In: (Block_t*) in $arg0 +# Out: Byte size in $etp_blk_sz +# + if ($arg0)->bhdr & 1 + # Free block + set $etp_blk_sz = ($arg0)->bhdr & ~7 + else + # Allocated block + if !$etp_MBC_ABLK_SZ_MASK + if etp_arch_bits == 64 + set $etp_MBC_ABLK_OFFSET_SHIFT = (64 - 24) + else + set $etp_MBC_ABLK_OFFSET_SHIFT = (32 - 9) + end + set $etp_MBC_ABLK_SZ_MASK = ((UWord)1 << $etp_MBC_ABLK_OFFSET_SHIFT) - 1 - 7 + end + set $etp_blk_sz = ($arg0)->bhdr & $etp_MBC_ABLK_SZ_MASK + end +end + +define etp-block2mbc-1 +# +# In: (Block_t*) in $arg0 +# Out: (Carrier_t*) in $etp-mbc +# + if (($arg0)->bhdr) & 1 + # Free block + set $etp_mbc = ($arg0)->u.carrier + else + # Allocated block + if !$etp_MBC_ABLK_OFFSET_SHIFT + if etp_arch_bits == 64 + set $etp_MBC_ABLK_OFFSET_SHIFT = (64 - 24) + else + set $etp_MBC_ABLK_OFFSET_SHIFT = (32 - 9) + end + end + set $etp_mbc = (Carrier_t*) ((((UWord)($arg0) >> 18) - (($arg0)->bhdr >> $etp_MBC_ABLK_OFFSET_SHIFT)) << 18) + end +end + +define etp-block2mbc + etp-block2mbc-1 ((Block_t*)$arg0) + print $etp_mbc +end + +document etp-block2mbc +%--------------------------------------------------------------------------- +% Print pointer to multiblock carrier containing the argument (Block_t*) +%--------------------------------------------------------------------------- +end + +define etp-block + etp-block-size-1 ((Block_t*)$arg0) + if ((Block_t*)$arg0)->bhdr & 1 + printf "%#lx: FREE sz=%#x\n", ($arg0), $etp_blk_sz + else + printf "%#lx: ALLOCATED sz=%#x\n", ($arg0), $etp_blk_sz + end +end + +document etp-block +%--------------------------------------------------------------------------- +% Print memory block (Block_t*) +%--------------------------------------------------------------------------- +end + +define etp-carrier-blocks + set $etp_crr = (Carrier_t*) $arg0 + set $etp_alc = (Allctr_t*)($etp_crr->allctr.counter & ~7) + set $etp_blk = (Block_t*) ((char*)$etp_crr + $etp_alc->mbc_header_size) + set $etp_prev_blk = 0 + set $etp_error_cnt = 0 + set $etp_ablk_cnt = 0 + set $etp_fblk_cnt = 0 + + if $argc == 2 + set $etp_be_silent = $arg1 + else + set $etp_be_silent = 0 + end + + while 1 + if !$etp_be_silent + etp-block $etp_blk + else + etp-block-size-1 $etp_blk + end + etp-block2mbc-1 $etp_blk + if $etp_mbc != $etp_crr + printf "ERROR: Invalid carrier pointer %#lx in block at %#lx\n", $etp_mbc, $etp_blk + set $etp_error_cnt = $etp_error_cnt + 1 + end + if $etp_prev_blk + if ($etp_prev_blk->bhdr & 1) + # Prev is FREE + if ($etp_blk->bhdr & 1) + printf "ERROR: Adjacent FREE blocks at %#lx and %#lx\n", $etp_prev_blk, $etp_blk + set $etp_error_cnt = $etp_error_cnt + 1 + end + if !($etp_blk->bhdr & 2) + printf "ERROR: Missing PREV_FREE_BLK_HDR_FLG (2) in block at %#lx\n", $etp_blk + set $etp_error_cnt = $etp_error_cnt + 1 + end + end + end + if $etp_blk->bhdr & 1 + set $etp_fblk_cnt = $etp_fblk_cnt + 1 + else + set $etp_ablk_cnt = $etp_ablk_cnt + 1 + end + if $etp_blk->bhdr & 4 + # Last block + loop_break + end + # All free blocks except the last have a footer + if ($etp_blk->bhdr & 1) && ((UWord*)((char*)$etp_blk + $etp_blk_sz))[-1] != $etp_blk_sz + printf "ERROR: Invalid footer of free block at %#lx\n", $etp_blk + end + set $etp_prev_blk = $etp_blk + set $etp_blk = (Block_t*) ((char*)$etp_blk + $etp_blk_sz) + end + + if ((char*)$etp_blk + $etp_blk_sz) != ((char*)$etp_crr + ($etp_crr->chdr & ~7)) + printf "ERROR: Last block not at end of carrier\n" + set $etp_error_cnt = $etp_error_cnt + 1 + end + printf "Allocated blocks: %u\n", $etp_ablk_cnt + printf "Free blocks: %u\n", $etp_fblk_cnt + if $etp_error_cnt + printf "%u ERRORs reported above\n", $etp-error-cnt + end +end + +document etp-carrier-blocks +%--------------------------------------------------------------------------- +% Check and (maybe) print all memory blocks in carrier +% Args: (Carrier_t*) [1=be_silent] +%--------------------------------------------------------------------------- +end + + +############################################################################ # Toolbox parameter handling # diff --git a/erts/etc/unix/run_erl.c b/erts/etc/unix/run_erl.c index a6fc4c2bf5..4b123b8911 100644 --- a/erts/etc/unix/run_erl.c +++ b/erts/etc/unix/run_erl.c @@ -40,9 +40,13 @@ #ifdef HAVE_CONFIG_H # include "config.h" #endif + #ifdef HAVE_WORKING_POSIX_OPENPT +#ifndef _XOPEN_SOURCE #define _XOPEN_SOURCE 600 #endif +#endif + #include <sys/types.h> #include <sys/wait.h> #include <sys/stat.h> diff --git a/erts/include/internal/ethr_mutex.h b/erts/include/internal/ethr_mutex.h index ee861065c5..6c931e0cd4 100644 --- a/erts/include/internal/ethr_mutex.h +++ b/erts/include/internal/ethr_mutex.h @@ -354,7 +354,7 @@ void ethr_rwmutex_rwunlock(ethr_rwmutex *); #ifdef ETHR_MTX_HARD_DEBUG #define ETHR_MTX_HARD_ASSERT(A) \ - ((void) ((A) ? 1 : ethr_assert_failed(__FILE__, __LINE__, #A))) + ((void) ((A) ? 1 : ethr_assert_failed(__FILE__, __LINE__, __func__,#A))) #else #define ETHR_MTX_HARD_ASSERT(A) ((void) 1) #endif diff --git a/erts/include/internal/ethread.h b/erts/include/internal/ethread.h index 64f1fae6d8..72c054b588 100644 --- a/erts/include/internal/ethread.h +++ b/erts/include/internal/ethread.h @@ -31,6 +31,7 @@ #endif #include <stdlib.h> +#include "ethread_inline.h" #include "erl_errno.h" #if defined(DEBUG) @@ -51,16 +52,12 @@ # endif #endif -#undef ETHR_INLINE -#if defined(__GNUC__) -# define ETHR_INLINE __inline__ -#elif defined(__WIN32__) -# define ETHR_INLINE __forceinline -#endif #if defined(ETHR_DEBUG) || !defined(ETHR_INLINE) || ETHR_XCHK \ || (defined(__GNUC__) && defined(ERTS_MIXED_CYGWIN_VC)) # undef ETHR_INLINE # define ETHR_INLINE +# undef ETHR_FORCE_INLINE +# define ETHR_FORCE_INLINE # undef ETHR_TRY_INLINE_FUNCS #endif @@ -285,19 +282,6 @@ ETHR_PROTO_NORETURN__ ethr_fatal_error__(const char *file, const char *func, int err); -#if !defined(__GNUC__) -# define ETHR_AT_LEAST_GCC_VSN__(MAJ, MIN, PL) 0 -#elif !defined(__GNUC_MINOR__) -# define ETHR_AT_LEAST_GCC_VSN__(MAJ, MIN, PL) \ - ((__GNUC__ << 24) >= (((MAJ) << 24) | ((MIN) << 12) | (PL))) -#elif !defined(__GNUC_PATCHLEVEL__) -# define ETHR_AT_LEAST_GCC_VSN__(MAJ, MIN, PL) \ - (((__GNUC__ << 24) | (__GNUC_MINOR__ << 12)) >= (((MAJ) << 24) | ((MIN) << 12) | (PL))) -#else -# define ETHR_AT_LEAST_GCC_VSN__(MAJ, MIN, PL) \ - (((__GNUC__ << 24) | (__GNUC_MINOR__ << 12) | __GNUC_PATCHLEVEL__) >= (((MAJ) << 24) | ((MIN) << 12) | (PL))) -#endif - #if !ETHR_AT_LEAST_GCC_VSN__(2, 96, 0) #define __builtin_expect(X, Y) (X) #endif @@ -396,6 +380,18 @@ extern ethr_runtime_t ethr_runtime__; #include "ethr_atomics.h" /* The atomics API */ +#if defined (ETHR_OSE_THREADS) +static ETHR_INLINE void +ose_yield(void) +{ + if (get_ptype(current_process()) == OS_PRI_PROC) { + set_pri(get_pri(current_process())); + } else { + delay(1); + } +} +#endif + #if defined(__GNUC__) && !defined(ETHR_OSE_THREADS) # ifndef ETHR_SPIN_BODY # if defined(__i386__) || defined(__x86_64__) @@ -414,9 +410,9 @@ extern ethr_runtime_t ethr_runtime__; # endif #elif defined(ETHR_OSE_THREADS) # ifndef ETHR_SPIN_BODY -# define ETHR_SPIN_BODY set_pri(get_pri(current_process())) +# define ETHR_SPIN_BODY ose_yield() # else -# error "OSE should use set_pri(get_pri(current_process()))" +# error "OSE should use ose_yield()" # endif #endif @@ -449,7 +445,7 @@ extern ethr_runtime_t ethr_runtime__; # define ETHR_YIELD() (pthread_yield(), 0) # endif # elif defined(ETHR_OSE_THREADS) -# define ETHR_YIELD() (set_pri(get_pri(current_process())), 0) +# define ETHR_YIELD() (ose_yield(), 0) # else # define ETHR_YIELD() (ethr_compiler_barrier(), 0) # endif diff --git a/erts/include/internal/ethread_inline.h b/erts/include/internal/ethread_inline.h new file mode 100644 index 0000000000..ffb756c84f --- /dev/null +++ b/erts/include/internal/ethread_inline.h @@ -0,0 +1,49 @@ +/* + * %CopyrightBegin% + * + * Copyright Ericsson AB 2004-2014. All Rights Reserved. + * + * The contents of this file are subject to the Erlang Public License, + * Version 1.1, (the "License"); you may not use this file except in + * compliance with the License. You should have received a copy of the + * Erlang Public License along with this software. If not, it can be + * retrieved online at http://www.erlang.org/. + * + * Software distributed under the License is distributed on an "AS IS" + * basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + * the License for the specific language governing rights and limitations + * under the License. + * + * %CopyrightEnd% + */ + +#ifndef ETHREAD_INLINE_H__ +#define ETHREAD_INLINE_H__ + +#if !defined(__GNUC__) +# define ETHR_AT_LEAST_GCC_VSN__(MAJ, MIN, PL) 0 +#elif !defined(__GNUC_MINOR__) +# define ETHR_AT_LEAST_GCC_VSN__(MAJ, MIN, PL) \ + ((__GNUC__ << 24) >= (((MAJ) << 24) | ((MIN) << 12) | (PL))) +#elif !defined(__GNUC_PATCHLEVEL__) +# define ETHR_AT_LEAST_GCC_VSN__(MAJ, MIN, PL) \ + (((__GNUC__ << 24) | (__GNUC_MINOR__ << 12)) >= (((MAJ) << 24) | ((MIN) << 12) | (PL))) +#else +# define ETHR_AT_LEAST_GCC_VSN__(MAJ, MIN, PL) \ + (((__GNUC__ << 24) | (__GNUC_MINOR__ << 12) | __GNUC_PATCHLEVEL__) >= (((MAJ) << 24) | ((MIN) << 12) | (PL))) +#endif + +#undef ETHR_INLINE +#if defined(__GNUC__) +# define ETHR_INLINE __inline__ +# if ETHR_AT_LEAST_GCC_VSN__(3, 1, 1) +# define ETHR_FORCE_INLINE __inline__ __attribute__((__always_inline__)) +# else +# define ETHR_FORCE_INLINE __inline__ +# endif +#elif defined(__WIN32__) +# define ETHR_INLINE __forceinline +# define ETHR_FORCE_INLINE __forceinline +#endif + +#endif /* #ifndef ETHREAD_INLINE_H__ */ diff --git a/erts/include/internal/win/ethr_membar.h b/erts/include/internal/win/ethr_membar.h index 8237660b2c..a17f2459fc 100644 --- a/erts/include/internal/win/ethr_membar.h +++ b/erts/include/internal/win/ethr_membar.h @@ -63,13 +63,13 @@ do { \ #pragma intrinsic(_mm_sfence) #pragma intrinsic(_mm_lfence) -static __forceinline void +static ETHR_FORCE_INLINE void ethr_cfence__(void) { _ReadWriteBarrier(); } -static __forceinline void +static ETHR_FORCE_INLINE void ethr_mfence__(void) { #if ETHR_SIZEOF_PTR == 4 @@ -80,7 +80,7 @@ ethr_mfence__(void) _mm_mfence(); } -static __forceinline void +static ETHR_FORCE_INLINE void ethr_sfence__(void) { #if ETHR_SIZEOF_PTR == 4 @@ -91,7 +91,7 @@ ethr_sfence__(void) _mm_sfence(); } -static __forceinline void +static ETHR_FORCE_INLINE void ethr_lfence__(void) { #if ETHR_SIZEOF_PTR == 4 diff --git a/erts/lib_src/Makefile.in b/erts/lib_src/Makefile.in index cf1aef518a..b680c03b1d 100644 --- a/erts/lib_src/Makefile.in +++ b/erts/lib_src/Makefile.in @@ -465,6 +465,7 @@ RELEASE_LIBS=$(ERTS_LIBS) INTERNAL_RELEASE_INCLUDES= \ $(ERTS_INCL_INT)/README \ $(ERTS_INCL_INT)/ethread.h \ + $(ERTS_INCL_INT)/ethread_inline.h \ $(ERTS_INCL_INT)/ethr_mutex.h \ $(ERTS_INCL_INT)/ethr_optimized_fallbacks.h \ $(ERTS_INCL_INT)/ethr_atomics.h \ diff --git a/erts/lib_src/common/erl_misc_utils.c b/erts/lib_src/common/erl_misc_utils.c index 5a271c5268..d58a28b5cb 100644 --- a/erts/lib_src/common/erl_misc_utils.c +++ b/erts/lib_src/common/erl_misc_utils.c @@ -25,6 +25,7 @@ # include <windows.h> #endif +#include "ethread_inline.h" #include "erl_misc_utils.h" #if defined(__WIN32__) @@ -191,7 +192,7 @@ struct erts_cpu_info_t_ { #if defined(__WIN32__) -static __forceinline int +static ETHR_FORCE_INLINE int get_proc_affinity(erts_cpu_info_t *cpuinfo, cpu_set_t *cpuset) { DWORD_PTR pamask; @@ -206,7 +207,7 @@ get_proc_affinity(erts_cpu_info_t *cpuinfo, cpu_set_t *cpuset) } } -static __forceinline int +static ETHR_FORCE_INLINE int set_thr_affinity(cpu_set_t *set) { if (*set == (cpu_set_t) 0) @@ -1157,7 +1158,7 @@ read_topology(erts_cpu_info_t *cpuinfo) #define ERTS_MU_RELATION_CACHE 2 /* RelationCache */ #define ERTS_MU_RELATION_PROCESSOR_PACKAGE 3 /* RelationProcessorPackage */ -static __forceinline int +static ETHR_FORCE_INLINE int rel_cmp_val(int r) { switch (r) { diff --git a/erts/lib_src/common/ethr_aux.c b/erts/lib_src/common/ethr_aux.c index ceecdcef64..b77f2178f2 100644 --- a/erts/lib_src/common/ethr_aux.c +++ b/erts/lib_src/common/ethr_aux.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2010-2012. All Rights Reserved. + * Copyright Ericsson AB 2010-2014. All Rights Reserved. * * The contents of this file are subject to the Erlang Public License, * Version 1.1, (the "License"); you may not use this file except in @@ -360,10 +360,10 @@ static ethr_ts_event *ts_event_pool(int size, ethr_ts_event **endpp) int i; ethr_aligned_ts_event *atsev; atsev = ethr_mem__.std.alloc(sizeof(ethr_aligned_ts_event) * size - + ETHR_CACHE_LINE_SIZE); + + ETHR_CACHE_LINE_SIZE - 1); if (!atsev) return NULL; - if ((((ethr_uint_t) atsev) & ETHR_CACHE_LINE_MASK) == 0) + if ((((ethr_uint_t) atsev) & ETHR_CACHE_LINE_MASK) != 0) atsev = ((ethr_aligned_ts_event *) ((((ethr_uint_t) atsev) & ~ETHR_CACHE_LINE_MASK) + ETHR_CACHE_LINE_SIZE)); diff --git a/erts/lib_src/common/ethr_mutex.c b/erts/lib_src/common/ethr_mutex.c index 72b44033ad..4e56efaf8b 100644 --- a/erts/lib_src/common/ethr_mutex.c +++ b/erts/lib_src/common/ethr_mutex.c @@ -1433,7 +1433,7 @@ void LeaveCriticalSection(CRITICAL_SECTION *cs) #define ETHR_CND_WAIT__ ((ethr_sint32_t) 0x11dead11) #define ETHR_CND_WAKEUP__ ((ethr_sint32_t) 0x11beef11) -static __forceinline void +static ETHR_FORCE_INLINE void cond_wakeup(ethr_ts_event *tse) { ETHR_ASSERT(ethr_atomic32_read(&tse->uaflgs) == ETHR_CND_WAIT__); @@ -1574,7 +1574,7 @@ ethr_cond_wait(ethr_cond *cnd, ethr_mutex *mtx) return 0; } -static __forceinline void +static ETHR_FORCE_INLINE void posix_compliant_mtx_enqueue(ethr_mutex *mtx, ethr_ts_event *tse_start, ethr_ts_event *tse_end) @@ -1614,7 +1614,7 @@ posix_compliant_mtx_enqueue(ethr_mutex *mtx, } } -static __forceinline void +static ETHR_FORCE_INLINE void enqueue_cond_wakeups(ethr_ts_event *queue, int posix_compliant) { if (queue) { diff --git a/erts/preloaded/ebin/erl_prim_loader.beam b/erts/preloaded/ebin/erl_prim_loader.beam Binary files differindex 26f851fd7a..001c9c76ed 100644 --- a/erts/preloaded/ebin/erl_prim_loader.beam +++ b/erts/preloaded/ebin/erl_prim_loader.beam diff --git a/erts/preloaded/ebin/erlang.beam b/erts/preloaded/ebin/erlang.beam Binary files differindex 344176b71b..c5cf4b459f 100644 --- a/erts/preloaded/ebin/erlang.beam +++ b/erts/preloaded/ebin/erlang.beam diff --git a/erts/preloaded/ebin/erts_internal.beam b/erts/preloaded/ebin/erts_internal.beam Binary files differindex bd402f9a98..b467633a4d 100644 --- a/erts/preloaded/ebin/erts_internal.beam +++ b/erts/preloaded/ebin/erts_internal.beam diff --git a/erts/preloaded/ebin/init.beam b/erts/preloaded/ebin/init.beam Binary files differindex 5d49eed469..915a2d1aef 100644 --- a/erts/preloaded/ebin/init.beam +++ b/erts/preloaded/ebin/init.beam diff --git a/erts/preloaded/ebin/otp_ring0.beam b/erts/preloaded/ebin/otp_ring0.beam Binary files differindex 2607739d12..df1bf932a3 100644 --- a/erts/preloaded/ebin/otp_ring0.beam +++ b/erts/preloaded/ebin/otp_ring0.beam diff --git a/erts/preloaded/ebin/prim_eval.beam b/erts/preloaded/ebin/prim_eval.beam Binary files differindex d665d35ab1..95b4bbca2d 100644 --- a/erts/preloaded/ebin/prim_eval.beam +++ b/erts/preloaded/ebin/prim_eval.beam diff --git a/erts/preloaded/ebin/prim_file.beam b/erts/preloaded/ebin/prim_file.beam Binary files differindex 201252de34..5e4fc5ba84 100644 --- a/erts/preloaded/ebin/prim_file.beam +++ b/erts/preloaded/ebin/prim_file.beam diff --git a/erts/preloaded/ebin/prim_inet.beam b/erts/preloaded/ebin/prim_inet.beam Binary files differindex ff8265414e..9d9a4886d9 100644 --- a/erts/preloaded/ebin/prim_inet.beam +++ b/erts/preloaded/ebin/prim_inet.beam diff --git a/erts/preloaded/ebin/prim_zip.beam b/erts/preloaded/ebin/prim_zip.beam Binary files differindex f80d4a96e5..d98b0275f4 100644 --- a/erts/preloaded/ebin/prim_zip.beam +++ b/erts/preloaded/ebin/prim_zip.beam diff --git a/erts/preloaded/ebin/zlib.beam b/erts/preloaded/ebin/zlib.beam Binary files differindex dba3ec9546..0744bdb21d 100644 --- a/erts/preloaded/ebin/zlib.beam +++ b/erts/preloaded/ebin/zlib.beam diff --git a/erts/preloaded/src/add_abstract_code b/erts/preloaded/src/add_abstract_code index e670156d21..211a60c930 100644 --- a/erts/preloaded/src/add_abstract_code +++ b/erts/preloaded/src/add_abstract_code @@ -27,8 +27,18 @@ main([BeamFile,AbstrFile]) -> {ok,_,Chunks0} = beam_lib:all_chunks(BeamFile), {ok,Abstr} = file:consult(AbstrFile), - Chunks = lists:keyreplace("Abst", 1, Chunks0, - {"Abst",term_to_binary({raw_abstract_v1,Abstr})}), + Chunks1 = lists:keyreplace("Abst", 1, Chunks0, + {"Abst",term_to_binary({raw_abstract_v1,Abstr})}), + {"CInf",CInf0} = lists:keyfind("CInf", 1, Chunks1), + CInf = fix_options(CInf0), + Chunks = lists:keyreplace("CInf", 1, Chunks1, {"CInf",CInf}), {ok,Module} = beam_lib:build_module(Chunks), ok = file:write_file(BeamFile, Module), init:stop(). + +fix_options(CInf0) -> + CInf1 = binary_to_term(CInf0), + {options,Opts0} = lists:keyfind(options, 1, CInf1), + Opts = Opts0 -- [from_asm], + CInf = lists:keyreplace(options, 1, CInf1, {options,Opts}), + term_to_binary(CInf). diff --git a/erts/preloaded/src/erl_prim_loader.erl b/erts/preloaded/src/erl_prim_loader.erl index 578913b633..6b86a427ba 100644 --- a/erts/preloaded/src/erl_prim_loader.erl +++ b/erts/preloaded/src/erl_prim_loader.erl @@ -42,11 +42,11 @@ %% Public -export([start/3, set_path/1, get_path/0, get_file/1, get_files/2, - list_dir/1, read_file_info/1, get_cwd/0, get_cwd/1]). + list_dir/1, read_file_info/1, read_link_info/1, get_cwd/0, get_cwd/1]). %% Used by erl_boot_server -export([prim_init/0, prim_get_file/2, prim_list_dir/2, - prim_read_file_info/2, prim_get_cwd/2]). + prim_read_file_info/3, prim_get_cwd/2]). %% Used by escript and code -export([set_primary_archive/4, release_archives/0]). @@ -223,6 +223,12 @@ list_dir(Dir) -> read_file_info(File) -> check_file_result(read_file_info, File, request({read_file_info,File})). +-spec read_link_info(Filename) -> {'ok', FileInfo} | 'error' when + Filename :: string(), + FileInfo :: file:file_info(). +read_link_info(File) -> + check_file_result(read_link_info, File, request({read_link_info,File})). + -spec get_cwd() -> {'ok', string()} | 'error'. get_cwd() -> check_file_result(get_cwd, [], request({get_cwd,[]})). @@ -325,6 +331,9 @@ loop(State, Parent, Paths) -> {read_file_info,File} -> {Res,State1} = handle_read_file_info(State, File), {Res,State1,Paths}; + {read_link_info,File} -> + {Res,State1} = handle_read_link_info(State, File), + {Res,State1,Paths}; {get_cwd,[]} -> {Res,State1} = handle_get_cwd(State, []), {Res,State1,Paths}; @@ -387,10 +396,15 @@ handle_list_dir(State = #state{loader = inet}, Dir) -> ?SAFE2(inet_list_dir(State, Dir), State). handle_read_file_info(State = #state{loader = efile}, File) -> - ?SAFE2(efile_read_file_info(State, File), State); + ?SAFE2(efile_read_file_info(State, File, true), State); handle_read_file_info(State = #state{loader = inet}, File) -> ?SAFE2(inet_read_file_info(State, File), State). +handle_read_link_info(State = #state{loader = efile}, File) -> + ?SAFE2(efile_read_file_info(State, File, false), State); +handle_read_link_info(State = #state{loader = inet}, File) -> + ?SAFE2(inet_read_link_info(State, File), State). + handle_get_cwd(State = #state{loader = efile}, Drive) -> ?SAFE2(efile_get_cwd(State, Drive), State); handle_get_cwd(State = #state{loader = inet}, Drive) -> @@ -514,8 +528,8 @@ efile_list_dir(#state{prim_state = PS} = State, Dir) -> {Res, PS2} = prim_list_dir(PS, Dir), {Res, State#state{prim_state = PS2}}. -efile_read_file_info(#state{prim_state = PS} = State, File) -> - {Res, PS2} = prim_read_file_info(PS, File), +efile_read_file_info(#state{prim_state = PS} = State, File, FollowLinks) -> + {Res, PS2} = prim_read_file_info(PS, File, FollowLinks), {Res, State#state{prim_state = PS2}}. efile_get_cwd(#state{prim_state = PS} = State, Drive) -> @@ -718,6 +732,10 @@ inet_list_dir(State, Dir) -> inet_read_file_info(State, File) -> inet_send_and_rcv({read_file_info,File}, read_file_info, State). +%% -> {{ok,Info},State} | {{error,Reason},State} +inet_read_link_info(State, File) -> + inet_send_and_rcv({read_link_info,File}, read_link_info, State). + %% -> {{ok,Cwd},State} | {{error,Reason},State} inet_get_cwd(State, []) -> inet_send_and_rcv(get_cwd, get_cwd, State); @@ -951,16 +969,18 @@ prim_list_dir(PS, Dir) -> debug(PS, {return, Res2}), {Res2, PS3}. --spec prim_read_file_info(prim_state(), file:filename()) -> +-spec prim_read_file_info(prim_state(), file:filename(), boolean()) -> {{'ok', #file_info{}}, prim_state()} | {{'error', term()}, prim_state()}. -prim_read_file_info(PS, File) -> +prim_read_file_info(PS, File, FollowLinks) -> debug(PS, {read_file_info, File}), {Res2, PS2} = case name_split(PS#prim_state.primary_archive, File) of {file, PrimFile} -> - Res = prim_file:read_file_info(PrimFile), - {Res, PS}; + case FollowLinks of + true -> {prim_file:read_file_info(PrimFile), PS}; + false -> {prim_file:read_link_info(PrimFile), PS} + end; {archive, ArchiveFile, []} -> %% Fake top directory debug(PS, {archive_read_file_info, ArchiveFile}), @@ -1487,7 +1507,14 @@ real_path(Name,[Path|Paths],Acc,Links) -> [""|_] = LinkPaths -> real_path(Name,LinkPaths++Paths,[],[ThisFile|Links]); LinkPaths -> - real_path(Name,LinkPaths++Paths,Acc,[ThisFile|Links]) + % windows currently does not allow creation of relative symlinks + % across different drives + case erlang:system_info(os_type) of + {win32, _} -> + real_path(Name,LinkPaths++Paths,[],[ThisFile|Links]); + _ -> + real_path(Name,LinkPaths++Paths,Acc,[ThisFile|Links]) + end end; _ -> real_path(Name,Paths,This,Links) diff --git a/erts/preloaded/src/erlang.erl b/erts/preloaded/src/erlang.erl index fbc37bd955..98d7a942a6 100644 --- a/erts/preloaded/src/erlang.erl +++ b/erts/preloaded/src/erlang.erl @@ -79,7 +79,7 @@ -export([binary_to_integer/1,binary_to_integer/2]). -export([binary_to_list/1]). -export([binary_to_list/3, binary_to_term/1, binary_to_term/2]). --export([bit_size/1, bitsize/1, bitstr_to_list/1, bitstring_to_list/1]). +-export([bit_size/1, bitsize/1, bitstring_to_list/1]). -export([bump_reductions/1, byte_size/1, call_on_load_function/1]). -export([cancel_timer/1, check_old_code/1, check_process_code/2, check_process_code/3, crc32/1]). @@ -91,7 +91,7 @@ -export([external_size/2, finish_after_on_load/2, finish_loading/1, float/1]). -export([float_to_binary/1, float_to_binary/2, float_to_list/1, float_to_list/2]). --export([fun_info/2, fun_to_list/1, function_exported/3]). +-export([fun_info/2, fun_info_mfa/1, fun_to_list/1, function_exported/3]). -export([garbage_collect/0, garbage_collect/1, garbage_collect/2]). -export([garbage_collect_message_area/0, get/0, get/1, get_keys/1]). -export([get_module_info/1, get_stacktrace/0, group_leader/0]). @@ -100,7 +100,7 @@ -export([integer_to_binary/1, integer_to_list/1]). -export([iolist_size/1, iolist_to_binary/1]). -export([is_alive/0, is_builtin/3, is_process_alive/1, length/1, link/1]). --export([list_to_atom/1, list_to_binary/1, list_to_bitstr/1]). +-export([list_to_atom/1, list_to_binary/1]). -export([list_to_bitstring/1, list_to_existing_atom/1, list_to_float/1]). -export([list_to_integer/1, list_to_integer/2]). -export([list_to_pid/1, list_to_tuple/1, loaded/0]). @@ -361,25 +361,15 @@ binary_to_list(_Binary, _Start, _Stop) -> %% binary_to_term/1 -spec binary_to_term(Binary) -> term() when Binary :: ext_binary(). -binary_to_term(Binary) -> - %% This BIF may throw badarg while trapping - try - erts_internal:binary_to_term(Binary) - catch - error:Reason -> erlang:error(Reason,[Binary]) - end. +binary_to_term(_Binary) -> + erlang:nif_error(undefined). %% binary_to_term/2 -spec binary_to_term(Binary, Opts) -> term() when Binary :: ext_binary(), Opts :: [safe]. -binary_to_term(Binary, Opts) -> - %% This BIF may throw badarg while trapping - try - erts_internal:binary_to_term(Binary,Opts) - catch - error:Reason -> erlang:error(Reason,[Binary,Opts]) - end. +binary_to_term(_Binary, _Opts) -> + erlang:nif_error(undefined). %% bit_size/1 %% Shadowed by erl_bif_types: erlang:bit_size/1 @@ -394,12 +384,6 @@ bit_size(_Bitstring) -> bitsize(_P1) -> erlang:nif_error(undefined). -%% bitstr_to_list/1 --spec erlang:bitstr_to_list(P1) -> [byte() | bitstring()] when - P1 :: bitstring(). -bitstr_to_list(_P1) -> - erlang:nif_error(undefined). - %% bitstring_to_list/1 -spec bitstring_to_list(Bitstring) -> [byte() | bitstring()] when Bitstring :: bitstring(). @@ -843,6 +827,15 @@ float_to_list(_Float, _Options) -> fun_info(_Fun, _Item) -> erlang:nif_error(undefined). +%% fun_info_mfa/1 +-spec erlang:fun_info_mfa(Fun) -> {Mod, Name, Arity} when + Fun :: function(), + Mod :: atom(), + Name :: atom(), + Arity :: non_neg_integer(). +fun_info_mfa(_Fun) -> + erlang:nif_error(undefined). + %% fun_to_list/1 -spec erlang:fun_to_list(Fun) -> string() when Fun :: function(). @@ -1082,12 +1075,6 @@ list_to_atom(_String) -> list_to_binary(_IoList) -> erlang:nif_error(undefined). -%% list_to_bitstr/1 --spec erlang:list_to_bitstr(P1) -> bitstring() when - P1 :: bitstring_list(). -list_to_bitstr(_P1) -> - erlang:nif_error(undefined). - %% list_to_bitstring/1 -spec list_to_bitstring(BitstringList) -> bitstring() when BitstringList :: bitstring_list(). @@ -1745,9 +1732,9 @@ is_pid(_Term) -> erlang:nif_error(undefined). %% Shadowed by erl_bif_types: erlang:is_map/1 --spec is_map(Map) -> boolean() when - Map :: map(). -is_map(_Map) -> +-spec is_map(Term) -> boolean() when + Term :: term(). +is_map(_Term) -> erlang:nif_error(undefined). %% Shadowed by erl_bif_types: erlang:is_port/1 @@ -2264,7 +2251,6 @@ tuple_to_list(_Tuple) -> (modified_timing_level) -> integer() | undefined; (multi_scheduling) -> disabled | blocked | enabled; (multi_scheduling_blockers) -> [PID :: pid()]; - (otp_correction_package) -> string(); (otp_release) -> string(); (port_count) -> non_neg_integer(); (port_limit) -> pos_integer(); @@ -2287,6 +2273,7 @@ tuple_to_list(_Tuple) -> (system_architecture) -> string(); (threads) -> boolean(); (thread_pool_size) -> non_neg_integer(); + (tolerant_timeofday) -> enabled | disabled; (trace_control_word) -> non_neg_integer(); (update_cpu_info) -> changed | unchanged; (version) -> string(); diff --git a/erts/preloaded/src/erts.app.src b/erts/preloaded/src/erts.app.src index a15da3a421..345a6ae3be 100644 --- a/erts/preloaded/src/erts.app.src +++ b/erts/preloaded/src/erts.app.src @@ -35,7 +35,6 @@ {registered, []}, {applications, []}, {env, []}, - {mod, {erts, []}}, {runtime_dependencies, ["stdlib-2.0", "kernel-3.0", "sasl-2.4"]} ]}. diff --git a/erts/preloaded/src/erts_internal.erl b/erts/preloaded/src/erts_internal.erl index edcd50c77e..2c5bd82cf0 100644 --- a/erts/preloaded/src/erts_internal.erl +++ b/erts/preloaded/src/erts_internal.erl @@ -29,8 +29,8 @@ -module(erts_internal). -export([await_port_send_result/3]). --export([binary_to_term/1, binary_to_term/2]). -export([cmp_term/2]). +-export([map_to_tuple_keys/1]). -export([port_command/3, port_connect/2, port_close/1, port_control/3, port_call/3, port_info/1, port_info/2]). @@ -161,17 +161,6 @@ request_system_task(_Pid, _Prio, _Request) -> check_process_code(_Module, _OptionList) -> erlang:nif_error(undefined). --spec binary_to_term(Binary) -> term() when - Binary :: binary(). -binary_to_term(_Binary) -> - erlang:nif_error(undefined). - --spec binary_to_term(Binary, Opts) -> term() when - Binary :: binary(), - Opts :: [safe]. -binary_to_term(_Binary, _Opts) -> - erlang:nif_error(undefined). - %% term compare where integer() < float() = true -spec cmp_term(A,B) -> Result when @@ -181,3 +170,11 @@ binary_to_term(_Binary, _Opts) -> cmp_term(_A,_B) -> erlang:nif_error(undefined). + +%% return the internal key tuple for map keys +-spec map_to_tuple_keys(M) -> Keys when + M :: map(), + Keys :: tuple(). + +map_to_tuple_keys(_M) -> + erlang:nif_error(undefined). diff --git a/erts/preloaded/src/init.erl b/erts/preloaded/src/init.erl index ab8464956c..e95e11b3e6 100644 --- a/erts/preloaded/src/init.erl +++ b/erts/preloaded/src/init.erl @@ -1041,7 +1041,7 @@ start_em([]) -> ok. start_it([]) -> ok; start_it({eval,Bin}) -> - Str = binary_to_list(Bin), + Str = b2s(Bin), {ok,Ts,_} = erl_scan:string(Str), Ts1 = case reverse(Ts) of [{dot,_}|_] -> Ts; diff --git a/erts/preloaded/src/prim_inet.erl b/erts/preloaded/src/prim_inet.erl index 143c718130..79ff013c77 100644 --- a/erts/preloaded/src/prim_inet.erl +++ b/erts/preloaded/src/prim_inet.erl @@ -25,7 +25,7 @@ %% Primitive inet_drv interface --export([open/3, open/4, fdopen/4, close/1]). +-export([open/3, open/4, fdopen/4, fdopen/5, close/1]). -export([bind/3, listen/1, listen/2, peeloff/2]). -export([connect/3, connect/4, async_connect/4]). -export([accept/1, accept/2, async_accept/2]). @@ -70,7 +70,12 @@ open(Protocol, Family, Type, Opts) -> open(Protocol, Family, Type, Opts, ?INET_REQ_OPEN, []). fdopen(Protocol, Family, Type, Fd) when is_integer(Fd) -> - open(Protocol, Family, Type, [], ?INET_REQ_FDOPEN, ?int32(Fd)). + fdopen(Protocol, Family, Type, Fd, true). + +fdopen(Protocol, Family, Type, Fd, Bound) + when is_integer(Fd), Bound == true orelse Bound == false -> + open(Protocol, Family, Type, [], ?INET_REQ_FDOPEN, + [?int32(Fd), enc_value_2(bool, Bound)]). open(Protocol, Family, Type, Opts, Req, Data) -> Drv = protocol2drv(Protocol), diff --git a/erts/test/otp_SUITE.erl b/erts/test/otp_SUITE.erl index 1fb452501f..229d10ccee 100644 --- a/erts/test/otp_SUITE.erl +++ b/erts/test/otp_SUITE.erl @@ -328,7 +328,9 @@ erl_file_encoding(_Config) -> Wc = filename:join([Root,"**","*.erl"]), ErlFiles = ordsets:subtract(ordsets:from_list(filelib:wildcard(Wc)), release_files(Root, "*.erl")), + {ok, MP} = re:compile(".*lib/(ic)|(orber)|(cos).*", [unicode]), Fs = [F || F <- ErlFiles, + filter_use_latin1_coding(F, MP), case epp:read_encoding(F) of none -> false; _ -> true @@ -342,6 +344,14 @@ erl_file_encoding(_Config) -> ?t:fail() end. +filter_use_latin1_coding(F, MP) -> + case re:run(F, MP) of + nomatch -> + true; + {match, _} -> + false + end. + xml_file_encoding(_Config) -> XmlFiles = xml_files(), Fs = [F || F <- XmlFiles, is_bad_encoding(F)], @@ -382,12 +392,18 @@ is_bad_encoding(File) -> end. runtime_dependencies(Config) -> + %% Ignore applications intentionally not declaring dependencies + %% found by xref. + IgnoreApps = [diameter], + + %% Verify that (at least) OTP application runtime dependencies found %% by xref are listed in the runtime_dependencies field of the .app file %% of each application. Server = ?config(xref_server, Config), {ok, AE} = xref:q(Server, "AE"), SAE = lists:keysort(1, AE), + put(ignored_failures, []), {AppDep, AppDeps} = lists:foldl(fun ({App, App}, Acc) -> Acc; ({App, Dep}, {undefined, []}) -> @@ -399,8 +415,45 @@ runtime_dependencies(Config) -> end, {undefined, []}, SAE), - [] = check_apps_deps([AppDep|AppDeps]), - ok. + [] = lists:filter(fun ({missing_runtime_dependency, + AppFile, + common_test}) -> + %% The test_server app is contaminated by + %% common_test when run in a source tree. It + %% should however *not* be contaminated + %% when run in an installation. + case {filename:basename(AppFile), + is_run_in_src_tree()} of + {"test_server.app", true} -> + false; + _ -> + true + end; + (_) -> + true + end, + check_apps_deps([AppDep|AppDeps], IgnoreApps)), + case IgnoreApps of + [] -> + ok; + _ -> + Comment = lists:flatten(io_lib:format("Ignored applications: ~p " + "Ignored failures: ~p", + [IgnoreApps, + get(ignored_failures)])), + {comment, Comment} + end. + +is_run_in_src_tree() -> + %% At least currently run_erl is not present in <code-root>/bin + %% in the source tree, but present in <code-root>/bin of an + %% ordinary installation. + case file:read_file_info(filename:join([code:root_dir(), + "bin", + "run_erl"])) of + {ok, _} -> false; + {error, _} -> true + end. have_rdep(_App, [], _Dep) -> false; @@ -414,28 +467,43 @@ have_rdep(App, [RDep | RDeps], Dep) -> have_rdep(App, RDeps, Dep) end. -check_app_deps(_App, _AppFile, _AFDeps, []) -> +check_app_deps(_App, _AppFile, _AFDeps, [], _IgnoreApps) -> []; -check_app_deps(App, AppFile, AFDeps, [XRDep | XRDeps]) -> - ResOtherDeps = check_app_deps(App, AppFile, AFDeps, XRDeps), +check_app_deps(App, AppFile, AFDeps, [XRDep | XRDeps], IgnoreApps) -> + ResOtherDeps = check_app_deps(App, AppFile, AFDeps, XRDeps, IgnoreApps), case have_rdep(App, AFDeps, XRDep) of true -> ResOtherDeps; false -> - [{missing_runtime_dependency, AppFile, XRDep} | ResOtherDeps] + Failure = {missing_runtime_dependency, AppFile, XRDep}, + case lists:member(App, IgnoreApps) of + true -> + put(ignored_failures, [Failure | get(ignored_failures)]), + ResOtherDeps; + false -> + [Failure | ResOtherDeps] + end end. -check_apps_deps([]) -> +check_apps_deps([], _IgnoreApps) -> []; -check_apps_deps([{App, Deps}|AppDeps]) -> - ResOtherApps = check_apps_deps(AppDeps), +check_apps_deps([{App, Deps}|AppDeps], IgnoreApps) -> + ResOtherApps = check_apps_deps(AppDeps, IgnoreApps), AppFile = code:where_is_file(atom_to_list(App) ++ ".app"), {ok,[{application, App, Info}]} = file:consult(AppFile), case lists:keyfind(runtime_dependencies, 1, Info) of {runtime_dependencies, RDeps} -> - check_app_deps(App, AppFile, RDeps, Deps) ++ ResOtherApps; + check_app_deps(App, AppFile, RDeps, Deps, IgnoreApps) + ++ ResOtherApps; false -> - [{missing_runtime_dependencies_key, AppFile} | ResOtherApps] + Failure = {missing_runtime_dependencies_key, AppFile}, + case lists:member(App, IgnoreApps) of + true -> + put(ignored_failures, [Failure | get(ignored_failures)]), + ResOtherApps; + false -> + [Failure | ResOtherApps] + end end. %%% diff --git a/erts/test/upgrade_SUITE.erl b/erts/test/upgrade_SUITE.erl index ee84a5dfd7..d5a920e03d 100644 --- a/erts/test/upgrade_SUITE.erl +++ b/erts/test/upgrade_SUITE.erl @@ -264,7 +264,7 @@ do_upgrade(FromVsn,FromApps,ToRel,ToApps,InstallDir) -> %%%----------------------------------------------------------------- %%% Library functions previous_major("17") -> - "r16"; + "r16b"; previous_major(Rel) -> integer_to_list(list_to_integer(Rel)-1). diff --git a/erts/vsn.mk b/erts/vsn.mk index 081fb66398..b6a38f9361 100644 --- a/erts/vsn.mk +++ b/erts/vsn.mk @@ -17,7 +17,7 @@ # %CopyrightEnd% # -VSN = 6.0 +VSN = 6.2 # Port number 4365 in 4.2 # Port number 4366 in 4.3 diff --git a/lib/asn1/c_src/asn1_erl_nif.c b/lib/asn1/c_src/asn1_erl_nif.c index 8a0e4b1cf0..53e3aa1678 100644 --- a/lib/asn1/c_src/asn1_erl_nif.c +++ b/lib/asn1/c_src/asn1_erl_nif.c @@ -941,16 +941,31 @@ static int ber_decode_value(ErlNifEnv* env, ERL_NIF_TERM *value, unsigned char * int maybe_ret; unsigned int len = 0; unsigned int lenoflen = 0; - int indef = 0; unsigned char *tmp_out_buff; ERL_NIF_TERM term = 0, curr_head = 0; if (((in_buf[*ib_index]) & 0x80) == ASN1_SHORT_DEFINITE_LENGTH) { len = in_buf[*ib_index]; - } else if (in_buf[*ib_index] == ASN1_INDEFINITE_LENGTH - ) - indef = 1; - else /* long definite length */{ + } else if (in_buf[*ib_index] == ASN1_INDEFINITE_LENGTH) { + (*ib_index)++; + curr_head = enif_make_list(env, 0); + if (*ib_index+1 >= in_buf_len) { + return ASN1_INDEF_LEN_ERROR; + } + while (!(in_buf[*ib_index] == 0 && in_buf[*ib_index + 1] == 0)) { + maybe_ret = ber_decode(env, &term, in_buf, ib_index, in_buf_len); + if (maybe_ret <= ASN1_ERROR) { + return maybe_ret; + } + curr_head = enif_make_list_cell(env, term, curr_head); + if (*ib_index+1 >= in_buf_len) { + return ASN1_INDEF_LEN_ERROR; + } + } + enif_make_reverse_list(env, curr_head, value); + (*ib_index) += 2; /* skip the indefinite length end bytes */ + return ASN1_OK; + } else /* long definite length */{ lenoflen = (in_buf[*ib_index] & 0x7f); /*length of length */ if (lenoflen > (in_buf_len - (*ib_index + 1))) return ASN1_LEN_ERROR; @@ -965,23 +980,7 @@ static int ber_decode_value(ErlNifEnv* env, ERL_NIF_TERM *value, unsigned char * if (len > (in_buf_len - (*ib_index + 1))) return ASN1_VALUE_ERROR; (*ib_index)++; - if (indef == 1) { /* in this case it is desireably to check that indefinite length - end bytes exist in inbuffer */ - curr_head = enif_make_list(env, 0); - while (!(in_buf[*ib_index] == 0 && in_buf[*ib_index + 1] == 0)) { - if (*ib_index >= in_buf_len) - return ASN1_INDEF_LEN_ERROR; - - if ((maybe_ret = ber_decode(env, &term, in_buf, ib_index, in_buf_len)) - <= ASN1_ERROR - ) - return maybe_ret; - curr_head = enif_make_list_cell(env, term, curr_head); - } - enif_make_reverse_list(env, curr_head, value); - (*ib_index) += 2; /* skip the indefinite length end bytes */ - } else if (form == ASN1_CONSTRUCTED) - { + if (form == ASN1_CONSTRUCTED) { int end_index = *ib_index + len; if (end_index > in_buf_len) return ASN1_LEN_ERROR; diff --git a/lib/asn1/doc/src/asn1_ug.xml b/lib/asn1/doc/src/asn1_ug.xml index ee54fdffd7..8b33497dd3 100644 --- a/lib/asn1/doc/src/asn1_ug.xml +++ b/lib/asn1/doc/src/asn1_ug.xml @@ -34,23 +34,25 @@ <section> <title>Features</title> - <p>The Asn1 application provides: - </p> + <p>The Asn1 application provides:</p> <list type="bulleted"> <item>An ASN.1 compiler for Erlang, which generates encode and decode functions to be used by Erlang programs sending and receiving ASN.1 specified data.</item> <item>Run-time functions used by the generated code.</item> - <item>The supported encoding rules are: + <item>Support for the following encoding rules: <list> <item> Basic Encoding Rules (<em>BER</em>) </item> <item> - Distinguished Encoding Rules (<em>DER</em>), a specialized form of BER that is used in security-conscious applications. + Distinguished Encoding Rules (<em>DER</em>), a specialized + form of BER that is used in security-conscious + applications. </item> <item> - Packed Encoding Rules (<em>PER</em>) both the aligned and unaligned variant. + Packed Encoding Rules (<em>PER</em>); both the aligned and + unaligned variant. </item> </list> </item> @@ -59,71 +61,41 @@ <section> <title>Overview</title> - <p>ASN.1 (Abstract Syntax Notation 1) is a formal language for describing data structures to be exchanged between distributed computer systems. - The purpose of ASN.1 is to have - a platform and programming language independent notation to express - types using a - standardized set of rules for the transformation of values of - a defined type, into a stream of bytes. This stream of bytes - can then be sent on a communication channel set up by the - lower layers in the stack of communication protocols e.g. - TCP/IP or encapsulated within UDP packets. This way, two - different applications written in two completely different - programming languages running on different computers with - different internal representation of data can exchange - instances of structured data types (instead of exchanging - bytes or bits). This makes programming faster and easier since no code - has to be written to process the transport format of the - data. - </p> - <p>To write a network application which processes ASN.1 encoded - messages, it is prudent and sometimes essential to have a set - of off-line development tools such as an ASN.1 compiler which - can generate the encode and decode logic for the specific ASN.1 - data types. It is also necessary to combine this with some - general language-specific runtime support for ASN.1 encoding and - decoding. - </p> - <p>The ASN.1 compiler must be directed towards a target language - or a set of closely related languages. This manual describes a - compiler which is directed towards the functional language - Erlang. In order to use this compiler, familiarity with the - language Erlang is essential. Therefore, the runtime support for ASN.1 is - also closely related to the language Erlang and - consist of a number of functions, which the - compiler uses. The types in ASN.1 and how to represent - values of those types in Erlang are described in this manual. - </p> - <p>The following document is structured so that the first part describes - how to use ASN.1 compiler, and then there are descriptions of all - the primitive and constructed ASN.1 types and their representation - in Erlang, - </p> + <p>ASN.1 (Abstract Syntax Notation One) is a formal language for + describing data structures to be exchanged between distributed + computer systems. The purpose of ASN.1 is to have a platform + and programming language independent notation to express types + using a standardized set of rules for the transformation of + values of a defined type into a stream of bytes. This stream of + bytes can then be sent on any type of communication + channel. This way, two applications written in different + programming languages running on different computers with + different internal representation of data can exchange instances + of structured data types.</p> </section> <section> <title>Prerequisites</title> - <p>It is assumed that the reader is familiar with the ASN.1 notation - as documented in the standard definition [<cite id="X.680"></cite>] which is - the primary text. It may also be helpful, but not necessary, - to read the standard definitions - [<cite id="X.681"></cite>] [<cite id="X.682"></cite>] [<cite id="X.683"></cite>] - [<cite id="X.690"></cite>] [<cite id="X.691"></cite>]. </p> - <p>A very good book explaining those reference texts is - [<cite id="DUBUISSON"></cite>], free to download at - <url href="http://www.oss.com/asn1/dubuisson.html">http://www.oss.com/asn1/dubuisson.html </url>. + <p>It is assumed that the reader is familiar with the ASN.1 + notation as documented in the standard definition [<cite + id="X.680"></cite>] which is the primary text. It may also be + helpful, but not necessary, to read the standard definitions + [<cite id="X.681"></cite>] [<cite id="X.682"></cite>] [<cite + id="X.683"></cite>] [<cite id="X.690"></cite>] [<cite + id="X.691"></cite>]. </p> + <p>A good book explaining those reference texts is + [<cite id="DUBUISSON"></cite>], which is free to download at + <url href="http://www.oss.com/asn1/dubuisson.html">http://www.oss.com/asn1/dubuisson.html</url>. </p> </section> <section> - <title>Capability</title> + <title>Capabilities</title> <p>This application covers all features of ASN.1 up to the 1997 - edition of the specification. In the 2002 edition of ASN.1 a number of - new features where introduced of which some are supported while - others are not. For example the - ECN (Encoding Control Notation) and XML notation are still - unsupported. Though, the other features of the 2002 edition are - fully or partly supported as shown below:</p> + edition of the specification. In the 2002 edition of ASN.1 a + number of new features were introduced. The following features + of the 2002 edition are fully or partly supported as shown + below:</p> <list type="bulleted"> <item> <p>Decimal notation (e.g., "1.5e3") for REAL values. The @@ -131,7 +103,7 @@ supported.</p> </item> <item> - <p>The RELATIVE-OID type for relative object identifiers are + <p>The RELATIVE-OID type for relative object identifiers is fully supported.</p> </item> <item> @@ -141,16 +113,16 @@ constraint is not a PER-visible constraint.</p> </item> <item> - <p>The subtype constraint by regular expressions (PATTERN) for character string types is parsed when compiling, but no further action is taken. This constraint is not a PER-visible constraint.</p> + <p>The subtype constraint by regular expressions (PATTERN) + for character string types is parsed when compiling, but no + further action is taken. This constraint is not a + PER-visible constraint.</p> </item> <item> <p>Multiple-line comments as in C, <c>/* ... */</c>, are supported.</p> </item> </list> - <p>It should also be added here that the encoding formats - supported are <em>BER</em>, <em>DER</em>, <em>PER aligned - basic</em> variant and <em>PER unaligned basic</em> variant.</p> </section> </section> @@ -162,19 +134,17 @@ <title>A First Example</title> <p>The following example demonstrates the basic functionality used to run the Erlang ASN.1 compiler.</p> - <p>First, create a file called <c>People.asn</c> containing the following:</p> + <p>Create a file called <c>People.asn</c> containing the following:</p> <pre> -People DEFINITIONS IMPLICIT TAGS ::= - +People DEFINITIONS AUTOMATIC TAGS ::= BEGIN -EXPORTS Person; - -Person ::= [PRIVATE 19] SEQUENCE { - name PrintableString, - location INTEGER {home(0),field(1),roving(2)}, - age INTEGER OPTIONAL } + Person ::= SEQUENCE { + name PrintableString, + location INTEGER {home(0),field(1),roving(2)}, + age INTEGER OPTIONAL + } END </pre> - <p>This file (<c>people.asn</c>) must be compiled before it can be + <p>This file (<c>People.asn</c>) must be compiled before it can be used. The ASN.1 compiler checks that the syntax is correct and that the text represents proper ASN.1 code before generating an abstract @@ -186,14 +156,14 @@ END </pre> The following shows how the compiler can be called from the Erlang shell:</p> <pre> -1><input>asn1ct:compile("People", [ber]).</input> +1><input> asn1ct:compile("People", [ber]).</input> ok 2> </pre> <p>The <c>verbose</c> option can be given to have information about the generated files printed:</p> <pre> -2><input>asn1ct:compile("People", [ber,verbose]).</input> +2><input> asn1ct:compile("People", [ber,verbose]).</input> Erlang ASN.1 compiling "People.asn" --{generated,"People.asn1db"}-- --{generated,"People.hrl"}-- @@ -201,17 +171,17 @@ Erlang ASN.1 compiling "People.asn" ok 3> </pre> - <p>The ASN.1 module People is now accepted and the abstract syntax tree - is saved in the <c>People.asn1db</c> file, the - generated Erlang code is compiled using the Erlang compiler and - loaded into the Erlang runtime system. Now there is a user interface - for <c>encode/2</c> and <c>decode/2</c> in the module People, - which is invoked by: - <br></br> -<c><![CDATA['People':encode(<Type name>,<Value>),]]></c> <br></br> - + <p>The ASN.1 module <c>People</c> is now accepted and the + abstract syntax tree is saved in the <c>People.asn1db</c> file; + the generated Erlang code is compiled using the Erlang compiler + and loaded into the Erlang run-time system. Now there is an API + for <c>encode/2</c> and <c>decode/2</c> in the module + <c>People</c>, which is invoked by: <br></br> + <c><![CDATA['People':encode(<Type name>, <Value>)]]></c> + <br></br> or <br></br> -<c><![CDATA['People':decode(<Type name>,<Value>),]]></c></p> +<c><![CDATA['People':decode(<Type name>, <Value>)]]></c></p> + <p>Assume there is a network application which receives instances of the ASN.1 defined type Person, modifies and sends them back again:</p> @@ -234,8 +204,7 @@ receive constructed and encoded using <c>'People':encode('Person',Answer)</c> which takes an instance of a defined ASN.1 type and transforms it to a - binary according to the BER or PER - encoding-rules. + binary according to the BER or PER encoding rules. <br></br> The encoder and the decoder can also be run from the shell.</p> @@ -252,13 +221,13 @@ The encoder and the decoder can also be run from <section> <title>Module dependencies</title> - <p>It is common that asn1 modules import defined types, values and - other entities from another asn1 module.</p> - <p>Earlier versions of the asn1 compiler required that modules that + <p>It is common that ASN.1 modules import defined types, values and + other entities from another ASN.1 module.</p> + <p>Earlier versions of the ASN.1 compiler required that modules that were imported from had to be compiled before the module that - imported. This caused problems when asn1 modules had circular + imported. This caused problems when ASN.1 modules had circular dependencies.</p> - <p>Now are referenced modules parsed when the compiler finds an + <p>Referenced modules are now parsed when the compiler finds an entity that is imported. There will not be any code generated for the referenced module. However, the compiled module rely on that the referenced modules also will be compiled.</p> @@ -310,7 +279,7 @@ erlc -o ../asnfiles -I ../asnfiles -I /usr/local/standards/asn1 Person.asn </item> <tag><c>-I IncludeDir</c></tag> <item> - <p>Where to search for <c>.asn1db</c> files and asn1 + <p>Where to search for <c>.asn1db</c> files and ASN.1 source specs in order to resolve references to other modules. This option can be repeated many times if there are several places to search in. The compiler will always @@ -322,26 +291,26 @@ erlc -o ../asnfiles -I ../asnfiles -I /usr/local/standards/asn1 Person.asn </item> <tag><c>+asn1config</c></tag> <item> - <p>This functionality works together with the flags - <c>ber</c>. It enables the + <p>This functionality works together with the + <c>ber</c> option. It enables the specialized decodes, see the <seealso marker="asn1_spec">Specialized Decode</seealso> chapter. </p> </item> <tag><c>+undec_rest</c></tag> <item> - <p>A buffer that holds a message, being decoded may - also have some following bytes. Now it is possible to get - those following bytes returned together with the decoded - value. If an asn1 spec is compiled with this option a tuple - <c>{ok,Value,Rest}</c> is returned. <c>Rest</c> may be a - list or a binary. Earlier versions of the compiler ignored - those following bytes.</p> + <p>A buffer that holds a message being decoded may also have + trailing bytes. If those trailing bytes are important they + can be returned along with the decoded value by compiling + the ASN.1 specification with the <c>+undec_rest</c> option. + The return value from the decoder will be + <c>{ok,Value,Rest}</c> where <c>Rest</c> is a binary + containing the trailing bytes.</p> </item> <tag><c>+'Any Erlc Option'</c></tag> <item> <p>You may add any option to the Erlang compiler when compiling the generated Erlang files. Any option - unrecognised by the asn1 compiler will be passed to the + unrecognized by the ASN.1 compiler will be passed to the Erlang compiler.</p> </item> </taglist> @@ -366,10 +335,6 @@ asn1ct:compile("H323-MESSAGES.asn1",[ber]). </pre> asn1ct:compile("H323-MESSAGES.asn1",[per]). </pre> <p>The generic encode and decode functions can be invoked like this:</p> <pre> -asn1ct:encode('H323-MESSAGES','SomeChoiceType',{call,"octetstring"}). -asn1ct:decode('H323-MESSAGES','SomeChoiceType',Bytes). </pre> - <p>Or, preferable like:</p> - <pre> 'H323-MESSAGES':encode('SomeChoiceType',{call,"octetstring"}). 'H323-MESSAGES':decode('SomeChoiceType',Bytes). </pre> </section> @@ -389,7 +354,7 @@ asn1ct:decode('H323-MESSAGES','SomeChoiceType',Bytes). </pre> compile time appear on the screen together with a line number indicating where in the source file the error was detected. If no errors are found, an Erlang ASN.1 module will - be created as default.</p> + be created.</p> <p>The run-time encoders and decoders execute within a catch and returns <c>{ok, Data}</c> or <c>{error, {asn1, Description}}</c> where @@ -400,18 +365,18 @@ asn1ct:decode('H323-MESSAGES','SomeChoiceType',Bytes). </pre> <section> <marker id="inlineExamples"></marker> - <title>Multi File Compilation</title> - <p>There are various reasons for using a multi file compilation:</p> + <title>Multi-file Compilation</title> + <p>There are various reasons for using multi-file compilation:</p> <list type="bulleted"> - <item>You want to choose name for the generated module by - any reason. Maybe you need to compile the same specs for - different encoding/decoding standards.</item> + <item>You want to choose the name for the generated module, + perhaps because you need to compile the same specs for + different encoding rules.</item> <item>You want only one resulting module.</item> </list> - <p>You need to specify which asn1 specs you will + <p>You need to specify which ASN.1 specs you will compile in a module that must have the extension <c>.set.asn</c>. You chose name of the module and provide the - names of the asn1 specs. For instance, if you have the specs + names of the ASN.1 specs. For instance, if you have the specs <c>File1.asn</c>, <c>File2.asn</c> and <c>File3.asn</c> your module <c>MyModule.set.asn</c> will look like:</p> <pre> @@ -422,11 +387,45 @@ File3.asn </pre> <code type="none"> ~> erlc MyModule.set.asn </code> <p>the result will be one merged module <c>MyModule.erl</c> with - the generated code from the three asn1 specs. + the generated code from the three ASN.1 specs. </p> </section> <section> + <title>A quick note about tags</title> + + <p>Tags used to be important for all users of ASN.1, because it + was necessary to manually add tags to certain constructs in order + for the ASN.1 specification to be valid. Here is an example of + an old-style specification:</p> + + <pre> +Tags DEFINITIONS ::= +BEGIN + Afters ::= CHOICE { cheese [0] IA5String, + dessert [1] IA5String } +END </pre> + + <p>Without the tags (the numbers in square brackets) the ASN.1 + compiler would refuse to compile the file.</p> + + <p>In 1994 the global tagging mode AUTOMATIC TAGS was introduced. + By putting AUTOMATIC TAGS in the module header, the ASN.1 compiler + will automatically add tags when needed. Here is the same + specification in AUTOMATIC TAGS mode:</p> + + <pre> +Tags DEFINITIONS AUTOMATIC TAGS ::= +BEGIN + Afters ::= CHOICE { cheese IA5String, + dessert IA5String } +END +</pre> + + <p>Tags will not be mentioned any more in this manual.</p> + </section> + + <section> <marker id="ASN1Types"></marker> <title>The ASN.1 Types</title> <p>This section describes the ASN.1 types including their @@ -497,7 +496,7 @@ Operational ::= BOOLEAN --ASN.1 definition </pre> <p>In Erlang code it may look like:</p> <pre> Val = true, -{ok,Bytes}=asn1rt:encode(MyModule,'Operational',Val), </pre> +{ok,Bytes} = MyModule:encode('Operational', Val), </pre> <p>Below follows a description of how values of each type can be represented in Erlang. </p> @@ -563,20 +562,18 @@ T6value3 = white <section> <marker id="REAL"></marker> <title>REAL</title> - <p>In this version reals are not implemented. When they are, - the following - ASN.1 type is used:</p> + <p>The following ASN.1 type is used for real numbers:</p> <pre> R1 ::= REAL </pre> - <p>Can be assigned a value in Erlang as:</p> + <p>It can be assigned a value in Erlang as:</p> <pre> -R1value1 = 2.14, +R1value1 = "2.14", R1value2 = {256,10,-2}, </pre> <p>In the last line note that the tuple {256,10,-2} is the real number 2.56 in a special notation, which will encode faster than simply - stating the number as 2.56. The arity three tuple is + stating the number as <c>"2.56"</c>. The arity three tuple is <c>{Mantissa,Base,Exponent}</c> i.e. Mantissa * Base^Exponent.</p> </section> @@ -736,13 +733,11 @@ O2Val = <<"must be exactly 28 chars....">>,</pre> specified for a type are especially important for PER, where they affect the encoding. </p> - <p>Please note that <em>all</em> the Character strings are - supported and it is possible to use the following ASN.1 type - definitions:</p> + <p>Here are some examples:</p> <pre> Digs ::= NumericString (SIZE(1..3)) TextFile ::= IA5String (SIZE(0..64000)) </pre> - <p>and the following Erlang assignments:</p> + <p>with corresponding Erlang assignments:</p> <pre> DigsVal1 = "456", DigsVal2 = "123", @@ -755,70 +750,86 @@ TextFileVal2 = [88,76,55,44,99,121 .......... a lot of characters here ....] characters are all represented by quadruples beginning with three zeros like {0,0,0,65} for the 'A' character. When decoding a value for these strings the result is a list of - quadruples, or integers when the value is an ASCII character. - The following example shows how it works:</p> - <p>In a file <c>PrimStrings.asn1</c> the type <c>BMP</c> is defined as - <br></br> -<c>BMP ::= BMPString</c> then using BER encoding (<c>ber</c> - option)the input/output format will be:</p> + quadruples, or integers when the value is an ASCII character.</p> + + <p>The following example shows how it works. We have the following + specification in the file <c>PrimStrings.asn1</c>.</p> + <pre> +PrimStrings DEFINITIONS AUTOMATIC TAGS ::= +BEGIN + BMP ::= BMPString +END + </pre> + + <p>Encoding and decoding some strings:</p> + <pre> -1> <input>{ok,Bytes1} = asn1rt:encode('PrimStrings','BMP',[{0,0,53,53},{0,0,45,56}]).</input> -{ok,[30,4,"55-8"]} -2> <input>asn1rt:decode('PrimStrings','BMP',list_to_binary(Bytes1)).</input> +1> <input>asn1ct:compile('PrimStrings', [ber]).</input> +ok +2> <input>{ok,Bytes1} = 'PrimStrings':encode('BMP', [{0,0,53,53},{0,0,45,56}]).</input> +{ok,<<30,4,53,54,45,56>>} +3> <input>'PrimStrings':decode('BMP', Bytes1).</input> {ok,[{0,0,53,53},{0,0,45,56}]} -3> <input>{ok,Bytes2} = asn1rt:encode('PrimStrings','BMP',[{0,0,53,53},{0,0,0,65}]).</input> -{ok,[30,4,[53,53,0,65]]} -4> <input>asn1rt:decode('PrimStrings','BMP',list_to_binary(Bytes2)).</input> +4> <input>{ok,Bytes2} = 'PrimStrings':encode('BMP', [{0,0,53,53},{0,0,0,65}]).</input> +{ok,<<30,4,53,53,0,65>>} +5> <input>'PrimStrings':decode('BMP', Bytes2).</input> {ok,[{0,0,53,53},65]} -5> <input>{ok,Bytes3} = asn1rt:encode('PrimStrings','BMP',"BMP string").</input> -{ok,[30,20,[0,66,0,77,0,80,0,32,0,115,0,116,0,114,0,105,0,110,0,103]]} -6> <input>asn1rt:decode('PrimStrings','BMP',list_to_binary(Bytes3)).</input> +6> <input>{ok,Bytes3} = 'PrimStrings':encode('BMP', "BMP string").</input> +{ok,<<30,20,0,66,0,77,0,80,0,32,0,115,0,116,0,114,0,105,0,110,0,103>>} +7> <input>'PrimStrings':decode('BMP', Bytes3).</input> {ok,"BMP string"} </pre> - <p>The UTF8String is represented in Erlang as a list of integers, - where each integer represents the unicode value of one - character. When a value shall be encoded one first has to - transform it to a UTF8 encoded binary, then it can be encoded by - asn1. When decoding the result is a UTF8 encoded binary, which - may be transformed to an integer list. The transformation - functions, <c>utf8_binary_to_list</c> and - <c>utf8_list_to_binary</c>, are in the <c>asn1rt</c> module. In - the example below we assume an asn1 definition <c>UTF ::= UTF8String</c> in a module <c>UTF.asn</c>:</p> + + <p>The UTF8String type is represented as a UTF-8 encoded binary in + Erlang. Such binaries can be created directly using the binary syntax + or by converting from a list of Unicode code points using the + <c>unicode:characters_to_binary/1</c> function.</p> + + <p>Here are some examples showing how UTF-8 encoded binaries can + be created and manipulated:</p> + + <pre> +1> <input>Gs = "Мой маленький Гном".</input> +[1052,1086,1081,32,1084,1072,1083,1077,1085,1100,1082,1080, + 1081,32,1043,1085,1086,1084] +2> <input>Gbin = unicode:characters_to_binary(Gs).</input> +<<208,156,208,190,208,185,32,208,188,208,176,208,187,208, + 181,208,189,209,140,208,186,208,184,208,185,32,208,147, + 208,...>> +3> <input>Gbin = <<"Мой маленький Гном"/utf8>>.</input> +<<208,156,208,190,208,185,32,208,188,208,176,208,187,208, + 181,208,189,209,140,208,186,208,184,208,185,32,208,147, + 208,...>> +4> <input>Gs = unicode:characters_to_list(Gbin).</input> +[1052,1086,1081,32,1084,1072,1083,1077,1085,1100,1082,1080, + 1081,32,1043,1085,1086,1084] + </pre> + + <p>See the <seealso marker="stdlib:unicode">unicode</seealso> module + for more details.</p> + + <p>In the following example we will use this ASN.1 specification:</p> <pre> -1> <input>asn1ct:compile('UTF',[ber]).</input> -Erlang ASN.1 version "1.4.3.3" compiling "UTF.asn" -Compiler Options: [ber] ---{generated,"UTF.asn1db"}-- ---{generated,"UTF.erl"}-- +UTF DEFINITIONS AUTOMATIC TAGS ::= +BEGIN + UTF ::= UTF8String +END + </pre> + + <p>Encoding and decoding a string with Unicode characters:</p> + + <pre> +5> <input>asn1ct:compile('UTF', [ber]).</input> +ok +6> <input>{ok,Bytes1} = 'UTF':encode('UTF', <<"Гном"/utf8>>).</input> +{ok,<<12,8,208,147,208,189,208,190,208,188>>} +7> <input>{ok,Bin1} = 'UTF':decode('UTF', Bytes1).</input> +{ok,<<208,147,208,189,208,190,208,188>>} +8> <input>io:format("~ts\n", [Bin1]).</input> +Гном ok -2> <input>UTF8Val1 = "hello".</input> -"hello" -3> <input>{ok,UTF8bin1} = asn1rt:utf8_list_to_binary(UTF8Val1).</input> -{ok,<<104,101,108,108,111>>} -4> <input>{ok,B}='UTF':encode('UTF',UTF8bin1).</input> -{ok,[12, - 5, - <<104,101,108,108,111>>]} -5> <input>Bin = list_to_binary(B).</input> -<<12,5,104,101,108,108,111>> -6> <input>{ok,UTF8bin1}='UTF':decode('UTF',Bin).</input> -{ok,<<104,101,108,108,111>>} -7> <input>asn1rt:utf8_binary_to_list(UTF8bin1).</input> -{ok,"hello"} -8> <input>UTF8Val2 = [16#00,16#100,16#ffff,16#ffffff].</input> -[0,256,65535,16777215] -9> <input>{ok,UTF8bin2} = asn1rt:utf8_list_to_binary(UTF8Val2).</input> -{ok,<<0,196,128,239,191,191,248,191,191,191,191>>} -10> <input>{ok,B2} = 'UTF':encode('UTF',UTF8bin2).</input> -{ok,[12, - 11, - <<0,196,128,239,191,191,248,191,191,191,191>>]} -11> <input>Bin2 = list_to_binary(B2).</input> -<<12,11,0,196,128,239,191,191,248,191,191,191,191>> -12> <input>{ok,UTF8bin2} = 'UTF':decode('UTF',Bin2).</input> -{ok,<<0,196,128,239,191,191,248,191,191,191,191>>} -13> <input>asn1rt:utf8_binary_to_list(UTF8bin2).</input> -{ok,[0,256,65535,16777215]} -14> </pre> +9> <input>unicode:characters_to_list(Bin1).</input> +[1043,1085,1086,1084] + </pre> </section> <section> @@ -853,9 +864,11 @@ OidVal1 = {1,2,55}, <section> <marker id="Object Descriptor"></marker> <title>Object Descriptor</title> - <p>Values of this type can be assigned a value as an ordinary string i.e. <br></br> + <p>Values of this type can be assigned a value as an ordinary string + like this:</p> - "This is the value of an Object descriptor"</p> + <pre> + "This is the value of an Object descriptor"</pre> </section> <section> @@ -898,19 +911,31 @@ Pdu ::= SEQUENCE { <pre> MyPdu = #'Pdu'{a=22,b=77.99,c={0,1,2,3,4},d='NULL'}. </pre> <p>The decode functions will return a record as result when decoding - a <c>SEQUENCE</c> or a <c>SET</c>. - <marker id="DEFAULT"></marker> -</p> - <p>A <c>SEQUENCE</c> and a <c>SET</c> may contain a component with a - <c>DEFAULT</c> key word followed by the actual value that is the - default value. In case of BER encoding it is optional to encode the - value if it equals the default value. If the application uses the - atom asn1_DEFAULT as value or if the value is a primitive value - that equals the default value the encoding omits the bytes for - this value, which is more efficient and it results in fever - bytes to send to the receiving application.</p> - <p>For instance, if the following types exists in a file "File.asn":</p> + a <c>SEQUENCE</c> or a <c>SET</c>.</p> + + <p>A <c>SEQUENCE</c> and a <c>SET</c> may contain a component + with a <c>DEFAULT</c> key word followed by the actual value that + is the default value. The <c>DEFAULT</c> keyword means that the + application doing the encoding can omit encoding of the value, + thus resulting in fewer bytes to send to the receiving + application.</p> + + <p>An application can use the atom <c>asn1_DEFAULT</c> to indicate + that the encoding should be omitted for that position in + the SEQUENCE.</p> + + <p>Depending on the encoding rules, the encoder may also compare + the given value to the default value and automatically omit the + encoding if they are equal. How much effort the encoder makes to + to compare the values depends on the encoding rules. The DER + encoding rules forbids encoding a value equal to the default value, + so it has a more thorough and time-consuming comparison than the + encoders for the other encoding rules.</p> + + <p>In the following example we will use this ASN.1 specification:</p> <pre> +File DEFINITIONS AUTOMATIC TAGS ::= +BEGIN Seq1 ::= SEQUENCE { a INTEGER DEFAULT 1, b Seq2 DEFAULT {aa TRUE, bb 15} @@ -920,131 +945,50 @@ Seq2 ::= SEQUENCE { aa BOOLEAN, bb INTEGER } - </pre> - <p>Some values and the corresponding encoding in an Erlang terminal - is shown below:</p> + +Seq3 ::= SEQUENCE { + bs BIT STRING {a(0), b(1), c(2)} DEFAULT {a, c} +} +END </pre> + <p>Here is an example where the BER encoder is able to omit encoding + of the default values:</p> <pre> -1> <input>asn1ct:compile('File').</input> -Erlang ASN.1 version "1.3.2" compiling "File.asn1" -Compiler Options: [] ---{generated,"File.asn1db"}-- ---{generated,"File.hrl"}-- ---{generated,"File.erl"}-- +1> <input>asn1ct:compile('File', [ber]).</input> ok -2> <input>'File':encode('Seq1',{'Seq1',asn1_DEFAULT,asn1_DEFAULT}).</input> -{ok,["0",[0],[[],[]]]} -3> <input>lists:flatten(["0",[0],[[],[]]]).</input> -[48,0] -4> <input>'File':encode('Seq1',{'Seq1',1,{'Seq2',true,15}}).</input> -{ok,["0","\\b",[[],["\\241",[6],[[[128],[1],"\\377"],[[129],[1],[15]]]]]]} -5> <input>lists:flatten(["0","\\b",[[],["\\241",[6],[[[128],[1],"\\377"],[[129],[1],[15]]]]]]).</input> -[48,8,161,6,128,1,255,129,1,15] -6> </pre> - <p>The result after command line 3, in the example above,shows that the - encoder omits the encoding of default values when they are specific - by asn1_DEFAULT. Line 5 shows that even primitive values that equals - the default value are detected and not encoded. But the constructed - value of component <c>b</c> in <c>Seq1</c> is not recognized as the - default value. Checking of default values in <c>BER</c> is not done - in case of complex values, because it would be to expensive. - <marker id="DEFAULT DER"></marker> -</p> - <p>But, the DER encoding format has stronger requirements regarding - default values both for SET and SEQUENCE. A more elaborate and time - expensive check of default values will take place. The following is - an example with the same types and values as above but with der - encoding format.</p> - <pre> -1> <input>asn1ct:compile('File',[der]).</input> -Erlang ASN.1 version "1.3.2" compiling "File.asn1" -Compiler Options: [der] ---{generated,"File.asn1db"}-- ---{generated,"File.hrl"}-- ---{generated,"File.erl"}-- +2> <input>'File':encode('Seq1', {'Seq1',asn1_DEFAULT,asn1_DEFAULT}).</input> +{ok,<<48,0>>} +3> <input>'File':encode('Seq1', {'Seq1',1,{'Seq2',true,15}}).</input> +{ok,<<48,0>>} </pre> + + <p>And here is an example with a named BIT STRING where the BER + encoder will not omit the encoding:</p> + <pre> +4> <input>'File':encode('Seq3', {'Seq3',asn1_DEFAULT).</input> +{ok,<<48,0>>} +5> <input>'File':encode('Seq3', {'Seq3',<<16#101:3>>).</input> +{ok,<<48,4,128,2,5,160>>} </pre> + + <p>The DER encoder will omit the encoding for the same BIT STRING:</p> + <pre> +6> <input>asn1ct:compile('File', [ber,der]).</input> ok -2> <input>'File':encode('Seq1',{'Seq1',asn1_DEFAULT,asn1_DEFAULT}).</input> -{ok,["0",[0],[[],[]]]} -3> <input>lists:flatten(["0",[0],[[],[]]]).</input> -[48,0] -4> <input>'File':encode('Seq1',{'Seq1',1,{'Seq2',true,15}}).</input> -{ok,["0",[0],[[],[]]]} -5> <input>lists:flatten(["0",[0],[[],[]]]).</input> -[48,0] -6> - </pre> - <p>Line 5 shows that even values of constructed types is checked and if - it equals the default value it will not be encoded.</p> +7> <input>'File':encode('Seq3', {'Seq3',asn1_DEFAULT).</input> +{ok,<<48,0>>} +8> <input>'File':encode('Seq3', {'Seq3',<<16#101:3>>).</input> +{ok,<<48,0>>} </pre> </section> <section> <marker id="SET"></marker> <title>SET</title> - <p>The SET type is an unusual construct and normally the SEQUENCE - type is more appropriate to use. Set is also inefficient compared with SEQUENCE, as the components can be in any order. Hence, it must be possible - to distinguish every component in 'SET', both when - encoding and decoding a value of a type defined to be a SET. - The tags of all components must be different from each other - in order to be easily recognizable.</p> - <p>A SET may be defined as:</p> - <pre> -Pdu2 ::= SET { - a INTEGER, - b BOOLEAN, - c ENUMERATED {on(0),off(1)} } </pre> - <p>A SET is represented as an Erlang record. - For each SEQUENCE and <c>SET</c> in - an ASN.1 module an Erlang record declaration is generated. For - <c>Pdu2</c> above a record is defined like this:</p> - <pre> --record('Pdu2',{a, b, c}). </pre> - <p>The record declarations for a module <c>M</c> are placed in a - separate <c>M.hrl</c> file.</p> - <p>Values can be assigned in Erlang as demonstrated below:</p> - <pre> -V = #'Pdu2'{a=44,b=false,c=off}. </pre> - <p>The decode functions will return a record as result when decoding - a SET. - </p> - <p>The difference between SET and SEQUENCE is that the order of - the components (in the BER encoded format) is undefined for SET - and defined as the lexical order from the ASN.1 definition for - SEQUENCE. The ASN.1 compiler for Erlang will always encode a - SET in the lexical order. The decode routines can handle SET - components encoded in any order but will always return the - result as a record. Since all components of the SET must be - distinguishable both in the encoding phase as well as the - decoding phase the following type is not allowed in a module - with EXPLICIT or IMPLICIT as tag-default :</p> - <p></p> - <pre> -Bad ::= SET {i INTEGER, - j INTEGER } </pre> - <p>The ASN.1 to Erlang compiler rejects the above type. We - shall not explain the concept of tag further here, we refer to - [<cite id="X.680"></cite>]. - </p> - <p>Encoding of a SET with components with DEFAULT values behaves - similar as a SEQUENCE, <seealso marker="#DEFAULT">see above</seealso>. The DER encoding format restrictions on DEFAULT - values is the same for SET as for SEQUENCE, and is supported by - the compiler, <seealso marker="#DEFAULT DER">see above</seealso>.</p> - <p>Moreover, in DER the elements of a SET will be sorted. If a - component is an un-tagged choice the sorting have to take place - in run-time. This fact emphasizes the following recommendation - if DER encoding format is used.</p> - <p>The concept of SET is an unusual - construct and one cannot think of one single application - where the set type is essential. (Imagine if someone - "invented'' the shuffled array in 'C') People tend to think - that 'SET' sounds nicer and more mathematical than 'SEQUENCE' - and hence use it when 'SEQUENCE' would have been more - appropriate. It is also most inefficient, since every correct - implementation of SET must always be prepared to accept the - components in any order. So, if possible use SEQUENCE instead - of SET.</p> + <p>In Erlang, the SET type is used exactly as SEQUENCE. Note + that if the BER or DER encoding rules are used, decoding a + SET is slower than decoding a SEQUENCE because the components + must be sorted.</p> </section> <section> - <title>Notes about Extend-ability for SEQUENCE and SET</title> + <title>Notes about extensibility for SEQUENCE and SET</title> <p>When a SEQUENCE or SET contains an extension marker and extension components like this:</p> <pre> @@ -1071,51 +1015,28 @@ SExt ::= SEQUENCE { <marker id="CHOICE"></marker> <title>CHOICE</title> <p>The CHOICE type is a space saver and is similar to the concept of a - 'union' in the C-language. As with the previous SET-type, the - tags of all components of a CHOICE need to be distinct. If - AUTOMATIC TAGS are defined for the module (which is - preferable) the tags can be omitted completely in the ASN.1 - specification of a CHOICE. - </p> + 'union' in the C language.</p> <p>Assume:</p> <pre> +SomeModuleName DEFINITIONS AUTOMATIC TAGS ::= +BEGIN T ::= CHOICE { - x [0] REAL, - y [1] INTEGER, - z [2] OBJECT IDENTIFIER } - </pre> + x REAL, + y INTEGER, + z OBJECT IDENTIFIER } +END </pre> <p>It is then possible to assign values:</p> <pre> TVal1 = {y,17}, TVal2 = {z,{0,1,2}}, </pre> - <p>A CHOICE value is always represented as the tuple + <p>A CHOICE value is always represented as the tuple <c>{ChoiceAlternative, Val}</c> where <c>ChoiceAlternative</c> - is an atom denoting the selected choice - alternative. - </p> - <p>It is also allowed to have a CHOICE type tagged as follow:</p> - <p></p> - <pre> -C ::= [PRIVATE 111] CHOICE { - C1, - C2 } - -C1 ::= CHOICE { - a [0] INTEGER, - b [1] BOOLEAN } - -C2 ::= CHOICE { - c [2] INTEGER, - d [3] OCTET STRING } </pre> - <p>In this case, the top type C appears to have no tags at all in - its components, however, both C1 and C2 are also defined as - CHOICE types and they have distinct tags among themselves. - Hence, the above type C is both legal and allowed. + is an atom denoting the selected choice alternative. </p> <section> - <title>Extendable CHOICE</title> + <title>Extensible CHOICE</title> <p>When a CHOICE contains an extension marker and the decoder detects an unknown alternative of the CHOICE the value is represented as:</p> <pre> @@ -1192,26 +1113,29 @@ Arr2Val = ["abc",[14,34,54],"Octets"], </pre> Where <c>Value</c> may be a value of yet another type T2.</p> <p>For example:</p> <pre> +EmbeddedExample DEFINITIONS AUTOMATIC TAGS ::= +BEGIN B ::= SEQUENCE { a Arr1, - b [0] T } + b T } Arr1 ::= SET SIZE (5) OF INTEGER (4..9) T ::= CHOICE { - x [0] REAL, - y [1] INTEGER, - z [2] OBJECT IDENTIFIER } </pre> - <p>The above example can be assigned like this in Erlang:</p> + x REAL, + y INTEGER, + z OBJECT IDENTIFIER } + END </pre> + <p>The SEQUENCE b can be encoded like this in Erlang:</p> <pre> -V2 = #'B'{a=[4,5,6,7,8], b={x,7.77}}. - </pre> +1> 'EmbeddedExample':encode('B', {'B',[4,5,6,7,8],{x,"7.77"}}). +{ok,<<5,56,0,8,3,55,55,55,46,69,45,50>>} </pre> </section> </section> <section> <title>Naming of Records in .hrl Files</title> - <p>When an asn1 specification is compiled all defined types of + <p>When an ASN.1 specification is compiled all defined types of type SET or SEQUENCE will result in a corresponding record in the generated hrl file. This is because the values for SET/SEQUENCE as mentioned in sections above are represented as records.</p> @@ -1227,8 +1151,8 @@ V2 = #'B'{a=[4,5,6,7,8], b={x,7.77}}. Emb ::= SEQUENCE { a SEQUENCE OF OCTET STRING, b SET { - a [0] INTEGER, - b [1] INTEGER DEFAULT 66}, + a INTEGER, + b INTEGER DEFAULT 66}, c CHOICE { a INTEGER, b FooType } } @@ -1299,7 +1223,7 @@ PType{T} ::= SEQUENCE{ <p>Types may refer to themselves. Suppose:</p> <pre> Rec ::= CHOICE { - nothing [0] NULL, + nothing NULL, something SEQUENCE { a INTEGER, b OCTET STRING, @@ -1331,7 +1255,7 @@ tt TT ::= {a 77,b {"kalle","kula"}} </pre> Firstly, it could be used as the value in some DEFAULT component:</p> <pre> SS ::= SET { - s [0] OBJECT IDENTIFIER, + s OBJECT IDENTIFIER, val TT DEFAULT tt } </pre> <p>It could also be used from inside an Erlang program. If the above ASN.1 code was defined in ASN.1 module <c>Values</c>, then the ASN.1 value @@ -1365,8 +1289,8 @@ SS ::= SET { <marker id="Information Object"></marker> <title>ASN.1 Information Objects (X.681)</title> <p>Information Object Classes, Information Objects and Information - Object Sets, (in the following called classes, objects and - object sets respectively), are defined in the standard + Object Sets (in the following called classes, objects and + object sets respectively) are defined in the standard definition [<cite id="X.681"></cite>]. In the following only a brief explanation is given. </p> <p>These constructs makes it possible to define open types, @@ -1435,9 +1359,26 @@ StartMessage ::= SEQUENCE { <p><c>StartMessage</c> can in the <c>content</c> field be encoded with a value of any type that an object in the <c>GENERAL-PROCEDURES</c> object set has in its <c>NEW MESSAGE</c> field. This field refers to a type field - <c><![CDATA[&Message]]></c> in the class. The <c>msgId</c> field is always + <c>&Message</c> in the class. The <c>msgId</c> field is always encoded as a PrintableString, since the field refers to a fixed type in the class.</p> + <p>In practice, object sets are usually declared to be extensible so + so that more objects can be added to the set later. Extensibility is + indicated like this:</p> + <pre> +GENERAL-PROCEDURES GENERAL-PROCEDURE ::= { + object1 | object2, ...} </pre> + <p>When decoding a type that uses an extensible set constraint, + there is always the possibility that the value in the UNIQUE + field is unknown (i.e. the type has been encoded with a later + version of the ASN.1 specification). When that happens, the + unencoded data will be returned wrapped in a tuple like this:</p> + + <pre> +{asn1_OPENTYPE,Binary}</pre> + <p>where <c>Binary</c> is an Erlang binary that contains the encoded + data. (If the option <c>legacy_erlang_types</c> has been given, + just the binary will be returned.)</p> </section> <section> @@ -1449,7 +1390,7 @@ StartMessage ::= SEQUENCE { instance, if a Type is used in a definition with certain purpose, one want the type-name to express the intention. This can be done with parameterization.</p> - <p>When many types (or an other ASN.1 entity) only differs in some + <p>When many types (or another ASN.1 entity) only differs in some minor cases, but the structure of the types are similar, only one general type can be defined and the differences may be supplied through parameters. </p> @@ -1466,132 +1407,11 @@ T1 ::= General{PrintableString} T2 ::= General{BIT STRING} </pre> <p>An example of a value that can be encoded as type T1 is {12,"hello"}.</p> - <p>Observe that the compiler not generates encode/decode functions for - parameterized types, only for the instances of the parameterized - types. So, if a file contains the types General{}, T1 and T2 above, + <p>Note that the compiler does not generate encode/decode functions for + parameterized types, but only for the instances of the parameterized + types. Therefore, if a file contains the types General{}, T1 and T2 above, encode/decode functions will only be generated for T1 and T2. </p> </section> - - <section> - <title>Tags</title> - <p>Every built-in ASN.1 type, except CHOICE and ANY have a universal tag. - This is a unique number that clearly identifies the type. <br></br> - - It is essential for all users of ASN.1 to - understand all the details about tags.</p> - <p>Tags are implicitly encoded in the BER encoding as shown below, but - are hardly not accounted for in the PER encoding. In PER tags are - used for instance to sort the components of a SET.</p> - <p>There are four different types of tags.</p> - <taglist> - <tag><em>universal</em></tag> - <item> - <p>For types whose meaning is the same in all - applications. Such as integers, sequences and so on; that is, all the built in - types.</p> - </item> - <tag><em>application</em></tag> - <item> - <p>For application specific types for example, the types in - X.400 Message handling service have this sort of tag.</p> - </item> - <tag><em>private</em></tag> - <item> - <p>For your own private types.</p> - </item> - <tag><em>context</em></tag> - <item> - <p>This is used to distinguish otherwise indistinguishable - types in a specific context. For example, if we have two - components of a - CHOICE type that are both <c>INTEGER</c> values, there is no - way for the decoder to - decipher which component was actually chosen, since both - components will be - tagged as <c>INTEGER</c>. When this or similar situations occur, - one or both of the components should be given a context specific - to resolve the ambiguity.</p> - </item> - </taglist> - <p>The tag in the case of the 'Apdu' type [PRIVATE 1] is encoded to a - sequence of bytes making it possible for a - decoder to look at the (initial) bytes that arrive and determine - whether the rest of the bytes must be of the type associated - with that particular sequence of bytes. This means that each - tag must be uniquely associated with <em>only</em> one ASN.1 - type. - </p> - <p>Immediately following the tag is a sequence of bytes - informing the decoder of the length of the instance. This is - sometimes referred to as TLV (Tag length value) encoding. - Hence, the structure of a BER encoded series of bytes is as shown in the table below.</p> - <p></p> - <table> - <row> - <cell align="left" valign="middle">Tag</cell> - <cell align="left" valign="middle">Len</cell> - <cell align="left" valign="middle">Value</cell> - </row> - <tcaption>Structure of a BER encoded series of bytes</tcaption> - </table> - </section> - - <section> - <title>Encoding Rules</title> - <p>When the first recommendation on ASN.1 was released 1988 it was - accompanied with the Basic Encoding Rules, BER, as the only - alternative for encoding. - BER is a somewhat verbose protocol. It adopts a so-called TLV (type, - length, value) approach to encoding in which every element of the - encoding carries some type information, some length information and - then the value of that element. Where the element is itself - structured, then the Value part of the element is itself a series of - embedded TLV components, to whatever depth is necessary. In summary, - BER is not a compact encoding but is relatively fast and easy to - produce.</p> - <p>The DER (Distinguished Encoding Rule) encoding format was included in - the standard in 1994. It is a specialized form of BER, which gives - the encoder the option to encode some entities differently. For - instance, is the value for TRUE any octet with any bit set to one. But, - DER does not leave any such choices. The value for TRUE in the DER - case is encoded as the octet <c>11111111</c>. So, the same value - encoded by two different DER encoders must result in the same bit - stream.</p> - <p>A more compact encoding is achieved with the Packed Encoding - Rules PER which was introduced together with the revised - recommendation in 1994. PER takes a rather different approach from - that taken by BER. The first difference is that the tag part in - the TLV is omitted from the encodings, and any tags in the - notation are not encoded. The potential ambiguities are resolved - as follows:</p> - <list type="bulleted"> - <item> - <p>A CHOICE is encoded by first encoding a choice index which - identifies the chosen - alternative by its position in the notation.</p> - </item> - <item> - <p>The elements of a SEQUENCE are transmitted in textual - order. OPTIONAL or DEFAULT elements are preceded by a bit map - to identify which elements are present. After sorting the - elements of a SET in the "canonical tag order" as defined in - X.680 8.6 they are treated as a SEQUENCE regarding OPTIONAL - and DEFAULT elements. A SET is transferred in the sorted - order.</p> - </item> - </list> - <p>A second difference is that PER takes full account of the sub-typing - information in that the encoded bytes are affected by the constraints. - The BER encoded bytes are unaffected by the constraints. - PER uses the sub-typing information to for example omit length fields - whenever possible. </p> - <p>The run-time functions, sometimes take the constraints into account - both for BER and PER. For instance are SIZE constrained strings checked.</p> - <p>There are two variants of PER, <em>aligned</em> and <em>unaligned</em>. - In summary, PER results in compact encodings which require much more - computation to produce than BER. - </p> - </section> </chapter> diff --git a/lib/asn1/doc/src/asn1ct.xml b/lib/asn1/doc/src/asn1ct.xml index 4d5a1a402a..32ff2d52cf 100644 --- a/lib/asn1/doc/src/asn1ct.xml +++ b/lib/asn1/doc/src/asn1ct.xml @@ -45,10 +45,11 @@ <p>By default in OTP 17, the representation of the BIT STRING and OCTET STRING types as Erlang terms have changed. BIT STRING values are now Erlang bitstrings and OCTET STRING values - are binaries. For details see <seealso - marker="asn1_ug#BIT STRING">BIT STRING</seealso> and <seealso - marker="asn1_ug#OCTET STRING">OCTET STRING</seealso> in User's - Guide.</p> + are binaries. Also, an undecoded open type will now be wrapped in + a <c>asn1_OPENTYPE</c> tuple. For details see <seealso + marker="asn1_ug#BIT STRING">BIT STRING</seealso>, <seealso + marker="asn1_ug#OCTET STRING">OCTET STRING</seealso>, and + <seealso marker="asn1_ug#Information%20Object">ASN.1 Information Objects</seealso> in User's Guide.</p> <p>To revert to the old representation of the types, use the <c>legacy_erlang_types</c> option.</p> </note> diff --git a/lib/asn1/doc/src/notes.xml b/lib/asn1/doc/src/notes.xml index ff7962edd9..a7032737bd 100644 --- a/lib/asn1/doc/src/notes.xml +++ b/lib/asn1/doc/src/notes.xml @@ -31,6 +31,153 @@ <p>This document describes the changes made to the asn1 application.</p> +<section><title>Asn1 3.0.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Several problems where the ASN.1 compiler would crash + when attempting to compile correct specifications have + been corrected.</p> + <p> + Own Id: OTP-12125</p> + </item> + <item> + <p> + Robustness when decoding incorrect BER messages has been + improved.</p> + <p> + Own Id: OTP-12145</p> + </item> + </list> + </section> + +</section> + +<section><title>Asn1 3.0.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + The ASN.1 compiler now generates code that don't trigger + Dialyzer warnings. Along the way, a few minor bugs were + fixed.</p> + <p> + Own Id: OTP-11372 Aux Id: seq12397 </p> + </item> + </list> + </section> + +</section> + +<section><title>Asn1 3.0</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Subtyping an extensible ENUMERATED would cause an + compilation error. (Thanks to Morten Nygaard Åsnes for + reporting this bug.)</p> + <p> + Own Id: OTP-11700</p> + </item> + <item> + <p>When specifying the value for an OCTET STRING in a + specification, the ASN.1 standard clearly states that the + value must be either a bstring or an hstring, but NOT a + cstring. The ASN.1 compiler will now generate a + compilation error if the value of an OCTET STRING is + given as a character string.</p> + <p>That is, the following example is now illegal:</p> + <p><c>string OCTET STRING ::= "Now illegal"</c></p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-11727</p> + </item> + <item> + <p> + Application upgrade (appup) files are corrected for the + following applications: </p> + <p> + <c>asn1, common_test, compiler, crypto, debugger, + dialyzer, edoc, eldap, erl_docgen, et, eunit, gs, hipe, + inets, observer, odbc, os_mon, otp_mibs, parsetools, + percept, public_key, reltool, runtime_tools, ssh, + syntax_tools, test_server, tools, typer, webtool, wx, + xmerl</c></p> + <p> + A new test utility for testing appup files is added to + test_server. This is now used by most applications in + OTP.</p> + <p> + (Thanks to Tobias Schlager)</p> + <p> + Own Id: OTP-11744</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + By giving --enable-static-{nifs,drivers} to configure it + is now possible to statically linking of nifs and drivers + to the main Erlang VM binary. At the moment only the asn1 + and crypto nifs of the Erlang/OTP nifs and drivers have + been prepared to be statically linked. For more details + see the Installation Guide in the System documentation.</p> + <p> + Own Id: OTP-11258</p> + </item> + <item> + <p>Code generation for the <c>per</c> and <c>uper</c> + backends has been somewhat improved.</p> + <p> + Own Id: OTP-11573</p> + </item> + <item> + <p>The OCTET STRING and BIT STRING types now have a more + natural mapping to Erlang types (binary and bitstring, + respectively), which is more efficient and will avoid + useless conversions between lists and + binaries/bitstrings.</p> + <p>This is an incompatible change. To revert to the old + mapping to support existing applications, use the + <c>legacy_erlang_types</c> option.</p> + <p>Impact: There is a potential for better performance, + as it is now possible to avoid conversions between lists + and binaries both in the generated ASN.1 encode/decode + code and in the application itself.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-11594</p> + </item> + <item> + <p>All functions in the <c>asn1rt</c> module, as well as + <c>asn1ct:decode/3</c> and <c>asn1ct:encode/3</c>, are + now deprecated.</p> + <p> + Own Id: OTP-11731</p> + </item> + <item> + <p> + Generated .hrl files are now protected from being + included more than once.</p> + <p> + Own Id: OTP-11804</p> + </item> + </list> + </section> + +</section> + <section><title>Asn1 2.0.4</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/asn1/src/Makefile b/lib/asn1/src/Makefile index 500f4a1358..6798da0072 100644 --- a/lib/asn1/src/Makefile +++ b/lib/asn1/src/Makefile @@ -52,6 +52,7 @@ CT_MODULES= \ asn1ct_pretty_format \ asn1ct_func \ asn1ct_gen \ + asn1ct_gen_check \ asn1ct_gen_per \ asn1ct_name \ asn1ct_constructed_per \ diff --git a/lib/asn1/src/asn1_records.hrl b/lib/asn1/src/asn1_records.hrl index 396ba0fcfa..6c1cf1b12a 100644 --- a/lib/asn1/src/asn1_records.hrl +++ b/lib/asn1/src/asn1_records.hrl @@ -37,7 +37,7 @@ -record('ObjectClassFieldType',{classname,class,fieldname,type}). -record(typedef,{checked=false,pos,name,typespec}). --record(classdef,{checked=false,pos,name,typespec}). +-record(classdef, {checked=false,pos,name,module,typespec}). -record(valuedef,{checked=false,pos,name,type,value,module}). -record(ptypedef,{checked=false,pos,name,args,typespec}). -record(pvaluedef,{checked=false,pos,name,args,type,value}). @@ -45,7 +45,6 @@ -record(pobjectdef,{checked=false,pos,name,args,class,def}). -record(pobjectsetdef,{checked=false,pos,name,args,class,def}). --record(identifier,{pos,val}). -record('Constraint',{'SingleValue'=no,'SizeConstraint'=no,'ValueRange'=no,'PermittedAlphabet'=no, 'ContainedSubtype'=no, 'TypeConstraint'=no,'InnerSubtyping'=no,e=no,'Other'=no}). -record(simpletableattributes,{objectsetname,c_name,c_index,usedclassfield, @@ -73,6 +72,15 @@ % Externalvaluereference -> modulename '.' typename -record('Externalvaluereference',{pos,module,value}). +%% Used to hold a tag for a field in a SEQUENCE/SET. It can also +%% be used for identifiers in OBJECT IDENTIFIER values, since the +%% parser cannot always distinguish a SEQUENCE with one element from +%% an OBJECT IDENTIFIER. +-record(seqtag, + {pos :: integer(), + module :: atom(), + val :: atom()}). + -record(state,{module,mname,type,tname,value,vname,erule,parameters=[], inputmodules,abscomppath=[],recordtopname=[],options, sourcedir}). diff --git a/lib/asn1/src/asn1ct.erl b/lib/asn1/src/asn1ct.erl index 8470e5a1b4..df341e5aab 100644 --- a/lib/asn1/src/asn1ct.erl +++ b/lib/asn1/src/asn1ct.erl @@ -43,7 +43,7 @@ add_tobe_refed_func/1,add_generated_refed_func/1, maybe_rename_function/3,current_sindex/0, set_current_sindex/1,maybe_saved_sindex/2, - parse_and_save/2,verbose/3,warning/3,warning/4,error/3]). + parse_and_save/2,verbose/3,warning/3,warning/4,error/3,format_error/1]). -export([get_bit_string_format/0,use_legacy_types/0]). -include("asn1_records.hrl"). @@ -143,7 +143,8 @@ parse_and_save_passes() -> {pass,save,fun save_pass/1}]. common_passes() -> - [{pass,check,fun check_pass/1}, + [{iff,parse,{pass,parse_listing,fun parse_listing/1}}, + {pass,check,fun check_pass/1}, {iff,abs,{pass,abs_listing,fun abs_listing/1}}, {pass,generate,fun generate_pass/1}, {unless,noobj,{pass,compile,fun compile_pass/1}}]. @@ -243,6 +244,16 @@ save_pass(#st{code=M,erule=Erule,dbfile=DbFile}=St) -> asn1_db:dbsave(DbFile,M#module.name), {ok,St}. +parse_listing(#st{code=Code,outfile=OutFile0}=St) -> + OutFile = OutFile0 ++ ".parse", + case file:write_file(OutFile, io_lib:format("~p\n", [Code])) of + ok -> + done; + {error,Reason} -> + Error = {write_error,OutFile,Reason}, + {error,St#st{error=[{structured_error,{OutFile0,none},?MODULE,Error}]}} + end. + abs_listing(#st{code={M,_},outfile=OutFile}) -> pretty2(M#module.name, OutFile++".abs"), done. @@ -2430,6 +2441,10 @@ verbose(Format, Args, S) -> ok end. +format_error({write_error,File,Reason}) -> + io_lib:format("writing output file ~s failed: ~s", + [File,file:format_error(Reason)]). + is_error(S) when is_record(S, state) -> is_error(S#state.options); is_error(O) -> diff --git a/lib/asn1/src/asn1ct_check.erl b/lib/asn1/src/asn1ct_check.erl index e788aa5c6c..5d8740b92e 100644 --- a/lib/asn1/src/asn1ct_check.erl +++ b/lib/asn1/src/asn1ct_check.erl @@ -91,7 +91,7 @@ check(S,{Types,Values,ParameterizedTypes,Classes,Objects,ObjectSets}) -> save_asn1db_uptodate(S,S#state.erule,S#state.mname), put(top_module,S#state.mname), - _ = checkp(S, ParameterizedTypes), %must do this before the templates are used + ParamError = checkp(S, ParameterizedTypes), %must do this before the templates are used %% table to save instances of parameterized objects,object sets asn1ct_table:new(parameterized_objects), @@ -160,8 +160,10 @@ check(S,{Types,Values,ParameterizedTypes,Classes,Objects,ObjectSets}) -> Exporterror = check_exports(S,S#state.module), ImportError = check_imports(S,S#state.module), - case {Terror3,Verror5,Cerror,Oerror,Exporterror,ImportError} of - {[],[],[],[],[],[]} -> + AllErrors = lists:flatten([ParamError,Terror3,Verror5,Cerror, + Oerror,Exporterror,ImportError]), + case AllErrors of + [] -> ContextSwitchTs = context_switch_in_spec(), InstanceOf = instance_of_in_spec(S#state.mname), NewTypes = lists:subtract(Types,AddClasses) ++ ContextSwitchTs @@ -175,8 +177,7 @@ check(S,{Types,Values,ParameterizedTypes,Classes,Objects,ObjectSets}) -> lists:subtract(NewObjects,ExclO)++InlinedObjects, lists:subtract(NewObjectSets,ExclOS)++ParObjectSetNames}}; _ -> - {error,lists:flatten([Terror3,Verror5,Cerror, - Oerror,Exporterror,ImportError])} + {error,AllErrors} end. context_switch_in_spec() -> @@ -549,14 +550,10 @@ check_class(S = #state{mname=M,tname=T},ClassSpec) #objectclass{fields=Def}; % in case of recursive definitions Tref = #'Externaltypereference'{type=TName} -> {MName,RefType} = get_referenced_type(S,Tref), - case is_class(S,RefType) of - true -> - NewState = update_state(S#state{type=RefType, - tname=TName},MName), - check_class(NewState,get_class_def(S,RefType)); - _ -> - error({class,{internal_error,RefType},S}) - end; + #classdef{} = CD = get_class_def(S, RefType), + NewState = update_state(S#state{type=RefType, + tname=TName}, MName), + check_class(NewState, CD); {pt,ClassRef,Params} -> %% parameterized class {_,PClassDef} = get_referenced_type(S,ClassRef), @@ -950,6 +947,8 @@ prepare_objset(ObjDef={object,definedsyntax,_ObjFields}) -> {set,[ObjDef],false}; prepare_objset({ObjDef=#type{},Ext}) when is_list(Ext) -> {set,[ObjDef|Ext],true}; +prepare_objset({#type{}=Type,#type{}=Ext}) -> + {set,[Type,Ext],true}; prepare_objset(Ret) -> Ret. @@ -1277,10 +1276,25 @@ get_fieldname_element(_S,Def,[{_RefType,_FieldName}|_RestFName]) check_fieldname_element(S,{value,{_,Def}}) -> check_fieldname_element(S,Def); -check_fieldname_element(S,TDef) when is_record(TDef,typedef) -> - check_type(S,TDef,TDef#typedef.typespec); -check_fieldname_element(S,VDef) when is_record(VDef,valuedef) -> - check_value(S,VDef); +check_fieldname_element(S, #typedef{typespec=Ts}=TDef) -> + case Ts of + #'Object'{} -> + check_object(S, TDef, Ts); + _ -> + check_type(S, TDef, Ts) + end; +check_fieldname_element(S, #valuedef{}=VDef) -> + try + check_value(S, VDef) + catch + throw:{objectdef} -> + #valuedef{checked=C,pos=Pos,name=N,type=Type, + value=Def} = VDef, + ClassName = Type#type.def, + NewSpec = #'Object'{classname=ClassName,def=Def}, + NewDef = #typedef{checked=C,pos=Pos,name=N,typespec=NewSpec}, + check_fieldname_element(S, NewDef) + end; check_fieldname_element(S,Eref) when is_record(Eref,'Externaltypereference'); is_record(Eref,'Externalvaluereference') -> @@ -1803,12 +1817,10 @@ convert_to_defaultfield(S,ObjFieldName,[OFS|RestOFS],CField)-> FieldName); ValSetting = #valuedef{} -> ValSetting; - ValSetting = {'CHOICE',{Alt,_ChVal}} when is_atom(Alt) -> - #valuedef{type=element(3,CField), - value=ValSetting, - module=S#state.mname}; ValSetting -> - #identifier{val=ValSetting} + #valuedef{type=element(3,CField), + value=ValSetting, + module=S#state.mname} end, ?dbg("fixedtypevaluefield ValRef: ~p~n",[ValRef]), case ValRef of @@ -2292,22 +2304,23 @@ validate_oid(_, S, OID, [Id|Vrest], Acc) error({value, {"illegal "++to_string(OID),[Id,Vrest],Acc}, S}) end end; -validate_oid(_, S, OID, [{Atom,Value}],[]) +validate_oid(_, S, OID, [{#seqtag{module=Mod,val=Atom},Value}], []) when is_atom(Atom),is_integer(Value) -> %% this case when an OBJECT IDENTIFIER value has been parsed as a %% SEQUENCE value - Rec = #'Externalvaluereference'{module=S#state.mname, + Rec = #'Externalvaluereference'{module=Mod, value=Atom}, validate_objectidentifier1(S, OID, [Rec,Value]); -validate_oid(_, S, OID, [{Atom,EVRef}],[]) +validate_oid(_, S, OID, [{#seqtag{module=Mod,val=Atom},EVRef}], []) when is_atom(Atom),is_record(EVRef,'Externalvaluereference') -> %% this case when an OBJECT IDENTIFIER value has been parsed as a %% SEQUENCE value OTP-4354 - Rec = #'Externalvaluereference'{module=EVRef#'Externalvaluereference'.module, + Rec = #'Externalvaluereference'{module=Mod, value=Atom}, validate_objectidentifier1(S, OID, [Rec,EVRef]); -validate_oid(_, S, OID, [Atom|Rest],Acc) when is_atom(Atom) -> - Rec = #'Externalvaluereference'{module=S#state.mname, +validate_oid(_, S, OID, [#seqtag{module=Mod,val=Atom}|Rest], Acc) + when is_atom(Atom) -> + Rec = #'Externalvaluereference'{module=Mod, value=Atom}, validate_oid(true,S, OID, [Rec|Rest],Acc); validate_oid(_, S, OID, V, Acc) -> @@ -2689,20 +2702,20 @@ normalize_set(S,Value,Components,NameList) -> normalized_record('SET',S,SortedVal,Components,NameList) end. -sort_value(Components,Value) -> - ComponentNames = lists:map(fun(#'ComponentType'{name=Cname}) -> Cname end, - Components), - sort_value1(ComponentNames,Value,[]). -sort_value1(_,V=#'Externalvaluereference'{},_) -> - %% sort later, get the value in normalize_seq_or_set - V; -sort_value1([N|Ns],Value,Acc) -> - case lists:keysearch(N,1,Value) of - {value,V} ->sort_value1(Ns,Value,[V|Acc]); - _ -> sort_value1(Ns,Value,Acc) - end; -sort_value1([],_,Acc) -> - lists:reverse(Acc). +sort_value(Components, Value0) when is_list(Value0) -> + {Keys0,_} = lists:mapfoldl(fun(#'ComponentType'{name=N}, I) -> + {{N,I},I+1} + end, 0, Components), + Keys = gb_trees:from_orddict(orddict:from_list(Keys0)), + Value1 = [{case gb_trees:lookup(N, Keys) of + {value,K} -> K; + none -> 'end' + end,Pair} || {#seqtag{val=N},_}=Pair <- Value0], + Value = lists:sort(Value1), + [Pair || {_,Pair} <- Value]; +sort_value(_Components, #'Externalvaluereference'{}=Value) -> + %% Sort later. + Value. sort_val_if_set(['SET'|_],Val,Type) -> sort_value(Type,Val); @@ -2735,9 +2748,9 @@ is_record_normalized(_S,Name,Value,NumComps) when is_tuple(Value) -> is_record_normalized(_,_,_,_) -> false. -normalize_seq_or_set(SorS,S,[{Cname,V}|Vs], +normalize_seq_or_set(SorS, S, [{#seqtag{val=Cname},V}|Vs], [#'ComponentType'{name=Cname,typespec=TS}|Cs], - NameList,Acc) -> + NameList, Acc) -> NewNameList = case TS#type.def of #'Externaltypereference'{type=TName} -> @@ -2915,8 +2928,7 @@ get_canonic_type(S,Type,NameList) -> check_ptype(S,Type,Ts) when is_record(Ts,type) -> - %Tag = Ts#type.tag, - %Constr = Ts#type.constraint, + check_formal_parameters(S, Type#ptypedef.args), Def = Ts#type.def, NewDef= case Def of @@ -2942,6 +2954,16 @@ check_ptype(S,Type,Ts) when is_record(Ts,type) -> check_ptype(_S,_PTDef,Ts) when is_record(Ts,objectclass) -> throw({asn1_param_class,Ts}). +check_formal_parameters(S, Args) -> + _ = [check_formal_parameter(S, A) || A <- Args], + ok. + +check_formal_parameter(_, {_,_}) -> + ok; +check_formal_parameter(_, #'Externaltypereference'{}) -> + ok; +check_formal_parameter(S, #'Externalvaluereference'{value=Name}=Ref) -> + asn1_error(S, Ref, {illegal_typereference,Name}). % check_type(S,Type,ObjSpec={{objectclassname,_},_}) -> % check_class(S,ObjSpec); @@ -2989,9 +3011,9 @@ check_type(S=#state{recordtopname=TopName},Type,Ts) when is_record(Ts,type) -> {TmpRefMod,TmpRefDef} -> {TmpRefMod,TmpRefDef,false} end, - case is_class(S,RefTypeDef) of - true -> throw({asn1_class,RefTypeDef}); - _ -> ok + case get_class_def(S, RefTypeDef) of + none -> ok; + #classdef{} -> throw({asn1_class,RefTypeDef}) end, Ct = TestFun(Ext), {RefType,ExtRef} = @@ -3372,23 +3394,17 @@ get_type_from_object(S,Object,TypeField) ObjSpec = check_object(S,ObjectDef,ObjectDef#typedef.typespec), get_fieldname_element(S,ObjectDef#typedef{typespec=ObjSpec},TypeField). -is_class(_S,#classdef{}) -> - true; -is_class(S,#typedef{typespec=#type{def=Eref}}) - when is_record(Eref,'Externaltypereference')-> - is_class(S,Eref); -is_class(S,Eref) when is_record(Eref,'Externaltypereference')-> - {_,NextDef} = get_referenced_type(S,Eref), - is_class(S,NextDef); -is_class(_,_) -> - false. - -get_class_def(_S,CD=#classdef{}) -> +%% get_class_def(S, Type) -> #classdef{} | 'none'. +get_class_def(S, #typedef{typespec=#type{def=#'Externaltypereference'{}=Eref}}) -> + {_,NextDef} = get_referenced_type(S, Eref), + get_class_def(S, NextDef); +get_class_def(S, #'Externaltypereference'{}=Eref) -> + {_,NextDef} = get_referenced_type(S, Eref), + get_class_def(S, NextDef); +get_class_def(_S, #classdef{}=CD) -> CD; -get_class_def(S,#typedef{typespec=#type{def=Eref}}) - when is_record(Eref,'Externaltypereference') -> - {_,NextDef} = get_referenced_type(S,Eref), - get_class_def(S,NextDef). +get_class_def(_S, _) -> + none. maybe_illicit_implicit_tag(Kind,Tag) -> case Tag of @@ -3595,109 +3611,54 @@ match_args(_,_, _, _) -> %% categorize_arg(S,FormalArg,ActualArg) -> {FormalArg,CatgorizedActualArg} %% categorize_arg(S,{Governor,Param},ActArg) -> - case {governor_category(S,Governor),parameter_name_style(Param,ActArg)} of -%% {absent,beginning_uppercase} -> %% a type -%% categorize(S,type,ActArg); - {type,beginning_lowercase} -> %% a value - categorize(S,value,Governor,ActArg); - {type,beginning_uppercase} -> %% a value set - categorize(S,value_set,ActArg); -%% {absent,entirely_uppercase} -> %% a class -%% categorize(S,class,ActArg); + case {governor_category(S, Governor),parameter_name_style(Param)} of + {type,beginning_lowercase} -> %a value + categorize(S, value, Governor, ActArg); + {type,beginning_uppercase} -> %a value set + categorize(ActArg); {{class,ClassRef},beginning_lowercase} -> - categorize(S,object,ActArg,ClassRef); + categorize(S, object, ActArg, ClassRef); {{class,ClassRef},beginning_uppercase} -> - categorize(S,object_set,ActArg,ClassRef); - _ -> - [ActArg] + categorize(S, object_set, ActArg, ClassRef) end; -categorize_arg(S,FormalArg,ActualArg) -> - %% governor is absent => a type or a class - case FormalArg of - #'Externaltypereference'{type=Name} -> - case is_class_name(Name) of - true -> - categorize(S,class,ActualArg); - _ -> - categorize(S,type,ActualArg) - end; - FA -> - throw({error,{unexpected_formal_argument,FA}}) - end. - -governor_category(S,#type{def=Eref}) - when is_record(Eref,'Externaltypereference') -> - governor_category(S,Eref); -governor_category(_S,#type{}) -> +categorize_arg(_S, _FormalArg, ActualArg) -> + %% Governor is absent -- must be a type or a class. We have already + %% checked that the FormalArg begins with an uppercase letter. + categorize(ActualArg). + +%% governor_category(S, Item) -> type | {class,#'Externaltypereference'{}} +%% Determine whether Item is a type or a class. +governor_category(S, #type{def=#'Externaltypereference'{}=Eref}) -> + governor_category(S, Eref); +governor_category(_S, #type{}) -> type; -governor_category(S,Ref) when is_record(Ref,'Externaltypereference') -> - case is_class(S,Ref) of - true -> - {class,Ref}; - _ -> +governor_category(S, #'Externaltypereference'{}=Ref) -> + case get_class_def(S, Ref) of + #classdef{pos=Pos,module=Mod,name=Name} -> + {class,#'Externaltypereference'{pos=Pos,module=Mod,type=Name}}; + none -> type - end; -governor_category(_,Class) - when Class == 'TYPE-IDENTIFIER'; Class == 'ABSTRACT-SYNTAX' -> - class. -%% governor_category(_,_) -> -%% absent. + end. %% parameter_name_style(Param,Data) -> Result %% gets the Parameter and the name of the Data and if it exists tells %% whether it begins with a lowercase letter or is partly or entirely %% spelled with uppercase letters. Otherwise returns undefined %% -parameter_name_style(_,#'Externaltypereference'{type=Name}) -> - name_category(Name); -parameter_name_style(_,#'Externalvaluereference'{value=Name}) -> - name_category(Name); -parameter_name_style(_,{valueset,_}) -> - %% It is a object set or value set +parameter_name_style(#'Externaltypereference'{}) -> beginning_uppercase; -parameter_name_style(#'Externalvaluereference'{},_) -> - beginning_lowercase; -parameter_name_style(#'Externaltypereference'{type=Name},_) -> - name_category(Name); -parameter_name_style(_,_) -> - undefined. - -name_category(Atom) when is_atom(Atom) -> - name_category(atom_to_list(Atom)); -name_category([H|T]) -> - case is_lowercase(H) of - true -> - beginning_lowercase; - _ -> - case is_class_name(T) of - true -> - entirely_uppercase; - _ -> - beginning_uppercase - end - end; -name_category(_) -> - undefined. +parameter_name_style(#'Externalvaluereference'{}) -> + beginning_lowercase. is_lowercase(X) when X >= $A,X =< $W -> false; is_lowercase(_) -> true. - -is_class_name(Name) when is_atom(Name) -> - is_class_name(atom_to_list(Name)); -is_class_name(Name) -> - case [X||X <- Name, X >= $a,X =< $w] of - [] -> - true; - _ -> - false - end. -%% categorize(S,Category,Parameter) -> CategorizedParameter +%% categorize(Parameter) -> CategorizedParameter %% If Parameter has an abstract syntax of another category than %% Category, transform it to a known syntax. -categorize(_S,type,{object,_,Type}) -> +categorize({object,_,Type}) -> %% One example of this case is an object with a parameterized type %% having a locally defined type as parameter. Def = fun(D = #type{}) -> @@ -3709,11 +3670,12 @@ categorize(_S,type,{object,_,Type}) -> D end, [Def(X)||X<-Type]; -categorize(_S,type,Def) when is_record(Def,type) -> +categorize(#type{}=Def) -> [#typedef{name = new_reference_name("type_argument"), typespec = Def#type{inlined=yes}}]; -categorize(_,_,Def) -> +categorize(Def) -> [Def]. + categorize(S,object_set,Def,ClassRef) -> NewObjSetSpec = check_object(S,Def,#'ObjectSet'{class = ClassRef, @@ -4546,55 +4508,43 @@ check_reference(S,#'Externaltypereference'{pos=Pos,module=Emod,type=Name}) -> #'Externaltypereference'{pos=Pos,module=ModName,type=Name} end. +get_referenced_type(S, T) -> + case do_get_referenced_type(S, T) of + {_,#type{def=#'Externaltypereference'{}=ERef}} -> + get_referenced_type(S, ERef); + {_,#type{def=#'Externalvaluereference'{}=VRef}} -> + get_referenced_type(S, VRef); + {_,_}=Res -> + Res + end. -get_referenced_type(S,Ext) when is_record(Ext,'Externaltypereference') -> - case match_parameters(S,Ext, S#state.parameters) of - Ext -> - #'Externaltypereference'{pos=Pos,module=Emod,type=Etype} = Ext, - case S#state.mname of - Emod -> % a local reference in this module - get_referenced1(S,Emod,Etype,Pos); - _ ->% always when multi file compiling - case lists:member(Emod,S#state.inputmodules) of - true -> - get_referenced1(S,Emod,Etype,Pos); - false -> - get_referenced(S,Emod,Etype,Pos) - end - end; - ERef = #'Externaltypereference'{} -> - get_referenced_type(S,ERef); - Other -> - {undefined,Other} - end; -get_referenced_type(S=#state{mname=Emod}, - ERef=#'Externalvaluereference'{pos=P,module=Emod, - value=Eval}) -> - case match_parameters(S,ERef,S#state.parameters) of - ERef -> - get_referenced1(S,Emod,Eval,P); - OtherERef when is_record(OtherERef,'Externalvaluereference') -> - get_referenced_type(S,OtherERef); - Value -> - {Emod,Value} - end; -get_referenced_type(S,ERef=#'Externalvaluereference'{pos=Pos,module=Emod, - value=Eval}) -> - case match_parameters(S,ERef,S#state.parameters) of - ERef -> - case lists:member(Emod,S#state.inputmodules) of - true -> - get_referenced1(S,Emod,Eval,Pos); - false -> - get_referenced(S,Emod,Eval,Pos) - end; - OtherERef -> - get_referenced_type(S,OtherERef) - end; -get_referenced_type(S,#identifier{val=Name,pos=Pos}) -> - get_referenced1(S,undefined,Name,Pos); -get_referenced_type(_S,Type) -> - {undefined,Type}. +do_get_referenced_type(#state{parameters=Ps}=S, T0) -> + case match_parameters(S, T0, Ps) of + T0 -> + do_get_ref_type_1(S, T0); + T -> + do_get_referenced_type(S, T) + end. + +do_get_ref_type_1(S, #'Externaltypereference'{pos=P, + module=M, + type=T}) -> + do_get_ref_type_2(S, P, M, T); +do_get_ref_type_1(S, #'Externalvaluereference'{pos=P, + module=M, + value=V}) -> + do_get_ref_type_2(S, P, M, V); +do_get_ref_type_1(_, T) -> + {undefined,T}. + +do_get_ref_type_2(#state{mname=Current,inputmodules=Modules}=S, + Pos, M, T) -> + case M =:= Current orelse lists:member(M, Modules) of + true -> + get_referenced1(S, M, T, Pos); + false -> + get_referenced(S, M, T, Pos) + end. %% get_referenced/3 %% The referenced entity Ename may in case of an imported parameterized @@ -6760,6 +6710,8 @@ format_error({illegal_instance_of,Class}) -> [Class]); format_error(illegal_octet_string_value) -> "expecting a bstring or an hstring as value for an OCTET STRING"; +format_error({illegal_typereference,Name}) -> + io_lib:format("'~p' is used as a typereference, but does not start with an uppercase letter", [Name]); format_error({invalid_fields,Fields,Obj}) -> io_lib:format("invalid ~s in ~p", [format_fields(Fields),Obj]); format_error({invalid_bit_number,Bit}) -> @@ -7006,7 +6958,7 @@ include_default_class1(_,[]) -> include_default_class1(Module,[{Name,TS}|Rest]) -> case asn1_db:dbget(Module,Name) of undefined -> - C = #classdef{checked=true,name=Name, + C = #classdef{checked=true,module=Module,name=Name, typespec=TS}, asn1_db:dbput(Module,Name,C); _ -> ok diff --git a/lib/asn1/src/asn1ct_constructed_ber_bin_v2.erl b/lib/asn1/src/asn1ct_constructed_ber_bin_v2.erl index a38da8bcc2..5fadd0495a 100644 --- a/lib/asn1/src/asn1ct_constructed_ber_bin_v2.erl +++ b/lib/asn1/src/asn1ct_constructed_ber_bin_v2.erl @@ -962,8 +962,7 @@ gen_enc_line(Erules,TopType,Cname,Type,Element,Indent,OptOrMand,Assign,EncObj) WhatKind = asn1ct_gen:type(InnerType), emit(IndDeep), emit(Assign), - gen_optormand_case(OptOrMand,Erules,TopType,Cname,Type,InnerType,WhatKind, - Element), + gen_optormand_case(OptOrMand, Erules, TopType, Cname, Type, Element), case {Type,asn1ct_gen:get_constraint(Type#type.constraint, componentrelation)} of % #type{constraint=[{tableconstraint_info,RefedFieldName}], @@ -1029,26 +1028,19 @@ gen_enc_line(Erules,TopType,Cname,Type,Element,Indent,OptOrMand,Assign,EncObj) emit([nl,indent(7),"end"]) end. -gen_optormand_case(mandatory,_Erules,_TopType,_Cname,_Type,_InnerType,_WhatKind, - _Element) -> +gen_optormand_case(mandatory, _Erules, _TopType, _Cname, _Type, _Element) -> ok; -gen_optormand_case('OPTIONAL',Erules,_TopType,_Cname,_Type,_InnerType,_WhatKind, - Element) -> +gen_optormand_case('OPTIONAL', Erules, _TopType, _Cname, _Type, Element) -> emit([" case ",Element," of",nl]), emit([indent(9),"asn1_NOVALUE -> {", empty_lb(Erules),",0};",nl]), emit([indent(9),"_ ->",nl,indent(12)]); -gen_optormand_case({'DEFAULT',DefaultValue},Erules,TopType,Cname,Type, - InnerType,WhatKind,Element) -> +gen_optormand_case({'DEFAULT',DefaultValue}, Erules, _TopType, + _Cname, Type, Element) -> CurrMod = get(currmod), case catch lists:member(der,get(encoding_options)) of true -> - emit(" case catch "), - asn1ct_gen:gen_check_call(TopType,Cname,Type,InnerType, - WhatKind,{asis,DefaultValue}, - Element), - emit([" of",nl]), - emit([indent(12),"true -> {[],0};",nl]); + asn1ct_gen_check:emit(Type, DefaultValue, Element); _ -> emit([" case ",Element," of",nl]), emit([indent(9),"asn1_DEFAULT -> {", @@ -1063,10 +1055,9 @@ gen_optormand_case({'DEFAULT',DefaultValue},Erules,TopType,Cname,Type, emit([indent(9),{asis, DefaultValue}," -> {", empty_lb(Erules),",0};",nl]) - end - end, - emit([indent(9),"_ ->",nl,indent(12)]). - + end, + emit([indent(9),"_ ->",nl,indent(12)]) + end. gen_dec_line(Erules,TopType,Cname,CTags,Type,OptOrMand,DecObjInf) -> @@ -1210,11 +1201,11 @@ gen_dec_call({typefield,_},_,_,Cname,Type,BytesVar,Tag,_,_,_DecObjInf,OptOrMandC (Type#type.def)#'ObjectClassFieldType'.fieldname, [{Cname,RefedFieldName,asn1ct_gen:mk_var(asn1ct_name:curr(term)), asn1ct_gen:mk_var(asn1ct_name:curr(tmpterm)),Tag,OptOrMandComp}]; -gen_dec_call(InnerType,Erules,TopType,Cname,Type,BytesVar,Tag,PrimOptOrMand, - OptOrMand,DecObjInf,_) -> +gen_dec_call(InnerType, _Erules, TopType, Cname, Type, BytesVar, + Tag, _PrimOptOrMand, _OptOrMand, DecObjInf,_) -> WhatKind = asn1ct_gen:type(InnerType), - gen_dec_call1(WhatKind,InnerType,Erules,TopType,Cname,Type,BytesVar,Tag, - PrimOptOrMand,OptOrMand), + gen_dec_call1(WhatKind, InnerType, TopType, Cname, + Type, BytesVar, Tag), case DecObjInf of {Cname,{_,OSet,_UniqueFName,ValIndex}} -> Term = asn1ct_gen:mk_var(asn1ct_name:curr(term)), @@ -1226,8 +1217,9 @@ gen_dec_call(InnerType,Erules,TopType,Cname,Type,BytesVar,Tag,PrimOptOrMand, ok end, []. -gen_dec_call1({primitive,bif},InnerType,Erules,TopType,Cname,Type,BytesVar, - Tag,OptOrMand,_) -> + +gen_dec_call1({primitive,bif}, InnerType, TopType, Cname, + Type, BytesVar, Tag) -> case {asn1ct:get_gen_state_field(namelist),InnerType} of {[{Cname,undecoded}|Rest],_} -> asn1ct:add_generated_refed_func({[Cname|TopType],undecoded, @@ -1236,11 +1228,10 @@ gen_dec_call1({primitive,bif},InnerType,Erules,TopType,Cname,Type,BytesVar, emit(["{'",asn1ct_gen:list2name([Cname|TopType]),"',", BytesVar,"}"]); _ -> - ?ASN1CT_GEN_BER:gen_dec_prim(Erules,Type,BytesVar,Tag,[], - ?PRIMITIVE,OptOrMand) + ?ASN1CT_GEN_BER:gen_dec_prim(Type, BytesVar, Tag) end; -gen_dec_call1('ASN1_OPEN_TYPE',_InnerType,Erules,TopType,Cname,Type,BytesVar, - Tag,OptOrMand,_) -> +gen_dec_call1('ASN1_OPEN_TYPE', _InnerType, TopType, Cname, + Type, BytesVar, Tag) -> case {asn1ct:get_gen_state_field(namelist),Type#type.def} of {[{Cname,undecoded}|Rest],_} -> asn1ct:add_generated_refed_func({[Cname|TopType],undecoded, @@ -1249,15 +1240,12 @@ gen_dec_call1('ASN1_OPEN_TYPE',_InnerType,Erules,TopType,Cname,Type,BytesVar, emit(["{'",asn1ct_gen:list2name([Cname|TopType]),"',", BytesVar,"}"]); {_,#'ObjectClassFieldType'{type=OpenType}} -> - ?ASN1CT_GEN_BER:gen_dec_prim(Erules,#type{def=OpenType}, - BytesVar,Tag,[], - ?PRIMITIVE,OptOrMand); + ?ASN1CT_GEN_BER:gen_dec_prim(#type{def=OpenType}, + BytesVar, Tag); _ -> - ?ASN1CT_GEN_BER:gen_dec_prim(Erules,Type,BytesVar,Tag,[], - ?PRIMITIVE,OptOrMand) + ?ASN1CT_GEN_BER:gen_dec_prim(Type, BytesVar, Tag) end; -gen_dec_call1(WhatKind,_,_Erules,TopType,Cname,Type,BytesVar, - Tag,_,_OptOrMand) -> +gen_dec_call1(WhatKind, _, TopType, Cname, Type, BytesVar, Tag) -> case asn1ct:get_gen_state_field(namelist) of [{Cname,undecoded}|Rest] -> asn1ct:add_generated_refed_func({[Cname|TopType],undecoded, diff --git a/lib/asn1/src/asn1ct_constructed_per.erl b/lib/asn1/src/asn1ct_constructed_per.erl index ed3f6f886e..a91404ed54 100644 --- a/lib/asn1/src/asn1ct_constructed_per.erl +++ b/lib/asn1/src/asn1ct_constructed_per.erl @@ -426,8 +426,7 @@ gen_dec_open_type(Erule, Val, {Xmod,Xtype}, LeadingAttr, emit([Term," = ",{asis,F},"(",TmpTerm,", ",Val,")"]). dec_objset_optional(N, {'DEFAULT',Val}) -> - dec_objset_optional_1(N, Val), - dec_objset_optional_1(N, asn1_DEFAULT); + dec_objset_optional_1(N, Val); dec_objset_optional(N, 'OPTIONAL') -> dec_objset_optional_1(N, asn1_NOVALUE); dec_objset_optional(_N, mandatory) -> ok. diff --git a/lib/asn1/src/asn1ct_func.erl b/lib/asn1/src/asn1ct_func.erl index 33f998722a..fb94f65b32 100644 --- a/lib/asn1/src/asn1ct_func.erl +++ b/lib/asn1/src/asn1ct_func.erl @@ -19,7 +19,8 @@ %% -module(asn1ct_func). --export([start_link/0,need/1,call/3,call_gen/3,call_gen/4,generate/1]). +-export([start_link/0,need/1,call/3,call_gen/3,call_gen/4, + generate/1,is_used/1]). -export([init/1,handle_call/3,handle_cast/2,terminate/2]). start_link() -> @@ -63,6 +64,10 @@ generate(Fd) -> Funcs = sofs:to_external(Funcs0), ok = file:write(Fd, Funcs). +is_used({_,_,_}=MFA) -> + req({is_used,MFA}). + + req(Req) -> gen_server:call(get(?MODULE), Req, infinity). @@ -103,7 +108,10 @@ handle_call({gen_func,Prefix,Key,GenFun}, _From, #st{gen=G0,gc=Gc0}=St) -> {reply,Name,St#st{gen=G,gc=Gc}}; {value,{Name,_}} -> {reply,Name,St} - end. + end; +handle_call({is_used,MFA}, _From, #st{used=Used}=St) -> + {reply,gb_sets:is_member(MFA, Used),St}. + terminate(_, _) -> ok. diff --git a/lib/asn1/src/asn1ct_gen.erl b/lib/asn1/src/asn1ct_gen.erl index 44b050e59d..450d309688 100644 --- a/lib/asn1/src/asn1ct_gen.erl +++ b/lib/asn1/src/asn1ct_gen.erl @@ -29,7 +29,6 @@ list2rname/1, constructed_suffix/2, unify_if_string/1, - gen_check_call/7, get_constraint/2, insert_once/2, ct_gen_module/1, @@ -43,6 +42,8 @@ -export([gen_encode_constructed/4, gen_decode_constructed/4]). +-define(SUPPRESSION_FUNC, 'dialyzer-suppressions'). + %% pgen(Outfile, Erules, Module, TypeOrVal, Options) %% Generate Erlang module (.erl) and (.hrl) file corresponding to an ASN.1 module %% .hrl file is only generated if necessary @@ -85,12 +86,18 @@ pgen_module(OutFile,Erules,Module, "%%%",nl, "%%% Run-time functions.",nl, "%%%",nl]), + dialyzer_suppressions(Erules), Fd = get(gen_file_out), asn1ct_func:generate(Fd), close_output_file(), _ = erase(outfile), asn1ct:verbose("--~p--~n",[{generated,ErlFile}],Options). +dialyzer_suppressions(Erules) -> + emit([nl, + {asis,?SUPPRESSION_FUNC},"(Arg) ->",nl]), + Rtmod = ct_gen_module(Erules), + Rtmod:dialyzer_suppressions(Erules). pgen_typeorval(Erules,Module,N2nConvEnums,{Types,Values,_Ptypes,_Classes,Objects,ObjectSets}) -> Rtmod = ct_gen_module(Erules), @@ -98,11 +105,6 @@ pgen_typeorval(Erules,Module,N2nConvEnums,{Types,Values,_Ptypes,_Classes,Objects pgen_values(Erules,Module,Values), pgen_objects(Rtmod,Erules,Module,Objects), pgen_objectsets(Rtmod,Erules,Module,ObjectSets), - case catch lists:member(der,get(encoding_options)) of - true -> - pgen_check_defaultval(Erules,Module); - _ -> ok - end, pgen_partial_decode(Rtmod,Erules,Module). pgen_values(_,_,[]) -> @@ -178,23 +180,6 @@ pgen_objectsets(Rtmod,Erules,Module,[H|T]) -> Rtmod:gen_objectset_code(Erules,TypeDef), pgen_objectsets(Rtmod,Erules,Module,T). -pgen_check_defaultval(Erules,Module) -> - CheckObjects = asn1ct_table:to_list(check_functions), - case get(asndebug) of - true -> - FileName = lists:concat([Module,".table"]), - {ok,IoDevice} = file:open(FileName,[write]), - Fun = - fun(X)-> - io:format(IoDevice,"~n~n************~n~n~p~n~n*****" - "********~n~n",[X]) - end, - lists:foreach(Fun,CheckObjects), - ok = file:close(IoDevice); - _ -> ok - end, - gen_check_defaultval(Erules,Module,CheckObjects). - pgen_partial_decode(Rtmod,Erule,Module) when Erule == ber -> pgen_partial_inc_dec(Rtmod,Erule,Module), pgen_partial_dec(Rtmod,Erule,Module); @@ -542,8 +527,7 @@ gen_part_decode_funcs({constructed,bif},TypeName, emit([" 'dec_",TypeName,"'(Data,",{asis,Tag},")"]); gen_part_decode_funcs({primitive,bif},_TypeName, {_Name,undecoded,Tag,Type}) -> - % Argument no 6 is 0, i.e. bit 6 for primitive encoding. - asn1ct_gen_ber_bin_v2:gen_dec_prim(ber_bin_v2,Type,"Data",Tag,[],0,", mandatory, "); + asn1ct_gen_ber_bin_v2:gen_dec_prim(Type, "Data", Tag); gen_part_decode_funcs(WhatKind,_TypeName,{_,Directive,_,_}) -> throw({error,{asn1,{"Not implemented yet",WhatKind," partial incomplete directive:",Directive}}}). @@ -576,131 +560,6 @@ gen_types(Erules,Tname,Type) when is_record(Type,type) -> asn1ct_name:clear(), Rtmod:gen_decode(Erules,Tname,Type). -gen_check_defaultval(Erules,Module,[{Name,Type}|Rest]) -> - gen_check_func(Name,Type), - gen_check_defaultval(Erules,Module,Rest); -gen_check_defaultval(_,_,[]) -> - ok. - -gen_check_func(Name,FType = #type{def=Def}) -> - EncName = ensure_atom(Name), - emit({{asis,EncName},"(_V,asn1_DEFAULT) ->",nl," true;",nl}), - emit({{asis,EncName},"(V,V) ->",nl," true;",nl}), - emit({{asis,EncName},"(V,{_,V}) ->",nl," true;",nl}), - case Def of - {'SEQUENCE OF',Type} -> - gen_check_sof(Name,'SEQOF',Type); - {'SET OF',Type} -> - gen_check_sof(Name,'SETOF',Type); - #'SEQUENCE'{components=Components} -> - gen_check_sequence(Name,Components); - #'SET'{components=Components} -> - gen_check_sequence(Name,Components); - {'CHOICE',Components} -> - gen_check_choice(Name,Components); - #'Externaltypereference'{type=T} -> - emit({{asis,EncName},"(DefaultValue,Value) ->",nl}), - emit({" '",list2name([T,check]),"'(DefaultValue,Value).",nl}); - MaybePrim -> - InnerType = get_inner(MaybePrim), - case type(InnerType) of - {primitive,bif} -> - emit({{asis,EncName},"(DefaultValue,Value) ->",nl," "}), - gen_prim_check_call(get_inner(InnerType),"DefaultValue","Value", - FType), - emit({".",nl,nl}); - _ -> - throw({asn1_error,{unknown,type,MaybePrim}}) - end - end. - -gen_check_sof(Name,SOF,Type) -> - EncName = ensure_atom(Name), - NewName = ensure_atom(list2name([sorted,Name])), - emit({{asis,EncName},"(V1,V2) ->",nl}), - emit({" ",{asis,NewName},"(lists:sort(V1),lists:sort(V2)).",nl,nl}), - emit({{asis,NewName},"([],[]) ->",nl," true;",nl}), - emit({{asis,NewName},"([DV|DVs],[V|Vs]) ->",nl," "}), - InnerType = get_inner(Type#type.def), - case type(InnerType) of - {primitive,bif} -> - gen_prim_check_call(get_inner(InnerType),"DV","V",Type), - emit({",",nl}); - {constructed,bif} -> - emit([{asis,ensure_atom(list2name([SOF,Name]))},"(DV, V),",nl]); - #'Externaltypereference'{type=T} -> - emit([{asis,ensure_atom(list2name([T,check]))},"(DV,V),",nl]); - 'ASN1_OPEN_TYPE' -> - emit(["DV = V,",nl]); - _ -> - emit(["DV = V,",nl]) - end, - emit({" ",{asis,NewName},"(DVs,Vs).",nl,nl}). - -gen_check_sequence(Name, []) -> - emit([{asis,ensure_atom(Name)},"(_,_) ->",nl, - " throw(badval).",nl,nl]); -gen_check_sequence(Name,Components) -> - emit([{asis,ensure_atom(Name)},"(DefaultValue,Value) ->",nl]), - gen_check_sequence(Name,Components,1). - -gen_check_sequence(Name,[#'ComponentType'{name=N,typespec=Type}|Cs],Num) -> - InnerType = get_inner(Type#type.def), - NthDefV = ["element(",Num+1,",DefaultValue)"], - NthV = ["element(",Num+1,",Value)"], - gen_check_func_call(Name,Type,InnerType,NthDefV,NthV,N), - case Cs of - [] -> - emit({".",nl,nl}); - _ -> - emit({",",nl}), - gen_check_sequence(Name,Cs,Num+1) - end. - -gen_check_choice(Name,CList=[#'ComponentType'{}|_Cs]) -> - emit([{asis,ensure_atom(Name)},"({Id,DefaultValue},{Id,Value}) ->",nl]), - emit([" case Id of",nl]), - gen_check_choice_components(Name,CList,1). - -gen_check_choice_components(_,[],_)-> - ok; -gen_check_choice_components(Name,[#'ComponentType'{name=N,typespec=Type}| - Cs],Num) -> - Ind6 = " ", - InnerType = get_inner(Type#type.def), - emit({Ind6,"'",N,"' ->",nl,Ind6}), - gen_check_func_call(Name,Type,InnerType,{var,"defaultValue"}, - {var,"value"},N), - case Cs of - [] -> - emit({nl," end.",nl,nl}); - _ -> - emit({";",nl}), - gen_check_choice_components(Name,Cs,Num+1) - end. - -gen_check_func_call(Name,Type,InnerType,DefVal,Val,N) -> - case type(InnerType) of - {primitive,bif} -> - emit(" "), - gen_prim_check_call(get_inner(InnerType),DefVal,Val,Type); - #'Externaltypereference'{type=T} -> - emit({" ",{asis,ensure_atom(list2name([T,check]))},"(",DefVal,",",Val,")"}); - 'ASN1_OPEN_TYPE' -> - emit([" if",nl, - " ",DefVal," == ",Val," -> true;",nl, - " true -> throw({error,{asn1_open_type}})",nl, - " end",nl]); - {constructed,bif} -> - emit([" ",{asis,ensure_atom(list2name([N,Name]))},"(",DefVal,",",Val,")"]); - _ -> - emit([" if",nl, - " ",DefVal," == ",Val," -> true;",nl, - " true -> throw({error,{asn1_open_type}})",nl, - " end",nl]) - end. - - %% VARIOUS GENERATOR STUFF %% ************************************************* %%************************************************** @@ -790,8 +649,9 @@ gen_decode_constructed(Erules,Typename,InnerType,D) when is_record(D,typedef) -> pgen_exports(Erules,_Module,{Types,Values,_,_,Objects,ObjectSets}) -> - emit(["-export([encoding_rule/0,bit_string_format/0," + emit(["-export([encoding_rule/0,bit_string_format/0,",nl, " legacy_erlang_types/0]).",nl]), + emit(["-export([",{asis,?SUPPRESSION_FUNC},"/1]).",nl]), case Types of [] -> ok; _ -> @@ -1077,9 +937,10 @@ gen_partial_inc_dispatcher() -> ok; {Data1,Data2} -> % io:format("partial_incomplete_decode: ~p~ninc_type_pattern: ~p~n",[Data,Data2]), - gen_partial_inc_dispatcher(Data1,Data2) + gen_partial_inc_dispatcher(Data1, Data2, "") end. -gen_partial_inc_dispatcher([{FuncName,TopType,_Pattern}|Rest],TypePattern) -> + +gen_partial_inc_dispatcher([{FuncName,TopType,_Pattern}|Rest], TypePattern, Sep) -> TPattern = case lists:keysearch(FuncName,1,TypePattern) of {value,{_,TP}} -> TP; @@ -1093,13 +954,13 @@ gen_partial_inc_dispatcher([{FuncName,TopType,_Pattern}|Rest],TypePattern) -> _ -> atom_to_list(TopType) end, - emit(["decode_partial_inc_disp('",TopTypeName,"',Data) ->",nl, + emit([Sep, + "decode_partial_inc_disp('",TopTypeName,"',Data) ->",nl, " ",{asis,list_to_atom(lists:concat(["dec-inc-",FuncName2]))}, - "(Data);",nl]), - gen_partial_inc_dispatcher(Rest,TypePattern); -gen_partial_inc_dispatcher([],_) -> - emit(["decode_partial_inc_disp(Type,_Data) ->",nl, - " exit({error,{asn1,{undefined_type,Type}}}).",nl]). + "(Data)"]), + gen_partial_inc_dispatcher(Rest, TypePattern, ";\n"); +gen_partial_inc_dispatcher([], _, _) -> + emit([".",nl]). gen_dispatcher([F1,F2|T],FuncName,Prefix,ExtraArg) -> emit([FuncName,"('",F1,"',Data) -> '",Prefix,F1,"'(Data",ExtraArg,")",";",nl]), @@ -1465,171 +1326,6 @@ to_textual_order(Cs=[#'ComponentType'{textual_order=undefined}|_]) -> to_textual_order(Cs) when is_list(Cs) -> lists:keysort(#'ComponentType'.textual_order,Cs). - -gen_check_call(TopType,Cname,Type,InnerType,WhatKind,DefaultValue,Element) -> - case WhatKind of - {primitive,bif} -> - gen_prim_check_call(InnerType,DefaultValue,Element,Type); - #'Externaltypereference'{module=M,type=T} -> - %% generate function call - Name = list2name([T,check]), - emit({"'",Name,"'(",DefaultValue,", ",Element,")"}), - %% insert in ets table and do look ahead check - Typedef = asn1_db:dbget(M,T), - RefType = Typedef#typedef.typespec, - InType = asn1ct_gen:get_inner(RefType#type.def), - case insert_once(check_functions,{Name,RefType}) of - true -> - lookahead_innertype([T],InType,RefType); - _ -> - ok - end; - {constructed,bif} -> - NameList = [Cname|TopType], - Name = list2name(NameList ++ [check]), - emit({"'",Name,"'(",DefaultValue,", ",Element,")"}), - asn1ct_table:insert(check_functions, {Name, Type}), - %% Must look for check functions in InnerType, - %% that may be referenced or internal defined - %% constructed types not used elsewhere. - lookahead_innertype(NameList,InnerType,Type); - _ -> - %% Generate Dummy function call i.e. anything is accepted - emit(["fun() -> true end ()"]) - end. - -gen_prim_check_call(PrimType, Default, Element, Type) -> - case unify_if_string(PrimType) of - 'BOOLEAN' -> - check_call(check_bool, [Default,Element]); - 'INTEGER' -> - NNL = case Type#type.def of - {_,NamedNumberList} -> NamedNumberList; - _ -> [] - end, - check_call(check_int, [Default,Element,{asis,NNL}]); - 'BIT STRING' -> - case Type#type.def of - {_,[]} -> - check_call(check_bitstring, - [Default,Element]); - {_,[_|_]=NBL} -> - check_call(check_named_bitstring, - [Default,Element,{asis,NBL}]) - end; - 'OCTET STRING' -> - check_call(check_octetstring, [Default,Element]); - 'NULL' -> - check_call(check_null, [Default,Element]); - 'OBJECT IDENTIFIER' -> - check_call(check_objectidentifier, [Default,Element]); - 'RELATIVE-OID' -> - check_call(check_objectidentifier, [Default,Element]); - 'ObjectDescriptor' -> - check_call(check_objectdescriptor, [Default,Element]); - 'REAL' -> - check_call(check_real, [Default,Element]); - 'ENUMERATED' -> - {_,Enumerations} = Type#type.def, - check_call(check_enum, [Default,Element,{asis,Enumerations}]); - restrictedstring -> - check_call(check_restrictedstring, [Default,Element]) - end. - -check_call(F, Args) -> - asn1ct_func:call(check, F, Args). - -%% lokahead_innertype/3 traverses Type and checks if check functions -%% have to be generated, i.e. for all constructed or referenced types. -lookahead_innertype(Name,'SEQUENCE',Type) -> - Components = (Type#type.def)#'SEQUENCE'.components, - lookahead_components(Name,Components); -lookahead_innertype(Name,'SET',Type) -> - Components = (Type#type.def)#'SET'.components, - lookahead_components(Name,Components); -lookahead_innertype(Name,'CHOICE',Type) -> - {_,Components} = Type#type.def, - lookahead_components(Name,Components); -lookahead_innertype(Name,'SEQUENCE OF',SeqOf) -> - lookahead_sof(Name,'SEQOF',SeqOf); -lookahead_innertype(Name,'SET OF',SeqOf) -> - lookahead_sof(Name,'SETOF',SeqOf); -lookahead_innertype(_Name,#'Externaltypereference'{module=M,type=T},_) -> - Typedef = asn1_db:dbget(M,T), - RefType = Typedef#typedef.typespec, - insert_once(check_functions,{list2name([T,check]),RefType}), - InType = asn1ct_gen:get_inner(RefType#type.def), - case type(InType) of - {constructed,bif} -> - lookahead_innertype([T],InType,RefType); - Ref = #'Externaltypereference'{} -> - lookahead_reference(Ref); - _ -> - ok - end; -lookahead_innertype(_,_,_) -> - ok. - -lookahead_components(_,[]) -> ok; -lookahead_components(Name,[C|Cs]) -> - #'ComponentType'{name=Cname,typespec=Type} = C, - InType = asn1ct_gen:get_inner(Type#type.def), - case asn1ct_gen:type(InType) of - {constructed,bif} -> - case insert_once(check_functions, - {list2name([Cname|Name] ++ [check]),Type}) of - true -> - lookahead_innertype([Cname|Name],InType,Type); - _ -> - ok - end; - #'Externaltypereference'{module=RefMod,type=RefName} -> - Typedef = asn1_db:dbget(RefMod,RefName), - RefType = Typedef#typedef.typespec, - case insert_once(check_functions,{list2name([RefName,check]), - RefType}) of - true -> - lookahead_innertype([RefName],InType,RefType); - _ -> - ok - end; - _ -> - ok - end, - lookahead_components(Name,Cs). - -lookahead_sof(Name,SOF,SOFType) -> - Type = case SOFType#type.def of - {_,_Type} -> _Type; - _Type -> _Type - end, - InnerType = asn1ct_gen:get_inner(Type#type.def), - case asn1ct_gen:type(InnerType) of - {constructed,bif} -> - %% this is if a constructed type is defined in - %% the SEQUENCE OF type - NameList = [SOF|Name], - insert_once(check_functions, - {list2name(NameList ++ [check]),Type}), - lookahead_innertype(NameList,InnerType,Type); - Ref = #'Externaltypereference'{} -> - lookahead_reference(Ref); - _ -> - ok - end. - -lookahead_reference(#'Externaltypereference'{module=M,type=T}) -> - Typedef = asn1_db:dbget(M,T), - RefType = Typedef#typedef.typespec, - InType = get_inner(RefType#type.def), - case insert_once(check_functions, - {list2name([T,check]),RefType}) of - true -> - lookahead_innertype([T],InType,RefType); - _ -> - ok - end. - insert_once(Table,Object) -> case asn1ct_table:lookup(Table, element(1, Object)) of [] -> @@ -1683,6 +1379,11 @@ conform_value(#type{def={'BIT STRING',[]}}, Bs) -> bitstring when is_bitstring(Bs) -> Bs end; +conform_value(#type{def='OCTET STRING'}, String) -> + case asn1ct:use_legacy_types() of + false -> String; + true -> binary_to_list(String) + end; conform_value(_, Value) -> Value. named_bitstring_value(List, Names) -> @@ -1901,11 +1602,6 @@ get_constraint(C,Key) -> {value,Cnstr} -> Cnstr end. - -ensure_atom(Atom) when is_atom(Atom) -> - Atom; -ensure_atom(List) when is_list(List) -> - list_to_atom(List). get_record_name_prefix() -> case lists:keysearch(record_name_prefix,1,get(encoding_options)) of diff --git a/lib/asn1/src/asn1ct_gen_ber_bin_v2.erl b/lib/asn1/src/asn1ct_gen_ber_bin_v2.erl index bea0ec8968..e51b0898be 100644 --- a/lib/asn1/src/asn1ct_gen_ber_bin_v2.erl +++ b/lib/asn1/src/asn1ct_gen_ber_bin_v2.erl @@ -27,11 +27,12 @@ -export([decode_class/1, decode_type/1]). -export([gen_encode/2,gen_encode/3,gen_decode/2,gen_decode/3]). -export([gen_encode_prim/4]). --export([gen_dec_prim/7]). +-export([gen_dec_prim/3]). -export([gen_objectset_code/2, gen_obj_code/3]). -export([encode_tag_val/3]). -export([gen_inc_decode/2,gen_decode_selected/3]). -export([extaddgroup2sequence/1]). +-export([dialyzer_suppressions/1]). -import(asn1ct_gen, [emit/1,demit/1]). @@ -65,6 +66,23 @@ %%=============================================================================== %%=============================================================================== +dialyzer_suppressions(_) -> + case asn1ct:use_legacy_types() of + false -> ok; + true -> suppress({ber,encode_bit_string,4}) + end, + suppress({ber,decode_selective,2}), + emit([" ok.",nl]). + +suppress({M,F,A}=MFA) -> + case asn1ct_func:is_used(MFA) of + false -> + ok; + true -> + Args = [lists:concat(["element(",I,", Arg)"]) || I <- lists:seq(1, A)], + emit([" ",{call,M,F,Args},com,nl]) + end. + %%=============================================================================== %% encode #{typedef, {pos, name, typespec}} %%=============================================================================== @@ -163,6 +181,12 @@ gen_encode_user(Erules, #typedef{}=D, Wrapper) -> gen_encode_prim(_Erules, #type{}=D, DoTag, Value) -> BitStringConstraint = get_size_constraint(D#type.constraint), + MaxBitStrSize = case BitStringConstraint of + [] -> none; + {_,'MAX'} -> none; + {_,Max} -> Max; + Max when is_integer(Max) -> Max + end, asn1ct_name:new(enumval), Type = case D#type.def of 'OCTET STRING' -> restricted_string; @@ -208,11 +232,11 @@ gen_encode_prim(_Erules, #type{}=D, DoTag, Value) -> "end"]); {'BIT STRING',[]} -> case asn1ct:use_legacy_types() of - false when BitStringConstraint =:= [] -> + false when MaxBitStrSize =:= none -> call(encode_unnamed_bit_string, [Value,DoTag]); false -> call(encode_unnamed_bit_string, - [{asis,BitStringConstraint},Value,DoTag]); + [{asis,MaxBitStrSize},Value,DoTag]); true -> call(encode_bit_string, [{asis,BitStringConstraint},Value, @@ -220,12 +244,12 @@ gen_encode_prim(_Erules, #type{}=D, DoTag, Value) -> end; {'BIT STRING',NamedNumberList} -> case asn1ct:use_legacy_types() of - false when BitStringConstraint =:= [] -> + false when MaxBitStrSize =:= none -> call(encode_named_bit_string, [Value,{asis,NamedNumberList},DoTag]); false -> call(encode_named_bit_string, - [{asis,BitStringConstraint},Value, + [{asis,MaxBitStrSize},Value, {asis,NamedNumberList},DoTag]); true -> call(encode_bit_string, @@ -254,14 +278,20 @@ emit_enc_enumerated_cases(L, Tags) -> emit_enc_enumerated_cases(L, Tags, noext). emit_enc_enumerated_cases([{EnumName,EnumVal}|T], Tags, Ext) -> + Bytes = encode_pos_integer(EnumVal, []), + Len = length(Bytes), emit([{asis,EnumName}," -> ", - {call,ber,encode_enumerated,[EnumVal,Tags]},";",nl]), + {call,ber,encode_tags,[Tags,{asis,Bytes},Len]},";",nl]), emit_enc_enumerated_cases(T, Tags, Ext); emit_enc_enumerated_cases([], _Tags, _Ext) -> %% FIXME: Should extension be handled? emit([{curr,enumval}," -> exit({error,{asn1, {enumerated_not_in_range,",{curr, enumval},"}}})"]), emit([nl,"end"]). +encode_pos_integer(0, [B|_Acc] = L) when B < 128 -> + L; +encode_pos_integer(N, Acc) -> + encode_pos_integer(N bsr 8, [N band 255|Acc]). %%=============================================================================== %%=============================================================================== @@ -339,15 +369,11 @@ gen_decode_selected_type(_Erules,TypeDef) -> case asn1ct_gen:type(InnerType) of 'ASN1_OPEN_TYPE' -> asn1ct_name:new(len), - gen_dec_prim(ber, Def#type{def='ASN1_OPEN_TYPE'}, - BytesVar,Tag, [] , - ?PRIMITIVE,"OptOrMand"); -% emit({";",nl}); + gen_dec_prim(Def#type{def='ASN1_OPEN_TYPE'}, + BytesVar, Tag); {primitive,bif} -> asn1ct_name:new(len), - gen_dec_prim(ber, Def, BytesVar,Tag,[] , - ?PRIMITIVE,"OptOrMand"); -% emit([";",nl]); + gen_dec_prim(Def, BytesVar, Tag); {constructed,bif} -> TopType = case TypeDef#typedef.name of A when is_atom(A) -> [A]; @@ -461,14 +487,12 @@ gen_decode_user(Erules,D) when is_record(D,typedef) -> case asn1ct_gen:type(InnerType) of 'ASN1_OPEN_TYPE' -> asn1ct_name:new(len), - gen_dec_prim(ber, Def#type{def='ASN1_OPEN_TYPE'}, - BytesVar,{string,"TagIn"}, [] , - ?PRIMITIVE,"OptOrMand"), + gen_dec_prim(Def#type{def='ASN1_OPEN_TYPE'}, + BytesVar, {string,"TagIn"}), emit({".",nl,nl}); {primitive,bif} -> asn1ct_name:new(len), - gen_dec_prim(ber, Def, BytesVar,{string,"TagIn"},[] , - ?PRIMITIVE,"OptOrMand"), + gen_dec_prim(Def, BytesVar, {string,"TagIn"}), emit([".",nl,nl]); {constructed,bif} -> asn1ct:update_namelist(D#typedef.name), @@ -481,17 +505,10 @@ gen_decode_user(Erules,D) when is_record(D,typedef) -> end. -gen_dec_prim(_Erules, Att, BytesVar, DoTag, _TagIn, _Form, _OptOrMand) -> +gen_dec_prim(Att, BytesVar, DoTag) -> Typename = Att#type.def, -%% Currently not used for BER replaced with [] as place holder -%% Constraint = Att#type.constraint, -%% Constraint = [], Constraint = get_size_constraint(Att#type.constraint), IntConstr = int_constr(Att#type.constraint), - AsBin = case get(binary_strings) of - true -> "_as_bin"; - _ -> "" - end, NewTypeName = case Typename of 'NumericString' -> restricted_string; 'TeletexString' -> restricted_string; @@ -505,107 +522,77 @@ gen_dec_prim(_Erules, Att, BytesVar, DoTag, _TagIn, _Form, _OptOrMand) -> 'ObjectDescriptor'-> restricted_string; 'UTCTime' -> restricted_string; 'GeneralizedTime' -> restricted_string; + 'OCTET STRING' -> + case asn1ct:use_legacy_types() of + true -> restricted_string; + false -> Typename + end; _ -> Typename end, + TagStr = case DoTag of + {string,Tag1} -> Tag1; + _ when is_list(DoTag) -> {asis,DoTag} + end, case NewTypeName of 'BOOLEAN'-> - emit(["decode_boolean(",BytesVar,","]), - need(decode_boolean, 2); + call(decode_boolean, [BytesVar,TagStr]); 'INTEGER' -> - case IntConstr of - [] -> - emit(["decode_integer(",BytesVar,","]), - need(decode_integer, 2); - {_,_} -> - emit(["decode_integer(",BytesVar,",", - {asis,IntConstr},","]), - need(decode_integer, 3) - end; - {'INTEGER',NamedNumberList} -> - case IntConstr of - [] -> - emit(["decode_named_integer(",BytesVar,",", - {asis,NamedNumberList},","]), - need(decode_named_integer, 3); - {_,_} -> - emit(["decode_named_integer(",BytesVar,",", - {asis,IntConstr},",", - {asis,NamedNumberList},","]), - need(decode_named_integer, 4) - end; - {'ENUMERATED',NamedNumberList} -> - emit(["decode_enumerated(",BytesVar,",", - {asis,NamedNumberList},","]), - need(decode_enumerated, 3); + check_constraint(decode_integer, [BytesVar,TagStr], + IntConstr, + identity, + identity); + {'INTEGER',NNL} -> + check_constraint(decode_integer, + [BytesVar,TagStr], + IntConstr, + identity, + fun(Val) -> + asn1ct_name:new(val), + emit([{curr,val}," = "]), + Val(), + emit([com,nl, + {call,ber,number2name, + [{curr,val},{asis,NNL}]}]) + end); + {'ENUMERATED',NNL} -> + gen_dec_enumerated(BytesVar, NNL, TagStr); 'REAL' -> - ok; - {'BIT STRING',_NamedNumberList} -> - ok; + asn1ct_name:new(tmpbuf), + emit(["begin",nl, + {curr,tmpbuf}," = ", + {call,ber,match_tags,[BytesVar,TagStr]},com,nl, + {call,real_common,decode_real,[{curr,tmpbuf}]},nl, + "end",nl]); + {'BIT STRING',NNL} -> + gen_dec_bit_string(BytesVar, Constraint, NNL, TagStr); 'NULL' -> - emit(["decode_null(",BytesVar,","]), - need(decode_null, 2); + call(decode_null, [BytesVar,TagStr]); 'OBJECT IDENTIFIER' -> - emit(["decode_object_identifier(",BytesVar,","]), - need(decode_object_identifier, 2); + call(decode_object_identifier, [BytesVar,TagStr]); 'RELATIVE-OID' -> - emit(["decode_relative_oid(",BytesVar,","]), - need(decode_relative_oid, 2); + call(decode_relative_oid, [BytesVar,TagStr]); 'OCTET STRING' -> - F = case asn1ct:use_legacy_types() of - false -> decode_octet_string; - true -> decode_restricted_string - end, - emit([{asis,F},"(",BytesVar,","]), - case Constraint of - [] -> - need(F, 2); - _ -> - emit([{asis,Constraint},","]), - need(F, 3) - end; + check_constraint(decode_octet_string, [BytesVar,TagStr], + Constraint, {erlang,byte_size}, identity); restricted_string -> - emit(["decode_restricted_string",AsBin,"(",BytesVar,","]), - case Constraint of - [] -> - need(decode_restricted_string, 2); - _ -> - emit([{asis,Constraint},","]), - need(decode_restricted_string, 3) - end; + check_constraint(decode_restricted_string, [BytesVar,TagStr], + Constraint, + {erlang,byte_size}, + fun(Val) -> + emit("binary_to_list("), + Val(), + emit(")") + end); 'UniversalString' -> - emit(["decode_universal_string",AsBin,"(", - BytesVar,",",{asis,Constraint},","]), - need(decode_universal_string, 3); + check_constraint(decode_universal_string, [BytesVar,TagStr], + Constraint, {erlang,length}, identity); 'UTF8String' -> - emit(["decode_UTF8_string",AsBin,"(", - BytesVar,","]), - need(decode_UTF8_string, 2); + call(decode_UTF8_string, [BytesVar,TagStr]); 'BMPString' -> - emit(["decode_BMP_string",AsBin,"(", - BytesVar,",",{asis,Constraint},","]), - need(decode_BMP_string, 3); + check_constraint(decode_BMP_string, [BytesVar,TagStr], + Constraint, {erlang,length}, identity); 'ASN1_OPEN_TYPE' -> - emit(["decode_open_type_as_binary(", - BytesVar,","]), - need(decode_open_type_as_binary, 2) - end, - - TagStr = case DoTag of - {string,Tag1} -> Tag1; - _ when is_list(DoTag) -> {asis,DoTag} - end, - case NewTypeName of - {'BIT STRING',NNL} -> - gen_dec_bit_string(BytesVar, Constraint, NNL, TagStr); - 'REAL' -> - asn1ct_name:new(tmpbuf), - emit(["begin",nl, - {curr,tmpbuf}," = ", - {call,ber,match_tags,[BytesVar,TagStr]},com,nl, - {call,real_common,decode_real,[{curr,tmpbuf}]},nl, - "end",nl]); - _ -> - emit([TagStr,")"]) + call(decode_open_type_as_binary, [BytesVar,TagStr]) end. %% Simplify an integer constraint so that we can efficiently test it. @@ -621,7 +608,7 @@ int_constr(C) -> [{'ValueRange',{_,_}=Range}] -> Range; [{'SingleValue',Sv}] -> - {Sv,Sv}; + Sv; [] -> [] end. @@ -632,16 +619,108 @@ gen_dec_bit_string(BytesVar, _Constraint, [_|_]=NNL, TagStr) -> gen_dec_bit_string(BytesVar, Constraint, [], TagStr) -> case asn1ct:get_bit_string_format() of compact -> - call(decode_compact_bit_string, - [BytesVar,{asis,Constraint},TagStr]); + check_constraint(decode_compact_bit_string, + [BytesVar,TagStr], + Constraint, + {ber,compact_bit_string_size}, + identity); legacy -> - call(decode_legacy_bit_string, - [BytesVar,{asis,Constraint},TagStr]); + check_constraint(decode_native_bit_string, + [BytesVar,TagStr], + Constraint, + {erlang,bit_size}, + fun(Val) -> + asn1ct_name:new(val), + emit([{curr,val}," = "]), + Val(), + emit([com,nl, + {call,ber,native_to_legacy_bit_string, + [{curr,val}]}]) + end); bitstring -> - call(decode_native_bit_string, - [BytesVar,{asis,Constraint},TagStr]) + check_constraint(decode_native_bit_string, + [BytesVar,TagStr], + Constraint, + {erlang,bit_size}, + identity) + end. + +check_constraint(F, Args, Constr, PreConstr0, ReturnVal0) -> + PreConstr = case PreConstr0 of + identity -> + fun(V) -> V end; + {Mod,Name} -> + fun(V) -> + asn1ct_name:new(c), + emit([{curr,c}," = ", + {call,Mod,Name,[V]},com,nl]), + {curr,c} + end + end, + ReturnVal = case ReturnVal0 of + identity -> fun(Val) -> Val() end; + _ -> ReturnVal0 + end, + case Constr of + [] when ReturnVal0 =:= identity -> + %% No constraint, no complications. + call(F, Args); + [] -> + %% No constraint, but the return value could consist + %% of more than one statement. + emit(["begin",nl]), + ReturnVal(fun() -> call(F, Args) end), + emit([nl, + "end",nl]); + _ -> + %% There is a constraint. + asn1ct_name:new(val), + emit(["begin",nl, + {curr,val}," = ",{call,ber,F,Args},com,nl]), + PreVal0 = asn1ct_gen:mk_var(asn1ct_name:curr(val)), + PreVal = PreConstr(PreVal0), + emit("if "), + case Constr of + {Min,Max} -> + emit([{asis,Min}," =< ",PreVal,", ", + PreVal," =< ",{asis,Max}]); + Sv when is_integer(Sv) -> + emit([PreVal," =:= ",{asis,Sv}]) + end, + emit([" ->",nl]), + ReturnVal(fun() -> emit(PreVal0) end), + emit([";",nl, + "true ->",nl, + "exit({error,{asn1,bad_range}})",nl, + "end",nl, + "end"]) end. +gen_dec_enumerated(BytesVar, NNL0, TagStr) -> + asn1ct_name:new(enum), + emit(["case ", + {call,ber,decode_integer,[BytesVar,TagStr]}, + " of",nl]), + NNL = case NNL0 of + {L1,L2} -> + L1 ++ L2 ++ [accept]; + [_|_] -> + NNL0 ++ [error] + end, + gen_dec_enumerated_1(NNL), + emit("end"). + +gen_dec_enumerated_1([accept]) -> + asn1ct_name:new(default), + emit([{curr,default}," -> {asn1_enum,",{curr,default},"}",nl]); +gen_dec_enumerated_1([error]) -> + asn1ct_name:new(default), + emit([{curr,default}," -> exit({error,{asn1,{illegal_enumerated,", + {curr,default},"}}})",nl]); +gen_dec_enumerated_1([{V,K}|T]) -> + emit([{asis,K}," -> ",{asis,V},";",nl]), + gen_dec_enumerated_1(T). + %% Object code generating for encoding and decoding %% ------------------------------------------------ @@ -986,9 +1065,8 @@ gen_decode_field_call(ObjName,FieldName,Bytes,Type) -> Tag = [(decode_class(X#tag.class) bsl 10) + X#tag.number || X <- OTag], case Type#typedef.name of - {primitive,bif} -> %%tag should be the primitive tag - gen_dec_prim(ber,Def,Bytes,Tag,"TagIn",?PRIMITIVE, - opt_or_default), + {primitive,bif} -> + gen_dec_prim(Def, Bytes, Tag), []; {constructed,bif} -> emit({" 'dec_",ObjName,'_',FieldName, @@ -1016,8 +1094,7 @@ gen_decode_default_call(ClassName,FieldName,Bytes,Type) -> FieldName])), typespec=Type}]; {primitive,bif} -> - gen_dec_prim(ber,Type,Bytes,Tag,"TagIn", - ?PRIMITIVE,opt_or_default), + gen_dec_prim(Type, Bytes, Tag), []; #'Externaltypereference'{module=CurrentMod,type=Etype} -> emit([" 'dec_",Etype,"'(",Bytes, " ,",{asis,Tag},")",nl]), @@ -1386,7 +1463,7 @@ emit_dec_open_type(I) -> end, emit(S). -emit_inner_of_decfun(#typedef{name={ExtName,Name},typespec=Type},Prop, +emit_inner_of_decfun(#typedef{name={ExtName,Name},typespec=Type}, _Prop, InternalDefFunName) -> OTag = Type#type.tag, %% Tag = [X#tag{class=decode_class(X#tag.class)}|| X <- OTag], @@ -1394,8 +1471,7 @@ emit_inner_of_decfun(#typedef{name={ExtName,Name},typespec=Type},Prop, case {ExtName,Name} of {primitive,bif} -> emit(indent(12)), - gen_dec_prim(ber,Type,"Bytes",Tag,"TagIn", - ?PRIMITIVE,Prop), + gen_dec_prim(Type, "Bytes", Tag), 0; {constructed,bif} -> emit([indent(12),"'dec_", @@ -1412,7 +1488,7 @@ emit_inner_of_decfun(#typedef{name={ExtName,Name},typespec=Type},Prop, emit_inner_of_decfun(#typedef{name=Name},_Prop,_) -> emit([indent(12),"'dec_",Name,"'(Bytes)"]), 0; -emit_inner_of_decfun(Type,Prop,_) when is_record(Type,type) -> +emit_inner_of_decfun(#type{}=Type, _Prop, _) -> OTag = Type#type.tag, %% Tag = [X#tag{class=decode_class(X#tag.class)}|| X <- OTag], Tag = [(decode_class(X#tag.class) bsl 10) + X#tag.number || X <- OTag], @@ -1423,8 +1499,7 @@ emit_inner_of_decfun(Type,Prop,_) when is_record(Type,type) -> case WhatKind of {primitive,bif} -> emit([indent(9),Def," ->",nl,indent(12)]), - gen_dec_prim(ber,Type,"Bytes",Tag,"TagIn", - ?PRIMITIVE,Prop); + gen_dec_prim(Type, "Bytes", Tag); #'Externaltypereference'{module=CurrMod,type=T} -> emit([indent(9),T," ->",nl,indent(12),"'dec_",T, % "'(Bytes, ",Prop,")"]); @@ -1561,6 +1636,3 @@ extaddgroup2sequence(ExtList) when is_list(ExtList) -> call(F, Args) -> asn1ct_func:call(ber, F, Args). - -need(F, Arity) -> - asn1ct_func:need({ber,F,Arity}). diff --git a/lib/asn1/src/asn1ct_gen_check.erl b/lib/asn1/src/asn1ct_gen_check.erl new file mode 100644 index 0000000000..d80a02dfbf --- /dev/null +++ b/lib/asn1/src/asn1ct_gen_check.erl @@ -0,0 +1,271 @@ +%% vim: tabstop=8:shiftwidth=4 +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2014. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% + +-module(asn1ct_gen_check). +-export([emit/3]). + +-import(asn1ct_gen, [emit/1]). +-include("asn1_records.hrl"). + +emit(Type, Default, Value) -> + Key = {Type,Default}, + Gen = fun(Fd, Name) -> + file:write(Fd, gen(Name, Type, Default)) + end, + emit(" case "), + asn1ct_func:call_gen("is_default_", Key, Gen, [Value]), + emit([" of",nl, + "true -> {[],0};",nl, + "false ->",nl]). + +gen(Name, #type{def=T}, Default) -> + NameStr = atom_to_list(Name), + [NameStr,"(asn1_DEFAULT) ->\n", + "true;\n"|case do_gen(T, Default) of + {literal,Literal} -> + [NameStr,"(",term2str(Literal),") ->\n","true;\n", + NameStr,"(_) ->\n","false.\n\n"]; + {exception,Func,Args} -> + [NameStr,"(Value) ->\n", + "try ",Func,"(Value",arg2str(Args),") of\n", + "_ -> true\n" + "catch throw:false -> false\n" + "end.\n\n"] + end]. + +do_gen(_, asn1_NOVALUE) -> + {literal,asn1_NOVALUE}; +do_gen(#'Externaltypereference'{module=M,type=T}, Default) -> + #typedef{typespec=#type{def=Td}} = asn1_db:dbget(M, T), + do_gen(Td, Default); +do_gen('BOOLEAN', Default) -> + {literal,Default}; +do_gen({'BIT STRING',[]}, Default) -> + true = is_bitstring(Default), %Assertion. + case asn1ct:use_legacy_types() of + false -> + {literal,Default}; + true -> + {exception,need(check_legacy_bitstring, 2),[Default]} + end; +do_gen({'BIT STRING',[_|_]=NBL}, Default) -> + do_named_bitstring(NBL, Default); +do_gen({'ENUMERATED',_}, Default) -> + {literal,Default}; +do_gen('INTEGER', Default) -> + {literal,Default}; +do_gen({'INTEGER',NNL}, Default) -> + {exception,need(check_int, 3),[Default,NNL]}; +do_gen('NULL', Default) -> + {literal,Default}; +do_gen('OCTET STRING', Default) -> + true = is_binary(Default), %Assertion. + case asn1ct:use_legacy_types() of + false -> + {literal,Default}; + true -> + {exception,need(check_octetstring, 2),[Default]} + end; +do_gen('OBJECT IDENTIFIER', Default0) -> + Default = pre_process_oid(Default0), + {exception,need(check_objectidentifier, 2),[Default]}; +do_gen({'CHOICE',Cs}, Default) -> + {Tag,Value} = Default, + [Type] = [Type || #'ComponentType'{name=T,typespec=Type} <- Cs, + T =:= Tag], + case do_gen(Type#type.def, Value) of + {literal,Lit} -> + {literal,{Tag,Lit}}; + {exception,Func0,Args} -> + Key = {Tag,Func0,Args}, + Gen = fun(Fd, Name) -> + S = gen_choice(Name, Tag, Func0, Args), + ok = file:write(Fd, S) + end, + Func = asn1ct_func:call_gen("is_default_choice", Key, Gen), + {exception,atom_to_list(Func),[]} + end; +do_gen(#'SEQUENCE'{components=Cs}, Default) -> + do_seq_set(Cs, Default); +do_gen({'SEQUENCE OF',Type}, Default) -> + do_sof(Type, Default); +do_gen(#'SET'{components=Cs}, Default) -> + do_seq_set(Cs, Default); +do_gen({'SET OF',Type}, Default) -> + do_sof(Type, Default); +do_gen(Type, Default) -> + case asn1ct_gen:unify_if_string(Type) of + restrictedstring -> + {exception,need(check_restrictedstring, 2),[Default]}; + _ -> + %% Open type. Do our best. + {literal,Default} + end. + +do_named_bitstring(NBL, Default0) when is_list(Default0) -> + Default = lists:sort(Default0), + Bs = asn1ct_gen:named_bitstring_value(Default, NBL), + Func = case asn1ct:use_legacy_types() of + false -> check_named_bitstring; + true -> check_legacy_named_bitstring + end, + {exception,need(Func, 4),[Default,Bs,bit_size(Bs)]}; +do_named_bitstring(_, Default) when is_bitstring(Default) -> + Func = case asn1ct:use_legacy_types() of + false -> check_named_bitstring; + true -> check_legacy_named_bitstring + end, + {exception,need(Func, 3),[Default,bit_size(Default)]}. + +do_seq_set(Cs0, Default) -> + Tag = element(1, Default), + Cs1 = [T || #'ComponentType'{typespec=T} <- Cs0], + Cs = components(Cs1, tl(tuple_to_list(Default))), + case are_all_literals(Cs) of + true -> + Literal = list_to_tuple([Tag|[L || {literal,L} <- Cs]]), + {literal,Literal}; + false -> + Key = {Cs,Default}, + Gen = fun(Fd, Name) -> + S = gen_components(Name, Tag, Cs), + ok = file:write(Fd, S) + end, + Func = asn1ct_func:call_gen("is_default_cs_", Key, Gen), + {exception,atom_to_list(Func),[]} + end. + +do_sof(Type, Default0) -> + Default = lists:sort(Default0), + Cs0 = lists:duplicate(length(Default), Type), + Cs = components(Cs0, Default), + case are_all_literals(Cs) of + true -> + Literal = [Lit || {literal,Lit} <- Cs], + {exception,need(check_literal_sof, 2),[Literal]}; + false -> + Key = Cs, + Gen = fun(Fd, Name) -> + S = gen_sof(Name, Cs), + ok = file:write(Fd, S) + end, + Func = asn1ct_func:call_gen("is_default_sof", Key, Gen), + {exception,atom_to_list(Func),[]} + end. + +are_all_literals([{literal,_}|T]) -> + are_all_literals(T); +are_all_literals([_|_]) -> + false; +are_all_literals([]) -> true. + +gen_components(Name, Tag, Cs) -> + [atom_to_list(Name),"(Value) ->\n", + "case Value of\n", + "{",term2str(Tag)|gen_cs_1(Cs, 1, [])]. + +gen_cs_1([{literal,Lit}|T], I, Acc) -> + [",\n",term2str(Lit)|gen_cs_1(T, I, Acc)]; +gen_cs_1([H|T], I, Acc) -> + Var = "E"++integer_to_list(I), + [",\n",Var|gen_cs_1(T, I+1, [{Var,H}|Acc])]; +gen_cs_1([], _, Acc) -> + ["} ->\n"|gen_cs_2(Acc, "")]. + +gen_cs_2([{Var,{exception,Func,Args}}|T], Sep) -> + [Sep,Func,"(",Var,arg2str(Args),")"|gen_cs_2(T, ",\n")]; +gen_cs_2([], _) -> + [";\n", + "_ ->\n" + "throw(false)\n" + "end.\n"]. + +gen_sof(Name, Cs) -> + [atom_to_list(Name),"(Value) ->\n", + "case length(Value) of\n", + integer_to_list(length(Cs))," -> ok;\n" + "_ -> throw(false)\n" + "end,\n" + "T0 = lists:sort(Value)"|gen_sof_1(Cs, 1)]. + +gen_sof_1([{exception,Func,Args}|Cs], I) -> + NumStr = integer_to_list(I), + H = "H" ++ NumStr, + T = "T" ++ NumStr, + Prev = "T" ++ integer_to_list(I-1), + [",\n", + "[",H,case Cs of + [] -> []; + [_|_] -> ["|",T] + end,"] = ",Prev,",\n", + Func,"(",H,arg2str(Args),")"|gen_sof_1(Cs, I+1)]; +gen_sof_1([], _) -> + ".\n". + +components([#type{def=Def}|Ts], [V|Vs]) -> + [do_gen(Def, V)|components(Ts, Vs)]; +components([], []) -> []. + +gen_choice(Name, Tag, Func, Args) -> + NameStr = atom_to_list(Name), + [NameStr,"({",term2str(Tag),",Value}) ->\n" + " ",Func,"(Value",arg2str(Args),");\n", + NameStr,"(_) ->\n" + " throw(false).\n"]. + +pre_process_oid(Oid) -> + Reserved = reserved_oid(), + pre_process_oid(tuple_to_list(Oid), Reserved, []). + +pre_process_oid([H|T]=Tail, Res0, Acc) -> + case lists:keyfind(H, 2, Res0) of + false -> + {lists:reverse(Acc),Tail}; + {Names0,H,Res} -> + Names = case is_list(Names0) of + false -> [Names0]; + true -> Names0 + end, + Keys = [H|Names], + pre_process_oid(T, Res, [Keys|Acc]) + end. + +reserved_oid() -> + [{['itu-t',ccitt],0, + [{recommendation,0,[]}, + {question,1,[]}, + {administration,2,[]}, + {'network-operator',3,[]}, + {'identified-organization',4,[]}]}, + {iso,1,[{standard,0,[]}, + {'member-body',2,[]}, + {'identified-organization',3,[]}]}, + {['joint-iso-itu-t','joint-iso-ccitt'],2,[]}]. + +arg2str(Args) -> + [", "++term2str(Arg) || Arg <- Args]. + +term2str(T) -> + io_lib:format("~w", [T]). + +need(F, A) -> + asn1ct_func:need({check,F,A}), + atom_to_list(F). diff --git a/lib/asn1/src/asn1ct_gen_per.erl b/lib/asn1/src/asn1ct_gen_per.erl index 519ce9f054..39cc0536f8 100644 --- a/lib/asn1/src/asn1ct_gen_per.erl +++ b/lib/asn1/src/asn1ct_gen_per.erl @@ -32,6 +32,7 @@ -export([gen_encode/2, gen_encode/3]). -export([gen_dec_external/2]). -export([extaddgroup2sequence/1]). +-export([dialyzer_suppressions/1]). -import(asn1ct_gen, [emit/1,demit/1]). -import(asn1ct_func, [call/3]). @@ -40,6 +41,15 @@ %% Generate ENCODING ****************************** %%****************************************x +dialyzer_suppressions(Erules) -> + case asn1ct_func:is_used({Erules,complete,1}) of + false -> + ok; + true -> + emit([" _ = complete(Arg),",nl]) + end, + emit([" ok.",nl]). + gen_encode(Erules,Type) when is_record(Type,typedef) -> gen_encode_user(Erules,Type). diff --git a/lib/asn1/src/asn1ct_imm.erl b/lib/asn1/src/asn1ct_imm.erl index fde39c674e..bdd14871d1 100644 --- a/lib/asn1/src/asn1ct_imm.erl +++ b/lib/asn1/src/asn1ct_imm.erl @@ -82,15 +82,8 @@ per_dec_enumerated(NamedList0, Aligned) -> Ub = length(NamedList0) - 1, Constraint = [{'ValueRange',{0,Ub}}], Int = per_dec_integer(Constraint, Aligned), - EnumTail = case matched_range(Int) of - {0,Ub} -> - %% The error case can never happen. - []; - _ -> - [enum_error] - end, - NamedList = per_dec_enumerated_fix_list(NamedList0, EnumTail, 0), - {map,Int,NamedList}. + NamedList = per_dec_enumerated_fix_list(NamedList0, [enum_error], 0), + {map,Int,opt_map(NamedList, Int)}. per_dec_enumerated(BaseNamedList, NamedListExt0, Aligned) -> Base = per_dec_enumerated(BaseNamedList, Aligned), @@ -124,7 +117,7 @@ per_dec_length(no, AllowZero, Aligned) -> per_dec_named_integer(Constraint, NamedList0, Aligned) -> Int = per_dec_integer(Constraint, Aligned), NamedList = [{K,V} || {V,K} <- NamedList0] ++ [integer_default], - {map,Int,NamedList}. + {map,Int,opt_map(NamedList, Int)}. per_dec_k_m_string(StringType, Constraint, Aligned) -> SzConstr = effective_constraint(bitstring, Constraint), @@ -175,6 +168,8 @@ per_enc_bit_string(Val0, NNL0, Constraint0, Aligned) -> ToBs = case ExtraArgs of [] -> {call,per_common,bs_drop_trailing_zeroes,[Val]}; + [0] -> + {call,per_common,bs_drop_trailing_zeroes,[Val]}; [Lower] -> {call,per_common,adjust_trailing_zeroes,[Val,Lower]} end, @@ -203,6 +198,7 @@ per_enc_legacy_bit_string(Val0, NNL0, Constraint0, Aligned) -> Constraint = effective_constraint(bitstring, Constraint0), ExtraArgs = case constr_min_size(Constraint) of no -> []; + 0 -> []; Lb -> [Lb] end, B ++ [{'try', @@ -265,10 +261,6 @@ per_enc_k_m_string(Val0, StringType, Constraint, Aligned) -> SzConstraint = effective_constraint(bitstring, Constraint), Unit = string_num_bits(StringType, Constraint, Aligned), Chars0 = char_tab(Constraint, StringType, Unit), - Args = case enc_char_tab(Chars0) of - notab -> [Val,Unit]; - Chars -> [Val,Unit,Chars] - end, Enc = case Unit of 16 -> {call,per_common,encode_chars_16bit,[Val],Bin}; @@ -277,7 +269,15 @@ per_enc_k_m_string(Val0, StringType, Constraint, Aligned) -> 8 -> {call,erlang,list_to_binary,[Val],Bin}; _ -> - {call,per_common,encode_chars,Args,Bin} + case enc_char_tab(Chars0) of + notab -> + {call,per_common,encode_chars,[Val,Unit],Bin}; + {tab,Tab} -> + {call,per_common,encode_chars,[Val,Unit,Tab],Bin}; + {compact_map,Map} -> + {call,per_common,encode_chars_compact_map, + [Val,Unit,Map],Bin} + end end, case Unit of 8 -> @@ -581,14 +581,42 @@ per_num_bits(N) when N =< 64 -> 6; per_num_bits(N) when N =< 128 -> 7; per_num_bits(N) when N =< 255 -> 8. +opt_map(Map, Imm) -> + case matched_range(Imm) of + unknown -> Map; + {Lb,Ub} -> opt_map_1(Map, Lb, Ub) + end. + +opt_map_1([{I,_}=Pair|T], Lb, Ub) -> + if + I =:= Lb, I =< Ub -> + [Pair|opt_map_1(T, Lb+1, Ub)]; + Lb < I, I =< Ub -> + [Pair|opt_map_1(T, Lb, Ub)]; + true -> + opt_map_1(T, Lb, Ub) + end; +opt_map_1(Map, Lb, Ub) -> + if + Lb =< Ub -> + Map; + true -> + [] + end. + matched_range({get_bits,Bits0,[U|Flags]}) when is_integer(U) -> - case lists:member(signed, Flags) of - false -> + case not lists:member(signed, Flags) andalso is_integer(Bits0) of + true -> Bits = U*Bits0, {0,(1 bsl Bits) - 1}; - true -> + false -> unknown end; +matched_range({add,Imm,Add}) -> + case matched_range(Imm) of + unknown -> unknown; + {Lb,Ub} -> {Lb+Add,Ub+Add} + end; matched_range(_Op) -> unknown. string_num_bits(StringType, Constraint, Aligned) -> @@ -1289,6 +1317,8 @@ eval_cond_1({eq,[],[]}) -> true; eval_cond_1({eq,I,N}) when is_integer(I), is_integer(N) -> I =:= N; +eval_cond_1({ge,I,N}) when is_integer(I), is_integer(N) -> + I >= N; eval_cond_1({lt,I,N}) when is_integer(I), is_integer(N) -> I < N; eval_cond_1(_) -> maybe. @@ -1303,9 +1333,15 @@ prepend_to_cond_1([Check|T], Code) -> enc_char_tab(notab) -> notab; enc_char_tab(Tab0) -> - Tab = tuple_to_list(Tab0), - First = hd(Tab), - {First-1,list_to_tuple(enc_char_tab_1(Tab, First, 0))}. + Tab1 = tuple_to_list(Tab0), + First = hd(Tab1), + Tab = enc_char_tab_1(Tab1, First, 0), + case lists:member(ill, Tab) of + false -> + {compact_map,{First,tuple_size(Tab0)}}; + true -> + {tab,{First-1,list_to_tuple(Tab)}} + end. enc_char_tab_1([H|T], H, I) -> [I|enc_char_tab_1(T, H+1, I+1)]; diff --git a/lib/asn1/src/asn1ct_parser2.erl b/lib/asn1/src/asn1ct_parser2.erl index 283616b157..3891fce8d3 100644 --- a/lib/asn1/src/asn1ct_parser2.erl +++ b/lib/asn1/src/asn1ct_parser2.erl @@ -25,7 +25,8 @@ %% Only used internally within this module. -record(typereference, {pos,val}). --record(constraint,{c,e}). +-record(constraint, {c,e}). +-record(identifier, {pos,val}). %% parse all types in module parse(Tokens) -> @@ -112,6 +113,9 @@ parse_ModuleDefinition(Tokens) -> parse_Exports([{'EXPORTS',_L1},{';',_L2}|Rest]) -> {{exports,[]},Rest}; +parse_Exports([{'EXPORTS',_},{'ALL',_},{';',_}|Rest]) -> + %% Same as no exports definition. + {{exports,all},Rest}; parse_Exports([{'EXPORTS',_L1}|Rest]) -> {SymbolList,Rest2} = parse_SymbolList(Rest), case Rest2 of @@ -1037,10 +1041,6 @@ parse_DefinedObjectClass([{typereference,_,_ModName},{'.',_},Tr={typereference,_ parse_DefinedObjectClass([Tr={typereference,_,_ObjClName}|Rest]) -> % {{objectclassname,tref2Exttref(Tr)},Rest}; {tref2Exttref(Tr),Rest}; -parse_DefinedObjectClass([{'TYPE-IDENTIFIER',_}|Rest]) -> - {'TYPE-IDENTIFIER',Rest}; -parse_DefinedObjectClass([{'ABSTRACT-SYNTAX',_}|Rest]) -> - {'ABSTRACT-SYNTAX',Rest}; parse_DefinedObjectClass(Tokens) -> throw({asn1_error,{get_line(hd(Tokens)),get(asn1_module), [got,get_token(hd(Tokens)),expected, @@ -1051,7 +1051,8 @@ parse_DefinedObjectClass(Tokens) -> parse_ObjectClassAssignment([{typereference,L1,ObjClName},{'::=',_}|Rest]) -> {Type,Rest2} = parse_ObjectClass(Rest), - {#classdef{pos=L1,name=ObjClName,typespec=Type},Rest2}; + {#classdef{pos=L1,name=ObjClName,module=resolve_module(Type), + typespec=Type},Rest2}; parse_ObjectClassAssignment(Tokens) -> throw({asn1_assignment_error,{get_line(hd(Tokens)),get(asn1_module), [got,get_token(hd(Tokens)),expected, @@ -2134,8 +2135,7 @@ parse_ParameterizedObjectSetAssignment(Tokens) -> %% Parameter = {Governor,Reference} | Reference %% Governor = Type | DefinedObjectClass %% Type = #type{} -%% DefinedObjectClass = #'Externaltypereference'{} | -%% 'ABSTRACT-SYNTAX' | 'TYPE-IDENTIFIER' +%% DefinedObjectClass = #'Externaltypereference'{} %% Reference = #'Externaltypereference'{} | #'Externalvaluereference'{} parse_ParameterList([{'{',_}|Rest]) -> parse_ParameterList(Rest,[]); @@ -2863,13 +2863,14 @@ parse_SequenceValue(Tokens) -> throw({asn1_error,{get_line(hd(Tokens)),get(asn1_module), [got,get_token(hd(Tokens)),expected,'{']}}). -parse_SequenceValue([{identifier,_,IdName}|Rest],Acc) -> +parse_SequenceValue([{identifier,Pos,IdName}|Rest],Acc) -> {Value,Rest2} = parse_Value(Rest), + SeqTag = #seqtag{pos=Pos,module=get(asn1_module),val=IdName}, case Rest2 of [{',',_}|Rest3] -> - parse_SequenceValue(Rest3,[{IdName,Value}|Acc]); + parse_SequenceValue(Rest3, [{SeqTag,Value}|Acc]); [{'}',_}|Rest3] -> - {lists:reverse([{IdName,Value}|Acc]),Rest3}; + {lists:reverse(Acc, [{SeqTag,Value}]),Rest3}; _ -> throw({asn1_error,{get_line(hd(Rest2)),get(asn1_module), [got,get_token(hd(Rest2)),expected,'}']}}) diff --git a/lib/asn1/src/asn1ct_tok.erl b/lib/asn1/src/asn1ct_tok.erl index 33f4379173..8687ed955c 100644 --- a/lib/asn1/src/asn1ct_tok.erl +++ b/lib/asn1/src/asn1ct_tok.erl @@ -309,7 +309,6 @@ check_hex(_) -> %% returns rstrtype if A is a reserved word in the group %% RestrictedCharacterStringType reserved_word('ABSENT') -> true; -%reserved_word('ABSTRACT-SYNTAX') -> true; % impl as predef item reserved_word('ALL') -> true; reserved_word('ANY') -> true; reserved_word('APPLICATION') -> true; @@ -380,7 +379,6 @@ reserved_word('T61String') -> rstrtype; reserved_word('TAGS') -> true; reserved_word('TeletexString') -> rstrtype; reserved_word('TRUE') -> true; -%% reserved_word('TYPE-IDENTIFIER') -> true; % impl as predef item reserved_word('UNION') -> true; reserved_word('UNIQUE') -> true; reserved_word('UNIVERSAL') -> true; diff --git a/lib/asn1/src/asn1rtt_ber.erl b/lib/asn1/src/asn1rtt_ber.erl index 4bd814769f..c4cd872368 100644 --- a/lib/asn1/src/asn1rtt_ber.erl +++ b/lib/asn1/src/asn1rtt_ber.erl @@ -26,25 +26,24 @@ skip_ExtensionAdditions/2]). -export([encode_boolean/2,decode_boolean/2, encode_integer/2,encode_integer/3, - decode_integer/2,decode_integer/3, - decode_named_integer/3,decode_named_integer/4, - encode_enumerated/2,decode_enumerated/3, + decode_integer/2, + number2name/2, encode_unnamed_bit_string/2,encode_unnamed_bit_string/3, encode_named_bit_string/3,encode_named_bit_string/4, encode_bit_string/4, decode_named_bit_string/3, - decode_compact_bit_string/3, - decode_legacy_bit_string/3, - decode_native_bit_string/3, + decode_compact_bit_string/2,compact_bit_string_size/1, + decode_native_bit_string/2, + native_to_legacy_bit_string/1, encode_null/2,decode_null/2, encode_relative_oid/2,decode_relative_oid/2, encode_object_identifier/2,decode_object_identifier/2, encode_restricted_string/2, - decode_octet_string/2,decode_octet_string/3, - decode_restricted_string/2,decode_restricted_string/3, - encode_universal_string/2,decode_universal_string/3, + decode_octet_string/2, + decode_restricted_string/2, + encode_universal_string/2,decode_universal_string/2, encode_UTF8_string/2,decode_UTF8_string/2, - encode_BMP_string/2,decode_BMP_string/3]). + encode_BMP_string/2,decode_BMP_string/2]). -export([encode_open_type/2,decode_open_type/2, decode_open_type_as_binary/2]). @@ -591,8 +590,6 @@ encode_tags(TagIn, {BytesSoFar,LenSoFar}) -> encode_open_type(Val, T) when is_list(Val) -> encode_open_type(list_to_binary(Val), T); -encode_open_type(Val, []) -> - {Val,byte_size(Val)}; encode_open_type(Val, Tag) -> encode_tags(Tag, Val, byte_size(Val)). @@ -697,41 +694,14 @@ encode_integer_neg(N, Acc) -> %%=============================================================================== %% decode integer -%% (Buffer, Range, HasTag, TotalLen) -> {Integer, Remain, RemovedBytes} -%% (Buffer, Range, NamedNumberList, HasTag, TotalLen) -> {Integer, Remain, RemovedBytes} %%=============================================================================== -decode_named_integer(Tlv, NamedNumberList, TagIn) -> - V = match_tags(Tlv, TagIn), - Int = decode_integer(V), - number2name(Int, NamedNumberList). - -decode_named_integer(Tlv, Range, NamedNumberList, TagIn) -> - V = match_tags(Tlv, TagIn), - Int = range_check_integer(decode_integer(V), Range), - number2name(Int, NamedNumberList). - decode_integer(Tlv, TagIn) -> - V = match_tags(Tlv, TagIn), - decode_integer(V). - -decode_integer(Tlv, Range, TagIn) -> - V = match_tags(Tlv, TagIn), - Int = decode_integer(V), - range_check_integer(Int, Range). - -decode_integer(Bin) -> + Bin = match_tags(Tlv, TagIn), Len = byte_size(Bin), <<Int:Len/signed-unit:8>> = Bin, Int. -range_check_integer(Int, {Lb,Ub}) when Lb =< Int, Int =< Ub -> - Int; -range_check_integer(Int, Range) -> - exit({error,{asn1,{integer_range,Range,Int}}}). - -number2name(Int, []) -> - Int; number2name(Int, NamedNumberList) -> case lists:keyfind(Int, 2, NamedNumberList) of {NamedVal,_} -> @@ -740,49 +710,6 @@ number2name(Int, NamedNumberList) -> Int end. - -%%============================================================================ -%% Enumerated value, ITU_T X.690 Chapter 8.4 - -%% encode enumerated value -%%============================================================================ -encode_enumerated(Val, TagIn) when is_integer(Val) -> - encode_tags(TagIn, encode_integer(Val)). - -%%============================================================================ -%% decode enumerated value -%% (Buffer, Range, NamedNumberList, HasTag, TotalLen) -> Value -%%=========================================================================== -decode_enumerated(Tlv, NamedNumberList, Tags) -> - Buffer = match_tags(Tlv, Tags), - decode_enumerated_notag(Buffer, NamedNumberList, Tags). - -decode_enumerated_notag(Buffer, {NamedNumberList,ExtList}, _Tags) -> - IVal = decode_integer(Buffer), - case decode_enumerated1(IVal, NamedNumberList) of - {asn1_enum,IVal} -> - decode_enumerated1(IVal,ExtList); - EVal -> - EVal - end; -decode_enumerated_notag(Buffer, NNList, _Tags) -> - IVal = decode_integer(Buffer), - case decode_enumerated1(IVal, NNList) of - {asn1_enum,_} -> - exit({error,{asn1, {illegal_enumerated, IVal}}}); - EVal -> - EVal - end. - -decode_enumerated1(Val, NamedNumberList) -> - %% it must be a named integer - case lists:keyfind(Val, 2, NamedNumberList) of - {NamedVal, _} -> - NamedVal; - _ -> - {asn1_enum,Val} - end. - %%============================================================================ %% Bitstring value, ITU_T X.690 Chapter 8.6 %% @@ -794,45 +721,46 @@ encode_unnamed_bit_string(Bits, TagIn) -> Bin = <<Unused,Bits/bitstring,0:Unused>>, encode_tags(TagIn, Bin, byte_size(Bin)). -encode_unnamed_bit_string(C, Bits, TagIn) -> +encode_unnamed_bit_string(MaxBits, Bits, TagIn) -> NumBits = bit_size(Bits), Unused = (8 - (NumBits band 7)) band 7, Bin = <<Unused,Bits/bitstring,0:Unused>>, - case C of - {_Min,Max} -> - if - NumBits > Max -> - exit({error,{asn1, - {bitstring_length, - {{was,NumBits},{maximum,Max}}}}}); - true -> - ok - end; - Size -> - if NumBits =< Size -> - ok; - true -> - exit({error,{asn1, - {bitstring_length, - {{was,NumBits},{should_be,Size}}}}}) - end - end, - encode_tags(TagIn, Bin, byte_size(Bin)). + if + NumBits > MaxBits -> + exit({error,{asn1, + {bitstring_length, + {{was,NumBits},{maximum,MaxBits}}}}}); + true -> + encode_tags(TagIn, Bin, byte_size(Bin)) + end. encode_named_bit_string([H|_]=Bits, NamedBitList, TagIn) when is_atom(H) -> - encode_bit_string_named([], Bits, NamedBitList, TagIn); + do_encode_named_bit_string(Bits, NamedBitList, TagIn); encode_named_bit_string([{bit,_}|_]=Bits, NamedBitList, TagIn) -> - encode_bit_string_named([], Bits, NamedBitList, TagIn); + do_encode_named_bit_string(Bits, NamedBitList, TagIn); encode_named_bit_string(Bits, _NamedBitList, TagIn) when is_bitstring(Bits) -> encode_unnamed_bit_string(Bits, TagIn). encode_named_bit_string(C, [H|_]=Bits, NamedBitList, TagIn) when is_atom(H) -> - encode_bit_string_named(C, Bits, NamedBitList, TagIn); + do_encode_named_bit_string(C, Bits, NamedBitList, TagIn); encode_named_bit_string(C, [{bit,_}|_]=Bits, NamedBitList, TagIn) -> - encode_bit_string_named(C, Bits, NamedBitList, TagIn); + do_encode_named_bit_string(C, Bits, NamedBitList, TagIn); encode_named_bit_string(C, Bits, _NamedBitList, TagIn) when is_bitstring(Bits) -> encode_unnamed_bit_string(C, Bits, TagIn). +do_encode_named_bit_string([FirstVal | RestVal], NamedBitList, TagIn) -> + ToSetPos = get_all_bitposes([FirstVal | RestVal], NamedBitList, []), + Size = lists:max(ToSetPos) + 1, + BitList = make_and_set_list(Size, ToSetPos, 0), + {Len,Unused,OctetList} = encode_bitstring(BitList), + encode_tags(TagIn, [Unused|OctetList],Len+1). + +do_encode_named_bit_string(Size, [FirstVal | RestVal], NamedBitList, TagIn) -> + ToSetPos = get_all_bitposes([FirstVal | RestVal], NamedBitList, []), + BitList = make_and_set_list(Size, ToSetPos, 0), + {Len, Unused, OctetList} = encode_bitstring(BitList), + encode_tags(TagIn, [Unused|OctetList], Len+1). + %%============================================================================ %% Bitstring value, ITU_T X.690 Chapter 8.6 %% @@ -932,15 +860,14 @@ remove_unused_then_dotag(TagIn,Unused,BinBits) -> encode_bit_string_named(C, [FirstVal | RestVal], NamedBitList, TagIn) -> ToSetPos = get_all_bitposes([FirstVal | RestVal], NamedBitList, []), - Size = - case C of - [] -> - lists:max(ToSetPos)+1; - {_Min,Max} -> - Max; - TSize -> - TSize - end, + Size = case C of + [] -> + lists:max(ToSetPos) + 1; + {_Min,Max} -> + Max; + TSize -> + TSize + end, BitList = make_and_set_list(Size, ToSetPos, 0), {Len, Unused, OctetList} = encode_bitstring(BitList), encode_tags(TagIn, [Unused|OctetList],Len+1). @@ -1098,33 +1025,23 @@ unused_bitlist([Bit | Rest], Trail, Ack) -> %% decode bitstring value %%============================================================================ -decode_compact_bit_string(Buffer, Range, Tags) -> +decode_compact_bit_string(Buffer, Tags) -> case match_and_collect(Buffer, Tags) of - <<0>> -> - check_restricted_string({0,<<>>}, 0, Range); - <<Unused,Bits/binary>> -> - Val = {Unused,Bits}, - Len = bit_size(Bits) - Unused, - check_restricted_string(Val, Len, Range) + <<0>> -> {0,<<>>}; + <<Unused,Bits/binary>> -> {Unused,Bits} end. -decode_legacy_bit_string(Buffer, Range, Tags) -> - Val = case match_and_collect(Buffer, Tags) of - <<0>> -> - []; - <<Unused,Bits/binary>> -> - decode_bitstring2(byte_size(Bits), Unused, Bits) - end, - check_restricted_string(Val, length(Val), Range). +compact_bit_string_size({Unused,Bits}) -> + bit_size(Bits) - Unused. -decode_native_bit_string(Buffer, Range, Tags) -> +decode_native_bit_string(Buffer, Tags) -> case match_and_collect(Buffer, Tags) of <<0>> -> - check_restricted_string(<<>>, 0, Range); + <<>>; <<Unused,Bits/binary>> -> Size = bit_size(Bits) - Unused, <<Val:Size/bitstring,_:Unused/bitstring>> = Bits, - check_restricted_string(Val, Size, Range) + Val end. decode_named_bit_string(Buffer, NamedNumberList, Tags) -> @@ -1147,6 +1064,9 @@ decode_bitstring2(Len, Unused, [B7,B6,B5,B4,B3,B2,B1,B0| decode_bitstring2(Len - 1, Unused, Buffer)]. +native_to_legacy_bit_string(Bits) -> + [B || <<B:1>> <= Bits]. + %%---------------------------------------- %% Decode the bitlist to names %%---------------------------------------- @@ -1310,31 +1230,12 @@ decode_octet_string(Tlv, TagsIn) -> Bin = match_and_collect(Tlv, TagsIn), binary:copy(Bin). -decode_octet_string(Tlv, Range, TagsIn) -> - Bin0 = match_and_collect(Tlv, TagsIn), - Bin = binary:copy(Bin0), - check_restricted_string(Bin, byte_size(Bin), Range). - %%============================================================================ %% decode Numeric Printable Teletex Videotex Visible IA5 Graphic General strings %%============================================================================ decode_restricted_string(Tlv, TagsIn) -> - Bin = match_and_collect(Tlv, TagsIn), - binary_to_list(Bin). - -decode_restricted_string(Tlv, Range, TagsIn) -> - Bin = match_and_collect(Tlv, TagsIn), - check_restricted_string(binary_to_list(Bin), byte_size(Bin), Range). - -check_restricted_string(Val, _Len, []) -> - Val; -check_restricted_string(Val, Len, {Lb,Ub}) when Lb =< Len, Len =< Ub -> - Val; -check_restricted_string(Val, Len, Len) -> - Val; -check_restricted_string(Val, _Len, Range) -> - exit({error,{asn1,{length,Range,Val}}}). + match_and_collect(Tlv, TagsIn). %%============================================================================ %% encode Universal string @@ -1360,10 +1261,9 @@ mk_uni_list([H|T],List) -> %% {String, Remain, RemovedBytes} %%=========================================================================== -decode_universal_string(Buffer, Range, Tags) -> +decode_universal_string(Buffer, Tags) -> Bin = match_and_collect(Buffer, Tags), - Val = mk_universal_string(binary_to_list(Bin)), - check_restricted_string(Val, length(Val), Range). + mk_universal_string(binary_to_list(Bin)). mk_universal_string(In) -> mk_universal_string(In, []). @@ -1423,10 +1323,9 @@ mk_BMP_list([H|T], List) -> %% (Buffer, Range, StringType, HasTag, TotalLen) -> %% {String, Remain, RemovedBytes} %%============================================================================ -decode_BMP_string(Buffer, Range, Tags) -> +decode_BMP_string(Buffer, Tags) -> Bin = match_and_collect(Buffer, Tags), - Val = mk_BMP_string(binary_to_list(Bin)), - check_restricted_string(Val, length(Val), Range). + mk_BMP_string(binary_to_list(Bin)). mk_BMP_string(In) -> mk_BMP_string(In,[]). diff --git a/lib/asn1/src/asn1rtt_check.erl b/lib/asn1/src/asn1rtt_check.erl index be4f9c8bff..0083867a10 100644 --- a/lib/asn1/src/asn1rtt_check.erl +++ b/lib/asn1/src/asn1rtt_check.erl @@ -18,43 +18,41 @@ %% -module(asn1rtt_check). --export([check_bool/2, +-export([check_fail/1, check_int/3, - check_bitstring/2,check_named_bitstring/3, + check_legacy_bitstring/2, + check_legacy_named_bitstring/3, + check_legacy_named_bitstring/4, + check_named_bitstring/3, + check_named_bitstring/4, + check_literal_sof/2, check_octetstring/2, - check_null/2, check_objectidentifier/2, check_objectdescriptor/2, check_real/2, - check_enum/3, check_restrictedstring/2]). -check_bool(_Bool, asn1_DEFAULT) -> - true; -check_bool(Bool, Bool) when is_boolean(Bool) -> - true; -check_bool(_Bool1, Bool2) -> - throw({error,Bool2}). +check_fail(_) -> + throw(false). -check_int(_, asn1_DEFAULT, _) -> - true; check_int(Value, Value, _) when is_integer(Value) -> true; -check_int(DefValue, Value, NNL) when is_atom(Value) -> +check_int(Value, DefValue, NNL) when is_atom(Value) -> case lists:keyfind(Value, 1, NNL) of {_,DefValue} -> true; _ -> - throw({error,DefValue}) + throw(false) end; -check_int(DefaultValue, _Value, _) -> - throw({error,DefaultValue}). +check_int(_, _, _) -> + throw(false). + +check_legacy_bitstring(Value, Default) -> + check_bitstring(Default, Value). %% check_bitstring(Default, UserBitstring) -> true|false %% Default = bitstring() %% UserBitstring = integeger() | list(0|1) | {Unused,binary()} | bitstring() -check_bitstring(_, asn1_DEFAULT) -> - true; check_bitstring(DefVal, {Unused,Binary}) -> %% User value in compact format. Sz = bit_size(Binary) - Unused, @@ -62,7 +60,7 @@ check_bitstring(DefVal, {Unused,Binary}) -> check_bitstring(DefVal, Val); check_bitstring(DefVal, Val) when is_bitstring(Val) -> case Val =:= DefVal of - false -> throw(error); + false -> throw(false); true -> true end; check_bitstring(Def, Val) when is_list(Val) -> @@ -75,178 +73,95 @@ check_bitstring_list(<<H:1,T1/bitstring>>, [H|T2]) -> check_bitstring_list(<<>>, []) -> true; check_bitstring_list(_, _) -> - throw(error). + throw(false). check_bitstring_integer(<<H:1,T1/bitstring>>, Int) when H =:= Int band 1 -> check_bitstring_integer(T1, Int bsr 1); check_bitstring_integer(<<>>, 0) -> true; check_bitstring_integer(_, _) -> - throw(error). - -check_named_bitstring(_, asn1_DEFAULT, _) -> - true; -check_named_bitstring(V, V, _) -> - true; -%% Default value and user value as lists of ones and zeros -check_named_bitstring(L1=[H1|_T1], L2=[H2|_T2], NBL=[_H|_T]) when is_integer(H1), is_integer(H2) -> - L2new = remove_trailing_zeros(L2), - check_named_bitstring(L1, L2new, NBL); -%% Default value as a list of 1 and 0 and user value as a list of atoms -check_named_bitstring(L1=[H1|_T1], L2=[H2|_T2], NBL) when is_integer(H1), is_atom(H2) -> - L3 = bit_list_to_nbl(L1, NBL, 0, []), - check_named_bitstring(L3, L2, NBL); -%% Both default value and user value as a list of atoms -check_named_bitstring(L1=[H1|T1], L2=[H2|_T2], _) - when is_atom(H1), is_atom(H2), length(L1) =:= length(L2) -> - case lists:member(H1, L2) of - true -> - check_bitstring1(T1, L2); - false -> throw({error,L2}) - end; -%% Default value as a list of atoms and user value as a list of 1 and 0 -check_named_bitstring(L1=[H1|_T1], L2=[H2|_T2], NBL) when is_atom(H1), is_integer(H2) -> - L3 = bit_list_to_nbl(L2, NBL, 0, []), - check_named_bitstring(L1, L3, NBL); -%% User value in compact format -check_named_bitstring(DefVal,CBS={_,_}, NBL) -> - NewVal = cbs_to_bit_list(CBS), - check_named_bitstring(DefVal, NewVal, NBL); -%% User value as a binary -check_named_bitstring(DefVal, CBS, NBL) when is_binary(CBS) -> - NewVal = cbs_to_bit_list({0,CBS}), - check_named_bitstring(DefVal, NewVal, NBL); -%% User value as a bitstring -check_named_bitstring(DefVal, CBS, NBL) when is_bitstring(CBS) -> - BitSize = bit_size(CBS), - Unused = 8 - (BitSize band 7), - NewVal = cbs_to_bit_list({Unused,<<CBS:BitSize/bits,0:Unused>>}), - check_named_bitstring(DefVal, NewVal, NBL); -check_named_bitstring(DV, V, _) -> - throw({error,DV,V}). - -int_to_bit_list(0, Acc, 0) -> - Acc; -int_to_bit_list(Int, Acc, Len) when Len > 0 -> - int_to_bit_list(Int bsr 1, [Int band 1|Acc], Len - 1). - -bit_list_to_nbl([0|T], NBL, Pos, Acc) -> - bit_list_to_nbl(T, NBL, Pos+1, Acc); -bit_list_to_nbl([1|T], NBL, Pos, Acc) -> - case lists:keyfind(Pos, 2, NBL) of - {N,_} -> - bit_list_to_nbl(T, NBL, Pos+1, [N|Acc]); + throw(false). + +check_legacy_named_bitstring([Int|_]=Val, Bs, BsSize) when is_integer(Int) -> + check_named_bitstring(<< <<B:1>> || B <- Val >>, Bs, BsSize); +check_legacy_named_bitstring({Unused,Val0}, Bs, BsSize) -> + Sz = bit_size(Val0) - Unused, + <<Val:Sz/bits,_/bits>> = Val0, + check_named_bitstring(Val, Bs, BsSize); +check_legacy_named_bitstring(Val, Bs, BsSize) when is_integer(Val) -> + L = legacy_int_to_bitlist(Val), + check_named_bitstring(<< <<B:1>> || B <- L >>, Bs, BsSize); +check_legacy_named_bitstring(Val, Bs, BsSize) -> + check_named_bitstring(Val, Bs, BsSize). + +check_legacy_named_bitstring([Int|_]=Val, Names, Bs, BsSize) when is_integer(Int) -> + check_named_bitstring(<< <<B:1>> || B <- Val >>, Names, Bs, BsSize); +check_legacy_named_bitstring({Unused,Val0}, Names, Bs, BsSize) -> + Sz = bit_size(Val0) - Unused, + <<Val:Sz/bits,_/bits>> = Val0, + check_named_bitstring(Val, Names, Bs, BsSize); +check_legacy_named_bitstring(Val, Names, Bs, BsSize) when is_integer(Val) -> + L = legacy_int_to_bitlist(Val), + check_named_bitstring(<< <<B:1>> || B <- L >>, Names, Bs, BsSize); +check_legacy_named_bitstring(Val, Names, Bs, BsSize) -> + check_named_bitstring(Val, Names, Bs, BsSize). + +legacy_int_to_bitlist(0) -> + []; +legacy_int_to_bitlist(Int) -> + [Int band 1|legacy_int_to_bitlist(Int bsr 1)]. + +check_named_bitstring(Bs, Bs, _) -> + true; +check_named_bitstring(Val, Bs, BsSize) -> + Rest = bit_size(Val) - BsSize, + case Val of + <<Bs:BsSize/bits,0:Rest>> -> + true; _ -> - throw({error,{no,named,element,at,pos,Pos}}) - end; -bit_list_to_nbl([], _, _, Acc) -> - Acc. - -remove_trailing_zeros(L2) -> - remove_trailing_zeros1(lists:reverse(L2)). -remove_trailing_zeros1(L) -> - lists:reverse(lists:dropwhile(fun(0)->true; - (_) ->false - end, - L)). - -check_bitstring1([H|T], NBL) -> - case lists:member(H, NBL) of - true -> check_bitstring1(T, NBL); - V -> throw({error,V}) - end; -check_bitstring1([], _) -> - true. - -cbs_to_bit_list({Unused, <<B7:1,B6:1,B5:1,B4:1,B3:1,B2:1,B1:1,B0:1,Rest/binary>>}) when byte_size(Rest) >= 1 -> - [B7,B6,B5,B4,B3,B2,B1,B0|cbs_to_bit_list({Unused,Rest})]; -cbs_to_bit_list({0,<<B7:1,B6:1,B5:1,B4:1,B3:1,B2:1,B1:1,B0:1>>}) -> - [B7,B6,B5,B4,B3,B2,B1,B0]; -cbs_to_bit_list({Unused,Bin}) when byte_size(Bin) =:= 1 -> - Used = 8-Unused, - <<Int:Used,_:Unused>> = Bin, - int_to_bit_list(Int, [], Used). - + throw(false) + end. -check_octetstring(_, asn1_DEFAULT) -> - true; -check_octetstring(L, L) -> - true; -check_octetstring(L, Int) when is_list(L), is_integer(Int) -> - case integer_to_octetlist(Int) of - L -> true; - V -> throw({error,V}) +check_named_bitstring([_|_]=Val, Names, _, _) -> + case lists:sort(Val) of + Names -> true; + _ -> throw(false) end; -check_octetstring(_, V) -> - throw({error,V}). - -integer_to_octetlist(Int) -> - integer_to_octetlist(Int, []). -integer_to_octetlist(0, Acc) -> - Acc; -integer_to_octetlist(Int, Acc) -> - integer_to_octetlist(Int bsr 8, [(Int band 255)|Acc]). - -check_null(_, asn1_DEFAULT) -> +check_named_bitstring(Bs, _, Bs, _) -> true; -check_null('NULL', 'NULL') -> - true; -check_null(_, V) -> - throw({error,V}). +check_named_bitstring(Val, _, Bs, BsSize) -> + Rest = bit_size(Val) - BsSize, + case Val of + <<Bs:BsSize/bits,0:Rest>> -> + true; + _ -> + throw(false) + end. -check_objectidentifier(_, asn1_DEFAULT) -> - true; -check_objectidentifier(OI, OI) -> +check_octetstring(V, V) -> true; -check_objectidentifier(DOI, OI) when is_tuple(DOI), is_tuple(OI) -> - check_objectidentifier1(tuple_to_list(DOI), tuple_to_list(OI)); -check_objectidentifier(_, OI) -> - throw({error,OI}). - -check_objectidentifier1([V|Rest1], [V|Rest2]) -> - check_objectidentifier1(Rest1, Rest2, V); -check_objectidentifier1([V1|Rest1], [V2|Rest2]) -> - case reserved_objectid(V2, []) of - V1 -> - check_objectidentifier1(Rest1, Rest2, [V1]); - V -> - throw({error,V}) - end. -check_objectidentifier1([V|Rest1], [V|Rest2], Above) -> - check_objectidentifier1(Rest1, Rest2, [V|Above]); -check_objectidentifier1([V1|Rest1], [V2|Rest2], Above) -> - case reserved_objectid(V2, Above) of - V1 -> - check_objectidentifier1(Rest1, Rest2, [V1|Above]); - V -> - throw({error,V}) +check_octetstring(V, Def) when is_list(V) -> + case list_to_binary(V) of + Def -> true; + _ -> throw(false) + end; +check_octetstring(_, _) -> + throw(false). + +check_objectidentifier(Value, {Prefix,Tail}) when is_tuple(Value) -> + check_oid(tuple_to_list(Value), Prefix, Tail); +check_objectidentifier(_, _) -> + throw(false). + +check_oid([H|T], [K|Ks], Tail) -> + case lists:member(H, K) of + false -> throw(false); + true -> check_oid(T, Ks, Tail) end; -check_objectidentifier1([], [], _) -> +check_oid(Tail, [], Tail) -> true; -check_objectidentifier1(_, V, _) -> - throw({error,object,identifier,V}). - -%% ITU-T Rec. X.680 Annex B - D -reserved_objectid('itu-t', []) -> 0; -reserved_objectid('ccitt', []) -> 0; -%% arcs below "itu-t" -reserved_objectid('recommendation', [0]) -> 0; -reserved_objectid('question', [0]) -> 1; -reserved_objectid('administration', [0]) -> 2; -reserved_objectid('network-operator', [0]) -> 3; -reserved_objectid('identified-organization', [0]) -> 4; - -reserved_objectid(iso, []) -> 1; -%% arcs below "iso", note that number 1 is not used -reserved_objectid('standard', [1]) -> 0; -reserved_objectid('member-body', [1]) -> 2; -reserved_objectid('identified-organization', [1]) -> 3; - -reserved_objectid('joint-iso-itu-t', []) -> 2; -reserved_objectid('joint-iso-ccitt', []) -> 2; - -reserved_objectid(_, _) -> false. - +check_oid(_, _, _) -> + throw(false). check_objectdescriptor(_, asn1_DEFAULT) -> true; @@ -262,21 +177,6 @@ check_real(R, R) -> check_real(_, _) -> throw({error,{not_implemented_yet,check_real}}). -check_enum(_, asn1_DEFAULT, _) -> - true; -check_enum(Val, Val, _) -> - true; -check_enum(Int, Atom, Enumerations) when is_integer(Int), is_atom(Atom) -> - case lists:keyfind(Atom, 1, Enumerations) of - {_,Int} -> true; - _ -> throw({error,{enumerated,Int,Atom}}) - end; -check_enum(DefVal, Val, _) -> - throw({error,{enumerated,DefVal,Val}}). - - -check_restrictedstring(_, asn1_DEFAULT) -> - true; check_restrictedstring(Val, Val) -> true; check_restrictedstring([V|Rest1], [V|Rest2]) -> @@ -295,7 +195,15 @@ check_restrictedstring({V1,V2,V3,V4}, [V1,V2,V3,V4]) -> check_restrictedstring([V1,V2,V3,V4], {V1,V2,V3,V4}) -> true; %% character string list -check_restrictedstring(V1, V2) when is_list(V1), is_tuple(V2) -> - check_restrictedstring(V1, tuple_to_list(V2)); -check_restrictedstring(V1, V2) -> - throw({error,{restricted,string,V1,V2}}). +check_restrictedstring(V1, V2) when is_tuple(V1) -> + check_restrictedstring(tuple_to_list(V1), V2); +check_restrictedstring(_, _) -> + throw(false). + +check_literal_sof(Value, Default) -> + case lists:sort(Value) of + Default -> + true; + _ -> + throw(false) + end. diff --git a/lib/asn1/src/asn1rtt_per_common.erl b/lib/asn1/src/asn1rtt_per_common.erl index 71fec411a0..0290c75a28 100644 --- a/lib/asn1/src/asn1rtt_per_common.erl +++ b/lib/asn1/src/asn1rtt_per_common.erl @@ -30,6 +30,7 @@ decode_big_chars/2, decode_oid/1,decode_relative_oid/1, encode_chars/2,encode_chars/3, + encode_chars_compact_map/3, encode_chars_16bit/1,encode_big_chars/1, encode_fragmented/2, encode_oid/1,encode_relative_oid/1, @@ -108,6 +109,9 @@ encode_chars(Val, NumBits) -> encode_chars(Val, NumBits, {Lb,Tab}) -> << <<(enc_char(C, Lb, Tab)):NumBits>> || C <- Val >>. +encode_chars_compact_map(Val, NumBits, {Lb,Limit}) -> + << <<(enc_char_cm(C, Lb, Limit)):NumBits>> || C <- Val >>. + encode_chars_16bit(Val) -> L = [case C of {0,0,A,B} -> [A,B]; @@ -383,6 +387,15 @@ enc_char(C0, Lb, Tab) -> illegal_char_error() end. +enc_char_cm(C0, Lb, Limit) -> + C = C0 - Lb, + if + 0 =< C, C < Limit -> + C; + true -> + illegal_char_error() + end. + illegal_char_error() -> error({error,{asn1,"value forbidden by FROM constraint"}}). diff --git a/lib/asn1/test/asn1_SUITE.erl b/lib/asn1/test/asn1_SUITE.erl index 782217ed2d..888339a4d2 100644 --- a/lib/asn1/test/asn1_SUITE.erl +++ b/lib/asn1/test/asn1_SUITE.erl @@ -50,13 +50,14 @@ all() -> {group, performance}]. groups() -> - [{compile, parallel([]), + Parallel = asn1_test_lib:parallel(), + [{compile, Parallel, [c_syntax, c_string, c_implicit_before_choice, constraint_equivalence]}, - {ber, parallel([]), + {ber, Parallel, [ber_choiceinseq, % Uses 'SOpttest' ber_optional]}, @@ -65,7 +66,7 @@ groups() -> {appup_test, [], [{asn1_appup_test, all}]}, - {parallel, parallel([]), + {parallel, Parallel, [cover, xref, {group, ber}, @@ -133,14 +134,13 @@ groups() -> testChoiceIndefinite, per_open_type, testInfObjectClass, - testParameterizedInfObj, + testParam, testFragmented, testMergeCompile, testobj, testDeepTConstr, testExport, testImport, - testParamBasic, testDER, testDEFAULT, testMvrasn6, @@ -173,13 +173,6 @@ groups() -> testTimer_per, testTimer_uper]}]. -parallel(Options) -> - case erlang:system_info(smp_support) andalso - erlang:system_info(schedulers) > 1 of - true -> [parallel|Options]; - false -> Options - end. - %%------------------------------------------------------------------------------ %% Init/end %%------------------------------------------------------------------------------ @@ -428,14 +421,12 @@ testMultipleLevels(Config, Rule, Opts) -> asn1_test_lib:compile("MultipleLevels", Config, [Rule|Opts]), testMultipleLevels:main(Rule). -testDef(Config) -> test(Config, fun testDef/3). -testDef(Config, Rule, Opts) -> - asn1_test_lib:compile("Def", Config, [Rule|Opts]), - testDef:main(Rule). - testDEFAULT(Config) -> test(Config, fun testDEFAULT/3, [ber,{ber,[der]},per,uper]). testDEFAULT(Config, Rule, Opts) -> + asn1_test_lib:compile_all(["Def","Default"], Config, [Rule|Opts]), + testDef:main(Rule), + testSeqSetDefaultVal:main(Rule, Opts), asn1_test_lib:compile_all(["Def","Default"], Config, [legacy_erlang_types,Rule|Opts]), testDef:main(Rule), @@ -528,12 +519,6 @@ testSetDefault(Config, Rule, Opts) -> asn1_test_lib:compile("SetDefault", Config, [Rule|Opts]), testSetDefault:main(Rule). -testParamBasic(Config) -> - test(Config, fun testParamBasic/3, [ber,{ber,[der]},per,uper]). -testParamBasic(Config, Rule, Opts) -> - asn1_test_lib:compile("ParamBasic", Config, [Rule|Opts]), - testParamBasic:main(Rule). - testSetOptional(Config) -> test(Config, fun testSetOptional/3). testSetOptional(Config, Rule, Opts) -> asn1_test_lib:compile("SetOptional", Config, [Rule|Opts]), @@ -766,11 +751,12 @@ testInfObjectClass(Config, Rule, Opts) -> testInfObjectClass:main(Rule), testInfObj:main(Rule). -testParameterizedInfObj(Config) -> - test(Config, fun testParameterizedInfObj/3). -testParameterizedInfObj(Config, Rule, Opts) -> - Files = ["Param","Param2"], +testParam(Config) -> + test(Config, fun testParam/3, [ber,{ber,[der]},per,uper]). +testParam(Config, Rule, Opts) -> + Files = ["ParamBasic","Param","Param2"], asn1_test_lib:compile_all(Files, Config, [Rule|Opts]), + testParamBasic:main(Rule), testParameterizedInfObj:main(Config, Rule), asn1_test_lib:compile("Param", Config, [legacy_erlang_types,Rule|Opts]), diff --git a/lib/asn1/test/asn1_SUITE_data/Constraints.py b/lib/asn1/test/asn1_SUITE_data/Constraints.py index c3b3aebd6d..3495cd841b 100644 --- a/lib/asn1/test/asn1_SUITE_data/Constraints.py +++ b/lib/asn1/test/asn1_SUITE_data/Constraints.py @@ -16,6 +16,7 @@ SemiConstrained ::= INTEGER (100..MAX) NegSemiConstrained ::= INTEGER (-128..MAX) SemiConstrainedExt ::= INTEGER (42..MAX, ...) NegSemiConstrainedExt ::= INTEGER (-128..MAX, ...) +SemiNamed ::= INTEGER {a(100), b(200)} (100..MAX) -- Extensions -- LongLongExt ::= INTEGER (0..18446744073709551615, ..., -5000..-1) Range256to65536Ext ::= INTEGER (256..65536, ..., 1000000..9000000) @@ -65,10 +66,12 @@ Wednesday ::= Day(wednesday) Thing ::= INTEGER {fred (0),fred2 (1),fred3 (2)} - - AnotherThing ::= Thing (fred | fred2) +OneMoreThing ::= INTEGER {wilma(0), fred(1), betty(3), barney(2)} +OneMoreThing-1 ::= OneMoreThing (wilma | fred) +OneMoreThing-2 ::= OneMoreThing (fred | barney) + I ::= INTEGER (0|15..269) -- OTP-5457 X1 ::= INTEGER (1..4 | 8 | 10 | 20) -- OTP-9946 diff --git a/lib/asn1/test/asn1_SUITE_data/Default.asn b/lib/asn1/test/asn1_SUITE_data/Default.asn index 168ce50bb2..b91660381a 100644 --- a/lib/asn1/test/asn1_SUITE_data/Default.asn +++ b/lib/asn1/test/asn1_SUITE_data/Default.asn @@ -25,6 +25,10 @@ SeqBS ::= SEQUENCE { e BIT STRING DEFAULT '01011010'B } +SeqBS2 ::= SEQUENCE { + bs BIT STRING {a(0), z(25)} DEFAULT '101'B +} + SetBS ::= SET { a BIT STRING DEFAULT '1010110'B, b BIT STRING DEFAULT 'A8A'H, @@ -156,4 +160,23 @@ four INTEGER ::= 4 cr IA5String ::= {0,13} +SeqNamedInts ::= SEQUENCE { + i1 INTEGER {first(0), last(31)} DEFAULT 15, + i2 INTEGER {first(0), last(31)} DEFAULT 31 +} + +S5 ::= SEQUENCE { + s3 S3 DEFAULT {}, + so SEQUENCE OF OBJECT IDENTIFIER DEFAULT { + {itu-t question 999}, + {itu-t question 555} + }, + soe SEQUENCE OF OBJECT IDENTIFIER DEFAULT { } +} + +SOI ::= SEQUENCE { + soi SEQUENCE OF OBJECT IDENTIFIER + DEFAULT { {iso member-body f(250) 9 55}, {iso member-body f(250) 3 4} } +} + END diff --git a/lib/asn1/test/asn1_SUITE_data/InfObj.asn b/lib/asn1/test/asn1_SUITE_data/InfObj.asn index 880e81c3b1..719119f418 100644 --- a/lib/asn1/test/asn1_SUITE_data/InfObj.asn +++ b/lib/asn1/test/asn1_SUITE_data/InfObj.asn @@ -255,6 +255,51 @@ Multiple-Optionals ::= SEQUENCE { t3 [2] MULTIPLE-OPTIONALS.&T3 ({Multiple-Optionals-Set}{@id}) OPTIONAL } +-- Test different ways of constructing object sets. + +OBJECT-SET-TEST ::= CLASS { + &id INTEGER UNIQUE, + &Type +} WITH SYNTAX { + &id IS &Type +} + +ObjectSetTest{OBJECT-SET-TEST:ObjectSet} ::= + SEQUENCE { + id OBJECT-SET-TEST.&id ({ObjectSet}), + type OBJECT-SET-TEST.&Type ({ObjectSet}{@id}) + } + +ost1 OBJECT-SET-TEST ::= { 1 IS BIT STRING } +ost2 OBJECT-SET-TEST ::= { 2 IS OCTET STRING } +ost3 OBJECT-SET-TEST ::= { 3 IS ENUMERATED {donald,scrooge} } +ost4 OBJECT-SET-TEST ::= { 4 IS BOOLEAN } +ost5 OBJECT-SET-TEST ::= { 5 IS INTEGER (0..15) } + +Ost12 OBJECT-SET-TEST ::= { ost1 | ost2 } +Ost123 OBJECT-SET-TEST ::= { ost3 | Ost12 } +Ost1234 OBJECT-SET-TEST ::= { Ost123 | ost4 } +Ost45 OBJECT-SET-TEST ::= { ost4 | ost5 } +Ost12345 OBJECT-SET-TEST ::= { Ost123 | Ost45 } + +OstSeq12 ::= ObjectSetTest{ {Ost12} } +OstSeq123 ::= ObjectSetTest{ {Ost123} } +OstSeq1234 ::= ObjectSetTest{ {Ost1234} } +OstSeq45 ::= ObjectSetTest{ {Ost45} } +OstSeq12345 ::= ObjectSetTest{ {Ost12345} } + +ExOst12 OBJECT-SET-TEST ::= { ost1, ..., ost2 } +ExOst123 OBJECT-SET-TEST ::= { ost3, ..., ExOst12 } +--ExOst1234 OBJECT-SET-TEST ::= { ExOst123, ..., ost4 } +ExOst45 OBJECT-SET-TEST ::= { ost4, ..., ost5 } +ExOst12345 OBJECT-SET-TEST ::= { ExOst123, ..., ExOst45 } + +ExOstSeq12 ::= ObjectSetTest{ {ExOst12} } +ExOstSeq123 ::= ObjectSetTest{ {ExOst123} } +--ExOstSeq1234 ::= ObjectSetTest{ {ExOst1234} } +ExOstSeq45 ::= ObjectSetTest{ {ExOst45} } +ExOstSeq12345 ::= ObjectSetTest{ {ExOst12345} } + END diff --git a/lib/asn1/test/asn1_SUITE_data/ParamBasic.asn1 b/lib/asn1/test/asn1_SUITE_data/ParamBasic.asn1 index 491bdf8956..68fc782f33 100644 --- a/lib/asn1/test/asn1_SUITE_data/ParamBasic.asn1 +++ b/lib/asn1/test/asn1_SUITE_data/ParamBasic.asn1 @@ -20,4 +20,26 @@ T21 ::= General2{PrintableString} T22 ::= General2{BIT STRING} + +-- +-- Test a class parameter that is the governor for another parameter. +-- + +AlgorithmIdentifier{ALGORITHM-TYPE, ALGORITHM-TYPE:AlgorithmSet} ::= + SEQUENCE { + algorithm ALGORITHM-TYPE.&id ({AlgorithmSet}), + type ALGORITHM-TYPE.&Type ({AlgorithmSet}{@algorithm}) + } + +AnAlgorithm ::= AlgorithmIdentifier{ SIGNATURE-ALGORITHM, + { {KEY 1 CONTAINING INTEGER} | + {KEY 2 CONTAINING BOOLEAN} } } + +SIGNATURE-ALGORITHM ::= CLASS { + &id INTEGER UNIQUE, + &Type +} WITH SYNTAX { + KEY &id CONTAINING &Type +} + END diff --git a/lib/asn1/test/asn1_SUITE_data/Set.py b/lib/asn1/test/asn1_SUITE_data/Set.py index 4062f6b804..3928004e6b 100644 --- a/lib/asn1/test/asn1_SUITE_data/Set.py +++ b/lib/asn1/test/asn1_SUITE_data/Set.py @@ -80,8 +80,8 @@ SetOpt3 ::= SET SetIn ::= SET { - boolIn BOOLEAN, - intIn INTEGER + boolIn BOOLEAN OPTIONAL, + intIn INTEGER OPTIONAL } diff --git a/lib/asn1/test/asn1_app_test.erl b/lib/asn1/test/asn1_app_test.erl index 1225e36778..d800846b3f 100644 --- a/lib/asn1/test/asn1_app_test.erl +++ b/lib/asn1/test/asn1_app_test.erl @@ -134,13 +134,13 @@ get_ebin_mods(App) -> check_asn1ct_modules(Extra) -> ASN1CTMods = [asn1ct,asn1ct_check,asn1_db,asn1ct_pretty_format, - asn1ct_gen,asn1ct_gen_per,asn1ct_gen_per_rt2ct, + asn1ct_gen,asn1ct_gen_check,asn1ct_gen_per, asn1ct_name,asn1ct_constructed_per,asn1ct_constructed_ber, asn1ct_gen_ber,asn1ct_constructed_ber_bin_v2, asn1ct_gen_ber_bin_v2,asn1ct_value, asn1ct_tok,asn1ct_parser2,asn1ct_table, asn1ct_imm,asn1ct_func,asn1ct_rtt, - asn1ct_eval_ext,asn1ct_eval_per,asn1ct_eval_uper], + asn1ct_eval_ext], case Extra -- ASN1CTMods of [] -> ok; diff --git a/lib/asn1/test/asn1_test_lib.erl b/lib/asn1/test/asn1_test_lib.erl index 417380159e..da07cd1118 100644 --- a/lib/asn1/test/asn1_test_lib.erl +++ b/lib/asn1/test/asn1_test_lib.erl @@ -21,19 +21,72 @@ -export([compile/3,compile_all/3,compile_erlang/3, hex_to_bin/1, + parallel/0, roundtrip/3,roundtrip/4,roundtrip_enc/3,roundtrip_enc/4]). -include_lib("test_server/include/test_server.hrl"). +run_dialyzer() -> + false. + compile(File, Config, Options) -> compile_all([File], Config, Options). compile_all(Files, Config, Options) -> DataDir = ?config(data_dir, Config), CaseDir = ?config(case_dir, Config), - [compile_file(filename:join(DataDir, F), [{outdir, CaseDir}|Options]) + [compile_file(filename:join(DataDir, F), [{outdir, CaseDir}, + debug_info|Options]) || F <- Files], + dialyze(Files, Options), ok. +parallel() -> + case erlang:system_info(schedulers) > 1 andalso not run_dialyzer() of + true -> [parallel]; + false -> [] + end. + +dialyze(Files, Options) -> + case not run_dialyzer() orelse lists:member(abs, Options) of + true -> ok; + false -> dialyze(Files) + end. + +dialyze(Files) -> + Beams0 = [code:which(module(F)) || F <- Files], + Beams = [code:which(asn1rt_nif)|Beams0], + case dialyzer:run([{files,Beams}, + {warnings,[no_improper_lists]}, + {get_warnings,true}]) of + [] -> + ok; + [_|_]=Ws -> + io:put_chars([[B,$\n] || B <- Beams]), + io:put_chars([dialyzer:format_warning(W) || W <- Ws]), + error(dialyzer_warnings) + end. + +module(F0) -> + F1 = filename:basename(F0), + F2 = case filename:extension(F1) of + ".asn" -> + filename:rootname(F1); + ".asn1" -> + filename:rootname(F1); + ".py" -> + filename:rootname(F1); + "" -> + F1 + end, + F = case filename:extension(F2) of + ".set" -> + filename:rootname(F2); + "" -> + F2 + end, + list_to_atom(F). +%% filename:join(CaseDir, F ++ ".beam"). + compile_file(File, Options) -> try ok = asn1ct:compile(File, [warnings_as_errors|Options]) @@ -59,6 +112,7 @@ roundtrip(Mod, Type, Value) -> roundtrip(Mod, Type, Value, ExpectedValue) -> {ok,Encoded} = Mod:encode(Type, Value), {ok,ExpectedValue} = Mod:decode(Type, Encoded), + test_ber_indefinite(Mod, Type, Encoded, ExpectedValue), ok. roundtrip_enc(Mod, Type, Value) -> @@ -67,6 +121,7 @@ roundtrip_enc(Mod, Type, Value) -> roundtrip_enc(Mod, Type, Value, ExpectedValue) -> {ok,Encoded} = Mod:encode(Type, Value), {ok,ExpectedValue} = Mod:decode(Type, Encoded), + test_ber_indefinite(Mod, Type, Encoded, ExpectedValue), Encoded. %%% @@ -76,3 +131,52 @@ roundtrip_enc(Mod, Type, Value, ExpectedValue) -> hex2num(C) when $0 =< C, C =< $9 -> C - $0; hex2num(C) when $A =< C, C =< $F -> C - $A + 10; hex2num(C) when $a =< C, C =< $f -> C - $a + 10. + +test_ber_indefinite(Mod, Type, Encoded, ExpectedValue) -> + case Mod:encoding_rule() of + ber -> + Indefinite = iolist_to_binary(ber_indefinite(Encoded)), + {ok,ExpectedValue} = Mod:decode(Type, Indefinite); + _ -> + ok + end. + +%% Rewrite all definite lengths for constructed values to an +%% indefinite length. +ber_indefinite(Bin0) -> + case ber_get_tag(Bin0) of + done -> + []; + primitive -> + Bin0; + {constructed,Tag,Bin1} -> + {Len,Bin2} = ber_get_len(Bin1), + <<Val0:Len/binary,Bin/binary>> = Bin2, + Val = iolist_to_binary(ber_indefinite(Val0)), + [<<Tag/binary,16#80,Val/binary,0,0>>|ber_indefinite(Bin)] + end. + +ber_get_tag(<<>>) -> + done; +ber_get_tag(<<_:2,0:1,_:5,_/binary>>) -> + primitive; +ber_get_tag(<<_:2,1:1,_:5,_/binary>>=Bin0) -> + TagLen = ber_tag_length(Bin0), + <<Tag:TagLen/binary,Bin/binary>> = Bin0, + {constructed,Tag,Bin}. + +ber_tag_length(<<_:3,2#11111:5,T/binary>>) -> + ber_tag_length_1(T, 1); +ber_tag_length(_) -> + 1. + +ber_tag_length_1(<<1:1,_:7,T/binary>>, N) -> + ber_tag_length_1(T, N+1); +ber_tag_length_1(<<0:1,_:7,_/binary>>, N) -> + N+1. + +ber_get_len(<<0:1,L:7,T/binary>>) -> + {L,T}; +ber_get_len(<<1:1,Octets:7,T0/binary>>) -> + <<L:Octets/unit:8,T/binary>> = T0, + {L,T}. diff --git a/lib/asn1/test/ber_decode_error.erl b/lib/asn1/test/ber_decode_error.erl index 8be92292ee..6fd2450c62 100644 --- a/lib/asn1/test/ber_decode_error.erl +++ b/lib/asn1/test/ber_decode_error.erl @@ -51,4 +51,18 @@ run([]) -> {error,{asn1,{invalid_value,_}}} = (catch 'Constructed':decode('I', <<8,7>>)), + %% Short indefinite length. Make sure that the decoder doesn't look + %% beyond the end of binary when looking for a 0,0 terminator. + {error,{asn1,{invalid_length,_}}} = + (catch 'Constructed':decode('S', sub(<<8,16#80,0,0>>, 3))), + {error,{asn1,{invalid_length,_}}} = + (catch 'Constructed':decode('S', sub(<<8,16#80,0,0>>, 2))), + {error,{asn1,{invalid_length,_}}} = + (catch 'Constructed':decode('S', sub(<<40,16#80,1,1,255,0,0>>, 6))), + {error,{asn1,{invalid_length,_}}} = + (catch 'Constructed':decode('S', sub(<<40,16#80,1,1,255,0,0>>, 5))), ok. + +sub(Bin, Bytes) -> + <<B:Bytes/binary,_/binary>> = Bin, + B. diff --git a/lib/asn1/test/error_SUITE.erl b/lib/asn1/test/error_SUITE.erl index 8a0414708d..1edd60f7c8 100644 --- a/lib/asn1/test/error_SUITE.erl +++ b/lib/asn1/test/error_SUITE.erl @@ -20,7 +20,8 @@ -module(error_SUITE). -export([suite/0,all/0,groups/0, already_defined/1,bitstrings/1,enumerated/1, - imports/1,instance_of/1,integers/1,objects/1,values/1]). + imports/1,instance_of/1,integers/1,objects/1, + parameterization/1,values/1]). -include_lib("test_server/include/test_server.hrl"). @@ -38,6 +39,7 @@ groups() -> instance_of, integers, objects, + parameterization, values]}]. parallel() -> @@ -219,6 +221,19 @@ objects(Config) -> } = run(P, Config), ok. +parameterization(Config) -> + M = 'Parameterization', + P = {M, + <<"Parameterization DEFINITIONS AUTOMATIC TAGS ::= BEGIN\n" + " NotUppercase{lowercase} ::= INTEGER (lowercase)\n" + "END\n">>}, + {error, + [{structured_error,{'Parameterization',2},asn1ct_check, + {illegal_typereference,lowercase}} + ] + } = run(P, Config), + ok. + values(Config) -> M = 'Values', P = {M, diff --git a/lib/asn1/test/testInfObj.erl b/lib/asn1/test/testInfObj.erl index 311595cfda..37c134b1b9 100644 --- a/lib/asn1/test/testInfObj.erl +++ b/lib/asn1/test/testInfObj.erl @@ -118,7 +118,41 @@ main(_Erule) -> roundtrip('InfObj', 'Multiple-Optionals', {'Multiple-Optionals',1,42,true,asn1_NOVALUE}), roundtrip('InfObj', 'Multiple-Optionals', - {'Multiple-Optionals',1,asn1_NOVALUE,asn1_NOVALUE,asn1_NOVALUE}). + {'Multiple-Optionals',1,asn1_NOVALUE,asn1_NOVALUE,asn1_NOVALUE}), + + test_objset('OstSeq12', [1,2]), + test_objset('OstSeq123', [1,2,3]), + test_objset('OstSeq1234', [1,2,3,4]), + test_objset('OstSeq45', [4,5]), + test_objset('OstSeq12345', [1,2,3,4,5]), + + test_objset('ExOstSeq12', [1,2]), + test_objset('ExOstSeq123', [1,2,3]), + %%test_objset('ExOstSeq1234', [1,2,3,4]), + test_objset('ExOstSeq45', [4,5]), + test_objset('ExOstSeq12345', [1,2,3,4,5]), + + ok. + +test_objset(Type, Keys) -> + _ = [test_object(Type, Key) || Key <- Keys], + _ = [(catch test_object(Type, Key)) || + Key <- lists:seq(1, 5) -- Keys], + ok. + +test_object(T, 1) -> + roundtrip('InfObj', T, {T,1,<<42:7>>}); +test_object(T, 2) -> + roundtrip('InfObj', T, {T,2,<<"abc">>}); +test_object(T, 3) -> + roundtrip('InfObj', T, {T,3,donald}), + roundtrip('InfObj', T, {T,3,scrooge}); +test_object(T, 4) -> + roundtrip('InfObj', T, {T,4,true}), + roundtrip('InfObj', T, {T,4,false}); +test_object(T, 5) -> + roundtrip('InfObj', T, {T,5,0}), + roundtrip('InfObj', T, {T,5,15}). roundtrip(M, T, V) -> asn1_test_lib:roundtrip(M, T, V). diff --git a/lib/asn1/test/testNBAPsystem.erl b/lib/asn1/test/testNBAPsystem.erl index 57cb483374..e37e22163a 100644 --- a/lib/asn1/test/testNBAPsystem.erl +++ b/lib/asn1/test/testNBAPsystem.erl @@ -79,13 +79,14 @@ powerRaiseLimit, dLPowerAveragingWindowSize, 'iE-Extensions' = asn1_NOVALUE}). compile(Config, Options) -> - [asn1_test_lib:compile(filename:join([nbapsystem, M]), Config, Options) - || M <- ["NBAP-CommonDataTypes.asn", - "NBAP-IEs.asn", - "NBAP-PDU-Contents.asn", - "NBAP-PDU-Discriptions.asn", - "NBAP-Constants.asn", - "NBAP-Containers.asn"]], + Fs = [filename:join("nbapsystem", M) || + M <- ["NBAP-CommonDataTypes.asn", + "NBAP-IEs.asn", + "NBAP-PDU-Contents.asn", + "NBAP-PDU-Discriptions.asn", + "NBAP-Constants.asn", + "NBAP-Containers.asn"]], + asn1_test_lib:compile_all(Fs, Config, Options), ok. diff --git a/lib/asn1/test/testParamBasic.erl b/lib/asn1/test/testParamBasic.erl index 3db89ca174..39f7947e8d 100644 --- a/lib/asn1/test/testParamBasic.erl +++ b/lib/asn1/test/testParamBasic.erl @@ -43,6 +43,9 @@ main(Rules) -> #'T12'{number=11,string = <<10:4>>}); _ -> ok end, + roundtrip('AnAlgorithm', {'AnAlgorithm',1,42}), + roundtrip('AnAlgorithm', {'AnAlgorithm',2,true}), + roundtrip('AnAlgorithm', {'AnAlgorithm',2,false}), ok. roundtrip(Type, Value) -> diff --git a/lib/asn1/test/testSeqSetDefaultVal.erl b/lib/asn1/test/testSeqSetDefaultVal.erl index 79992a0a94..c3d9ce33b7 100644 --- a/lib/asn1/test/testSeqSetDefaultVal.erl +++ b/lib/asn1/test/testSeqSetDefaultVal.erl @@ -36,6 +36,7 @@ c = asn1_DEFAULT, d = asn1_DEFAULT, e = asn1_DEFAULT}). +-record('SeqBS2',{bs = asn1_DEFAULT}). -record('SetBS',{a = asn1_DEFAULT, b = asn1_DEFAULT, c = asn1_DEFAULT, @@ -93,6 +94,13 @@ b = asn1_DEFAULT}). -record('S4_b',{ba = asn1_DEFAULT, bb = asn1_DEFAULT}). +-record('SeqNamedInts', + {i1 = asn1_DEFAULT, + i2 = asn1_DEFAULT}). +-record('S5',{s3 = asn1_DEFAULT, + so = asn1_DEFAULT, + soe = asn1_DEFAULT}). +-record('SOI', {soi = asn1_DEFAULT}). main(ber, []) -> %% Nothing to test because plain BER will only use @@ -105,7 +113,11 @@ main(Rule, Opts) -> case {Rule,Opts} of {ber,[der]} -> - der(); + der(), + case 'Default':legacy_erlang_types() of + false -> der_new_types(); + true -> der_legacy() + end; {_,_} -> ok end, @@ -118,45 +130,45 @@ main(Rule, Opts) -> {#'SeqBS'{}, [{#'SeqBS'.a, - [asn1_DEFAULT, - 2#0110101, + [asn1_DEFAULT, %Always. + <<1:1,0:1,1:1,0:1,1:1,1:1,0:1>>], + [2#0110101, %Legacy only. [1,0,1,0,1,1,0], - {1,<<16#AC>>}, - <<1:1,0:1,1:1,0:1,1:1,1:1,0:1>>]}, + {1,<<16#AC>>}]}, {#'SeqBS'.b, [asn1_DEFAULT, - 2#10100010101, + <<16#A8:8,16#A:4>>], + [2#10100010101, [1,0,1,0,1,0,0,0,1,0,1,0], - {4,<<16#A8,16#A0>>}, - <<16#A8:8,16#A:4>>]}, + {4,<<16#A8,16#A0>>}]}, {#'SeqBS'.c, [asn1_DEFAULT, [second], - [0,1], - {6,<<0:1,1:1,0:6>>}, - <<1:2>>]}, + <<1:2>>], + [[0,1], + {6,<<0:1,1:1,0:6>>}]}, {#'SeqBS'.c, %Zeroes on the right [asn1_DEFAULT, [second], - [0,1,0,0,0], - {4,<<0:1,1:1,0:6>>}, - <<1:2,0:17>>]}, + <<1:2,0:17>>], + [[0,1,0,0,0], + {4,<<0:1,1:1,0:6>>}]}, {#'SeqBS'.d, [asn1_DEFAULT, - 2#1001, + <<2#1001:4>>], + [2#1001, [1,0,0,1], - {4,<<2#1001:4,0:4>>}, - <<2#1001:4>>]}, + {4,<<2#1001:4,0:4>>}]}, {#'SeqBS'.e, [asn1_DEFAULT, - [0,1,0,1,1,0,1,0], - {0,<<2#01011010:8>>}, - <<2#01011010:8>>]}, + <<2#01011010:8>>], + [[0,1,0,1,1,0,1,0], + {0,<<2#01011010:8>>}]}, %% Not EQUAL to DEFAULT. {#'SeqBS'.b, + [<<6:3>>], [[1,1,0], %Not equal to DEFAULT - {5,<<6:3,0:5>>}, - <<6:3>>]} + {5,<<6:3,0:5>>}]} ]}, {#'SeqOS'{}, @@ -170,15 +182,14 @@ main(Rule, Opts) -> {1,2,14,15}]}, {#'SeqOI'.b, [asn1_DEFAULT, -%% {iso,'member-body',250,3,4}, + %% {iso,'member-body',250,3,4}, {1,2,250,3,4}]}, {#'SeqOI'.c, [asn1_DEFAULT, -%% {iso,standard,8571,2,250,4}, + %% {iso,standard,8571,2,250,4}, {1,0,8571,2,250,4}]}]} ], - io:format("~p\n", [Ts]), - R0 = [[consistency(Rec, Pos, Vs) || {Pos,Vs} <- Fs] || {Rec,Fs} <- Ts], + R0 = [[consistency(Rec, PosVs) || PosVs <- Fs] || {Rec,Fs} <- Ts], case lists:flatten(R0) of [] -> ok; @@ -187,8 +198,20 @@ main(Rule, Opts) -> ?t:fail() end. -consistency(Rec0, Pos, [V|Vs]) -> +legacy_filter({_,_}=Keep) -> + Keep; +legacy_filter({Rec,Standard,Legacy}) -> + case 'Default':legacy_erlang_types() of + false -> + {Rec,Standard}; + true -> + {Rec,Standard++Legacy} + end. + +consistency(Rec0, PosVs) -> + {Pos,[V|Vs]=AllVs} = legacy_filter(PosVs), T = element(1, Rec0), + io:format("~p: ~p\n", [T,AllVs]), Rec = setelement(Pos, Rec0, V), {ok,Enc} = 'Default':encode(T, Rec), {ok,_SmokeTest} = 'Default':decode(T, Enc), @@ -206,7 +229,7 @@ consistency_1([V|Vs], Rec0, Pos, Enc) -> consistency_1([], _, _, _) -> []. der() -> - io:put_chars("Peforming DER-specific tests..."), + io:put_chars("Performing DER-specific tests..."), roundtrip(<<48,0>>, 'SeqInts', #'SeqInts'{a=asn1_DEFAULT,b=asn1_DEFAULT, @@ -227,98 +250,6 @@ der() -> #'SetInts'{a=1,b=-1,c=three,d=1}, #'SetInts'{a=1,b=-1,c=3,d=1}), - - roundtrip(<<48,0>>, - 'SeqBS', - #'SeqBS'{a=2#0110101, - b=2#010100010101, - c=[second], - d=[1,0,0,1]}, - #'SeqBS'{a = <<2#1010110:7>>, b = <<16#A8A:12>>, - c=[second], d = <<2#1001:4>>, - e = <<2#01011010:8>>}), - roundtrip(<<48,0>>, - 'SeqBS', - #'SeqBS'{a=[1,0,1,0,1,1,0], - b=[1,0,1,0,1,0,0,0,1,0,1,0], - c={5,<<64>>}, - d=2#1001}, - #'SeqBS'{a = <<2#1010110:7>>, b = <<16#A8A:12>>, - c=[second], d = <<2#1001:4>>, - e = <<2#01011010:8>>}), - roundtrip(<<48,3,131,1,0>>, - 'SeqBS', - #'SeqBS'{a=[1,0,1,0,1,1,0], - b=[1,0,1,0,1,0,0,0,1,0,1,0], - c={5,<<64>>}, - d=0}, - #'SeqBS'{a = <<2#1010110:7>>, b = <<16#A8A:12>>, - c=[second], d = <<>>, - e = <<2#01011010:8>>}), - roundtrip(<<48,3,131,1,0>>, - 'SeqBS', - #'SeqBS'{a = <<1:1,0:1,1:1,0:1,1:1,1:1,0:1>>, - b = <<1:1,0:1,1:1,0:1,1:1,0:1,0:1,0:1,1:1,0:1,1:1,0:1>>, - c = <<2:3>>, - d=0, - e = <<16#5A:8>>}, - #'SeqBS'{a = <<2#1010110:7>>, b = <<16#A8A:12>>, - c=[second], d = <<>>, - e = <<2#01011010:8>>}), - - %% None of the default values are used. - roundtrip(<<48,19,128,2,7,128,129,2,5,64,130,2,5,32,131,1,0,132,2,5,224>>, - 'SeqBS', - #'SeqBS'{a = <<1:1>>, - b = {5,<<64>>}, - c = [third], - d = 0, - e = <<7:3>>}, - #'SeqBS'{a = <<1:1>>, - b = <<2:3>>, - c = [third], - d = <<>>, - e = <<7:3>>}), - - roundtrip(<<49,0>>, - 'SetBS', - #'SetBS'{a=2#0110101, - b=2#010100010101, - c=[second], - d=[1,0,0,1]}, - #'SetBS'{a = <<2#1010110:7>>, b = <<16#A8A:12>>, - c=[second], d = <<2#1001:4>>}), - roundtrip(<<49,0>>, - 'SetBS', - #'SetBS'{a=[1,0,1,0,1,1,0], - b=[1,0,1,0,1,0,0,0,1,0,1,0], - c={5,<<64>>}, - d=9}, - #'SetBS'{a = <<2#1010110:7>>, b = <<16#A8A:12>>, - c=[second], d = <<2#1001:4>>}), - roundtrip(<<49,3,131,1,0>>, - 'SetBS', - #'SetBS'{a=[1,0,1,0,1,1,0], - b=[1,0,1,0,1,0,0,0,1,0,1,0], - c={5,<<64>>}, - d=0}, - #'SetBS'{a = <<2#1010110:7>>, b = <<16#A8A:12>>, - c=[second], d = <<>>}), - roundtrip(<<49,3,131,1,0>>, - 'SetBS', - #'SetBS'{a = <<1:1,0:1,1:1,0:1,1:1,1:1,0:1>>, - b = <<1:1,0:1,1:1,0:1,1:1,0:1,0:1,0:1,1:1,0:1,1:1,0:1>>, - c = <<2:3>>, - d=0}, - #'SetBS'{a = <<2#1010110:7>>, b = <<16#A8A:12>>, - c=[second], d = <<>>}), - - roundtrip(<<48,0>>, 'SeqOS', - #'SeqOS'{a = <<172>>,b = <<16#A8,16#A0>>,c='NULL'}), - - roundtrip(<<49,0>>, 'SetOS', - #'SetOS'{a = <<172>>,b = <<16#A8,16#A0>>,c='NULL'}), - roundtrip(<<48,0>>, 'SeqOI', #'SeqOI'{a={1,2,14,15}, @@ -443,6 +374,184 @@ der() -> #'S4'{a=#'S2'{a=1,b=asn1_NOVALUE},b=#'S4_b'{ba=true,bb=0}}, #'S4'{a=#'S2'{a=1,b=asn1_NOVALUE},b=#'S4_b'{ba=true,bb=0}}), + roundtrip(<<48,0>>, + 'SeqBS', + #'SeqBS'{a = <<2#1010110:7>>, b = <<16#A8A:12>>, + c=[second], d = <<2#1001:4>>, + e = <<2#01011010:8>>}), + roundtrip(<<49,0>>, + 'SetBS', + #'SetBS'{a = <<2#1010110:7>>, b = <<16#A8A:12>>, + c=[second], d = <<2#1001:4>>}), + + %% None of the default values are used. + roundtrip(<<48,19,128,2,7,128,129,2,5,64,130,2,5,32,131,1,0,132,2,5,224>>, + 'SeqBS', + #'SeqBS'{a = <<1:1>>, + b = <<2:3>>, + c = [third], + d = <<>>, + e = <<7:3>>}), + roundtrip(<<49,3,131,1,0>>, + 'SetBS', + #'SetBS'{a = <<2#1010110:7>>, b = <<16#A8A:12>>, + c=[second], d = <<>>}), + + %% SeqNamedInts + roundtrip(<<48,0>>, + 'SeqNamedInts', + #'SeqNamedInts'{i1=15,i2=31}), + roundtrip(<<48,0>>, + 'SeqNamedInts', + #'SeqNamedInts'{}, + #'SeqNamedInts'{i1=15,i2=31}), + roundtrip(<<48,0>>, + 'SeqNamedInts', + #'SeqNamedInts'{i2=last}, + #'SeqNamedInts'{i1=15,i2=31}), + roundtrip(<<48,3,128,1,0>>, + 'SeqNamedInts', + #'SeqNamedInts'{i1=first,i2=31}, + #'SeqNamedInts'{i1=first,i2=31}), + + %% S5 + roundtrip(<<48,0>>, + 'S5', + #'S5'{s3=#'S3'{a=[11,12,13], + b=[{a,11},{b,true},{c,13}], + c=[1,2,3,4], + d=[#'S2'{a=20,b=true},#'S2'{a=30,b=false}]}, + so=[{0,1,999},{0,1,555}], + soe=[]}), + roundtrip(<<48,0>>, + 'S5', + #'S5'{}, + #'S5'{s3=#'S3'{a=[11,12,13], + b=[{a,11},{b,true},{c,13}], + c=[1,2,3,4], + d=[#'S2'{a=20,b=true},#'S2'{a=30,b=false}]}, + so=[{0,1,999},{0,1,555}], + soe=[]}), + + %% SOI + roundtrip(<<48,0>>, + 'SOI', + #'SOI'{}, + #'SOI'{soi=[{1,2,250,9,55},{1,2,250,3,4}]}), + + %% SeqBS2 + roundtrip(<<48,0>>, + 'SeqBS2', + #'SeqBS2'{bs= <<16#5:3>>}), + roundtrip(<<48,0>>, + 'SeqBS2', + #'SeqBS2'{bs= <<16#5:3,0:4>>}, + #'SeqBS2'{bs= <<16#5:3>>}), + + ok. + +der_new_types() -> + io:put_chars("Performing DER-specific tests with new types..."), + + roundtrip(<<48,0>>, 'SeqOS', + #'SeqOS'{a = <<172>>,b = <<16#A8,16#A0>>,c='NULL'}), + + roundtrip(<<49,0>>, 'SetOS', + #'SetOS'{a = <<172>>,b = <<16#A8,16#A0>>,c='NULL'}), + ok. + +der_legacy() -> + io:put_chars("Performing DER-specific tests with legacy types..."), + + roundtrip(<<48,0>>, 'SeqOS', + #'SeqOS'{a=[172],b=[16#A8,16#A0],c='NULL'}), + roundtrip(<<49,0>>, 'SetOS', + #'SetOS'{a=[172],b=[16#A8,16#A0],c='NULL'}), + + roundtrip(<<48,0>>, + 'SeqBS', + #'SeqBS'{a=2#0110101, + b=2#010100010101, + c=[second], + d=[1,0,0,1]}, + #'SeqBS'{a = <<2#1010110:7>>, b = <<16#A8A:12>>, + c=[second], d = <<2#1001:4>>, + e = <<2#01011010:8>>}), + roundtrip(<<48,0>>, + 'SeqBS', + #'SeqBS'{a=[1,0,1,0,1,1,0], + b=[1,0,1,0,1,0,0,0,1,0,1,0], + c={5,<<64>>}, + d=2#1001}, + #'SeqBS'{a = <<2#1010110:7>>, b = <<16#A8A:12>>, + c=[second], d = <<2#1001:4>>, + e = <<2#01011010:8>>}), + roundtrip(<<48,3,131,1,0>>, + 'SeqBS', + #'SeqBS'{a=[1,0,1,0,1,1,0], + b=[1,0,1,0,1,0,0,0,1,0,1,0], + c={5,<<64>>}, + d=0}, + #'SeqBS'{a = <<2#1010110:7>>, b = <<16#A8A:12>>, + c=[second], d = <<>>, + e = <<2#01011010:8>>}), + roundtrip(<<48,3,131,1,0>>, + 'SeqBS', + #'SeqBS'{a = <<1:1,0:1,1:1,0:1,1:1,1:1,0:1>>, + b = <<1:1,0:1,1:1,0:1,1:1,0:1,0:1,0:1,1:1,0:1,1:1,0:1>>, + c = <<2:3>>, + d=0, + e = <<16#5A:8>>}, + #'SeqBS'{a = <<2#1010110:7>>, b = <<16#A8A:12>>, + c=[second], d = <<>>, + e = <<2#01011010:8>>}), + + %% None of the default values are used. + roundtrip(<<48,19,128,2,7,128,129,2,5,64,130,2,5,32,131,1,0,132,2,5,224>>, + 'SeqBS', + #'SeqBS'{a = <<1:1>>, + b = {5,<<64>>}, + c = [third], + d = 0, + e = <<7:3>>}, + #'SeqBS'{a = <<1:1>>, + b = <<2:3>>, + c = [third], + d = <<>>, + e = <<7:3>>}), + roundtrip(<<49,0>>, + 'SetBS', + #'SetBS'{a=2#0110101, + b=2#010100010101, + c=[second], + d=[1,0,0,1]}, + #'SetBS'{a = <<2#1010110:7>>, b = <<16#A8A:12>>, + c=[second], d = <<2#1001:4>>}), + roundtrip(<<49,0>>, + 'SetBS', + #'SetBS'{a=[1,0,1,0,1,1,0], + b=[1,0,1,0,1,0,0,0,1,0,1,0], + c={5,<<64>>}, + d=9}, + #'SetBS'{a = <<2#1010110:7>>, b = <<16#A8A:12>>, + c=[second], d = <<2#1001:4>>}), + roundtrip(<<49,3,131,1,0>>, + 'SetBS', + #'SetBS'{a=[1,0,1,0,1,1,0], + b=[1,0,1,0,1,0,0,0,1,0,1,0], + c={5,<<64>>}, + d=0}, + #'SetBS'{a = <<2#1010110:7>>, b = <<16#A8A:12>>, + c=[second], d = <<>>}), + roundtrip(<<49,3,131,1,0>>, + 'SetBS', + #'SetBS'{a = <<1:1,0:1,1:1,0:1,1:1,1:1,0:1>>, + b = <<1:1,0:1,1:1,0:1,1:1,0:1,0:1,0:1,1:1,0:1,1:1,0:1>>, + c = <<2:3>>, + d=0}, + #'SetBS'{a = <<2#1010110:7>>, b = <<16#A8A:12>>, + c=[second], d = <<>>}), + ok. roundtrip(Encoded, Type, Value) -> diff --git a/lib/asn1/test/testTcapsystem.erl b/lib/asn1/test/testTcapsystem.erl index 4979a385b2..fcc9e084e0 100644 --- a/lib/asn1/test/testTcapsystem.erl +++ b/lib/asn1/test/testTcapsystem.erl @@ -21,44 +21,42 @@ -export([compile/2]). --include_lib("test_server/include/test_server.hrl"). - compile(Config, Options) -> - [asn1_test_lib:compile(filename:join([tcapsystem, M]), Config, Options) - || M <- ["DialoguePDUs.asn", - "MAP-ApplicationContexts.asn", - "MAP-BS-Code.asn", - "MAP-CallHandlingOperations.asn", - "MAP-CH-DataTypes.asn", - "MAP-CommonDataTypes.asn", - "MAP-DialogueInformation.asn", - "MAP-ER-DataTypes.asn", - "MAP-Errors.asn", - "MAP-ExtensionDataTypes.asn", - "MAP-GR-DataTypes.asn", - "MAP-Group-Call-Operations.asn", - "MAP-LCS-DataTypes.asn", - "MAP-LocationServiceOperations.asn", - "MAP-MobileServiceOperations.asn", - "MAP-MS-DataTypes.asn", - "MAP-OM-DataTypes.asn", - "MAP-OperationAndMaintenanceOperations.asn", - "MAP-Protocol.asn", - "MAP-SecureTransportOperations.asn", - "MAP-ShortMessageServiceOperations.asn", - "MAP-SM-DataTypes.asn", - "MAP-SS-Code.asn", - "MAP-SS-DataTypes.asn", - "MAP-ST-DataTypes.asn", - "MAP-SupplementaryServiceOperations.asn", - "MAP-TS-Code.asn", - "MobileDomainDefinitions.asn", - "Remote-Operations-Generic-ROS-PDUs.asn", - "Remote-Operations-Information-Objects.asn", - "Remote-Operations-Useful-Definitions.asn", - "TCAP-Examples.asn", - "TCAPMessages.asn", - "TCAP-Tools.asn", - "TC-Notation-Extensions.asn", - "UnidialoguePDUs.asn"]], - ok. + Fs = [filename:join("tcapsystem", M) || + M <- ["DialoguePDUs.asn", + "MAP-ApplicationContexts.asn", + "MAP-BS-Code.asn", + "MAP-CallHandlingOperations.asn", + "MAP-CH-DataTypes.asn", + "MAP-CommonDataTypes.asn", + "MAP-DialogueInformation.asn", + "MAP-ER-DataTypes.asn", + "MAP-Errors.asn", + "MAP-ExtensionDataTypes.asn", + "MAP-GR-DataTypes.asn", + "MAP-Group-Call-Operations.asn", + "MAP-LCS-DataTypes.asn", + "MAP-LocationServiceOperations.asn", + "MAP-MobileServiceOperations.asn", + "MAP-MS-DataTypes.asn", + "MAP-OM-DataTypes.asn", + "MAP-OperationAndMaintenanceOperations.asn", + "MAP-Protocol.asn", + "MAP-SecureTransportOperations.asn", + "MAP-ShortMessageServiceOperations.asn", + "MAP-SM-DataTypes.asn", + "MAP-SS-Code.asn", + "MAP-SS-DataTypes.asn", + "MAP-ST-DataTypes.asn", + "MAP-SupplementaryServiceOperations.asn", + "MAP-TS-Code.asn", + "MobileDomainDefinitions.asn", + "Remote-Operations-Generic-ROS-PDUs.asn", + "Remote-Operations-Information-Objects.asn", + "Remote-Operations-Useful-Definitions.asn", + "TCAP-Examples.asn", + "TCAPMessages.asn", + "TCAP-Tools.asn", + "TC-Notation-Extensions.asn", + "UnidialoguePDUs.asn"]], + asn1_test_lib:compile_all(Fs, Config, Options). diff --git a/lib/asn1/vsn.mk b/lib/asn1/vsn.mk index 1f16f31f6b..d87c50637d 100644 --- a/lib/asn1/vsn.mk +++ b/lib/asn1/vsn.mk @@ -1,2 +1,2 @@ #next version number to use is 2.0 -ASN1_VSN = 3.0 +ASN1_VSN = 3.0.2 diff --git a/lib/common_test/configure.in b/lib/common_test/configure.in index b2e6ad997a..b2e6ad997a 100755..100644 --- a/lib/common_test/configure.in +++ b/lib/common_test/configure.in diff --git a/lib/common_test/doc/src/Makefile b/lib/common_test/doc/src/Makefile index 99161ce68a..57233a7f6c 100644 --- a/lib/common_test/doc/src/Makefile +++ b/lib/common_test/doc/src/Makefile @@ -47,6 +47,7 @@ CT_MODULES = \ ct_snmp \ unix_telnet \ ct_slave \ + ct_property_test \ ct_netconfc CT_XML_FILES = $(CT_MODULES:=.xml) diff --git a/lib/common_test/doc/src/cover_chapter.xml b/lib/common_test/doc/src/cover_chapter.xml index a215c8c2f3..accb94e1a9 100644 --- a/lib/common_test/doc/src/cover_chapter.xml +++ b/lib/common_test/doc/src/cover_chapter.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2006</year><year>2013</year> + <year>2006</year><year>2014</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -81,10 +81,7 @@ specify that previously exported data should be imported and included in the analysis for a test (you can specify multiple import files). This way it is possible to analyse total code coverage - without necessarily running all tests at once. Note that even if - you run separate tests in one test run, code coverage data will - not be passed on from one test to another unless you specify an - export file for Common Test to use for this purpose.</p> + without necessarily running all tests at once.</p> <p>To activate the code coverage support, you simply specify the name of the cover specification file as you start Common Test. @@ -266,10 +263,20 @@ ct_cover:cross_cover_analyse(Level, [{s1,S1LogDir},{s2,S2LogDir}]).</code> <section> <title>Logging</title> - <p>To view the result of a code coverage test, follow the - "Coverage log" link on the test suite results page. This - takes you to the code coverage overview page. If you have - successfully performed a detailed coverage analysis, you + <p>To view the result of a code coverage test, click the button + labled "COVER LOG" in the top level index page for the test run.</p> + + <p>Prior to Erlang/OTP 17.1, if your test run consisted of + multiple tests, cover would be started and stopped for each test + within the test run. Separate logs would be available via the + "Coverage log" link on the test suite result pages. These links + are still available, but now they all point to the same page as + the button on the top level index page. The log contains the + accumulated results for the complete test run. See the release + notes for more information about this change.</p> + + <p>The buttonc takes you to the code coverage overview page. If you + have successfully performed a detailed coverage analysis, you find links to each individual module coverage page here.</p> <p>If cross cover analysis has been performed, and there are diff --git a/lib/common_test/doc/src/ct_hooks.xml b/lib/common_test/doc/src/ct_hooks.xml index 859ff9df14..cab6dfea51 100644 --- a/lib/common_test/doc/src/ct_hooks.xml +++ b/lib/common_test/doc/src/ct_hooks.xml @@ -450,12 +450,15 @@ </func> <func> - <name>Module:on_tc_fail(TestcaseName, Reason, CTHState) -> + <name>Module:on_tc_fail(TestName, Reason, CTHState) -> NewCTHState</name> <fsummary>Called after the CTH scope ends</fsummary> <type> - <v>TestcaseName = init_per_suite | end_per_suite | - init_per_group | end_per_group | atom()</v> + <v>TestName = init_per_suite | end_per_suite | + {init_per_group,GroupName} | {end_per_group,GroupName} | + {FuncName,GroupName} | FuncName</v> + <v>FuncName = atom()</v> + <v>GroupName = atom()</v> <v>Reason = term()</v> <v>CTHState = NewCTHState = term()</v> </type> @@ -463,14 +466,16 @@ <desc> <p> OPTIONAL </p> - <p>This function is called whenever a testcase fails. - It is called after the post function has been called for - the testcase which failed. i.e. - if init_per_suite fails this function is called after + <p>This function is called whenever a test case (or config function) + fails. It is called after the post function has been called for + the failed test case. I.e. if init_per_suite fails, this function + is called after <seealso marker="#Module:post_init_per_suite-4"> - post_init_per_suite</seealso>, and if a testcase fails it is called + post_init_per_suite</seealso>, and if a test case fails, it is called after <seealso marker="#Module:post_end_per_testcase-4"> - post_end_per_testcase</seealso>.</p> + post_end_per_testcase</seealso>. If the failed test case belongs + to a test case group, the first argument is a tuple + <c>{FuncName,GroupName}</c>, otherwise simply the function name.</p> <p>The data which comes with the Reason follows the same format as the <seealso marker="event_handler_chapter#failreason">FailReason @@ -481,12 +486,14 @@ </func> <func> - <name>Module:on_tc_skip(TestcaseName, Reason, CTHState) -> + <name>Module:on_tc_skip(TestName, Reason, CTHState) -> NewCTHState</name> <fsummary>Called after the CTH scope ends</fsummary> <type> - <v>TestcaseName = end_per_suite | {init_per_group,GroupName} | - {end_per_group,GroupName} | atom()</v> + <v>TestName = init_per_suite | end_per_suite | + {init_per_group,GroupName} | {end_per_group,GroupName} | + {FuncName,GroupName} | FuncName</v> + <v>FuncName = atom()</v> <v>GroupName = atom()</v> <v>Reason = {tc_auto_skip | tc_user_skip, term()}</v> <v>CTHState = NewCTHState = term()</v> @@ -495,14 +502,17 @@ <desc> <p> OPTIONAL </p> - <p>This function is called whenever a testcase is skipped. - It is called after the post function has been called for the - testcase which was skipped. - i.e. if init_per_group is skipped this function is called after - <seealso marker="#Module:post_init_per_suite-4">post_init_per_group - </seealso>, and if a testcase is skipped it is called after - <seealso marker="#Module:post_end_per_testcase-4">post_end_per_testcase - </seealso>.</p> + <p>This function is called whenever a test case (or config function) + is skipped. It is called after the post function has been called + for the skipped test case. I.e. if init_per_group is skipped, this + function is called after + <seealso marker="#Module:post_init_per_group-4"> + post_init_per_group</seealso>, and if a test case is skipped, + it is called after + <seealso marker="#Module:post_end_per_testcase-4"> + post_end_per_testcase</seealso>. If the skipped test case belongs to a + test case group, the first argument is a tuple <c>{FuncName,GroupName}</c>, + otherwise simply the function name.</p> <p>The data which comes with the Reason follows the same format as <seealso marker="event_handler_chapter#tc_auto_skip">tc_auto_skip diff --git a/lib/common_test/doc/src/ct_run.xml b/lib/common_test/doc/src/ct_run.xml index 39259b092a..d8e79ca80e 100644 --- a/lib/common_test/doc/src/ct_run.xml +++ b/lib/common_test/doc/src/ct_run.xml @@ -108,6 +108,7 @@ EvHandler2 InitArg2 and .. EvHandlerN InitArgN] [-include InclDir1 InclDir2 .. InclDirN] [-no_auto_compile] + [-abort_if_missing_suites] [-muliply_timetraps Multiplier] [-scale_timetraps] [-create_priv_dir auto_per_run | auto_per_tc | manual_per_tc] @@ -144,6 +145,7 @@ EvHandler2 InitArg2 and .. EvHandlerN InitArgN] [-include InclDir1 InclDir2 .. InclDirN] [-no_auto_compile] + [-abort_if_missing_suites] [-muliply_timetraps Multiplier] [-scale_timetraps] [-create_priv_dir auto_per_run | auto_per_tc | manual_per_tc] @@ -171,6 +173,7 @@ [-decrypt_key Key] | [-decrypt_file KeyFile] [-include InclDir1 InclDir2 .. InclDirN] [-no_auto_compile] + [-abort_if_missing_suites] [-muliply_timetraps Multiplier] [-scale_timetraps] [-create_priv_dir auto_per_run | auto_per_tc | manual_per_tc] diff --git a/lib/common_test/doc/src/event_handler_chapter.xml b/lib/common_test/doc/src/event_handler_chapter.xml index 47d0ba59fb..45f01c12ec 100644 --- a/lib/common_test/doc/src/event_handler_chapter.xml +++ b/lib/common_test/doc/src/event_handler_chapter.xml @@ -227,11 +227,13 @@ <item> <marker id="tc_auto_skip"></marker> - <c>#event{name = tc_auto_skip, data = {Suite,Func,Reason}}</c> + <c>#event{name = tc_auto_skip, data = {Suite,TestName,Reason}}</c> <p><c>Suite = atom()</c>, the name of the suite.</p> - <p><c>Func = atom() | {end_per_group,GroupName}</c>, the name of the test case - or configuration function.</p> - <p><c>GroupName = atom()</c>, name of the group.</p> + <p><c>TestName = init_per_suite | end_per_suite | + {init_per_group,GroupName} | {end_per_group,GroupName} | + {FuncName,GroupName} | FuncName</c></p> + <p><c>FuncName = atom()</c>, the name of the test case or configuration function.</p> + <p><c>GroupName = atom()</c>, the name of the test case group.</p> <p><c>Reason = {failed,FailReason} | {require_failed_in_suite0,RequireInfo}</c>, reason for auto skipping <c>Func</c>.</p> @@ -252,21 +254,26 @@ <c>init_per_group</c>, a failed <c>require</c> in <c>suite/0</c>, or a failed test case in a sequence. Note that this event is never received as a result of a test case getting skipped because of <c>init_per_testcase</c> failing, since that information is carried with - the <c>tc_done</c> event. + the <c>tc_done</c> event. If a failed test case belongs to a test case group, the second + data element is a tuple <c>{FuncName,GroupName}</c>, otherwise simply the function name. </p></item> - + <item> <marker id="tc_user_skip"></marker> - <c>#event{name = tc_user_skip, data = {Suite,Func,Comment}}</c> + <c>#event{name = tc_user_skip, data = {Suite,TestName,Comment}}</c> <p><c>Suite = atom()</c>, the name of the suite.</p> - <p><c>Func = atom() | {end_per_group,GroupName}</c>, the name of the test case - or configuration function.</p> - <p><c>GroupName = atom()</c>, name of the group.</p> + <p><c>TestName = init_per_suite | end_per_suite | + {init_per_group,GroupName} | {end_per_group,GroupName} | + {FuncName,GroupName} | FuncName</c></p> + <p><c>FuncName = atom()</c>, the name of the test case or configuration function.</p> + <p><c>GroupName = atom()</c>, the name of the test case group.</p> <p><c>Comment = string()</c>, reason for skipping the test case.</p> <p>This event specifies that a test case has been skipped by the user. It is only ever received if the skip was declared in a test specification. Otherwise, user skip information is received as a <c>{skipped,SkipReason}</c> - result in the <c>tc_done</c> event for the test case. + result in the <c>tc_done</c> event for the test case. If a skipped test case belongs + to a test case group, the second data element is a tuple <c>{FuncName,GroupName}</c>, + otherwise simply the function name. </p></item> <item><c>#event{name = test_stats, data = {Ok,Failed,Skipped}}</c> diff --git a/lib/common_test/doc/src/notes.xml b/lib/common_test/doc/src/notes.xml index f10d5f85bf..f4ce5369f7 100644 --- a/lib/common_test/doc/src/notes.xml +++ b/lib/common_test/doc/src/notes.xml @@ -32,6 +32,297 @@ <file>notes.xml</file> </header> +<section><title>Common_Test 1.8.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Ticket OTP-11971 introduced a runtime dependency towards + test_server-3.7.1, since the interface between + test_server and common_test was changed. Erroneously, the + common_test.app file was not updated according to this. + This has now been corrected.</p> + <p> + Own Id: OTP-12037</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Warning: this is experimental and may disappear or change + without previous warning.</p> + <p> + Experimental support for running Quickcheck and PropEr + tests from common_test suites is added to common_test. + See the reference manual for the new module + <c>ct_property_testing</c>.</p> + <p> + Experimental property tests are added under + <c>lib/{inet,ssh}/test/property_test</c>. They can be run + directly or from the commont_test suites + <c>inet/ftp_property_test_SUITE.erl</c> and + <c>ssh/test/ssh_property_test_SUITE.erl</c>.</p> + <p> + See the code in the <c>test</c> directories and the man + page for details.</p> + <p> + (Thanks to Tuncer Ayaz for a patch adding Triq)</p> + <p> + Own Id: OTP-12119</p> + </item> + </list> + </section> + +</section> + +<section><title>Common_Test 1.8.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Substrings in long telnet messages would sometimes get + wrongly reversed. This error has been corrected.</p> + <p> + Own Id: OTP-11871 Aux Id: seq12581 </p> + </item> + <item> + <p> + The basic_html logging mode in Common Test (for + compatibility with old browsers) generated HTML code with + unbalanced tags. This has been fixed.</p> + <p> + Own Id: OTP-11917 Aux Id: seq12598 </p> + </item> + <item> + <p> + The mechanism for running code cover analysis with + common_test has been improved. Earlier, if a test run + consisted of multiple tests, cover would be started and + stopped for each test. This would give "intermediate" + cover logs available from the "Coverage log" link on the + test suite result pages. To accumulate cover data over + all tests, the 'export' option had to be used in the + cover spec file. This was not well documented, and the + functionality was quite confusing.</p> + <p> + Using the 'nodes' option in the cover spec file would + fail when the test run consisted of multiple tests, since + the specified nodes would only be included in the cover + analysis of the first test.</p> + <p> + The repeated compilation and analysis of the same modules + was also very time consuming.</p> + <p> + To overcome these problems, ct will now only cover + compile and analyze modules once per test run, i.e. once + for each cover spec file. The log file is available via a + new button on the top level index page. The old "Coverage + log" links on the test suite result pages still exist, + but they all point to the same log containing the + accumulated result.</p> + <p> + Own Id: OTP-11971</p> + </item> + <item> + <p> + If multiple tests would run simultaneously on different + Erlang nodes, writing their logs to the same directory, + then there would often be entries in the all_runs.html + log file showing incomplete results (all zeroes) upon + completion. This problem was caused by a bug in the + Common Test log cache mechanism, which has been fixed.</p> + <p> + Own Id: OTP-11988 Aux Id: seq12611 </p> + </item> + </list> + </section> + +</section> + +<section><title>Common_Test 1.8</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + The error generated if a test case process received an + exit from a linked process while executing + init_per_testcase/2, was handled incorrectly by Common + Test. The problem has been solved, and Common Test now + reports this type of error correctly, with proper error + reason and exit location as well.</p> + <p> + Own Id: OTP-11643</p> + </item> + <item> + <p> + Running a parallel test case group with two or more + instances of the same test case would result in identical + log file names, and one test case instance would + overwrite the log file of another. This problem has been + solved.</p> + <p> + Own Id: OTP-11644</p> + </item> + <item> + <p> + Application upgrade (appup) files are corrected for the + following applications: </p> + <p> + <c>asn1, common_test, compiler, crypto, debugger, + dialyzer, edoc, eldap, erl_docgen, et, eunit, gs, hipe, + inets, observer, odbc, os_mon, otp_mibs, parsetools, + percept, public_key, reltool, runtime_tools, ssh, + syntax_tools, test_server, tools, typer, webtool, wx, + xmerl</c></p> + <p> + A new test utility for testing appup files is added to + test_server. This is now used by most applications in + OTP.</p> + <p> + (Thanks to Tobias Schlager)</p> + <p> + Own Id: OTP-11744</p> + </item> + <item> + <p> + The <c>cth_surefire</c> hook would crash in + <c>pre_init_per_suite/3</c> if a previous hook returned + <c>{skip,Reason}</c> or <c>{fail,Reason}</c> instead of a + <c>Config</c> list. This error has been corrected, and + <c>cth_surefire</c> will now simply propagate the + received <c>InitData</c> value instead.</p> + <p> + Own Id: OTP-11811</p> + </item> + <item> + <p> + Specs of return values are corrected for + <c>ct_netconfc:get/2,3</c>, + <c>ct_netconfc:get_config/3,4</c>, + <c>ct_netconfc:action/2,3</c>, + <c>ct_netconfc:send_rpc/2,3</c> and + <c>ct_netconfc:send/2,3</c>.</p> + <p> + Own Id: OTP-11834 Aux Id: seq12574 </p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + ct_telnet can now log all communication taking place + during a telnet session. Previously, only information + about ct_telnet operations and commands, as well as + explicitly requested data from the server, was logged.</p> + <p> + Furthermore, a logging mechanism based on an Error Logger + event handler and a dedicated Common Test hook, + <c>cth_conn_log</c>, now makes it possible to print data + for individual connections to separate log files. Please + see the <c>ct_telnet</c> reference manual for more + information and examples.</p> + <p> + Important note: A new argument, <c>ConnName</c> has been + added to the <c>unix_telnet:connect/5</c> callback + function. This forces users that use private ct_telnet + callback modules to update their code according to + <c>unix_telnet:connect/6</c>. Please see the + <c>unix_telnet</c> reference manual and source code + module for details.</p> + <p> + Own Id: OTP-11440 Aux Id: seq12457 </p> + </item> + <item> + <p> + A new timeout option has been introduced for the + <c>ct_telnet:expect/3</c> function. With + <c>{total_timeout,Time}</c> it's possible to set a time + limit for the complete expect operation. After + <c>Time</c> milliseconds, <c>expect/3</c> returns + <c>{error,timeout}</c>. The default value used if + <c>total_timeout</c> is not specified, is infinity (i.e. + no time limit). Please see the <c>ct_telnet</c> reference + manual for more information.</p> + <p> + Own Id: OTP-11689</p> + </item> + <item> + <p> + Some function specs are corrected or moved and some edoc + comments are corrected in order to allow use of edoc. + (Thanks to Pierre Fenoll)</p> + <p> + Own Id: OTP-11702</p> + </item> + <item> + <p> + Test case group name information has been added to the + data sent with <c>tc_user_skip</c> and + <c>tc_auto_skip</c> event messages, as well as the data + passed in calls to the CT Hook functions + <c>on_tc_skip/3</c> and <c>on_tc_fail/3</c>. The + modification only affects the function name + element/argument. This value remains an atom if the test + case in question does not belong to a test case group. + Otherwise a tuple <c>{FuncName,GroupName}</c> + (<c>{atom(),atom()}</c>) is passed instead.</p> + <p> + Note that this change may (depending on the patterns used + for matching) require modifications of user event + handlers and hook modules. Please see the Event Handling + chapter in the Common Test User's Guide, and the + reference manual for <c>ct_hooks</c>, for details.</p> + <p> + Note also that the Test Server framework callback + function <c>report/2</c> has been modified. This change + only affects users with test frameworks interfacing Test + Server rather than Common Test. See the + <c>test_server_ctrl</c> reference manual for details.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-11732 Aux Id: seq12541 </p> + </item> + <item> + <p> + If Common Test can't prompt the user to abort or continue + the test run when one or more test suites fail to + compile, a new option, + <c>{abort_if_missing_suites,Bool}</c>, can be used to + specify whether it should proceed with the test run, or + stop execution. The default value of <c>Bool</c> is + <c>false</c> (i.e. to proceed even if suites are + missing).</p> + <p> + Own Id: OTP-11769</p> + </item> + </list> + </section> + + + <section><title>Known Bugs and Problems</title> + <list> + <item> + <p> + common_test: Fix problems reported by Dialyzer.</p> + <p> + Own Id: OTP-11525</p> + </item> + </list> + </section> + +</section> + <section><title>Common_Test 1.7.4</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/common_test/doc/src/ref_man.xml b/lib/common_test/doc/src/ref_man.xml index 2f5c892e60..c266b70d00 100644 --- a/lib/common_test/doc/src/ref_man.xml +++ b/lib/common_test/doc/src/ref_man.xml @@ -78,6 +78,7 @@ <xi:include href="unix_telnet.xml"/> <xi:include href="ct_slave.xml"/> <xi:include href="ct_hooks.xml"/> + <xi:include href="ct_property_test.xml"/> </application> diff --git a/lib/common_test/doc/src/run_test_chapter.xml b/lib/common_test/doc/src/run_test_chapter.xml index a4a77ee400..864f82cb63 100644 --- a/lib/common_test/doc/src/run_test_chapter.xml +++ b/lib/common_test/doc/src/run_test_chapter.xml @@ -59,7 +59,15 @@ <p>If compilation should fail for one or more suites, the compilation errors are printed to tty and the operator is asked if the test run should proceed without the missing suites, or be aborted. If the operator chooses to proceed, - it is noted in the HTML log which tests have missing suites.</p> + it is noted in the HTML log which tests have missing suites. If Common Test is + unable to prompt the user after compilation failure (if Common Test doesn't + control stdin), the test run will proceed automatically without the missing + suites. This behaviour can however be modified with the + <c><![CDATA[ct_run]]></c> flag <c><![CDATA[-abort_if_missing_suites]]></c>, + or the <c><![CDATA[ct:run_test/1]]></c> option + <c><![CDATA[{abort_if_missing_suites,TrueOrFalse}]]></c>. If + <c><![CDATA[abort_if_missing_suites]]></c> is set (to true), the test run + will stop immediately if some suites fail to compile.</p> <p>Any help module (i.e. regular Erlang module with name not ending with "_SUITE") that resides in the same test object directory as a suite @@ -167,6 +175,7 @@ <seealso marker="ct_hooks_chapter#builtin_cths">Built-in Common Test Hooks</seealso>. Default is <c>true</c>.</item> <item><c><![CDATA[-include]]></c>, specifies include directories (see above).</item> <item><c><![CDATA[-no_auto_compile]]></c>, disables the automatic test suite compilation feature (see above).</item> + <item><c><![CDATA[-abort_if_missing_suites]]></c>, aborts the test run if one or more suites fail to compile (see above).</item> <item><c><![CDATA[-multiply_timetraps <n>]]></c>, extends <seealso marker="write_test_chapter#timetraps">timetrap timeout</seealso> values.</item> <item><c><![CDATA[-scale_timetraps <bool>]]></c>, enables automatic <seealso marker="write_test_chapter#timetraps">timetrap @@ -589,8 +598,8 @@ Common Test will either execute one test run per specification file, or join the files and perform all tests within one single test run. The first behaviour is the default one. The latter requires that the start - flag/option <c>join_suites</c> is provided, e.g. - <c>run_test -spec ./my_tests1.ts ./my_tests2.ts -join_suites</c>.</p> + flag/option <c>join_specs</c> is provided, e.g. + <c>run_test -spec ./my_tests1.ts ./my_tests2.ts -join_specs</c>.</p> <p>Joining a number of specifications, or running them separately, can also be accomplished with (and may be combined with) test specification @@ -744,6 +753,9 @@ {auto_compile, Bool}, {auto_compile, NodeRefs, Bool}, + {abort_if_missing_suites, Bool}, + {abort_if_missing_suites, NodeRefs, Bool}, + {config, ConfigFiles}. {config, ConfigDir, ConfigBaseNames}. {config, NodeRefs, ConfigFiles}. diff --git a/lib/common_test/priv/run_test.in b/lib/common_test/priv/run_test.in index 1508751e4f..1508751e4f 100755..100644 --- a/lib/common_test/priv/run_test.in +++ b/lib/common_test/priv/run_test.in diff --git a/lib/common_test/src/Makefile b/lib/common_test/src/Makefile index 4600c0ad78..8d74546880 100644 --- a/lib/common_test/src/Makefile +++ b/lib/common_test/src/Makefile @@ -74,7 +74,8 @@ MODULES= \ ct_netconfc \ ct_conn_log_h \ cth_conn_log \ - ct_groups + ct_groups \ + ct_property_test TARGET_MODULES= $(MODULES:%=$(EBIN)/%) BEAM_FILES= $(MODULES:%=$(EBIN)/%.$(EMULATOR)) diff --git a/lib/common_test/src/common_test.app.src b/lib/common_test/src/common_test.app.src index e28751fb59..580d5dbd7b 100644 --- a/lib/common_test/src/common_test.app.src +++ b/lib/common_test/src/common_test.app.src @@ -1,7 +1,7 @@ % This is an -*- erlang -*- file. %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009-2012. All Rights Reserved. +%% Copyright Ericsson AB 2009-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -64,7 +64,7 @@ {applications, [kernel,stdlib]}, {env, []}, {runtime_dependencies,["xmerl-1.3.7","webtool-0.8.10","tools-2.6.14", - "test_server-3.7","stdlib-2.0","ssh-3.0.1", + "test_server-3.7.1","stdlib-2.0","ssh-3.0.1", "snmp-4.25.1","sasl-2.4","runtime_tools-1.8.14", "kernel-3.0","inets-5.10","erts-6.0", "debugger-4.0","crypto-3.3","compiler-5.0"]}]}. diff --git a/lib/common_test/src/ct.erl b/lib/common_test/src/ct.erl index e6732f7fc7..85afdc7834 100644 --- a/lib/common_test/src/ct.erl +++ b/lib/common_test/src/ct.erl @@ -150,7 +150,8 @@ run(TestDirs) -> %%% {silent_connections,Conns} | {stylesheet,CSSFile} | %%% {cover,CoverSpecFile} | {cover_stop,Bool} | {step,StepOpts} | %%% {event_handler,EventHandlers} | {include,InclDirs} | -%%% {auto_compile,Bool} | {create_priv_dir,CreatePrivDir} | +%%% {auto_compile,Bool} | {abort_if_missing_suites,Bool} | +%%% {create_priv_dir,CreatePrivDir} | %%% {multiply_timetraps,M} | {scale_timetraps,Bool} | %%% {repeat,N} | {duration,DurTime} | {until,StopTime} | %%% {force_stop,ForceStop} | {decrypt,DecryptKeyOrFile} | @@ -772,7 +773,7 @@ comment(Format, Args) when is_list(Format), is_list(Args) -> send_html_comment(Comment) -> Html = "<font color=\"green\">" ++ Comment ++ "</font>", - ct_util:set_testdata({comment,Html}), + ct_util:set_testdata({{comment,group_leader()},Html}), test_server:comment(Html). %%%----------------------------------------------------------------- diff --git a/lib/common_test/src/ct_cover.erl b/lib/common_test/src/ct_cover.erl index ae671c750a..c7f446dee9 100644 --- a/lib/common_test/src/ct_cover.erl +++ b/lib/common_test/src/ct_cover.erl @@ -47,18 +47,21 @@ add_nodes(Nodes) -> undefined -> {error,cover_not_running}; _ -> - {File,Nodes0,Import,Export,AppInfo} = ct_util:get_testdata(cover), + Nodes0 = cover:which_nodes(), Nodes1 = [Node || Node <- Nodes, lists:member(Node,Nodes0) == false], ct_logs:log("COVER INFO", "Adding nodes to cover test: ~w", [Nodes1]), case cover:start(Nodes1) of - Result = {ok,_} -> - ct_util:set_testdata({cover,{File,Nodes1++Nodes0, - Import,Export,AppInfo}}), - + Result = {ok,StartedNodes} -> + ct_logs:log("COVER INFO", + "Successfully added nodes to cover test: ~w", + [StartedNodes]), Result; Error -> + ct_logs:log("COVER INFO", + "Failed to add nodes to cover test: ~tp", + [Error]), Error end end. @@ -81,19 +84,20 @@ remove_nodes(Nodes) -> undefined -> {error,cover_not_running}; _ -> - {File,Nodes0,Import,Export,AppInfo} = ct_util:get_testdata(cover), + Nodes0 = cover:which_nodes(), ToRemove = [Node || Node <- Nodes, lists:member(Node,Nodes0)], ct_logs:log("COVER INFO", - "Removing nodes from cover test: ~w", [ToRemove]), + "Removing nodes from cover test: ~w", [ToRemove]), case cover:stop(ToRemove) of ok -> - Nodes1 = lists:foldl(fun(N,Deleted) -> - lists:delete(N,Deleted) - end, Nodes0, ToRemove), - ct_util:set_testdata({cover,{File,Nodes1, - Import,Export,AppInfo}}), + ct_logs:log("COVER INFO", + "Successfully removed nodes from cover test.", + []), ok; Error -> + ct_logs:log("COVER INFO", + "Failed to remove nodes from cover test: ~tp", + [Error]), Error end end. @@ -124,20 +128,20 @@ get_spec(File) -> catch get_spec_test(File). get_spec_test(File) -> - FullName = filename:absname(File), - case filelib:is_file(FullName) of + Dir = filename:dirname(File), % always abs path in here, set in ct_run + case filelib:is_file(File) of true -> - case file:consult(FullName) of + case file:consult(File) of {ok,Terms} -> Import = case lists:keysearch(import, 1, Terms) of {value,{_,Imps=[S|_]}} when is_list(S) -> ImpsFN = lists:map(fun(F) -> - filename:absname(F) + filename:absname(F,Dir) end, Imps), test_files(ImpsFN, ImpsFN); {value,{_,Imp=[IC|_]}} when is_integer(IC) -> - ImpFN = filename:absname(Imp), + ImpFN = filename:absname(Imp,Dir), test_files([ImpFN], [ImpFN]); _ -> [] @@ -145,11 +149,11 @@ get_spec_test(File) -> Export = case lists:keysearch(export, 1, Terms) of {value,{_,Exp=[EC|_]}} when is_integer(EC) -> - filename:absname(Exp); + filename:absname(Exp,Dir); {value,{_,[Exp]}} -> - filename:absname(Exp); + filename:absname(Exp,Dir); _ -> - [] + undefined end, Nodes = case lists:keysearch(nodes, 1, Terms) of @@ -175,7 +179,7 @@ get_spec_test(File) -> E; [CoverSpec] -> CoverSpec1 = remove_excludes_and_dups(CoverSpec), - {FullName,Nodes,Import,Export,CoverSpec1}; + {File,Nodes,Import,Export,CoverSpec1}; _ -> {error,multiple_apps_in_cover_spec} end; @@ -186,7 +190,7 @@ get_spec_test(File) -> {error,{invalid_cover_spec,Error}} end; false -> - {error,{cant_read_cover_spec_file,FullName}} + {error,{cant_read_cover_spec_file,File}} end. collect_apps([{level,Level}|Ts], Apps) -> diff --git a/lib/common_test/src/ct_framework.erl b/lib/common_test/src/ct_framework.erl index 7d577462b0..e8ea7992b4 100644 --- a/lib/common_test/src/ct_framework.erl +++ b/lib/common_test/src/ct_framework.erl @@ -657,7 +657,18 @@ end_tc(Mod,Func,TCPid,Result,Args,Return) -> _ -> ok end, - ct_util:delete_testdata(comment), + if Func == end_per_group; Func == end_per_suite -> + %% clean up any saved comments + ct_util:match_delete_testdata({comment,'_'}); + true -> + %% attemp to delete any saved comment for this TC + case process_info(TCPid, group_leader) of + {group_leader,TCGL} -> + ct_util:delete_testdata({comment,TCGL}); + _ -> + ok + end + end, ct_util:delete_suite_data(last_saved_config), FuncSpec = group_or_func(Func,Args), @@ -850,7 +861,7 @@ error_notification(Mod,Func,_Args,{Error,Loc}) -> _ -> %% this notification comes from the test case process, so %% we can add error info to comment with test_server:comment/1 - case ct_util:get_testdata(comment) of + case ct_util:get_testdata({comment,group_leader()}) of undefined -> test_server:comment(ErrorHtml); Comment -> @@ -1233,38 +1244,7 @@ report(What,Data) -> ct_logs:make_all_suites_index({TestName,RunDir}), ok; tests_start -> - case ct_util:get_testdata(cover) of - undefined -> - ok; - {_CovFile,_CovNodes,CovImport,CovExport,_CovAppData} -> - %% Always import cover data from files specified by CovImport - %% if no CovExport defined. If CovExport is defined, only - %% import from CovImport files initially, then use CovExport - %% to pass coverdata between proceeding tests (in the same run). - Imps = - case CovExport of - [] -> % don't export data between tests - CovImport; - _ -> - case filelib:is_file(CovExport) of - true -> - [CovExport]; - false -> - CovImport - end - end, - lists:foreach( - fun(Imp) -> - case cover:import(Imp) of - ok -> - ok; - {error,Reason} -> - ct_logs:log("COVER INFO", - "Importing cover data from: ~ts fails! " - "Reason: ~p", [Imp,Reason]) - end - end, Imps) - end; + ok; tests_done -> ok; severe_error -> @@ -1274,7 +1254,7 @@ report(What,Data) -> ct_util:set_testdata({What,Data}), ok; tc_start -> - %% Data = {Suite,{Func,GroupName}},LogFileName} + %% Data = {{Suite,{Func,GroupName}},LogFileName} Data1 = case Data of {{Suite,{Func,undefined}},LFN} -> {{Suite,Func},LFN}; _ -> Data diff --git a/lib/common_test/src/ct_logs.erl b/lib/common_test/src/ct_logs.erl index a4ad65c0a4..43eabb18d5 100644 --- a/lib/common_test/src/ct_logs.erl +++ b/lib/common_test/src/ct_logs.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2003-2013. All Rights Reserved. +%% Copyright Ericsson AB 2003-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -62,6 +62,7 @@ -define(totals_name, "totals.info"). -define(log_cache_name, "ct_log_cache"). -define(misc_io_log, "misc_io.log.html"). +-define(coverlog_name, "cover.html"). % must be same as in test_server_ctrl -define(table_color1,"#ADD8E6"). -define(table_color2,"#E4F0FE"). @@ -1304,7 +1305,8 @@ total_row(Success, Fail, UserSkip, AutoSkip, NotBuilt, All) -> "<td align=right>",integer_to_list(AllSkip), " (",UserSkipStr,"/",AutoSkipStr,")</td>\n", "<td align=right><b>",integer_to_list(NotBuilt),"<b></td>\n", - AllInfo, "</tr>\n</tfoot>\n"]. + AllInfo, "</tr>\n", + xhtml("","</tfoot>\n")]. not_built(_BaseName,_LogDir,_All,[]) -> 0; @@ -1368,6 +1370,19 @@ index_header(Label, StartTime) -> format_time(StartTime), {[],[1],[2,3,4,5]}) end, + Cover = + case filelib:is_regular(?abs(?coverlog_name)) of + true -> + xhtml(["<p><a href=\"",?coverlog_name, + "\">Cover Log</a></p><br>\n"], + ["<br />" + "<div id=\"button_holder\" class=\"btn\">\n" + "<a href=\"",?coverlog_name, + "\">COVER LOG</a>\n</div><br /><br />"]); + false -> + xhtml("<br>\n", "<br /><br /><br />\n") + end, + [Head | ["<center>\n", xhtml(["<p><a href=\"",?ct_log_name, @@ -1375,8 +1390,8 @@ index_header(Label, StartTime) -> ["<br />" "<div id=\"button_holder\" class=\"btn\">\n" "<a href=\"",?ct_log_name, - "\">COMMON TEST FRAMEWORK LOG</a>\n</div>"]), - xhtml("<br>\n", "<br /><br /><br />\n"), + "\">COMMON TEST FRAMEWORK LOG</a>\n</div><br>\n"]), + Cover, xhtml(["<table border=\"3\" cellpadding=\"5\" " "bgcolor=\"",?table_color3,"\">\n"], ["<table id=\"",?sortable_table_name,"\">\n", @@ -1519,7 +1534,8 @@ all_suites_index_footer() -> xhtml("<br><br>\n", "<br /><br />\n") | footer()]. all_runs_index_footer() -> - ["</tbody>\n</table>\n", + [xhtml("", "</tbody>\n"), + "</table>\n", "</center>\n", xhtml("<br><br>\n", "<br /><br />\n") | footer()]. @@ -1676,7 +1692,7 @@ config_table(Vars) -> config_table_header() -> [ xhtml(["<h2>Configuration</h2>\n" - "<table border=\"3\" cellpadding=\"5\" bgcolor=\"",?table_color1,"\"\n"], + "<table border=\"3\" cellpadding=\"5\" bgcolor=\"",?table_color1,"\">\n"], ["<h4>CONFIGURATION</h4>\n", "<table id=\"",?sortable_table_name,"\">\n", "<thead>\n"]), @@ -1692,7 +1708,7 @@ config_table1([{Key,Value}|Vars]) -> "<td>", io_lib:format("~p",[Value]), "</td>\n</tr>\n"]) | config_table1(Vars)]; config_table1([]) -> - ["</tbody>\n</table>\n"]. + [xhtml("","</tbody>\n"),"</table>\n"]. make_all_runs_index(When) -> @@ -1842,14 +1858,27 @@ dir_diff_all_runs(LogDirs=[Dir|Dirs], Cached=[CElem|CElems], LatestInCache, AllRunsDirs) -> DirDate = datestr_from_dirname(Dir), if DirDate > LatestInCache -> - %% Dir is a new run entry + %% Dir is a new run entry (not cached) dir_diff_all_runs(Dirs, Cached, LatestInCache, [Dir|AllRunsDirs]); DirDate == LatestInCache, CElems /= [] -> - %% Dir is an existing run entry + %% Dir is an existing (cached) run entry + + %% Only add the cached element instead of Dir if the totals + %% are "non-empty" (a test might be executing on a different + %% node and results haven't been saved yet) + ElemToAdd = + case CElem of + {_CDir,{_NodeStr,_Label,_Logs,{0,0,0,0,0}},_IxLink} -> + %% "empty" element in cache - this could be an + %% incomplete test and should be checked again + Dir; + _ -> + CElem + end, dir_diff_all_runs(Dirs, CElems, datestr_from_dirname(element(1,hd(CElems))), - [CElem|AllRunsDirs]); + [ElemToAdd|AllRunsDirs]); DirDate == LatestInCache, CElems == [] -> %% we're done, Dirs must all be new lists:reverse(Dirs)++[CElem|AllRunsDirs]; diff --git a/lib/common_test/src/ct_netconfc.erl b/lib/common_test/src/ct_netconfc.erl index 6fc840745d..a3861dc745 100644 --- a/lib/common_test/src/ct_netconfc.erl +++ b/lib/common_test/src/ct_netconfc.erl @@ -536,7 +536,7 @@ send(Client, SimpleXml) -> Client :: client(), SimpleXml :: simple_xml(), Timeout :: timeout(), - Result :: ok | {error,error_reason()}. + Result :: simple_xml() | {error,error_reason()}. %% @doc Send an XML document to the server. %% %% The given XML document is sent as is to the server. This function @@ -556,7 +556,7 @@ send_rpc(Client, SimpleXml) -> Client :: client(), SimpleXml :: simple_xml(), Timeout :: timeout(), - Result :: ok | {error,error_reason()}. + Result :: [simple_xml()] | {error,error_reason()}. %% @doc Send a Netconf <code>rpc</code> request to the server. %% %% The given XML document is wrapped in a valid Netconf @@ -635,7 +635,7 @@ get(Client, Filter) -> Client :: client(), Filter :: simple_xml() | xpath(), Timeout :: timeout(), - Result :: {ok,simple_xml()} | {error,error_reason()}. + Result :: {ok,[simple_xml()]} | {error,error_reason()}. %% @doc Get data. %% %% This operation returns both configuration and state data from the @@ -661,7 +661,7 @@ get_config(Client, Source, Filter) -> Source :: netconf_db(), Filter :: simple_xml() | xpath(), Timeout :: timeout(), - Result :: {ok,simple_xml()} | {error,error_reason()}. + Result :: {ok,[simple_xml()]} | {error,error_reason()}. %% @doc Get configuration data. %% %% To be able to access another source than `running', the server @@ -759,7 +759,7 @@ action(Client,Action) -> Client :: client(), Action :: simple_xml(), Timeout :: timeout(), - Result :: {ok,simple_xml()} | {error,error_reason()}. + Result :: {ok,[simple_xml()]} | {error,error_reason()}. %% @doc Execute an action. %% %% @end diff --git a/lib/common_test/src/ct_property_test.erl b/lib/common_test/src/ct_property_test.erl new file mode 100644 index 0000000000..52acda5388 --- /dev/null +++ b/lib/common_test/src/ct_property_test.erl @@ -0,0 +1,186 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2003-2014. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%%% %%% +%%% WARNING %%% +%%% %%% +%%% This is experimental code which may be changed or removed %%% +%%% anytime without any warning. %%% +%%% %%% +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + + +%%% @doc EXPERIMENTAL support in common-test for calling property based tests. +%%% +%%% <p>This module is a first step towards running Property Based testing in the +%%% Common Test framework. A property testing tool like QuickCheck or PropEr is +%%% assumed to be installed.</p> +%%% +%%% <p>The idea is to have a common_test testsuite calling a property testing +%%% tool with special property test suites as defined by that tool. In this manual +%%% we assume the usual Erlang Application directory structure. The tests are +%%% collected in the application's <c>test</c> directory. The test directory +%%% has a sub-directory called <c>property_test</c> where everything needed for +%%% the property tests are collected.</p> +%%% +%%% <p>A typical ct test suite using <c>ct_property_test</c> is organized as follows: +%%% </p> +%%% ``` +%%% -include_lib("common_test/include/ct.hrl"). +%%% +%%% all() -> [prop_ftp_case]. +%%% +%%% init_per_suite(Config) -> +%%% ct_property_test:init_per_suite(Config). +%%% +%%% %%%---- test case +%%% prop_ftp_case(Config) -> +%%% ct_property_test:quickcheck( +%%% ftp_simple_client_server:prop_ftp(Config), +%%% Config +%%% ). +%%% ''' +%%% +%%% <warning> +%%% <p> +%%% This is experimental code which may be changed or removed +%%% anytime without any warning. +%%% </p> +%%% </warning> +%%% +%%% @end + +-module(ct_property_test). + +%% API +-export([init_per_suite/1, + quickcheck/2]). + +-include_lib("common_test/include/ct.hrl"). + +%%%----------------------------------------------------------------- +%%% @spec init_per_suite(Config) -> Config | {skip,Reason} +%%% +%%% @doc Initializes Config for property testing. +%%% +%%% <p>The function investigates if support is available for either Quickcheck, PropEr, +%%% or Triq. +%%% The options <c>{property_dir,AbsPath}</c> and +%%% <c>{property_test_tool,Tool}</c> is set in the Config returned.</p> +%%% <p>The function is intended to be called in the init_per_suite in the test suite.</p> +%%% <p>The property tests are assumed to be in the subdirectory <c>property_test</c>.</p> +%%% @end + +init_per_suite(Config) -> + case which_module_exists([eqc,proper,triq]) of + {ok,ToolModule} -> + ct:pal("Found property tester ~p",[ToolModule]), + Path = property_tests_path("property_test", Config), + case compile_tests(Path,ToolModule) of + error -> + {fail, "Property test compilation failed in "++Path}; + up_to_date -> + add_code_pathz(Path), + [{property_dir,Path}, + {property_test_tool,ToolModule} | Config] + end; + + not_found -> + ct:pal("No property tester found",[]), + {skip, "No property testing tool found"} + end. + +%%%----------------------------------------------------------------- +%%% @spec quickcheck(Property, Config) -> true | {fail,Reason} +%%% +%%% @doc Call quickcheck and return the result in a form suitable for common_test. +%%% +%%% <p>The function is intended to be called in the test cases in the test suite.</p> +%%% @end + +quickcheck(Property, Config) -> + Tool = proplists:get_value(property_test_tool,Config), + F = function_name(quickcheck, Tool), + mk_ct_return( Tool:F(Property), Tool ). + + +%%%================================================================ +%%% +%%% Local functions +%%% + +%%% Make return values back to the calling Common Test suite +mk_ct_return(true, _Tool) -> + true; +mk_ct_return(Other, Tool) -> + try lists:last(hd(Tool:counterexample())) + of + {set,{var,_},{call,M,F,Args}} -> + {fail, io_lib:format("~p:~p/~p returned bad result",[M,F,length(Args)])} + catch + _:_ -> + {fail, Other} + end. + +%%% Check if a property testing tool is found +which_module_exists([Module|Modules]) -> + case module_exists(Module) of + true -> {ok,Module}; + false -> which_module_exists(Modules) + end; +which_module_exists(_) -> + not_found. + +module_exists(Module) -> + is_list(catch Module:module_info()). + +%%% The path to the property tests +property_tests_path(Dir, Config) -> + DataDir = proplists:get_value(data_dir, Config), + filename:join(lists:droplast(filename:split(DataDir))++[Dir]). + +%%% Extend the code path with Dir if it not already present +add_code_pathz(Dir) -> + case lists:member(Dir, code:get_path()) of + true -> ok; + false -> code:add_pathz(Dir) + end. + +compile_tests(Path, ToolModule) -> + MacroDefs = macro_def(ToolModule), + {ok,Cwd} = file:get_cwd(), + ok = file:set_cwd(Path), + {ok,FileNames} = file:list_dir("."), + BeamFiles = [F || F<-FileNames, + filename:extension(F) == ".beam"], + [file:delete(F) || F<-BeamFiles], + ct:pal("Compiling in ~p:~n Deleted ~p~n MacroDefs=~p",[Path,BeamFiles,MacroDefs]), + Result = make:all([load|MacroDefs]), + file:set_cwd(Cwd), + Result. + + +macro_def(eqc) -> [{d, 'EQC'}]; +macro_def(proper) -> [{d, 'PROPER'}]; +macro_def(triq) -> [{d, 'TRIQ'}]. + +function_name(quickcheck, triq) -> check; +function_name(F, _) -> F. + diff --git a/lib/common_test/src/ct_run.erl b/lib/common_test/src/ct_run.erl index 03cf06abed..00d0aab507 100644 --- a/lib/common_test/src/ct_run.erl +++ b/lib/common_test/src/ct_run.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2013. All Rights Reserved. +%% Copyright Ericsson AB 2004-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -1646,7 +1646,7 @@ do_run(Tests, Misc, LogDir, LogOpts) when is_list(Misc), do_run(Tests, [], Opts#opts{logdir = LogDir}, []); do_run(Tests, Skip, Opts, Args) when is_record(Opts, opts) -> - #opts{label = Label, profile = Profile, cover = Cover, + #opts{label = Label, profile = Profile, verbosity = VLvls} = Opts, %% label - used by ct_logs TestLabel = @@ -1670,22 +1670,6 @@ do_run(Tests, Skip, Opts, Args) when is_record(Opts, opts) -> non_existing -> {error,no_path_to_test_server}; _ -> - Opts1 = if Cover == undefined -> - Opts; - true -> - case ct_cover:get_spec(Cover) of - {error,Reason} -> - exit({error,Reason}); - CoverSpec -> - CoverStop = - case Opts#opts.cover_stop of - undefined -> true; - Stop -> Stop - end, - Opts#opts{coverspec = CoverSpec, - cover_stop = CoverStop} - end - end, %% This env variable is used by test_server to determine %% which framework it runs under. case os:getenv("TEST_SERVER_FRAMEWORK") of @@ -1711,7 +1695,7 @@ do_run(Tests, Skip, Opts, Args) when is_record(Opts, opts) -> _Pid -> ct_util:set_testdata({starter,Opts#opts.starter}), compile_and_run(Tests, Skip, - Opts1#opts{verbosity=Verbosity}, Args) + Opts#opts{verbosity=Verbosity}, Args) end end. @@ -2146,67 +2130,11 @@ check_and_add([{TestDir0,M,_} | Tests], Added, PA) -> check_and_add([], _, PA) -> {ok,PA}. -do_run_test(Tests, Skip, Opts) -> +do_run_test(Tests, Skip, Opts0) -> case check_and_add(Tests, [], []) of {ok,AddedToPath} -> ct_util:set_testdata({stats,{0,0,{0,0}}}), - ct_util:set_testdata({cover,undefined}), test_server_ctrl:start_link(local), - case Opts#opts.coverspec of - CovData={CovFile, - CovNodes, - _CovImport, - CovExport, - #cover{app = CovApp, - level = CovLevel, - excl_mods = CovExcl, - incl_mods = CovIncl, - cross = CovCross, - src = _CovSrc}} -> - ct_logs:log("COVER INFO", - "Using cover specification file: ~ts~n" - "App: ~w~n" - "Cross cover: ~w~n" - "Including ~w modules~n" - "Excluding ~w modules", - [CovFile,CovApp,CovCross, - length(CovIncl),length(CovExcl)]), - - %% cover export file will be used for export and import - %% between tests so make sure it doesn't exist initially - case filelib:is_file(CovExport) of - true -> - DelResult = file:delete(CovExport), - ct_logs:log("COVER INFO", - "Warning! " - "Export file ~ts already exists. " - "Deleting with result: ~p", - [CovExport,DelResult]); - false -> - ok - end, - - %% tell test_server which modules should be cover compiled - %% note that actual compilation is done when tests start - test_server_ctrl:cover(CovApp, CovFile, CovExcl, CovIncl, - CovCross, CovExport, CovLevel, - Opts#opts.cover_stop), - %% save cover data (used e.g. to add nodes dynamically) - ct_util:set_testdata({cover,CovData}), - %% start cover on specified nodes - if (CovNodes /= []) and (CovNodes /= undefined) -> - ct_logs:log("COVER INFO", - "Nodes included in cover " - "session: ~w", - [CovNodes]), - cover:start(CovNodes); - true -> - ok - end, - true; - _ -> - false - end, %% let test_server expand the test tuples and count no of cases {Suites,NoOfCases} = count_test_cases(Tests, Skip), @@ -2231,24 +2159,31 @@ do_run_test(Tests, Skip, Opts) -> end, %% if the verbosity level is set lower than ?STD_IMPORTANCE, tell %% test_server to ignore stdout printouts to the test case log file - case proplists:get_value(default, Opts#opts.verbosity) of + case proplists:get_value(default, Opts0#opts.verbosity) of VLvl when is_integer(VLvl), (?STD_IMPORTANCE < (100-VLvl)) -> test_server_ctrl:reject_io_reqs(true); _Lower -> ok end, - test_server_ctrl:multiply_timetraps(Opts#opts.multiply_timetraps), - test_server_ctrl:scale_timetraps(Opts#opts.scale_timetraps), + test_server_ctrl:multiply_timetraps(Opts0#opts.multiply_timetraps), + test_server_ctrl:scale_timetraps(Opts0#opts.scale_timetraps), test_server_ctrl:create_priv_dir(choose_val( - Opts#opts.create_priv_dir, + Opts0#opts.create_priv_dir, auto_per_run)), + + {ok,LogDir} = ct_logs:get_log_dir(true), + {TsCoverInfo,Opts} = maybe_start_cover(Opts0, LogDir), + ct_event:notify(#event{name=start_info, node=node(), data={NoOfTests,NoOfSuites,NoOfCases}}), CleanUp = add_jobs(Tests, Skip, Opts, []), unlink(whereis(test_server_ctrl)), catch test_server_ctrl:wait_finish(), + + maybe_stop_cover(Opts, TsCoverInfo, LogDir), + %% check if last testcase has left a "dead" trace window %% behind, and if so, kill it case ct_util:get_testdata(interpret) of @@ -2281,6 +2216,102 @@ do_run_test(Tests, Skip, Opts) -> exit(Error) end. +maybe_start_cover(Opts=#opts{cover=Cover,cover_stop=CoverStop0},LogDir) -> + if Cover == undefined -> + {undefined,Opts}; + true -> + case ct_cover:get_spec(Cover) of + {error,Reason} -> + exit({error,Reason}); + CoverSpec -> + CoverStop = + case CoverStop0 of + undefined -> true; + Stop -> Stop + end, + start_cover(Opts#opts{coverspec=CoverSpec, + cover_stop=CoverStop}, + LogDir) + end + end. + +start_cover(Opts=#opts{coverspec=CovData,cover_stop=CovStop},LogDir) -> + {CovFile, + CovNodes, + CovImport, + _CovExport, + #cover{app = CovApp, + level = CovLevel, + excl_mods = CovExcl, + incl_mods = CovIncl, + cross = CovCross, + src = _CovSrc}} = CovData, + ct_logs:log("COVER INFO", + "Using cover specification file: ~ts~n" + "App: ~w~n" + "Cross cover: ~w~n" + "Including ~w modules~n" + "Excluding ~w modules", + [CovFile,CovApp,CovCross, + length(CovIncl),length(CovExcl)]), + + %% Tell test_server to print a link in its coverlog + %% pointing to the real coverlog which will be written in + %% maybe_stop_cover/2 + test_server_ctrl:cover({log,LogDir}), + + %% Cover compile all modules + {ok,TsCoverInfo} = test_server_ctrl:cover_compile(CovApp,CovFile, + CovExcl,CovIncl, + CovCross,CovLevel, + CovStop), + ct_logs:log("COVER INFO", + "Compilation completed - test_server cover info: ~tp", + [TsCoverInfo]), + + %% start cover on specified nodes + if (CovNodes /= []) and (CovNodes /= undefined) -> + ct_logs:log("COVER INFO", + "Nodes included in cover " + "session: ~w", + [CovNodes]), + cover:start(CovNodes); + true -> + ok + end, + lists:foreach( + fun(Imp) -> + case cover:import(Imp) of + ok -> + ok; + {error,Reason} -> + ct_logs:log("COVER INFO", + "Importing cover data from: ~ts fails! " + "Reason: ~p", [Imp,Reason]) + end + end, CovImport), + {TsCoverInfo,Opts}. + +maybe_stop_cover(_,undefined,_) -> + ok; +maybe_stop_cover(#opts{coverspec=CovData},TsCoverInfo,LogDir) -> + {_CovFile, + _CovNodes, + _CovImport, + CovExport, + _AppData} = CovData, + case CovExport of + undefined -> ok; + _ -> + ct_logs:log("COVER INFO","Exporting cover data to ~tp",[CovExport]), + cover:export(CovExport) + end, + ct_logs:log("COVER INFO","Analysing cover data to ~tp",[LogDir]), + test_server_ctrl:cover_analyse(TsCoverInfo,LogDir), + ct_logs:log("COVER INFO","Analysis completed.",[]), + ok. + + delete_dups([S | Suites]) -> Suites1 = lists:delete(S, Suites), [S | delete_dups(Suites1)]; diff --git a/lib/common_test/src/ct_telnet.erl b/lib/common_test/src/ct_telnet.erl index 0a067b3a08..3b2652d06c 100644 --- a/lib/common_test/src/ct_telnet.erl +++ b/lib/common_test/src/ct_telnet.erl @@ -46,100 +46,80 @@ %% %% == Logging == %% -%% `ct_telnet' can be configured to uses the `error_logger' for logging telnet -%% traffic. A special purpose error handler is implemented in -%% `ct_conn_log_h'. To use this error handler, add the `cth_conn_log' -%% hook in your test suite, e.g: -%% +%% The default logging behaviour of `ct_telnet' is to print information +%% to the test case HTML log about performed operations and commands +%% and their corresponding results. What won't be printed to the HTML log +%% are text strings sent from the telnet server that are not explicitly +%% received by means of a `ct_telnet' function such as `expect/3'. +%% `ct_telnet' may however be configured to use a special purpose event handler, +%% implemented in `ct_conn_log_h', for logging <b>all</b> telnet traffic. +%% To use this handler, you need to install a Common Test hook named +%% `cth_conn_log'. Example (using the test suite info function): %% %% ``` %% suite() -> -%% [{ct_hooks, [{cth_conn_log, [{conn_mod(),hook_options()}]}]}]. -%%''' +%% [{ct_hooks, [{cth_conn_log, [{conn_mod(),hook_options()}]}]}]. +%% ''' %% %% `conn_mod()' is the name of the common_test module implementing %% the connection protocol, i.e. `ct_telnet'. %% -%% The hook option `log_type' specifies the type of logging: -%% -%% <dl> -%% <dt>`raw'</dt> -%% <dd>The sent and received telnet data is logged to a separate -%% text file as is, without any formatting. A link to the file is -%% added to the test case HTML log.</dd> +%% The `cth_conn_log' hook performs unformatted logging of telnet data to +%% a separate text file. All telnet communication is captured and printed, +%% including arbitrary data sent from the server. The link to this text file +%% can be found on the top of the test case HTML log. %% -%% <dt>`html (default)'</dt> -%% <dd>The sent and received telnet traffic is pretty printed -%% directly in the test case HTML log.</dd> -%% -%% <dt>`silent'</dt> -%% <dd>Telnet traffic is not logged.</dd> -%% </dl> -%% -%% By default, all telnet traffic is logged in one single log -%% file. However, it is possible to have different connections logged -%% in separate files. To do this, use the hook option `hosts' and -%% list the names of the servers/connections that will be used in the -%% suite. Note that the connections must be named for this to work +%% By default, data for all telnet connections is logged in one common +%% file (named `default'), which might get messy e.g. if multiple telnet +%% sessions are running in parallel. It is therefore possible to create a +%% separate log file for each connection. To configure this, use the hook +%% option `hosts' and list the names of the servers/connections that will be +%% used in the suite. Note that the connections must be named for this to work %% (see the `open' function below). %% -%% The `hosts' option has no effect if `log_type' is set to `html' or -%% `silent'. -%% -%% The hook options can also be specified in a configuration file with -%% the configuration variable `ct_conn_log': +%% The hook option named `log_type' may be used to change the `cth_conn_log' +%% behaviour. The default value of this option is `raw', which results in the +%% behaviour described above. If the value is set to `html', all telnet +%% communication is printed to the test case HTML log instead. %% -%% ``` -%% {ct_conn_log,[{conn_mod(),hook_options()}]}. -%% ''' -%% -%% For example: +%% All `cth_conn_log' hook options described above can also be specified in +%% a configuration file with the configuration variable `ct_conn_log'. Example: %% %% ``` -%% {ct_conn_log,[{ct_telnet,[{log_type,raw}, -%% {hosts,[key_or_name()]}]}]} +%% {ct_conn_log, [{ct_telnet,[{log_type,raw}, +%% {hosts,[key_or_name()]}]}]} %% ''' %% %% <b>Note</b> that hook options specified in a configuration file -%% will overwrite any hardcoded hook options in the test suite. +%% will overwrite any hardcoded hook options in the test suite! %% -%% === Logging example 1 === +%% === Logging example === %% -%% The following `ct_hooks' statement will cause raw printing of -%% telnet traffic to separate logs for the connections named -%% `server1' and `server2'. Any other connections will be logged -%% to default telnet log. +%% The following `ct_hooks' statement will cause printing of telnet traffic +%% to separate logs for the connections named `server1' and `server2'. +%% Traffic for any other connections will be logged in the default telnet log. %% %% ``` %% suite() -> -%% [{ct_hooks, [{cth_conn_log, [{ct_telnet,[{log_type,raw}}, -%% {hosts,[server1,server2]}]} -%% ]}]}]. +%% [{ct_hooks, +%% [{cth_conn_log, [{ct_telnet,[{hosts,[server1,server2]}]}]}]}]. %%''' %% -%% === Logging example 2 === -%% -%% The following configuration file will cause raw logging of all -%% telnet traffic into one single text file. +%% As previously explained, the above specification could also be provided +%% by means of an entry like this in a configuration file: %% %% ``` -%% {ct_conn_log,[{ct_telnet,[{log_type,raw}]}]}. +%% {ct_conn_log, [{ct_telnet,[{hosts,[server1,server2]}]}]}. %% ''' %% -%% The `ct_hooks' statement must look like this: +%% in which case the `ct_hooks' statement in the test suite may simply look +%% like this: %% %% ``` %% suite() -> -%% [{ct_hooks, [{cth_conn_log, []}]}]. +%% [{ct_hooks, [{cth_conn_log, []}]}]. %% ''' %% -%% The same `ct_hooks' statement without the configuration file would -%% cause HTML logging of all telnet connections into the test case -%% HTML log. -%% -%% <b>Note</b> that if the `cth_conn_log' hook is not added, telnet -%% traffic is still logged in the test case HTML log file (on the legacy -%% `ct_telnet' format). %% @end %% @type connection_type() = telnet | ts1 | ts2 @@ -205,6 +185,7 @@ open(Name) -> %%% Name = target_name() %%% ConnType = ct_telnet:connection_type() %%% Handle = ct_telnet:handle() +%%% Reason = term() %%% %%% @doc Open a telnet connection to the specified target host. open(Name,ConnType) -> @@ -234,6 +215,7 @@ open(KeyOrName,ConnType,TargetMod) -> %%% TargetMod = atom() %%% Extra = term() %%% Handle = handle() +%%% Reason = term() %%% %%% @doc Open a telnet connection to the specified target host. %%% @@ -295,7 +277,8 @@ open(KeyOrName,ConnType,TargetMod,Extra) -> %%%----------------------------------------------------------------- %%% @spec close(Connection) -> ok | {error,Reason} -%%% Connection = ct_telnet:connection() +%%% Connection = ct_telnet:connection() +%%% Reason = term() %%% %%% @doc Close the telnet connection and stop the process managing it. %%% @@ -308,7 +291,7 @@ close(Connection) -> {ok,Pid} -> log(undefined,close,"Connection closed, handle: ~w",[Pid]), case ct_gen_conn:stop(Pid) of - {error,{process_down,Pid,noproc}} -> + {error,{process_down,Pid,_}} -> {error,already_closed}; Result -> Result @@ -330,6 +313,7 @@ cmd(Connection,Cmd) -> %%% Cmd = string() %%% Timeout = integer() %%% Data = [string()] +%%% Reason = term() %%% @doc Send a command via telnet and wait for prompt. cmd(Connection,Cmd,Timeout) -> case get_handle(Connection) of @@ -350,6 +334,7 @@ cmdf(Connection,CmdFormat,Args) -> %%% Args = list() %%% Timeout = integer() %%% Data = [string()] +%%% Reason = term() %%% @doc Send a telnet command and wait for prompt %%% (uses a format string and list of arguments to build the command). cmdf(Connection,CmdFormat,Args,Timeout) when is_list(Args) -> @@ -360,6 +345,7 @@ cmdf(Connection,CmdFormat,Args,Timeout) when is_list(Args) -> %%% @spec get_data(Connection) -> {ok,Data} | {error,Reason} %%% Connection = ct_telnet:connection() %%% Data = [string()] +%%% Reason = term() %%% @doc Get all data which has been received by the telnet client %%% since last command was sent. get_data(Connection) -> @@ -374,6 +360,7 @@ get_data(Connection) -> %%% @spec send(Connection,Cmd) -> ok | {error,Reason} %%% Connection = ct_telnet:connection() %%% Cmd = string() +%%% Reason = term() %%% @doc Send a telnet command and return immediately. %%% %%% <p>The resulting output from the command can be read with @@ -391,6 +378,7 @@ send(Connection,Cmd) -> %%% Connection = ct_telnet:connection() %%% CmdFormat = string() %%% Args = list() +%%% Reason = term() %%% @doc Send a telnet command and return immediately (uses a format %%% string and a list of arguments to build the command). sendf(Connection,CmdFormat,Args) when is_list(Args) -> @@ -616,9 +604,12 @@ handle_msg({cmd,Cmd,Timeout},State) -> end_gen_log(), {Return,State#state{buffer=NewBuffer,prompt=Prompt}}; handle_msg({send,Cmd},State) -> + start_gen_log(heading(send,State#state.name)), log(State,send,"Sending: ~p",[Cmd]), + debug_cont_gen_log("Throwing Buffer:",[]), debug_log_lines(State#state.buffer), + case {State#state.type,State#state.prompt} of {ts,_} -> silent_teln_expect(State#state.name, @@ -638,6 +629,7 @@ handle_msg({send,Cmd},State) -> ok end, ct_telnet_client:send_data(State#state.teln_pid,Cmd), + end_gen_log(), {ok,State#state{buffer=[],prompt=false}}; handle_msg(get_data,State) -> start_gen_log(heading(get_data,State#state.name)), @@ -765,8 +757,8 @@ check_if_prompt_was_reached(Data,_) when is_list(Data) -> check_if_prompt_was_reached(_,_) -> false. -%%% @hidden -%% Functions for logging ct_telnet reports and telnet data +%%%----------------------------------------------------------------- +%%% Functions for logging ct_telnet reports and telnet data heading(Action,undefined) -> io_lib:format("~w ~w",[?MODULE,Action]); @@ -776,6 +768,8 @@ heading(Action,Name) -> force_log(State,Action,String,Args) -> log(State,Action,String,Args,true). +%%%----------------------------------------------------------------- +%%% @hidden log(State,Action,String,Args) when is_record(State, state) -> log(State,Action,String,Args,false); log(Name,Action,String,Args) when is_atom(Name) -> @@ -783,6 +777,8 @@ log(Name,Action,String,Args) when is_atom(Name) -> log(TelnPid,Action,String,Args) when is_pid(TelnPid) -> log(#state{teln_pid=TelnPid},Action,String,Args,false). +%%%----------------------------------------------------------------- +%%% @hidden log(undefined,String,Args) -> log(#state{},undefined,String,Args,false); log(Name,String,Args) when is_atom(Name) -> @@ -790,6 +786,8 @@ log(Name,String,Args) when is_atom(Name) -> log(TelnPid,String,Args) when is_pid(TelnPid) -> log(#state{teln_pid=TelnPid},undefined,String,Args). +%%%----------------------------------------------------------------- +%%% @hidden log(#state{name=Name,teln_pid=TelnPid,host=Host,port=Port}, Action,String,Args,ForcePrint) -> Name1 = if Name == undefined -> get({ct_telnet_pid2name,TelnPid}); @@ -839,6 +837,8 @@ log(#state{name=Name,teln_pid=TelnPid,host=Host,port=Port}, end end. +%%%----------------------------------------------------------------- +%%% @hidden start_gen_log(Heading) -> %% check if output is suppressed case ct_util:is_silenced(telnet) of @@ -846,6 +846,8 @@ start_gen_log(Heading) -> false -> ct_gen_conn:start_log(Heading) end. +%%%----------------------------------------------------------------- +%%% @hidden end_gen_log() -> %% check if output is suppressed case ct_util:is_silenced(telnet) of @@ -871,14 +873,13 @@ teln_cmd(Pid,Cmd,Prx,Timeout) -> teln_receive_until_prompt(Pid,Prx,Timeout). teln_get_all_data(Pid,Prx,Data,Acc,LastLine) -> - case check_for_prompt(Prx,lists:reverse(LastLine) ++ Data) of + case check_for_prompt(Prx,LastLine++Data) of {prompt,Lines,_PromptType,Rest} -> teln_get_all_data(Pid,Prx,Rest,[Lines|Acc],[]); {noprompt,Lines,LastLine1} -> case ct_telnet_client:get_data(Pid) of {ok,[]} -> - {ok,lists:reverse(lists:append([Lines|Acc])), - lists:reverse(LastLine1)}; + {ok,lists:reverse(lists:append([Lines|Acc])),LastLine1}; {ok,Data1} -> teln_get_all_data(Pid,Prx,Data1,[Lines|Acc],LastLine1) end @@ -1336,7 +1337,7 @@ teln_receive_until_prompt(Pid,Prx,Timeout) -> teln_receive_until_prompt(Pid,Prx,Acc,LastLine) -> {ok,Data} = ct_telnet_client:get_data(Pid), - case check_for_prompt(Prx,LastLine ++ Data) of + case check_for_prompt(Prx,LastLine++Data) of {prompt,Lines,PromptType,Rest} -> Return = lists:reverse(lists:append([Lines|Acc])), {ok,Return,PromptType,Rest}; diff --git a/lib/common_test/src/ct_util.erl b/lib/common_test/src/ct_util.erl index f5eb3a72f0..56027586d1 100644 --- a/lib/common_test/src/ct_util.erl +++ b/lib/common_test/src/ct_util.erl @@ -37,7 +37,7 @@ save_suite_data_async/3, save_suite_data_async/2, read_suite_data/1, delete_suite_data/0, delete_suite_data/1, match_delete_suite_data/1, - delete_testdata/0, delete_testdata/1, + delete_testdata/0, delete_testdata/1, match_delete_testdata/1, set_testdata/1, get_testdata/1, get_testdata/2, set_testdata_async/1, update_testdata/2, update_testdata/3, set_verbosity/1, get_verbosity/1]). @@ -270,6 +270,9 @@ delete_testdata() -> delete_testdata(Key) -> call({delete_testdata, Key}). +match_delete_testdata(KeyPat) -> + call({match_delete_testdata, KeyPat}). + update_testdata(Key, Fun) -> update_testdata(Key, Fun, []). @@ -361,7 +364,25 @@ loop(Mode,TestData,StartDir) -> {{delete_testdata,Key},From} -> TestData1 = lists:keydelete(Key,1,TestData), return(From,ok), - loop(From,TestData1,StartDir); + loop(From,TestData1,StartDir); + {{match_delete_testdata,{Key1,Key2}},From} -> + %% handles keys with 2 elements + TestData1 = + lists:filter(fun({Key,_}) when not is_tuple(Key) -> + true; + ({Key,_}) when tuple_size(Key) =/= 2 -> + true; + ({{_,KeyB},_}) when Key1 == '_' -> + KeyB =/= Key2; + ({{KeyA,_},_}) when Key2 == '_' -> + KeyA =/= Key1; + (_) when Key1 == '_' ; Key2 == '_' -> + false; + (_) -> + true + end, TestData), + return(From,ok), + loop(From,TestData1,StartDir); {{set_testdata,New = {Key,_Val}},From} -> TestData1 = lists:keydelete(Key,1,TestData), return(From,ok), diff --git a/lib/common_test/src/cth_conn_log.erl b/lib/common_test/src/cth_conn_log.erl index 0e6c877c5d..1e60f2751e 100644 --- a/lib/common_test/src/cth_conn_log.erl +++ b/lib/common_test/src/cth_conn_log.erl @@ -91,12 +91,15 @@ merge_log_info([{Mod,ConfOpts}|ConfList],HookList) -> {value,{_,HookOpts},HL1} -> {ConfOpts ++ HookOpts, HL1} % ConfOpts overwrites HookOpts! end, - [{Mod,get_log_opts(Opts)} | merge_log_info(ConfList,HookList1)]; + [{Mod,get_log_opts(Mod,Opts)} | merge_log_info(ConfList,HookList1)]; merge_log_info([],HookList) -> - [{Mod,get_log_opts(Opts)} || {Mod,Opts} <- HookList]. + [{Mod,get_log_opts(Mod,Opts)} || {Mod,Opts} <- HookList]. -get_log_opts(Opts) -> - LogType = proplists:get_value(log_type,Opts,html), +get_log_opts(Mod,Opts) -> + DefaultLogType = if Mod == ct_telnet -> raw; + true -> html + end, + LogType = proplists:get_value(log_type,Opts,DefaultLogType), Hosts = proplists:get_value(hosts,Opts,[]), {LogType,Hosts}. diff --git a/lib/common_test/src/unix_telnet.erl b/lib/common_test/src/unix_telnet.erl index b05386a5ab..10666b979d 100644 --- a/lib/common_test/src/unix_telnet.erl +++ b/lib/common_test/src/unix_telnet.erl @@ -17,8 +17,8 @@ %% %CopyrightEnd% %% -%%% @doc Callback module for ct_telnet for talking telnet -%%% to a unix host. +%%% @doc Callback module for ct_telnet, for connecting to a telnet +%%% server on a unix host. %%% %%% <p>It requires the following entry in the config file:</p> %%% <pre> @@ -28,15 +28,15 @@ %%% {password,Password}, %%% {keep_alive,Bool}]}. % optional</pre> %%% -%%% <p>To talk telnet to the host specified by +%%% <p>To communicate via telnet to the host specified by %%% <code>HostNameOrIpAddress</code>, use the interface functions in -%%% <code>ct</code>, e.g. <code>open(Name), cmd(Name,Cmd), ...</code>.</p> +%%% <code>ct_telnet</code>, e.g. <code>open(Name), cmd(Name,Cmd), ...</code>.</p> %%% %%% <p><code>Name</code> is the name you allocated to the unix host in %%% your <code>require</code> statement. E.g.</p> -%%% <pre> suite() -> [{require,Name,{unix,[telnet,username,password]}}].</pre> +%%% <pre> suite() -> [{require,Name,{unix,[telnet]}}].</pre> %%% <p>or</p> -%%% <pre> ct:require(Name,{unix,[telnet,username,password]}).</pre> +%%% <pre> ct:require(Name,{unix,[telnet]}).</pre> %%% %%% <p>The "keep alive" activity (i.e. that Common Test sends NOP to the server %%% every 10 seconds if the connection is idle) may be enabled or disabled for one @@ -62,20 +62,18 @@ -define(prx,"login: |Password: |\\\$ |> "). %%%----------------------------------------------------------------- -%%% @hidden %%% @spec get_prompt_regexp() -> PromptRegexp %%% PromptRegexp = ct_telnet:prompt_regexp() %%% %%% @doc Callback for ct_telnet.erl. %%% -%%% <p>Return the prompt regexp for telnet connections to the -%%% interwatch instrument.</p> +%%% <p>Return a suitable regexp string that will match common +%%% prompts for users on unix hosts.</p> get_prompt_regexp() -> ?prx. %%%----------------------------------------------------------------- -%%% @hidden %%% @spec connect(ConnName,Ip,Port,Timeout,KeepAlive,Extra) -> %%% {ok,Handle} | {error,Reason} %%% ConnName = ct:target_name() @@ -83,14 +81,15 @@ get_prompt_regexp() -> %%% Port = integer() %%% Timeout = integer() %%% KeepAlive = bool() -%%% Extra = {Username,Password} +%%% Extra = ct:target_name() | {Username,Password} %%% Username = string() %%% Password = string() %%% Handle = ct_telnet:handle() +%%% Reason = term() %%% %%% @doc Callback for ct_telnet.erl. %%% -%%% <p>Setup telnet connection to a UNIX host.</p> +%%% <p>Setup telnet connection to a unix host.</p> connect(ConnName,Ip,Port,Timeout,KeepAlive,Extra) -> case Extra of {Username,Password} -> diff --git a/lib/common_test/test/Makefile b/lib/common_test/test/Makefile index 085f19d023..a0ac47f12a 100644 --- a/lib/common_test/test/Makefile +++ b/lib/common_test/test/Makefile @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 2008-2013. All Rights Reserved. +# Copyright Ericsson AB 2008-2014. All Rights Reserved. # # The contents of this file are subject to the Erlang Public License, # Version 1.1, (the "License"); you may not use this file except in @@ -61,6 +61,7 @@ MODULES= \ ct_snmp_SUITE \ ct_group_leader_SUITE \ ct_cover_SUITE \ + ct_cover_nomerge_SUITE \ ct_groups_search_SUITE \ ct_surefire_SUITE \ ct_telnet_SUITE diff --git a/lib/common_test/test/ct_cover_SUITE.erl b/lib/common_test/test/ct_cover_SUITE.erl index ec2680f664..87ba4ae1b9 100644 --- a/lib/common_test/test/ct_cover_SUITE.erl +++ b/lib/common_test/test/ct_cover_SUITE.erl @@ -76,7 +76,8 @@ all() -> cover_node_option, ct_cover_add_remove_nodes, otp_9956, - cross + cross, + export_import ]. %%-------------------------------------------------------------------- @@ -172,8 +173,8 @@ cross(Config) -> check_calls(Events2,1), %% Get the log dirs for each test and run cross cover analyse - [D11,D12] = lists:sort(get_run_dirs(Events1)), - [D21,D22] = lists:sort(get_run_dirs(Events2)), + [D11,D12] = lists:sort(get_log_dirs(Events1)), + [D21,D22] = lists:sort(get_log_dirs(Events2)), ct_cover:cross_cover_analyse(details,[{cross1,D11},{cross2,D21}]), ct_cover:cross_cover_analyse(details,[{cross1,D12},{cross2,D22}]), @@ -199,6 +200,20 @@ cross(Config) -> ok. +export_import(Config) -> + DataDir = ?config(data_dir,Config), + false = check_cover(Config), + CoverSpec1 = + default_cover_file_content() ++ [{export,"export_import.coverdata"}], + CoverFile1 = create_cover_file(export_import1,CoverSpec1,Config), + {ok,Events1} = run_test(export_import1,default,[{cover,CoverFile1}],Config), + check_calls(Events1,1), + CoverSpec2 = + default_cover_file_content() ++ [{import,"export_import.coverdata"}], + CoverFile2 = create_cover_file(export_import2,CoverSpec2,Config), + {ok,Events2} = run_test(export_import2,default,[{cover,CoverFile2}],Config), + check_calls(Events2,2), + ok. %%%----------------------------------------------------------------- %%% HELP FUNCTIONS @@ -267,18 +282,17 @@ check_cover(Node) when is_atom(Node) -> false end. -%% Get the log dir "run.<timestamp>" for all (both!) tests -get_run_dirs(Events) -> - [filename:dirname(TCLog) || +%% Get the log dir "ct_run.<timestamp>" for all (both!) tests +get_log_dirs(Events) -> + [LogDir || {ct_test_support_eh, - {event,tc_logfile,_Node, - {{?suite,init_per_suite},TCLog}}} <- Events]. + {event,start_logging,_Node,LogDir}} <- Events]. %% Check that each coverlog includes N calls to ?mod:foo/0 check_calls(Events,N) -> check_calls(Events,{?mod,foo,0},N). check_calls(Events,MFA,N) -> - CoverLogs = [filename:join(D,"all.coverdata") || D <- get_run_dirs(Events)], + CoverLogs = [filename:join(D,"all.coverdata") || D <- get_log_dirs(Events)], do_check_logs(CoverLogs,MFA,N). do_check_logs([CoverLog|CoverLogs],{Mod,_,_} = MFA,N) -> diff --git a/lib/common_test/test/ct_cover_nomerge_SUITE.erl b/lib/common_test/test/ct_cover_nomerge_SUITE.erl new file mode 100644 index 0000000000..8e2ee1b500 --- /dev/null +++ b/lib/common_test/test/ct_cover_nomerge_SUITE.erl @@ -0,0 +1,221 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2014. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% + +%%%------------------------------------------------------------------- +%%% File: ct_cover_nomerge_SUITE +%%% +%%% Description: +%%% Test code cover analysis support when merge_tests=false +%%% +%%%------------------------------------------------------------------- +-module(ct_cover_nomerge_SUITE). + +-compile(export_all). + +-include_lib("common_test/include/ct.hrl"). +-include_lib("common_test/include/ct_event.hrl"). + +-define(eh, ct_test_support_eh). +-define(mod, cover_test_mod). + +%%-------------------------------------------------------------------- +%% TEST SERVER CALLBACK FUNCTIONS +%%-------------------------------------------------------------------- + +%%-------------------------------------------------------------------- +%% Description: Since Common Test starts another Test Server +%% instance, the tests need to be performed on a separate node (or +%% there will be clashes with logging processes etc). +%%-------------------------------------------------------------------- +init_per_suite(Config) -> + case test_server:is_cover() of + true -> + {skip,"Test server is running cover already - skipping"}; + false -> + ct_test_support:init_per_suite(Config) + end. + +end_per_suite(Config) -> + ct_test_support:end_per_suite(Config). + +init_per_testcase(TestCase, Config) -> + ct_test_support:init_per_testcase(TestCase, Config). + +end_per_testcase(TestCase, Config) -> + try apply(?MODULE,TestCase,[cleanup,Config]) + catch error:undef -> ok + end, + ct_test_support:end_per_testcase(TestCase, Config). + +suite() -> [{ct_hooks,[ts_install_cth]}]. + +all() -> + [ + local, + remote, + remote_nostop + ]. + +%%-------------------------------------------------------------------- +%% TEST CASES +%%-------------------------------------------------------------------- + +local(Config) -> + DataDir = ?config(data_dir, Config), + Spec = filename:join(DataDir, "local.spec"), + CoverSpec = [{incl_mods,[?mod]}], + CoverFile = create_cover_file(local,CoverSpec,Config), + {Opts,ERPid} = setup([{spec,Spec},{label,local},{cover,CoverFile}], Config), + {ok,Events} = execute(local, local, Opts, ERPid, Config), + false = check_cover(Config), + check_calls(Events,2), + ok. + +remote(Config) -> + DataDir = ?config(data_dir, Config), + Spec = filename:join(DataDir, "remote.spec"), + %% extending some timers for slow test hosts + {ok,Node} = ct_slave:start(ct_nomerge,[{boot_timeout,15}, + {init_timeout,15}, + {startup_timeout,15}]), + + CoverSpec = [{nodes,[Node]}, + {incl_mods,[?mod]}], + CoverFile = create_cover_file(remote,CoverSpec,Config), + {Opts,ERPid} = setup([{spec,Spec},{label,remote},{cover,CoverFile}], Config), + {ok,Events} = execute(remote, remote, Opts, ERPid, Config), + false = check_cover(Config), + check_calls(Events,2), + ok. +remote(cleanup,_Config) -> + {ok,_} = ct_slave:stop(ct_nomerge), + ok. + +remote_nostop(Config) -> + DataDir = ?config(data_dir, Config), + Spec = filename:join(DataDir, "remote_nostop.spec"), + %% extending some timers for slow test hosts + {ok,Node} = ct_slave:start(ct_nomerge,[{boot_timeout,15}, + {init_timeout,15}, + {startup_timeout,15}]), + + CoverSpec = [{nodes,[Node]}, + {incl_mods,[?mod]}], + CoverFile = create_cover_file(remote_nostop,CoverSpec,Config), + {Opts,ERPid} = setup([{spec,Spec},{label,remote_nostop}, + {cover,CoverFile},{cover_stop,false}], + Config), + {ok,Events} = execute(remote_nostop, remote_nostop, Opts, ERPid, Config), + {true,[Node],[cover_test_mod]} = check_cover(Config), + check_calls(Events,2), + ok. +remote_nostop(cleanup,Config) -> + CtNode = ?config(ct_node,Config), + ok = rpc:call(CtNode,cover,stop,[]), + {ok,_} = ct_slave:stop(ct_nomerge), + ok. + + +%%%----------------------------------------------------------------- +%%% HELP FUNCTIONS +%%%----------------------------------------------------------------- +setup(Test, Config) -> + Opts0 = ct_test_support:get_opts(Config), + Level = ?config(trace_level, Config), + EvHArgs = [{cbm,ct_test_support},{trace_level,Level}], + Opts = Opts0 ++ [{event_handler,{?eh,EvHArgs}}|Test], + ERPid = ct_test_support:start_event_receiver(Config), + {Opts,ERPid}. + +execute(Name, Testcase, Opts, ERPid, Config) -> + ok = ct_test_support:run(Opts, Config), + Events = ct_test_support:get_events(ERPid, Config), + + ct_test_support:log_events(Name, + reformat(Events, ?eh), + ?config(priv_dir, Config), + Opts), + TestEvents = events_to_check(Testcase), + R = ct_test_support:verify_events(TestEvents, Events, Config), + {R,Events}. + +reformat(Events, EH) -> + ct_test_support:reformat(Events, EH). + +events_to_check(local) -> + events_to_check1(cover_nomerge_local_SUITE); +events_to_check(remote) -> + events_to_check1(cover_nomerge_remote_SUITE); +events_to_check(remote_nostop) -> + events_to_check1(cover_nomerge_remote_nostop_SUITE). +events_to_check1(Suite) -> + OneTest = + [{?eh,start_logging,{'DEF','RUNDIR'}}] ++ + [{?eh,tc_done,{Suite,t1,ok}}] ++ + [{?eh,tc_done,{Suite,t2,ok}}] ++ + [{?eh,stop_logging,[]}], + + %% 2 tests (ct:run_test + script_start) is default + OneTest ++ OneTest. + +check_cover(Config) when is_list(Config) -> + CTNode = proplists:get_value(ct_node, Config), + check_cover(CTNode); +check_cover(Node) when is_atom(Node) -> + case rpc:call(Node,test_server,is_cover,[]) of + true -> + {true, + rpc:call(Node,cover,which_nodes,[]), + rpc:call(Node,cover,modules,[])}; + false -> + false + end. + +%% Get the log dir "ct_run.<timestamp>" for all (both!) tests +get_log_dirs(Events) -> + [LogDir || + {ct_test_support_eh, + {event,start_logging,_Node,LogDir}} <- Events]. + +%% Check that each coverlog includes N calls to ?mod:foo/0 +check_calls(Events,N) -> + check_calls(Events,{?mod,foo,0},N). +check_calls(Events,MFA,N) -> + CoverLogs = [filename:join(D,"all.coverdata") || D <- get_log_dirs(Events)], + do_check_logs(CoverLogs,MFA,N). + +do_check_logs([CoverLog|CoverLogs],{Mod,_,_} = MFA,N) -> + {ok,_} = cover:start(), + ok = cover:import(CoverLog), + {ok,Calls} = cover:analyse(Mod,calls,function), + ok = cover:stop(), + {MFA,N} = lists:keyfind(MFA,1,Calls), + do_check_logs(CoverLogs,MFA,N); +do_check_logs([],_,_) -> + ok. + +create_cover_file(Filename,Terms,Config) -> + PrivDir = ?config(priv_dir,Config), + File = filename:join(PrivDir,Filename) ++ ".cover", + {ok,Fd} = file:open(File,[write]), + lists:foreach(fun(Term) -> + file:write(Fd,io_lib:format("~p.~n",[Term])) + end,Terms), + ok = file:close(Fd), + File. diff --git a/lib/common_test/test/ct_cover_nomerge_SUITE_data/cover_nomerge_local_SUITE.erl b/lib/common_test/test/ct_cover_nomerge_SUITE_data/cover_nomerge_local_SUITE.erl new file mode 100644 index 0000000000..e1fe3b5fc9 --- /dev/null +++ b/lib/common_test/test/ct_cover_nomerge_SUITE_data/cover_nomerge_local_SUITE.erl @@ -0,0 +1,63 @@ +%%-------------------------------------------------------------------- +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2014. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%%---------------------------------------------------------------------- +-module(cover_nomerge_local_SUITE). +-include_lib("common_test/include/ct.hrl"). + +-compile(export_all). + +%% Default timetrap timeout (set in init_per_testcase). +-define(default_timeout, ?t:minutes(1)). + +suite() -> + []. + +all() -> + [t1,t2]. + +init_per_suite(Config) -> + Config. + +end_per_suite(Config) -> + Config. + +init_per_testcase(_Case, Config) -> + Dog = test_server:timetrap(?default_timeout), + [{watchdog, Dog}|Config]. + +end_per_testcase(Case, Config) -> + Dog=?config(watchdog, Config), + test_server:timetrap_cancel(Dog), + ok. + +%%%----------------------------------------------------------------- +%%% Test cases +break(_Config) -> + test_server:break(""), + ok. + +t1(_Config) -> + cover_compiled = code:which(cover_test_mod), + ok = cover_test_mod:foo(), + ok. + +t2(_Config) -> + cover_compiled = code:which(cover_test_mod), + ok = cover_test_mod:foo(), + ok. diff --git a/lib/common_test/test/ct_cover_nomerge_SUITE_data/cover_nomerge_remote_SUITE.erl b/lib/common_test/test/ct_cover_nomerge_SUITE_data/cover_nomerge_remote_SUITE.erl new file mode 100644 index 0000000000..a77ae0c2db --- /dev/null +++ b/lib/common_test/test/ct_cover_nomerge_SUITE_data/cover_nomerge_remote_SUITE.erl @@ -0,0 +1,75 @@ +%%-------------------------------------------------------------------- +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2014. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%%---------------------------------------------------------------------- +-module(cover_nomerge_remote_SUITE). +-include_lib("common_test/include/ct.hrl"). + +-compile(export_all). + +%% Default timetrap timeout (set in init_per_testcase). +-define(default_timeout, ?t:minutes(1)). + +suite() -> + []. + +all() -> + [t1,t2]. + +init_per_suite(Config) -> + {ok,Host} = inet:gethostname(), + Node = list_to_atom("ct_nomerge@"++Host), + pong = net_adm:ping(Node), + +%% Include this row, and exclude the equivalent row in end_per_suite => +%% fails every now and then with missing data. Why? +%% ct_cover:remove_nodes([Node]), + ct_cover:add_nodes([Node]), + [{node,Node}|Config]. + +end_per_suite(Config) -> + Node = ?config(node,Config), + ct_cover:remove_nodes([Node]), + Config. + +init_per_testcase(_Case, Config) -> + Dog = test_server:timetrap(?default_timeout), + [{watchdog, Dog}|Config]. + +end_per_testcase(Case, Config) -> + Dog=?config(watchdog, Config), + test_server:timetrap_cancel(Dog), + ok. + +%%%----------------------------------------------------------------- +%%% Test cases +break(_Config) -> + test_server:break(""), + ok. + +t1(Config) -> + Node = ?config(node,Config), + cover_compiled = rpc:call(Node, code, which, [cover_test_mod]), + ok = rpc:call(Node, cover_test_mod, foo, []), + ok. + +t2(Config) -> + Node = ?config(node,Config), + cover_compiled = rpc:call(Node, code, which, [cover_test_mod]), + ok = rpc:call(Node, cover_test_mod, foo, []), + ok. diff --git a/lib/common_test/test/ct_cover_nomerge_SUITE_data/cover_nomerge_remote_nostop_SUITE.erl b/lib/common_test/test/ct_cover_nomerge_SUITE_data/cover_nomerge_remote_nostop_SUITE.erl new file mode 100644 index 0000000000..0b3159f2c3 --- /dev/null +++ b/lib/common_test/test/ct_cover_nomerge_SUITE_data/cover_nomerge_remote_nostop_SUITE.erl @@ -0,0 +1,68 @@ +%%-------------------------------------------------------------------- +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2014. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%%---------------------------------------------------------------------- +-module(cover_nomerge_remote_nostop_SUITE). +-include_lib("common_test/include/ct.hrl"). + +-compile(export_all). + +%% Default timetrap timeout (set in init_per_testcase). +-define(default_timeout, ?t:minutes(1)). + +suite() -> + []. + +all() -> + [t1,t2]. + +init_per_suite(Config) -> + {ok,Host} = inet:gethostname(), + Node = list_to_atom("ct_nomerge@"++Host), + pong = net_adm:ping(Node), + [{node,Node}|Config]. + +end_per_suite(Config) -> + Config. + +init_per_testcase(_Case, Config) -> + Dog = test_server:timetrap(?default_timeout), + [{watchdog, Dog}|Config]. + +end_per_testcase(Case, Config) -> + Dog=?config(watchdog, Config), + test_server:timetrap_cancel(Dog), + ok. + +%%%----------------------------------------------------------------- +%%% Test cases +break(_Config) -> + test_server:break(""), + ok. + +t1(Config) -> + Node = ?config(node,Config), + cover_compiled = rpc:call(Node, code, which, [cover_test_mod]), + ok = rpc:call(Node, cover_test_mod, foo, []), + ok. + +t2(Config) -> + Node = ?config(node,Config), + cover_compiled = rpc:call(Node, code, which, [cover_test_mod]), + ok = rpc:call(Node, cover_test_mod, foo, []), + ok. diff --git a/lib/common_test/test/ct_cover_nomerge_SUITE_data/cover_test_mod.erl b/lib/common_test/test/ct_cover_nomerge_SUITE_data/cover_test_mod.erl new file mode 100644 index 0000000000..d4f69452c3 --- /dev/null +++ b/lib/common_test/test/ct_cover_nomerge_SUITE_data/cover_test_mod.erl @@ -0,0 +1,4 @@ +-module(cover_test_mod). +-compile(export_all). +foo() -> + ok. diff --git a/lib/common_test/test/ct_cover_nomerge_SUITE_data/local.spec b/lib/common_test/test/ct_cover_nomerge_SUITE_data/local.spec new file mode 100644 index 0000000000..893c48b010 --- /dev/null +++ b/lib/common_test/test/ct_cover_nomerge_SUITE_data/local.spec @@ -0,0 +1,6 @@ +{merge_tests,false}. + +{alias,dir,"."}. + +{cases, dir, cover_nomerge_local_SUITE, [t1]}. +{cases, dir, cover_nomerge_local_SUITE, [t2]}. diff --git a/lib/common_test/test/ct_cover_nomerge_SUITE_data/remote.spec b/lib/common_test/test/ct_cover_nomerge_SUITE_data/remote.spec new file mode 100644 index 0000000000..78c4332270 --- /dev/null +++ b/lib/common_test/test/ct_cover_nomerge_SUITE_data/remote.spec @@ -0,0 +1,6 @@ +{merge_tests,false}. + +{alias,dir,"."}. + +{cases, dir, cover_nomerge_remote_SUITE, [t1]}. +{cases, dir, cover_nomerge_remote_SUITE, [t2]}. diff --git a/lib/common_test/test/ct_cover_nomerge_SUITE_data/remote_nostop.spec b/lib/common_test/test/ct_cover_nomerge_SUITE_data/remote_nostop.spec new file mode 100644 index 0000000000..049f586c72 --- /dev/null +++ b/lib/common_test/test/ct_cover_nomerge_SUITE_data/remote_nostop.spec @@ -0,0 +1,6 @@ +{merge_tests,false}. + +{alias,dir,"."}. + +{cases, dir, cover_nomerge_remote_nostop_SUITE, [t1]}. +{cases, dir, cover_nomerge_remote_nostop_SUITE, [t2]}. diff --git a/lib/common_test/test/ct_snmp_SUITE_data/snmp.cfg b/lib/common_test/test/ct_snmp_SUITE_data/snmp.cfg index 895e097de6..7ff356e49a 100644 --- a/lib/common_test/test/ct_snmp_SUITE_data/snmp.cfg +++ b/lib/common_test/test/ct_snmp_SUITE_data/snmp.cfg @@ -22,23 +22,23 @@ {agent_target_param_def,{data_dir_file,"target_params.conf"}}, {agent_vacm,{data_dir_file,"vacm.conf"}}]}. {snmp_app1,[{manager, [{config, [{verbosity, silence}]}, - {server,[{verbosity,silence}]}, - {net_if,[{verbosity,silence}]}, + {server,[{verbosity,log}]}, + {net_if,[{verbosity,log}]}, {versions,[v2]} ]}, {agent, [{config, [{verbosity, silence}]}, - {net_if,[{verbosity,silence}]}, + {net_if,[{verbosity,log}]}, {mib_server,[{verbosity,silence}]}, {local_db,[{verbosity,silence}]}, - {agent_verbosity,silence} + {agent_verbosity,log} ]}]}. {snmp_app2,[{manager, [{config, [{verbosity, silence}]}, - {server,[{verbosity,silence}]}, - {net_if,[{verbosity,silence}]} + {server,[{verbosity,log}]}, + {net_if,[{verbosity,log}]} ]}, {agent, [{config, [{verbosity, silence}]}, - {net_if,[{verbosity,silence}]}, + {net_if,[{verbosity,log}]}, {mib_server,[{verbosity,silence}]}, {local_db,[{verbosity,silence}]}, - {agent_verbosity,silence} + {agent_verbosity,log} ]}]}. diff --git a/lib/common_test/test/ct_snmp_SUITE_data/snmp_SUITE.erl b/lib/common_test/test/ct_snmp_SUITE_data/snmp_SUITE.erl index 16b2b5690c..e20832e1e7 100644 --- a/lib/common_test/test/ct_snmp_SUITE_data/snmp_SUITE.erl +++ b/lib/common_test/test/ct_snmp_SUITE_data/snmp_SUITE.erl @@ -1,7 +1,7 @@ %%-------------------------------------------------------------------- %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2012. All Rights Reserved. +%% Copyright Ericsson AB 2012-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -288,6 +288,9 @@ override_usm(Config) -> %% Check that usm.conf is overwritten {ok,MyUsm} = snmpa_conf:read_usm_config(DataDir), {ok,UsedUsm} = snmpa_conf:read_usm_config(ConfDir), + ct:pal( + "MyUsm = ~p~nUsedUsm = ~p", + [MyUsm, UsedUsm]), true = (MyUsm == UsedUsm), %% Check that the usm user is actually configured... @@ -304,6 +307,9 @@ override_standard(Config) -> %% Check that standard.conf is overwritten {ok,MyStandard} = snmpa_conf:read_standard_config(DataDir), {ok,UsedStandard} = snmpa_conf:read_standard_config(ConfDir), + ct:pal( + "MyStandard = ~p~nUsedStandard = ~p", + [MyStandard, UsedStandard]), true = (MyStandard == UsedStandard), %% Check that the values from standard.conf is actually configured... @@ -319,6 +325,9 @@ override_context(Config) -> %% Check that context.conf is overwritten {ok,MyContext} = snmpa_conf:read_context_config(DataDir), {ok,UsedContext} = snmpa_conf:read_context_config(ConfDir), + ct:pal( + "MyContext = ~p~nUsedContext = ~p", + [MyContext, UsedContext]), true = (MyContext == UsedContext), ok. @@ -330,6 +339,9 @@ override_community(Config) -> %% Check that community.conf is overwritten {ok,MyCommunity} = snmpa_conf:read_community_config(DataDir), {ok,UsedCommunity} = snmpa_conf:read_community_config(ConfDir), + ct:pal( + "MyCommunity = ~p~nUsedCommunity = ~p", + [MyCommunity, UsedCommunity]), true = (MyCommunity == UsedCommunity), ok. @@ -341,6 +353,9 @@ override_notify(Config) -> %% Check that notify.conf is overwritten {ok,MyNotify} = snmpa_conf:read_notify_config(DataDir), {ok,UsedNotify} = snmpa_conf:read_notify_config(ConfDir), + ct:pal( + "MyNotify = ~p~nUsedNotify = ~p", + [MyNotify, UsedNotify]), true = (MyNotify == UsedNotify), ok. @@ -352,6 +367,9 @@ override_target_addr(Config) -> %% Check that target_addr.conf is overwritten {ok,MyTargetAddr} = snmpa_conf:read_target_addr_config(DataDir), {ok,UsedTargetAddr} = snmpa_conf:read_target_addr_config(ConfDir), + ct:pal( + "MyTargetAddr = ~p~nUsedTargetAddr = ~p", + [MyTargetAddr, UsedTargetAddr]), true = (MyTargetAddr == UsedTargetAddr), ok. @@ -363,6 +381,9 @@ override_target_params(Config) -> %% Check that target_params.conf is overwritten {ok,MyTargetParams} = snmpa_conf:read_target_params_config(DataDir), {ok,UsedTargetParams} = snmpa_conf:read_target_params_config(ConfDir), + ct:pal( + "MyTargetParams = ~p~nUsedTargetParams = ~p", + [MyTargetParams, UsedTargetParams]), true = (MyTargetParams == UsedTargetParams), ok. @@ -374,6 +395,9 @@ override_vacm(Config) -> %% Check that vacm.conf is overwritten {ok,MyVacm} = snmpa_conf:read_vacm_config(DataDir), {ok,UsedVacm} = snmpa_conf:read_vacm_config(ConfDir), + ct:pal( + "MyVacm = ~p~nUsedVacm = ~p", + [MyVacm, UsedVacm]), true = (MyVacm == UsedVacm), ok. diff --git a/lib/common_test/test/ct_snmp_SUITE_data/snmp_SUITE_data/target_addr.conf b/lib/common_test/test/ct_snmp_SUITE_data/snmp_SUITE_data/target_addr.conf index d02672a074..d3ce2fa60e 100644 --- a/lib/common_test/test/ct_snmp_SUITE_data/snmp_SUITE_data/target_addr.conf +++ b/lib/common_test/test/ct_snmp_SUITE_data/snmp_SUITE_data/target_addr.conf @@ -1,2 +1,2 @@ -{"target1", snmpUDPDomain, [147,214,122,73], 5000, 1500, 3, "std_trap", "target_v3", "", [], 2048}. -{"target2", snmpUDPDomain, [147,214,122,73], 5000, 1500, 3, "std_inform", "target_v3", "", [], 2048}. +{"target1", snmpUDPDomain, {[147,214,122,73], 5000}, 1500, 3, "std_trap", "target_v3", "", [], 2048}. +{"target2", snmpUDPDomain, {[147,214,122,73], 5000}, 1500, 3, "std_inform", "target_v3", "", [], 2048}. diff --git a/lib/common_test/test/ct_telnet_SUITE.erl b/lib/common_test/test/ct_telnet_SUITE.erl index f5cff76fd1..84e69c2b54 100644 --- a/lib/common_test/test/ct_telnet_SUITE.erl +++ b/lib/common_test/test/ct_telnet_SUITE.erl @@ -180,14 +180,20 @@ telnet_config(unix_telnet, legacy) -> {ct_conn_log,[]}]; %% LogType same as GroupName telnet_config(unix_telnet, LogType) -> + LogTypeTerm = if LogType == raw -> []; + true -> [{log_type,LogType}] + end, [{unix, ct:get_config(unix)}, {ct_conn_log, - [{ct_telnet,[{log_type,LogType}, - {hosts,[telnet_server_conn1, - telnet_server_conn2, - telnet_server_conn3, - telnet_server_conn4]}]}]}]; + [{ct_telnet, LogTypeTerm ++ + [{hosts,[telnet_server_conn1, + telnet_server_conn2, + telnet_server_conn3, + telnet_server_conn4]}]}]}]; telnet_config(_, LogType) -> + LogTypeTerm = if LogType == raw -> []; + true -> [{log_type,LogType}] + end, [{unix,[{telnet,"localhost"}, {port, ?erl_telnet_server_port}, {username,?erl_telnet_server_user}, @@ -202,11 +208,11 @@ telnet_config(_, LogType) -> [{ct_conn_log,[]}]; true -> [{ct_conn_log, - [{ct_telnet,[{log_type,LogType}, - {hosts,[telnet_server_conn1, - telnet_server_conn2, - telnet_server_conn3, - telnet_server_conn4]}]}]}] + [{ct_telnet, LogTypeTerm ++ + [{hosts,[telnet_server_conn1, + telnet_server_conn2, + telnet_server_conn3, + telnet_server_conn4]}]}]}] end]. %%%----------------------------------------------------------------- diff --git a/lib/common_test/test/ct_telnet_SUITE_data/ct_telnet_own_server_SUITE.erl b/lib/common_test/test/ct_telnet_SUITE_data/ct_telnet_own_server_SUITE.erl index 0ee0525216..c0f79d0f10 100644 --- a/lib/common_test/test/ct_telnet_SUITE_data/ct_telnet_own_server_SUITE.erl +++ b/lib/common_test/test/ct_telnet_SUITE_data/ct_telnet_own_server_SUITE.erl @@ -16,7 +16,8 @@ suite() -> ]. all() -> - [expect, + [ + expect, expect_repeat, expect_sequence, expect_error_prompt, @@ -31,8 +32,10 @@ all() -> ignore_prompt_repeat, ignore_prompt_sequence, ignore_prompt_timeout, + large_string, server_speaks, - server_disconnects]. + server_disconnects + ]. groups() -> []. @@ -214,6 +217,41 @@ no_prompt_check_timeout(_) -> ok = ct_telnet:close(Handle), ok. +%% Check that it's possible to receive multiple chunks of data sent from +%% the server with one get_data call +large_string(_) -> + {ok, Handle} = ct_telnet:open(telnet_server_conn1), + String = "abcd efgh ijkl mnop qrst uvwx yz ", + BigString = lists:flatmap(fun(S) -> S end, + [String || _ <- lists:seq(1,10)]), + VerifyStr = [C || C <- BigString, C/=$ ], + + {ok,Data} = ct_telnet:cmd(Handle, "echo_sep "++BigString), + ct:log("[CMD] Received ~w chars: ~s", [length(lists:flatten(Data)),Data]), + VerifyStr = [C || C <- lists:flatten(Data), C/=$ , C/=$\r, C/=$\n, C/=$>], + + %% Test #1: With a long sleep value, all data gets gets buffered and + %% ct_telnet can receive it with one single request to ct_telnet_client. + %% Test #2: With a short sleep value, ct_telnet needs multiple calls to + %% ct_telnet_client to collect the data. This iterative operation should + %% yield the same result as the single request case. + + ok = ct_telnet:send(Handle, "echo_sep "++BigString), + timer:sleep(1000), + {ok,Data1} = ct_telnet:get_data(Handle), + ct:log("[GET DATA #1] Received ~w chars: ~s", + [length(lists:flatten(Data1)),Data1]), + VerifyStr = [C || C <- lists:flatten(Data1), C/=$ , C/=$\r, C/=$\n, C/=$>], + + ok = ct_telnet:send(Handle, "echo_sep "++BigString), + timer:sleep(50), + {ok,Data2} = ct_telnet:get_data(Handle), + ct:log("[GET DATA #2] Received ~w chars: ~s", [length(lists:flatten(Data2)),Data2]), + VerifyStr = [C || C <- lists:flatten(Data2), C/=$ , C/=$\r, C/=$\n, C/=$>], + + ok = ct_telnet:close(Handle), + ok. + %% The server says things. Manually check that it gets printed correctly %% in the general IO log. server_speaks(_) -> diff --git a/lib/common_test/test/telnet_server.erl b/lib/common_test/test/telnet_server.erl index ae56787819..1d341d6106 100644 --- a/lib/common_test/test/telnet_server.erl +++ b/lib/common_test/test/telnet_server.erl @@ -198,6 +198,14 @@ do_handle_data(Data,#state{authorized={user,_}}=State) -> do_handle_data("echo " ++ Data,State) -> send(Data++"\r\n> ",State), {ok,State}; +do_handle_data("echo_sep " ++ Data,State) -> + Msgs = string:tokens(Data," "), + lists:foreach(fun(Msg) -> + send(Msg,State), + timer:sleep(10) + end, Msgs), + send("\r\n> ",State), + {ok,State}; do_handle_data("echo_no_prompt " ++ Data,State) -> send(Data,State), {ok,State}; diff --git a/lib/common_test/vsn.mk b/lib/common_test/vsn.mk index f8a5aab686..00c0925b40 100644 --- a/lib/common_test/vsn.mk +++ b/lib/common_test/vsn.mk @@ -1 +1 @@ -COMMON_TEST_VSN = 1.8 +COMMON_TEST_VSN = 1.8.2 diff --git a/lib/compiler/doc/src/notes.xml b/lib/compiler/doc/src/notes.xml index cb39b28d3d..d48a0a5599 100644 --- a/lib/compiler/doc/src/notes.xml +++ b/lib/compiler/doc/src/notes.xml @@ -31,6 +31,272 @@ <p>This document describes the changes made to the Compiler application.</p> +<section><title>Compiler 5.0.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Corrected a bug with incorrect code generation when + inlining was turned on.</p> + <p> + Own Id: OTP-12132</p> + </item> + </list> + </section> + +</section> + +<section><title>Compiler 5.0.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + A Dialyzer crash involving analysis of Map types has now + been fixed.</p> + <p> + Own Id: OTP-11947</p> + </item> + <item> + <p>The compiler would fail to compile a file with a + latin-1 character in the false branch of an <c>-ifdef</c> + or <c>-indef</c>.</p> + <p> + Own Id: OTP-11987</p> + </item> + </list> + </section> + +</section> + +<section><title>Compiler 5.0</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Line numbers would not be correct when a binary + construction such as + '<c><<Bin/binary,...>></c>' fails. (Thanks to + Stanislav Seletskiy for reporting this bug.)</p> + <p> + Own Id: OTP-11572</p> + </item> + <item> + <p> + The compiler now properly annotates the code in value in + the '<c>after</c>' clause for a '<c>try</c>' so that + Dialyzer no longer generates a false warning for an + unmatched return.</p> + <p> + Own Id: OTP-11580</p> + </item> + <item> + <p> + Some case statements where no clause would match could + cause an internal error in the compiler. (Thanks to Erik + Soe Sorensen for reporting this bug.)</p> + <p> + Own Id: OTP-11610</p> + </item> + <item> + <p> + With <c>--Wunmatched_returns</c>, dialyzer will no longer + warn when the value of a list comprehension is ignored, + provided that the each value in the list would be an + atomic value (such as integer or atoms, as opposed to + tuples and lists). Example: ignoring '<c>[io:format(...) + || ...]</c>' will not cause a warning, while ignoring + '<c>[file:close(Fd) || ...]</c>' will.</p> + <p> + Own Id: OTP-11626</p> + </item> + <item> + <p> + Matching out a binary and applying the binary as if it + were a fun would crash the run-time system. (Thanks to + Loïc Hoguin.)</p> + <p> + Own Id: OTP-11672</p> + </item> + <item> + <p> + Some local implementations of removing the last element + from a list are replaced by <c>lists:droplast/1</c>. Note + that this requires at least <c>stdlib-2.0</c>, which is + the stdlib version delivered in OTP 17.0. (Thanks to Hans + Svensson)</p> + <p> + Own Id: OTP-11678</p> + </item> + <item> + <p> + Allow all auto imports to be suppressed at once. + Introducing the no_auto_import attribute: + -compile(no_auto_import). Useful for code generation + tools that always use the qualified function names and + want to avoid the auto imported functions clashing with + local ones. (Thanks to José Valim.)</p> + <p> + Own Id: OTP-11682</p> + </item> + <item> + <p> + Application upgrade (appup) files are corrected for the + following applications: </p> + <p> + <c>asn1, common_test, compiler, crypto, debugger, + dialyzer, edoc, eldap, erl_docgen, et, eunit, gs, hipe, + inets, observer, odbc, os_mon, otp_mibs, parsetools, + percept, public_key, reltool, runtime_tools, ssh, + syntax_tools, test_server, tools, typer, webtool, wx, + xmerl</c></p> + <p> + A new test utility for testing appup files is added to + test_server. This is now used by most applications in + OTP.</p> + <p> + (Thanks to Tobias Schlager)</p> + <p> + Own Id: OTP-11744</p> + </item> + <item> + <p> + Adapt 'asm' deprecation message to new version scheme. + (Thanks to Tuncer Ayaz)</p> + <p> + Own Id: OTP-11751</p> + </item> + <item> + <p> + A number of compiler errors where unusual or nonsensical + code would crash the compiler have been reported by Ulf + Norell and corrected by Anthony Ramine.</p> + <p> + Own Id: OTP-11770</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Compilation times for modules with a huge number for + record accesses using the dot operator has been improved.</p> + <p> + Own Id: OTP-10652</p> + </item> + <item> + <p> + The compiler can generate somewhat better code by moving + let expressions into sequences. (Thanks to Anthony + Ramine.)</p> + <p> + Own Id: OTP-11056</p> + </item> + <item> + <p> + Forbid unsized fields in patterns of binary generators + and simplified v3_core's translation of bit string + generators. (Thanks to Anthony Ramine.)</p> + <p> + Own Id: OTP-11186</p> + </item> + <item> + <p> + Funs can now be a given a name. Thanks to to Richard + O'Keefe for the idea (EEP37) and to Anthony Ramine for + the implementation.</p> + <p> + Own Id: OTP-11537</p> + </item> + <item> + <p> + Using the <c>from_asm</c> option to produce a BEAM file + starting from BEAM assembly code would often fail because + early optimization passes would not understand + instructions that later optimization passes would + introduce. (Thanks to Anthony Ramine.)</p> + <p> + Own Id: OTP-11544</p> + </item> + <item> + <p> + The <c>.core</c> and <c>.S</c> extensions are now + documented in the <c>erlc</c> documentation, and the + '<c>from_core</c>' and '<c>from_asm</c>' options are now + documented in the compiler documentation. (Thanks to + Tuncer Ayaz.)</p> + <p> + Own Id: OTP-11547</p> + </item> + <item> + <p>Optimization of case expressions that build tuples or + lists have been improved.</p> + <p> + Own Id: OTP-11584</p> + </item> + <item> + <p> + EEP43: New data type - Maps</p> + <p> + With Maps you may for instance: <taglist> <item><c>M0 = + #{ a => 1, b => 2}, % create + associations</c></item> <item><c>M1 = M0#{ a := 10 }, % + update values</c></item> <item><c>M2 = M1#{ "hi" => + "hello"}, % add new associations</c></item> <item><c>#{ + "hi" := V1, a := V2, b := V3} = M2. % match keys with + values</c></item> </taglist></p> + <p> + For information on how to use Maps please see Map Expressions in the + <seealso marker="doc/reference_manual:expressions#map_expressions"> + Reference Manual</seealso>.</p> + <p> + The current implementation is without the following + features: <taglist> <item>No variable keys</item> + <item>No single value access</item> <item>No map + comprehensions</item> </taglist></p> + <p> + Note that Maps is <em>experimental</em> during OTP 17.0.</p> + <p> + Own Id: OTP-11616</p> + </item> + <item> + <p> + Some function specs are corrected or moved and some edoc + comments are corrected in order to allow use of edoc. + (Thanks to Pierre Fenoll)</p> + <p> + Own Id: OTP-11702</p> + </item> + <item> + <p> + Thanks to Anthony Ramine for several improvements to the + optimizations in the BEAM compiler and for cleaning up + the code the code that transforms list and binary + comprehensions to Core Erlang.</p> + <p> + Own Id: OTP-11720</p> + </item> + <item> + <p> + The default encoding for Erlang source files is now + UTF-8. As a temporary measure to ease the transition from + the old default of latin-1, if the compiler encounters + byte sequences that are not valid UTF-8 sequences, the + compiler will re-try the compilation in latin-1 mode. + This workaround will be removed in a future release.</p> + <p> + Own Id: OTP-11791</p> + </item> + </list> + </section> + +</section> + <section><title>Compiler 4.9.4</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/compiler/src/beam_disasm.erl b/lib/compiler/src/beam_disasm.erl index 4bdfe4e0c2..c45596f236 100644 --- a/lib/compiler/src/beam_disasm.erl +++ b/lib/compiler/src/beam_disasm.erl @@ -1030,6 +1030,7 @@ resolve_inst({gc_bif2,Args},Imports,_,_) -> [F,Live,Bif,A1,A2,Reg] = resolve_args(Args), {extfunc,_Mod,BifName,_Arity} = lookup(Bif+1,Imports), {gc_bif,BifName,F,Live,[A1,A2],Reg}; + %% %% New instruction in R14, gc_bif with 3 arguments %% @@ -1146,21 +1147,17 @@ resolve_inst({put_map_assoc,Args},_,_,_) -> [FLbl,Src,Dst,{u,N},{{z,1},{u,_Len},List0}] = Args, List = resolve_args(List0), {put_map_assoc,FLbl,Src,Dst,N,{list,List}}; - resolve_inst({put_map_exact,Args},_,_,_) -> [FLbl,Src,Dst,{u,N},{{z,1},{u,_Len},List0}] = Args, List = resolve_args(List0), {put_map_exact,FLbl,Src,Dst,N,{list,List}}; - -resolve_inst({is_map,Args0},_,_,_) -> +resolve_inst({is_map=I,Args0},_,_,_) -> [FLbl|Args] = resolve_args(Args0), - {test, is_map, FLbl, Args}; - + {test,I,FLbl,Args}; resolve_inst({has_map_fields,Args0},_,_,_) -> [FLbl,Src,{{z,1},{u,_Len},List0}] = Args0, List = resolve_args(List0), {test,has_map_fields,FLbl,Src,{list,List}}; - resolve_inst({get_map_elements,Args0},_,_,_) -> [FLbl,Src,{{z,1},{u,_Len},List0}] = Args0, List = resolve_args(List0), diff --git a/lib/compiler/src/cerl.erl b/lib/compiler/src/cerl.erl index e400e4f185..9d6768b157 100644 --- a/lib/compiler/src/cerl.erl +++ b/lib/compiler/src/cerl.erl @@ -126,20 +126,18 @@ map_es/1, map_arg/1, update_c_map/3, + c_map/1, is_c_map_empty/1, ann_c_map/2, ann_c_map/3, map_pair_op/1,map_pair_key/1,map_pair_val/1, update_c_map_pair/4, + c_map_pair/2, ann_c_map_pair/4 ]). -export_type([c_binary/0, c_bitstr/0, c_call/0, c_clause/0, c_cons/0, c_fun/0, - c_literal/0, c_map_pair/0, c_module/0, c_tuple/0, + c_literal/0, c_map/0, c_map_pair/0, c_module/0, c_tuple/0, c_values/0, c_var/0, cerl/0, var_name/0]). -%% HiPE does not understand Maps -%% (guard functions is_map/1 and map_size/1 in ann_c_map/3) --compile(no_native). - -include("core_parse.hrl"). -type c_alias() :: #c_alias{}. @@ -1586,9 +1584,20 @@ map_es(#c_map{es = Es}) -> -spec map_arg(c_map()) -> c_map() | c_literal(). -map_arg(#c_map{arg = M}) -> +map_arg(#c_map{arg=M}) -> M. +-spec c_map([c_map_pair()]) -> c_map(). + +c_map(Pairs) -> + #c_map{es=Pairs}. + +-spec is_c_map_empty(c_map() | c_literal()) -> boolean(). + +is_c_map_empty(#c_map{ es=[] }) -> true; +is_c_map_empty(#c_literal{val=M}) when is_map(M),map_size(M) =:= 0 -> true; +is_c_map_empty(_) -> false. + -spec ann_c_map([term()], [cerl()]) -> c_map() | c_literal(). ann_c_map(As,Es) -> @@ -1648,6 +1657,11 @@ map_pair_key(#c_map_pair{key=K}) -> K. map_pair_val(#c_map_pair{val=V}) -> V. map_pair_op(#c_map_pair{op=Op}) -> Op. +-spec c_map_pair(cerl(), cerl()) -> c_map_pair(). + +c_map_pair(Key,Val) -> + #c_map_pair{op=#c_literal{val=assoc},key=Key,val=Val}. + -spec ann_c_map_pair([term()], cerl(), cerl(), cerl()) -> c_map_pair(). @@ -4249,6 +4263,9 @@ ann_make_tree(As, bitstr, [[V],[S],[U],[T],[Fs]]) -> ann_c_bitstr(As, V, S, U, T, Fs); ann_make_tree(As, cons, [[H], [T]]) -> ann_c_cons(As, H, T); ann_make_tree(As, tuple, [Es]) -> ann_c_tuple(As, Es); +ann_make_tree(As, map, [Es]) -> ann_c_map(As, Es); +ann_make_tree(As, map, [[A], Es]) -> ann_c_map(As, A, Es); +ann_make_tree(As, map_pair, [[Op], [K], [V]]) -> ann_c_map_pair(As, Op, K, V); ann_make_tree(As, 'let', [Vs, [A], [B]]) -> ann_c_let(As, Vs, A, B); ann_make_tree(As, seq, [[A], [B]]) -> ann_c_seq(As, A, B); ann_make_tree(As, apply, [[Op], Es]) -> ann_c_apply(As, Op, Es); diff --git a/lib/compiler/src/core_lib.erl b/lib/compiler/src/core_lib.erl index 93ec3bbad5..2792fd8fa5 100644 --- a/lib/compiler/src/core_lib.erl +++ b/lib/compiler/src/core_lib.erl @@ -59,7 +59,7 @@ is_lit_bin(Es) -> %% Return the value of LitExpr. -spec literal_value(cerl:c_literal() | cerl:c_binary() | - cerl:c_cons() | cerl:c_tuple()) -> term(). + cerl:c_map() | cerl:c_cons() | cerl:c_tuple()) -> term(). literal_value(#c_literal{val=V}) -> V; literal_value(#c_binary{segments=Es}) -> @@ -67,7 +67,14 @@ literal_value(#c_binary{segments=Es}) -> literal_value(#c_cons{hd=H,tl=T}) -> [literal_value(H)|literal_value(T)]; literal_value(#c_tuple{es=Es}) -> - list_to_tuple(literal_value_list(Es)). + list_to_tuple(literal_value_list(Es)); +literal_value(#c_map{arg=Cm,es=Cmps}) -> + M = literal_value(Cm), + lists:foldl(fun(#c_map_pair{ key=Ck, val=Cv },Mi) -> + K = literal_value(Ck), + V = literal_value(Cv), + maps:put(K,V,Mi) + end, M, Cmps). literal_value_list(Vals) -> [literal_value(V) || V <- Vals]. diff --git a/lib/compiler/src/core_pp.erl b/lib/compiler/src/core_pp.erl index a76327457d..83412ecdd7 100644 --- a/lib/compiler/src/core_pp.erl +++ b/lib/compiler/src/core_pp.erl @@ -120,7 +120,11 @@ format_1(#c_literal{anno=A,val=Bitstring}, Ctxt) when is_bitstring(Bitstring) -> format_1(#c_binary{anno=A,segments=Segs}, Ctxt); format_1(#c_literal{anno=A,val=M},Ctxt) when is_map(M) -> Pairs = maps:to_list(M), - Cpairs = [#c_map_pair{op=#c_literal{val=assoc}, + Op = case Ctxt of + #ctxt{ class = clause } -> exact; + _ -> assoc + end, + Cpairs = [#c_map_pair{op=#c_literal{val=Op}, key=#c_literal{val=V}, val=#c_literal{val=K}} || {K,V} <- Pairs], format_1(#c_map{anno=A,arg=#c_literal{val=#{}},es=Cpairs},Ctxt); diff --git a/lib/compiler/src/sys_core_fold.erl b/lib/compiler/src/sys_core_fold.erl index 52d6dfe184..82817a987a 100644 --- a/lib/compiler/src/sys_core_fold.erl +++ b/lib/compiler/src/sys_core_fold.erl @@ -1556,16 +1556,8 @@ map_pair_pattern_list(Ps0, Isub, Osub0) -> {Ps,Osub}. map_pair_pattern(#c_map_pair{op=#c_literal{val=exact},key=K0,val=V0}=Pair,{Isub,Osub0}) -> - {K,Osub1} = case cerl:type(K0) of - binary -> - K1 = eval_binary(K0), - case cerl:type(K1) of - literal -> {K1,Osub0}; - _ -> pattern(K0,Isub,Osub0) - end; - _ -> pattern(K0,Isub,Osub0) - end, - {V,Osub} = pattern(V0,Isub,Osub1), + K = expr(K0, Isub), + {V,Osub} = pattern(V0,Isub,Osub0), {Pair#c_map_pair{key=K,val=V},{Isub,Osub}}. bin_pattern_list(Ps0, Isub, Osub0) -> @@ -2031,9 +2023,9 @@ case_opt(Arg, Cs0, Sub) -> case_opt_args([A0|As0], Cs0, Sub, LitExpr, Acc) -> case case_opt_arg(A0, Sub, Cs0, LitExpr) of - error -> + {error,Cs1} -> %% Nothing to be done. Move on to the next argument. - Cs = [{Ps,C,[P|PsAcc],Bs} || {[P|Ps],C,PsAcc,Bs} <- Cs0], + Cs = [{Ps,C,[P|PsAcc],Bs} || {[P|Ps],C,PsAcc,Bs} <- Cs1], case_opt_args(As0, Cs, Sub, LitExpr, [A0|Acc]); {ok,As1,Cs} -> %% The argument was either expanded (from tuple/list) or @@ -2052,7 +2044,7 @@ case_opt_arg(E0, Sub, Cs, LitExpr) -> E = maybe_replace_var(E0, Sub), case cerl:is_data(E) of false -> - error; + {error,Cs}; true -> case cerl:data_type(E) of {atomic,_} -> @@ -2102,35 +2094,44 @@ maybe_replace_var_1(E, #sub{t=Tdb}) -> %% pattern matching is tricky, so we will give up in that case. case_opt_lit(Lit, Cs0, LitExpr) -> - try case_opt_lit_1(Cs0, Lit, LitExpr) of + Cs1 = case_opt_lit_1(Lit, Cs0, LitExpr), + try case_opt_lit_2(Lit, Cs1) of Cs -> {ok,[],Cs} catch throw:impossible -> - error + {error,Cs1} end. -case_opt_lit_1([{[P|Ps],C,PsAcc,Bs0}|Cs], E, LitExpr) -> +case_opt_lit_1(E, [{[P|_],C,_,_}=Current|Cs], LitExpr) -> + case cerl_clauses:match(P, E) of + none -> + %% The pattern will not match the literal. Remove the clause. + %% Unless the entire case expression is a literal, also + %% emit a warning. + case LitExpr of + false -> add_warning(C, nomatch_clause_type); + true -> ok + end, + case_opt_lit_1(E, Cs, LitExpr); + _ -> + [Current|case_opt_lit_1(E, Cs, LitExpr)] + end; +case_opt_lit_1(_, [], _) -> []. + +case_opt_lit_2(E, [{[P|Ps],C,PsAcc,Bs0}|Cs]) -> + %% Non-matching clauses have already been removed in case_opt_lit_1/3. case cerl_clauses:match(P, E) of - none -> - %% The pattern will not match the literal. Remove the clause. - %% Unless the entire case expression is a literal, also - %% emit a warning. - case LitExpr of - false -> add_warning(C, nomatch_clause_type); - true -> ok - end, - case_opt_lit_1(Cs, E, LitExpr); {true,Bs} -> %% The pattern matches the literal. Remove the pattern %% and update the bindings. - [{Ps,C,PsAcc,Bs++Bs0}|case_opt_lit_1(Cs, E, LitExpr)]; + [{Ps,C,PsAcc,Bs++Bs0}|case_opt_lit_2(E, Cs)]; {false,_} -> %% Binary literal and pattern. We are not sure whether %% the pattern will match. throw(impossible) end; -case_opt_lit_1([], _, _) -> []. +case_opt_lit_2(_, []) -> []. %% case_opt_data(Expr, Clauses0, LitExpr) -> {ok,Exprs,Clauses} @@ -2247,23 +2248,23 @@ letify(#c_var{name=Vname}=Var, Val, Body) -> %% opt_case_in_let(LetExpr) -> LetExpr' -opt_case_in_let(#c_let{vars=Vs,arg=Arg,body=B}=Let) -> - opt_case_in_let_0(Vs, Arg, B, Let). +opt_case_in_let(#c_let{vars=Vs,arg=Arg,body=B}=Let, Sub) -> + opt_case_in_let_0(Vs, Arg, B, Let, Sub). opt_case_in_let_0([#c_var{name=V}], Arg, - #c_case{arg=#c_var{name=V},clauses=Cs}=Case, Let) -> + #c_case{arg=#c_var{name=V},clauses=Cs}=Case, Let, Sub) -> case opt_case_in_let_1(V, Arg, Cs) of impossible -> case is_simple_case_arg(Arg) andalso not core_lib:is_var_used(V, Case#c_case{arg=#c_literal{val=nil}}) of true -> - expr(opt_bool_case(Case#c_case{arg=Arg,clauses=Cs}), sub_new()); + expr(opt_bool_case(Case#c_case{arg=Arg,clauses=Cs}), sub_new(Sub)); false -> Let end; Expr -> Expr end; -opt_case_in_let_0(_, _, _, Let) -> Let. +opt_case_in_let_0(_, _, _, Let, _) -> Let. opt_case_in_let_1(V, Arg, Cs) -> try @@ -2606,7 +2607,7 @@ opt_simple_let_2(Let0, Vs0, Arg0, Body0, effect, Sub) -> expr(#c_seq{arg=Arg,body=Body}, effect, sub_new_preserve_types(Sub)); true -> Let = Let0#c_let{vars=Vs,arg=Arg,body=Body}, - opt_case_in_let_arg(opt_case_in_let(Let), effect, Sub) + opt_case_in_let_arg(opt_case_in_let(Let, Sub), effect, Sub) end end; opt_simple_let_2(Let, Vs0, Arg0, Body, value, Sub) -> @@ -2629,7 +2630,7 @@ opt_simple_let_2(Let, Vs0, Arg0, Body, value, Sub) -> expr(#c_seq{arg=Arg,body=Body}, value, sub_new_preserve_types(Sub)); {Vs,Arg,Body} -> opt_case_in_let_arg( - opt_case_in_let(Let#c_let{vars=Vs,arg=Arg,body=Body}), + opt_case_in_let(Let#c_let{vars=Vs,arg=Arg,body=Body}, Sub), value, Sub) end. diff --git a/lib/compiler/src/sys_pre_expand.erl b/lib/compiler/src/sys_pre_expand.erl index 91a46a20fe..761ae8409c 100644 --- a/lib/compiler/src/sys_pre_expand.erl +++ b/lib/compiler/src/sys_pre_expand.erl @@ -232,7 +232,7 @@ pattern({map,Line,Ps}, St0) -> {TPs,St1} = pattern_list(Ps, St0), {{map,Line,TPs},St1}; pattern({map_field_exact,Line,K0,V0}, St0) -> - {K,St1} = pattern(K0, St0), + {K,St1} = expr(K0, St0), {V,St2} = pattern(V0, St1), {{map_field_exact,Line,K,V},St2}; %%pattern({struct,Line,Tag,Ps}, St0) -> diff --git a/lib/compiler/src/v3_codegen.erl b/lib/compiler/src/v3_codegen.erl index f1331d1fe7..47a357c23d 100644 --- a/lib/compiler/src/v3_codegen.erl +++ b/lib/compiler/src/v3_codegen.erl @@ -960,7 +960,6 @@ select_extract_map(Src, Vs, Fail, I, Vdb, Bef, St) -> end, {{[],[]},Bef}, Vs), Code = case {HasKs,GetVs} of - {[],[]} -> {[],Aft,St}; {HasKs,[]} -> [{test,has_map_fields,{f,Fail},Rsrc,{list,HasKs}}]; {[],GetVs} -> @@ -1553,8 +1552,7 @@ map_pair_strip_and_termsort(Es) -> Ls = [{K,V}||{_,K,V}<-Es], lists:sort(fun ({{_,A},_}, {{_,B},_}) -> erts_internal:cmp_term(A,B) =< 0; ({nil,_}, {{_,B},_}) -> [] =< B; - ({{_,A},_}, {nil,_}) -> A =< []; - ({nil,_}, {nil,_}) -> true + ({{_,A},_}, {nil,_}) -> A =< [] end, Ls). %%% diff --git a/lib/compiler/src/v3_core.erl b/lib/compiler/src/v3_core.erl index 04210ae243..83cf76f241 100644 --- a/lib/compiler/src/v3_core.erl +++ b/lib/compiler/src/v3_core.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2013. All Rights Reserved. +%% Copyright Ericsson AB 1999-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -82,6 +82,8 @@ -include("core_parse.hrl"). +-define(REC_OFFSET, 100000000). % Also in erl_expand_records. + %% Internal core expressions and help functions. %% N.B. annotations fields in place as normal Core expressions. @@ -501,7 +503,7 @@ expr({cons,L,H0,T0}, St0) -> {H1,Hps,St1} = safe(H0, St0), {T1,Tps,St2} = safe(T0, St1), A = lineno_anno(L, St2), - {ann_c_cons(A, H1, T1),Hps ++ Tps,St2}; + {annotate_cons(A, H1, T1, St2),Hps ++ Tps,St2}; expr({lc,L,E,Qs0}, St0) -> {Qs1,St1} = preprocess_quals(L, Qs0, St0), lc_tq(L, E, Qs1, #c_literal{anno=lineno_anno(L, St1),val=[]}, St1); @@ -509,20 +511,31 @@ expr({bc,L,E,Qs}, St) -> bc_tq(L, E, Qs, {nil,L}, St); expr({tuple,L,Es0}, St0) -> {Es1,Eps,St1} = safe_list(Es0, St0), - A = lineno_anno(L, St1), - {ann_c_tuple(A, Es1),Eps,St1}; + A = record_anno(L, St1), + {annotate_tuple(A, Es1, St1),Eps,St1}; expr({map,L,Es0}, St0) -> % erl_lint should make sure only #{ K => V } are allowed % in map construction. - {Es1,Eps,St1} = map_pair_list(Es0, St0), - A = lineno_anno(L, St1), - {ann_c_map(A,Es1),Eps,St1}; + try map_pair_list(Es0, St0) of + {Es1,Eps,St1} -> + A = lineno_anno(L, St1), + {ann_c_map(A,Es1),Eps,St1} + catch + throw:{bad_map,Warning} -> + St = add_warning(L, Warning, St0), + LineAnno = lineno_anno(L, St), + As = [#c_literal{anno=LineAnno,val=badarg}], + {#icall{anno=#a{anno=LineAnno}, %Must have an #a{} + module=#c_literal{anno=LineAnno,val=erlang}, + name=#c_literal{anno=LineAnno,val=error}, + args=As},[],St} + end; expr({map,L,M0,Es0}, St0) -> try expr_map(M0,Es0,lineno_anno(L, St0),St0) of {_,_,_}=Res -> Res catch - throw:bad_map -> - St = add_warning(L, bad_map, St0), + throw:{bad_map,Warning} -> + St = add_warning(L, Warning, St0), LineAnno = lineno_anno(L, St), As = [#c_literal{anno=LineAnno,val=badarg}], {#icall{anno=#a{anno=LineAnno}, %Must have an #a{} @@ -762,7 +775,7 @@ expr_map(M0,Es0,A,St0) -> {Es1,Eps,St2} = map_pair_list(Es0, St1), {ann_c_map(A,M1,Es1),Mps++Eps,St2} end; - false -> throw(bad_map) + false -> throw({bad_map,bad_map}) end. is_valid_map_src(#c_literal{val = M}) when is_map(M) -> true; @@ -774,18 +787,23 @@ map_pair_list(Es, St) -> foldr(fun ({map_field_assoc,L,K0,V0}, {Ces,Esp,St0}) -> {K,Ep0,St1} = safe(K0, St0), + ok = ensure_valid_map_key(K), {V,Ep1,St2} = safe(V0, St1), A = lineno_anno(L, St2), Pair = #c_map_pair{op=#c_literal{val=assoc},anno=A,key=K,val=V}, {[Pair|Ces],Ep0 ++ Ep1 ++ Esp,St2}; ({map_field_exact,L,K0,V0}, {Ces,Esp,St0}) -> {K,Ep0,St1} = safe(K0, St0), + ok = ensure_valid_map_key(K), {V,Ep1,St2} = safe(V0, St1), A = lineno_anno(L, St2), Pair = #c_map_pair{op=#c_literal{val=exact},anno=A,key=K,val=V}, {[Pair|Ces],Ep0 ++ Ep1 ++ Esp,St2} end, {[],[],St}, Es). +ensure_valid_map_key(#c_literal{}) -> ok; +ensure_valid_map_key(_) -> throw({bad_map,bad_map_key}). + %% try_exception([ExcpClause], St) -> {[ExcpVar],Handler,St}. try_exception(Ecs0, St0) -> @@ -1541,9 +1559,9 @@ pattern({atom,L,A}, St) -> #c_literal{anno=lineno_anno(L, St),val=A}; pattern({string,L,S}, St) -> #c_literal{anno=lineno_anno(L, St),val=S}; pattern({nil,L}, St) -> #c_literal{anno=lineno_anno(L, St),val=[]}; pattern({cons,L,H,T}, St) -> - ann_c_cons(lineno_anno(L, St), pattern(H, St), pattern(T, St)); + annotate_cons(lineno_anno(L, St), pattern(H, St), pattern(T, St), St); pattern({tuple,L,Ps}, St) -> - ann_c_tuple(lineno_anno(L, St), pattern_list(Ps, St)); + annotate_tuple(record_anno(L, St), pattern_list(Ps, St), St); pattern({map,L,Ps}, St) -> #c_map{anno=lineno_anno(L, St), es=pattern_map_pairs(Ps, St)}; pattern({bin,L,Ps}, St) -> @@ -1589,24 +1607,17 @@ pattern_alias_map_pair_patterns([Cv1,Cv2|Cvs]) -> pattern_alias_map_pair_patterns([pat_alias(Cv1,Cv2)|Cvs]). pattern_map_pair({map_field_exact,L,K,V}, St) -> - %% FIXME: Better way to construct literals? or missing case - %% {Key,_,_} = expr(K, St), - Key = case K of - {bin,L,Es0} -> - case constant_bin(Es0) of - error -> - throw(badmatch); - Bin -> - #c_literal{anno=lineno_anno(L,St),val=Bin} - end; + case expr(K,St) of + {#c_literal{}=Key,_,_} -> + #c_map_pair{anno=lineno_anno(L, St), + op=#c_literal{val=exact}, + key=Key, + val=pattern(V, St)}; _ -> - pattern(K,St) - end, - #c_map_pair{anno=lineno_anno(L, St), - op=#c_literal{val=exact}, - key=Key, - val=pattern(V, St)}. - + %% this will throw a cryptic error message + %% but it is better than nothing + throw(nomatch) + end. %% pat_bin([BinElement], State) -> [BinSeg]. @@ -1712,6 +1723,26 @@ fail_clause(Pats, Anno, Arg) -> body=[#iprimop{anno=#a{anno=Anno},name=#c_literal{val=match_fail}, args=[Arg]}]}. +annotate_tuple(A, Es, St) -> + case member(dialyzer, St#core.opts) of + true -> + %% Do not coalesce constant tuple elements. A Hack. + Node = cerl:ann_c_tuple(A, [cerl:c_var(any)]), + cerl:update_c_tuple_skel(Node, Es); + false -> + ann_c_tuple(A, Es) + end. + +annotate_cons(A, H, T, St) -> + case member(dialyzer, St#core.opts) of + true -> + %% Do not coalesce constant conses. A Hack. + Node= cerl:ann_c_cons(A, cerl:c_var(any), cerl:c_var(any)), + cerl:update_c_cons_skel(Node, H, T); + false -> + ann_c_cons(A, H, T) + end. + ubody(B, St) -> uexpr(B, [], St). %% uclauses([Lclause], [KnownVar], State) -> {[Lclause],State}. @@ -2229,6 +2260,23 @@ bitstr_vars(Segs, Vs) -> lit_vars(V, lit_vars(S, Vs0)) end, Vs, Segs). +record_anno(L, St) when L >= ?REC_OFFSET -> + case member(dialyzer, St#core.opts) of + true -> + [record | lineno_anno(L - ?REC_OFFSET, St)]; + false -> + lineno_anno(L, St) + end; +record_anno(L, St) when L < -?REC_OFFSET -> + case member(dialyzer, St#core.opts) of + true -> + [record | lineno_anno(L + ?REC_OFFSET, St)]; + false -> + lineno_anno(L, St) + end; +record_anno(L, St) -> + lineno_anno(L, St). + lineno_anno(L, St) -> {line, Line} = erl_parse:get_attribute(L, line), if @@ -2299,6 +2347,8 @@ format_error(nomatch) -> "pattern cannot possibly match"; format_error(bad_binary) -> "binary construction will fail because of a type mismatch"; +format_error(bad_map_key) -> + "map construction will fail because of none literal key (large binaries are not literals)"; format_error(bad_map) -> "map construction will fail because of a type mismatch". diff --git a/lib/compiler/src/v3_kernel.erl b/lib/compiler/src/v3_kernel.erl index d3b785aa14..40d2f72b4c 100644 --- a/lib/compiler/src/v3_kernel.erl +++ b/lib/compiler/src/v3_kernel.erl @@ -581,8 +581,7 @@ map_key_clean(#k_literal{val=V}) -> {k_literal,V}; map_key_clean(#k_int{val=V}) -> {k_int,V}; map_key_clean(#k_float{val=V}) -> {k_float,V}; map_key_clean(#k_atom{val=V}) -> {k_atom,V}; -map_key_clean(#k_nil{}) -> k_nil; -map_key_clean(#k_var{name=V}) -> {k_var,V}. +map_key_clean(#k_nil{}) -> k_nil. %% call_type(Module, Function, Arity) -> call | bif | apply | error. diff --git a/lib/compiler/src/v3_life.erl b/lib/compiler/src/v3_life.erl index c4f54a7970..cd4b5fd674 100644 --- a/lib/compiler/src/v3_life.erl +++ b/lib/compiler/src/v3_life.erl @@ -324,8 +324,7 @@ type(k_binary) -> binary; type(k_bin_seg) -> bin_seg; type(k_bin_int) -> bin_int; type(k_bin_end) -> bin_end; -type(k_map) -> map; -type(k_map_pair) -> map_pair. +type(k_map) -> map. %% variable(Klit) -> Lit. %% var_list([Klit]) -> [Lit]. diff --git a/lib/compiler/test/core_fold_SUITE.erl b/lib/compiler/test/core_fold_SUITE.erl index 9c986576d5..6a7036d728 100644 --- a/lib/compiler/test/core_fold_SUITE.erl +++ b/lib/compiler/test/core_fold_SUITE.erl @@ -23,7 +23,7 @@ t_element/1,setelement/1,t_length/1,append/1,t_apply/1,bifs/1, eq/1,nested_call_in_case/1,guard_try_catch/1,coverage/1, unused_multiple_values_error/1,unused_multiple_values/1, - multiple_aliases/1,redundant_boolean_clauses/1]). + multiple_aliases/1,redundant_boolean_clauses/1,mixed_matching_clauses/1]). -export([foo/0,foo/1,foo/2,foo/3]). @@ -40,7 +40,7 @@ groups() -> [t_element,setelement,t_length,append,t_apply,bifs, eq,nested_call_in_case,guard_try_catch,coverage, unused_multiple_values_error,unused_multiple_values, - multiple_aliases,redundant_boolean_clauses]}]. + multiple_aliases,redundant_boolean_clauses,mixed_matching_clauses]}]. init_per_suite(Config) -> @@ -373,5 +373,15 @@ redundant_boolean_clauses(Config) when is_list(Config) -> true -> yes end. +mixed_matching_clauses(Config) when is_list(Config) -> + 0 = case #{} of + #{} -> 0; + a -> 1 + end, + 0 = case <<>> of + <<>> -> 0; + a -> 1 + end, + ok. id(I) -> I. diff --git a/lib/compiler/test/map_SUITE.erl b/lib/compiler/test/map_SUITE.erl index 90eae6fb4f..403b7e8405 100644 --- a/lib/compiler/test/map_SUITE.erl +++ b/lib/compiler/test/map_SUITE.erl @@ -40,6 +40,8 @@ t_build_and_match_over_alloc/1, t_build_and_match_empty_val/1, t_build_and_match_val/1, + t_build_and_match_nil/1, + t_build_and_match_structure/1, %% errors in 17.0-rc1 t_update_values/1, @@ -68,6 +70,8 @@ all() -> [ t_build_and_match_over_alloc, t_build_and_match_empty_val, t_build_and_match_val, + t_build_and_match_nil, + t_build_and_match_structure, %% errors in 17.0-rc1 t_update_values, @@ -109,6 +113,10 @@ t_build_and_match_literals(Config) when is_list(Config) -> M = #{ map_1:=#{ map_2:=#{value_3 := third}, value_2:= second}, value_1:=first} = id(#{ map_1=>#{ map_2=>#{value_3 => third}, value_2=> second}, value_1=>first}), + %% map key + #{ #{} := 42 } = id(#{ #{} => 42 }), + #{ #{ "a" => 3 } := 42 } = id(#{ #{ "a" => 3} => 42 }), + %% nil key #{[]:=ok,1:=2} = id(#{[]=>ok,1=>2}), @@ -118,6 +126,8 @@ t_build_and_match_literals(Config) when is_list(Config) -> {'EXIT',{{badmatch,_},_}} = (catch (#{x:=3} = id({a,b,c}))), {'EXIT',{{badmatch,_},_}} = (catch (#{x:=3} = id(#{y=>3}))), {'EXIT',{{badmatch,_},_}} = (catch (#{x:=3} = id(#{x=>"three"}))), + {'EXIT',{badarg,_}} = (catch id(#{<<0:258>> =>"three"})), + {'EXIT',{{badmatch,_},_}} = (catch (#{#{"a"=>42} := 3}=id(#{#{"a"=>3}=>42}))), ok. t_build_and_match_aliasing(Config) when is_list(Config) -> @@ -234,7 +244,8 @@ t_update_assoc(Config) when is_list(Config) -> %% Errors cases. BadMap = id(badmap), {'EXIT',{badarg,_}} = (catch BadMap#{nonexisting=>val}), - + {'EXIT',{badarg,_}} = (catch <<>>#{nonexisting=>val}), + {'EXIT',{badarg,_}} = (catch M0#{<<0:257>> => val}), %% limitation ok. t_update_exact(Config) when is_list(Config) -> @@ -262,6 +273,8 @@ t_update_exact(Config) when is_list(Config) -> {'EXIT',{badarg,_}} = (catch M0#{1.0:=v,1.0=>v2}), {'EXIT',{badarg,_}} = (catch M0#{42.0:=v,42:=v2}), {'EXIT',{badarg,_}} = (catch M0#{42=>v1,42.0:=v2,42:=v3}), + {'EXIT',{badarg,_}} = (catch <<>>#{nonexisting:=val}), + {'EXIT',{badarg,_}} = (catch M0#{<<0:257>> := val}), %% limitation ok. t_update_values(Config) when is_list(Config) -> @@ -561,6 +574,32 @@ t_build_and_match_val(Config) when is_list(Config) -> test_server:fail({no_match, Other}) end. +t_build_and_match_nil(Config) when is_list(Config) -> + %% literals removed the coverage + V1 = id(cookie), + V2 = id(cake), + V3 = id(crisps), + + #{ [] := V1, "treat" := V2, {your,treat} := V3 } = id(#{ + {your,treat} => V3, + "treat" => V2, + [] => V1 }), + #{ [] := V3, [] := V3 } = id(#{ [] => V1, [] => V3 }), + ok. + +t_build_and_match_structure(Config) when is_list(Config) -> + V2 = id("it"), + S = id([42,{"hi", "=)", #{ "a" => 42, any => any, val => "get_" ++ V2}}]), + + %% match deep map values + V2 = case S of + [42,{"hi",_, #{ "a" := 42, val := "get_" ++ V1, any := _ }}] -> V1 + end, + %% match deep map + ok = case S of + [42,{"hi",_, #{ }}] -> ok + end, + ok. %% Use this function to avoid compile-time evaluation of an expression. id(I) -> I. diff --git a/lib/compiler/test/warnings_SUITE.erl b/lib/compiler/test/warnings_SUITE.erl index c3b02819f9..0637041873 100644 --- a/lib/compiler/test/warnings_SUITE.erl +++ b/lib/compiler/test/warnings_SUITE.erl @@ -573,7 +573,48 @@ maps(Config) when is_list(Config) -> ">>, [], {warnings,[{3,sys_core_fold,no_clause_match}, - {9,sys_core_fold,nomatch_clause_type}]}}], + {9,sys_core_fold,nomatch_clause_type}]}}, + {bad_map_src1, + <<" + t() -> + M = {a,[]}, + {'EXIT',{badarg,_}} = (catch(M#{ a => 1})), + ok. + ">>, + [], + {warnings,[{4,v3_kernel,bad_map}]}}, + {bad_map_src2, + <<" + t() -> + M = id({a,[]}), + {'EXIT',{badarg,_}} = (catch(M#{ a => 1})), + ok. + id(I) -> I. + ">>, + [inline], + {warnings,[{4,v3_kernel,bad_map}]}}, + {bad_map_src3, + <<" + t() -> + {'EXIT',{badarg,_}} = (catch <<>>#{ a := 1}), + ok. + ">>, + [], + {warnings,[{3,v3_core,bad_map}]}}, + {bad_map_literal_key, + <<" + t() -> + V = id(1), + M = id(#{ <<$h,$i>> => V }), + V = case M of + #{ <<0:257>> := Val } -> Val; + #{ <<$h,$i>> := Val } -> Val + end, + ok. + id(I) -> I. + ">>, + [], + {warnings,[{6,v3_core,nomatch}]}}], run(Config, Ts), ok. @@ -621,6 +662,20 @@ latin1_fallback(Conf) when is_list(Conf) -> {warnings,[{1,compile,reparsing_invalid_unicode}]} }], [] = run(Conf, Ts2), + + Ts3 = [{latin1_fallback3, + %% Test that the compiler fall backs to latin-1 with + %% a warning if a file has no encoding and does not + %% contain correct UTF-8 sequences. + <<"-ifdef(NOTDEFINED). + t(_) -> \"",246,"\"; + t(x) -> ok. + -endif. + ">>, + [], + {warnings,[{2,compile,reparsing_invalid_unicode}]}}], + [] = run(Conf, Ts3), + ok. %%% diff --git a/lib/compiler/vsn.mk b/lib/compiler/vsn.mk index c0c3d56472..d042596557 100644 --- a/lib/compiler/vsn.mk +++ b/lib/compiler/vsn.mk @@ -1 +1 @@ -COMPILER_VSN = 5.0 +COMPILER_VSN = 5.0.2 diff --git a/lib/cosEvent/doc/src/notes.xml b/lib/cosEvent/doc/src/notes.xml index df05f5634e..8f519447fc 100644 --- a/lib/cosEvent/doc/src/notes.xml +++ b/lib/cosEvent/doc/src/notes.xml @@ -32,7 +32,23 @@ <file>notes.xml</file> </header> - <section><title>cosEvent 2.1.14</title> + <section><title>cosEvent 2.1.15</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> The default encoding of Erlang files has been changed + from ISO-8859-1 to UTF-8. </p> <p> The encoding of XML + files has also been changed to UTF-8. </p> + <p> + Own Id: OTP-10907</p> + </item> + </list> + </section> + +</section> + +<section><title>cosEvent 2.1.14</title> <section><title>Improvements and New Features</title> <list> diff --git a/lib/cosEventDomain/doc/src/notes.xml b/lib/cosEventDomain/doc/src/notes.xml index f88d101a69..2c3bf16411 100644 --- a/lib/cosEventDomain/doc/src/notes.xml +++ b/lib/cosEventDomain/doc/src/notes.xml @@ -31,7 +31,23 @@ <file>notes.xml</file> </header> - <section><title>cosEventDomain 1.1.13</title> + <section><title>cosEventDomain 1.1.14</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> The default encoding of Erlang files has been changed + from ISO-8859-1 to UTF-8. </p> <p> The encoding of XML + files has also been changed to UTF-8. </p> + <p> + Own Id: OTP-10907</p> + </item> + </list> + </section> + +</section> + +<section><title>cosEventDomain 1.1.13</title> <section><title>Improvements and New Features</title> <list> diff --git a/lib/cosFileTransfer/doc/src/notes.xml b/lib/cosFileTransfer/doc/src/notes.xml index 5cca5e307b..1d0c826d40 100644 --- a/lib/cosFileTransfer/doc/src/notes.xml +++ b/lib/cosFileTransfer/doc/src/notes.xml @@ -30,7 +30,23 @@ <file>notes.xml</file> </header> - <section><title>cosFileTransfer 1.1.15</title> + <section><title>cosFileTransfer 1.1.16</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> The default encoding of Erlang files has been changed + from ISO-8859-1 to UTF-8. </p> <p> The encoding of XML + files has also been changed to UTF-8. </p> + <p> + Own Id: OTP-10907</p> + </item> + </list> + </section> + +</section> + +<section><title>cosFileTransfer 1.1.15</title> <section><title>Improvements and New Features</title> <list> diff --git a/lib/cosNotification/doc/src/notes.xml b/lib/cosNotification/doc/src/notes.xml index b700e0984c..2e4f922142 100644 --- a/lib/cosNotification/doc/src/notes.xml +++ b/lib/cosNotification/doc/src/notes.xml @@ -31,7 +31,23 @@ <file>notes.xml</file> </header> - <section><title>cosNotification 1.1.20</title> + <section><title>cosNotification 1.1.21</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> The default encoding of Erlang files has been changed + from ISO-8859-1 to UTF-8. </p> <p> The encoding of XML + files has also been changed to UTF-8. </p> + <p> + Own Id: OTP-10907</p> + </item> + </list> + </section> + +</section> + +<section><title>cosNotification 1.1.20</title> <section><title>Improvements and New Features</title> <list> diff --git a/lib/cosProperty/doc/src/notes.xml b/lib/cosProperty/doc/src/notes.xml index 22b1a73d29..739f41617f 100644 --- a/lib/cosProperty/doc/src/notes.xml +++ b/lib/cosProperty/doc/src/notes.xml @@ -31,7 +31,23 @@ <file>notes.xml</file> </header> - <section><title>cosProperty 1.1.16</title> + <section><title>cosProperty 1.1.17</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> The default encoding of Erlang files has been changed + from ISO-8859-1 to UTF-8. </p> <p> The encoding of XML + files has also been changed to UTF-8. </p> + <p> + Own Id: OTP-10907</p> + </item> + </list> + </section> + +</section> + +<section><title>cosProperty 1.1.16</title> <section><title>Improvements and New Features</title> <list> diff --git a/lib/cosTime/doc/src/notes.xml b/lib/cosTime/doc/src/notes.xml index c1d000a5ff..f218f19a6b 100644 --- a/lib/cosTime/doc/src/notes.xml +++ b/lib/cosTime/doc/src/notes.xml @@ -32,7 +32,23 @@ <file>notes.xml</file> </header> - <section><title>cosTime 1.1.13</title> + <section><title>cosTime 1.1.14</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> The default encoding of Erlang files has been changed + from ISO-8859-1 to UTF-8. </p> <p> The encoding of XML + files has also been changed to UTF-8. </p> + <p> + Own Id: OTP-10907</p> + </item> + </list> + </section> + +</section> + +<section><title>cosTime 1.1.13</title> <section><title>Improvements and New Features</title> <list> diff --git a/lib/cosTransactions/doc/src/notes.xml b/lib/cosTransactions/doc/src/notes.xml index eab402aee8..4b20f23efb 100644 --- a/lib/cosTransactions/doc/src/notes.xml +++ b/lib/cosTransactions/doc/src/notes.xml @@ -32,7 +32,23 @@ <file>notes.xml</file> </header> - <section><title>cosTransactions 1.2.13</title> + <section><title>cosTransactions 1.2.14</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> The default encoding of Erlang files has been changed + from ISO-8859-1 to UTF-8. </p> <p> The encoding of XML + files has also been changed to UTF-8. </p> + <p> + Own Id: OTP-10907</p> + </item> + </list> + </section> + +</section> + +<section><title>cosTransactions 1.2.13</title> <section><title>Improvements and New Features</title> <list> diff --git a/lib/crypto/c_src/crypto.c b/lib/crypto/c_src/crypto.c index fca08c4eed..e7215eeb64 100644 --- a/lib/crypto/c_src/crypto.c +++ b/lib/crypto/c_src/crypto.c @@ -215,6 +215,7 @@ static ERL_NIF_TERM des_cfb_crypt(ErlNifEnv* env, int argc, const ERL_NIF_TERM a static ERL_NIF_TERM des_ecb_crypt(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); static ERL_NIF_TERM des_ede3_cbc_crypt(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); static ERL_NIF_TERM des_ede3_cfb_crypt_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); +static ERL_NIF_TERM aes_cfb_8_crypt(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); static ERL_NIF_TERM aes_cfb_128_crypt(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); static ERL_NIF_TERM aes_ctr_encrypt(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); static ERL_NIF_TERM aes_ctr_stream_encrypt(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); @@ -344,6 +345,7 @@ static ErlNifFunc nif_funcs[] = { {"des_ecb_crypt", 3, des_ecb_crypt}, {"des_ede3_cbc_crypt", 6, des_ede3_cbc_crypt}, {"des_ede3_cfb_crypt_nif", 6, des_ede3_cfb_crypt_nif}, + {"aes_cfb_8_crypt", 4, aes_cfb_8_crypt}, {"aes_cfb_128_crypt", 4, aes_cfb_128_crypt}, {"aes_ctr_encrypt", 3, aes_ctr_encrypt}, {"aes_ctr_decrypt", 3, aes_ctr_encrypt}, @@ -460,9 +462,11 @@ static void hmac_context_dtor(ErlNifEnv* env, struct hmac_context*); /* #define PRINTF_ERR0(FMT) enif_fprintf(stderr, FMT "\n") #define PRINTF_ERR1(FMT, A1) enif_fprintf(stderr, FMT "\n", A1) +#define PRINTF_ERR2(FMT, A1, A2) enif_fprintf(stderr, FMT "\n", A1, A2) */ #define PRINTF_ERR0(FMT) #define PRINTF_ERR1(FMT,A1) +#define PRINTF_ERR2(FMT,A1,A2) #ifdef __OSE__ @@ -504,7 +508,33 @@ static int init_ose_crypto() { #define CHECK_OSE_CRYPTO() #endif + +static int verify_lib_version(void) +{ + const unsigned long libv = SSLeay(); + const unsigned long hdrv = OPENSSL_VERSION_NUMBER; + +# define MAJOR_VER(V) ((unsigned long)(V) >> (7*4)) + + if (MAJOR_VER(libv) != MAJOR_VER(hdrv)) { + PRINTF_ERR2("CRYPTO: INCOMPATIBLE SSL VERSION" + " lib=%lx header=%lx\n", libv, hdrv); + return 0; + } + return 1; +} + + #ifdef HAVE_DYNAMIC_CRYPTO_LIB + +# if defined(DEBUG) +static char crypto_callback_name[] = "crypto_callback.debug"; +# elif defined(VALGRIND) +static char crypto_callback_name[] = "crypto_callback.valgrind"; +# else +static char crypto_callback_name[] = "crypto_callback"; +# endif + static int change_basename(ErlNifBinary* bin, char* buf, int bufsz, const char* newfile) { int i; @@ -543,6 +573,9 @@ static int init(ErlNifEnv* env, ERL_NIF_TERM load_info) if (!INIT_OSE_CRYPTO()) return 0; + if (!verify_lib_version()) + return 0; + /* load_info: {301, <<"/full/path/of/this/library">>} */ if (!enif_get_tuple(env, load_info, &tpl_arity, &tpl_array) || tpl_arity != 2 @@ -611,7 +644,7 @@ static int init(ErlNifEnv* env, ERL_NIF_TERM load_info) #ifdef HAVE_DYNAMIC_CRYPTO_LIB { void* handle; - if (!change_basename(&lib_bin, lib_buf, sizeof(lib_buf), "crypto_callback")) { + if (!change_basename(&lib_bin, lib_buf, sizeof(lib_buf), crypto_callback_name)) { return 0; } if (!(handle = enif_dlopen(lib_buf, &error_handler, NULL))) { @@ -1382,6 +1415,7 @@ static ERL_NIF_TERM hmac_init(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[ ErlNifBinary key; struct hmac_context* obj; const EVP_MD *md; + ERL_NIF_TERM ret; CHECK_OSE_CRYPTO(); @@ -1413,7 +1447,9 @@ static ERL_NIF_TERM hmac_init(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[ HMAC_CTX_init(&obj->ctx); HMAC_Init(&obj->ctx, key.data, key.size, md); - return enif_make_resource(env, obj); + ret = enif_make_resource(env, obj); + enif_release_resource(obj); + return ret; } static ERL_NIF_TERM hmac_update(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) @@ -1600,6 +1636,30 @@ static ERL_NIF_TERM des_ede3_cfb_crypt_nif(ErlNifEnv* env, int argc, const ERL_N #endif } +static ERL_NIF_TERM aes_cfb_8_crypt(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) +{/* (Key, IVec, Data, IsEncrypt) */ + ErlNifBinary key, ivec, text; + AES_KEY aes_key; + unsigned char ivec_clone[16]; /* writable copy */ + int new_ivlen = 0; + ERL_NIF_TERM ret; + + if (!enif_inspect_iolist_as_binary(env, argv[0], &key) || key.size != 16 + || !enif_inspect_binary(env, argv[1], &ivec) || ivec.size != 16 + || !enif_inspect_iolist_as_binary(env, argv[2], &text)) { + return enif_make_badarg(env); + } + + memcpy(ivec_clone, ivec.data, 16); + AES_set_encrypt_key(key.data, 128, &aes_key); + AES_cfb8_encrypt((unsigned char *) text.data, + enif_make_new_binary(env, text.size, &ret), + text.size, &aes_key, ivec_clone, &new_ivlen, + (argv[3] == atom_true)); + CONSUME_REDS(env,text); + return ret; +} + static ERL_NIF_TERM aes_cfb_128_crypt(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) {/* (Key, IVec, Data, IsEncrypt) */ ErlNifBinary key, ivec, text; @@ -2480,6 +2540,7 @@ static ERL_NIF_TERM dss_sign_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM ar return enif_make_binary(env, &ret_bin); } else { + enif_release_binary(&ret_bin); return atom_error; } } @@ -2742,6 +2803,7 @@ static ERL_NIF_TERM dh_compute_key_nif(ErlNifEnv* env, int argc, const ERL_NIF_T ret = enif_make_binary(env, &ret_bin); } else { + enif_release_binary(&ret_bin); ret = atom_error; } } @@ -2866,8 +2928,8 @@ static ERL_NIF_TERM srp_user_secret_nif(ErlNifEnv* env, int argc, const ERL_NIF_ /* a + (u * x) */ bn_exp2 = BN_new(); - BN_mod_mul(bn_result, bn_u, bn_exponent, bn_prime, bn_ctx); - BN_mod_add(bn_exp2, bn_a, bn_result, bn_prime, bn_ctx); + BN_mul(bn_result, bn_u, bn_exponent, bn_ctx); + BN_add(bn_exp2, bn_a, bn_result); /* (B - (k * g^x)) ^ (a + (u * x)) % N */ BN_mod_exp(bn_result, bn_base, bn_exp2, bn_prime, bn_ctx); @@ -3213,6 +3275,7 @@ out: if (bn_order) BN_free(bn_order); if (cofactor) BN_free(cofactor); if (group) EC_GROUP_free(group); + if (point) EC_POINT_free(point); return key; } @@ -3375,8 +3438,11 @@ static ERL_NIF_TERM ec_key_generate(ErlNifEnv* env, int argc, const ERL_NIF_TERM EC_KEY_free(key); return enif_make_tuple2(env, pub_key, priv_key); } - else + else { + if (key) + EC_KEY_free(key); return enif_make_badarg(env); + } #else return atom_notsup; #endif diff --git a/lib/crypto/c_src/crypto_callback.c b/lib/crypto/c_src/crypto_callback.c index a08dcec463..b4c175ae43 100644 --- a/lib/crypto/c_src/crypto_callback.c +++ b/lib/crypto/c_src/crypto_callback.c @@ -107,8 +107,6 @@ static INLINE void locking(int mode, ErlNifRWLock* lock) static void locking_function(int mode, int n, const char *file, int line) { - ASSERT(n>=0 && n<CRYPTO_num_locks()); - locking(mode, lock_vec[n]); } diff --git a/lib/crypto/doc/src/crypto.xml b/lib/crypto/doc/src/crypto.xml index e88bf01491..7712173ed8 100644 --- a/lib/crypto/doc/src/crypto.xml +++ b/lib/crypto/doc/src/crypto.xml @@ -128,7 +128,7 @@ <p><code>stream_cipher() = rc4 | aes_ctr </code></p> - <p><code>block_cipher() = aes_cbc128 | aes_cfb128 | aes_ige256 | blowfish_cbc | + <p><code>block_cipher() = aes_cbc128 | aes_cfb8 | aes_cfb128 | aes_ige256 | blowfish_cbc | blowfish_cfb64 | des_cbc | des_cfb | des3_cbc | des3_cbf | des_ede3 | rc2_cbc </code></p> @@ -152,7 +152,7 @@ Note that both md4 and md5 are recommended only for compatibility with existing applications. </p> <p><code> cipher_algorithms() = des_cbc | des_cfb | des3_cbc | des3_cbf | des_ede3 | - blowfish_cbc | blowfish_cfb64 | aes_cbc128 | aes_cfb128| aes_cbc256 | aes_ige256 | rc2_cbc | aes_ctr| rc4 </code> </p> + blowfish_cbc | blowfish_cfb64 | aes_cbc128 | aes_cfb8 | aes_cfb128| aes_cbc256 | aes_ige256 | rc2_cbc | aes_ctr| rc4 </code> </p> <p><code> public_key_algorithms() = rsa |dss | ecdsa | dh | ecdh | ec_gf2m</code> Note that ec_gf2m is not strictly a public key algorithm, but a restriction on what curves are supported with ecdsa and ecdh. diff --git a/lib/crypto/doc/src/notes.xml b/lib/crypto/doc/src/notes.xml index 53249479f1..82b6de9acd 100644 --- a/lib/crypto/doc/src/notes.xml +++ b/lib/crypto/doc/src/notes.xml @@ -30,6 +30,163 @@ </header> <p>This document describes the changes made to the Crypto application.</p> +<section><title>Crypto 3.4.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Make <c>crypto</c> verify major version number of OpenSSL + header files and runtime library. Loading of + <c>crypto</c> will fail if there is a version mismatch.</p> + <p> + Own Id: OTP-12146 Aux Id: seq12700 </p> + </item> + </list> + </section> + +</section> + +<section><title>Crypto 3.4</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fix memory leak in <c>crypto:hmac_init/upgrade/final</c> + functions for all data and in <c>crypto:hmac/3/4</c> for + data larger than 20000 bytes. Bug exists since OTP 17.0.</p> + <p> + Own Id: OTP-11953</p> + </item> + <item> + <p> + Fix memory leak in <c>crypto</c> for elliptic curve.</p> + <p> + Own Id: OTP-11999</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Add <c>aes_cfb8</c> cypher to <c>crypto:block_encrypt</c> + and <c>block_decrypt</c>.</p> + <p> + Own Id: OTP-11911</p> + </item> + </list> + </section> + +</section> + +<section><title>Crypto 3.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fix memory leaks and invalid deallocations in + <c>mod_pow</c>, <c>mod_exp</c> and + <c>generate_key(srp,...)</c> when bad arguments are + passed. (Thanks to Florian Zumbiehi)</p> + <p> + Own Id: OTP-11550</p> + </item> + <item> + <p> + Correction of the word 'ChipherText' throughout the + documentation (Thanks to Andrew Tunnell-Jones)</p> + <p> + Own Id: OTP-11609</p> + </item> + <item> + <p> + Fix fatal bug when using a hmac context variable in more + than one call to <c>hmac_update</c> or <c>hmac_final</c>. + The reuse of hmac contexts has never worked as the + underlying OpenSSL implementation does not support it. It + is now documented as having undefined behaviour, but it + does not crash or corrupt the VM anymore.</p> + <p> + Own Id: OTP-11724</p> + </item> + <item> + <p> + Crypto handles out-of-memory with a controlled abort + instead of crash/corruption. (Thanks to Florian Zumbiehi)</p> + <p> + Own Id: OTP-11725</p> + </item> + <item> + <p> + Application upgrade (appup) files are corrected for the + following applications: </p> + <p> + <c>asn1, common_test, compiler, crypto, debugger, + dialyzer, edoc, eldap, erl_docgen, et, eunit, gs, hipe, + inets, observer, odbc, os_mon, otp_mibs, parsetools, + percept, public_key, reltool, runtime_tools, ssh, + syntax_tools, test_server, tools, typer, webtool, wx, + xmerl</c></p> + <p> + A new test utility for testing appup files is added to + test_server. This is now used by most applications in + OTP.</p> + <p> + (Thanks to Tobias Schlager)</p> + <p> + Own Id: OTP-11744</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + By giving --enable-static-{nifs,drivers} to configure it + is now possible to statically linking of nifs and drivers + to the main Erlang VM binary. At the moment only the asn1 + and crypto nifs of the Erlang/OTP nifs and drivers have + been prepared to be statically linked. For more details + see the Installation Guide in the System documentation.</p> + <p> + Own Id: OTP-11258</p> + </item> + <item> + <p> + Add IGE mode for AES cipher in crypto (Thanks to Yura + Beznos).</p> + <p> + Own Id: OTP-11522</p> + </item> + <item> + <p> + Moved elliptic curve definition from the crypto + NIF/OpenSSL into Erlang code, adds the RFC-5639 brainpool + curves and makes TLS use them (RFC-7027).</p> + <p> + Thanks to Andreas Schultz</p> + <p> + Own Id: OTP-11578</p> + </item> + <item> + <p> + Remove all obsolete application processes from crypto and + make it into a pure library application.</p> + <p> + Own Id: OTP-11619</p> + </item> + </list> + </section> + +</section> + <section><title>Crypto 3.2</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/crypto/src/crypto.erl b/lib/crypto/src/crypto.erl index 5bf52fc8a4..e1fbbf9ab8 100644 --- a/lib/crypto/src/crypto.erl +++ b/lib/crypto/src/crypto.erl @@ -210,7 +210,7 @@ supports()-> [{hashs, Hashs}, {ciphers, [des_cbc, des_cfb, des3_cbc, des_ede3, blowfish_cbc, - blowfish_cfb64, blowfish_ofb64, blowfish_ecb, aes_cbc128, aes_cfb128, + blowfish_cfb64, blowfish_ofb64, blowfish_ecb, aes_cbc128, aes_cfb8, aes_cfb128, aes_cbc256, rc2_cbc, aes_ctr, rc4] ++ Ciphers}, {public_keys, [rsa, dss, dh, srp] ++ PubKeys} ]. @@ -281,7 +281,7 @@ hmac_final_n(_Context, _HashLen) -> ? nif_stub. %% Ecrypt/decrypt %%% -spec block_encrypt(des_cbc | des_cfb | des3_cbc | des3_cbf | des_ede3 | blowfish_cbc | - blowfish_cfb64 | aes_cbc128 | aes_cfb128 | aes_cbc256 | rc2_cbc, + blowfish_cfb64 | aes_cbc128 | aes_cfb8 | aes_cfb128 | aes_cbc256 | rc2_cbc, Key::iodata(), Ivec::binary(), Data::iodata()) -> binary(). block_encrypt(des_cbc, Key, Ivec, Data) -> @@ -306,6 +306,8 @@ block_encrypt(aes_cbc256, Key, Ivec, Data) -> aes_cbc_256_encrypt(Key, Ivec, Data); block_encrypt(aes_ige256, Key, Ivec, Data) -> aes_ige_256_encrypt(Key, Ivec, Data); +block_encrypt(aes_cfb8, Key, Ivec, Data) -> + aes_cfb_8_encrypt(Key, Ivec, Data); block_encrypt(aes_cfb128, Key, Ivec, Data) -> aes_cfb_128_encrypt(Key, Ivec, Data); block_encrypt(rc2_cbc, Key, Ivec, Data) -> @@ -313,7 +315,7 @@ block_encrypt(rc2_cbc, Key, Ivec, Data) -> -spec block_decrypt(des_cbc | des_cfb | des3_cbc | des3_cbf | des_ede3 | blowfish_cbc | blowfish_cfb64 | blowfish_ofb64 | aes_cbc128 | aes_cbc256 | aes_ige256 | - aes_cfb128 | rc2_cbc, + aes_cfb8 | aes_cfb128 | rc2_cbc, Key::iodata(), Ivec::binary(), Data::iodata()) -> binary(). block_decrypt(des_cbc, Key, Ivec, Data) -> @@ -338,6 +340,8 @@ block_decrypt(aes_cbc256, Key, Ivec, Data) -> aes_cbc_256_decrypt(Key, Ivec, Data); block_decrypt(aes_ige256, Key, Ivec, Data) -> aes_ige_256_decrypt(Key, Ivec, Data); +block_decrypt(aes_cfb8, Key, Ivec, Data) -> + aes_cfb_8_decrypt(Key, Ivec, Data); block_decrypt(aes_cfb128, Key, Ivec, Data) -> aes_cfb_128_decrypt(Key, Ivec, Data); block_decrypt(rc2_cbc, Key, Ivec, Data) -> @@ -1159,7 +1163,21 @@ blowfish_ofb64_encrypt(_Key, _IVec, _Data) -> ?nif_stub. %% -%% AES in cipher feedback mode (CFB) +%% AES in cipher feedback mode (CFB) - 8 bit shift +%% +-spec aes_cfb_8_encrypt(iodata(), binary(), iodata()) -> binary(). +-spec aes_cfb_8_decrypt(iodata(), binary(), iodata()) -> binary(). + +aes_cfb_8_encrypt(Key, IVec, Data) -> + aes_cfb_8_crypt(Key, IVec, Data, true). + +aes_cfb_8_decrypt(Key, IVec, Data) -> + aes_cfb_8_crypt(Key, IVec, Data, false). + +aes_cfb_8_crypt(_Key, _IVec, _Data, _IsEncrypt) -> ?nif_stub. + +%% +%% AES in cipher feedback mode (CFB) - 128 bit shift %% -spec aes_cfb_128_encrypt(iodata(), binary(), iodata()) -> binary(). -spec aes_cfb_128_decrypt(iodata(), binary(), iodata()) -> binary(). diff --git a/lib/crypto/test/crypto_SUITE.erl b/lib/crypto/test/crypto_SUITE.erl index 63552d2e70..03aa3964a5 100644 --- a/lib/crypto/test/crypto_SUITE.erl +++ b/lib/crypto/test/crypto_SUITE.erl @@ -55,6 +55,7 @@ all() -> {group, blowfish_cfb64}, {group, blowfish_ofb64}, {group, aes_cbc128}, + {group, aes_cfb8}, {group, aes_cfb128}, {group, aes_cbc256}, {group, aes_ige256}, @@ -90,6 +91,7 @@ groups() -> {des3_cbf,[], [block]}, {rc2_cbc,[], [block]}, {aes_cbc128,[], [block]}, + {aes_cfb8,[], [block]}, {aes_cfb128,[], [block]}, {aes_cbc256,[], [block]}, {aes_ige256,[], [block]}, @@ -688,7 +690,7 @@ group_config(ecdsa = Type, Config) -> SignVerify = [{Type, sha, Public, Private, Msg}], [{sign_verify, SignVerify} | Config]; group_config(srp, Config) -> - GenerateCompute = [srp3(), srp6(), srp6a()], + GenerateCompute = [srp3(), srp6(), srp6a(), srp6a_smaller_prime()], [{generate_compute, GenerateCompute} | Config]; group_config(ecdh, Config) -> Compute = ecdh(), @@ -723,6 +725,9 @@ group_config(aes_cbc256, Config) -> group_config(aes_ige256, Config) -> Block = aes_ige256(), [{block, Block} | Config]; +group_config(aes_cfb8, Config) -> + Block = aes_cfb8(), + [{block, Block} | Config]; group_config(aes_cfb128, Config) -> Block = aes_cfb128(), [{block, Block} | Config]; @@ -1164,6 +1169,25 @@ aes_ige256() -> hexstr2bin("f69f2445df4f9b17ad2b417be66c3710")} ]. +aes_cfb8() -> + [{aes_cfb8, + hexstr2bin("2b7e151628aed2a6abf7158809cf4f3c"), + hexstr2bin("000102030405060708090a0b0c0d0e0f"), + hexstr2bin("6bc1bee22e409f96e93d7e117393172a")}, + {aes_cfb8, + hexstr2bin("2b7e151628aed2a6abf7158809cf4f3c"), + hexstr2bin("3B3FD92EB72DAD20333449F8E83CFB4A"), + hexstr2bin("ae2d8a571e03ac9c9eb76fac45af8e51")}, + {aes_cfb8, + hexstr2bin("2b7e151628aed2a6abf7158809cf4f3c"), + hexstr2bin("C8A64537A0B3A93FCDE3CDAD9F1CE58B"), + hexstr2bin("30c81c46a35ce411e5fbc1191a0a52ef")}, + {aes_cfb8, + hexstr2bin("2b7e151628aed2a6abf7158809cf4f3c"), + hexstr2bin("26751F67A3CBB140B1808CF187A4F4DF"), + hexstr2bin("f69f2445df4f9b17ad2b417be66c3710")} + ]. + aes_cfb128() -> [{aes_cfb128, hexstr2bin("2b7e151628aed2a6abf7158809cf4f3c"), @@ -1472,6 +1496,32 @@ srp6() -> ClientPublic = crypto:mod_pow(Generator, ClientPrivate, Prime), srp(ClientPrivate, Generator, Prime, Version, Verifier, ServerPublic, ServerPrivate, UserPassHash, Scrambler, SessionKey). + +srp6a_smaller_prime() -> + Username = <<"alice">>, + Password = <<"password123">>, + Salt = <<"mystrongsalt">>, + Prime = hexstr2bin("894B645E89E1535BBDAD5B8B290650530801B18EBFBF5E8FAB3C82872A3E9BB7"), + Generator = <<7>>, + Version = '6a', + Scrambler = hexstr2bin("18DE4A002AD05EF464B19AE2B6929F9B1319C7AA"), + Verifier = hexstr2bin("867401D5DE10964768184EAF246B322760C847604075FA66A4423907" + "8428BCA5"), + ClientPrivate = hexstr2bin("C49F832EE8D67ECF9E7F2785EB0622D8B3FE2344C00F96E1AEF4103C" + "A44D51F9"), + ServerPrivate = hexstr2bin("6C78CCEAAEC15E69068A87795B2A20ED7B45CFC5A254EBE2F17F144A" + "4D99DB18"), + ClientPublic = hexstr2bin("2452A57166BBBF690DB77539BAF9C57CD1ED99D5AA15ED925AD9B5C3" + "64BBEDFF"), + ServerPublic = hexstr2bin("2C0464DE84B91E4963A3546CAC0EFE55F31F49208C3F0AD7EE55F444" + "8F38BA7F"), + + SessionKey = hexstr2bin("65581B2302580BD26F522A5A421CF969B9CCBCE4051196B034A2A9D22065D848"), + UserPassHash = crypto:hash(sha, [Salt, crypto:hash(sha, [Username, <<$:>>, Password])]), + Verifier = crypto:mod_pow(Generator, UserPassHash, Prime), + ClientPublic = crypto:mod_pow(Generator, ClientPrivate, Prime), + srp(ClientPrivate, Generator, Prime, Version, Verifier, ServerPublic, ServerPrivate, UserPassHash, Scrambler, SessionKey). + srp6a() -> Username = <<"alice">>, Password = <<"password123">>, diff --git a/lib/crypto/vsn.mk b/lib/crypto/vsn.mk index a2bd6f851a..2a7f3c4558 100644 --- a/lib/crypto/vsn.mk +++ b/lib/crypto/vsn.mk @@ -1 +1 @@ -CRYPTO_VSN = 3.3 +CRYPTO_VSN = 3.4.1 diff --git a/lib/debugger/doc/src/notes.xml b/lib/debugger/doc/src/notes.xml index a3543a1e11..c1ba1eec6b 100644 --- a/lib/debugger/doc/src/notes.xml +++ b/lib/debugger/doc/src/notes.xml @@ -32,6 +32,90 @@ <p>This document describes the changes made to the Debugger application.</p> +<section><title>Debugger 4.0.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fix evaluation of map updates in the debugger and + erl_eval</p> + <p> + Reported-by: José Valim</p> + <p> + Own Id: OTP-11922</p> + </item> + </list> + </section> + +</section> + +<section><title>Debugger 4.0</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + The debugger now correctly evaluates code such as '<c>X = + true andalso X</c>'. (Thanks to Anthony Ramine.)</p> + <p> + Own Id: OTP-11553</p> + </item> + <item> + <p> + A few subtle bugs in the evaluation of code in the + debugger has been corrected. (Thanks to Anthony Ramine.)</p> + <p> + Own Id: OTP-11676</p> + </item> + <item> + <p> + Application upgrade (appup) files are corrected for the + following applications: </p> + <p> + <c>asn1, common_test, compiler, crypto, debugger, + dialyzer, edoc, eldap, erl_docgen, et, eunit, gs, hipe, + inets, observer, odbc, os_mon, otp_mibs, parsetools, + percept, public_key, reltool, runtime_tools, ssh, + syntax_tools, test_server, tools, typer, webtool, wx, + xmerl</c></p> + <p> + A new test utility for testing appup files is added to + test_server. This is now used by most applications in + OTP.</p> + <p> + (Thanks to Tobias Schlager)</p> + <p> + Own Id: OTP-11744</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Removed gs based applications and gs based backends. The + <c>observer</c> application replaces the removed + applications.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-10915</p> + </item> + <item> + <p> + Support Maps syntax in debugger (Thanks to Anthony + Ramine).</p> + <p> + Own Id: OTP-11673</p> + </item> + </list> + </section> + +</section> + <section><title>Debugger 3.2.12</title> <section><title>Improvements and New Features</title> diff --git a/lib/debugger/src/Makefile b/lib/debugger/src/Makefile index 90189dd297..d61519f1ad 100644 --- a/lib/debugger/src/Makefile +++ b/lib/debugger/src/Makefile @@ -63,7 +63,7 @@ MODULES= \ HRL_FILES= -INTERNAL_HRL_FILES= dbg_ieval.hrl +INTERNAL_HRL_FILES= dbg_ieval.hrl dbg_wx_filedialog_win.hrl ERL_FILES= $(MODULES:%=%.erl) diff --git a/lib/debugger/src/dbg_ieval.erl b/lib/debugger/src/dbg_ieval.erl index 0653ce4c00..77297de0f3 100644 --- a/lib/debugger/src/dbg_ieval.erl +++ b/lib/debugger/src/dbg_ieval.erl @@ -665,11 +665,11 @@ expr({map,Line,E0,Fs0}, Bs0, Ieval0) -> {value,E,Bs1} = expr(E0, Bs0, Ieval), case E of #{} -> - {Fs,Bs2} = eval_map_fields(Fs0, Bs1, Ieval), + {Fs,Bs2} = eval_map_fields(Fs0, Bs0, Ieval), Value = lists:foldl(fun ({map_assoc,K,V}, Mi) -> maps:put(K,V,Mi); ({map_exact,K,V}, Mi) -> maps:update(K,V,Mi) end, E, Fs), - {value,Value,Bs2}; + {value,Value,merge_bindings(Bs2, Bs1, Ieval)}; _ -> exception(error, {badarg,E}, Bs1, Ieval) end; diff --git a/lib/debugger/test/int_eval_SUITE.erl b/lib/debugger/test/int_eval_SUITE.erl index ecbd68ab40..9d7ef238e3 100644 --- a/lib/debugger/test/int_eval_SUITE.erl +++ b/lib/debugger/test/int_eval_SUITE.erl @@ -294,6 +294,7 @@ stacktrace(Config) when is_list(Config) -> maps(Config) when is_list(Config) -> Fun = fun () -> ?IM:empty_map_update([camembert]) end, {'EXIT',{{badarg,[camembert]},_}} = spawn_eval(Fun), + [#{hello := 0, price := 0}] = spawn_eval(fun () -> ?IM:update_in_fun() end), ok. diff --git a/lib/debugger/test/int_eval_SUITE_data/my_int_eval_module.erl b/lib/debugger/test/int_eval_SUITE_data/my_int_eval_module.erl index e047a33d8c..7f55360f48 100644 --- a/lib/debugger/test/int_eval_SUITE_data/my_int_eval_module.erl +++ b/lib/debugger/test/int_eval_SUITE_data/my_int_eval_module.erl @@ -29,7 +29,7 @@ -export([more_catch/1,more_nocatch/1,exit_me/0]). -export([f/1, f_try/1, f_catch/1]). -export([otp_5837/1, otp_8310/0]). --export([empty_map_update/1]). +-export([empty_map_update/1, update_in_fun/0]). %% Internal exports. -export([echo/2,my_subtract/2,catch_a_ball/0,throw_a_ball/0]). @@ -244,3 +244,6 @@ otp_8310() -> ok. empty_map_update(Map) -> Map#{}. + +update_in_fun() -> + lists:map(fun (X) -> X#{price := 0} end, [#{hello => 0, price => nil}]). diff --git a/lib/debugger/vsn.mk b/lib/debugger/vsn.mk index cd107599e9..33481a1537 100644 --- a/lib/debugger/vsn.mk +++ b/lib/debugger/vsn.mk @@ -1 +1 @@ -DEBUGGER_VSN = 4.0 +DEBUGGER_VSN = 4.0.1 diff --git a/lib/dialyzer/doc/src/dialyzer.xml b/lib/dialyzer/doc/src/dialyzer.xml index a92b890a80..4e26a9e95e 100644 --- a/lib/dialyzer/doc/src/dialyzer.xml +++ b/lib/dialyzer/doc/src/dialyzer.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>2006</year><year>2013</year> + <year>2006</year><year>2014</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -50,33 +50,31 @@ <p>Dialyzer also has a command line version for automated use. Below is a brief description of the list of its options. The same information can be obtained by writing</p> - <code type="none"><![CDATA[ - dialyzer --help - ]]></code> + <code type="none"> + dialyzer --help</code> <p>in a shell. Please refer to the GUI description for more details on the operation of Dialyzer.</p> <p>The exit status of the command line version is:</p> - <code type="none"><![CDATA[ + <code type="none"> 0 - No problems were encountered during the analysis and no warnings were emitted. 1 - Problems were encountered during the analysis. - 2 - No problems were encountered, but warnings were emitted. - ]]></code> + 2 - No problems were encountered, but warnings were emitted.</code> <p>Usage:</p> - <code type="none"><![CDATA[ + <code type="none"> dialyzer [--help] [--version] [--shell] [--quiet] [--verbose] [-pa dir]* [--plt plt] [--plts plt*] [-Ddefine]* - [-I include_dir]* [--output_plt file] [-Wwarn]* + [-I include_dir]* [--output_plt file] [-Wwarn]* [--raw] [--src] [--gui] [files_or_dirs] [-r dirs] [--apps applications] [-o outfile] [--build_plt] [--add_to_plt] [--remove_from_plt] [--check_plt] [--no_check_plt] [--plt_info] [--get_warnings] - [--no_native] [--fullpath] - ]]></code> + [--dump_callgraph file] [--no_native] [--fullpath] + [--statistics]</code> <p>Options:</p> <taglist> <tag><c><![CDATA[files_or_dirs]]></c> (for backwards compatibility also - as: <c><![CDATA[-c files_or_dirs]]></c></tag> + as: <c><![CDATA[-c files_or_dirs]]></c>)</tag> <item>Use Dialyzer from the command line to detect defects in the specified files or directories containing <c><![CDATA[.erl]]></c> or <c><![CDATA[.beam]]></c> files, depending on the type of the @@ -88,16 +86,14 @@ analysis.</item> <tag><c><![CDATA[--apps applications]]></c></tag> <item>Option typically used when building or modifying a plt as in: - <code type="none"><![CDATA[ - dialyzer --build_plt --apps erts kernel stdlib mnesia ... - ]]></code> + <code type="none"> + dialyzer --build_plt --apps erts kernel stdlib mnesia ...</code> to conveniently refer to library applications corresponding to the Erlang/OTP installation. However, the option is general and can also be used during analysis in order to refer to Erlang/OTP applications. In addition, file or directory names can also be included, as in: - <code type="none"><![CDATA[ - dialyzer --apps inets ssl ./ebin ../other_lib/ebin/my_module.beam - ]]></code></item> + <code type="none"> + dialyzer --apps inets ssl ./ebin ../other_lib/ebin/my_module.beam</code></item> <tag><c><![CDATA[-o outfile]]></c> (or <c><![CDATA[--output outfile]]></c>)</tag> <item>When using Dialyzer from the command line, send the analysis @@ -129,19 +125,16 @@ that the plts are disjoint (i.e., do not have any module appearing in more than one plt). The plts are created in the usual way: - <code type="none"><![CDATA[ + <code type="none"> dialyzer --build_plt --output_plt plt_1 files_to_include ... - dialyzer --build_plt --output_plt plt_n files_to_include - ]]></code> + dialyzer --build_plt --output_plt plt_n files_to_include</code> and then can be used in either of the following ways: - <code type="none"><![CDATA[ - dialyzer files_to_analyze --plts plt_1 ... plt_n - ]]></code> + <code type="none"> + dialyzer files_to_analyze --plts plt_1 ... plt_n</code> or: - <code type="none"><![CDATA[ - dialyzer --plts plt_1 ... plt_n -- files_to_analyze - ]]></code> + <code type="none"> + dialyzer --plts plt_1 ... plt_n -- files_to_analyze</code> (Note the -- delimiter in the second case)</item> <tag><c><![CDATA[-Wwarn]]></c></tag> <item>A family of options which selectively turn on/off warnings @@ -220,8 +213,6 @@ <item>Suppress warnings for unused functions.</item> <tag><c><![CDATA[-Wno_improper_lists]]></c></tag> <item>Suppress warnings for construction of improper lists.</item> - <tag><c><![CDATA[-Wno_tuple_as_fun]]></c></tag> - <item>Suppress warnings for using tuples instead of funs.</item> <tag><c><![CDATA[-Wno_fun_app]]></c></tag> <item>Suppress warnings for fun applications that will fail.</item> <tag><c><![CDATA[-Wno_match]]></c></tag> @@ -229,9 +220,16 @@ match.</item> <tag><c><![CDATA[-Wno_opaque]]></c></tag> <item>Suppress warnings for violations of opaqueness of data types.</item> + <tag><c><![CDATA[-Wno_fail_call]]></c></tag> + <item>Suppress warnings for failing calls.</item> + <tag><c><![CDATA[-Wno_contracts]]></c></tag> + <item>Suppress warnings about invalid contracts.</item> <tag><c><![CDATA[-Wno_behaviours]]></c></tag> <item>Suppress warnings about behaviour callbacks which drift from the published recommended interfaces.</item> + <tag><c><![CDATA[-Wno_undefined_callbacks]]></c></tag> + <item>Suppress warnings about behaviours that have no + <c>-callback</c> attributes for their callbacks.</item> <tag><c><![CDATA[-Wunmatched_returns]]></c>***</tag> <item>Include warnings for function calls which ignore a structured return value or do not match against one of many possible return @@ -278,13 +276,13 @@ </type> <desc> <p>Dialyzer GUI version.</p> - <code type="none"><![CDATA[ + <code type="none"> OptList :: [Option] Option :: {files, [Filename :: string()]} | {files_rec, [DirName :: string()]} | {defines, [{Macro: atom(), Value : term()}]} - | {from, src_code | byte_code} %% Defaults to byte_code - | {init_plt, FileName :: string()} %% If changed from default + | {from, src_code | byte_code} %% Defaults to byte_code + | {init_plt, FileName :: string()} %% If changed from default | {plts, [FileName :: string()]} %% If changed from default | {include_dirs, [DirName :: string()]} | {output_file, FileName :: string()} @@ -304,14 +302,15 @@ WarnOpts :: no_return | no_match | no_opaque | no_fail_call + | no_contracts + | no_behaviours + | no_undefined_callbacks + | unmatched_returns | error_handling | race_conditions - | behaviours - | unmatched_returns | overspecs | underspecs - | specdiffs - ]]></code> + | specdiffs</code> </desc> </func> <func> @@ -323,17 +322,30 @@ WarnOpts :: no_return </type> <desc> <p>Dialyzer command line version.</p> - <code type="none"><![CDATA[ + <code type="none"> Warnings :: [{Tag, Id, Msg}] -Tag :: 'warn_return_no_exit' | 'warn_return_only_exit' - | 'warn_not_called' | 'warn_non_proper_list' - | 'warn_fun_app' | 'warn_matching' - | 'warn_failing_call' | 'warn_contract_types' - | 'warn_contract_syntax' | 'warn_contract_not_equal' - | 'warn_contract_subtype' | 'warn_contract_supertype' +Tag :: 'warn_behaviour' + | 'warn_bin_construction' + | 'warn_callgraph' + | 'warn_contract_not_equal' + | 'warn_contract_range' + | 'warn_contract_subtype' + | 'warn_contract_supertype' + | 'warn_contract_syntax' + | 'warn_contract_types' + | 'warn_failing_call' + | 'warn_fun_app' + | 'warn_matching' + | 'warn_non_proper_list' + | 'warn_not_called' + | 'warn_opaque' + | 'warn_race_condition' + | 'warn_return_no_exit' + | 'warn_return_only_exit' + | 'warn_umatched_return' + | 'warn_undefined_callbacks' Id = {File :: string(), Line :: integer()} -Msg = msg() -- Undefined -]]></code> +Msg = msg() -- Undefined</code> </desc> </func> <func> diff --git a/lib/dialyzer/doc/src/notes.xml b/lib/dialyzer/doc/src/notes.xml index b61b1af1b0..d35639aa32 100644 --- a/lib/dialyzer/doc/src/notes.xml +++ b/lib/dialyzer/doc/src/notes.xml @@ -31,6 +31,241 @@ <p>This document describes the changes made to the Dialyzer application.</p> +<section><title>Dialyzer 2.7.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> A bug concerning <c>is_record/2,3</c> has been fixed, + as well as some cases where Dialyzer could crash due to + reaching system limits. </p> + <p> + Own Id: OTP-12018</p> + </item> + <item> + <p> When given the <c>-Wunderspecs</c> flag Dialyzer + sometimes output bogus warnings for parametrized types. + This bug has been fixed. </p> + <p> + Own Id: OTP-12111</p> + </item> + <item> + <p>Dialyzer now fetch the compile options from beam + files, and use them when creating core from the abstract + code. Previously the options were ignored. </p> + <p> + Own Id: OTP-12150</p> + </item> + </list> + </section> + +</section> + +<section><title>Dialyzer 2.7.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> Fix a bug concerning opaque types. Thanks to Shayan + Pooya for pointing out the bug.</p> + <p> + Own Id: OTP-11869</p> + </item> + <item> + <p> A bug where Dialyzer failed to handle typed records + with fields containing remote types has been fixed. + Thanks to Erik Søe Sørensen for reporting the bug. </p> + <p> + Own Id: OTP-11918</p> + </item> + <item> + <p> Make sure that only literal records are checked + against the types of record definitions. Until now the + elements of tuples have been checked against record field + types if the tag och size of the tuple matches the record + definition, often with surprising results. </p> + <p> + Own Id: OTP-11935 Aux Id: seq12590 </p> + </item> + <item> + <p> + A Dialyzer crash involving analysis of Map types has now + been fixed.</p> + <p> + Own Id: OTP-11947</p> + </item> + </list> + </section> + +</section> + +<section><title>Dialyzer 2.7</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> Dialyzer will no longer emit warnings when inspecting + or modifying opaque types within the scope of a module. + </p> <p> Hitherto the shape of terms (tuple, list, etc.) + has been used to determine the opaque terms, but now the + contracts are used for decorating types with opaqueness. + </p> + <p> + Own Id: OTP-10397</p> + </item> + <item> + <p> + With <c>--Wunmatched_returns</c>, dialyzer will no longer + warn when the value of a list comprehension is ignored, + provided that the each value in the list would be an + atomic value (such as integer or atoms, as opposed to + tuples and lists). Example: ignoring '<c>[io:format(...) + || ...]</c>' will not cause a warning, while ignoring + '<c>[file:close(Fd) || ...]</c>' will.</p> + <p> + Own Id: OTP-11626</p> + </item> + <item> + <p> + The man page for dialyzer now contains correct + information regarding -Wno_behaviours. (Thanks to Steve + Vinosky.)</p> + <p> + Own Id: OTP-11706</p> + </item> + <item> + <p> Fix handling of 'on_load' attribute. (Thanks to + Kostis Sagonas.) </p> + <p> + Own Id: OTP-11743</p> + </item> + <item> + <p> + Application upgrade (appup) files are corrected for the + following applications: </p> + <p> + <c>asn1, common_test, compiler, crypto, debugger, + dialyzer, edoc, eldap, erl_docgen, et, eunit, gs, hipe, + inets, observer, odbc, os_mon, otp_mibs, parsetools, + percept, public_key, reltool, runtime_tools, ssh, + syntax_tools, test_server, tools, typer, webtool, wx, + xmerl</c></p> + <p> + A new test utility for testing appup files is added to + test_server. This is now used by most applications in + OTP.</p> + <p> + (Thanks to Tobias Schlager)</p> + <p> + Own Id: OTP-11744</p> + </item> + <item> + <p> The generalization of guard constraints has been + modified. </p> + <p> + Own Id: OTP-11798 Aux Id: seq12547 </p> + </item> + <item> + <p> Dialyzer now plays nicely with funs that come as + "external" arguments. (Thanks to Stavros Aronis for + fixing the bug.) </p> + <p> + Own Id: OTP-11826</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> The pre-defined types <c>array/0</c>, <c>dict/0</c>, + <c>digraph/0</c>, <c>gb_set/0</c>, <c>gb_tree/0</c>, + <c>queue/0</c>, <c>set/0</c>, and <c>tid/0</c> have been + deprecated. They will be removed in Erlang/OTP 18.0. </p> + <p> Instead the types <c>array:array/0</c>, + <c>dict:dict/0</c>, <c>digraph:graph/0</c>, + <c>gb_set:set/0</c>, <c>gb_tree:tree/0</c>, + <c>queue:queue/0</c>, <c>sets:set/0</c>, and + <c>ets:tid/0</c> can be used. (Note: it has always been + necessary to use <c>ets:tid/0</c>.) </p> <p> It is + allowed in Erlang/OTP 17.0 to locally re-define the types + <c>array/0</c>, <c>dict/0</c>, and so on. </p> <p> New + types <c>array:array/1</c>, <c>dict:dict/2</c>, + <c>gb_sets:set/1</c>, <c>gb_trees:tree/2</c>, + <c>queue:queue/1</c>, and <c>sets:set/1</c> have been + added. </p> <p> A compiler option, + <c>nowarn_deprecated_type</c>, has been introduced. By + including the attribute </p> <c> + -compile(nowarn_deprecated_type).</c> <p> in an Erlang + source file, warnings about deprecated types can be + avoided in Erlang/OTP 17.0. </p> <p> The option can also + be given as a compiler flag: </p> <c> erlc + +nowarn_deprecated_type file.erl</c> + <p> + Own Id: OTP-10342</p> + </item> + <item> + <p> + Removed gs based applications and gs based backends. The + <c>observer</c> application replaces the removed + applications.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-10915</p> + </item> + <item> + <p> + Forbid unsized fields in patterns of binary generators + and simplified v3_core's translation of bit string + generators. (Thanks to Anthony Ramine.)</p> + <p> + Own Id: OTP-11186</p> + </item> + <item> + <p> + EEP43: New data type - Maps</p> + <p> + With Maps you may for instance: <taglist> <item><c>M0 = + #{ a => 1, b => 2}, % create + associations</c></item> <item><c>M1 = M0#{ a := 10 }, % + update values</c></item> <item><c>M2 = M1#{ "hi" => + "hello"}, % add new associations</c></item> <item><c>#{ + "hi" := V1, a := V2, b := V3} = M2. % match keys with + values</c></item> </taglist></p> + <p> + For information on how to use Maps please see Map Expressions in the + <seealso marker="doc/reference_manual:expressions#map_expressions"> + Reference Manual</seealso>.</p> + <p> + The current implementation is without the following + features: <taglist> <item>No variable keys</item> + <item>No single value access</item> <item>No map + comprehensions</item> </taglist></p> + <p> + Note that Maps is <em>experimental</em> during OTP 17.0.</p> + <p> + Own Id: OTP-11616</p> + </item> + <item> + <p> Parameterized opaque types have been introduced. </p> + <p> + Own Id: OTP-11625</p> + </item> + <item> + <p> + Some function specs are corrected or moved and some edoc + comments are corrected in order to allow use of edoc. + (Thanks to Pierre Fenoll)</p> + <p> + Own Id: OTP-11702</p> + </item> + </list> + </section> + +</section> + <section><title>Dialyzer 2.6.1</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/dialyzer/src/Makefile b/lib/dialyzer/src/Makefile index d7265ba31a..91fbdca5bd 100644 --- a/lib/dialyzer/src/Makefile +++ b/lib/dialyzer/src/Makefile @@ -88,7 +88,7 @@ APPUP_TARGET= $(EBIN)/$(APPUP_FILE) ifeq ($(NATIVE_LIBS_ENABLED),yes) ERL_COMPILE_FLAGS += +native endif -ERL_COMPILE_FLAGS += +warn_exported_vars +warn_unused_import +warn_untyped_record +warn_missing_spec +warnings_as_errors +ERL_COMPILE_FLAGS += +warn_export_vars +warn_unused_import +warn_untyped_record +warn_missing_spec +warnings_as_errors # ---------------------------------------------------- # Targets diff --git a/lib/dialyzer/src/dialyzer_analysis_callgraph.erl b/lib/dialyzer/src/dialyzer_analysis_callgraph.erl index 2a633c5e37..af1c2b7e3a 100644 --- a/lib/dialyzer/src/dialyzer_analysis_callgraph.erl +++ b/lib/dialyzer/src/dialyzer_analysis_callgraph.erl @@ -249,7 +249,7 @@ compile_and_store(Files, #analysis_state{codeserver = CServer, timing_server = Timing, parent = Parent} = State) -> send_log(Parent, "Reading files and computing callgraph... "), - {T1, _} = statistics(runtime), + {T1, _} = statistics(wall_clock), Callgraph = dialyzer_callgraph:new(), CompileInit = make_compile_init(State, Callgraph), {{Failed, NoWarn, Modules}, NextLabel} = @@ -272,13 +272,13 @@ compile_and_store(Files, #analysis_state{codeserver = CServer, [[Reason || {_Filename, Reason} <- Failed]]), exit({error, Msg}) end, - {T2, _} = statistics(runtime), + {T2, _} = statistics(wall_clock), Msg1 = io_lib:format("done in ~.2f secs\nRemoving edges... ", [(T2-T1)/1000]), send_log(Parent, Msg1), Callgraph = ?timing(Timing, "clean", _C2, cleanup_callgraph(State, CServer2, Callgraph, Modules)), - {T3, _} = statistics(runtime), + {T3, _} = statistics(wall_clock), Msg2 = io_lib:format("done in ~.2f secs\n", [(T3-T2)/1000]), send_log(Parent, Msg2), {Callgraph, sets:from_list(NoWarn), CServer2}. @@ -373,7 +373,16 @@ compile_byte(File, Callgraph, CServer, UseContracts) -> {error, " Could not get abstract code for: " ++ File ++ "\n" ++ " Recompile with +debug_info or analyze starting from source code"}; {ok, AbstrCode} -> - compile_common(File, AbstrCode, [], Callgraph, CServer, UseContracts) + compile_byte(File, AbstrCode, Callgraph, CServer, UseContracts) + end. + +compile_byte(File, AbstrCode, Callgraph, CServer, UseContracts) -> + case dialyzer_utils:get_compile_options_from_beam(File) of + error -> + {error, " Could not get compile options for: " ++ File ++ "\n" ++ + " Recompile or analyze starting from source code"}; + {ok, CompOpts} -> + compile_common(File, AbstrCode, CompOpts, Callgraph, CServer, UseContracts) end. compile_common(File, AbstrCode, CompOpts, Callgraph, CServer, UseContracts) -> @@ -620,9 +629,9 @@ dump_callgraph(CallGraph, State, #analysis{callgraph_file = File} = Analysis) -> Extension = filename:extension(File), Start_Msg = io_lib:format("Dumping the callgraph... ", []), send_log(State#analysis_state.parent, Start_Msg), - {T1, _} = statistics(runtime), + {T1, _} = statistics(wall_clock), dump_callgraph(CallGraph, State, Analysis, Extension), - {T2, _} = statistics(runtime), + {T2, _} = statistics(wall_clock), Finish_Msg = io_lib:format("done in ~2f secs\n", [(T2-T1)/1000]), send_log(State#analysis_state.parent, Finish_Msg), ok. diff --git a/lib/dialyzer/src/dialyzer_cl.erl b/lib/dialyzer/src/dialyzer_cl.erl index e013d39a0e..3e7d9dfa99 100644 --- a/lib/dialyzer/src/dialyzer_cl.erl +++ b/lib/dialyzer/src/dialyzer_cl.erl @@ -504,9 +504,7 @@ hipe_compile(Files, #options{erlang_mode = ErlangMode} = Options) -> _ -> Mods = [lists, dict, digraph, digraph_utils, ets, gb_sets, gb_trees, ordsets, sets, sofs, - %cerl, % uses maps instructions - %erl_types, % uses maps instructions - cerl_trees, erl_bif_types, + cerl, erl_types, cerl_trees, erl_bif_types, dialyzer_analysis_callgraph, dialyzer, dialyzer_behaviours, dialyzer_codeserver, dialyzer_contracts, dialyzer_coordinator, dialyzer_dataflow, dialyzer_dep, diff --git a/lib/dialyzer/src/dialyzer_cl_parse.erl b/lib/dialyzer/src/dialyzer_cl_parse.erl index db27b2037d..04ce0e8bc3 100644 --- a/lib/dialyzer/src/dialyzer_cl_parse.erl +++ b/lib/dialyzer/src/dialyzer_cl_parse.erl @@ -2,7 +2,7 @@ %%----------------------------------------------------------------------- %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2006-2013. All Rights Reserved. +%% Copyright Ericsson AB 2006-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -357,12 +357,13 @@ help_warnings() -> help_message() -> S = "Usage: dialyzer [--help] [--version] [--shell] [--quiet] [--verbose] [-pa dir]* [--plt plt] [--plts plt*] [-Ddefine]* - [-I include_dir]* [--output_plt file] [-Wwarn]* + [-I include_dir]* [--output_plt file] [-Wwarn]* [--raw] [--src] [--gui] [files_or_dirs] [-r dirs] [--apps applications] [-o outfile] [--build_plt] [--add_to_plt] [--remove_from_plt] [--check_plt] [--no_check_plt] [--plt_info] [--get_warnings] - [--no_native] [--fullpath] [--statistics] + [--dump_callgraph file] [--no_native] [--fullpath] + [--statistics] Options: files_or_dirs (for backwards compatibility also as: -c files_or_dirs) Use Dialyzer from the command line to detect defects in the @@ -495,14 +496,16 @@ warning_options_msg() -> Suppress warnings for unused functions. -Wno_improper_lists Suppress warnings for construction of improper lists. - -Wno_tuple_as_fun - Suppress warnings for using tuples instead of funs. -Wno_fun_app Suppress warnings for fun applications that will fail. -Wno_match Suppress warnings for patterns that are unused or cannot match. -Wno_opaque Suppress warnings for violations of opaqueness of data types. + -Wno_fail_call + Suppress warnings for failing calls. + -Wno_contracts + Suppress warnings about invalid contracts. -Wno_behaviours Suppress warnings about behaviour callbacks which drift from the published recommended interfaces. diff --git a/lib/dialyzer/src/dialyzer_contracts.erl b/lib/dialyzer/src/dialyzer_contracts.erl index 283031eb9a..f27fc1a842 100644 --- a/lib/dialyzer/src/dialyzer_contracts.erl +++ b/lib/dialyzer/src/dialyzer_contracts.erl @@ -20,8 +20,6 @@ -module(dialyzer_contracts). --compile(export_all). - -export([check_contract/2, check_contracts/4, contracts_without_fun/3, @@ -686,7 +684,7 @@ picky_contract_check(CSig0, Sig0, MFA, FileLine, Contract, RecDict, Acc) -> true -> Acc; false -> case extra_contract_warning(MFA, FileLine, Contract, - CSig, Sig, RecDict) of + CSig0, Sig0, RecDict) of no_warning -> Acc; {warning, Warning} -> [Warning|Acc] end @@ -752,14 +750,8 @@ is_remote_types_related(Contract, CSig, Sig, RecDict) -> t_from_forms_without_remote([{FType, []}], RecDict) -> Type0 = erl_types:t_from_form(FType, RecDict), - Map = - fun(Type) -> - case erl_types:t_is_remote(Type) of - true -> erl_types:t_none(); - false -> Type - end - end, - {ok, erl_types:t_map(Map, Type0)}; + Type1 = erl_types:subst_all_remote(Type0, erl_types:t_none()), + {ok, erl_types:subst_all_vars_to_any(Type1)}; t_from_forms_without_remote([{_FType, _Constrs}], _RecDict) -> %% 'When' constraints unsupported; diff --git a/lib/dialyzer/src/dialyzer_dataflow.erl b/lib/dialyzer/src/dialyzer_dataflow.erl index 692684cd99..03005e689f 100644 --- a/lib/dialyzer/src/dialyzer_dataflow.erl +++ b/lib/dialyzer/src/dialyzer_dataflow.erl @@ -53,7 +53,7 @@ t_bitstr/0, t_bitstr/2, t_bitstr_concat/1, t_bitstr_match/2, t_cons/0, t_cons/2, t_cons_hd/2, t_cons_tl/2, t_contains_opaque/2, - t_find_opaque_mismatch/2, t_float/0, t_from_range/2, t_from_term/1, + t_find_opaque_mismatch/3, t_float/0, t_from_range/2, t_from_term/1, t_fun/0, t_fun/2, t_fun_args/1, t_fun_args/2, t_fun_range/1, t_fun_range/2, t_integer/0, t_integers/1, t_is_any/1, t_is_atom/1, t_is_atom/2, t_is_any_atom/3, @@ -93,6 +93,8 @@ -define(TYPE_LIMIT, 3). +-define(BITS, 128). + -record(state, {callgraph :: dialyzer_callgraph:callgraph(), envs :: env_tab(), fun_tab :: fun_tab(), @@ -136,11 +138,10 @@ get_warnings(Tree, Plt, Callgraph, Records, NoWarnUnused) -> State1 = analyze_module(Tree, Plt, Callgraph, Records, true), - State2 = find_mismatched_record_patterns(Tree, State1), - State3 = - state__renew_warnings(state__get_warnings(State2, NoWarnUnused), State2), - State4 = state__get_race_warnings(State3), - {State4#state.warnings, state__all_fun_types(State4)}. + State2 = + state__renew_warnings(state__get_warnings(State1, NoWarnUnused), State1), + State3 = state__get_race_warnings(State2), + {State3#state.warnings, state__all_fun_types(State3)}. -spec get_fun_types(cerl:c_module(), dialyzer_plt:plt(), dialyzer_callgraph:callgraph(), @@ -277,13 +278,8 @@ traverse(Tree, Map, State) -> {State1, Map1} = lists:foldl(FoldFun, {State, Map}, Defs), traverse(Body, Map1, State1); literal -> - %% This is needed for finding records - case cerl:unfold_literal(Tree) of - Tree -> - Type = literal_type(Tree), - {State, Map, Type}; - NewTree -> traverse(NewTree, Map, State) - end; + Type = literal_type(Tree), + {State, Map, Type}; module -> handle_module(Tree, Map, State); primop -> @@ -363,20 +359,24 @@ traverse_list([], Map, State, Acc) -> handle_apply(Tree, Map, State) -> Args = cerl:apply_args(Tree), Op = cerl:apply_op(Tree), - {State1, Map1, ArgTypes} = traverse_list(Args, Map, State), - {State2, Map2, OpType} = traverse(Op, Map1, State1), + {State0, Map1, ArgTypes} = traverse_list(Args, Map, State), + {State1, Map2, OpType} = traverse(Op, Map1, State0), case any_none(ArgTypes) of true -> - {State2, Map2, t_none()}; + {State1, Map2, t_none()}; false -> - {CallSitesKnown, FunList} = - case state__lookup_call_site(Tree, State2) of - error -> {false, []}; - {ok, [external]} -> {false, []}; - {ok, List} -> {true, List} + FunList = + case state__lookup_call_site(Tree, State) of + error -> [external]; %% so that we go directly in the fallback + {ok, List} -> List end, - case CallSitesKnown of - false -> + FunInfoList = [{local, state__fun_info(Fun, State)} || Fun <- FunList], + case + handle_apply_or_call(FunInfoList, Args, ArgTypes, Map2, Tree, State1) + of + {had_external, State2} -> + %% Fallback: use whatever info we collected from traversing the op + %% instead of the result that has been generalized to t_any(). Arity = length(Args), OpType1 = t_inf(OpType, t_fun(Arity, t_any())), case t_is_none(OpType1) of @@ -408,25 +408,23 @@ handle_apply(Tree, Map, State) -> {State2, enter_type(Op, OpType1, Map3), Range} end end; - true -> - FunInfoList = [{local, state__fun_info(Fun, State)} - || Fun <- FunList], - handle_apply_or_call(FunInfoList, Args, ArgTypes, Map2, Tree, State1) + Normal -> Normal end end. handle_apply_or_call(FunInfoList, Args, ArgTypes, Map, Tree, State) -> None = t_none(), handle_apply_or_call(FunInfoList, Args, ArgTypes, Map, Tree, State, - [None || _ <- ArgTypes], None). + [None || _ <- ArgTypes], None, false). handle_apply_or_call([{local, external}|Left], Args, ArgTypes, Map, Tree, State, - _AccArgTypes, _AccRet) -> + _AccArgTypes, _AccRet, _HadExternal) -> handle_apply_or_call(Left, Args, ArgTypes, Map, Tree, State, - ArgTypes, t_any()); + ArgTypes, t_any(), true); handle_apply_or_call([{TypeOfApply, {Fun, Sig, Contr, LocalRet}}|Left], Args, ArgTypes, Map, Tree, - #state{opaques = Opaques} = State, AccArgTypes, AccRet) -> + #state{opaques = Opaques} = State, + AccArgTypes, AccRet, HadExternal) -> Any = t_any(), AnyArgs = [Any || _ <- Args], GenSig = {AnyArgs, fun(_) -> t_any() end}, @@ -573,11 +571,16 @@ handle_apply_or_call([{TypeOfApply, {Fun, Sig, Contr, LocalRet}}|Left], NewAccRet = t_sup(AccRet, TotalRet), ?debug("NewAccRet: ~s\n", [t_to_string(NewAccRet)]), handle_apply_or_call(Left, Args, ArgTypes, Map, Tree, - State3, NewAccArgTypes, NewAccRet); + State3, NewAccArgTypes, NewAccRet, HadExternal); handle_apply_or_call([], Args, _ArgTypes, Map, _Tree, State, - AccArgTypes, AccRet) -> - NewMap = enter_type_lists(Args, AccArgTypes, Map), - {State, NewMap, AccRet}. + AccArgTypes, AccRet, HadExternal) -> + case HadExternal of + false -> + NewMap = enter_type_lists(Args, AccArgTypes, Map), + {State, NewMap, AccRet}; + true -> + {had_external, State} + end. apply_fail_reason(FailedSig, FailedBif, FailedContract) -> if @@ -1103,7 +1106,7 @@ handle_tuple(Tree, Map, State) -> %% Let's find out if this is a record case Elements of [Tag|Left] -> - case cerl:is_c_atom(Tag) of + case cerl:is_c_atom(Tag) andalso is_literal_record(Tree) of true -> TagVal = cerl:atom_val(Tag), case state__lookup_record(TagVal, length(Left), State1) of @@ -1233,15 +1236,10 @@ do_clause(C, Arg, ArgType0, OrigArgType, Map, State) -> false -> {State1, Map, t_none(), ArgType0}; true -> - PatString = - case ErrorType of - bind -> format_patterns(Pats); - record -> format_patterns(Pats); - opaque -> format_patterns(NewPats) - end, {Msg, Force} = case t_is_none(ArgType0) of true -> + PatString = format_patterns(Pats), PatTypes = [PatString, format_type(OrigArgType, State1)], %% See if this is covered by an earlier clause or if it %% simply cannot match @@ -1291,6 +1289,12 @@ do_clause(C, Arg, ArgType0, OrigArgType, Map, State) -> false -> true end, + PatString = + case ErrorType of + bind -> format_patterns(Pats); + record -> format_patterns(NewPats); + opaque -> format_patterns(NewPats) + end, PatTypes = case ErrorType of bind -> [PatString, format_type(ArgType0, State1)]; record -> [PatString, format_type(Type, State1)]; @@ -1437,7 +1441,7 @@ bind_pat_vars([Pat|PatLeft], [Type|TypeLeft], Acc, Map, State, Rev) -> BinType = t_inf(t_bitstr(), Type, Opaques), case t_is_none(BinType) of true -> - case t_find_opaque_mismatch(t_bitstr(), Type) of + case t_find_opaque_mismatch(t_bitstr(), Type, Opaques) of {ok, T1, T2} -> bind_error([Pat], T1, T2, opaque); error -> @@ -1453,7 +1457,7 @@ bind_pat_vars([Pat|PatLeft], [Type|TypeLeft], Acc, Map, State, Rev) -> Cons = t_inf(Type, t_cons(), Opaques), case t_is_none(Cons) of true -> - bind_opaque_pats(t_cons(), Type, Pat, Map, State, Rev); + bind_opaque_pats(t_cons(), Type, Pat, State); false -> {Map1, [HdType, TlType]} = bind_pat_vars([cerl:cons_hd(Pat), cerl:cons_tl(Pat)], @@ -1466,7 +1470,7 @@ bind_pat_vars([Pat|PatLeft], [Type|TypeLeft], Acc, Map, State, Rev) -> Literal = literal_type(Pat), case t_is_none(t_inf(Literal, Type, Opaques)) of true -> - bind_opaque_pats(Literal, Type, Pat, Map, State, Rev); + bind_opaque_pats(Literal, Type, Pat, State); false -> {Map, Literal} end; map -> @@ -1477,7 +1481,7 @@ bind_pat_vars([Pat|PatLeft], [Type|TypeLeft], Acc, Map, State, Rev) -> case Es of [] -> {false, t_tuple([])}; [Tag|Left] -> - case cerl:is_c_atom(Tag) of + case cerl:is_c_atom(Tag) andalso is_literal_record(Pat) of true -> TagAtom = cerl:atom_val(Tag), case state__lookup_record(TagAtom, length(Left), State) of @@ -1493,7 +1497,7 @@ bind_pat_vars([Pat|PatLeft], [Type|TypeLeft], Acc, Map, State, Rev) -> Tuple = t_inf(Prototype, Type, Opaques), case t_is_none(Tuple) of true -> - bind_opaque_pats(Prototype, Type, Pat, Map, State, Rev); + bind_opaque_pats(Prototype, Type, Pat, State); false -> SubTuples = t_tuple_subtypes(Tuple, Opaques), %% Need to call the top function to get the try-catch wrapper @@ -1542,7 +1546,7 @@ bind_pat_vars([Pat|PatLeft], [Type|TypeLeft], Acc, Map, State, Rev) -> VarType2 = t_inf(VarType1, Type, Opaques), case t_is_none(VarType2) of true -> - case t_find_opaque_mismatch(VarType1, Type) of + case t_find_opaque_mismatch(VarType1, Type, Opaques) of {ok, T1, T2} -> bind_error([Pat], T1, T2, opaque); error -> @@ -1608,10 +1612,18 @@ bind_bin_segs([Seg|Segs], BinType, Acc, Map, State) -> SizeVal = lists:max(List), Flags = cerl:concrete(cerl:bitstr_flags(Seg)), N = SizeVal * UnitVal, - case lists:member(signed, Flags) of - true -> t_from_range(-(1 bsl (N - 1)), 1 bsl (N - 1) - 1); - false -> t_from_range(0, 1 bsl N - 1) - end + case N >= ?BITS of + true -> + case lists:member(signed, Flags) of + true -> t_from_range(neg_inf, pos_inf); + false -> t_from_range(0, pos_inf) + end; + false -> + case lists:member(signed, Flags) of + true -> t_from_range(-(1 bsl (N - 1)), 1 bsl (N - 1) - 1); + false -> t_from_range(0, 1 bsl N - 1) + end + end end end, {Map2, [_]} = bind_pat_vars([Val], [ValConstr], [], Map1, State, false), @@ -1624,21 +1636,26 @@ bind_bin_segs([Seg|Segs], BinType, Acc, Map, State) -> bind_bin_segs([], _BinType, Acc, Map, _State) -> {Map, lists:reverse(Acc)}. -bind_error(Pats, Type, OpaqueType, Error) -> +bind_error(Pats, Type, OpaqueType, Error0) -> + Error = case {Error0, Pats} of + {bind, [Pat]} -> + case is_literal_record(Pat) of + true -> record; + false -> Error0 + end; + _ -> Error0 + end, throw({error, Error, Pats, Type, OpaqueType}). -bind_opaque_pats(GenType, Type, Pat, Map, State, Rev) -> - case t_find_opaque_mismatch(GenType, Type) of +-spec bind_opaque_pats(type(), type(), cerl:c_literal(), state()) -> + no_return(). + +bind_opaque_pats(GenType, Type, Pat, State) -> + case t_find_opaque_mismatch(GenType, Type, State#state.opaques) of {ok, T1, T2} -> - case erl_types:is_opaque_type(T2, State#state.opaques) of - true -> - NewType = erl_types:t_struct_from_opaque(Type, [T2]), - {Map1, _} = - bind_pat_vars([Pat], [NewType], [], Map, State, Rev), - {Map1, T2}; - false -> bind_error([Pat], T1, T2, opaque) - end; - error -> bind_error([Pat], Type, t_none(), bind) + bind_error([Pat], T1, T2, opaque); + error -> + bind_error([Pat], Type, t_none(), bind) end. %%---------------------------------------- @@ -1836,9 +1853,9 @@ handle_guard_comp(Guard, Comp, Map, Env, Eval, State) -> [Type1, Type2] = ArgTypes, IsInt1 = t_is_integer(Type1, Opaques), IsInt2 = t_is_integer(Type2, Opaques), - case {cerl:type(Arg1), cerl:type(Arg2)} of - {literal, literal} -> - case erlang:Comp(cerl:concrete(Arg1), cerl:concrete(Arg2)) of + case {type(Arg1), type(Arg2)} of + {{literal, Lit1}, {literal, Lit2}} -> + case erlang:Comp(cerl:concrete(Lit1), cerl:concrete(Lit2)) of true when Eval =:= pos -> {Map, t_atom(true)}; true when Eval =:= dont_know -> {Map, t_atom(true)}; true when Eval =:= neg -> {Map, t_atom(true)}; @@ -1847,13 +1864,13 @@ handle_guard_comp(Guard, Comp, Map, Env, Eval, State) -> false when Eval =:= dont_know -> {Map, t_atom(false)}; false when Eval =:= neg -> {Map, t_atom(false)} end; - {literal, var} when IsInt1 andalso IsInt2 andalso (Eval =:= pos) -> - case bind_comp_literal_var(Arg1, Arg2, Type2, Comp, Map1, Opaques) of + {{literal, Lit1}, var} when IsInt1 andalso IsInt2 andalso (Eval =:= pos) -> + case bind_comp_literal_var(Lit1, Arg2, Type2, Comp, Map1, Opaques) of error -> signal_guard_fail(Eval, Guard, ArgTypes, State); {ok, NewMap} -> {NewMap, t_atom(true)} end; - {var, literal} when IsInt1 andalso IsInt2 andalso (Eval =:= pos) -> - case bind_comp_literal_var(Arg2, Arg1, Type1, invert_comp(Comp), + {var, {literal, Lit2}} when IsInt1 andalso IsInt2 andalso (Eval =:= pos) -> + case bind_comp_literal_var(Lit2, Arg1, Type1, invert_comp(Comp), Map1, Opaques) of error -> signal_guard_fail(Eval, Guard, ArgTypes, State); {ok, NewMap} -> {NewMap, t_atom(true)} @@ -1973,15 +1990,15 @@ handle_guard_is_record(Guard, Map, Env, Eval, State) -> handle_guard_eq(Guard, Map, Env, Eval, State) -> [Arg1, Arg2] = cerl:call_args(Guard), - case {cerl:type(Arg1), cerl:type(Arg2)} of - {literal, literal} -> - case cerl:concrete(Arg1) =:= cerl:concrete(Arg2) of + case {type(Arg1), type(Arg2)} of + {{literal, Lit1}, {literal, Lit2}} -> + case cerl:concrete(Lit1) =:= cerl:concrete(Lit2) of true -> if Eval =:= pos -> {Map, t_atom(true)}; Eval =:= neg -> - ArgTypes = [t_from_term(cerl:concrete(Arg1)), - t_from_term(cerl:concrete(Arg2))], + ArgTypes = [t_from_term(cerl:concrete(Lit1)), + t_from_term(cerl:concrete(Lit2))], signal_guard_fail(Eval, Guard, ArgTypes, State); Eval =:= dont_know -> {Map, t_atom(true)} end; @@ -1990,28 +2007,28 @@ handle_guard_eq(Guard, Map, Env, Eval, State) -> Eval =:= neg -> {Map, t_atom(false)}; Eval =:= dont_know -> {Map, t_atom(false)}; Eval =:= pos -> - ArgTypes = [t_from_term(cerl:concrete(Arg1)), - t_from_term(cerl:concrete(Arg2))], + ArgTypes = [t_from_term(cerl:concrete(Lit1)), + t_from_term(cerl:concrete(Lit2))], signal_guard_fail(Eval, Guard, ArgTypes, State) end end; - {literal, _} when Eval =:= pos -> - case cerl:concrete(Arg1) of + {{literal, Lit1}, _} when Eval =:= pos -> + case cerl:concrete(Lit1) of Atom when is_atom(Atom) -> - bind_eqeq_guard_lit_other(Guard, Arg1, Arg2, Map, Env, State); + bind_eqeq_guard_lit_other(Guard, Lit1, Arg2, Map, Env, State); [] -> - bind_eqeq_guard_lit_other(Guard, Arg1, Arg2, Map, Env, State); + bind_eqeq_guard_lit_other(Guard, Lit1, Arg2, Map, Env, State); _ -> - bind_eq_guard(Guard, Arg1, Arg2, Map, Env, Eval, State) + bind_eq_guard(Guard, Lit1, Arg2, Map, Env, Eval, State) end; - {_, literal} when Eval =:= pos -> - case cerl:concrete(Arg2) of + {_, {literal, Lit2}} when Eval =:= pos -> + case cerl:concrete(Lit2) of Atom when is_atom(Atom) -> - bind_eqeq_guard_lit_other(Guard, Arg2, Arg1, Map, Env, State); + bind_eqeq_guard_lit_other(Guard, Lit2, Arg1, Map, Env, State); [] -> - bind_eqeq_guard_lit_other(Guard, Arg2, Arg1, Map, Env, State); + bind_eqeq_guard_lit_other(Guard, Lit2, Arg1, Map, Env, State); _ -> - bind_eq_guard(Guard, Arg1, Arg2, Map, Env, Eval, State) + bind_eq_guard(Guard, Arg1, Lit2, Map, Env, Eval, State) end; {_, _} -> bind_eq_guard(Guard, Arg1, Arg2, Map, Env, Eval, State) @@ -2043,13 +2060,14 @@ bind_eq_guard(Guard, Arg1, Arg2, Map, Env, Eval, State) -> handle_guard_eqeq(Guard, Map, Env, Eval, State) -> [Arg1, Arg2] = cerl:call_args(Guard), - case {cerl:type(Arg1), cerl:type(Arg2)} of - {literal, literal} -> - case cerl:concrete(Arg1) =:= cerl:concrete(Arg2) of + case {type(Arg1), type(Arg2)} of + {{literal, Lit1}, {literal, Lit2}} -> + + case cerl:concrete(Lit1) =:= cerl:concrete(Lit2) of true -> if Eval =:= neg -> - ArgTypes = [t_from_term(cerl:concrete(Arg1)), - t_from_term(cerl:concrete(Arg2))], + ArgTypes = [t_from_term(cerl:concrete(Lit1)), + t_from_term(cerl:concrete(Lit2))], signal_guard_fail(Eval, Guard, ArgTypes, State); Eval =:= pos -> {Map, t_atom(true)}; Eval =:= dont_know -> {Map, t_atom(true)} @@ -2058,15 +2076,15 @@ handle_guard_eqeq(Guard, Map, Env, Eval, State) -> if Eval =:= neg -> {Map, t_atom(false)}; Eval =:= dont_know -> {Map, t_atom(false)}; Eval =:= pos -> - ArgTypes = [t_from_term(cerl:concrete(Arg1)), - t_from_term(cerl:concrete(Arg2))], + ArgTypes = [t_from_term(cerl:concrete(Lit1)), + t_from_term(cerl:concrete(Lit2))], signal_guard_fail(Eval, Guard, ArgTypes, State) end end; - {literal, _} when Eval =:= pos -> - bind_eqeq_guard_lit_other(Guard, Arg1, Arg2, Map, Env, State); - {_, literal} when Eval =:= pos -> - bind_eqeq_guard_lit_other(Guard, Arg2, Arg1, Map, Env, State); + {{literal, Lit1}, _} when Eval =:= pos -> + bind_eqeq_guard_lit_other(Guard, Lit1, Arg2, Map, Env, State); + {_, {literal, Lit2}} when Eval =:= pos -> + bind_eqeq_guard_lit_other(Guard, Lit2, Arg1, Map, Env, State); {_, _} -> bind_eqeq_guard(Guard, Arg1, Arg2, Map, Env, Eval, State) end. @@ -3275,12 +3293,17 @@ get_file([_|Tail]) -> get_file(Tail). is_compiler_generated(Ann) -> lists:member(compiler_generated, Ann) orelse (get_line(Ann) < 1). +is_literal_record(Tree) -> + Ann = cerl:get_ann(Tree), + lists:member(record, Ann). + -spec format_args([cerl:cerl()], [type()], state()) -> nonempty_string(). format_args([], [], _State) -> "()"; -format_args(ArgList, TypeList, State) -> +format_args(ArgList0, TypeList, State) -> + ArgList = fold_literals(ArgList0), "(" ++ format_args_1(ArgList, TypeList, State) ++ ")". format_args_1([Arg], [Type], State) -> @@ -3339,7 +3362,8 @@ format_cerl(Tree) -> {ribbon, 100000} %% newlines. ]). -format_patterns(Pats) -> +format_patterns(Pats0) -> + Pats = fold_literals(Pats0), NewPats = map_pats(cerl:c_values(Pats)), String = format_cerl(NewPats), case Pats of @@ -3371,6 +3395,23 @@ map_pats(Pats) -> end, cerl_trees:map(Fun, Pats). +fold_literals(TreeList) -> + [cerl:fold_literal(Tree) || Tree <- TreeList]. + +type(Tree) -> + Folded = cerl:fold_literal(Tree), + case cerl:type(Folded) of + literal -> {literal, Folded}; + Type -> Type + end. + +is_literal(Tree) -> + Folded = cerl:fold_literal(Tree), + case cerl:is_literal(Folded) of + true -> {yes, Folded}; + false -> no + end. + parent_allows_this(FunLbl, #state{callgraph = Callgraph, plt = Plt} =State) -> case state__is_escaping(FunLbl, State) of false -> false; % if it isn't escaping it can't be a return value @@ -3415,18 +3456,18 @@ find_terminals(Tree) -> M0 = cerl:call_module(Tree), F0 = cerl:call_name(Tree), A = length(cerl:call_args(Tree)), - case cerl:is_literal(M0) andalso cerl:is_literal(F0) of - false -> - %% We cannot make assumptions. Say that both are true. - {true, true}; - true -> - M = cerl:concrete(M0), - F = cerl:concrete(F0), + case {is_literal(M0), is_literal(F0)} of + {{yes, LitM}, {yes, LitF}} -> + M = cerl:concrete(LitM), + F = cerl:concrete(LitF), case (erl_bif_types:is_known(M, F, A) andalso t_is_none(erl_bif_types:type(M, F, A))) of true -> {true, false}; false -> {false, true} - end + end; + _ -> + %% We cannot make assumptions. Say that both are true. + {true, true} end; 'case' -> find_terminals_list(cerl:case_clauses(Tree)); 'catch' -> find_terminals(cerl:catch_body(Tree)); @@ -3471,66 +3512,6 @@ find_terminals_list([], Explicit, Normal) -> %%---------------------------------------------------------------------------- -%% If you write a record pattern in a matching that violates the -%% definition it will never match. However, the warning is lost in the -%% regular analysis. This after-pass catches it. - -find_mismatched_record_patterns(Tree, State) -> - cerl_trees:fold( - fun(SubTree, AccState) -> - case cerl:is_c_clause(SubTree) of - true -> lists:foldl(fun(P, AccState1) -> - find_rec_warnings(P, AccState1) - end, AccState, cerl:clause_pats(SubTree)); - false -> AccState - end - end, State, Tree). - -find_rec_warnings(Tree, State) -> - cerl_trees:fold( - fun(SubTree, AccState) -> - case cerl:is_c_tuple(SubTree) of - true -> find_rec_warnings_tuple(SubTree, AccState); - false -> AccState - end - end, State, Tree). - -find_rec_warnings_tuple(Tree, State) -> - Elements = cerl:tuple_es(Tree), - {_, _, EsType} = traverse_list(Elements, map__new(), State), - TupleType = t_tuple(EsType), - case t_is_none(TupleType) of - true -> State; - false -> - %% Let's find out if this is a record construction. - case Elements of - [Tag|Left] -> - case cerl:is_c_atom(Tag) of - true -> - TagVal = cerl:atom_val(Tag), - case state__lookup_record(TagVal, length(Left), State) of - error -> State; - {ok, Prototype} -> - InfTupleType = t_inf(Prototype, TupleType), - case t_is_none(InfTupleType) of - true -> - Msg = {record_matching, - [format_patterns([Tree]), TagVal]}, - state__add_warning(State, ?WARN_MATCHING, Tree, Msg); - false -> - State - end - end; - false -> - State - end; - _ -> - State - end - end. - -%%---------------------------------------------------------------------------- - -ifdef(DEBUG_PP). debug_pp(Tree, true) -> io:put_chars(cerl_prettypr:format(Tree, [{hook, cerl_typean:pp_hook()}])), diff --git a/lib/dialyzer/src/dialyzer_dep.erl b/lib/dialyzer/src/dialyzer_dep.erl index e3ece144c9..572e60278d 100644 --- a/lib/dialyzer/src/dialyzer_dep.erl +++ b/lib/dialyzer/src/dialyzer_dep.erl @@ -124,8 +124,10 @@ traverse(Tree, Out, State, CurrentFun) -> TmpState = state__add_deps(Label, O1, State), state__add_deps(CurrentFun, O2,TmpState) end, - {BodyFuns, State2} = traverse(Body, Out, State1, - cerl_trees:get_label(Tree)), + Vars = cerl:fun_vars(Tree), + Out1 = bind_single(Vars, output(set__singleton(external)), Out), + {BodyFuns, State2} = + traverse(Body, Out1, State1, cerl_trees:get_label(Tree)), {output(set__singleton(Label)), state__add_esc(BodyFuns, State2)}; 'let' -> Vars = cerl:let_vars(Tree), diff --git a/lib/dialyzer/src/dialyzer_gui_wx.erl b/lib/dialyzer/src/dialyzer_gui_wx.erl index 7070fa240d..868857d675 100644 --- a/lib/dialyzer/src/dialyzer_gui_wx.erl +++ b/lib/dialyzer/src/dialyzer_gui_wx.erl @@ -699,8 +699,7 @@ handle_add_files(#gui_state{chosen_box = ChosenBox, file_box = FileBox, end. handle_add_dir(#gui_state{chosen_box = ChosenBox, dir_entry = DirBox, - files_to_analyze = FileList, - mode = Mode} = State) -> + files_to_analyze = FileList, mode = Mode} = State) -> case wxDirPickerCtrl:getPath(DirBox) of "" -> State; @@ -714,8 +713,8 @@ handle_add_dir(#gui_state{chosen_box = ChosenBox, dir_entry = DirBox, State#gui_state{files_to_analyze = add_files(filter_mods(NewDir1,Ext), FileList, ChosenBox, Ext)} end. -handle_add_rec(#gui_state{chosen_box = ChosenBox, dir_entry = DirBox, files_to_analyze = FileList, - mode = Mode} = State) -> +handle_add_rec(#gui_state{chosen_box = ChosenBox, dir_entry = DirBox, + files_to_analyze = FileList, mode = Mode} = State) -> case wxDirPickerCtrl:getPath(DirBox) of "" -> State; @@ -723,11 +722,11 @@ handle_add_rec(#gui_state{chosen_box = ChosenBox, dir_entry = DirBox, files_to_a NewDir = ordsets:new(), NewDir1 = ordsets:add_element(Dir,NewDir), TargetDirs = ordsets:union(NewDir1, all_subdirs(NewDir1)), - case wxRadioBox:getSelection(Mode) of - 0 -> Ext = ".beam"; - 1-> Ext = ".erl" - end, - State#gui_state{files_to_analyze = add_files(filter_mods(TargetDirs,Ext), FileList, ChosenBox, Ext)} + Ext = case wxRadioBox:getSelection(Mode) of + 0 -> ".beam"; + 1 -> ".erl" + end, + State#gui_state{files_to_analyze = add_files(filter_mods(TargetDirs, Ext), FileList, ChosenBox, Ext)} end. handle_file_delete(#gui_state{chosen_box = ChosenBox, @@ -886,13 +885,10 @@ config_gui_start(State) -> wxRadioBox:disable(State#gui_state.mode). save_file(#gui_state{frame = Frame, warnings_box = WBox, log = Log} = State, Type) -> - case Type of - warnings -> - Message = "Save Warnings", - Box = WBox; - log -> Message = "Save Log", - Box = Log - end, + {Message, Box} = case Type of + warnings -> {"Save Warnings", WBox}; + log -> {"Save Log", Log} + end, case wxTextCtrl:getValue(Box) of "" -> error_sms(State,"There is nothing to save...\n"); _ -> @@ -936,8 +932,7 @@ include_dialog(#gui_state{gui = Wx, frame = Frame, options = Options}) -> wxButton:connect(DeleteAllButton, command_button_clicked), wxButton:connect(Ok, command_button_clicked), wxButton:connect(Cancel, command_button_clicked), - Dirs = [io_lib:format("~s", [X]) - || X <- Options#options.include_dirs], + Dirs = [io_lib:format("~s", [X]) || X <- Options#options.include_dirs], wxListBox:set(Box, Dirs), Layout = wxBoxSizer:new(?wxVERTICAL), Buttons = wxBoxSizer:new(?wxHORIZONTAL), diff --git a/lib/dialyzer/src/dialyzer_races.erl b/lib/dialyzer/src/dialyzer_races.erl index b1f849b16f..2a8aba5d8f 100644 --- a/lib/dialyzer/src/dialyzer_races.erl +++ b/lib/dialyzer/src/dialyzer_races.erl @@ -990,8 +990,7 @@ fixup_race_forward_helper(CurrFun, CurrFunLabel, Fun, FunLabel, NewRaceVarMap, Args, NewFunArgs, NewFunTypes, NestingLevel}; {CurrFun, Fun} -> NewCallsToAnalyze = lists:delete(Head, CallsToAnalyze), - NewRaceVarMap = - race_var_map(Args, NewFunArgs, RaceVarMap, bind), + NewRaceVarMap = race_var_map(Args, NewFunArgs, RaceVarMap, bind), RetC = case Fun of InitFun -> @@ -1018,8 +1017,7 @@ fixup_race_forward_helper(CurrFun, CurrFunLabel, Fun, FunLabel, label = FunLabel, var_map = NewRaceVarMap, def_vars = Args, call_vars = NewFunArgs, arg_types = NewFunTypes}| - lists:reverse(StateRaceList)] ++ - RetC; + lists:reverse(StateRaceList)] ++ RetC; _ -> [#curr_fun{status = in, mfa = Fun, label = FunLabel, var_map = NewRaceVarMap, @@ -1054,13 +1052,9 @@ fixup_race_backward(CurrFun, Calls, CallsToAnalyze, Parents, Height) -> false -> [CurrFun|Parents] end; [Head|Tail] -> - MorePaths = - case Head of - {Parent, CurrFun} -> true; - {Parent, _TupleB} -> false - end, - case MorePaths of - true -> + {Parent, TupleB} = Head, + case TupleB =:= CurrFun of + true -> % more paths are needed NewCallsToAnalyze = lists:delete(Head, CallsToAnalyze), NewParents = fixup_race_backward(Parent, NewCallsToAnalyze, @@ -1854,7 +1848,8 @@ ets_tuple_argtypes1(Str, Tuple, TupleList, NestingLevel) -> end. format_arg(?bypassed) -> ?no_label; -format_arg(Arg) -> +format_arg(Arg0) -> + Arg = cerl:fold_literal(Arg0), case cerl:type(Arg) of var -> cerl_trees:get_label(Arg); tuple -> list_to_tuple([format_arg(A) || A <- cerl:tuple_es(Arg)]); @@ -1884,7 +1879,7 @@ format_args_1([Arg|Args], [Type|Types], CleanState) -> case Arg =:= ?bypassed of true -> [?no_label, format_type(Type, CleanState)]; false -> - case cerl:is_literal(Arg) of + case cerl:is_literal(cerl:fold_literal(Arg)) of true -> [?no_label, format_cerl(Arg)]; false -> [format_arg(Arg), format_type(Type, CleanState)] end @@ -2154,7 +2149,8 @@ race_var_map_guard_helper1(Arg, Pats, RaceVarMap, Op) -> end end. -race_var_map_guard_helper2(Arg, Pat, Bool, RaceVarMap, Op) -> +race_var_map_guard_helper2(Arg, Pat0, Bool, RaceVarMap, Op) -> + Pat = cerl:fold_literal(Pat0), case cerl:type(Pat) of literal -> [Arg1, Arg2] = cerl:call_args(Arg), diff --git a/lib/dialyzer/src/dialyzer_typesig.erl b/lib/dialyzer/src/dialyzer_typesig.erl index 31ceaf5ac5..3d03ed3ab3 100644 --- a/lib/dialyzer/src/dialyzer_typesig.erl +++ b/lib/dialyzer/src/dialyzer_typesig.erl @@ -389,13 +389,8 @@ traverse(Tree, DefinedVars, State) -> {State2, _} = traverse_list(Funs, DefinedVars1, State1), traverse(Body, DefinedVars1, State2); literal -> - %% This is needed for finding records - case cerl:unfold_literal(Tree) of - Tree -> - Type = t_from_term(cerl:concrete(Tree)), - {State, Type}; - NewTree -> traverse(NewTree, DefinedVars, State) - end; + Type = t_from_term(cerl:concrete(Tree)), + {State, Type}; module -> Defs = cerl:module_defs(Tree), Funs = [Fun || {_Var, Fun} <- Defs], @@ -462,7 +457,7 @@ traverse(Tree, DefinedVars, State) -> end, case Elements of [Tag|Fields] -> - case cerl:is_c_atom(Tag) of + case cerl:is_c_atom(Tag) andalso is_literal_record(Tree) of true -> %% Check if a record is constructed. Arity = length(Fields), @@ -874,7 +869,8 @@ get_underapprox_from_guard(Tree, Map) -> MFA -> case get_type_test(MFA) of {ok, Type} -> - [Arg] = cerl:call_args(Tree), + [Arg0] = cerl:call_args(Tree), + Arg = cerl:fold_literal(Arg0), {ArgType, Map1} = get_underapprox_from_guard(Arg, Map), Inf = t_inf(Type, ArgType), case t_is_none(Inf) of @@ -891,7 +887,9 @@ get_underapprox_from_guard(Tree, Map) -> {erlang, '=:=', 2} -> throw(dont_know); {erlang, '==', 2} -> throw(dont_know); {erlang, 'and', 2} -> - [Arg1, Arg2] = cerl:call_args(Tree), + [Arg1_0, Arg2_0] = cerl:call_args(Tree), + Arg1 = cerl:fold_literal(Arg1_0), + Arg2 = cerl:fold_literal(Arg2_0), case ((cerl:is_c_var(Arg1) orelse cerl:is_literal(Arg1)) andalso (cerl:is_c_var(Arg2) orelse cerl:is_literal(Arg2))) of @@ -3272,6 +3270,10 @@ lookup_record(Records, Tag, Arity) -> error end. +is_literal_record(Tree) -> + Ann = cerl:get_ann(Tree), + lists:member(record, Ann). + family(L) -> sofs:to_external(sofs:rel2fam(sofs:relation(L))). diff --git a/lib/dialyzer/src/dialyzer_utils.erl b/lib/dialyzer/src/dialyzer_utils.erl index 21183e3459..4e2ec67b35 100644 --- a/lib/dialyzer/src/dialyzer_utils.erl +++ b/lib/dialyzer/src/dialyzer_utils.erl @@ -31,6 +31,7 @@ format_sig/1, format_sig/2, get_abstract_code_from_beam/1, + get_compile_options_from_beam/1, get_abstract_code_from_src/1, get_abstract_code_from_src/2, get_core_from_abstract_code/1, @@ -136,6 +137,26 @@ get_abstract_code_from_beam(File) -> error end. +-spec get_compile_options_from_beam(file:filename()) -> 'error' | {'ok', [compile:option()]}. + +get_compile_options_from_beam(File) -> + case beam_lib:chunks(File, [compile_info]) of + {ok, {_, List}} -> + case lists:keyfind(compile_info, 1, List) of + {compile_info, CompInfo} -> compile_info_to_options(CompInfo); + _ -> error + end; + _ -> + %% No or unsuitable compile info. + error + end. + +compile_info_to_options(CompInfo) -> + case lists:keyfind(options, 1, CompInfo) of + {options, CompOpts} -> {ok, CompOpts}; + _ -> error + end. + -type get_core_from_abs_ret() :: {'ok', cerl:c_module()} | 'error'. -spec get_core_from_abstract_code(abstract_code()) -> get_core_from_abs_ret(). @@ -150,7 +171,9 @@ get_core_from_abstract_code(AbstrCode, Opts) -> %% performed them. In some cases we end up in trouble when %% performing them again. AbstrCode1 = cleanup_parse_transforms(AbstrCode), - try compile:forms(AbstrCode1, Opts ++ src_compiler_opts()) of + %% Remove parse_transforms (and other options) from compile options. + Opts2 = cleanup_compile_options(Opts), + try compile:forms(AbstrCode1, Opts2 ++ src_compiler_opts()) of {ok, _, Core} -> {ok, Core}; _What -> error catch @@ -402,7 +425,7 @@ sets_filter([Mod|Mods], ExpTypes) -> src_compiler_opts() -> [no_copt, to_core, binary, return_errors, no_inline, strict_record_tests, strict_record_updates, - no_is_record_optimization]. + dialyzer]. -spec get_module(abstract_code()) -> module(). @@ -419,6 +442,24 @@ cleanup_parse_transforms([Other|Left]) -> cleanup_parse_transforms([]) -> []. +-spec cleanup_compile_options([compile:option()]) -> [compile:option()]. + +%% Using abstract, not asm or core. +cleanup_compile_options([from_asm|Opts]) -> + Opts; +cleanup_compile_options([asm|Opts]) -> + Opts; +cleanup_compile_options([from_core|Opts]) -> + Opts; +%% The parse transform will already have been applied, may cause problems if it +%% is re-applied. +cleanup_compile_options([{parse_transform, _}|Opts]) -> + Opts; +cleanup_compile_options([Other|Opts]) -> + [Other|cleanup_compile_options(Opts)]; +cleanup_compile_options([]) -> + []. + -spec format_errors([{module(), string()}]) -> [string()]. format_errors([{Mod, Errors}|Left]) -> diff --git a/lib/dialyzer/test/dialyzer_SUITE.erl b/lib/dialyzer/test/dialyzer_SUITE.erl index 1b62291a00..8507525597 100644 --- a/lib/dialyzer/test/dialyzer_SUITE.erl +++ b/lib/dialyzer/test/dialyzer_SUITE.erl @@ -30,12 +30,12 @@ -export([init_per_testcase/2, end_per_testcase/2]). %% Test cases must be exported. --export([app_test/1, appup_test/1]). +-export([app_test/1, appup_test/1, beam_tests/1]). suite() -> [{ct_hooks,[ts_install_cth]}]. all() -> - [app_test, appup_test]. + [app_test, appup_test, beam_tests]. groups() -> []. @@ -75,3 +75,38 @@ app_test(Config) when is_list(Config) -> %% Test that the .appup file does not contain any `basic' errors appup_test(Config) when is_list(Config) -> ok = ?t:appup_test(dialyzer). + +beam_tests(Config) when is_list(Config) -> + Prog = <<" + -module(no_auto_import). + + %% Copied from erl_lint_SUITE.erl, clash6 + + -export([size/1]). + + size([]) -> + 0; + size({N,_}) -> + N; + size([_|T]) -> + 1+size(T). + ">>, + Opts = [no_auto_import], + {ok, BeamFile} = compile(Config, Prog, no_auto_import, Opts), + [] = run_dialyzer([BeamFile]), + ok. + +compile(Config, Prog, Module, CompileOpts) -> + Source = lists:concat([Module, ".erl"]), + PrivDir = ?config(priv_dir,Config), + Filename = filename:join([PrivDir, Source]), + ok = file:write_file(Filename, Prog), + Opts = [{outdir, PrivDir}, debug_info | CompileOpts], + {ok, Module} = compile:file(Filename, Opts), + {ok, filename:join([PrivDir, lists:concat([Module, ".beam"])])}. + +run_dialyzer(Files) -> + dialyzer:run([{analysis_type, plt_build}, + {files, Files}, + {from, byte_code}, + {check_plt, false}]). diff --git a/lib/dialyzer/test/opaque_SUITE_data/results/simple b/lib/dialyzer/test/opaque_SUITE_data/results/simple index 072ac9be8f..29864d6065 100644 --- a/lib/dialyzer/test/opaque_SUITE_data/results/simple +++ b/lib/dialyzer/test/opaque_SUITE_data/results/simple @@ -14,12 +14,17 @@ is_rec.erl:53: The call erlang:is_record(A::simple1_adt:d1(),A::simple1_adt:d1() is_rec.erl:57: Guard test is_record(A::simple1_adt:d1(),'r',2) breaks the opaqueness of its argument is_rec.erl:61: The record #r{f1::simple1_adt:d1()} violates the declared type for #r{} is_rec.erl:65: The call erlang:is_record({simple1_adt:d1(),1},'r',2) contains an opaque term as 1st argument when terms of different types are expected in these positions -rec_api.erl:22: Record construction #r1{f1::10} violates the declared type of field f1::'undefined' | rec_api:a() -rec_api.erl:23: The pattern {'r1', 10} violates the declared type for #r1{} -rec_api.erl:27: The attempt to match a term of type rec_adt:r1() against the pattern {'r1', 'a'} breaks the opaqueness of the term -rec_api.erl:29: Invalid type specification for function rec_api:adt_t1/1. The success typing is (#r1{f1::'a'}) -> #r1{f1::'a'} -rec_api.erl:34: Invalid type specification for function rec_api:adt_r1/0. The success typing is () -> #r1{f1::'a'} -rec_api.erl:77: The attempt to match a term of type rec_api:f() against the variable _ breaks the opaqueness of the term +rec_api.erl:104: Matching of pattern {'r2', 10} tagged with a record name violates the declared type of #r2{f1::10} +rec_api.erl:113: The attempt to match a term of type #r3{f1::queue:queue(_)} against the pattern {'r3', 'a'} breaks the opaqueness of queue:queue(_) +rec_api.erl:118: Record construction #r3{f1::10} violates the declared type of field f1::queue:queue(_) +rec_api.erl:123: The attempt to match a term of type #r3{f1::10} against the pattern {'r3', 10} breaks the opaqueness of queue:queue(_) +rec_api.erl:24: Record construction #r1{f1::10} violates the declared type of field f1::'undefined' | rec_api:a() +rec_api.erl:29: Matching of pattern {'r1', 10} tagged with a record name violates the declared type of #r1{f1::10} +rec_api.erl:33: The attempt to match a term of type rec_adt:r1() against the pattern {'r1', 'a'} breaks the opaqueness of the term +rec_api.erl:35: Invalid type specification for function rec_api:adt_t1/1. The success typing is (#r1{f1::'a'}) -> #r1{f1::'a'} +rec_api.erl:40: Invalid type specification for function rec_api:adt_r1/0. The success typing is () -> #r1{f1::'a'} +rec_api.erl:85: The attempt to match a term of type rec_api:f() against the variable _ breaks the opaqueness of rec_adt:f() +rec_api.erl:99: Record construction #r2{f1::10} violates the declared type of field f1::rec_api:a() simple1_api.erl:113: The test simple1_api:d1() =:= simple1_api:d2() can never evaluate to 'true' simple1_api.erl:118: Guard test simple1_api:d2() =:= A::simple1_api:d1() can never succeed simple1_api.erl:142: Attempt to test for equality between a term of type simple1_adt:o2() and a term of opaque type simple1_adt:o1() @@ -58,7 +63,7 @@ simple1_api.erl:381: Invalid type specification for function simple1_api:bool_ad simple1_api.erl:407: The size simple1_adt:i1() breaks the opaqueness of A simple1_api.erl:418: The attempt to match a term of type non_neg_integer() against the variable A breaks the opaqueness of simple1_adt:i1() simple1_api.erl:425: The attempt to match a term of type non_neg_integer() against the variable B breaks the opaqueness of simple1_adt:i1() -simple1_api.erl:432: The attempt to match a term of type non_neg_integer() against the variable B breaks the opaqueness of simple1_api:o1() +simple1_api.erl:432: The pattern <<_:B/integer-unit:1>> can never match the type any() simple1_api.erl:448: The attempt to match a term of type non_neg_integer() against the variable Sz breaks the opaqueness of simple1_adt:i1() simple1_api.erl:460: The attempt to match a term of type simple1_adt:bit1() against the pattern <<_/binary-unit:8>> breaks the opaqueness of the term simple1_api.erl:478: The call 'foo':A(A::simple1_adt:a()) breaks the opaqueness of the term A :: simple1_adt:a() diff --git a/lib/dialyzer/test/opaque_SUITE_data/results/timer b/lib/dialyzer/test/opaque_SUITE_data/results/timer index e917b76b08..b1cfcd4e9f 100644 --- a/lib/dialyzer/test/opaque_SUITE_data/results/timer +++ b/lib/dialyzer/test/opaque_SUITE_data/results/timer @@ -1,4 +1,4 @@ timer_use.erl:16: The pattern 'gazonk' can never match the type {'error',_} | {'ok',timer:tref()} -timer_use.erl:17: The attempt to match a term of type {'ok',timer:tref()} against the pattern {'ok', 42} breaks the opaqueness of timer:tref() +timer_use.erl:17: The attempt to match a term of type {'error',_} | {'ok',timer:tref()} against the pattern {'ok', 42} breaks the opaqueness of timer:tref() timer_use.erl:18: The attempt to match a term of type {'error',_} | {'ok',timer:tref()} against the pattern {Tag, 'gazonk'} breaks the opaqueness of timer:tref() diff --git a/lib/dialyzer/test/opaque_SUITE_data/src/opaque/opaque_bug5.erl b/lib/dialyzer/test/opaque_SUITE_data/src/opaque/opaque_bug5.erl new file mode 100644 index 0000000000..28d739de8e --- /dev/null +++ b/lib/dialyzer/test/opaque_SUITE_data/src/opaque/opaque_bug5.erl @@ -0,0 +1,10 @@ +%% Second arg of is_record call wasn't checked properly + +-module(opaque_bug5). + +-export([b/0]). + +b() -> + is_record(id({a}), id(a)). + +id(I) -> I. diff --git a/lib/dialyzer/test/opaque_SUITE_data/src/simple/rec_api.erl b/lib/dialyzer/test/opaque_SUITE_data/src/simple/rec_api.erl index d9b1d59f0c..fb6d59d263 100644 --- a/lib/dialyzer/test/opaque_SUITE_data/src/simple/rec_api.erl +++ b/lib/dialyzer/test/opaque_SUITE_data/src/simple/rec_api.erl @@ -1,9 +1,9 @@ -module(rec_api). --export([t1/0, t2/0, adt_t1/0, adt_t1/1, adt_r1/0, - t/1, t_adt/0, r/0, r_adt/0]). +-export([t1/0, t2/0, t3/0, adt_t1/0, adt_t1/1, adt_r1/0, + t/1, t_adt/0, r/0, r_adt/0, u1/0, u2/0, u3/0, v1/0, v2/0, v3/0]). --export_type([{a,0},{r1,0}]). +-export_type([{a,0},{r1,0}, r2/0, r3/0]). -export_type([f/0, op_t/0, r/0, tup/0]). @@ -19,8 +19,14 @@ t1() -> {r1, a} = A. t2() -> - A = {r1, 10}, % violates the type of #r1{} - {r1, 10} = A. % violates the type of #r1{} + A = {r1, 10}, + {r1, 10} = A, + A = #r1{f1 = 10}, % violates the type of field f1 + #r1{f1 = 10} = A. + +t3() -> + A = {r1, 10}, + #r1{f1 = 10} = A. % violates the type of #r1{} adt_t1() -> R = rec_adt:r1(), @@ -66,7 +72,8 @@ t_adt() -> -spec r() -> _. r() -> - {r, f(), 2}. % OK, f() is a local opaque type + {{r, f(), 2}, + #r{f = f(), o = 2}}. % OK, f() is a local opaque type -spec f() -> f(). @@ -74,4 +81,43 @@ f() -> fun(_) -> 3 end. r_adt() -> - {r, rec_adt:f(), 2}. % breaks the opaqueness + {{r, rec_adt:f(), 2}, + #r{f = rec_adt:f(), o = 2}}. % breaks the opaqueness + +-record(r2, % like #r1{}, but with initial value + {f1 = a :: a()}). + +-opaque r2() :: #r2{}. + +u1() -> + A = #r2{f1 = a}, + {r2, a} = A. + +u2() -> + A = {r2, 10}, + {r2, 10} = A, + A = #r2{f1 = 10}, % violates the type of field f1 + #r2{f1 = 10} = A. + +u3() -> + A = {r2, 10}, + #r2{f1 = 10} = A. % violates the type of #r2{} + +-record(r3, % like #r1{}, but an opaque type + {f1 = queue:new():: queue:queue()}). + +-opaque r3() :: #r3{}. + +v1() -> + A = #r3{f1 = queue:new()}, + {r3, a} = A. % breaks the opaqueness + +v2() -> + A = {r3, 10}, + {r3, 10} = A, + A = #r3{f1 = 10}, % violates the type of field f1 + #r3{f1 = 10} = A. + +v3() -> + A = {r3, 10}, + #r3{f1 = 10} = A. % breaks the opaqueness diff --git a/lib/dialyzer/test/opaque_SUITE_data/src/simple/simple1_api.erl b/lib/dialyzer/test/opaque_SUITE_data/src/simple/simple1_api.erl index 5135eb8e59..eef2074e0c 100644 --- a/lib/dialyzer/test/opaque_SUITE_data/src/simple/simple1_api.erl +++ b/lib/dialyzer/test/opaque_SUITE_data/src/simple/simple1_api.erl @@ -428,8 +428,8 @@ bit_adt_t3(A) -> bit_t5(A) -> B = o1(), - case none:none() of - <<A:B>> -> 1 % breaks the opaqueness + case none:none() of % the type is any(); should fix that XXX + <<A:B>> -> 1 % can never match (local opaque type is OK) end. -spec bit_t4(<<_:1>>) -> integer(). diff --git a/lib/dialyzer/test/options1_SUITE_data/results/compiler b/lib/dialyzer/test/options1_SUITE_data/results/compiler index 6399e3e36b..30b6f4814a 100644 --- a/lib/dialyzer/test/options1_SUITE_data/results/compiler +++ b/lib/dialyzer/test/options1_SUITE_data/results/compiler @@ -4,7 +4,7 @@ beam_bool.erl:193: The pattern {[], _} can never match the type {[{_,_,_,_},...] beam_bool.erl:510: The pattern [{'set', [Dst], _, _}, {'%live', _}] can never match the type [{_,_,_,_}] beam_disasm.erl:537: The variable X can never match since previous clauses completely covered the type 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 beam_type.erl:284: The pattern <'pi', 0> can never match the type <_,1 | 2> -beam_validator.erl:396: The pattern <{'jump', {'f', _}}, Vst = {'vst', 'none', _}> can never match the type <_,#vst{current::#st{ct::[]}}> +beam_validator.erl:396: Matching of pattern {'vst', 'none', _} tagged with a record name violates the declared type of #vst{current::#st{ct::[]}} beam_validator.erl:690: The pattern <'term', OldT> can never match the type <{'tuple',[any(),...]},_> beam_validator.erl:693: Guard test 'or'('false','false') can never succeed beam_validator.erl:700: Guard test 'or'('false','false') can never succeed @@ -33,4 +33,4 @@ core_lint.erl:473: The pattern <{'c_atom', _, 'all'}, 'binary', _Def, St> can ne core_lint.erl:505: The pattern <_Req, 'unknown', St> can never match the type <non_neg_integer(),non_neg_integer(),_> v3_codegen.erl:1569: The call v3_codegen:load_reg_1(V::any(),I::0,Rs::any(),pos_integer()) will never return since it differs in the 4th argument from the success typing arguments: (any(),0,maybe_improper_list(),0) v3_codegen.erl:1571: The call v3_codegen:load_reg_1(V::any(),I::0,[],pos_integer()) will never return since it differs in the 4th argument from the success typing arguments: (any(),0,maybe_improper_list(),0) -v3_core.erl:646: The pattern <Prim = {'iprimop', _, _, _}, St> can never match the type <#c_nil{anno::[any(),...]} | {'c_atom' | 'c_char' | 'c_float' | 'c_int' | 'c_string' | 'c_tuple' | 'c_var' | 'ibinary' | 'icatch' | 'ireceive1',[any(),...] | {_,_,_,_},_} | #c_cons{anno::[any(),...]} | #c_fname{anno::[any(),...]} | #iletrec{anno::{_,_,_,_},defs::[any(),...],body::[any(),...]} | #icase{anno::{_,_,_,_},args::[any()],clauses::[any()],fc::{_,_,_,_,_,_}} | #ireceive2{anno::{_,_,_,_},clauses::[any()],action::[any()]} | #ifun{anno::{_,_,_,_},id::[any(),...],vars::[any()],clauses::[any(),...],fc::{_,_,_,_,_,_}} | #imatch{anno::{_,_,_,_},guard::[],fc::{_,_,_,_,_,_}} | #itry{anno::{_,_,_,_},args::[any()],vars::[any(),...],body::[any(),...],evars::[any(),...],handler::[any(),...]},_> +v3_core.erl:646: Matching of pattern {'iprimop', _, _, _} tagged with a record name violates the declared type of #c_nil{anno::[any(),...]} | {'c_atom' | 'c_char' | 'c_float' | 'c_int' | 'c_string' | 'c_tuple' | 'c_var' | 'ibinary' | 'icatch' | 'ireceive1',[any(),...] | {_,_,_,_},_} | #c_cons{anno::[any(),...]} | #c_fname{anno::[any(),...]} | #iletrec{anno::{_,_,_,_},defs::[any(),...],body::[any(),...]} | #icase{anno::{_,_,_,_},args::[any()],clauses::[any()],fc::{_,_,_,_,_,_}} | #ireceive2{anno::{_,_,_,_},clauses::[any()],action::[any()]} | #ifun{anno::{_,_,_,_},id::[any(),...],vars::[any()],clauses::[any(),...],fc::{_,_,_,_,_,_}} | #imatch{anno::{_,_,_,_},guard::[],fc::{_,_,_,_,_,_}} | #itry{anno::{_,_,_,_},args::[any()],vars::[any(),...],body::[any(),...],evars::[any(),...],handler::[any(),...]} diff --git a/lib/dialyzer/test/r9c_SUITE_data/results/asn1 b/lib/dialyzer/test/r9c_SUITE_data/results/asn1 index c11105b76d..1cf03346ee 100644 --- a/lib/dialyzer/test/r9c_SUITE_data/results/asn1 +++ b/lib/dialyzer/test/r9c_SUITE_data/results/asn1 @@ -5,7 +5,7 @@ asn1ct.erl:1673: The pattern 'all' can never match the type 'asn1_module' | 'exc asn1ct.erl:672: The pattern <{'false', Result}, _, _> can never match the type <{'true','true'},atom() | binary() | [atom() | [any()] | char()],[any()]> asn1ct.erl:909: Guard test is_atom(Ext::[49 | 97 | 98 | 100 | 110 | 115]) can never succeed asn1ct_check.erl:1698: The pattern {'error', _} can never match the type [any()] -asn1ct_check.erl:2733: The pattern {'type', Tag, _, _, _, _} can never match the type 'ASN1_OPEN_TYPE' | {_,_} | {'fixedtypevaluefield',_,_} +asn1ct_check.erl:2733: Matching of pattern {'type', Tag, _, _, _, _} tagged with a record name violates the declared type of 'ASN1_OPEN_TYPE' | {_,_} | {'fixedtypevaluefield',_,_} asn1ct_check.erl:2738: The pattern <_S, _> can never match since previous clauses completely covered the type <#state{},#'ObjectClassFieldType'{class::#objectclass{fields::maybe_improper_list() | {_,_,_,_}},fieldname::{_,maybe_improper_list()},type::'ASN1_OPEN_TYPE' | {_,_} | {'fixedtypevaluefield',_,_}}> asn1ct_check.erl:2887: The variable Other can never match since previous clauses completely covered the type any() asn1ct_check.erl:3188: The pattern <_S, [], B> can never match the type <#state{},{'SingleValue',_},{'ValueRange',_}> diff --git a/lib/dialyzer/test/small_SUITE_data/results/confusing_record_warning b/lib/dialyzer/test/small_SUITE_data/results/confusing_record_warning deleted file mode 100644 index ac3d89b02b..0000000000 --- a/lib/dialyzer/test/small_SUITE_data/results/confusing_record_warning +++ /dev/null @@ -1,3 +0,0 @@ - -confusing_record_warning.erl:18: Function test/1 has no local return -confusing_record_warning.erl:18: Matching of pattern {'r', [_]} tagged with a record name violates the declared type of #r{field::'binary' | 'undefined'} diff --git a/lib/dialyzer/test/small_SUITE_data/results/funs_from_outside b/lib/dialyzer/test/small_SUITE_data/results/funs_from_outside new file mode 100644 index 0000000000..3e597ef1bc --- /dev/null +++ b/lib/dialyzer/test/small_SUITE_data/results/funs_from_outside @@ -0,0 +1,7 @@ + +funs_from_outside.erl:18: The pattern 'error' can never match the type {'ok','nothing' | 'something'} +funs_from_outside.erl:32: Function run2/2 has no local return +funs_from_outside.erl:35: Function testb/3 has no local return +funs_from_outside.erl:41: The pattern 'error' can never match the type {'ok','nothing' | 'something'} +funs_from_outside.erl:78: Function test2/1 has no local return +funs_from_outside.erl:83: The pattern 'error' can never match the type 'ok' diff --git a/lib/dialyzer/test/small_SUITE_data/results/invalid_spec_2 b/lib/dialyzer/test/small_SUITE_data/results/invalid_spec_2 new file mode 100644 index 0000000000..4565112ea0 --- /dev/null +++ b/lib/dialyzer/test/small_SUITE_data/results/invalid_spec_2 @@ -0,0 +1,2 @@ + +scala_user.erl:5: Invalid type specification for function scala_user:is_list/2. The success typing is (maybe_improper_list() | tuple(),_) -> boolean() diff --git a/lib/dialyzer/test/small_SUITE_data/results/literals b/lib/dialyzer/test/small_SUITE_data/results/literals new file mode 100644 index 0000000000..03e161ca71 --- /dev/null +++ b/lib/dialyzer/test/small_SUITE_data/results/literals @@ -0,0 +1,14 @@ + +literals.erl:11: Function t1/0 has no local return +literals.erl:12: Record construction #r{id::'a'} violates the declared type of field id::'integer' | 'undefined' +literals.erl:14: Function t2/0 has no local return +literals.erl:15: Record construction #r{id::'a'} violates the declared type of field id::'integer' | 'undefined' +literals.erl:17: Function t3/0 has no local return +literals.erl:18: Record construction #r{id::'a'} violates the declared type of field id::'integer' | 'undefined' +literals.erl:21: Record construction #r{id::'a'} violates the declared type of field id::'integer' | 'undefined' +literals.erl:23: Function m1/1 has no local return +literals.erl:23: Matching of pattern {'r', 'a'} tagged with a record name violates the declared type of #r{id::'integer' | 'undefined'} +literals.erl:26: Function m2/1 has no local return +literals.erl:26: Matching of pattern {'r', 'a'} tagged with a record name violates the declared type of #r{id::'integer' | 'undefined'} +literals.erl:29: Function m3/1 has no local return +literals.erl:29: The pattern {{'r', 'a'}} can never match the type any() diff --git a/lib/dialyzer/test/small_SUITE_data/results/maps_difftype b/lib/dialyzer/test/small_SUITE_data/results/maps_difftype new file mode 100644 index 0000000000..8980321135 --- /dev/null +++ b/lib/dialyzer/test/small_SUITE_data/results/maps_difftype @@ -0,0 +1,3 @@ + +maps_difftype.erl:10: Function empty_mismatch/1 has no local return +maps_difftype.erl:11: The pattern ~{}~ can never match the type tuple() diff --git a/lib/dialyzer/test/small_SUITE_data/results/my_sofs b/lib/dialyzer/test/small_SUITE_data/results/my_sofs index bc97c08d62..0b933e6cd7 100644 --- a/lib/dialyzer/test/small_SUITE_data/results/my_sofs +++ b/lib/dialyzer/test/small_SUITE_data/results/my_sofs @@ -1,3 +1,3 @@ -my_sofs.erl:34: The pattern {'Set', _, _} can never match the type #'OrdSet'{} -my_sofs.erl:54: The pattern {'Set', _, _} can never match the type #'OrdSet'{} +my_sofs.erl:34: Matching of pattern {'Set', _, _} tagged with a record name violates the declared type of #'OrdSet'{} +my_sofs.erl:54: Matching of pattern {'Set', _, _} tagged with a record name violates the declared type of #'OrdSet'{} diff --git a/lib/dialyzer/test/small_SUITE_data/results/pretty_bitstring b/lib/dialyzer/test/small_SUITE_data/results/pretty_bitstring new file mode 100644 index 0000000000..0ad6eee766 --- /dev/null +++ b/lib/dialyzer/test/small_SUITE_data/results/pretty_bitstring @@ -0,0 +1,3 @@ + +pretty_bitstring.erl:7: Function t/0 has no local return +pretty_bitstring.erl:8: The call binary:copy(#{#<1>(8, 1, 'integer', ['unsigned', 'big']), #<2>(8, 1, 'integer', ['unsigned', 'big']), #<3>(3, 1, 'integer', ['unsigned', 'big'])}#,2) breaks the contract (Subject,N) -> binary() when is_subtype(Subject,binary()), is_subtype(N,non_neg_integer()) diff --git a/lib/dialyzer/test/small_SUITE_data/results/record_pat b/lib/dialyzer/test/small_SUITE_data/results/record_pat index 9a3f925e42..a46be6c451 100644 --- a/lib/dialyzer/test/small_SUITE_data/results/record_pat +++ b/lib/dialyzer/test/small_SUITE_data/results/record_pat @@ -1,2 +1,2 @@ -record_pat.erl:14: The pattern {'foo', 'baz'} violates the declared type for #foo{} +record_pat.erl:14: Matching of pattern {'foo', 'baz'} tagged with a record name violates the declared type of #foo{bar::'undefined' | integer()} diff --git a/lib/dialyzer/test/small_SUITE_data/results/record_test b/lib/dialyzer/test/small_SUITE_data/results/record_test index 9715f0dcfb..7060bfa200 100644 --- a/lib/dialyzer/test/small_SUITE_data/results/record_test +++ b/lib/dialyzer/test/small_SUITE_data/results/record_test @@ -1,3 +1,3 @@ -record_test.erl:19: The pattern {'foo', _} can never match the type 'foo' +record_test.erl:19: Matching of pattern {'foo', _} tagged with a record name violates the declared type of 'foo' record_test.erl:21: The variable _ can never match since previous clauses completely covered the type 'foo' diff --git a/lib/dialyzer/test/small_SUITE_data/results/relevant_record_warning b/lib/dialyzer/test/small_SUITE_data/results/relevant_record_warning new file mode 100644 index 0000000000..2e417e1b2a --- /dev/null +++ b/lib/dialyzer/test/small_SUITE_data/results/relevant_record_warning @@ -0,0 +1,3 @@ + +relevant_record_warning.erl:22: Function test/1 has no local return +relevant_record_warning.erl:23: Record construction #r{field::<<_:8>>} violates the declared type of field field::'binary' | 'undefined' diff --git a/lib/dialyzer/test/small_SUITE_data/src/ddfs_master/common_types.hrl b/lib/dialyzer/test/small_SUITE_data/src/ddfs_master/common_types.hrl new file mode 100644 index 0000000000..f362a06bca --- /dev/null +++ b/lib/dialyzer/test/small_SUITE_data/src/ddfs_master/common_types.hrl @@ -0,0 +1,6 @@ +-type host() :: nonempty_string(). +-type path() :: nonempty_string(). +-type url() :: binary(). + +% The host portion of a url, if available. +-type url_host() :: host() | none. diff --git a/lib/dialyzer/test/small_SUITE_data/src/ddfs_master/config.hrl b/lib/dialyzer/test/small_SUITE_data/src/ddfs_master/config.hrl new file mode 100644 index 0000000000..8cab65fc9c --- /dev/null +++ b/lib/dialyzer/test/small_SUITE_data/src/ddfs_master/config.hrl @@ -0,0 +1,148 @@ + +-define(SECOND, 1000). +-define(MINUTE, (60 * ?SECOND)). +-define(HOUR, (60 * ?MINUTE)). +-define(DAY, (24 * ?HOUR)). +-define(MB, (1024 * 1024)). + +% Maximum length of tag/blob prefix +-define(NAME_MAX, 511). + +% How long ddfs node startup can take. The most time-consuming part +% is the scanning of the tag objects in the node's DDFS volumes. +-define(NODE_STARTUP, (1 * ?MINUTE)). + +% How long to wait on the master for replies from nodes. +-define(NODE_TIMEOUT, (10 * ?SECOND)). + +% How long to wait for a reply from an operation coordinated by the +% master that accesses nodes. This value should be larger than +% NODE_TIMEOUT. +-define(NODEOP_TIMEOUT, (1 * ?MINUTE)). + +% The minimum amount of free space a node must have, to be considered +% a primary candidate host for a new blob. +-define(MIN_FREE_SPACE, (1024 * ?MB)). + +% The maximum number of active HTTP connections on a system (this +% applies separately for GET and PUT operations). +-define(HTTP_MAX_ACTIVE, 3). + +% The maximum number of waiting HTTP connections to queue up on a busy system. +-define(HTTP_QUEUE_LENGTH, 100). + +% The maximum number of simultaneous HTTP connections. Note that +% HTTP_MAX_CONNS * 2 * 2 + 32 < Maximum number of file descriptors, where +% 2 = Get and put, 2 = two FDs required for each connection (connection +% itself + a file it accesses), 32 = a guess how many extra fds is needed. +-define(HTTP_MAX_CONNS, 128). + +% How long to keep a PUT request in queue if the system is busy. +-define(PUT_WAIT_TIMEOUT, (1 * ?MINUTE)). + +% How long to keep a GET request in queue if the system is busy. +-define(GET_WAIT_TIMEOUT, (1 * ?MINUTE)). + +% An unused loaded tag expires in TAG_EXPIRES milliseconds. Note that +% if TAG_EXPIRES is not smaller than GC_INTERVAL, tags will never +% expire from the memory cache and will always take up memory. +-define(TAG_EXPIRES, (10 * ?HOUR)). + +% How often the master's cache of all known tag names is refreshed. +% This refresh is only needed to purge deleted tags eventually from +% the tag cache. It doesn't harm to have a long interval. +-define(TAG_CACHE_INTERVAL, (10 * ?MINUTE)). + +% How soon a tag object initialized in memory expires if it's content +% cannot be fetched from the cluster. +-define(TAG_EXPIRES_ONERROR, (1 * ?SECOND)). + +% How often a DDFS node should refresh its tag cache from disk. +-define(FIND_TAGS_INTERVAL, ?DAY). + +% How often buffered (delayed) updates to a tag need to be +% flushed. Tradeoff: The longer the interval, the more updates are +% bundled in a single commit. On the other hand, in the worst case +% the requester has to wait for the full interval before getting a +% reply. A long interval also increases the likelihood that the server +% crashes before the commit has finished successfully, making requests +% more unreliable. +-define(DELAYED_FLUSH_INTERVAL, (1 * ?SECOND)). + +% How long to wait between garbage collection runs. +-define(GC_INTERVAL, ?DAY). + +% Max duration for a GC run. This should be smaller than +% min(ORPHANED_{BLOB,TAG}_EXPIRES). +-define(GC_MAX_DURATION, (3 * ?DAY)). + +% How long to wait after startup for cluster to stabilize before +% starting the first GC run. +-define(GC_DEFAULT_INITIAL_WAIT, (5 * ?MINUTE)). + +% The longest potential interval between messages in the GC protocol; +% used to ensure GC makes forward progress. This can be set to the +% estimated time to traverse all the volumes on a DDFS node. +-define(GC_PROGRESS_INTERVAL, (30 * ?MINUTE)). + +% Number of extra replicas (i.e. lost replicas recovered during GC) to +% allow before deleting extra replicas. +-define(NUM_EXTRA_REPLICAS, 1). + +% Permissions for files backing blobs and tags. +-define(FILE_MODE, 8#00400). + +% How often to check available disk space in ddfs_node. +-define(DISKSPACE_INTERVAL, (10 * ?SECOND)). + +% The maximum size of payloads of HTTP requests to the /ddfs/tag/ +% prefix. +-define(MAX_TAG_BODY_SIZE, (512 * ?MB)). + +% Tag attribute names and values have a limited size, and there +% can be only a limited number of them. +-define(MAX_TAG_ATTRIB_NAME_SIZE, 1024). +-define(MAX_TAG_ATTRIB_VALUE_SIZE, 1024). +-define(MAX_NUM_TAG_ATTRIBS, 1000). + +% How long HTTP requests that perform tag updates should wait to +% finish (a long time). +-define(TAG_UPDATE_TIMEOUT, ?DAY). + +% Timeout for re-replicating a single blob over HTTP PUT. This +% depends on the largest blobs hosted by DDFS, and the speed of the +% cluster network. +-define(GC_PUT_TIMEOUT, (180 * ?MINUTE)). + +% Delete !partial files after this many milliseconds. +-define(PARTIAL_EXPIRES, ?DAY). + +% When orphaned blob can be deleted. This should be large enough that +% you can upload all the new blobs of a tag and perform the tag update +% within this time. +-define(ORPHANED_BLOB_EXPIRES, (5 * ?DAY)). + +% When orphaned tag can be deleted. +-define(ORPHANED_TAG_EXPIRES, (5 * ?DAY)). + +% How long a tag has to stay on the deleted list before +% we can permanently forget it, after all known instances +% of the tag object have been removed. This quarantine period +% ensures that a node that was temporarily unavailable +% and reactivates can't resurrect deleted tags. You +% must ensure that all temporarily inactive nodes +% are reactivated (or cleaned) within the ?DELETED_TAG_EXPIRES +% time frame. +% +% This value _must_ be larger than the other time-related DDFS +% parameters listed in this file. In particular, it must be larger +% than ORPHANED_TAG_EXPIRES. +-define(DELETED_TAG_EXPIRES, (30 * ?DAY)). + +% How many times a tag operation should be retried before aborting. +-define(MAX_TAG_OP_RETRIES, 3). + +% How long to wait before timing out a tag retrieval. This should be +% large enough to read a large tag object off the disk and send it +% over the network. +-define(GET_TAG_TIMEOUT, (5 * ?MINUTE)). diff --git a/lib/dialyzer/test/small_SUITE_data/src/ddfs_master/ddfs.hrl b/lib/dialyzer/test/small_SUITE_data/src/ddfs_master/ddfs.hrl new file mode 100644 index 0000000000..e43ec23fe1 --- /dev/null +++ b/lib/dialyzer/test/small_SUITE_data/src/ddfs_master/ddfs.hrl @@ -0,0 +1,9 @@ +-type volume_name() :: nonempty_string(). + +% Diskinfo is {FreeSpace, UsedSpace}. +-type diskinfo() :: {non_neg_integer(), non_neg_integer()}. +-type volume() :: {diskinfo(), volume_name()}. + +-type object_type() :: 'blob' | 'tag'. +-type object_name() :: binary(). +-type taginfo() :: {erlang:timestamp(), volume_name()}. diff --git a/lib/dialyzer/test/small_SUITE_data/src/ddfs_master/ddfs_gc.hrl b/lib/dialyzer/test/small_SUITE_data/src/ddfs_master/ddfs_gc.hrl new file mode 100644 index 0000000000..dc43f7586b --- /dev/null +++ b/lib/dialyzer/test/small_SUITE_data/src/ddfs_master/ddfs_gc.hrl @@ -0,0 +1,17 @@ +-type local_object() :: {object_name(), node()}. +-type phase() :: 'start' | 'build_map' | 'map_wait' | 'gc' + | 'rr_blobs' | 'rr_blobs_wait' | 'rr_tags'. +-type protocol_msg() :: {'check_blob', object_name()} | 'start_gc' | 'end_rr'. + +-type blob_update() :: {object_name(), 'filter' | [url()]}. + +-type check_blob_result() :: 'false' | {'true', volume_name()}. + +% GC statistics + +% {Files, Bytes} +-type gc_stat() :: {non_neg_integer(), non_neg_integer()}. +% {Kept, Deleted} +-type obj_stats() :: {gc_stat(), gc_stat()}. +% {Tags, Blobs}. +-type gc_run_stats() :: {obj_stats(), obj_stats()}. diff --git a/lib/dialyzer/test/small_SUITE_data/src/ddfs_master/ddfs_master.erl b/lib/dialyzer/test/small_SUITE_data/src/ddfs_master/ddfs_master.erl new file mode 100644 index 0000000000..2be2773dc5 --- /dev/null +++ b/lib/dialyzer/test/small_SUITE_data/src/ddfs_master/ddfs_master.erl @@ -0,0 +1,531 @@ +-module(ddfs_master). +-behaviour(gen_server). + +-export([start_link/0]). +-export([get_tags/1, get_tags/3, + get_nodeinfo/1, + get_read_nodes/0, + get_hosted_tags/1, + gc_blacklist/0, gc_blacklist/1, + gc_stats/0, + choose_write_nodes/3, + new_blob/4, new_blob/5, + safe_gc_blacklist/0, safe_gc_blacklist/1, + refresh_tag_cache/0, + tag_notify/2, + tag_operation/2, tag_operation/3, + update_gc_stats/1, + update_nodes/1 + ]). +-export([init/1, + handle_call/3, + handle_cast/2, + handle_info/2, + terminate/2, + code_change/3]). + +-define(WEB_PORT, 8011). + +-compile(nowarn_deprecated_type). + +-include("common_types.hrl"). +-include("gs_util.hrl"). +-include("config.hrl"). +-include("ddfs.hrl"). +-include("ddfs_tag.hrl"). +-include("ddfs_gc.hrl"). + +-type node_info() :: {node(), {non_neg_integer(), non_neg_integer()}}. +-type gc_stats() :: none | gc_run_stats(). + +-record(state, {tags = gb_trees:empty() :: gb_trees:tree(), + tag_cache = false :: false | gb_sets:set(), + cache_refresher :: pid(), + + nodes = [] :: [node_info()], + write_blacklist = [] :: [node()], + read_blacklist = [] :: [node()], + gc_blacklist = [] :: [node()], + safe_gc_blacklist = gb_sets:empty() :: gb_sets:set(), + gc_stats = none :: none | {gc_stats(), erlang:timestamp()}}). +-type state() :: #state{}. +-type replyto() :: {pid(), reference()}. + +-export_type([gc_stats/0, node_info/0]). + +%% =================================================================== +%% API functions + +-spec start_link() -> {ok, pid()}. +start_link() -> + lager:info("DDFS master starts"), + case gen_server:start_link({local, ?MODULE}, ?MODULE, [], []) of + {ok, Server} -> {ok, Server}; + {error, {already_started, Server}} -> {ok, Server} + end. + +-spec tag_operation(term(), tagname()) -> term(). +tag_operation(Op, Tag) -> + gen_server:call(?MODULE, {tag, Op, Tag}). +-spec tag_operation(term(), tagname(), non_neg_integer() | infinity) -> + term(). +tag_operation(Op, Tag, Timeout) -> + gen_server:call(?MODULE, {tag, Op, Tag}, Timeout). + +-spec tag_notify(term(), tagname()) -> ok. +tag_notify(Op, Tag) -> + gen_server:cast(?MODULE, {tag_notify, Op, Tag}). + +-spec get_nodeinfo(all) -> {ok, [node_info()]}. +get_nodeinfo(all) -> + gen_server:call(?MODULE, {get_nodeinfo, all}). + +-spec get_read_nodes() -> {ok, [node()], non_neg_integer()} | {error, term()}. +get_read_nodes() -> + gen_server:call(?MODULE, get_read_nodes, infinity). + +-spec gc_blacklist() -> {ok, [node()]}. +gc_blacklist() -> + gen_server:call(?MODULE, gc_blacklist). + +-spec gc_blacklist([node()]) -> ok. +gc_blacklist(Nodes) -> + gen_server:cast(?MODULE, {gc_blacklist, Nodes}). + +-spec gc_stats() -> {ok, none | {gc_stats(), erlang:timestamp()}} | {error, term()}. +gc_stats() -> + gen_server:call(?MODULE, gc_stats). + +-spec get_hosted_tags(host()) -> {ok, [tagname()]} | {error, term()}. +get_hosted_tags(Host) -> + gen_server:call(?MODULE, {get_hosted_tags, Host}). + +-spec choose_write_nodes(non_neg_integer(), [node()], [node()]) -> {ok, [node()]}. +choose_write_nodes(K, Include, Exclude) -> + gen_server:call(?MODULE, {choose_write_nodes, K, Include, Exclude}). + +-spec get_tags(gc) -> {ok, [tagname()], [node()]} | too_many_failed_nodes; + (safe) -> {ok, [binary()]} | too_many_failed_nodes. +get_tags(Mode) -> + get_tags(?MODULE, Mode, ?GET_TAG_TIMEOUT). + +-spec get_tags(server(), gc, non_neg_integer()) -> + {ok, [tagname()], [node()]} | too_many_failed_nodes; + (server(), safe, non_neg_integer()) -> + {ok, [binary()]} | too_many_failed_nodes. +get_tags(Server, Mode, Timeout) -> + disco_profile:timed_run( + fun() -> gen_server:call(Server, {get_tags, Mode}, Timeout) end, + get_tags). + +-spec new_blob(string()|object_name(), non_neg_integer(), [node()], [node()]) -> + too_many_replicas | {ok, [nonempty_string()]}. +new_blob(Obj, K, Include, Exclude) -> + gen_server:call(?MODULE, {new_blob, Obj, K, Include, Exclude}, infinity). + +-spec new_blob(server(), string()|object_name(), non_neg_integer(), [node()], [node()]) -> + too_many_replicas | {ok, [nonempty_string()]}. +new_blob(Master, Obj, K, Include, Exclude) -> + gen_server:call(Master, {new_blob, Obj, K, Include, Exclude}, infinity). + +-spec safe_gc_blacklist() -> {ok, [node()]} | {error, term()}. +safe_gc_blacklist() -> + gen_server:call(?MODULE, safe_gc_blacklist). + +-spec safe_gc_blacklist(gb_sets:set()) -> ok. +safe_gc_blacklist(SafeGCBlacklist) -> + gen_server:cast(?MODULE, {safe_gc_blacklist, SafeGCBlacklist}). + +-spec update_gc_stats(gc_run_stats()) -> ok. +update_gc_stats(Stats) -> + gen_server:cast(?MODULE, {update_gc_stats, Stats}). + +-type nodes_update() :: [{node(), boolean(), boolean()}]. +-spec update_nodes(nodes_update()) -> ok. +update_nodes(DDFSNodes) -> + gen_server:cast(?MODULE, {update_nodes, DDFSNodes}). + +-spec update_nodestats(gb_trees:tree()) -> ok. +update_nodestats(NewNodes) -> + gen_server:cast(?MODULE, {update_nodestats, NewNodes}). + +-spec update_tag_cache(gb_sets:set()) -> ok. +update_tag_cache(TagCache) -> + gen_server:cast(?MODULE, {update_tag_cache, TagCache}). + +-spec refresh_tag_cache() -> ok. +refresh_tag_cache() -> + gen_server:cast(?MODULE, refresh_tag_cache). + +%% =================================================================== +%% gen_server callbacks + +-spec init(_) -> gs_init(). +init(_Args) -> + _ = [disco_profile:new_histogram(Name) + || Name <- [get_tags, do_get_tags_all, do_get_tags_filter, + do_get_tags_safe, do_get_tags_gc]], + spawn_link(fun() -> monitor_diskspace() end), + spawn_link(fun() -> ddfs_gc:start_gc(disco:get_setting("DDFS_DATA")) end), + Refresher = spawn_link(fun() -> refresh_tag_cache_proc() end), + put(put_port, disco:get_setting("DDFS_PUT_PORT")), + {ok, #state{cache_refresher = Refresher}}. + +-type choose_write_nodes_msg() :: {choose_write_nodes, non_neg_integer(), [node()], [node()]}. +-type new_blob_msg() :: {new_blob, string() | object_name(), non_neg_integer(), [node()]}. +-type tag_msg() :: {tag, ddfs_tag:call_msg(), tagname()}. +-spec handle_call(dbg_state_msg(), from(), state()) -> + gs_reply(state()); + ({get_nodeinfo, all}, from(), state()) -> + gs_reply({ok, [node_info()]}); + (get_read_nodes, from(), state()) -> + gs_reply({ok, [node()], non_neg_integer}); + (gc_blacklist, from(), state()) -> + gs_reply({ok, [node()]}); + (gc_stats, from(), state()) -> + gs_reply({ok, gc_stats(), erlang:timestamp()}); + (choose_write_nodes_msg(), from(), state()) -> + gs_reply({ok, [node()]}); + (new_blob_msg(), from(), state()) -> + gs_reply(new_blob_result()); + (tag_msg(), from(), state()) -> + gs_reply({error, nonodes}) | gs_noreply(); + ({get_tags, gc | safe}, from(), state()) -> + gs_noreply(); + ({get_hosted_tags, host()}, from(), state()) -> + gs_noreply(); + (safe_gc_blacklist, from(), state()) -> + gs_reply({ok, [node()]}). +handle_call(dbg_get_state, _, S) -> + {reply, S, S}; + +handle_call({get_nodeinfo, all}, _From, #state{nodes = Nodes} = S) -> + {reply, {ok, Nodes}, S}; + +handle_call(get_read_nodes, _F, #state{nodes = Nodes, read_blacklist = RB} = S) -> + {reply, do_get_readable_nodes(Nodes, RB), S}; + +handle_call(gc_blacklist, _F, #state{gc_blacklist = Nodes} = S) -> + {reply, {ok, Nodes}, S}; + +handle_call(gc_stats, _F, #state{gc_stats = Stats} = S) -> + {reply, {ok, Stats}, S}; + +handle_call({choose_write_nodes, K, Include, Exclude}, _, + #state{nodes = N, write_blacklist = WBL, gc_blacklist = GBL} = S) -> + BL = lists:umerge(WBL, GBL), + {reply, do_choose_write_nodes(N, K, Include, Exclude, BL), S}; + +handle_call({new_blob, Obj, K, Include, Exclude}, _, + #state{nodes = N, gc_blacklist = GBL, write_blacklist = WBL} = S) -> + BL = lists:umerge(WBL, GBL), + {reply, do_new_blob(Obj, K, Include, Exclude, BL, N), S}; + +handle_call({tag, _M, _Tag}, _From, #state{nodes = []} = S) -> + {reply, {error, no_nodes}, S}; + +handle_call({tag, M, Tag}, From, S) -> + {noreply, do_tag_request(M, Tag, From, S)}; + +handle_call({get_tags, Mode}, From, #state{nodes = Nodes} = S) -> + spawn(fun() -> + gen_server:reply(From, do_get_tags(Mode, [N || {N, _} <- Nodes])) + end), + {noreply, S}; + +handle_call({get_hosted_tags, Host}, From, S) -> + spawn(fun() -> gen_server:reply(From, ddfs_gc:hosted_tags(Host)) end), + {noreply, S}; + +handle_call(safe_gc_blacklist, _From, #state{safe_gc_blacklist = SBL} = S) -> + {reply, {ok, gb_sets:to_list(SBL)}, S}. + +-spec handle_cast({tag_notify, ddfs_tag:cast_msg(), tagname()} + | {gc_blacklist, [node()]} + | {safe_gc_blacklist, gb_sets:set()} + | {update_gc_stats, gc_stats()} + | {update_tag_cache, gb_sets:set()} + | refresh_tag_cache + | {update_nodes, nodes_update()} + | {update_nodestats, gb_trees:tree()}, + state()) -> gs_noreply(). +handle_cast({tag_notify, M, Tag}, S) -> + {noreply, do_tag_notify(M, Tag, S)}; + +handle_cast({gc_blacklist, Nodes}, #state{safe_gc_blacklist = SBL} = S) -> + BLSet = gb_sets:from_list(Nodes), + NewSBL = gb_sets:intersection(BLSet, SBL), + {noreply, S#state{gc_blacklist = gb_sets:to_list(BLSet), + safe_gc_blacklist = NewSBL}}; + +handle_cast({safe_gc_blacklist, SafeBlacklist}, #state{gc_blacklist = BL} = S) -> + SBL = gb_sets:intersection(SafeBlacklist, gb_sets:from_list(BL)), + {noreply, S#state{safe_gc_blacklist = SBL}}; + +handle_cast({update_gc_stats, Stats}, S) -> + {noreply, S#state{gc_stats = {Stats, now()}}}; + +handle_cast({update_tag_cache, TagCache}, S) -> + {noreply, S#state{tag_cache = TagCache}}; + +handle_cast(refresh_tag_cache, #state{cache_refresher = Refresher} = S) -> + Refresher ! refresh, + {noreply, S}; + +handle_cast({update_nodes, NewNodes}, S) -> + {noreply, do_update_nodes(NewNodes, S)}; + +handle_cast({update_nodestats, NewNodes}, S) -> + {noreply, do_update_nodestats(NewNodes, S)}. + +-spec handle_info({'DOWN', _, _, pid(), _}, state()) -> gs_noreply(). +handle_info({'DOWN', _, _, Pid, _}, S) -> + {noreply, do_tag_exit(Pid, S)}. + +%% =================================================================== +%% gen_server callback stubs + +-spec terminate(term(), state()) -> ok. +terminate(Reason, _State) -> + lager:warning("DDFS master died: ~p", [Reason]). + +-spec code_change(term(), state(), term()) -> {ok, state()}. +code_change(_OldVsn, State, _Extra) -> {ok, State}. + +%% =================================================================== +%% internal functions + +-spec do_get_readable_nodes([node_info()], [node()]) -> + {ok, [node()], non_neg_integer()}. +do_get_readable_nodes(Nodes, ReadBlacklist) -> + NodeSet = gb_sets:from_ordset(lists:sort([Node || {Node, _} <- Nodes])), + BlackSet = gb_sets:from_ordset(ReadBlacklist), + ReadableNodeSet = gb_sets:subtract(NodeSet, BlackSet), + {ok, gb_sets:to_list(ReadableNodeSet), gb_sets:size(BlackSet)}. + +-spec do_choose_write_nodes([node_info()], non_neg_integer(), [node()], [node()], [node()]) -> + {ok, [node()]}. +do_choose_write_nodes(Nodes, K, Include, Exclude, BlackList) -> + % Include is the list of nodes that must be included + % + % Node selection algorithm: + % 1. try to choose K nodes randomly from all the nodes which have + % more than ?MIN_FREE_SPACE bytes free space available and which + % are not excluded or blacklisted. + % 2. if K nodes cannot be found this way, choose the K emptiest + % nodes which are not excluded or blacklisted. + Primary = ([N || {N, {Free, _Total}} <- Nodes, Free > ?MIN_FREE_SPACE / 1024] + -- (Exclude ++ BlackList)), + if length(Primary) >= K -> + {ok, Include ++ disco_util:choose_random(Primary -- Include , K - length(Include))}; + true -> + Preferred = [N || {N, _} <- lists:reverse(lists:keysort(2, Nodes))], + Secondary = Include ++ lists:sublist(Preferred -- (Include ++ Exclude ++ BlackList), + K - length(Include)), + {ok, Secondary} + end. + +-type new_blob_result() :: too_many_replicas | {ok, [nonempty_string()]}. +-spec do_new_blob(string()|object_name(), non_neg_integer(), [node()], [node()], [node()], [node_info()]) -> + new_blob_result(). +do_new_blob(_Obj, K, _Include, _Exclude, _BlackList, Nodes) when K > length(Nodes) -> + too_many_replicas; +do_new_blob(Obj, K, Include, Exclude, BlackList, Nodes) -> + {ok, WriteNodes} = do_choose_write_nodes(Nodes, K, Include, Exclude, BlackList), + Urls = [["http://", disco:host(N), ":", get(put_port), "/ddfs/", Obj] + || N <- WriteNodes], + {ok, Urls}. + +% Tag request: Start a new tag server if one doesn't exist already. Forward +% the request to the tag server. + +-spec get_tag_pid(tagname(), gb_trees:tree(), false | gb_sets:set()) -> + {pid(), gb_trees:tree()}. +get_tag_pid(Tag, Tags, Cache) -> + case gb_trees:lookup(Tag, Tags) of + none -> + NotFound = (Cache =/= false + andalso not gb_sets:is_element(Tag, Cache)), + {ok, Server} = ddfs_tag:start(Tag, NotFound), + erlang:monitor(process, Server), + {Server, gb_trees:insert(Tag, Server, Tags)}; + {value, P} -> + {P, Tags} + end. + +-spec do_tag_request(term(), tagname(), replyto(), state()) -> + state(). +do_tag_request(M, Tag, From, #state{tags = Tags, tag_cache = Cache} = S) -> + {Pid, TagsN} = get_tag_pid(Tag, Tags, Cache), + gen_server:cast(Pid, {M, From}), + S#state{tags = TagsN, + tag_cache = Cache =/= false andalso gb_sets:add(Tag, Cache)}. + +-spec do_tag_notify(term(), tagname(), state()) -> state(). +do_tag_notify(M, Tag, #state{tags = Tags, tag_cache = Cache} = S) -> + {Pid, TagsN} = get_tag_pid(Tag, Tags, Cache), + gen_server:cast(Pid, {notify, M}), + S#state{tags = TagsN, + tag_cache = Cache =/= false andalso gb_sets:add(Tag, Cache)}. + +-spec do_update_nodes(nodes_update(), state()) -> state(). +do_update_nodes(NewNodes, #state{nodes = Nodes, tags = Tags} = S) -> + WriteBlacklist = lists:sort([Node || {Node, false, _} <- NewNodes]), + ReadBlacklist = lists:sort([Node || {Node, _, false} <- NewNodes]), + OldNodes = gb_trees:from_orddict(Nodes), + UpdatedNodes = lists:keysort(1, [case gb_trees:lookup(Node, OldNodes) of + none -> + {Node, {0, 0}}; + {value, OldStats} -> + {Node, OldStats} + end || {Node, _WB, _RB} <- NewNodes]), + if + UpdatedNodes =/= Nodes -> + _ = [gen_server:cast(Pid, {die, none}) || Pid <- gb_trees:values(Tags)], + spawn(fun() -> + {ok, ReadableNodes, RBSize} = + do_get_readable_nodes(UpdatedNodes, ReadBlacklist), + refresh_tag_cache(ReadableNodes, RBSize) + end), + S#state{nodes = UpdatedNodes, + write_blacklist = WriteBlacklist, + read_blacklist = ReadBlacklist, + tag_cache = false, + tags = gb_trees:empty()}; + true -> + S#state{write_blacklist = WriteBlacklist, + read_blacklist = ReadBlacklist} + end. + +-spec do_update_nodestats(gb_trees:tree(), state()) -> state(). +do_update_nodestats(NewNodes, #state{nodes = Nodes} = S) -> + UpdatedNodes = [case gb_trees:lookup(Node, NewNodes) of + none -> + {Node, Stats}; + {value, NewStats} -> + {Node, NewStats} + end || {Node, Stats} <- Nodes], + S#state{nodes = UpdatedNodes}. + +-spec do_tag_exit(pid(), state()) -> state(). +do_tag_exit(Pid, S) -> + NewTags = [X || {_, V} = X <- gb_trees:to_list(S#state.tags), V =/= Pid], + S#state{tags = gb_trees:from_orddict(NewTags)}. + +-spec do_get_tags(all | filter, [node()]) -> {[node()], [node()], [binary()]}; + (safe, [node()]) -> {ok, [binary()]} | too_many_failed_nodes; + (gc, [node()]) -> {ok, [binary()], [node()]} | too_many_failed_nodes. +do_get_tags(all, Nodes) -> + disco_profile:timed_run( + fun() -> + {Replies, Failed} = + gen_server:multi_call(Nodes, ddfs_node, get_tags, ?NODE_TIMEOUT), + {OkNodes, Tags} = lists:unzip(Replies), + {OkNodes, Failed, lists:usort(lists:flatten(Tags))} + end, do_get_tags_all); + +do_get_tags(filter, Nodes) -> + disco_profile:timed_run( + fun() -> + {OkNodes, Failed, Tags} = do_get_tags(all, Nodes), + case tag_operation(get_tagnames, <<"+deleted">>, ?NODEOP_TIMEOUT) of + {ok, Deleted} -> + TagSet = gb_sets:from_ordset(Tags), + DelSet = gb_sets:insert(<<"+deleted">>, Deleted), + NotDeleted = gb_sets:to_list(gb_sets:subtract(TagSet, DelSet)), + {OkNodes, Failed, NotDeleted}; + E -> + E + end + end, do_get_tags_filter); + +do_get_tags(safe, Nodes) -> + disco_profile:timed_run( + fun() -> + TagMinK = list_to_integer(disco:get_setting("DDFS_TAG_MIN_REPLICAS")), + case do_get_tags(filter, Nodes) of + {_OkNodes, Failed, Tags} when length(Failed) < TagMinK -> + {ok, Tags}; + _ -> + too_many_failed_nodes + end + end, do_get_tags_safe); + +% The returned tag list may include +deleted. +do_get_tags(gc, Nodes) -> + disco_profile:timed_run( + fun() -> + {OkNodes, Failed, Tags} = do_get_tags(all, Nodes), + TagMinK = list_to_integer(disco:get_setting("DDFS_TAG_MIN_REPLICAS")), + case length(Failed) < TagMinK of + false -> + too_many_failed_nodes; + true -> + case tag_operation(get_tagnames, <<"+deleted">>, ?NODEOP_TIMEOUT) of + {ok, Deleted} -> + TagSet = gb_sets:from_ordset(Tags), + NotDeleted = gb_sets:subtract(TagSet, Deleted), + {ok, gb_sets:to_list(NotDeleted), OkNodes}; + E -> + E + end + end + end, do_get_tags_gc). + +% Timeouts in this call by the below processes can cause ddfs_master +% itself to crash, since the processes are linked to it. +-spec safe_get_read_nodes() -> {ok, [node()], non_neg_integer()} | error. +safe_get_read_nodes() -> + try get_read_nodes() of + {ok, _ReadableNodes, _RBSize} = RN -> + RN; + E -> + lager:error("unexpected response retrieving readable nodes: ~p", [E]), + error + catch + K:E -> + lager:error("error retrieving readable nodes: ~p:~p", [K, E]), + error + end. + +-spec monitor_diskspace() -> no_return(). +monitor_diskspace() -> + case safe_get_read_nodes() of + {ok, ReadableNodes, _RBSize} -> + {Space, _F} = gen_server:multi_call(ReadableNodes, + ddfs_node, + get_diskspace, + ?NODE_TIMEOUT), + update_nodestats(gb_trees:from_orddict(lists:keysort(1, Space))); + error -> + ok + end, + timer:sleep(?DISKSPACE_INTERVAL), + monitor_diskspace(). + +-spec refresh_tag_cache_proc() -> no_return(). +refresh_tag_cache_proc() -> + case safe_get_read_nodes() of + {ok, ReadableNodes, RBSize} -> + refresh_tag_cache(ReadableNodes, RBSize); + error -> + ok + end, + receive + refresh -> + ok + after ?TAG_CACHE_INTERVAL -> + ok + end, + refresh_tag_cache_proc(). + +-spec refresh_tag_cache([node()], non_neg_integer()) -> ok. +refresh_tag_cache(Nodes, BLSize) -> + TagMinK = list_to_integer(disco:get_setting("DDFS_TAG_MIN_REPLICAS")), + {Replies, Failed} = + gen_server:multi_call(Nodes, ddfs_node, get_tags, ?NODE_TIMEOUT), + if Nodes =/= [], length(Failed) + BLSize < TagMinK -> + {_OkNodes, Tags} = lists:unzip(Replies), + update_tag_cache(gb_sets:from_list(lists:flatten(Tags))); + true -> ok + end. diff --git a/lib/dialyzer/test/small_SUITE_data/src/ddfs_master/ddfs_tag.hrl b/lib/dialyzer/test/small_SUITE_data/src/ddfs_master/ddfs_tag.hrl new file mode 100644 index 0000000000..2920b67fc5 --- /dev/null +++ b/lib/dialyzer/test/small_SUITE_data/src/ddfs_master/ddfs_tag.hrl @@ -0,0 +1,19 @@ + +-type tokentype() :: 'read' | 'write'. +-type user_attr() :: [{binary(), binary()}]. +% An 'internal' token is also used by internal consumers, but never stored. +-type token() :: 'null' | binary(). + +-type tagname() :: binary(). +-type tagid() :: binary(). + +-type attrib() :: 'urls' | 'read_token' | 'write_token' | {'user', binary()}. + +-record(tagcontent, {id :: tagid(), + last_modified :: binary(), + read_token = null :: token(), + write_token = null :: token(), + urls = [] :: [[binary()]], + user = [] :: user_attr()}). + +-type tagcontent() :: #tagcontent{}. diff --git a/lib/dialyzer/test/small_SUITE_data/src/ddfs_master/gs_util.hrl b/lib/dialyzer/test/small_SUITE_data/src/ddfs_master/gs_util.hrl new file mode 100644 index 0000000000..d579e9a7d7 --- /dev/null +++ b/lib/dialyzer/test/small_SUITE_data/src/ddfs_master/gs_util.hrl @@ -0,0 +1,16 @@ +% This is a set of type utilities to be used when spec-cing the +% callbacks of a gen_server implementation. It should be included in +% the impl module, which needs to define the state() type. + +-type gs_init() :: {ok, state()}. +-type gs_reply(T) :: {reply, (T), state()}. +-type gs_noreply() :: {noreply, state()}. +-type gs_noreply_t() :: {noreply, state(), non_neg_integer()}. +-type gs_stop(T) :: {stop, (T), state()}. + +% Generic utilities. + +-type server() :: pid() | atom() | {atom(), node()}. +-type from() :: {pid(), term()}. + +-type dbg_state_msg() :: dbg_get_state. diff --git a/lib/dialyzer/test/small_SUITE_data/src/fun2ms.erl b/lib/dialyzer/test/small_SUITE_data/src/fun2ms.erl new file mode 100644 index 0000000000..9e7df85e4c --- /dev/null +++ b/lib/dialyzer/test/small_SUITE_data/src/fun2ms.erl @@ -0,0 +1,21 @@ +-module(fun2ms). +-export([return/0]). +-include_lib("stdlib/include/ms_transform.hrl"). + +-record(snapshot, {id :: integer(), arg1 :: atom(), arg2 :: tuple()}). + +return() -> + TableId = ets:new(table, [public, {keypos, #snapshot.id}]), + + ets:insert(TableId, [#snapshot{id = 1, arg1 = hard, arg2 = {1,2}}, + #snapshot{id = 2, arg1 = rock, arg2 = {1,2}}, + #snapshot{id = 3, arg1 = hallelujah, arg2 = + {1,2}}]), + + + Example = ets:fun2ms( + fun(#snapshot{id = Arg1, arg1 = Arg2}) -> + {Arg1, Arg2} + end), + + ets:select(TableId, Example). diff --git a/lib/dialyzer/test/small_SUITE_data/src/funs_from_outside.erl b/lib/dialyzer/test/small_SUITE_data/src/funs_from_outside.erl new file mode 100644 index 0000000000..f4cbf31160 --- /dev/null +++ b/lib/dialyzer/test/small_SUITE_data/src/funs_from_outside.erl @@ -0,0 +1,83 @@ +-module(funs_from_outside).
+
+-export([run1/2, run2/2, run3/2]).
+-export([test1/1, test2/1]).
+
+%%------------------------------------------------------------------------------
+
+run1(X, Y) ->
+ testa(fun do_something/1, X, Y).
+
+testa(Fun, X, Y) ->
+ F = case even(X) of
+ true -> Fun;
+ false -> fun do_nothing/1
+ end,
+ case F(Y) of
+ {ok, _} -> ok;
+ error -> error
+ end.
+
+do_nothing(_) -> {ok, nothing}.
+
+do_something(_) -> {ok, something}.
+
+even(X) ->
+ X rem 2 =:= 0.
+
+%%------------------------------------------------------------------------------
+
+%% Duplicating code since we are monovariant...
+
+run2(X, Y) ->
+ testb(fun do_something/1, X, Y).
+
+testb(Fun, X, Y) ->
+ F = case even(X) of
+ true -> Fun;
+ false -> fun do_nothing/1
+ end,
+ case F(Y) of
+ error -> error
+ end.
+
+%%------------------------------------------------------------------------------
+
+%% Duplicating code since we are monovariant...
+
+run3(X, Y) ->
+ testc(fun do_something_2/1, X, Y).
+
+testc(Fun, X, Y) ->
+ F = case even(X) of
+ true -> Fun;
+ false -> fun do_nothing/1
+ end,
+ case F(Y) of
+ {ok, _} -> ok;
+ %% This pattern can match.
+ error -> error
+ end.
+
+do_something_2(foo) -> {ok, something};
+do_something_2(_) -> error.
+
+%%------------------------------------------------------------------------------
+
+test1(Fun) ->
+ F = case get(test1) of
+ test1_t -> Fun;
+ test1_f -> fun fok/0
+ end,
+ error = F().
+
+fok() -> ok.
+
+%%------------------------------------------------------------------------------
+
+test2(Fun) ->
+ F = case get(test1) of
+ test1_t -> fun fok/0;
+ test1_f -> fun fok/0
+ end,
+ error = F().
diff --git a/lib/dialyzer/test/small_SUITE_data/src/invalid_spec_2/scala_data.erl b/lib/dialyzer/test/small_SUITE_data/src/invalid_spec_2/scala_data.erl new file mode 100644 index 0000000000..c26787fe24 --- /dev/null +++ b/lib/dialyzer/test/small_SUITE_data/src/invalid_spec_2/scala_data.erl @@ -0,0 +1,5 @@ +-module(scala_data). + +-export_type([data/0]). + +-opaque data() :: {'data', term()}. diff --git a/lib/dialyzer/test/small_SUITE_data/src/invalid_spec_2/scala_user.erl b/lib/dialyzer/test/small_SUITE_data/src/invalid_spec_2/scala_user.erl new file mode 100644 index 0000000000..4e981f3b74 --- /dev/null +++ b/lib/dialyzer/test/small_SUITE_data/src/invalid_spec_2/scala_user.erl @@ -0,0 +1,8 @@ +-module(scala_user). + +-export([is_list/2]). + +-spec is_list(atom(), scala_data:data()) -> boolean(). + +is_list( List,Data) when is_list(List) -> true; +is_list(Tuple,Data) when is_tuple(Tuple) -> false. diff --git a/lib/dialyzer/test/small_SUITE_data/src/limit.erl b/lib/dialyzer/test/small_SUITE_data/src/limit.erl new file mode 100644 index 0000000000..97ee585b77 --- /dev/null +++ b/lib/dialyzer/test/small_SUITE_data/src/limit.erl @@ -0,0 +1,20 @@ +%% Misc cases where Dialyzer would fail with system_limit or crash + +-module(limit). + +-export([tu/0, big/1, b2/0]). + +tu() -> + erlang:make_tuple(1 bsl 24, def, [{5,e},{1,a},{3,c}]). + +big(<<Int:1152921504606846976/unit:128,0,_/binary>>) -> {5,Int}. + +b2() -> + Maxbig = maxbig(), + _ = bnot Maxbig, + ok. + +maxbig() -> + %% We assume that the maximum arity is (1 bsl 19) - 1. + Ws = erlang:system_info(wordsize), + (((1 bsl ((16777184 * (Ws div 4))-1)) - 1) bsl 1) + 1. diff --git a/lib/dialyzer/test/small_SUITE_data/src/literals.erl b/lib/dialyzer/test/small_SUITE_data/src/literals.erl new file mode 100644 index 0000000000..abd7033712 --- /dev/null +++ b/lib/dialyzer/test/small_SUITE_data/src/literals.erl @@ -0,0 +1,33 @@ +-module(literals). + +%% Bad records inside structures used to be ignored. The reason: +%% v3_core:unfold() does not annotate the parts of a literal. +%% This example does not work perfectly yet, in particular Maps. + +-export([t1/0, t2/0, t3/0, t4/0, m1/1, m2/1, m3/1, m4/1]). + +-record(r, {id :: integer}). + +t1() -> + #r{id = a}. % violation + +t2() -> + [#r{id = a}]. % violation + +t3() -> + {#r{id = a}}. % violation + +t4() -> + #{a => #r{id = a}}. % violation found, but t4() returns... (bug) + +m1(#r{id = a}) -> % violation + ok. + +m2([#r{id = a}]) -> % violation + ok. + +m3({#r{id = a}}) -> % can never match; not so good + ok. + +m4(#{a := #r{id = a}}) -> % violation not found + ok. diff --git a/lib/dialyzer/test/small_SUITE_data/src/maps_difftype.erl b/lib/dialyzer/test/small_SUITE_data/src/maps_difftype.erl new file mode 100644 index 0000000000..19e61a7944 --- /dev/null +++ b/lib/dialyzer/test/small_SUITE_data/src/maps_difftype.erl @@ -0,0 +1,11 @@ +%% +%% File: maps_difftype.erl +%% Author: Björn-Egil Dahlberg +%% Created: 2014-04-29 +%% +-module(maps_difftype). + +-export([empty_mismatch/1]). + +empty_mismatch(Tuple) when is_tuple(Tuple) -> + case Tuple of #{} -> ok end. diff --git a/lib/dialyzer/test/small_SUITE_data/src/pretty_bitstring.erl b/lib/dialyzer/test/small_SUITE_data/src/pretty_bitstring.erl new file mode 100644 index 0000000000..3dbf5ab7a7 --- /dev/null +++ b/lib/dialyzer/test/small_SUITE_data/src/pretty_bitstring.erl @@ -0,0 +1,8 @@ +%% Prettyprint bitstrings. + +-module(pretty_bitstring). + +-export([t/0]). + +t() -> + binary:copy(<<1,2,3:3>>,2). diff --git a/lib/dialyzer/test/small_SUITE_data/src/confusing_record_warning.erl b/lib/dialyzer/test/small_SUITE_data/src/relevant_record_warning.erl index 8af74e0914..3ff65458df 100644 --- a/lib/dialyzer/test/small_SUITE_data/src/confusing_record_warning.erl +++ b/lib/dialyzer/test/small_SUITE_data/src/relevant_record_warning.erl @@ -1,3 +1,7 @@ +%% Formerly confusing_record_warning.erl. +%% The warning output is relevant as of Erlang/OTP 17.1. +%% The original comment kept below. + %%--------------------------------------------------------------------- %% A user complained that dialyzer produces a weird warning for the %% following program. I explained to him that there is an implicit @@ -9,7 +13,7 @@ %% The pattern {'r', [_]} can never match the type any() %% We should clearly give some less confusing warning in this case. %%--------------------------------------------------------------------- --module(confusing_record_warning). +-module(relevant_record_warning). -export([test/1]). diff --git a/lib/dialyzer/test/small_SUITE_data/src/remote_field.erl b/lib/dialyzer/test/small_SUITE_data/src/remote_field.erl new file mode 100644 index 0000000000..c34fa1b9dd --- /dev/null +++ b/lib/dialyzer/test/small_SUITE_data/src/remote_field.erl @@ -0,0 +1,11 @@ +-module(remote_field). + +-type f(T) :: {ssl:sslsocket(), T}. + +-record(r1, { f1 :: f(_) }). +-type r1(T) :: #r1{ f1 :: fun((ssl:sslsocket(), T) -> any()) }. + +-record(state, { + r :: r1(T), + arg :: T + }). diff --git a/lib/dialyzer/test/small_SUITE_data/src/remote_field2.erl b/lib/dialyzer/test/small_SUITE_data/src/remote_field2.erl new file mode 100644 index 0000000000..35687e22ec --- /dev/null +++ b/lib/dialyzer/test/small_SUITE_data/src/remote_field2.erl @@ -0,0 +1,17 @@ +-module(remote_field2). + +-export([handle_cast/2]). + +-record(state, {tcp_socket :: inet:socket()}). + +-spec handle_cast(_,_) -> + {noreply,_} | + {stop,{shutdown,connection_closed}, + #state{tcp_socket :: port()}}. +handle_cast({send, Message}, #state{tcp_socket = TCPSocket} = State) -> + case gen_tcp:send(TCPSocket, Message) of + ok -> + {noreply, State}; + {error, closed} -> + {stop, {shutdown, connection_closed}, State} + end. diff --git a/lib/dialyzer/test/underspecs_SUITE_data/results/arr b/lib/dialyzer/test/underspecs_SUITE_data/results/arr new file mode 100644 index 0000000000..9497d12eec --- /dev/null +++ b/lib/dialyzer/test/underspecs_SUITE_data/results/arr @@ -0,0 +1,4 @@ + +arr.erl:14: Type specification arr:test2(array:array(T),non_neg_integer(),T) -> array:array(T) is a supertype of the success typing: arr:test2(array:array(_),pos_integer(),_) -> array:array(_) +arr.erl:24: Type specification arr:test4(array:array(T),non_neg_integer(),_) -> array:array(T) is a supertype of the success typing: arr:test4(array:array(_),pos_integer(),_) -> array:array(_) +arr.erl:29: Type specification arr:test5(array:array(T),non_neg_integer(),T) -> array:array(T) is a supertype of the success typing: arr:test5(array:array(_),non_neg_integer(),integer()) -> array:array(_) diff --git a/lib/dialyzer/test/underspecs_SUITE_data/src/arr.erl b/lib/dialyzer/test/underspecs_SUITE_data/src/arr.erl new file mode 100644 index 0000000000..3b265ccec2 --- /dev/null +++ b/lib/dialyzer/test/underspecs_SUITE_data/src/arr.erl @@ -0,0 +1,41 @@ +-module(arr). + +%% http://erlang.org/pipermail/erlang-questions/2014-August/080445.html + +-define(A, array). + +-export([test/3, test2/3, test3/3, test4/3, test5/3, test6/3]). + +-spec test(?A:array(T), non_neg_integer(), T) -> ?A:array(T). + +test(Array, N, Value) -> + ?A:set(N, Value, Array). + +-spec test2(?A:array(T), non_neg_integer(), T) -> ?A:array(T). + +test2(Array, N, Value) when N > 0 -> + ?A:set(N, Value, Array). + +-spec test3(?A:array(T), non_neg_integer(), _) -> ?A:array(T). + +test3(Array, N, Value) -> + ?A:set(N, Value, Array). + +-spec test4(?A:array(T), non_neg_integer(), _) -> ?A:array(T). + +test4(Array, N, Value) when N > 0 -> + ?A:set(N, Value, Array). + +-spec test5(?A:array(T), non_neg_integer(), T) -> ?A:array(T). + +test5(Array, N, Value) when is_integer(Value) -> + ?A:set(N, Value, Array). + +%% One would ideally want a warning also for test6(), but the current +%% analysis of parametrized opaque types is not strong enough to +%% discover this. +-spec test6(?A:array(integer()), non_neg_integer(), integer()) -> + ?A:array(any()). + +test6(Array, N, Value) -> + ?A:set(N, Value, Array). diff --git a/lib/dialyzer/vsn.mk b/lib/dialyzer/vsn.mk index 95d2464e1d..58cc77c2fa 100644 --- a/lib/dialyzer/vsn.mk +++ b/lib/dialyzer/vsn.mk @@ -1 +1 @@ -DIALYZER_VSN = 2.7 +DIALYZER_VSN = 2.7.2 diff --git a/lib/diameter/doc/src/diameter.xml b/lib/diameter/doc/src/diameter.xml index 7d6a28e51c..ab9ad25a3a 100644 --- a/lib/diameter/doc/src/diameter.xml +++ b/lib/diameter/doc/src/diameter.xml @@ -227,7 +227,7 @@ question is as if a callback had taken place and returned <c>{error, failure}</c>.</p> <p> -Defaults to <c>report</c> if unspecified.</p> +Defaults to <c>discard</c> if unspecified.</p> </item> <tag><c>{request_errors, answer_3xxx|answer|callback}</c></tag> diff --git a/lib/diameter/doc/src/diameter_make.xml b/lib/diameter/doc/src/diameter_make.xml index 13ec5bbfc1..0c7e6b794d 100644 --- a/lib/diameter/doc/src/diameter_make.xml +++ b/lib/diameter/doc/src/diameter_make.xml @@ -52,7 +52,7 @@ under the License. <p> The function &codec; is used to compile a diameter &dictionary; into Erlang source. -The resulting source implements the interface diameter required +The resulting source implements the interface diameter requires to encode and decode the dictionary's messages and AVPs.</p> <p> diff --git a/lib/diameter/doc/src/notes.xml b/lib/diameter/doc/src/notes.xml index cd14195e78..7f69bdbfbf 100644 --- a/lib/diameter/doc/src/notes.xml +++ b/lib/diameter/doc/src/notes.xml @@ -11,7 +11,7 @@ <header> <copyright> <year>2011</year> -<year>2013</year> +<year>2014</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -42,6 +42,352 @@ first.</p> <!-- ===================================================================== --> +<section><title>diameter 1.7.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Don't leave extra bit in decoded AVP data.</p> + <p> + An extra bit could be communicated in the data field of a + diameter_avp record in the case of length errors. Of no + consequence for code using the record encoding of + Diameter messages, but code examining diameter_avp + records would see this bit.</p> + <p> + Dictionary files must be recompiled for the fix to have + effect.</p> + <p> + Own Id: OTP-12074</p> + </item> + <item> + <p> + Fix counting of outgoing requests and answers setting the + E-bit.</p> + <p> + OTP-11721 broke these counters for all outgoing requests + except DWR, and caused answers setting the E-bit to be + counted as unknown messages.</p> + <p> + Own Id: OTP-12080</p> + </item> + <item> + <p> + Fix Failed-AVP decode.</p> + <p> + The best-effort decode only worked for AVPs in the common + dictionary, not for those in the dictionary of the + application identified in the Diameter Header of the + answer message in question.</p> + <p> + Failed-AVP in an answer decoded with the RFC 3588 common + dictionary (diameter_gen_base_rfc3588) was regarded as an + error. The RFC 6733 dictionary was unaffected.</p> + <p> + Dictionary files must be recompiled for the fix to have + effect.</p> + <p> + Own Id: OTP-12094</p> + </item> + </list> + </section> + +</section> + +<section><title>diameter 1.7</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Improve robustness.</p> + <p> + Counters returned by diameter:service_info/2 now only + count messages known to the dictionary in question, so + that an attacker cannot cause arbitrarily many counters + to be created.</p> + <p> + Messages to the Erlang log have been minimized, and those + related to traffic have been removed entirely since an + attacker could cause a node to be logged to death. + Consequently, the default answer_errors configuration has + been changed from report to discard. A service needs to + be restarted for the change in default to take effect.</p> + <p> + Own Id: OTP-11721</p> + </item> + <item> + <p> + Fix request table leak.</p> + <p> + Outgoing Diameter requests are stored in a table until an + answer is received or times out. Calling + diameter:stop_service/1 before this took place would + orphan the entries, resulting in a memory leak.</p> + <p> + Own Id: OTP-11893</p> + </item> + <item> + <p> + Fix broken SCTP transport.</p> + <p> + OTP-11593 caused the sending of answer messages over SCTP + to fail.</p> + <p> + Own Id: OTP-11901 Aux Id: OTP-11593 </p> + </item> + <item> + <p> + Fix watchdog process leak.</p> + <p> + A failed capabilities exchange on a listening transport + would orphan a process, causing a memory leak.</p> + <p> + Own Id: OTP-11934</p> + </item> + <item> + <p> + Fix incorrect handling of incoming DPR.</p> + <p> + In the case of a listening transport, a reconnection by a + peer following DPR could transition the watchdog state to + REOPEN instead of OKAY.</p> + <p> + Own Id: OTP-11938</p> + </item> + <item> + <p> + Fix handling of AVP length errors on unknown AVPs.</p> + <p> + An AVP (Header) length that pointed past the end of the + message was not flagged as a 5014 error in this case. + Moreover, encoding such an AVP in the Failed-AVP of an + answer message as a consequence of other errors (eg. + M-bit, resulting in 5001) failed if the AVP contained a + complete header.</p> + <p> + Dictionary files must be recompiled for the fix to have + effect.</p> + <p> + Own Id: OTP-11946</p> + </item> + <item> + <p> + Fix broken check in dictionary compilation.</p> + <p> + That an AVP specified in the content of a @codecs or + @custom_types section was undefined went undetected, + causing compilation to fail when attempting to lookup the + AVP's type.</p> + <p> + Own Id: OTP-11958</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Add result code counters for CEA, DWA, and DPA.</p> + <p> + In addition to the existing result code counters on other + answer messages.</p> + <p> + Own Id: OTP-11891</p> + </item> + <item> + <p> + Add best-effort decode of AVPs within Failed-AVP.</p> + <p> + OTP-11007 disabled the decode of AVPs in Failed-AVP since + errors could cause the decode of Failed-AVP itself to + fail. Component AVPs are now decoded if possible, + otherwise not. AVPs of type Grouped are decoded as much + as possible, as deeply as possible.</p> + <p> + Dictionary files must be recompiled for the fix to have + effect.</p> + <p> + Own Id: OTP-11936 Aux Id: OTP-11007 </p> + </item> + <item> + <p> + Add counters for encode errors in outgoing Diameter + messages.</p> + <p> + In addition to the existing counters on decode errors. + The latter now count independently of result codes in + answer messages since decode errors do not preclude the + presence of a result code.</p> + <p> + Own Id: OTP-11937</p> + </item> + </list> + </section> + +</section> + +<section><title>diameter 1.6</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Add missing check at dictionary compilation.</p> + <p> + In particular, that an AVP defined as having type Grouped + in an @avp_types section has a corresponding definition + in a @grouped section.</p> + <p> + Own Id: OTP-11561</p> + </item> + <item> + <p> + Correct documentation on the setting of Origin-State-Id</p> + <p> + It was incorrectly stated that the AVP would be set in an + outgoing DPR/DPA.</p> + <p> + Own Id: OTP-11583</p> + </item> + <item> + <p> + Change interface for communicating outbound stream id to + diameter_sctp</p> + <p> + The module uses the transport_data field of record + diameter_packet to communicate the stream on which the an + incoming message is received and on which an outgoing + message should be sent, the previous interface being that + both are communicated as a tuple of the form {stream, + Id}. However, since diameter retains the value of an + incoming request's transport_data unless the + corresponding answer message specifies otherwise, the + behaviour in this case is to send an answer on the + outbound stream with the same identifier as the that of + the inbound stream on which the request was received. If + the inbound stream id is greater than or equal to the + number of outbound streams then this is guaranteed to + fail, causing the transport process in question to + terminate. There is no relationship between inbound and + outbound stream identifiers so diameter_sctp's imposition + of one is simply wrong.</p> + <p> + Outbound stream ids are now communicated with a different + tuple: {outstream, Id}, interpreted modulo the number of + outbound streams. Thus, retention of an inbound request's + transport_data has no effect on the selection of an + outbound stream.</p> + <p> + The change in interface is not strictly backwards + compatible because of the new atom for the outbound + stream. However, as there is currently no documented way + of obtaining the available number of outbound streams for + a peer connection, there is no way for a client to have + known the range of ids from which it could reliably have + chosen with the previous interface, so any setting of the + outbound stream has probably been unintentional. Not + explicitly specifying an outbound stream now results in a + round-robin selection.</p> + <p> + Thanks to Sharmila Pillai for reporting the problem.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-11593</p> + </item> + <item> + <p> + Fix unicode path failure in diameter_make:codec/2.</p> + <p> + A dictionary path containing a unicode codepoint > 255 + caused the function to fail.</p> + <p> + Own Id: OTP-11655</p> + </item> + <item> + <p> + Fix 'accept' config to diameter_sctp.</p> + <p> + OTP-10893 added support for {accept, Match} tuples to + specify addresses or regexps that should be matched + against peer addresses to decide whether or not a newly + established association should be retained, but this + hasn't been functional in the SCTP case because of + missing support in inet(3).</p> + <p> + The display of both local and peer addresses in + diameter:service_info/2 output has also been corrected.</p> + <p> + Own Id: OTP-11661 Aux Id: OTP-10229 </p> + </item> + <item> + <p> + Be lenient with the M-bit in Grouped AVPs.</p> + <p> + RFC 6733 says this, in 4.4:</p> + <p> + <taglist><item><p><c>Receivers of a Grouped AVP that does + not have the 'M' (mandatory) bit set and one or more of + the encapsulated AVPs within the group has the 'M' + (mandatory) bit set MAY simply be ignored if the Grouped + AVP itself is unrecognized. The rule applies even if the + encapsulated AVP with its 'M' (mandatory) bit set is + further encapsulated within other sub-groups, i.e., other + Grouped AVPs embedded within the Grouped + AVP.</c></p></item></taglist></p> + <p> + The first sentence is mangled but take it to mean this:</p> + <p> + <taglist><item><p><c>An unrecognized AVP of type Grouped + that does not set the 'M' bit MAY be ignored even if one + of its encapsulated AVPs sets the 'M' + bit.</c></p></item></taglist></p> + <p> + This is a bit of a non-statement since if the AVP is + unrecognized then its type is unknown. We therefore don't + know that its data bytes contain encapsulated AVPs, so + can't but ignore any of those that set the M-bit. Doing + anything else when the type *is* known would be + inconsistent.</p> + <p> + OTP-11087 (R16B03) caused the M-bit on any unrecognized + AVP to be regarded as an error, unrecognized being taken + to mean "not explicitly defined as a member of its + container". (That is, an AVP that can't be packed into a + dedicated record field, which is slightly stronger than + "not defined".) This fixed the original intention for + top-level AVPs but broke the required leniency for + Grouped AVPs whose type is known. This leniency is now + restored.</p> + <p> + Note that dictionary files need to be recompiled for the + change to have effect.</p> + <p> + Thanks to Rory McKeown for reporting the problem.</p> + <p> + Own Id: OTP-11675 Aux Id: OTP-11087 </p> + </item> + <item> + <p> + Fix pick_peer case clause failure.</p> + <p> + In the case of {call_mutates_state, true} configuration + on the service in question, any peer selection that + failed to select a peer resulted in a case clause + failure. This was noticed in the case of a peer failover + in which an alternate peer wasn't available.</p> + <p> + Own Id: OTP-11789</p> + </item> + </list> + </section> + +</section> + <section><title>diameter 1.5</title> <section><title>Improvements and New Features</title> @@ -73,12 +419,6 @@ first.</p> <p> Own Id: OTP-11361</p> </item> - <item> - <p> - Fix silent make rules (Thanks to Anthony Ramine)</p> - <p> - Own Id: OTP-11514</p> - </item> </list> </section> @@ -86,7 +426,7 @@ first.</p> <section><title>diameter 1.4.4</title> - <section><title>Known Bugs and Problems</title> + <section><title>Fixed Bugs and Malfunctions</title> <list> <item> <p> diff --git a/lib/diameter/examples/code/client.erl b/lib/diameter/examples/code/client.erl index bfe71b0e56..46eb4a55db 100644 --- a/lib/diameter/examples/code/client.erl +++ b/lib/diameter/examples/code/client.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2012. All Rights Reserved. +%% Copyright Ericsson AB 2010-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in diff --git a/lib/diameter/examples/code/client_cb.erl b/lib/diameter/examples/code/client_cb.erl index ee3dcb2fec..843cdd9262 100644 --- a/lib/diameter/examples/code/client_cb.erl +++ b/lib/diameter/examples/code/client_cb.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2012. All Rights Reserved. +%% Copyright Ericsson AB 2010-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -77,23 +77,11 @@ prepare_retransmit(Packet, SvcName, Peer) -> %% handle_answer/4 -%% Since client.erl has detached the call when using the list -%% encoding and not otherwise, output to the terminal in the -%% the former case, return in the latter. - -handle_answer(#diameter_packet{msg = Msg}, Request, _SvcName, _Peer) - when is_list(Request) -> - io:format("answer: ~p~n", [Msg]); - handle_answer(#diameter_packet{msg = Msg}, _Request, _SvcName, _Peer) -> {ok, Msg}. %% handle_error/4 -handle_error(Reason, Request, _SvcName, _Peer) - when is_list(Request) -> - io:format("error: ~p~n", [Reason]); - handle_error(Reason, _Request, _SvcName, _Peer) -> {error, Reason}. diff --git a/lib/diameter/examples/code/redirect_cb.erl b/lib/diameter/examples/code/redirect_cb.erl index 69836774a1..8d98b0d2df 100644 --- a/lib/diameter/examples/code/redirect_cb.erl +++ b/lib/diameter/examples/code/redirect_cb.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2013. All Rights Reserved. +%% Copyright Ericsson AB 2010-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -34,12 +34,10 @@ -define(UNEXPECTED, erlang:error({unexpected, ?MODULE, ?LINE})). -peer_up(_SvcName, {PeerRef, _}, State) -> - io:format("up: ~p~n", [PeerRef]), +peer_up(_SvcName, _Peer, State) -> State. -peer_down(_SvcName, {PeerRef, _}, State) -> - io:format("down: ~p~n", [PeerRef]), +peer_down(_SvcName, _Peer, State) -> State. pick_peer(_, _, _SvcName, _State) -> diff --git a/lib/diameter/examples/code/relay_cb.erl b/lib/diameter/examples/code/relay_cb.erl index 9f9cd8d5ae..68798014e6 100644 --- a/lib/diameter/examples/code/relay_cb.erl +++ b/lib/diameter/examples/code/relay_cb.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2012. All Rights Reserved. +%% Copyright Ericsson AB 2010-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -32,12 +32,10 @@ handle_error/5, handle_request/3]). -peer_up(_SvcName, {PeerRef, _}, State) -> - io:format("up: ~p~n", [PeerRef]), +peer_up(_SvcName, _Peer, State) -> State. -peer_down(_SvcName, {PeerRef, _}, State) -> - io:format("down: ~p~n", [PeerRef]), +peer_down(_SvcName, _Peer, State) -> State. %% Returning 'relay' from handle_request causes diameter to resend the diff --git a/lib/diameter/examples/code/server_cb.erl b/lib/diameter/examples/code/server_cb.erl index 0f6eb32ed6..9d8d395d06 100644 --- a/lib/diameter/examples/code/server_cb.erl +++ b/lib/diameter/examples/code/server_cb.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2012. All Rights Reserved. +%% Copyright Ericsson AB 2010-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -38,12 +38,10 @@ -define(UNEXPECTED, erlang:error({unexpected, ?MODULE, ?LINE})). -peer_up(_SvcName, {PeerRef, _}, State) -> - io:format("up: ~p~n", [PeerRef]), +peer_up(_SvcName, _Peer, State) -> State. -peer_down(_SvcName, {PeerRef, _}, State) -> - io:format("down: ~p~n", [PeerRef]), +peer_down(_SvcName, _Peer, State) -> State. pick_peer(_, _, _SvcName, _State) -> @@ -68,10 +66,13 @@ handle_request(#diameter_packet{msg = Req, errors = []}, _SvcName, {_, Caps}) origin_realm = {OR,_}} = Caps, #diameter_base_RAR{'Session-Id' = Id, - 'Re-Auth-Request-Type' = RT} + 'Re-Auth-Request-Type' = Type} = Req, - {reply, answer(RT, Id, OH, OR)}; + {reply, #diameter_base_RAA{'Result-Code' = rc(Type), + 'Origin-Host' = OH, + 'Origin-Realm' = OR, + 'Session-Id' = Id}}; %% ... or one that wasn't. 3xxx errors are answered by diameter itself %% but these are 5xxx errors for which we must contruct a reply. @@ -84,32 +85,18 @@ handle_request(#diameter_packet{msg = Req}, _SvcName, {_, Caps}) #diameter_base_RAR{'Session-Id' = Id} = Req, - Ans = #diameter_base_RAA{'Origin-Host' = OH, - 'Origin-Realm' = OR, - 'Session-Id' = Id}, + {reply, #diameter_base_RAA{'Origin-Host' = OH, + 'Origin-Realm' = OR, + 'Session-Id' = Id}}; - {reply, Ans}; +%% Answer that any other message is unsupported. +handle_request(#diameter_packet{}, _SvcName, _) -> + {answer_message, 3001}. %% DIAMETER_COMMAND_UNSUPPORTED -%% Should really reply to other base messages that we don't support -%% but simply discard them instead. -handle_request(#diameter_packet{}, _SvcName, {_,_}) -> - discard. +%% Map Re-Auth-Request-Type to Result-Code just for the purpose of +%% generating different answers. -%% --------------------------------------------------------------------------- - -%% Answer using the record or list encoding depending on -%% Re-Auth-Request-Type. This is just as an example. You would -%% typically just choose one, and this has nothing to do with the how -%% client.erl sends. - -answer(0, Id, OH, OR) -> - #diameter_base_RAA{'Result-Code' = 2001, %% DIAMETER_SUCCESS - 'Origin-Host' = OH, - 'Origin-Realm' = OR, - 'Session-Id' = Id}; - -answer(_, Id, OH, OR) -> - ['RAA', {'Result-Code', 5012}, %% DIAMETER_UNABLE_TO_COMPLY - {'Origin-Host', OH}, - {'Origin-Realm', OR}, - {'Session-Id', Id}]. +rc(0) -> + 2001; %% DIAMETER_SUCCESS +rc(_) -> + 5012. %% DIAMETER_UNABLE_TO_COMPLY diff --git a/lib/diameter/include/diameter.hrl b/lib/diameter/include/diameter.hrl index 79c4dce541..c2c271a9a3 100644 --- a/lib/diameter/include/diameter.hrl +++ b/lib/diameter/include/diameter.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2013. All Rights Reserved. +%% Copyright Ericsson AB 2010-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -58,8 +58,8 @@ -record(diameter_header, {version, %% 8-bit unsigned length, %% 24-bit unsigned - cmd_code, %% 8-bit unsigned - application_id, %% 24-bit unsigned + cmd_code, %% 24-bit unsigned + application_id, %% 32-bit unsigned hop_by_hop_id, %% 32-bit unsigned end_to_end_id, %% 32-bit unsigned is_request, %% boolean() R flag @@ -126,7 +126,7 @@ default, extra = []}). -%% The diameter service and diameter_apps records are only passed +%% The diameter service and diameter_app records are only passed %% through the transport interface when starting a transport process, %% although typically a transport implementation will (and probably %% should) only be interested host_ip_address. @@ -143,6 +143,7 @@ init_state, %% option 'state', initial callback state id, %% 32-bit unsigned application identifier = Dict:id() mutable = false, %% boolean(), do traffic callbacks modify state? - options = [{answer_errors, report}, %% | callback | discard + options = [{answer_errors, discard}, %% | callback | report {request_errors, answer_3xxx}]}). %% | callback | answer + -endif. %% -ifdef(diameter_hrl). diff --git a/lib/diameter/include/diameter_gen.hrl b/lib/diameter/include/diameter_gen.hrl index c8f706dc3e..bc25f7d472 100644 --- a/lib/diameter/include/diameter_gen.hrl +++ b/lib/diameter/include/diameter_gen.hrl @@ -30,6 +30,10 @@ %% error or not. See is_strict/0. -define(STRICT_KEY, strict). +%% Key that says whether or not we should do a best-effort decode +%% within Failed-AVP. +-define(FAILED_KEY, failed). + -type parent_name() :: atom(). %% parent = Message or AVP -type parent_record() :: tuple(). %% -type avp_name() :: atom(). @@ -286,15 +290,7 @@ decode(Name, 'AVP', Avp, Acc) -> %% d/3 -%% Don't try to decode the value of a Failed-AVP component since it -%% probably won't. Note that matching on 'Failed-AVP' assumes that -%% this is the RFC AVP, with code 279. Strictly, this doesn't need to -%% be the case, so we're assuming no one defines another Failed-AVP. -d('Failed-AVP' = Name, Avp, Acc) -> - decode_AVP(Name, Avp, Acc); - -%% Or try to decode. -d(Name, Avp, {Avps, Acc}) -> +d(Name, Avp, Acc) -> #diameter_avp{name = AvpName, data = Data, type = Type, @@ -307,51 +303,120 @@ d(Name, Avp, {Avps, Acc}) -> %% value around through the entire decode. The solution here is %% simple in comparison, both to implement and to understand. - Reset = relax(Type, M), + Strict = relax(Type, M), + + %% Use the process dictionary again to keep track of whether we're + %% decoding within Failed-AVP and should ignore decode errors + %% altogether. + + Failed = relax(Name), %% Not AvpName or else a failed Failed-AVP + %% decode is packed into 'AVP'. + Mod = dict(Failed), %% Dictionary to decode in. - try avp(decode, Data, AvpName) of + try Mod:avp(decode, Data, AvpName) of V -> + {Avps, T} = Acc, {H, A} = ungroup(V, Avp), - {[H | Avps], pack_avp(Name, A, Acc)} + {[H | Avps], pack_avp(Name, A, T)} catch error: Reason -> - %% Failures here won't be visible since they're a "normal" - %% occurrence if the peer sends a faulty AVP that we need to - %% respond sensibly to. Log the occurence for traceability, - %% but the peer will also receive info in the resulting - %% answer-message. - diameter_lib:log({decode, failure}, - ?MODULE, - ?LINE, - {Reason, Avp, erlang:get_stacktrace()}), - {Rec, Failed} = Acc, - {[Avp|Avps], {Rec, [rc(Reason, Avp) | Failed]}} + d(undefined == Failed orelse is_failed(), + Reason, + Name, + trim(Avp), + Acc) after - relax(Reset) + reset(?STRICT_KEY, Strict), + reset(?FAILED_KEY, Failed) end. +%% trim/1 +%% +%% Remove any extra bit that was added in diameter_codec to induce a +%% 5014 error. + +trim(#diameter_avp{data = <<0:1, Bin/binary>>} = Avp) -> + Avp#diameter_avp{data = Bin}; + +trim(Avp) -> + Avp. + +%% dict/1 +%% +%% Retrieve the dictionary for the best-effort decode of Failed-AVP, +%% as put by diameter_codec:decode/2. See that function for the +%% explanation. + +dict(true) -> + case get({diameter_codec, dictionary}) of + undefined -> + ?MODULE; + Mod -> + Mod + end; + +dict(_) -> + ?MODULE. + +%% d/5 + +%% Ignore a decode error within Failed-AVP ... +d(true, _, Name, Avp, Acc) -> + decode_AVP(Name, Avp, Acc); + +%% ... or not. Failures here won't be visible since they're a "normal" +%% occurrence if the peer sends a faulty AVP that we need to respond +%% sensibly to. Log the occurence for traceability, but the peer will +%% also receive info in the resulting answer message. +d(false, Reason, Name, Avp, {Avps, Acc}) -> + Stack = diameter_lib:get_stacktrace(), + diameter_lib:log(decode_error, + ?MODULE, + ?LINE, + {Reason, Name, Avp#diameter_avp.name, Stack}), + {Rec, Failed} = Acc, + {[Avp|Avps], {Rec, [rc(Reason, Avp) | Failed]}}. + +%% relax/2 + %% Set false in the process dictionary as soon as we see a Grouped AVP %% that doesn't set the M-bit, so that is_strict() can say whether or %% not to ignore the M-bit on an encapsulated AVP. relax('Grouped', M) -> - V = getr(?STRICT_KEY), - if V == undefined andalso not M -> + case getr(?STRICT_KEY) of + undefined when not M -> putr(?STRICT_KEY, M); - true -> + _ -> false end; relax(_, _) -> false. -%% Reset strictness. -relax(undefined) -> - eraser(?STRICT_KEY); -relax(false) -> - ok. - is_strict() -> false /= getr(?STRICT_KEY). +%% relax/1 +%% +%% Set true in the process dictionary as soon as we see Failed-AVP. +%% Matching on 'Failed-AVP' assumes that this is the RFC AVP. +%% Strictly, this doesn't need to be the case. + +relax('Failed-AVP') -> + is_failed() orelse putr(?FAILED_KEY, true); + +relax(_) -> + is_failed(). + +is_failed() -> + true == getr(?FAILED_KEY). + +%% reset/2 + +reset(Key, undefined) -> + eraser(Key); +reset(_, _) -> + ok. + %% decode_AVP/3 %% %% Don't know this AVP: see if it can be packed in an 'AVP' field @@ -410,8 +475,25 @@ pack_avp(_, Arity, Avp, Acc) -> %% pack_AVP/3 -pack_AVP(Name, #diameter_avp{is_mandatory = M} = Avp, Acc) -> - case pack_arity(Name, M) of +%% Length failure was induced because of a header/payload length +%% mismatch. The AVP Length is reset to match the received data if +%% this AVP is encoded in an answer message, since the length is +%% computed. +%% +%% Data is a truncated header if command_code = undefined, otherwise +%% payload bytes. The former is padded to the length of a header if +%% the AVP reaches an outgoing encode in diameter_codec. +%% +%% RFC 6733 says that an AVP returned with 5014 can contain a minimal +%% payload for the AVP's type, but in this case we don't know the +%% type. + +pack_AVP(_, #diameter_avp{data = <<0:1, Data/binary>>} = Avp, Acc) -> + {Rec, Failed} = Acc, + {Rec, [{5014, Avp#diameter_avp{data = Data}} | Failed]}; + +pack_AVP(Name, #diameter_avp{is_mandatory = M, name = AvpName} = Avp, Acc) -> + case pack_arity(Name, AvpName, M) of 0 -> {Rec, Failed} = Acc, {Rec, [{if M -> 5001; true -> 5008 end, Avp} | Failed]}; @@ -419,15 +501,32 @@ pack_AVP(Name, #diameter_avp{is_mandatory = M} = Avp, Acc) -> pack(Arity, 'AVP', Avp, Acc) end. -%% Give Failed-AVP special treatment since it'll contain any -%% unrecognized mandatory AVP's. -pack_arity(Name, M) -> - case Name /= 'Failed-AVP' andalso M andalso is_strict() of - true -> - 0; - false -> - avp_arity(Name, 'AVP') - end. +%% Give Failed-AVP special treatment since (1) it'll contain any +%% unrecognized mandatory AVP's and (2) the RFC 3588 grammar failed to +%% allow for Failed-AVP in an answer-message. + +pack_arity(Name, AvpName, M) -> + IsFailed = Name == 'Failed-AVP' orelse is_failed(), + + %% Not testing just Name /= 'Failed-AVP' means we're changing the + %% packing of AVPs nested within Failed-AVP, but the point of + %% ignoring errors within Failed-AVP is to decode as much as + %% possible, and failing because a mandatory AVP couldn't be + %% packed into a dedicated field defeats that point. Note that we + %% can't just test not is_failed() since this will be 'true' when + %% packing an unknown AVP directly within Failed-AVP. + + pack_arity(IsFailed + orelse {Name, AvpName} == {'answer-message', 'Failed-AVP'} + orelse not M + orelse not is_strict(), + Name). + +pack_arity(true, Name) -> + avp_arity(Name, 'AVP'); + +pack_arity(false, _) -> + 0. %% 3588: %% diff --git a/lib/diameter/src/Makefile b/lib/diameter/src/Makefile index 578bbaee2e..9afccf298c 100644 --- a/lib/diameter/src/Makefile +++ b/lib/diameter/src/Makefile @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 2010-2013. All Rights Reserved. +# Copyright Ericsson AB 2010-2014. All Rights Reserved. # # The contents of this file are subject to the Erlang Public License, # Version 1.1, (the "License"); you may not use this file except in @@ -41,7 +41,7 @@ INCDIR = ../include ABS_EBIN := $(shell cd $(EBIN) && pwd) # Where make should look for dependencies. -VPATH = .:base:compiler:transport:gen +VPATH = .:base:compiler:transport:gen:info # ---------------------------------------------------- # Target specs @@ -55,13 +55,13 @@ DICT_ERLS = $(DICT_MODULES:%=%.erl) DICT_HRLS = $(DICT_MODULES:%=%.hrl) # Modules to build before compiling dictionaries. -COMPILER_MODULES = $(notdir $(filter compiler/%, $(CT_MODULES))) \ - $(DICT_YRL) +COMPILER_MODULES = $(notdir $(CT_MODULES)) $(DICT_YRL) # All handwritten modules from which a depend.mk is generated. MODULES = \ $(RT_MODULES) \ - $(CT_MODULES) + $(CT_MODULES) \ + $(INFO_MODULES) # Modules whose names are inserted into the app file. APP_MODULES = \ @@ -72,6 +72,7 @@ APP_MODULES = \ TARGET_MODULES = \ $(APP_MODULES) \ $(CT_MODULES) \ + $(INFO_MODULES) \ $(DICT_YRL:%=gen/%) # What to build for the 'opt' target. @@ -147,14 +148,19 @@ gen/$(DICT_YRL).erl: compiler/$(DICT_YRL).yrl $(ERLC) -Werror -o $(@D) $< # Generate the app file. -$(APP_TARGET): $(APP_SRC) ../vsn.mk modules.mk +$(APP_TARGET): $(APP_SRC) ../vsn.mk modules.mk app.sed $(gen_verbose) \ M=`echo $(notdir $(APP_MODULES)) | tr ' ' ,`; \ + C=`echo $(COMPILER_MODULES) | tr ' ' ,`; \ + I=`echo $(notdir $(INFO_MODULES)) | tr ' ' ,`; \ R=`echo $(REGISTERED) | tr ' ' ,`; \ sed -e 's;%VSN%;$(VSN);' \ -e "s;%MODULES%;$$M;" \ + -e "s;%COMPILER%;$$C;" \ + -e "s;%INFO%;$$I;" \ -e "s;%REGISTERED%;$$R;" \ - $< > $@ + $< \ + | sed -f app.sed > $@ $(APPUP_TARGET): $(APPUP_SRC) ../vsn.mk $(vsn_verbose) \ @@ -177,6 +183,8 @@ info: @echo @$(call list,CT_MODULES) @echo + @$(call list,INFO_MODULES) + @echo @$(call list,TARGET_MODULES) @echo @$(call list,TARGET_DIRS) @@ -216,7 +224,7 @@ dialyze: opt $(PLT) -Wno_improper_lists \ $(EBIN)/diameter_gen_base_rfc3588.$(EMULATOR) \ $(patsubst %, $(EBIN)/%.$(EMULATOR), \ - $(notdir $(RT_MODULES) $(CT_MODULES))) + $(notdir $(RT_MODULES) $(CT_MODULES) $(INFO_MODULES))) # Omit all but the common dictionary module since these # (diameter_gen_relay in particular) generate warning depending on how # much of the included diameter_gen.hrl they use. diff --git a/lib/diameter/src/app.sed b/lib/diameter/src/app.sed new file mode 100644 index 0000000000..7916f65002 --- /dev/null +++ b/lib/diameter/src/app.sed @@ -0,0 +1,40 @@ +# +# %CopyrightBegin% +# +# Copyright Ericsson AB 2014. All Rights Reserved. +# +# The contents of this file are subject to the Erlang Public License, +# Version 1.1, (the "License"); you may not use this file except in +# compliance with the License. You should have received a copy of the +# Erlang Public License along with this software. If not, it can be +# retrieved online at http://www.erlang.org/. +# +# Software distributed under the License is distributed on an "AS IS" +# basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +# the License for the specific language governing rights and limitations +# under the License. +# +# %CopyrightEnd% + +# +# Generate runtime_dependencies from applications to avoid having to +# specify the same application more than once. +# + +/{runtime_dependencies,/b v +/{[-a-z]*, "[0-9.]*"}/!b +/{vsn,/b + +/%%/!H +s/{\([^,]*\)[^}]*}/\1/g +s/%%/%,/ +b + +:v + +p +x +s/\n// +s/%//g +s/\n */ /g +s/{\([^,]*\), "\([^"]*"\)}/"\1-\2/g diff --git a/lib/diameter/src/base/diameter_codec.erl b/lib/diameter/src/base/diameter_codec.erl index 0de4d53973..a2b04bfd63 100644 --- a/lib/diameter/src/base/diameter_codec.erl +++ b/lib/diameter/src/base/diameter_codec.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2013. All Rights Reserved. +%% Copyright Ericsson AB 2010-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -70,12 +70,15 @@ encode(Mod, #diameter_packet{} = Pkt) -> try e(Mod, Pkt) catch + exit: {Reason, Stack, #diameter_header{} = H} = T -> + %% Exit with a header in the reason to let the caller + %% count encode errors. + ?LOG(encode_error, {Reason, Stack, H}), + exit({?MODULE, encode, T}); error: Reason -> - %% Be verbose since a crash report may be truncated and - %% encode errors are self-inflicted. - X = {?MODULE, encode, {Reason, ?STACK}}, - diameter_lib:error_report(X, {?MODULE, encode, [Mod, Pkt]}), - exit(X) + T = {Reason, diameter_lib:get_stacktrace()}, + ?LOG(encode_error, T), + exit({?MODULE, encode, T}) end; encode(Mod, Msg) -> @@ -87,53 +90,62 @@ encode(Mod, Msg) -> msg = Msg}). e(_, #diameter_packet{msg = [#diameter_header{} = Hdr | As]} = Pkt) -> - Avps = encode_avps(As), - Length = size(Avps) + 20, - - #diameter_header{version = Vsn, - cmd_code = Code, - application_id = Aid, - hop_by_hop_id = Hid, - end_to_end_id = Eid} - = Hdr, - - Flags = make_flags(0, Hdr), - - Pkt#diameter_packet{header = Hdr, - bin = <<Vsn:8, Length:24, - Flags:8, Code:24, - Aid:32, - Hid:32, - Eid:32, - Avps/binary>>}; + try encode_avps(As) of + Avps -> + Length = size(Avps) + 20, + + #diameter_header{version = Vsn, + cmd_code = Code, + application_id = Aid, + hop_by_hop_id = Hid, + end_to_end_id = Eid} + = Hdr, + + Flags = make_flags(0, Hdr), + + Pkt#diameter_packet{header = Hdr, + bin = <<Vsn:8, Length:24, + Flags:8, Code:24, + Aid:32, + Hid:32, + Eid:32, + Avps/binary>>} + catch + error: Reason -> + exit({Reason, diameter_lib:get_stacktrace(), Hdr}) + end; -e(Mod, #diameter_packet{header = Hdr, msg = Msg} = Pkt) -> +e(Mod, #diameter_packet{header = Hdr0, msg = Msg} = Pkt) -> #diameter_header{version = Vsn, hop_by_hop_id = Hid, end_to_end_id = Eid} - = Hdr, + = Hdr0, MsgName = rec2msg(Mod, Msg), - {Code, Flags0, Aid} = msg_header(Mod, MsgName, Hdr), - Flags = make_flags(Flags0, Hdr), - - Avps = encode_avps(Mod, MsgName, values(Msg)), - Length = size(Avps) + 20, - - Pkt#diameter_packet{header = Hdr#diameter_header - {length = Length, - cmd_code = Code, - application_id = Aid, - is_request = 0 /= ?MASK(7, Flags), - is_proxiable = 0 /= ?MASK(6, Flags), - is_error = 0 /= ?MASK(5, Flags), - is_retransmitted = 0 /= ?MASK(4, Flags)}, - bin = <<Vsn:8, Length:24, - Flags:8, Code:24, - Aid:32, - Hid:32, - Eid:32, - Avps/binary>>}. + {Code, Flags0, Aid} = msg_header(Mod, MsgName, Hdr0), + Flags = make_flags(Flags0, Hdr0), + Hdr = Hdr0#diameter_header{cmd_code = Code, + application_id = Aid, + is_request = 0 /= ?MASK(7, Flags), + is_proxiable = 0 /= ?MASK(6, Flags), + is_error = 0 /= ?MASK(5, Flags), + is_retransmitted = 0 /= ?MASK(4, Flags)}, + Values = values(Msg), + + try encode_avps(Mod, MsgName, Values) of + Avps -> + Length = size(Avps) + 20, + Pkt#diameter_packet{header = Hdr#diameter_header{length = Length}, + bin = <<Vsn:8, Length:24, + Flags:8, Code:24, + Aid:32, + Hid:32, + Eid:32, + Avps/binary>>} + catch + error: Reason -> + exit({Reason, diameter_lib:get_stacktrace(), Hdr}) + end. %% make_flags/2 @@ -225,15 +237,35 @@ rec2msg(Mod, Rec) -> %% Unsuccessfully decoded AVPs will be placed in #diameter_packet.errors. --spec decode(module(), #diameter_packet{} | bitstring()) +-spec decode(module() | {module(), module()}, #diameter_packet{} | binary()) -> #diameter_packet{}. +%% An Answer setting the E-bit. The application dictionary is needed +%% for the best-effort decode of Failed-AVP, and the best way to make +%% this available to the AVP decode in diameter_gen.hrl, without +%% having to rewrite the entire codec generation, is to place it in +%% the process dictionary. It's the code in diameter_gen.hrl (that's +%% included by every generated codec module) that looks for the entry. +%% Not ideal, but it solves the problem relatively simply. +decode({Mod, Mod}, Pkt) -> + decode(Mod, Pkt); +decode({Mod, AppMod}, Pkt) -> + Key = {?MODULE, dictionary}, + put(Key, AppMod), + try + decode(Mod, Pkt) + after + erase(Key) + end; + +%% Or not: a request, or an answer not setting the E-bit. decode(Mod, Pkt) -> decode(Mod:id(), Mod, Pkt). -%% If we're a relay application then just extract the avp's without -%% any decoding of their data since we don't know the application in -%% question. +%% decode/3 + +%% Relay application: just extract the avp's without any decoding of +%% their data since we don't know the application in question. decode(?APP_ID_RELAY, _, #diameter_packet{} = Pkt) -> case collect_avps(Pkt) of {E, As} -> @@ -259,34 +291,36 @@ decode(_, Mod, #diameter_packet{header = Hdr} = Pkt) -> decode_avps(MsgName, Mod, Pkt, collect_avps(Pkt)); decode(Id, Mod, Bin) - when is_bitstring(Bin) -> + when is_binary(Bin) -> decode(Id, Mod, #diameter_packet{header = decode_header(Bin), bin = Bin}). +%% decode_avps/4 + decode_avps(MsgName, Mod, Pkt, {E, Avps}) -> - ?LOG(invalid, Pkt#diameter_packet.bin), + ?LOG(invalid_avp_length, Pkt#diameter_packet.header), #diameter_packet{errors = Failed} = P = decode_avps(MsgName, Mod, Pkt, Avps), P#diameter_packet{errors = [E | Failed]}; -decode_avps('', Mod, Pkt, Avps) -> %% unknown message ... - ?LOG(unknown, {Mod, Pkt#diameter_packet.header}), +decode_avps('', _, Pkt, Avps) -> %% unknown message ... + ?LOG(unknown_message, Pkt#diameter_packet.header), Pkt#diameter_packet{avps = lists:reverse(Avps), errors = [3001]}; %% DIAMETER_COMMAND_UNSUPPORTED %% msg = undefined identifies this case. decode_avps(MsgName, Mod, Pkt, Avps) -> %% ... or not - {Rec, As, Failed} = Mod:decode_avps(MsgName, Avps), - ?LOGC([] /= Failed, failed, {Mod, Failed}), + {Rec, As, Errors} = Mod:decode_avps(MsgName, Avps), + ?LOGC([] /= Errors, decode_errors, Pkt#diameter_packet.header), Pkt#diameter_packet{msg = Rec, - errors = Failed, + errors = Errors, avps = As}. %%% --------------------------------------------------------------------------- %%% # decode_header/1 %%% --------------------------------------------------------------------------- --spec decode_header(bitstring()) +-spec decode_header(binary()) -> #diameter_header{} | false. @@ -297,7 +331,7 @@ decode_header(<<Version:8, ApplicationId:32, HopByHopId:32, EndToEndId:32, - _/bitstring>>) -> + _/binary>>) -> <<R:1, P:1, E:1, T:1, _:4>> = CmdFlags, %% 3588 (ch 3) says that reserved bits MUST be set to 0 and ignored @@ -410,7 +444,7 @@ msg_id(#diameter_header{application_id = A, is_request = R}) -> {A, C, if R -> 1; true -> 0 end}; -msg_id(<<_:32, Rbit:1, _:7, CmdCode:24, ApplId:32, _/bitstring>>) -> +msg_id(<<_:32, Rbit:1, _:7, CmdCode:24, ApplId:32, _/binary>>) -> {ApplId, CmdCode, Rbit}. %%% --------------------------------------------------------------------------- @@ -421,17 +455,18 @@ msg_id(<<_:32, Rbit:1, _:7, CmdCode:24, ApplId:32, _/bitstring>>) -> %% order in the binary. Note also that grouped avp's aren't unraveled, %% only those at the top level. --spec collect_avps(#diameter_packet{} | bitstring()) +-spec collect_avps(#diameter_packet{} | binary()) -> [Avp] | {Error, [Avp]} when Avp :: #diameter_avp{}, Error :: {5014, #diameter_avp{}}. collect_avps(#diameter_packet{bin = Bin}) -> - <<_:20/binary, Avps/bitstring>> = Bin, + <<_:20/binary, Avps/binary>> = Bin, collect_avps(Avps); -collect_avps(Bin) -> +collect_avps(Bin) + when is_binary(Bin) -> collect_avps(Bin, 0, []). collect_avps(<<>>, _, Acc) -> @@ -461,7 +496,9 @@ collect_avps(Bin, N, Acc) -> split_avp(Bin) -> {Code, V, M, P, Len, HdrLen} = split_head(Bin), - {Data, B} = split_data(Bin, HdrLen, Len - HdrLen), + + <<_:HdrLen/binary, Rest/binary>> = Bin, + {Data, B} = split_data(Rest, Len - HdrLen), {B, #diameter_avp{code = Code, vendor_id = V, @@ -471,17 +508,15 @@ split_avp(Bin) -> %% split_head/1 -split_head(<<Code:32, 1:1, M:1, P:1, _:5, Len:24, V:32, _/bitstring>>) -> +split_head(<<Code:32, 1:1, M:1, P:1, _:5, Len:24, V:32, _/binary>>) -> {Code, V, M, P, Len, 12}; -split_head(<<Code:32, 0:1, M:1, P:1, _:5, Len:24, _/bitstring>>) -> +split_head(<<Code:32, 0:1, M:1, P:1, _:5, Len:24, _/binary>>) -> {Code, undefined, M, P, Len, 8}; -%% Header is truncated: pack_avp/1 will pad to the minimum header -%% length. -split_head(B) - when is_bitstring(B) -> - ?THROW({5014, #diameter_avp{data = B}}). +%% Header is truncated. +split_head(Bin) -> + ?THROW({5014, #diameter_avp{data = Bin}}). %% 3588: %% @@ -516,34 +551,27 @@ split_head(B) %% split_data/3 -split_data(Bin, HdrLen, Len) - when 0 =< Len -> - split_data(Bin, HdrLen, Len, (4 - (Len rem 4)) rem 4); - -split_data(_, _, _) -> - invalid_avp_length(). +split_data(Bin, Len) -> + Pad = (4 - (Len rem 4)) rem 4, -%% split_data/4 + %% Len might be negative here, but that ensures the failure of the + %% binary match. -split_data(Bin, HdrLen, Len, Pad) -> case Bin of - <<_:HdrLen/binary, Data:Len/binary, _:Pad/binary, Rest/bitstring>> -> + <<Data:Len/binary, _:Pad/binary, Rest/binary>> -> {Data, Rest}; _ -> - invalid_avp_length() + %% Header length points past the end of the message. As + %% stated in the 6733 text above, it's sufficient to + %% return a zero-filled minimal payload if this is a + %% request. Do this (in cases that we know the type) by + %% inducing a decode failure and letting the dictionary's + %% decode (in diameter_gen) deal with it. Here we don't + %% know type. If the type isn't known, then the decode + %% just strips the extra bit. + {<<0:1, Bin/binary>>, <<>>} end. -%% invalid_avp_length/0 -%% -%% AVP Length doesn't mesh with payload. Induce a decode error by -%% returning a payload that no valid Diameter type can have. This is -%% so that a known AVP will result in 5014 error with a zero'd -%% payload. Here we simply don't know how to construct this payload. -%% (Yes, this solution is an afterthought.) - -invalid_avp_length() -> - {<<0:1>>, <<>>}. - %%% --------------------------------------------------------------------------- %%% # pack_avp/1 %%% --------------------------------------------------------------------------- @@ -575,17 +603,23 @@ pack_avp(#diameter_avp{data = {Dict, Name, Value}} = A) -> {Name, Type} = Dict:avp_name(Code, Vid), pack_avp(A#diameter_avp{data = {Hdr, {Type, Value}}}); +%% ... with a truncated header ... pack_avp(#diameter_avp{code = undefined, data = B}) - when is_bitstring(B) -> + when is_binary(B) -> %% Reset the AVP Length of an AVP Header resulting from a 5014 %% error. The RFC doesn't explicitly say to do this but the %% receiver can't correctly extract this and following AVP's %% without a correct length. On the downside, the header doesn't %% reveal if the received header has been padded. Pad = 8*header_length(B) - bit_size(B), - Len = size(<<H:5/binary, _:24, T/binary>> = <<B/bitstring, 0:Pad>>), + Len = size(<<H:5/binary, _:24, T/binary>> = <<B/binary, 0:Pad>>), <<H/binary, Len:24, T/binary>>; +%% ... from a dictionary compiled against old code in diameter_gen ... +%% ... when ignoring errors in Failed-AVP ... +pack_avp(#diameter_avp{data = <<0:1, B/binary>>} = A) -> + pack_avp(A#diameter_avp{data = B}); + %% ... or as an iolist. pack_avp(#diameter_avp{code = Code, vendor_id = V, diff --git a/lib/diameter/src/base/diameter_config.erl b/lib/diameter/src/base/diameter_config.erl index f5ea459fd0..dd1c9b73bb 100644 --- a/lib/diameter/src/base/diameter_config.erl +++ b/lib/diameter/src/base/diameter_config.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2013. All Rights Reserved. +%% Copyright Ericsson AB 2010-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -753,7 +753,7 @@ app_acc({application, Opts} = T, Acc) -> Alias = get_opt(alias, Opts, Dict), ModS = get_opt(state, Opts, Alias), M = get_opt(call_mutates_state, Opts, false, [true]), - A = get_opt(answer_errors, Opts, report, [callback, discard]), + A = get_opt(answer_errors, Opts, discard, [callback, report]), P = get_opt(request_errors, Opts, answer_3xxx, [answer, callback]), [#diameter_app{alias = Alias, dictionary = Dict, diff --git a/lib/diameter/src/base/diameter_lib.erl b/lib/diameter/src/base/diameter_lib.erl index 44d81e2778..5b3a2063f8 100644 --- a/lib/diameter/src/base/diameter_lib.erl +++ b/lib/diameter/src/base/diameter_lib.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2013. All Rights Reserved. +%% Copyright Ericsson AB 2010-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -25,6 +25,8 @@ now_diff/1, time/1, eval/1, + eval_name/1, + get_stacktrace/0, ipaddr/1, spawn_opts/2, wait/1, @@ -32,6 +34,22 @@ log/4]). %% --------------------------------------------------------------------------- +%% # get_stacktrace/0 +%% --------------------------------------------------------------------------- + +%% Return a stacktrace with a leading, potentially large, argument +%% list replaced by an arity. Trace on stacktrace/0 to see the +%% original. + +get_stacktrace() -> + stacktrace(erlang:get_stacktrace()). + +stacktrace([{M,F,A,L} | T]) when is_list(A) -> + [{M, F, length(A), L} | T]; +stacktrace(L) -> + L. + +%% --------------------------------------------------------------------------- %% # info_report/2 %% --------------------------------------------------------------------------- @@ -60,9 +78,17 @@ warning_report(Reason, T) -> report(fun error_logger:warning_report/1, Reason, T). report(Fun, Reason, T) -> - Fun([{why, Reason}, {who, self()}, {what, T}]), + Fun(io_lib:format("diameter: ~" ++ fmt(Reason) ++ "~n ~p~n", + [Reason, T])), false. +fmt(T) -> + if is_list(T) -> + "s"; + true -> + "p" + end. + %% --------------------------------------------------------------------------- %% # now_diff/1 %% --------------------------------------------------------------------------- @@ -129,8 +155,8 @@ eval({M,F,A}) -> eval([{M,F,A} | X]) -> apply(M, F, X ++ A); -eval([[F|A] | X]) -> - eval([F | X ++ A]); +eval([[F|X] | A]) -> + eval([F | A ++ X]); eval([F|A]) -> apply(F,A); @@ -142,6 +168,28 @@ eval(F) -> F(). %% --------------------------------------------------------------------------- +%% eval_name/1 +%% --------------------------------------------------------------------------- + +eval_name({M,F,A}) -> + {M, F, length(A)}; + +eval_name([{M,F,A} | X]) -> + {M, F, length(A) + length(X)}; + +eval_name([[F|A] | X]) -> + eval_name([F | X ++ A]); + +eval_name([F|_]) -> + F; + +eval_name({F}) -> + eval_name(F); + +eval_name(F) -> + F. + +%% --------------------------------------------------------------------------- %% # ipaddr/1 %% %% Parse an IP address. diff --git a/lib/diameter/src/base/diameter_peer_fsm.erl b/lib/diameter/src/base/diameter_peer_fsm.erl index f76bd96c3c..86fc43cdc5 100644 --- a/lib/diameter/src/base/diameter_peer_fsm.erl +++ b/lib/diameter/src/base/diameter_peer_fsm.erl @@ -283,7 +283,7 @@ handle_info(T, #state{} = State) -> ok -> {noreply, State}; #state{state = X} = S -> - ?LOGC(X =/= State#state.state, transition, X), + ?LOGC(X /= State#state.state, transition, X), {noreply, S}; {stop, Reason} -> ?LOG(stop, Reason), @@ -292,15 +292,12 @@ handle_info(T, #state{} = State) -> ?LOG(stop, T), {stop, {shutdown, T}, State} catch - exit: {diameter_codec, encode, _} = Reason -> + exit: {diameter_codec, encode, T} = Reason -> + incr_error(send, T, State#state.dictionary), ?LOG(stop, Reason), - %% diameter_codec:encode/2 emits an error report. Only - %% indicate the probable reason here. - diameter_lib:info_report(probable_configuration_error, - insufficient_capabilities), {stop, {shutdown, Reason}, State}; {?MODULE, Tag, Reason} -> - ?LOG(Tag, {Reason, T}), + ?LOG(stop, Tag), {stop, {shutdown, Reason}, State} end. %% The form of the throw caught here is historical. It's @@ -476,12 +473,13 @@ send_CER(#state{state = {'Wait-Conn-Ack', Tmo}, orelse close({already_connected, Remote, LCaps}), CER = build_CER(S), - ?LOG(send, 'CER'), #diameter_packet{header = #diameter_header{end_to_end_id = Eid, hop_by_hop_id = Hid}} = Pkt = encode(CER, Dict), + incr(send, Pkt, Dict), send(TPid, Pkt), + ?LOG(send, 'CER'), start_timer(Tmo, S#state{state = {'Wait-CEA', Hid, Eid}}). %% Register ourselves as connecting to the remote endpoint in @@ -526,7 +524,6 @@ recv(#diameter_packet{header = #diameter_header{} = Hdr} = S) -> Name = diameter_codec:msg_name(Dict0, Hdr), Pid ! {recv, self(), Name, Pkt}, - diameter_stats:incr({msg_id(Name, Hdr), recv}), %% count received rcv(Name, Pkt, S); recv(#diameter_packet{header = undefined, @@ -553,42 +550,30 @@ recv(#diameter_header{length = Len} recv(#diameter_header{} = H, #diameter_packet{bin = Bin}, - #state{length_errors = E} - = S) -> - invalid(E, - invalid_message_length, - recv, - [size(Bin), bit_size(Bin) rem 8, H, S]); + #state{length_errors = E}) -> + T = {size(Bin), bit_size(Bin) rem 8, H}, + invalid(E, message_length_mismatch, T); -recv(false, Pkt, #state{length_errors = E} = S) -> - invalid(E, truncated_header, recv, [Pkt, S]). +recv(false, #diameter_packet{bin = Bin}, #state{length_errors = E}) -> + invalid(E, truncated_header, Bin). %% Note that counters here only count discarded messages. -invalid(E, Reason, F, A) -> +invalid(E, Reason, T) -> diameter_stats:incr(Reason), - abort(E, Reason, F, A). - -abort(exit, Reason, F, A) -> - diameter_lib:warning_report(Reason, {?MODULE, F, A}), - throw({?MODULE, abort, Reason}); - -abort(_, _, _, _) -> + E == exit andalso close({Reason, T}), + ?LOG(Reason, T), ok. -msg_id({_,_,_} = T, _) -> - T; -msg_id(_, Hdr) -> - {_,_,_} = diameter_codec:msg_id(Hdr). - %% rcv/3 %% Incoming CEA. -rcv('CEA', +rcv('CEA' = N, #diameter_packet{header = #diameter_header{end_to_end_id = Eid, hop_by_hop_id = Hid}} = Pkt, #state{state = {'Wait-CEA', Hid, Eid}} = S) -> + ?LOG(recv, N), handle_CEA(Pkt, S); %% Incoming CER @@ -609,34 +594,71 @@ rcv('DPR' = N, Pkt, S) -> %% DPA in response to DPR and with the expected identifiers. rcv('DPA' = N, #diameter_packet{header = #diameter_header{end_to_end_id = Eid, - hop_by_hop_id = Hid}}, - #state{transport = TPid, + hop_by_hop_id = Hid} + = H} + = Pkt, + #state{dictionary = Dict0, + transport = TPid, dpr = {Hid, Eid}}) -> + ?LOG(recv, N), + incr(recv, H, Dict0), + incr_rc(recv, diameter_codec:decode(Dict0, Pkt), Dict0), diameter_peer:close(TPid), {stop, N}; %% Ignore anything else, an unsolicited DPA in particular. +rcv(N, #diameter_packet{header = H}, _) + when N == 'CER'; + N == 'CEA'; + N == 'DPR'; + N == 'DPA' -> + ?LOG(ignored, N), + %% Note that these aren't counted in the normal recv counter. + diameter_stats:incr({diameter_codec:msg_id(H), recv, ignored}), + ok; + rcv(_, _, _) -> ok. +%% incr/3 + +incr(Dir, Hdr, Dict0) -> + diameter_traffic:incr(Dir, Hdr, self(), Dict0). + +%% incr_rc/3 + +incr_rc(Dir, Pkt, Dict0) -> + diameter_traffic:incr_rc(Dir, Pkt, self(), Dict0). + +%% incr_error/3 + +incr_error(Dir, Pkt, Dict0) -> + diameter_traffic:incr_error(Dir, Pkt, self(), Dict0). + %% send/2 %% Msg here could be a #diameter_packet or a binary depending on who's %% sending. In particular, the watchdog will send DWR as a binary %% while messages coming from clients will be in a #diameter_packet. send(Pid, Msg) -> - diameter_stats:incr({diameter_codec:msg_id(Msg), send}), diameter_peer:send(Pid, Msg). %% handle_request/3 +%% +%% Incoming CER or DPR. -handle_request(Type, #diameter_packet{} = Pkt, #state{dictionary = D} = S) -> - ?LOG(recv, Type), - send_answer(Type, diameter_codec:decode(D, Pkt), S). +handle_request(Name, + #diameter_packet{header = H} = Pkt, + #state{dictionary = Dict0} = S) -> + ?LOG(recv, Name), + incr(recv, H, Dict0), + send_answer(Name, diameter_codec:decode(Dict0, Pkt), S). %% send_answer/3 send_answer(Type, ReqPkt, #state{transport = TPid, dictionary = Dict} = S) -> + incr_error(recv, ReqPkt, Dict), + #diameter_packet{header = H, transport_data = TD} = ReqPkt, @@ -653,13 +675,19 @@ send_answer(Type, ReqPkt, #state{transport = TPid, dictionary = Dict} = S) -> msg = Msg, transport_data = TD}, - send(TPid, diameter_codec:encode(Dict, Pkt)), + AnsPkt = diameter_codec:encode(Dict, Pkt), + + incr(send, AnsPkt, Dict), + incr_rc(send, AnsPkt, Dict), + send(TPid, AnsPkt), + ?LOG(send, ans(Type)), eval(PostF, S). +ans('CER') -> 'CEA'; +ans('DPR') -> 'DPA'. + eval([F|A], S) -> apply(F, A ++ [S]); -eval(ok, S) -> - S; eval(T, _) -> close(T). @@ -723,8 +751,8 @@ cea(CEA, RC, Dict0) -> post('CER' = T, RC, Pkt, S) -> {T, caps(S), {RC, Pkt}}; -post('DPR', _, _, _) -> - ok. +post('DPR' = T, _, _, #state{parent = Pid}) -> + [fun(S) -> Pid ! {T, self()}, S end]. rejected({capabilities_cb, _F, Reason}, T, S) -> rejected(Reason, T, S); @@ -734,7 +762,7 @@ rejected(discard, T, _) -> rejected({N, Es}, T, S) -> {answer('CER', N, failed_avp(N, Es), S), T}; rejected(N, T, S) -> - rejected({N, []}, T, S). + {answer('CER', N, [], S), T}. failed_avp(RC, [{RC, Avp} | _]) -> [{'Failed-AVP', [[{'AVP', [Avp]}]]}]; @@ -848,28 +876,27 @@ recv_CER(CER, #state{service = Svc, dictionary = Dict}) -> close({'CER', CER, Svc, Dict, Reason}) end. -%% handle_CEA/1 +%% handle_CEA/2 -handle_CEA(#diameter_packet{bin = Bin} +handle_CEA(#diameter_packet{header = H} = Pkt, #state{dictionary = Dict0, service = #diameter_service{capabilities = LCaps}} - = S) - when is_binary(Bin) -> - ?LOG(recv, 'CEA'), + = S) -> + incr(recv, H, Dict0), - #diameter_packet{msg = CEA} + #diameter_packet{} = DPkt = diameter_codec:decode(Dict0, Pkt), + RC = result_code(incr_rc(recv, DPkt, Dict0)), + {SApps, IS, RCaps} = recv_CEA(DPkt, S), #diameter_caps{origin_host = {OH, DH}} = Caps = capz(LCaps, RCaps), - RC = Dict0:'#get-'('Result-Code', CEA), - %% Ensure that we don't already have a connection to the peer in %% question. This isn't the peer election of 3588 except in the %% sense that, since we don't know who we're talking to until we @@ -877,7 +904,7 @@ handle_CEA(#diameter_packet{bin = Bin} %% connection with the peer. try - ?IS_SUCCESS(RC) + is_integer(RC) andalso ?IS_SUCCESS(RC) orelse ?THROW(RC), [] == SApps andalso ?THROW(no_common_application), @@ -897,6 +924,11 @@ handle_CEA(#diameter_packet{bin = Bin} %% capabilities exchange could send DIAMETER_LIMITED_SUCCESS = 2002, %% even if this isn't required by RFC 3588. +result_code({'Result-Code', N}) -> + N; +result_code(_) -> + undefined. + %% recv_CEA/2 recv_CEA(#diameter_packet{header = #diameter_header{version @@ -988,19 +1020,13 @@ capz(#diameter_caps{} = L, #diameter_caps{} = R) -> tl(tuple_to_list(R)))]). %% close/1 +%% +%% A good function to trace on in case of problems with capabilities +%% exchange. close(Reason) -> - report(Reason), throw({?MODULE, close, Reason}). -%% Could possibly log more here. -report({M, _, _, _, _} = T) - when M == 'CER'; - M == 'CEA' -> - diameter_lib:error_report(failure, T); -report(_) -> - ok. - %% dpr/2 %% %% The RFC isn't clear on whether DPR should be send in a non-Open @@ -1034,7 +1060,7 @@ dpr(_Reason, _S) -> %% process and contact it. (eg. diameter:service_info/2) dpr([CB|Rest], [Reason | _] = Args, S) -> - try diameter_lib:eval([CB | Args]) of + case diameter_lib:eval([CB | Args]) of {dpr, Opts} when is_list(Opts) -> send_dpr(Reason, Opts, S); dpr -> @@ -1044,14 +1070,7 @@ dpr([CB|Rest], [Reason | _] = Args, S) -> ignore -> dpr(Rest, Args, S); T -> - No = {disconnect_cb, T}, - diameter_lib:error_report(invalid, No), - {stop, No} - catch - E:R -> - No = {disconnect_cb, E, R, ?STACK}, - diameter_lib:error_report(failure, No), - {stop, No} + ?ERROR({disconnect_cb, CB, Args, T}) end; dpr([], [Reason | _], S) -> @@ -1082,6 +1101,7 @@ send_dpr(Reason, Opts, #state{transport = TPid, {'Origin-Realm', OR}, {'Disconnect-Cause', Cause}], Dict), + incr(send, Pkt, Dict), send(TPid, Pkt), dpa_timer(Tmo), ?LOG(send, 'DPR'), diff --git a/lib/diameter/src/base/diameter_service.erl b/lib/diameter/src/base/diameter_service.erl index 1274e0fc48..ab56ca9cef 100644 --- a/lib/diameter/src/base/diameter_service.erl +++ b/lib/diameter/src/base/diameter_service.erl @@ -499,9 +499,21 @@ transition(Req, S) -> %% # terminate/2 %% --------------------------------------------------------------------------- -terminate(Reason, #state{service_name = Name} = S) -> +terminate(Reason, #state{service_name = Name, peerT = PeerT} = S) -> send_event(Name, stop), ets:delete(?STATE_TABLE, Name), + + %% Communicate pending loss of any peers that connection_down/3 + %% won't. This is needed when stopping a service since we don't + %% wait for watchdog state changes to take care of if. That this + %% takes place after deleting the state entry ensures that the + %% resulting failover by request processes accomplishes nothing. + ets:foldl(fun(#peer{pid = TPid}, _) -> + diameter_traffic:peer_down(TPid) + end, + ok, + PeerT), + shutdown == Reason %% application shutdown andalso shutdown(application, S). @@ -719,14 +731,27 @@ remotes(F) -> L when is_list(L) -> L; T -> - diameter_lib:error_report({invalid_return, T}, F), + ?LOG(invalid_return, {F,T}), + error_report(invalid_return, share_peers, F), [] catch E:R -> - diameter_lib:error_report({failure, {E, R, ?STACK}}, F), + ?LOG(failure, {E, R, F, diameter_lib:get_stacktrace()}), + error_report(failure, share_peers, F), [] end. +%% error_report/3 + +error_report(T, What, F) -> + Reason = io_lib:format("~s from ~p callback", [reason(T), What]), + diameter_lib:error_report(Reason, diameter_lib:eval_name(F)). + +reason(invalid_return) -> + "invalid return"; +reason(failure) -> + "failure". + %% --------------------------------------------------------------------------- %% # start/3 %% --------------------------------------------------------------------------- @@ -869,7 +894,7 @@ watchdog(TPid, [], ?WD_OKAY, ?WD_SUSPECT = To, Wd, State) -> %% Watchdog has lost its connection. watchdog(TPid, [], _, ?WD_DOWN = To, Wd, #state{peerT = PeerT} = S) -> - close(Wd, S), + close(Wd), watchdog_down(Wd, To, S), ets:delete(PeerT, TPid); @@ -1026,8 +1051,11 @@ peer_cb(App, F, A) -> true catch E:R -> - diameter_lib:error_report({failure, {E, R, ?STACK}}, - {App, F, A}), + %% Don't include arguments since a #diameter_caps{} strings + %% from the peer, which could be anything (especially, large). + [Mod|X] = App#diameter_app.module, + ?LOG(failure, {E, R, Mod, F, diameter_lib:get_stacktrace()}), + error_report(failure, F, {Mod, F, A ++ X}), false end. @@ -1187,26 +1215,16 @@ tc(false = No, _, _) -> %% removed %% another watchdog to be able to detect that it should transition %% from initial into reopen rather than okay. That someone is either %% the accepting watchdog upon reception of a CER from the previously -%% connected peer, or us after connect_timer timeout. +%% connected peer, or us after connect_timer timeout or immediately. -close(#watchdog{type = connect}, _) -> +close(#watchdog{type = connect}) -> ok; + close(#watchdog{type = accept, pid = Pid, - ref = Ref, - options = Opts}, - #state{service_name = SvcName}) -> - c(Pid, diameter_config:have_transport(SvcName, Ref), Opts). - -%% Tell watchdog to (maybe) die later ... -c(Pid, true, Opts) -> + options = Opts}) -> Tc = connect_timer(Opts, 2*?DEFAULT_TC), - erlang:send_after(Tc, Pid, close); - -%% ... or now. -c(Pid, false, _Opts) -> - Pid ! close. - + erlang:send_after(Tc, Pid, close). %% The RFC's only document the behaviour of Tc, our connect_timer, %% for the establishment of connections but we also give %% connect_timer semantics for a listener, being the time within @@ -1260,13 +1278,14 @@ cm([#diameter_app{alias = Alias} = App], Req, From, Svc) -> mod_state(Alias, ModS), {T, RC}; T -> - diameter_lib:error_report({invalid, T}, - {App, handle_call, Args}), + ModX = App#diameter_app.module, + ?LOG(invalid_return, {ModX, handle_call, Args, T}), invalid catch E: Reason -> - diameter_lib:error_report({failure, {E, Reason, ?STACK}}, - {App, handle_call, Args}), + ModX = App#diameter_app.module, + Stack = diameter_lib:get_stacktrace(), + ?LOG(failure, {E, Reason, ModX, handle_call, Stack}), failure end; @@ -1389,6 +1408,8 @@ pick_peer(Local, Remote, Pid, _SvcName, #diameter_app{mutable = true} = App) case call_service(Pid, {pick_peer, Local, Remote, App}) of {TPid, _} = T when is_pid(TPid) -> T; + false = No -> + No; {error, _} -> false end; @@ -1422,13 +1443,16 @@ pick_peer(Local, T; %% Accept returned state in the immutable {false = No, S} -> %% case as long it isn't changed. No; - T -> - diameter_lib:error_report({invalid, T, App}, - {App, pick_peer, Args}) + T when M -> + ModX = App#diameter_app.module, + ?LOG(invalid_return, {ModX, pick_peer, T}), + false catch - E: Reason -> - diameter_lib:error_report({failure, {E, Reason, ?STACK}}, - {App, pick_peer, Args}) + E: Reason when M -> + ModX = App#diameter_app.module, + Stack = diameter_lib:get_stacktrace(), + ?LOG(failure, {E, Reason, ModX, pick_peer, Stack}), + false end. %% peers/4 @@ -1549,7 +1573,8 @@ transports(#state{watchdogT = WatchdogT}) -> -define(OTHER_INFO, [connections, name, peers, - statistics]). + statistics, + info]). service_info(Item, S) when is_atom(Item) -> @@ -1639,6 +1664,7 @@ complete_info(Item, #state{service = Svc} = S) -> keys -> ?ALL_INFO ++ ?CAP_INFO ++ ?OTHER_INFO; all -> service_info(?ALL_INFO, S); statistics -> info_stats(S); + info -> info_info(S); connections -> info_connections(S); peers -> info_peers(S) end. @@ -1721,12 +1747,11 @@ peer_acc(PeerT, Acc, #watchdog{pid = Pid, state = WS, started = At, peer = TPid}) -> - dict:append(Ref, - [{type, Type}, - {options, Opts}, - {watchdog, {Pid, At, WS}} - | info_peer(PeerT, TPid, WS)], - Acc). + Info = [{type, Type}, + {options, Opts}, + {watchdog, {Pid, At, WS}} + | info_peer(PeerT, TPid, WS)], + dict:append(Ref, Info ++ [{info, info_process_info(Info)}], Acc). info_peer(PeerT, TPid, WS) when is_pid(TPid), WS /= ?WD_DOWN -> @@ -1738,6 +1763,49 @@ info_peer(PeerT, TPid, WS) info_peer(_, _, _) -> []. +info_process_info(Info) -> + lists:flatmap(fun ipi/1, Info). + +ipi({watchdog, {Pid, _, _}}) -> + info_pid(Pid); + +ipi({peer, {Pid, _}}) -> + info_pid(Pid); + +ipi({port, [{owner, Pid} | _]}) -> + info_pid(Pid); + +ipi(_) -> + []. + +info_pid(Pid) -> + case process_info(Pid, [message_queue_len, memory, binary]) of + undefined -> + []; + L -> + [{Pid, lists:map(fun({K,V}) -> {K, map_info(K,V)} end, L)}] + end. + +%% The binary list consists of 3-tuples {Ptr, Size, Count}, where Ptr +%% is a C pointer value, Size is the size of a referenced binary in +%% bytes, and Count is a global reference count. The same Ptr can +%% occur multiple times, once for each reference on the process heap. +%% In this case, the corresponding tuples will have Size in common but +%% Count may differ just because no global lock is taken when the +%% value is retrieved. +%% +%% The list can be quite large, and we aren't often interested in the +%% pointers or counts, so whittle this down to the number of binaries +%% referenced and their total byte count. +map_info(binary, L) -> + SzD = lists:foldl(fun({P,S,_}, D) -> dict:store(P,S,D) end, + dict:new(), + L), + {dict:size(SzD), dict:fold(fun(_,S,N) -> S + N end, 0, SzD)}; + +map_info(_, T) -> + T. + %% The point of extracting the config here is so that 'transport' info %% has one entry for each transport ref, the peer table only %% containing entries that have a living watchdog. @@ -1795,6 +1863,13 @@ mk_app(#diameter_app{} = A) -> info_pending(#state{} = S) -> diameter_traffic:pending(transports(S)). +%% info_info/1 +%% +%% Extract process_info from connections info. + +info_info(S) -> + [I || L <- conn_list(S), {info, I} <- L]. + %% info_connections/1 %% %% One entry per transport connection. Statistics for each entry are diff --git a/lib/diameter/src/base/diameter_traffic.erl b/lib/diameter/src/base/diameter_traffic.erl index 7fbb306b02..280d09d7e8 100644 --- a/lib/diameter/src/base/diameter_traffic.erl +++ b/lib/diameter/src/base/diameter_traffic.erl @@ -31,6 +31,11 @@ %% towards diameter_watchdog -export([receive_message/4]). +%% towards diameter_peer_fsm and diameter_watchdog +-export([incr/4, + incr_error/4, + incr_rc/4]). + %% towards diameter_service -export([make_recvdata/1, peer_up/1, @@ -44,6 +49,8 @@ -include_lib("diameter/include/diameter.hrl"). -include("diameter_internal.hrl"). +-define(LOGX(Reason, T), begin ?LOG(Reason, T), x({Reason, T}) end). + -define(RELAY, ?DIAMETER_DICT_RELAY). -define(BASE, ?DIAMETER_DICT_COMMON). %% Note: the RFC 3588 dictionary @@ -109,6 +116,72 @@ peer_down(TPid) -> failover(TPid). %% --------------------------------------------------------------------------- +%% incr/4 +%% --------------------------------------------------------------------------- + +incr(Dir, #diameter_packet{header = H}, TPid, Dict) -> + incr(Dir, H, TPid, Dict); + +incr(Dir, #diameter_header{} = H, TPid, Dict) -> + incr(TPid, {msg_id(H, Dict), Dir}). + +%% --------------------------------------------------------------------------- +%% incr_error/4 +%% --------------------------------------------------------------------------- + +%% Identify messages using the application dictionary, not the encode +%% dictionary, which may differ in the case of answer-message. +incr_error(Dir, T, Pid, {_Dict, AppDict}) -> + incr_error(Dir, T, Pid, AppDict); + +%% Decoded message without errors. +incr_error(recv, #diameter_packet{errors = []}, _, _) -> + ok; + +incr_error(recv = D, #diameter_packet{header = H}, TPid, Dict) -> + incr_error(D, H, TPid, Dict); + +%% Encoded message with errors and an identifiable header ... +incr_error(send = D, {_, _, #diameter_header{} = H}, TPid, Dict) -> + incr_error(D, H, TPid, Dict); + +%% ... or not. +incr_error(send = D, {_,_}, TPid, _) -> + incr_error(D, unknown, TPid); + +incr_error(Dir, #diameter_header{} = H, TPid, Dict) -> + incr_error(Dir, msg_id(H, Dict), TPid); + +incr_error(Dir, Id, TPid, _) -> + incr_error(Dir, Id, TPid). + +incr_error(Dir, Id, TPid) -> + incr(TPid, {Id, Dir, error}). + +%% --------------------------------------------------------------------------- +%% incr_rc/4 +%% --------------------------------------------------------------------------- + +-spec incr_rc(send|recv, Pkt, TPid, Dict0) + -> {Counter, non_neg_integer()} + | Reason + when Pkt :: #diameter_packet{}, + TPid :: pid(), + Dict0 :: module(), + Counter :: {'Result-Code', integer()} + | {'Experimental-Result', integer(), integer()}, + Reason :: atom(). + +incr_rc(Dir, Pkt, TPid, Dict0) -> + try + incr_result(Dir, Pkt, TPid, {Dict0, Dict0, Dict0}) + catch + exit: {E,_} when E == no_result_code; + E == invalid_error_bit -> + E + end. + +%% --------------------------------------------------------------------------- %% pending/1 %% --------------------------------------------------------------------------- @@ -182,7 +255,7 @@ spawn_request(TPid, Pkt, Dict0, Opts, RecvData) -> spawn_opt(fun() -> recv_request(TPid, Pkt, Dict0, RecvData) end, Opts) catch error: system_limit = E -> %% discard - ?LOG({error, E}, now()) + ?LOG(error, E) end. %% --------------------------------------------------------------------------- @@ -211,7 +284,9 @@ recv_R({#diameter_app{id = Id, dictionary = Dict} = App, Caps}, Pkt0, Dict0, RecvData) -> + incr(recv, Pkt0, TPid, Dict), Pkt = errors(Id, diameter_codec:decode(Id, Dict, Pkt0)), + incr_error(recv, Pkt, TPid, Dict), {Caps, Pkt, App, recv_R(App, TPid, Dict0, Caps, RecvData, Pkt)}; %% Note that the decode is different depending on whether or not Id is %% ?APP_ID_RELAY. @@ -283,23 +358,25 @@ rc(N) -> %% This error is returned when a request is received with an invalid %% message length. -errors(_, #diameter_packet{header = #diameter_header{length = Len}, +errors(_, #diameter_packet{header = #diameter_header{length = Len} = H, bin = Bin, errors = Es} = Pkt) when Len < 20; 0 /= Len rem 4; 8*Len /= bit_size(Bin) -> + ?LOG(invalid_message_length, {H, bit_size(Bin)}), Pkt#diameter_packet{errors = [5015 | Es]}; %% DIAMETER_UNSUPPORTED_VERSION 5011 %% This error is returned when a request was received, whose version %% number is unsupported. -errors(_, #diameter_packet{header = #diameter_header{version = V}, +errors(_, #diameter_packet{header = #diameter_header{version = V} = H, errors = Es} = Pkt) when V /= ?DIAMETER_VERSION -> + ?LOG(unsupported_version, H), Pkt#diameter_packet{errors = [5011 | Es]}; %% DIAMETER_COMMAND_UNSUPPORTED 3001 @@ -307,12 +384,13 @@ errors(_, #diameter_packet{header = #diameter_header{version = V}, %% recognize or support. This MUST be used when a Diameter node %% receives an experimental command that it does not understand. -errors(Id, #diameter_packet{header = #diameter_header{is_proxiable = P}, +errors(Id, #diameter_packet{header = #diameter_header{is_proxiable = P} = H, msg = M, errors = Es} = Pkt) when ?APP_ID_RELAY /= Id, undefined == M; %% don't know the command ?APP_ID_RELAY == Id, not P -> %% command isn't proxiable + ?LOG(command_unsupported, H), Pkt#diameter_packet{errors = [3001 | Es]}; %% DIAMETER_INVALID_HDR_BITS 3008 @@ -321,9 +399,11 @@ errors(Id, #diameter_packet{header = #diameter_header{is_proxiable = P}, %% inconsistent with the command code's definition. errors(_, #diameter_packet{header = #diameter_header{is_request = true, - is_error = true}, + is_error = true} + = H, errors = Es} = Pkt) -> + ?LOG(invalid_hdr_bits, H), Pkt#diameter_packet{errors = [3008 | Es]}; %% Green. @@ -396,7 +476,7 @@ send_A({Caps, Pkt}, TPid, Dict0, _RecvData) -> %% unsupported application #diameter_packet{errors = [RC|_]} = Pkt, send_A(answer_message(RC, Caps, Dict0, Pkt), TPid, - Dict0, + {Dict0, Dict0}, Pkt, [], []); @@ -404,7 +484,7 @@ send_A({Caps, Pkt}, TPid, Dict0, _RecvData) -> %% unsupported application send_A({Caps, Pkt, App, {T, EvalPktFs, EvalFs}}, TPid, Dict0, RecvData) -> send_A(answer(T, Caps, Pkt, App, Dict0, RecvData), TPid, - Dict0, + {App#diameter_app.dictionary, Dict0}, Pkt, EvalPktFs, EvalFs); @@ -414,8 +494,8 @@ send_A(_, _, _, _) -> %% send_A/6 -send_A(T, TPid, Dict0, ReqPkt, EvalPktFs, EvalFs) -> - reply(T, TPid, Dict0, EvalPktFs, ReqPkt), +send_A(T, TPid, DictT, ReqPkt, EvalPktFs, EvalFs) -> + reply(T, TPid, DictT, EvalPktFs, ReqPkt), lists:foreach(fun diameter_lib:eval/1, EvalFs). %% answer/6 @@ -479,7 +559,6 @@ answer_message(RC, origin_realm = {OR,_}}, Dict0, Pkt) -> - ?LOG({error, RC}, Pkt), {Dict0, answer_message(OH, OR, RC, Dict0, Pkt)}. %% resend/7 @@ -574,30 +653,32 @@ is_loop(Code, Vid, OH, Dict0, Avps) -> %% reply/5 %% Local answer ... -reply({Dict, Ans}, TPid, Dict0, Fs, ReqPkt) -> - reply(Ans, Dict, TPid, Dict0, Fs, ReqPkt); +reply({Dict, Ans}, TPid, {AppDict, Dict0}, Fs, ReqPkt) -> + local(Ans, TPid, {Dict, AppDict, Dict0}, Fs, ReqPkt); %% ... or relayed. reply(#diameter_packet{} = Pkt, TPid, _Dict0, Fs, _ReqPkt) -> eval_packet(Pkt, Fs), send(TPid, Pkt). -%% reply/6 +%% local/5 %% %% Send a locally originating reply. %% Skip the setting of Result-Code and Failed-AVP's below. This is %% undocumented and shouldn't be relied on. -reply([Msg], Dict, TPid, Dict0, Fs, ReqPkt) +local([Msg], TPid, DictT, Fs, ReqPkt) when is_list(Msg); is_tuple(Msg) -> - reply(Msg, Dict, TPid, Dict0, Fs, ReqPkt#diameter_packet{errors = []}); + local(Msg, TPid, DictT, Fs, ReqPkt#diameter_packet{errors = []}); -reply(Msg, Dict, TPid, Dict0, Fs, ReqPkt) -> - Pkt = encode(Dict, +local(Msg, TPid, {Dict, AppDict, Dict0} = DictT, Fs, ReqPkt) -> + Pkt = encode({Dict, AppDict}, + TPid, reset(make_answer_packet(Msg, ReqPkt), Dict, Dict0), Fs), - incr(send, Pkt, Dict, TPid, Dict0), %% count outgoing result codes + incr(send, Pkt, TPid, AppDict), + incr_result(send, Pkt, TPid, DictT), %% count outgoing send(TPid, Pkt). %% reset/3 @@ -962,35 +1043,63 @@ find(Pred, [H|T]) -> %% code, the missing vendor id, and a zero filled payload of the minimum %% required length for the omitted AVP will be added. -%% incr/4 +%% incr_result/5 %% %% Increment a stats counter for result codes in incoming and outgoing %% answers. %% Outgoing message as binary: don't count. (Sending binaries is only %% partially supported.) -incr(_, #diameter_packet{msg = undefined}, _, _, _) -> - ok; - -%% Incoming with decode errors. -incr(recv = D, #diameter_packet{header = H, errors = [_|_]}, _, TPid, _) -> - incr(TPid, {diameter_codec:msg_id(H), D, error}); +incr_result(_, #diameter_packet{msg = undefined = No}, _, _) -> + No; -%% Incoming without errors or outgoing. Outgoing with encode errors -%% never gets here since encode fails. -incr(Dir, Pkt, Dict, TPid, Dict0) -> +%% Incoming or outgoing. Outgoing with encode errors never gets here +%% since encode fails. +incr_result(Dir, Pkt, TPid, {Dict, AppDict, Dict0}) -> #diameter_packet{header = #diameter_header{is_error = E} = Hdr, - msg = Rec} + msg = Msg, + errors = Es} = Pkt, - RC = int(get_avp_value(Dict, 'Result-Code', Rec)), + Id = msg_id(Hdr, AppDict), + + %% Count incoming decode errors. + recv /= Dir orelse [] == Es orelse incr_error(Dir, Id, TPid, AppDict), - %% Exit on an improper Result-Code. + %% Exit on a missing result code. + T = rc_counter(Dict, Msg), + T == false andalso ?LOGX(no_result_code, {Dict, Dir, Hdr}), + {Ctr, RC} = T, + + %% Or on an inappropriate value. is_result(RC, E, Dict0) - orelse x({invalid_error_bit, RC}, answer, [Dir, Pkt]), + orelse ?LOGX(invalid_error_bit, {Dict, Dir, Hdr, RC}), + + incr(TPid, {Id, Dir, Ctr}), + Ctr. - irc(TPid, Hdr, Dir, rc_counter(Dict, Rec, RC)). +%% msg_id/2 + +msg_id(#diameter_packet{header = H}, Dict) -> + msg_id(H, Dict); + +%% Only count on known keys so as not to be vulnerable to attack: +%% there are 2^32 (application ids) * 2^24 (command codes) = 2^56 +%% pairs for an attacker to choose from. +msg_id(Hdr, Dict) -> + {_ApplId, Code, R} = Id = diameter_codec:msg_id(Hdr), + case Dict:msg_name(Code, 0 == R) of + '' -> + unknown(Dict:id(), R); + _ -> + Id + end. + +unknown(?APP_ID_RELAY, R) -> + {relay, R}; +unknown(_, _) -> + unknown. %% No E-bit: can't be 3xxx. is_result(RC, false, _Dict0) -> @@ -1006,12 +1115,6 @@ is_result(RC, true, _) -> orelse 5000 =< RC andalso RC < 6000. -irc(_, _, _, undefined) -> - false; - -irc(TPid, Hdr, Dir, Ctr) -> - incr(TPid, {diameter_codec:msg_id(Hdr), Dir, Ctr}). - %% incr/2 incr(TPid, Counter) -> @@ -1024,14 +1127,16 @@ incr(TPid, Counter) -> %% All Diameter answer messages defined in vendor-specific %% applications MUST include either one Result-Code AVP or one %% Experimental-Result AVP. -%% -%% Maintain statistics assuming one or the other, not both, which is -%% surely the intent of the RFC. -rc_counter(Dict, Rec, undefined) -> - rcc(get_avp_value(Dict, 'Experimental-Result', Rec)); -rc_counter(_, _, RC) -> - {'Result-Code', RC}. +rc_counter(Dict, Msg) -> + rcc(Dict, Msg, int(get_avp_value(Dict, 'Result-Code', Msg))). + +rcc(Dict, Msg, undefined) -> + rcc(get_avp_value(Dict, 'Experimental-Result', Msg)); + +rcc(_, _, N) + when is_integer(N) -> + {{'Result-Code', N}, N}. %% Outgoing answers may be in any of the forms messages can be sent %% in. Incoming messages will be records. We're assuming here that the @@ -1039,12 +1144,12 @@ rc_counter(_, _, RC) -> rcc([{_,_,N} = T | _]) when is_integer(N) -> - T; + {T,N}; rcc({_,_,N} = T) when is_integer(N) -> - T; + {T,N}; rcc(_) -> - undefined. + false. %% Extract the first good looking integer. There's no guarantee %% that what we're looking for has arity 1. @@ -1057,13 +1162,6 @@ int(N) int(_) -> undefined. --spec x(any(), atom(), list()) -> no_return(). - -%% Warn and exit request process on errors in an incoming answer. -x(Reason, F, A) -> - diameter_lib:warning_report(Reason, {?MODULE, F, A}), - x(Reason). - x(T) -> exit(T). @@ -1305,7 +1403,7 @@ send_R(Pkt0, {Pid, Ref}, SvcName, Fs) -> - Pkt = encode(Dict, Pkt0, Fs), + Pkt = encode(Dict, TPid, Pkt0, Fs), #options{timeout = Timeout} = Opts, @@ -1318,6 +1416,7 @@ send_R(Pkt0, packet = Pkt0}, try + incr(send, Pkt, TPid, Dict), TRef = send_request(TPid, Pkt, Req, SvcName, Timeout), Pid ! Ref, %% tell caller a send has been attempted handle_answer(SvcName, @@ -1353,14 +1452,14 @@ handle_answer(SvcName, App, {error, Req, Reason}) -> handle_error(App, Req, Reason, SvcName); handle_answer(SvcName, - #diameter_app{dictionary = Dict, + #diameter_app{dictionary = AppDict, id = Id} = App, {answer, Req, Dict0, Pkt}) -> - Mod = dict(Dict, Dict0, Pkt), - handle_A(errors(Id, diameter_codec:decode(Mod, Pkt)), + Dict = dict(AppDict, Dict0, Pkt), + handle_A(errors(Id, diameter_codec:decode({Dict, AppDict}, Pkt)), SvcName, - Mod, + Dict, Dict0, App, Req). @@ -1370,11 +1469,21 @@ handle_answer(SvcName, %% want to examine the answer? handle_A(Pkt, SvcName, Dict, Dict0, App, #request{transport = TPid} = Req) -> + AppDict = App#diameter_app.dictionary, + + incr(recv, Pkt, TPid, AppDict), + try - incr(recv, Pkt, Dict, TPid, Dict0) %% count incoming result codes + incr_result(recv, Pkt, TPid, {Dict, AppDict, Dict0}) %% count incoming of _ -> answer(Pkt, SvcName, App, Req) catch + exit: {no_result_code, _} -> + %% RFC 6733 requires one of Result-Code or + %% Experimental-Result, but the decode will have detected + %% a missing AVP. If both are optional in the dictionary + %% then this isn't a decode error: just continue on. + answer(Pkt, SvcName, App, Req); exit: {invalid_error_bit, RC} -> #diameter_packet{errors = Es} = Pkt, @@ -1382,6 +1491,8 @@ handle_A(Pkt, SvcName, Dict, Dict0, App, #request{transport = TPid} = Req) -> answer(Pkt#diameter_packet{errors = [E|Es]}, SvcName, App, Req) end. +%% answer/4 + answer(Pkt, SvcName, #diameter_app{module = ModX, @@ -1401,11 +1512,16 @@ a(#diameter_packet{errors = Es} callback == AE -> cb(ModX, handle_answer, [Pkt, msg(P), SvcName, {TPid, Caps}]); -a(Pkt, SvcName, _, report, Req) -> - x(errors, handle_answer, [SvcName, Req, Pkt]); +a(Pkt, SvcName, _, AE, _) -> + a(Pkt#diameter_packet.header, SvcName, AE). -a(Pkt, SvcName, _, discard, Req) -> - x({errors, handle_answer, [SvcName, Req, Pkt]}). +a(Hdr, SvcName, report) -> + MFA = {?MODULE, handle_answer, [SvcName, Hdr]}, + diameter_lib:warning_report(errors, MFA), + a(Hdr, SvcName, discard); + +a(Hdr, SvcName, discard) -> + x({answer_errors, {SvcName, Hdr}}). %% Note that we don't check that the application id in the answer's %% header is what we expect. (TODO: Does the rfc says anything about @@ -1463,10 +1579,10 @@ msg(#diameter_packet{msg = undefined, bin = Bin}) -> msg(#diameter_packet{msg = Msg}) -> Msg. -%% encode/3 +%% encode/4 -encode(Dict, Pkt, Fs) -> - P = encode(Dict, Pkt), +encode(Dict, TPid, Pkt, Fs) -> + P = encode(Dict, TPid, Pkt), eval_packet(P, Fs), P. @@ -1477,12 +1593,23 @@ encode(Dict, Pkt, Fs) -> %% an encoded binary. This isn't the usual case and doesn't properly %% support retransmission but is useful for test. +encode(Dict, TPid, Pkt) + when is_atom(Dict) -> + encode({Dict, Dict}, TPid, Pkt); + %% A message to be encoded. -encode(Dict, #diameter_packet{bin = undefined} = Pkt) -> - diameter_codec:encode(Dict, Pkt); +encode(DictT, TPid, #diameter_packet{bin = undefined} = Pkt) -> + {Dict, AppDict} = DictT, + try + diameter_codec:encode(Dict, Pkt) + catch + exit: {diameter_codec, encode, T} = Reason -> + incr_error(send, T, TPid, AppDict), + exit(Reason) + end; %% An encoded binary: just send. -encode(_, #diameter_packet{} = Pkt) -> +encode(_, _, #diameter_packet{} = Pkt) -> Pkt. %% send_request/5 @@ -1579,13 +1706,14 @@ resend_request(Pkt0, SvcName, Tmo, Fs) -> - Pkt = encode(Dict, Pkt0, Fs), + Pkt = encode(Dict, TPid, Pkt0, Fs), Req = Req0#request{transport = TPid, packet = Pkt0, caps = Caps}, - ?LOG(retransmission, Req), + ?LOG(retransmission, Pkt#diameter_packet.header), + incr(TPid, {msg_id(Pkt, Dict), send, retransmission}), TRef = send_request(TPid, Pkt, Req, SvcName, Tmo), {TRef, Req}. diff --git a/lib/diameter/src/base/diameter_watchdog.erl b/lib/diameter/src/base/diameter_watchdog.erl index 53e659e3f6..eff5096745 100644 --- a/lib/diameter/src/base/diameter_watchdog.erl +++ b/lib/diameter/src/base/diameter_watchdog.erl @@ -49,8 +49,6 @@ -define(IS_NATURAL(N), (is_integer(N) andalso 0 =< N)). --define(CHOOSE(B,T,F), if (B) -> T; true -> F end). - -record(config, {suspect = 1 :: non_neg_integer(), %% OKAY -> SUSPECT okay = 3 :: non_neg_integer()}). %% REOPEN -> OKAY @@ -221,7 +219,6 @@ dict0(_, _, Acc) -> Acc. config_error(T) -> - diameter_lib:error_report(configuration_error, T), exit({shutdown, {configuration_error, T}}). %% handle_call/3 @@ -270,7 +267,7 @@ event(Msg, TPid = tpid(F,T), E = {[TPid | data(Msg, TPid, From, To)], From, To}, send(Pid, {watchdog, self(), E}), - ?LOG(transition, {self(), E}). + ?LOG(transition, {From, To}). data(Msg, TPid, reopen, okay) -> {recv, TPid, 'DWA', _Pkt} = Msg, %% assert @@ -313,14 +310,13 @@ code_change(_, State, _) -> %% The state transitions documented here are extracted from RFC 3539, %% the commentary is ours. -%% Service or watchdog is telling the watchdog of an accepting -%% transport to die after connect_timer expiry or reestablished -%% connection (in another transport process) respectively. -transition(close, #watchdog{status = down}) -> +%% Service is telling the watchdog of an accepting transport to die +%% following transport death in state INITIAL, or after connect_timer +%% expiry; or another watchdog is saying the same after reestablishing +%% a connection previously had by this one. +transition(close, #watchdog{}) -> {{accept, _}, _, _} = getr(restart), %% assert stop; -transition(close, #watchdog{}) -> - ok; %% Service is asking for the peer to be taken down gracefully. transition({shutdown, Pid, _}, #watchdog{parent = Pid, @@ -332,6 +328,11 @@ transition({shutdown = T, Pid, Reason}, #watchdog{parent = Pid, send(TPid, {T, self(), Reason}), S#watchdog{shutdown = true}; +%% Transport is telling us that DPA has been sent in response to DPR: +%% its death should lead to ours. +transition({'DPR', TPid}, #watchdog{transport = TPid} = S) -> + S#watchdog{shutdown = true}; + %% Parent process has died, transition({'DOWN', _, process, Pid, _Reason}, #watchdog{parent = Pid}) -> @@ -403,18 +404,39 @@ transition({open = Key, TPid, _Hosts, T}, %% REOPEN Connection down CloseConnection() %% SetWatchdog() DOWN +%% Transport has died after DPA or service requested termination ... transition({'DOWN', _, process, TPid, _Reason}, #watchdog{transport = TPid, shutdown = true}) -> stop; +%% ... or not. transition({'DOWN', _, process, TPid, _Reason}, #watchdog{transport = TPid, - status = T} - = S) -> - set_watchdog(S#watchdog{status = ?CHOOSE(initial == T, T, down), - pending = false, - transport = undefined}); + status = T, + restrict = {_,R}} + = S0) -> + S = S0#watchdog{pending = false, + transport = undefined}, + {{M,_}, _, _} = getr(restart), + + %% Close an accepting watchdog immediately if there's no + %% restriction on the number of connections to the same peer: the + %% state machine never enters state REOPEN in this case. The + %% 'close' message (instead of stop) is so as not to bypass the + %% sending of messages to the service process in handle_info/2. + + if T /= initial, M == accept, not R -> + send(self(), close), + S#watchdog{status = down}; + T /= initial -> + set_watchdog(S#watchdog{status = down}); + M == connect -> + set_watchdog(S); + M == accept -> + send(self(), close), + S + end; %% Incoming message. transition({recv, TPid, Name, Pkt}, #watchdog{transport = TPid} = S) -> @@ -454,9 +476,7 @@ encode(dwr = M, Dict0, Mask) -> hop_by_hop_id = Seq}, Pkt = #diameter_packet{header = Hdr, msg = Msg}, - #diameter_packet{bin = Bin} = diameter_codec:encode(Dict0, Pkt), - Bin; - + diameter_codec:encode(Dict0, Pkt); encode(dwa, Dict0, #diameter_packet{header = H, transport_data = TD} = ReqPkt) -> @@ -525,10 +545,14 @@ send_watchdog(#watchdog{pending = false, dictionary = Dict0, sequence = Mask} = S) -> - send(TPid, {send, encode(dwr, Dict0, Mask)}), + #diameter_packet{bin = Bin} = EPkt = encode(dwr, Dict0, Mask), + diameter_traffic:incr(send, EPkt, TPid, Dict0), + send(TPid, {send, Bin}), ?LOG(send, 'DWR'), S#watchdog{pending = true}. +%% Dont' count encode errors since we don't expect any on DWR/DWA. + %% recv/3 recv(Name, Pkt, S) -> @@ -545,13 +569,29 @@ recv(Name, Pkt, S) -> rcv('DWR', Pkt, #watchdog{transport = TPid, dictionary = Dict0}) -> - send(TPid, {send, encode(dwa, Dict0, Pkt)}), + ?LOG(recv, 'DWR'), + DPkt = diameter_codec:decode(Dict0, Pkt), + diameter_traffic:incr(recv, DPkt, TPid, Dict0), + diameter_traffic:incr_error(recv, DPkt, TPid, Dict0), + EPkt = encode(dwa, Dict0, Pkt), + diameter_traffic:incr(send, EPkt, TPid, Dict0), + diameter_traffic:incr_rc(send, EPkt, TPid, Dict0), + + send(TPid, {send, EPkt}), ?LOG(send, 'DWA'); +rcv('DWA', Pkt, #watchdog{transport = TPid, + dictionary = Dict0}) -> + ?LOG(recv, 'DWA'), + diameter_traffic:incr(recv, Pkt, TPid, Dict0), + diameter_traffic:incr_rc(recv, + diameter_codec:decode(Dict0, Pkt), + TPid, + Dict0); + rcv(N, _, _) when N == 'CER'; N == 'CEA'; - N == 'DWA'; N == 'DPR'; N == 'DPA' -> false; @@ -740,7 +780,7 @@ timeout(#watchdog{status = T} = S) restart(#watchdog{transport = undefined} = S) -> restart(getr(restart), S); -restart(S) -> +restart(S) -> %% reconnect has won race with timeout S. %% restart/2 @@ -770,9 +810,10 @@ restart({{connect, _} = T, Opts, Svc}, %% die. Note that a state machine never enters state REOPEN in this %% case. restart({{accept, _}, _, _}, #watchdog{restrict = {_, false}}) -> - stop; + stop; %% 'DOWN' was in old code: 'close' was not sent -%% Otherwise hang around until told to die. +%% Otherwise hang around until told to die, either by the service or +%% by another watchdog. restart({{accept, _}, _, _}, S) -> S. diff --git a/lib/diameter/src/compiler/diameter_codegen.erl b/lib/diameter/src/compiler/diameter_codegen.erl index 22422f2ef2..d91a776321 100644 --- a/lib/diameter/src/compiler/diameter_codegen.erl +++ b/lib/diameter/src/compiler/diameter_codegen.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2013. All Rights Reserved. +%% Copyright Ericsson AB 2010-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -31,7 +31,8 @@ %% on the beam file of another dictionary. %% --export([from_dict/4]). +-export([from_dict/4, + is_printable_ascii/1]). %% used by ?TERM/1 in diameter_forms.hrl -include("diameter_forms.hrl"). -include("diameter_vsn.hrl"). @@ -121,6 +122,9 @@ eraser(Key) -> %% =========================================================================== %% =========================================================================== +is_printable_ascii(C) -> + 16#20 =< C andalso C =< 16#7F. + get_value(Key, Plist) -> proplists:get_value(Key, Plist, []). @@ -128,7 +132,7 @@ gen(parse, ParseD, _Mod) -> [?VERSION | ParseD]; gen(forms, ParseD, Mod) -> - pp(erl_forms(Mod, ParseD)); + preprocess(Mod, erl_forms(Mod, ParseD)); gen(hrl, ParseD, Mod) -> gen_hrl(Mod, ParseD); @@ -834,19 +838,19 @@ rec_name(Name, Prefix) -> Prefix ++ Name. %% =========================================================================== -%% pp/1 +%% preprocess/2 %% %% Preprocess forms as generated by 'forms' option. In particular, %% replace the include_lib attributes in generated forms by the %% corresponding forms, extracting the latter from an existing %% dictionary (diameter_gen_relay). The resulting forms can be %% compiled to beam using compile:forms/2 (which does no preprocessing -%% or it's own; DiY currently appears to be the only way to preprocess +%% of it's own; DiY currently appears to be the only way to preprocess %% a forms list). -pp(Forms) -> +preprocess(Mod, Forms) -> {_, Beam, _} = code:get_object_code(diameter_gen_relay), - pp(Forms, abstract_code(Beam)). + pp(Forms, remod(Mod, abstract_code(Beam))). pp(Forms, {ok, Code}) -> Files = files(Code, []), @@ -855,6 +859,25 @@ pp(Forms, {ok, Code}) -> pp(Forms, {error, Reason}) -> erlang:error({forms, Reason, Forms}). +%% Replace literal diameter_gen_relay atoms in the extracted forms. +%% ?MODULE for example. + +remod(Mod, L) + when is_list(L) -> + [remod(Mod, T) || T <- L]; + +remod(Mod, {atom, _, diameter_gen_relay} = T) -> + setelement(3, T, Mod); + +remod(Mod, T) + when is_tuple(T) -> + list_to_tuple(remod(Mod, tuple_to_list(T))); + +remod(_, T) -> + T. + +%% Replace include_lib by the corresponding forms. + include({attribute, _, include_lib, Path}, Files) -> Inc = filename:basename(Path), [{Inc, Forms}] = [T || {F, _} = T <- Files, F == Inc], %% expect one @@ -863,6 +886,8 @@ include({attribute, _, include_lib, Path}, Files) -> include(T, _) -> [T]. +%% Extract abstract code. + abstract_code(Beam) -> case beam_lib:chunks(Beam, [abstract_code]) of {ok, {_Mod, [{abstract_code, {_Vsn, Code}}]}} -> @@ -873,6 +898,8 @@ abstract_code(Beam) -> {E, Reason} end. +%% Extract filename/forms pairs for included forms. + files([{attribute, _, file, {Path, _}} | T], Acc) -> {Body, Rest} = lists:splitwith(fun({attribute, _, file, _}) -> false; (_) -> true diff --git a/lib/diameter/src/compiler/diameter_dict_util.erl b/lib/diameter/src/compiler/diameter_dict_util.erl index 136bba16cb..cf4741e563 100644 --- a/lib/diameter/src/compiler/diameter_dict_util.erl +++ b/lib/diameter/src/compiler/diameter_dict_util.erl @@ -731,8 +731,8 @@ no_messages_without_id(Dict) -> %% explode/4 %% -%% {avp_vendor_id, AvpName} -> [Lineno, Id::integer()] -%% {custom_types|codecs|inherits, AvpName} -> [Lineno, Mod::string()] +%% {avp_vendor_id, AvpName} -> [Lineno, Id::integer()] +%% {custom|inherits, AvpName} -> [Lineno, Mod::string()] explode({_, Line, AvpName}, Dict, {_, _, X} = T, K) -> true = K /= avp_vendor_id orelse is_uint32(T, [K]), @@ -1094,7 +1094,7 @@ explode_avps([{_, Line, Name} | Toks], Dict) -> Vid = avp_vendor_id(Flags, Name, Line, Dict), %% An AVP is uniquely defined by its AVP code and vendor id (if any). - %% Ensure there are no duplicate. + %% Ensure there are no duplicates. store_new({avp_types, {Code, Vid}}, [Line, Name], Dict, @@ -1302,8 +1302,7 @@ x({K, {Name, AvpName}}, [Line | _], Dict) %% Ditto. x({K, AvpName}, [Line | _], Dict) when K == avp_vendor_id; - K == custom_types; - K == codecs -> + K == custom -> true = avp_is_defined(AvpName, Dict, Line); %% Ensure that all local AVP's of type Grouped are also present in @grouped. diff --git a/lib/diameter/src/compiler/diameter_forms.hrl b/lib/diameter/src/compiler/diameter_forms.hrl index 9b14c1715a..dd03401b9e 100644 --- a/lib/diameter/src/compiler/diameter_forms.hrl +++ b/lib/diameter/src/compiler/diameter_forms.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2013. All Rights Reserved. +%% Copyright Ericsson AB 2010-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -57,4 +57,6 @@ -define(FIELDS(Fs), [{?record_field, ?ATOM(F), V} || {F,V} <- Fs]). %% Literal term. --define(TERM(T), erl_parse:abstract(T, ?LINE)). +-define(TERM(T), erl_parse:abstract(T, [ + {line, ?LINE}, + {encoding, fun diameter_codegen:is_printable_ascii/1}])). diff --git a/lib/diameter/src/compiler/diameter_make.erl b/lib/diameter/src/compiler/diameter_make.erl index adc7808e49..72f5d36da4 100644 --- a/lib/diameter/src/compiler/diameter_make.erl +++ b/lib/diameter/src/compiler/diameter_make.erl @@ -232,21 +232,29 @@ identify([Vsn | [T|_] = ParseD]) identify({path, File} = T) -> {T, File}; identify(File) -> - Bin = iolist_to_binary([File]), - case is_path(Bin) of + case is_path([File]) of true -> {{path, File}, File}; - false -> {Bin, ?DEFAULT_DICT_FILE} + false -> {File, ?DEFAULT_DICT_FILE} end. -%% Interpret anything containing \n or \r as a literal dictionary, -%% otherwise a path. (Which might be the wrong guess in the worst case.) -is_path(Bin) -> - try - [throw(C) || <<C>> <= Bin, $\n == C orelse $\r == C], - true - catch - throw:_ -> false - end. +%% Interpret anything containing \n or \r as a literal dictionary. + +is_path([<<C,B/binary>> | T]) -> + is_path([C, B | T]); + +is_path([[C|L] | T]) -> + is_path([C, L | T]); + +is_path([C|_]) + when $\n == C; + $\r == C -> + false; + +is_path([_|T]) -> + is_path(T); + +is_path([]) -> + true. make(File, Opts, Dict) -> ok(lists:foldl(fun(M,A) -> [make(File, Opts, Dict, M) | A] end, diff --git a/lib/diameter/src/diameter.app.src b/lib/diameter/src/diameter.app.src index 509de9e595..ac1d847753 100644 --- a/lib/diameter/src/diameter.app.src +++ b/lib/diameter/src/diameter.app.src @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2013. All Rights Reserved. +%% Copyright Ericsson AB 2010-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -20,11 +20,27 @@ {application, diameter, [{description, "Diameter protocol"}, {vsn, "%VSN%"}, - {modules, [%MODULES%]}, + {modules, [ + %MODULES% + %,%COMPILER% + %,%INFO% + ]}, {registered, [%REGISTERED%]}, - {applications, [stdlib, kernel]}, + {applications, [ + {stdlib, "2.0"}, {kernel, "3.0"}%, {erts, "6.0"} + %% {syntax-tools, "1.6.14"} + %% {runtime-tools, "1.8.14"} + %, {ssl, "5.3.4"} + ]}, {env, []}, {mod, {diameter_app, []}}, - {runtime_dependencies, ["syntax_tools-1.6.14","stdlib-2.0","ssl-5.3.4", - "runtime_tools-1.8.14","kernel-3.0","erts-6.0"]} + {runtime_dependencies, [ + ]} + %% + %% Note that ssl is only required if configured on TCP transports, + %% and syntax-tools and runtime-tools are only required if the + %% dictionary compiler and debug modules (respectively) are + %% needed/wanted at runtime, which they typically aren't. These + %% modules are the two commented lines in the 'modules' tuple. + %% ]}. diff --git a/lib/diameter/src/diameter.appup.src b/lib/diameter/src/diameter.appup.src index 0d421c229e..40580e3ce6 100644 --- a/lib/diameter/src/diameter.appup.src +++ b/lib/diameter/src/diameter.appup.src @@ -34,7 +34,28 @@ {"1.4.2", [{restart_application, diameter}]}, %% R16B01 {"1.4.3", [{restart_application, diameter}]}, %% R16B02 {"1.4.4", [{restart_application, diameter}]}, - {"1.5", [{restart_application, diameter}]} %% R16B03 + {"1.5", [{restart_application, diameter}]}, %% R16B03 + {"1.6", [{load_module, diameter_lib}, %% 17.0 + {load_module, diameter_traffic}, + {load_module, diameter_watchdog}, + {load_module, diameter_peer_fsm}, + {load_module, diameter_service}, + {load_module, diameter_gen_base_rfc6733}, + {load_module, diameter_gen_acct_rfc6733}, + {load_module, diameter_gen_base_rfc3588}, + {load_module, diameter_gen_base_accounting}, + {load_module, diameter_gen_relay}, + {load_module, diameter_codec}, + {load_module, diameter_sctp}]}, + {"1.7", [{load_module, diameter_service}, %% 17.1 + {load_module, diameter_codec}, + {load_module, diameter_gen_base_rfc6733}, + {load_module, diameter_gen_acct_rfc6733}, + {load_module, diameter_gen_base_rfc3588}, + {load_module, diameter_gen_base_accounting}, + {load_module, diameter_gen_relay}, + {load_module, diameter_traffic}, + {load_module, diameter_peer_fsm}]} ], [ {"0.9", [{restart_application, diameter}]}, @@ -51,6 +72,27 @@ {"1.4.2", [{restart_application, diameter}]}, {"1.4.3", [{restart_application, diameter}]}, {"1.4.4", [{restart_application, diameter}]}, - {"1.5", [{restart_application, diameter}]} + {"1.5", [{restart_application, diameter}]}, + {"1.6", [{load_module, diameter_sctp}, + {load_module, diameter_codec}, + {load_module, diameter_gen_relay}, + {load_module, diameter_gen_base_accounting}, + {load_module, diameter_gen_base_rfc3588}, + {load_module, diameter_gen_acct_rfc6733}, + {load_module, diameter_gen_base_rfc6733}, + {load_module, diameter_service}, + {load_module, diameter_peer_fsm}, + {load_module, diameter_watchdog}, + {load_module, diameter_traffic}, + {load_module, diameter_lib}]}, + {"1.7", [{load_module, diameter_peer_fsm}, + {load_module, diameter_traffic}, + {load_module, diameter_gen_relay}, + {load_module, diameter_gen_base_accounting}, + {load_module, diameter_gen_base_rfc3588}, + {load_module, diameter_gen_acct_rfc6733}, + {load_module, diameter_gen_base_rfc6733}, + {load_module, diameter_codec}, + {load_module, diameter_service}]} ] }. diff --git a/lib/diameter/src/base/diameter_dbg.erl b/lib/diameter/src/info/diameter_dbg.erl index 5b0ac3a3b6..b536e5e80b 100644 --- a/lib/diameter/src/base/diameter_dbg.erl +++ b/lib/diameter/src/info/diameter_dbg.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2011. All Rights Reserved. +%% Copyright Ericsson AB 2010-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -17,12 +17,15 @@ %% %CopyrightEnd% %% +%% +%% Information and debug functions. +%% + -module(diameter_dbg). -export([table/1, tables/0, fields/1, - help/0, modules/0, versions/0, version_info/0, @@ -30,7 +33,7 @@ procs/0, latest/0, nl/0, - log/4]). + sizes/0]). -export([diameter_config/0, diameter_peer/0, @@ -52,11 +55,9 @@ tp/1]). -include_lib("diameter/include/diameter.hrl"). --include("diameter_internal.hrl"). - --define(INFO, diameter_info). --define(SEP(), ?INFO:sep()). +-define(APP, diameter). +-define(I, diameter_info). -define(LOCAL, [diameter_config, diameter_peer, @@ -68,27 +69,25 @@ -define(VALUES(Rec), tl(tuple_to_list(Rec))). -log(_Slogan, _Mod, _Line, _Details) -> - ok. - -%%% ---------------------------------------------------------- -%%% # help() -%%% ---------------------------------------------------------- +%% ---------------------------------------------------------- +%% # sizes/0 +%% +%% Return sizes of named tables. +%% ---------------------------------------------------------- -help() -> - not_yet_implemented. +sizes() -> + [{T, ets:info(T, size)} || T <- ?LOCAL, T /= diameter_peer]. -%%% ---------------------------------------------------------- -%%% # table(TableName) -%%% -%%% Input: TableName = diameter table containing record entries. -%%% -%%% Output: Count | undefined -%%% ---------------------------------------------------------- +%% ---------------------------------------------------------- +%% # table/1 +%% +%% Pretty-print a diameter table. Returns the number of records +%% printed, or undefined. +%% ---------------------------------------------------------- table(T) when (T == diameter_peer) orelse (T == diameter_reg) -> - ?INFO:format(collect(T), fields(T), fun ?INFO:split/2); + ?I:format(collect(T), fields(T), fun ?I:split/2); table(Table) when is_atom(Table) -> @@ -96,7 +95,7 @@ table(Table) undefined = No -> No; Fields -> - ?INFO:format(Table, Fields, fun split/2) + ?I:format(Table, Fields, fun split/2) end. split([started, name | Fs], [S, N | Vs]) -> @@ -107,9 +106,9 @@ split([[F|FT]|Fs], [Rec|Vs]) -> split([F|Fs], [V|Vs]) -> {F, Fs, V, Vs}. -%%% ---------------------------------------------------------- -%%% # TableName() -%%% ---------------------------------------------------------- +%% ---------------------------------------------------------- +%% # TableName/0 +%% ---------------------------------------------------------- -define(TABLE(Name), Name() -> table(Name)). @@ -121,16 +120,15 @@ split([F|Fs], [V|Vs]) -> ?TABLE(diameter_service). ?TABLE(diameter_stats). -%%% ---------------------------------------------------------- -%%% # tables() -%%% -%%% Output: Number of records output. -%%% -%%% Description: Pretty-print records in diameter tables from all nodes. -%%% ---------------------------------------------------------- +%% ---------------------------------------------------------- +%% # tables/0 +%% +%% Pretty-print diameter tables from all nodes. Returns the number of +%% records printed. +%% ---------------------------------------------------------- tables() -> - ?INFO:format(field(?LOCAL), fun split/3, fun collect/1). + ?I:format(field(?LOCAL), fun split/3, fun collect/1). field(Tables) -> lists:map(fun(T) -> {T, fields(T)} end, lists:sort(Tables)). @@ -138,66 +136,66 @@ field(Tables) -> split(_, Fs, Vs) -> split(Fs, Vs). -%%% ---------------------------------------------------------- -%%% # modules() -%%% ---------------------------------------------------------- +%% ---------------------------------------------------------- +%% # modules/0 +%% ---------------------------------------------------------- modules() -> - Path = filename:join([appdir(), atom_to_list(?APPLICATION) ++ ".app"]), - {ok, [{application, ?APPLICATION, Attrs}]} = file:consult(Path), + Path = filename:join([appdir(), atom_to_list(?APP) ++ ".app"]), + {ok, [{application, ?APP, Attrs}]} = file:consult(Path), {modules, Mods} = lists:keyfind(modules, 1, Attrs), Mods. appdir() -> - [_|_] = code:lib_dir(?APPLICATION, ebin). + [_|_] = code:lib_dir(?APP, ebin). -%%% ---------------------------------------------------------- -%%% # versions() -%%% ---------------------------------------------------------- +%% ---------------------------------------------------------- +%% # versions/0 +%% ---------------------------------------------------------- versions() -> - ?INFO:versions(modules()). + ?I:versions(modules()). -%%% ---------------------------------------------------------- -%%% # versions() -%%% ---------------------------------------------------------- +%% ---------------------------------------------------------- +%% # version_info/0 +%% ---------------------------------------------------------- version_info() -> - ?INFO:version_info(modules()). + ?I:version_info(modules()). -%%% ---------------------------------------------------------- -%%% # compiled() -%%% ---------------------------------------------------------- +%% ---------------------------------------------------------- +%% # compiled/0 +%% ---------------------------------------------------------- compiled() -> - ?INFO:compiled(modules()). + ?I:compiled(modules()). -%%% ---------------------------------------------------------- -%%% procs() -%%% ---------------------------------------------------------- +%% ---------------------------------------------------------- +%% # procs/0 +%% ---------------------------------------------------------- procs() -> - ?INFO:procs(?APPLICATION). + ?I:procs(?APP). -%%% ---------------------------------------------------------- -%%% # latest() -%%% ---------------------------------------------------------- +%% ---------------------------------------------------------- +%% # latest/0 +%% ---------------------------------------------------------- latest() -> - ?INFO:latest(modules()). + ?I:latest(modules()). -%%% ---------------------------------------------------------- -%%% # nl() -%%% ---------------------------------------------------------- +%% ---------------------------------------------------------- +%% # nl/0 +%% ---------------------------------------------------------- nl() -> lists:foreach(fun(M) -> abcast = c:nl(M) end, modules()). -%%% ---------------------------------------------------------- -%%% # pp(Bin) -%%% -%%% Description: Pretty-print a message binary. -%%% ---------------------------------------------------------- +%% ---------------------------------------------------------- +%% # pp/1 +%% +%% Description: Pretty-print a message binary. +%% ---------------------------------------------------------- %% Network byte order = big endian. @@ -207,7 +205,7 @@ pp(<<Version:8, MsgLength:24, HbHid:32, E2Eid:32, AVPs/binary>>) -> - ?SEP(), + ?I:sep(), ppp(["Version", "Message length", "[Actual length]", @@ -227,7 +225,7 @@ pp(<<Version:8, MsgLength:24, HbHid, E2Eid]), N = avp_loop({AVPs, MsgLength - 20}, 0), - ?SEP(), + ?I:sep(), N; pp(<<_Version:8, MsgLength:24, _/binary>> = Bin) -> @@ -328,23 +326,23 @@ ppp(Fields, Values) -> ppp({Field, Value}) -> io:format(": ~-22s : ~p~n", [Field, Value]). -%%% ---------------------------------------------------------- -%%% # subscriptions() -%%% -%%% Output: list of {SvcName, Pid} -%%% ---------------------------------------------------------- +%% ---------------------------------------------------------- +%% # subscriptions/0 +%% +%% Returns a list of {SvcName, Pid}. +%% ---------------------------------------------------------- subscriptions() -> diameter_service:subscriptions(). -%%% ---------------------------------------------------------- -%%% # children() -%%% ---------------------------------------------------------- +%% ---------------------------------------------------------- +%% # children/0 +%% ---------------------------------------------------------- children() -> diameter_sup:tree(). -%%% ---------------------------------------------------------- +%% ---------------------------------------------------------- %% tracer/[12] @@ -430,7 +428,7 @@ peers(Name, Ts) -> mk_peers(Name, [_, {type, connect} | _] = Ts) -> [[Name | mk_peer(Ts)]]; -mk_peers(Name, [R, {type, listen}, O, {accept = A, As}]) -> +mk_peers(Name, [R, {type, listen}, O, {accept = A, As} | _]) -> [[Name | mk_peer([R, {type, A}, O | Ts])] || Ts <- As]. %% This is a bit lame: service_info works to build this list and out %% of something like what we want here and then we take it apart. @@ -485,13 +483,12 @@ fields(diameter_service) -> [started, name, record_info(fields, diameter_service), + watchdogT, peerT, - connT, - share_peers, - use_shared_peers, shared_peers, local_peers, - monitor]; + monitor, + options]; ?FIELDS(diameter_event); ?FIELDS(diameter_uri); diff --git a/lib/diameter/src/base/diameter_info.erl b/lib/diameter/src/info/diameter_info.erl index 39d32d07cd..10972f3231 100644 --- a/lib/diameter/src/base/diameter_info.erl +++ b/lib/diameter/src/info/diameter_info.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2011. All Rights Reserved. +%% Copyright Ericsson AB 2010-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -17,6 +17,11 @@ %% %CopyrightEnd% %% +%% +%% Generic functions for formatting table listings and more. Used by +%% diameter_dbg. +%% + -module(diameter_info). -export([usage/1, @@ -573,12 +578,7 @@ sys_info() -> {A,V}. os_info() -> - {os:version(), case os:type() of - {_Fam, _Name} = T -> - T; - Fam -> - {Fam, ""} - end}. + {os:version(), os:type()}. chomp(S) -> string:strip(S, right, $\n). diff --git a/lib/diameter/src/modules.mk b/lib/diameter/src/modules.mk index f8d3cf1d6f..a2a7a51892 100644 --- a/lib/diameter/src/modules.mk +++ b/lib/diameter/src/modules.mk @@ -1,8 +1,7 @@ -#-*-makefile-*- ; force emacs to enter makefile-mode # %CopyrightBegin% # -# Copyright Ericsson AB 2010-2013. All Rights Reserved. +# Copyright Ericsson AB 2010-2014. All Rights Reserved. # # The contents of this file are subject to the Erlang Public License, # Version 1.1, (the "License"); you may not use this file except in @@ -64,16 +63,19 @@ RT_MODULES = \ transport/diameter_transport \ transport/diameter_transport_sup -# Handwritten (compile time) modules not included in the app file. +# Handwritten compiler modules not included in the app file. CT_MODULES = \ - base/diameter_dbg \ - base/diameter_info \ compiler/diameter_codegen \ compiler/diameter_exprecs \ compiler/diameter_dict_scanner \ compiler/diameter_dict_util \ compiler/diameter_make +# Info/debug modules, also not included in the app file. +INFO_MODULES = \ + info/diameter_dbg \ + info/diameter_info + # Released hrl files in ../include intended for public consumption. EXTERNAL_HRLS = \ diameter.hrl \ diff --git a/lib/diameter/src/transport/diameter_sctp.erl b/lib/diameter/src/transport/diameter_sctp.erl index d0a01351f3..32e7aaca39 100644 --- a/lib/diameter/src/transport/diameter_sctp.erl +++ b/lib/diameter/src/transport/diameter_sctp.erl @@ -616,6 +616,8 @@ send(#diameter_packet{bin = Bin, transport_data = {outstream, SId}}, S; %% ... or not: rotate through all streams. +send(#diameter_packet{bin = Bin}, S) -> + send(Bin, S); send(Bin, #transport{streams = {_, OS}, os = N} = S) diff --git a/lib/diameter/test/diameter_app_SUITE.erl b/lib/diameter/test/diameter_app_SUITE.erl index 1e262895a6..f68a18b5c2 100644 --- a/lib/diameter/test/diameter_app_SUITE.erl +++ b/lib/diameter/test/diameter_app_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2013. All Rights Reserved. +%% Copyright Ericsson AB 2010-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -50,7 +50,7 @@ diameter_exprecs, diameter_make]). --define(HELP_MODULES, [diameter_dbg, +-define(INFO_MODULES, [diameter_dbg, diameter_info]). %% =========================================================================== @@ -99,13 +99,13 @@ vsn(Config) -> %% # modules/1 %% %% Ensure that the app file modules and installed modules differ by -%% compiler/help modules. +%% compiler/info modules. %% =========================================================================== modules(Config) -> Mods = fetch(modules, fetch(app, Config)), Installed = code_mods(), - Help = lists:sort(?HELP_MODULES ++ ?COMPILER_MODULES), + Help = lists:sort(?INFO_MODULES ++ ?COMPILER_MODULES), {[], Help} = {Mods -- Installed, lists:sort(Installed -- Mods)}. @@ -158,12 +158,15 @@ appvsn(Name) -> %% # xref/1 %% %% Ensure that no function in our application calls an undefined function -%% or one in an application we haven't specified as a dependency. (Almost.) +%% or one in an application we haven't declared as a dependency. (Almost.) %% =========================================================================== xref(Config) -> App = fetch(app, Config), - Mods = fetch(modules, App), + Mods = fetch(modules, App), %% modules listed in the app file + + %% List of application names extracted from runtime_dependencies. + Deps = lists:map(fun unversion/1, fetch(runtime_dependencies, App)), {ok, XRef} = xref:start(make_name(xref_test_name)), ok = xref:set_default(XRef, [{verbose, false}, {warnings, false}]), @@ -178,7 +181,9 @@ xref(Config) -> [?APP, erts | fetch(applications, App)]), {ok, Undefs} = xref:analyze(XRef, undefined_function_calls), - {ok, Called} = xref:analyze(XRef, {module_call, ?COMPILER_MODULES}), + {ok, RTmods} = xref:analyze(XRef, {module_use, Mods}), + {ok, CTmods} = xref:analyze(XRef, {module_use, ?COMPILER_MODULES}), + {ok, RTdeps} = xref:analyze(XRef, {module_call, Mods}), xref:stop(XRef), @@ -190,18 +195,41 @@ xref(Config) -> Undefs), %% diameter_tcp does call ssl despite the latter not being listed %% as a dependency in the app file since ssl is only required for - %% TLS security: it's up to a client who wants TLS it to start - %% ssl. - - [] = lists:filter(fun is_bad_dependency/1, Called). - -%% It's not strictly necessary that diameter compiler modules not -%% depend on other diameter modules but it's a simple source of build -%% errors if not encoded in the makefile (hence the test) so guard -%% against it. -is_bad_dependency(Mod) -> - lists:prefix("diameter", atom_to_list(Mod)) - andalso not lists:member(Mod, ?COMPILER_MODULES). + %% TLS security: it's up to a client who wants TLS to start ssl. + + %% Ensure that only runtime or info modules call runtime modules. + %% It's not strictly necessary that diameter compiler modules not + %% depend on other diameter modules but it's a simple source of + %% build errors if not properly encoded in the makefile so guard + %% against it. + [] = (RTmods -- Mods) -- ?INFO_MODULES, + + %% Ensure that runtime modules don't call compiler modules. + CTmods = CTmods -- Mods, + + %% Ensure that runtime modules only call other runtime modules, or + %% applications declared as in runtime_dependencies in the app + %% file. Note that the declared application versions are ignored + %% since we only know what we can see now. + [] = lists:filter(fun(M) -> not lists:member(app(M), Deps) end, + RTdeps -- Mods). + +unversion(App) -> + T = lists:dropwhile(fun is_vsn_ch/1, lists:reverse(App)), + lists:reverse(case T of [$-|TT] -> TT; _ -> T end). + +is_vsn_ch(C) -> + $0 =< C andalso C =< $9 orelse $. == C. + +app('$M_EXPR') -> %% could be anything but assume it's ok + "erts"; +app(Mod) -> + case code:which(Mod) of + preloaded -> + "erts"; + Path -> + unversion(lists:nth(3, lists:reverse(filename:split(Path)))) + end. add_application(XRef, App) -> add_application(XRef, App, code:lib_dir(App)). diff --git a/lib/diameter/test/diameter_compiler_SUITE.erl b/lib/diameter/test/diameter_compiler_SUITE.erl index df4dde6240..20c9275808 100644 --- a/lib/diameter/test/diameter_compiler_SUITE.erl +++ b/lib/diameter/test/diameter_compiler_SUITE.erl @@ -120,6 +120,16 @@ {avp_has_duplicate_flag, " -", " MM"}, + {ok, + "@vendor 0", + "@vendor 10415"}, + {ok, + [{"@vendor 0", "@vendor 10415"}, + {"Proxy-Info .*M$", "&V"}, + {"Proxy-Info ::= [^>]*", "& 10415 "}]}, + {grouped_vendor_id_without_flag, + [{"@vendor 0", "@vendor 10415"}, + {"Proxy-Info ::= [^>]*", "& 10415 "}]}, {avp_has_vendor_id, "@avp_types", "@avp_vendor_id 667 Class\n&"}, @@ -307,6 +317,21 @@ {avp_not_defined, "CEA ::=", "<XXX> &"}, + {ok, + "@avp_types", + "@codecs tmod Session-Id &"}, + {ok, + "@avp_types", + "@custom_types tmod Session-Id &"}, + {avp_not_defined, + "@avp_types", + "@codecs tmod OctetString &"}, + {avp_not_defined, + "@avp_types", + "@custom_types tmod OctetString &"}, + {avp_already_defined, + "@avp_types", + "@codecs tmod Session-Id @custom_types tmod Session-Id &"}, {not_loaded, [{"@avp_types", "@inherits nomod XXX &"}, {"CEA ::=", "<XXX> &"}]}, diff --git a/lib/diameter/test/diameter_dpr_SUITE.erl b/lib/diameter/test/diameter_dpr_SUITE.erl index 9252650bf7..f3f16b06e0 100644 --- a/lib/diameter/test/diameter_dpr_SUITE.erl +++ b/lib/diameter/test/diameter_dpr_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2012. All Rights Reserved. +%% Copyright Ericsson AB 2012-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -73,7 +73,7 @@ %% Valid values for Disconnect-Cause. -define(CAUSES, [0, rebooting, 1, busy, 2, goaway]). -%% Establish one client connection for element of this list, +%% Establish one client connection for each element of this list, %% configured with disconnect/5 as disconnect_cb and returning the %% specified value. -define(RETURNS, @@ -129,8 +129,8 @@ stop_service(Config) -> service == group(Config) andalso (ok = diameter:stop_service(?CLIENT)). -%% Check for callbacks and stop the service. (Not the other way around -%% for the timing reason explained below.) +%% Check for callbacks before diameter:stop/0, not the other way around +%% for the timing reason explained below. check(Config) -> Grp = group(Config), [Pid | Refs] = ?util:read_priv(Config, config), diff --git a/lib/diameter/test/diameter_examples_SUITE.erl b/lib/diameter/test/diameter_examples_SUITE.erl index 02c8d34361..aef4bc35ef 100644 --- a/lib/diameter/test/diameter_examples_SUITE.erl +++ b/lib/diameter/test/diameter_examples_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2013. All Rights Reserved. +%% Copyright Ericsson AB 2013-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -24,7 +24,10 @@ -module(diameter_examples_SUITE). -export([suite/0, - all/0]). + all/0, + groups/0, + init_per_group/2, + end_per_group/2]). %% testcases -export([dict/1, dict/0, @@ -46,7 +49,7 @@ %% The order here is significant and causes the server to listen %% before the clients connect. --define(NODES, [compile, server, client]). +-define(NODES, [server, client]). %% Options to ct_slave:start/2. -define(TIMEOUTS, [{T, 15000} || T <- [boot_timeout, @@ -63,6 +66,9 @@ %% Common dictionaries to inherit from examples. -define(DICT0, [rfc3588_base, rfc6733_base]). +%% Transport protocols over which the example Diameter nodes are run. +-define(PROTS, [tcp, sctp]). + %% =========================================================================== suite() -> @@ -71,7 +77,34 @@ suite() -> all() -> [dict, code, - slave, + {group, all}]. + +groups() -> + Tc = tc(), + [{all, [parallel], [{group, P} || P <- ?PROTS]} + | [{P, [], Tc} || P <- ?PROTS]]. + +init_per_group(all, Config) -> + Config; + +init_per_group(tcp = N, Config) -> + [{group, N} | Config]; + +init_per_group(sctp = N, Config) -> + case gen_sctp:open() of + {ok, Sock} -> + gen_sctp:close(Sock), + [{group, N} | Config]; + {error, E} when E == eprotonosupport; + E == esocktnosupport -> %% fail on any other reason + {skip, no_sctp} + end. + +end_per_group(_, _) -> + ok. + +tc() -> + [slave, enslave, start, traffic, @@ -88,7 +121,7 @@ dict() -> dict(_Config) -> Dirs = [filename:join(H ++ ["examples", "dict"]) || H <- [[code:lib_dir(diameter)], [here(), ".."]]], - [] = [{F,D,RC} || {_,F} <- sort(find_files(Dirs, ".*\\.dia")), + [] = [{F,D,RC} || {_,F} <- sort(find_files(Dirs, ".*\\.dia$")), D <- ?DICT0, RC <- [make(F,D)], RC /= ok]. @@ -184,17 +217,18 @@ make_name(Dict) -> %% Compile example code under examples/code. code(Config) -> - Node = slave(hd(?NODES), here()), + Node = slave(compile, here()), [] = rpc:call(Node, ?MODULE, install, - [proplists:get_value(priv_dir, Config)]). + [proplists:get_value(priv_dir, Config)]), + {ok, Node} = ct_slave:stop(compile). %% Compile on another node since the code path may be modified. install(PrivDir) -> Top = install(here(), PrivDir), Src = filename:join([Top, "examples", "code"]), - Files = find_files([Src], ".*\\.erl"), + Files = find_files([Src], ".*\\.erl$"), [] = [{F,E} || {_,F} <- Files, {error, _, _} = E <- [compile:file(F, [warnings_as_errors, return_errors])]]. @@ -226,7 +260,7 @@ install(Dir, PrivDir) -> Inc = filename:join([Top, "include"]), Gen = filename:join([Top, "src", "gen"]), - Files = find_files([Inc, Gen], ".*\\.hrl"), + Files = find_files([Inc, Gen], ".*\\.hrl$"), [] = [{F,E} || {_,F} <- Files, B <- [filename:basename(F)], D <- [filename:join([TmpInc, B])], @@ -280,9 +314,10 @@ now_diff(_) -> %% Start two nodes: one for the server, one for the client. enslave(Config) -> + Prot = proplists:get_value(group, Config), Dir = here(), - Nodes = [{N, slave(N, Dir)} || N <- tl(?NODES)], - ?util:write_priv(Config, nodes, Nodes). + Nodes = [{S, slave(N, Dir)} || S <- ?NODES, N <- [concat(Prot, S)]], + ?util:write_priv(Config, Prot, Nodes). slave(Name, Dir) -> {ok, Node} = ct_slave:start(Name, ?TIMEOUTS), @@ -292,6 +327,9 @@ slave(Name, Dir) -> [[Dir, filename:join([Dir, "..", "ebin"])]]), Node. +concat(Prot, Svc) -> + list_to_atom(atom_to_list(Prot) ++ atom_to_list(Svc)). + here() -> filename:dirname(code:which(?MODULE)). @@ -304,24 +342,25 @@ top(Dir, LibDir) -> %% start/1 -start(server) -> +start({server, Prot}) -> ok = diameter:start(), ok = server:start(), - {ok, Ref} = server:listen(tcp), - [_] = ?util:lport(tcp, Ref), + {ok, Ref} = server:listen(Prot), + [_] = ?util:lport(Prot, Ref), ok; -start(client) -> +start({client = Svc, Prot}) -> ok = diameter:start(), - true = diameter:subscribe(client), + true = diameter:subscribe(Svc), ok = client:start(), - {ok, Ref} = client:connect(tcp), + {ok, Ref} = client:connect(Prot), receive #diameter_event{info = {up, Ref, _, _, _}} -> ok end; start(Config) -> - Nodes = ?util:read_priv(Config, nodes), + Prot = proplists:get_value(group, Config), + Nodes = ?util:read_priv(Config, Prot), [] = [RC || {T,N} <- Nodes, - RC <- [rpc:call(N, ?MODULE, start, [T])], + RC <- [rpc:call(N, ?MODULE, start, [{T, Prot}])], RC /= ok]. %% traffic/1 @@ -336,7 +375,8 @@ traffic(client) -> receive {'DOWN', MRef, process, _, Reason} -> Reason end; traffic(Config) -> - Nodes = ?util:read_priv(Config, nodes), + Prot = proplists:get_value(group, Config), + Nodes = ?util:read_priv(Config, Prot), [] = [RC || {T,N} <- Nodes, RC <- [rpc:call(N, ?MODULE, traffic, [T])], RC /= ok]. @@ -355,5 +395,6 @@ stop(Name) {ok, _Node} = ct_slave:stop(Name), ok; -stop(_Config) -> - [] = [RC || N <- ?NODES, RC <- [stop(N)], RC /= ok]. +stop(Config) -> + Prot = proplists:get_value(group, Config), + [] = [RC || N <- ?NODES, RC <- [stop(concat(Prot, N))], RC /= ok]. diff --git a/lib/diameter/test/diameter_failover_SUITE.erl b/lib/diameter/test/diameter_failover_SUITE.erl index dfd3253827..c1494dcdb1 100644 --- a/lib/diameter/test/diameter_failover_SUITE.erl +++ b/lib/diameter/test/diameter_failover_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2013. All Rights Reserved. +%% Copyright Ericsson AB 2010-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -47,6 +47,7 @@ send_discard_1/1, send_discard_2/1, stop_services/1, + empty/1, stop/1]). %% diameter callbacks @@ -121,6 +122,7 @@ all() -> send_discard_1, send_discard_2, stop_services, + empty, stop]. %% =========================================================================== @@ -147,6 +149,10 @@ stop_services(_Config) -> T <- [diameter:stop_service(H)], T /= ok]. +%% Ensure transports have been removed from request table. +empty(_Config) -> + [] = ets:tab2list(diameter_request). + stop(_Config) -> ok = diameter:stop(). diff --git a/lib/diameter/test/diameter_traffic_SUITE.erl b/lib/diameter/test/diameter_traffic_SUITE.erl index a97c54fc04..4b67372016 100644 --- a/lib/diameter/test/diameter_traffic_SUITE.erl +++ b/lib/diameter/test/diameter_traffic_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2013. All Rights Reserved. +%% Copyright Ericsson AB 2010-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -43,7 +43,9 @@ send_protocol_error/1, send_arbitrary/1, send_unknown/1, + send_unknown_short/1, send_unknown_mandatory/1, + send_unknown_short_mandatory/1, send_noreply/1, send_unsupported/1, send_unsupported_app/1, @@ -54,7 +56,8 @@ send_zero_avp_length/1, send_invalid_avp_length/1, send_invalid_reject/1, - send_unrecognized_mandatory/1, + send_unexpected_mandatory_decode/1, + send_unexpected_mandatory/1, send_long/1, send_nopeer/1, send_noapp/1, @@ -266,7 +269,9 @@ tc() -> send_protocol_error, send_arbitrary, send_unknown, + send_unknown_short, send_unknown_mandatory, + send_unknown_short_mandatory, send_noreply, send_unsupported, send_unsupported_app, @@ -277,7 +282,8 @@ tc() -> send_zero_avp_length, send_invalid_avp_length, send_invalid_reject, - send_unrecognized_mandatory, + send_unexpected_mandatory_decode, + send_unexpected_mandatory, send_long, send_nopeer, send_noapp, @@ -447,6 +453,24 @@ send_unknown(Config) -> data = <<17>>}]} = lists:last(Avps). +%% Ditto, and point the AVP length past the end of the message. Expect +%% 5014. +send_unknown_short(Config) -> + send_unknown_short(Config, false, ?INVALID_AVP_LENGTH). + +send_unknown_short(Config, M, RC) -> + Req = ['ASR', {'AVP', [#diameter_avp{code = 999, + is_mandatory = M, + data = <<17>>}]}], + ['ASA', _SessionId, {'Result-Code', RC} | Avps] + = call(Config, Req), + [#'diameter_base_Failed-AVP'{'AVP' = As}] + = proplists:get_value('Failed-AVP', Avps), + [#diameter_avp{code = 999, + is_mandatory = M, + data = <<17, _/binary>>}] %% extra bits from padding + = As. + %% Ditto but set the M flag. send_unknown_mandatory(Config) -> Req = ['ASR', {'AVP', [#diameter_avp{code = 999, @@ -461,6 +485,27 @@ send_unknown_mandatory(Config) -> data = <<17>>}] = As. +%% Ditto, and point the AVP length past the end of the message. Expect +%% 5014 instead of 5001. +send_unknown_short_mandatory(Config) -> + send_unknown_short(Config, true, ?INVALID_AVP_LENGTH). + +%% Send an ACR containing an unexpected mandatory Session-Timeout. +%% Expect 5001, and check that the value in Failed-AVP was decoded. +send_unexpected_mandatory_decode(Config) -> + Req = ['ASR', {'AVP', [#diameter_avp{code = 27, %% Session-Timeout + is_mandatory = true, + data = <<12:32>>}]}], + ['ASA', _SessionId, {'Result-Code', ?AVP_UNSUPPORTED} | Avps] + = call(Config, Req), + [#'diameter_base_Failed-AVP'{'AVP' = As}] + = proplists:get_value('Failed-AVP', Avps), + [#diameter_avp{code = 27, + is_mandatory = true, + value = 12, + data = <<12:32>>}] + = As. + %% Send an STR that the server ignores. send_noreply(Config) -> Req = ['STR', {'Termination-Cause', ?BAD_ANSWER}], @@ -527,9 +572,9 @@ send_invalid_reject(Config) -> ?answer_message(?TOO_BUSY) = call(Config, Req). -%% Send an STR containing a known AVP, but one that's not allowed and -%% sets the M-bit. -send_unrecognized_mandatory(Config) -> +%% Send an STR containing a known AVP, but one that's not expected and +%% that sets the M-bit. +send_unexpected_mandatory(Config) -> Req = ['STR', {'Termination-Cause', ?LOGOUT}], ['STA', _SessionId, {'Result-Code', ?AVP_UNSUPPORTED} | _] @@ -836,6 +881,26 @@ log(#diameter_packet{bin = Bin} = P, T) %% prepare/4 prepare(Pkt, Caps, N, #group{client_dict0 = Dict0} = Group) + when N == send_unknown_short_mandatory; + N == send_unknown_short -> + Req = prepare(Pkt, Caps, Group), + + #diameter_packet{header = #diameter_header{length = L}, + bin = Bin} + = E + = diameter_codec:encode(Dict0, Pkt#diameter_packet{msg = Req}), + + %% Find the unknown AVP data at the end of the message and alter + %% its length header. + + {Padding, [17|_]} = lists:splitwith(fun(C) -> C == 0 end, + lists:reverse(binary_to_list(Bin))), + + Offset = L - length(Padding) - 4, + <<H:Offset/binary, Len:24, T/binary>> = Bin, + E#diameter_packet{bin = <<H/binary, (Len+9):24, T/binary>>}; + +prepare(Pkt, Caps, N, #group{client_dict0 = Dict0} = Group) when N == send_long_avp_length; N == send_short_avp_length; N == send_zero_avp_length -> @@ -876,8 +941,8 @@ prepare(Pkt, Caps, N, #group{client_dict0 = Dict0} = Group) <<V, L:24, H/binary>> = H0, %% assert E#diameter_packet{bin = <<V, (L+4):24, H/binary, 16:24, 0:32, T/binary>>}; -prepare(Pkt, Caps, send_unrecognized_mandatory, #group{client_dict0 = Dict0} - = Group) -> +prepare(Pkt, Caps, send_unexpected_mandatory, #group{client_dict0 = Dict0} + = Group) -> Req = prepare(Pkt, Caps, Group), #diameter_packet{bin = <<V, Len:24, T/binary>>} = E @@ -997,7 +1062,9 @@ answer(Rec, [_|_], N) N == send_short_avp_length; N == send_zero_avp_length; N == send_invalid_avp_length; - N == send_invalid_reject -> + N == send_invalid_reject; + N == send_unknown_short_mandatory; + N == send_unexpected_mandatory_decode -> Rec; answer(Rec, [], _) -> Rec. diff --git a/lib/diameter/vsn.mk b/lib/diameter/vsn.mk index 54019fa46c..4e54e4eafc 100644 --- a/lib/diameter/vsn.mk +++ b/lib/diameter/vsn.mk @@ -18,5 +18,5 @@ # %CopyrightEnd% APPLICATION = diameter -DIAMETER_VSN = 1.6 +DIAMETER_VSN = 1.7.1 APP_VSN = $(APPLICATION)-$(DIAMETER_VSN)$(PRE_VSN) diff --git a/lib/edoc/doc/src/notes.xml b/lib/edoc/doc/src/notes.xml index 394e7af09c..52b7529f70 100644 --- a/lib/edoc/doc/src/notes.xml +++ b/lib/edoc/doc/src/notes.xml @@ -31,6 +31,70 @@ <p>This document describes the changes made to the EDoc application.</p> +<section><title>Edoc 0.7.15</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fix spec to doc generation from erl_docgen and edoc for + maps</p> + <p> + Own Id: OTP-12058</p> + </item> + </list> + </section> + +</section> + +<section><title>Edoc 0.7.14</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> The default encoding for Erlang source files is now + UTF-8. As a temporary measure to ease the transition from + the old default of Latin-1, if EDoc encounters byte + sequences that are not valid UTF-8 sequences, EDoc will + re-try in Latin-1 mode. This workaround will be removed + in a future release. </p> + <p> + Own Id: OTP-12008</p> + </item> + </list> + </section> + +</section> + +<section><title>Edoc 0.7.13</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Application upgrade (appup) files are corrected for the + following applications: </p> + <p> + <c>asn1, common_test, compiler, crypto, debugger, + dialyzer, edoc, eldap, erl_docgen, et, eunit, gs, hipe, + inets, observer, odbc, os_mon, otp_mibs, parsetools, + percept, public_key, reltool, runtime_tools, ssh, + syntax_tools, test_server, tools, typer, webtool, wx, + xmerl</c></p> + <p> + A new test utility for testing appup files is added to + test_server. This is now used by most applications in + OTP.</p> + <p> + (Thanks to Tobias Schlager)</p> + <p> + Own Id: OTP-11744</p> + </item> + </list> + </section> + +</section> + <section><title>Edoc 0.7.12.1</title> <section><title>Improvements and New Features</title> diff --git a/lib/edoc/src/edoc.erl b/lib/edoc/src/edoc.erl index a87a8471e3..983f04e8b6 100644 --- a/lib/edoc/src/edoc.erl +++ b/lib/edoc/src/edoc.erl @@ -696,15 +696,44 @@ read_source_2(Name, Opts) -> %% The line of the dot token will be copied to the integer token. parse_file(Name, Includes, Macros) -> - case epp:open(Name, Includes, Macros) of - {ok, Epp} -> - try {ok, parse_file(Epp)} + case parse_file(utf8, Name, Includes, Macros) of + invalid_unicode -> + parse_file(latin1, Name, Includes, Macros); + Ret -> + Ret + end. + +parse_file(DefEncoding, Name, Includes, Macros) -> + Options = [{name, Name}, + {includes, Includes}, + {macros, Macros}, + {default_encoding, DefEncoding}], + case epp:open([extra | Options]) of + {ok, Epp, Extra} -> + try parse_file(Epp) of + Forms -> + Encoding = proplists:get_value(encoding, Extra), + case find_invalid_unicode(Forms) of + invalid_unicode when Encoding =/= utf8 -> + invalid_unicode; + _ -> + {ok, Forms} + end after _ = epp:close(Epp) end; Error -> Error end. +find_invalid_unicode([H|T]) -> + case H of + {error,{_Line,file_io_server,invalid_unicode}} -> + invalid_unicode; + _Other -> + find_invalid_unicode(T) + end; +find_invalid_unicode([]) -> none. + parse_file(Epp) -> case scan_and_parse(Epp) of {ok, Form} -> diff --git a/lib/edoc/src/edoc_layout.erl b/lib/edoc/src/edoc_layout.erl index e164ff060f..a102d432bc 100644 --- a/lib/edoc/src/edoc_layout.erl +++ b/lib/edoc/src/edoc_layout.erl @@ -831,8 +831,6 @@ t_type([#xmlElement{name = nonempty_list, content = Es}]) -> t_nonempty_list(Es); t_type([#xmlElement{name = map, content = Es}]) -> t_map(Es); -t_type([#xmlElement{name = map_field, content=Es}]) -> - t_map_field(Es); t_type([#xmlElement{name = tuple, content = Es}]) -> t_tuple(Es); t_type([#xmlElement{name = 'fun', content = Es}]) -> @@ -882,10 +880,11 @@ t_fun(Es) -> [") -> "] ++ t_utype(get_elem(type, Es))). t_map(Es) -> - ["#{"] ++ seq(fun t_utype_elem/1, Es, ["}"]). + Fs = get_elem(map_field, Es), + ["#{"] ++ seq(fun t_map_field/1, Fs, ["}"]). -t_map_field([K,V]) -> - [t_utype_elem(K) ++ " => " ++ t_utype_elem(V)]. +t_map_field(#xmlElement{content = [K,V]}) -> + t_utype_elem(K) ++ [" => "] ++ t_utype_elem(V). t_record(E, Es) -> Name = ["#"] ++ t_type(get_elem(atom, Es)), @@ -1082,6 +1081,8 @@ ot_type([#xmlElement{name = nonempty_list, content = Es}]) -> ot_nonempty_list(Es); ot_type([#xmlElement{name = tuple, content = Es}]) -> ot_tuple(Es); +ot_type([#xmlElement{name = map, content = Es}]) -> + ot_map(Es); ot_type([#xmlElement{name = 'fun', content = Es}]) -> ot_fun(Es); ot_type([#xmlElement{name = record, content = Es}]) -> @@ -1138,6 +1139,12 @@ ot_nonempty_list(Es) -> ot_tuple(Es) -> {type,0,tuple,[ot_utype_elem(E) || E <- Es]}. +ot_map(Es) -> + {type,0,map,[ot_map_field(E) || E <- get_elem(map_field,Es)]}. + +ot_map_field(#xmlElement{content=[K,V]}) -> + {type,0,map_field_assoc,ot_utype_elem(K), ot_utype_elem(V)}. + ot_fun(Es) -> Range = ot_utype(get_elem(type, Es)), Args = [ot_utype_elem(A) || A <- get_content(argtypes, Es)], diff --git a/lib/edoc/src/edoc_types.erl b/lib/edoc/src/edoc_types.erl index af8f1230fb..8a6c8eb33e 100644 --- a/lib/edoc/src/edoc_types.erl +++ b/lib/edoc/src/edoc_types.erl @@ -66,6 +66,7 @@ is_new_predefined(boolean, 0) -> true; is_new_predefined(byte, 0) -> true; is_new_predefined(iodata, 0) -> true; is_new_predefined(iolist, 0) -> true; +is_new_predefined(map, 0) -> true; is_new_predefined(maybe_improper_list, 0) -> true; is_new_predefined(maybe_improper_list, 2) -> true; is_new_predefined(mfa, 0) -> true; @@ -142,7 +143,7 @@ to_xml(#t_fun{args = As, range = T}, Env) -> {'fun', [{argtypes, map(fun wrap_utype/2, As, Env)}, wrap_utype(T, Env)]}; to_xml(#t_map{ types = Ts}, Env) -> - {map, map(fun wrap_utype/2, Ts, Env)}; + {map, map(fun to_xml/2, Ts, Env)}; to_xml(#t_map_field{ k_type=K, v_type=V}, Env) -> {map_field, [wrap_utype(K,Env), wrap_utype(V, Env)]}; to_xml(#t_tuple{types = Ts}, Env) -> diff --git a/lib/edoc/test/edoc_SUITE.erl b/lib/edoc/test/edoc_SUITE.erl index c9c7811afb..c63660c8c0 100644 --- a/lib/edoc/test/edoc_SUITE.erl +++ b/lib/edoc/test/edoc_SUITE.erl @@ -22,12 +22,12 @@ init_per_group/2,end_per_group/2]). %% Test cases --export([app/1,appup/1,build_std/1,build_map_module/1]). +-export([app/1,appup/1,build_std/1,build_map_module/1,otp_12008/1]). suite() -> [{ct_hooks,[ts_install_cth]}]. all() -> - [app,appup,build_std,build_map_module]. + [app,appup,build_std,build_map_module,otp_12008]. groups() -> []. @@ -77,3 +77,21 @@ build_map_module(Config) when is_list(Config) -> Filename = filename:join(DataDir, "map_module.erl"), ok = edoc:file(Filename, [{dir, PrivDir}]), ok. + +otp_12008(Config) when is_list(Config) -> + DataDir = ?config(data_dir, Config), + PrivDir = ?config(priv_dir, Config), + Un1 = filename:join(DataDir, "un1.erl"), + Un2 = filename:join(DataDir, "un2.erl"), + Un3 = filename:join(DataDir, "un3.erl"), + %% epp_dodger + Opts1 = [{dir, PrivDir}], + ok = edoc:files([Un1], Opts1), + ok = edoc:files([Un2], Opts1), + {'EXIT', error} = (catch edoc:files([Un3], Opts1)), + %% epp + Opts2 = [{preprocess, true}, {dir, PrivDir}], + ok = edoc:files([Un1], Opts2), + ok = edoc:files([Un2], Opts2), + {'EXIT', error} = (catch edoc:files([Un3], Opts2)), + ok. diff --git a/lib/edoc/test/edoc_SUITE_data/map_module.erl b/lib/edoc/test/edoc_SUITE_data/map_module.erl index 94ee7e6f26..f242721637 100644 --- a/lib/edoc/test/edoc_SUITE_data/map_module.erl +++ b/lib/edoc/test/edoc_SUITE_data/map_module.erl @@ -1,6 +1,6 @@ -module(map_module). --export([foo1/1,foo2/3]). +-export([foo1/1,foo2/3,start_child/2]). %% @type wazzup() = integer() %% @type some_type() = map() @@ -25,3 +25,43 @@ foo1(#{ a:= 1, b := V}) -> V. Map :: #{ get => 'value', 'value' => binary()}) -> binary(). foo2(Type1, {a,#{ "a" := _}}, #{get := value, value := B}) when is_map(Type1) -> B. + +%% from supervisor 18.0 + +-type child() :: 'undefined' | pid(). +-type child_id() :: term(). +-type mfargs() :: {M :: module(), F :: atom(), A :: [term()] | undefined}. +-type modules() :: [module()] | 'dynamic'. +-type restart() :: 'permanent' | 'transient' | 'temporary'. +-type shutdown() :: 'brutal_kill' | timeout(). +-type worker() :: 'worker' | 'supervisor'. +-type sup_ref() :: (Name :: atom()) + | {Name :: atom(), Node :: node()} + | {'global', Name :: atom()} + | {'via', Module :: module(), Name :: any()} + | pid(). +-type child_spec() :: #{name => child_id(), % mandatory + start => mfargs(), % mandatory + restart => restart(), % optional + shutdown => shutdown(), % optional + type => worker(), % optional + modules => modules()} % optional + | {Id :: child_id(), + StartFunc :: mfargs(), + Restart :: restart(), + Shutdown :: shutdown(), + Type :: worker(), + Modules :: modules()}. + +-type startchild_err() :: 'already_present' + | {'already_started', Child :: child()} | term(). +-type startchild_ret() :: {'ok', Child :: child()} + | {'ok', Child :: child(), Info :: term()} + | {'error', startchild_err()}. + + +-spec start_child(SupRef, ChildSpec) -> startchild_ret() when + SupRef :: sup_ref(), + ChildSpec :: child_spec() | (List :: [term()]). +start_child(Supervisor, ChildSpec) -> + {Supervisor,ChildSpec}. diff --git a/lib/edoc/test/edoc_SUITE_data/un1.erl b/lib/edoc/test/edoc_SUITE_data/un1.erl new file mode 100644 index 0000000000..0c48e7f940 --- /dev/null +++ b/lib/edoc/test/edoc_SUITE_data/un1.erl @@ -0,0 +1,7 @@ +-module(un1). + +-export([t/0]). + +%% @doc F�pp +t() -> + �rlig. diff --git a/lib/edoc/test/edoc_SUITE_data/un2.erl b/lib/edoc/test/edoc_SUITE_data/un2.erl new file mode 100644 index 0000000000..a6d13f4723 --- /dev/null +++ b/lib/edoc/test/edoc_SUITE_data/un2.erl @@ -0,0 +1,8 @@ +-module(un2). +%% coding: latin-1 + +-export([t/0]). + +%% @doc F�pp +t() -> + �rlig. diff --git a/lib/edoc/test/edoc_SUITE_data/un3.erl b/lib/edoc/test/edoc_SUITE_data/un3.erl new file mode 100644 index 0000000000..fbe9591dce --- /dev/null +++ b/lib/edoc/test/edoc_SUITE_data/un3.erl @@ -0,0 +1,8 @@ +-module(un3). +%% coding: utf-8 + +-export([t/0]). + +%% @doc F�pp +t() -> + �rlig. diff --git a/lib/edoc/vsn.mk b/lib/edoc/vsn.mk index 0172aac48b..b1cf115b29 100644 --- a/lib/edoc/vsn.mk +++ b/lib/edoc/vsn.mk @@ -1 +1 @@ -EDOC_VSN = 0.7.13 +EDOC_VSN = 0.7.15 diff --git a/lib/eldap/doc/src/notes.xml b/lib/eldap/doc/src/notes.xml index e7a3e20202..089bb731d4 100644 --- a/lib/eldap/doc/src/notes.xml +++ b/lib/eldap/doc/src/notes.xml @@ -30,7 +30,51 @@ </header> <p>This document describes the changes made to the Eldap application.</p> - <section><title>Eldap 1.0.2</title> + <section><title>Eldap 1.0.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Application upgrade (appup) files are corrected for the + following applications: </p> + <p> + <c>asn1, common_test, compiler, crypto, debugger, + dialyzer, edoc, eldap, erl_docgen, et, eunit, gs, hipe, + inets, observer, odbc, os_mon, otp_mibs, parsetools, + percept, public_key, reltool, runtime_tools, ssh, + syntax_tools, test_server, tools, typer, webtool, wx, + xmerl</c></p> + <p> + A new test utility for testing appup files is added to + test_server. This is now used by most applications in + OTP.</p> + <p> + (Thanks to Tobias Schlager)</p> + <p> + Own Id: OTP-11744</p> + </item> + <item> + <p> + Add support for IPv6 connections, By including the + [inet6] option in eldap:open/2. Default value is still + [inet] (Thanks to Edwin Fine)</p> + <p> + Own Id: OTP-11753</p> + </item> + <item> + <p> + Fixed bug where eldap:search returned binaries instead of + strings. (Thanks Simon MacMullen for the report)</p> + <p> + Own Id: OTP-11768</p> + </item> + </list> + </section> + +</section> + +<section><title>Eldap 1.0.2</title> <section><title>Fixed Bugs and Malfunctions</title> <list> diff --git a/lib/eldap/test/README b/lib/eldap/test/README index 8774db1504..ec774c1ae3 100644 --- a/lib/eldap/test/README +++ b/lib/eldap/test/README @@ -19,7 +19,7 @@ This will however not work, since slapd is guarded by apparmor that checks that To make a local extension of alowed operations: sudo emacs /etc/apparmor.d/local/usr.sbin.slapd -and, after the change (yes, at least on Ubuntu it is right to edit ../local/.. but run with an other file) : +and, after the change (yes, at least on Ubuntu it is right to edit ../local/.. but run with another file): sudo apparmor_parser -r /etc/apparmor.d/usr.sbin.slapd diff --git a/lib/erl_docgen/doc/src/notes.xml b/lib/erl_docgen/doc/src/notes.xml index 24d12b2bb7..f194fb6d6c 100644 --- a/lib/erl_docgen/doc/src/notes.xml +++ b/lib/erl_docgen/doc/src/notes.xml @@ -30,7 +30,64 @@ </header> <p>This document describes the changes made to the <em>erl_docgen</em> application.</p> - <section><title>Erl_Docgen 0.3.4.1</title> + <section><title>Erl_Docgen 0.3.6</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fix spec to doc generation from erl_docgen and edoc for + maps</p> + <p> + Own Id: OTP-12058</p> + </item> + </list> + </section> + +</section> + +<section><title>Erl_Docgen 0.3.5</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Application upgrade (appup) files are corrected for the + following applications: </p> + <p> + <c>asn1, common_test, compiler, crypto, debugger, + dialyzer, edoc, eldap, erl_docgen, et, eunit, gs, hipe, + inets, observer, odbc, os_mon, otp_mibs, parsetools, + percept, public_key, reltool, runtime_tools, ssh, + syntax_tools, test_server, tools, typer, webtool, wx, + xmerl</c></p> + <p> + A new test utility for testing appup files is added to + test_server. This is now used by most applications in + OTP.</p> + <p> + (Thanks to Tobias Schlager)</p> + <p> + Own Id: OTP-11744</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Handle map types in docgen_edoc_xml_cb</p> + <p> + Own Id: OTP-11776</p> + </item> + </list> + </section> + +</section> + +<section><title>Erl_Docgen 0.3.4.1</title> <section><title>Improvements and New Features</title> <list> diff --git a/lib/erl_docgen/src/docgen_otp_specs.erl b/lib/erl_docgen/src/docgen_otp_specs.erl index 886194598f..1075c47801 100644 --- a/lib/erl_docgen/src/docgen_otp_specs.erl +++ b/lib/erl_docgen/src/docgen_otp_specs.erl @@ -388,8 +388,8 @@ t_type([#xmlElement{name = nonempty_list, content = Es}]) -> t_nonempty_list(Es); t_type([#xmlElement{name = tuple, content = Es}]) -> t_tuple(Es); -t_type([#xmlElement{name = map}]) -> - t_map(); +t_type([#xmlElement{name = map, content = Es}]) -> + t_map(Es); t_type([#xmlElement{name = 'fun', content = Es}]) -> ["fun("] ++ t_fun(Es) ++ [")"]; t_type([E = #xmlElement{name = record, content = Es}]) -> @@ -432,8 +432,12 @@ t_nonempty_list(Es) -> t_tuple(Es) -> ["{"] ++ seq(fun t_utype_elem/1, Es, ["}"]). -t_map() -> - ["map()"]. +t_map(Es) -> + Fs = get_elem(map_field, Es), + ["#{"] ++ seq(fun t_map_field/1, Fs, ["}"]). + +t_map_field(#xmlElement{content = [K,V]}) -> + [t_utype_elem(K) ++ " => " ++ t_utype_elem(V)]. t_fun(Es) -> ["("] ++ seq(fun t_utype_elem/1, get_content(argtypes, Es), @@ -550,12 +554,14 @@ ot_type([#xmlElement{name = nonempty_list, content = Es}]) -> ot_nonempty_list(Es); ot_type([#xmlElement{name = tuple, content = Es}]) -> ot_tuple(Es); +ot_type([#xmlElement{name = map, content = Es}]) -> + ot_map(Es); ot_type([#xmlElement{name = 'fun', content = Es}]) -> ot_fun(Es); ot_type([#xmlElement{name = record, content = Es}]) -> ot_record(Es); ot_type([#xmlElement{name = abstype, content = Es}]) -> - ot_abstype(Es); + ot_abstype(Es); ot_type([#xmlElement{name = union, content = Es}]) -> ot_union(Es). @@ -606,6 +612,12 @@ ot_nonempty_list(Es) -> ot_tuple(Es) -> {type,0,tuple,[ot_utype_elem(E) || E <- Es]}. +ot_map(Es) -> + {type,0,map,[ot_map_field(E) || E <- get_elem(map_field,Es)]}. + +ot_map_field(#xmlElement{content=[K,V]}) -> + {type,0,map_field_assoc, ot_utype_elem(K), ot_utype_elem(V)}. + ot_fun(Es) -> Range = ot_utype(get_elem(type, Es)), Args = [ot_utype_elem(A) || A <- get_content(argtypes, Es)], diff --git a/lib/erl_docgen/vsn.mk b/lib/erl_docgen/vsn.mk index 0f89922275..8bfcc08698 100644 --- a/lib/erl_docgen/vsn.mk +++ b/lib/erl_docgen/vsn.mk @@ -1 +1 @@ -ERL_DOCGEN_VSN = 0.3.5 +ERL_DOCGEN_VSN = 0.3.6 diff --git a/lib/erl_interface/aclocal.m4 b/lib/erl_interface/aclocal.m4 index c51c26794a..ed492d55ff 100644 --- a/lib/erl_interface/aclocal.m4 +++ b/lib/erl_interface/aclocal.m4 @@ -83,8 +83,10 @@ AC_ARG_VAR(erl_xcomp_ose_LM_POST_LINK, [OSE postlink tool (only used when cross AC_ARG_VAR(erl_xcomp_ose_LM_SET_CONF, [Sets the configuration for an OSE load module (only used when cross compiling for OSE)]) AC_ARG_VAR(erl_xcomp_ose_LM_ELF_SIZE, [Prints the section size information for an OSE load module (only used when cross compiling for OSE)]) AC_ARG_VAR(erl_xcomp_ose_LM_LCF, [OSE load module linker configuration file (only used when cross compiling for OSE)]) -AC_ARG_VAR(erl_xcomp_ose_LM_CONF, [OSE load module default configuration file (only used when cross compiling for OSE)]) -AC_ARG_VAR(erl_xcomp_ose_CONFD, [OSE OSE confd source file]) +AC_ARG_VAR(erl_xcomp_ose_BEAM_LM_CONF, [BEAM OSE load module default configuration file (only used when cross compiling for OSE)]) +AC_ARG_VAR(erl_xcomp_ose_EPMD_LM_CONF, [EPMD OSE load module default configuration file (only used when cross compiling for OSE)]) +AC_ARG_VAR(erl_xcomp_ose_RUN_ERL_LM_CONF, [run_erl_lm OSE load module default configuration file (only used when cross compiling for OSE)]) +AC_ARG_VAR(erl_xcomp_ose_CONFD, [OSE confd source file]) AC_ARG_VAR(erl_xcomp_ose_CRT0_LM, [OSE crt0 lm source file]) ]) @@ -1116,7 +1118,7 @@ case "$THR_LIB_NAME" in [Define if you have the "ose_spi/ose_spi.h" header file.])) ;; esac - if test "x$THR_LIB_NAME" == "xpthread"; then + if test "x$THR_LIB_NAME" = "xpthread"; then case $host_os in openbsd*) # The default stack size is insufficient for our needs @@ -1220,7 +1222,7 @@ case "$THR_LIB_NAME" in dnl dnl Check for functions dnl - if test "x$THR_LIB_NAME" == "xpthread"; then + if test "x$THR_LIB_NAME" = "xpthread"; then AC_CHECK_FUNC(pthread_spin_lock, \ [ethr_have_native_spinlock=yes \ AC_DEFINE(ETHR_HAVE_PTHREAD_SPIN_LOCK, 1, \ diff --git a/lib/erl_interface/configure.in b/lib/erl_interface/configure.in index d511f2e240..ef78f0f87b 100644 --- a/lib/erl_interface/configure.in +++ b/lib/erl_interface/configure.in @@ -311,6 +311,26 @@ else fi fi +dnl ---------------------------------------------------------------------- +dnl Enable -fsanitize= flags. +dnl ---------------------------------------------------------------------- + +m4_define(DEFAULT_SANITIZERS, [address,undefined]) +AC_ARG_ENABLE( + sanitizers, + AS_HELP_STRING( + [--enable-sanitizers@<:@=comma-separated list of sanitizers@:>@], + [Default=DEFAULT_SANITIZERS]), +[ +case "$enableval" in + no) sanitizers= ;; + yes) sanitizers="-fsanitize=DEFAULT_SANITIZERS" ;; + *) sanitizers="-fsanitize=$enableval" ;; +esac +CFLAGS="$CFLAGS $sanitizers" +LDFLAGS="$LDFLAGS $sanitizers" +]) + # --------------------------------------------------------------------------- # XXX # --------------------------------------------------------------------------- diff --git a/lib/erl_interface/doc/src/notes.xml b/lib/erl_interface/doc/src/notes.xml index 4eb61015cc..1056d45ec6 100644 --- a/lib/erl_interface/doc/src/notes.xml +++ b/lib/erl_interface/doc/src/notes.xml @@ -30,6 +30,54 @@ </header> <p>This document describes the changes made to the Erl_interface application.</p> +<section><title>Erl_Interface 3.7.18</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Implement --enable-sanitizers[=sanitizers]. Similar to + debugging with Valgrind, it's very useful to enable + -fsanitize= switches to catch bugs at runtime.</p> + <p> + Own Id: OTP-12153</p> + </item> + </list> + </section> + +</section> + +<section><title>Erl_Interface 3.7.17</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Now works with Visual Studio.</p> + <p> + Own Id: OTP-11984</p> + </item> + </list> + </section> + +</section> + +<section><title>Erl_Interface 3.7.16</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fix memcheck warning in gen_challange (Thanks to Olivier + Girondel)</p> + <p> + Own Id: OTP-11608</p> + </item> + </list> + </section> + +</section> + <section><title>Erl_Interface 3.7.15</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/erl_interface/include/ei.h b/lib/erl_interface/include/ei.h index a3eb437f88..3f3435977d 100644 --- a/lib/erl_interface/include/ei.h +++ b/lib/erl_interface/include/ei.h @@ -39,7 +39,7 @@ #include <stdio.h> /* Need type FILE */ #include <errno.h> /* Need EHOSTUNREACH, ENOMEM, ... */ -#if !defined(__WIN32__) && !defined(VXWORKS) || (defined(VXWORKS) && defined(HAVE_SENS)) +#if !(defined(__WIN32__) || defined(_WIN32)) && !defined(VXWORKS) || (defined(VXWORKS) && defined(HAVE_SENS)) # include <netdb.h> #endif diff --git a/lib/erl_interface/src/connect/ei_connect.c b/lib/erl_interface/src/connect/ei_connect.c index 2e8418d61e..45c000ef76 100644 --- a/lib/erl_interface/src/connect/ei_connect.c +++ b/lib/erl_interface/src/connect/ei_connect.c @@ -761,7 +761,7 @@ int ei_close_connection(int fd) #endif /* - * Accept and initiate a connection from an other + * Accept and initiate a connection from another * Erlang node. Return a file descriptor at success, * otherwise -1; */ diff --git a/lib/erl_interface/src/legacy/erl_connect.c b/lib/erl_interface/src/legacy/erl_connect.c index ae0265a388..d70d914b79 100644 --- a/lib/erl_interface/src/legacy/erl_connect.c +++ b/lib/erl_interface/src/legacy/erl_connect.c @@ -190,7 +190,7 @@ int erl_close_connection(int fd) } /* - * Accept and initiate a connection from an other + * Accept and initiate a connection from another * Erlang node. Return a file descriptor at success, * otherwise -1; */ diff --git a/lib/erl_interface/vsn.mk b/lib/erl_interface/vsn.mk index 8731283265..38a13944e7 100644 --- a/lib/erl_interface/vsn.mk +++ b/lib/erl_interface/vsn.mk @@ -1,2 +1,2 @@ -EI_VSN = 3.7.16 +EI_VSN = 3.7.18 ERL_INTERFACE_VSN = $(EI_VSN) diff --git a/lib/et/doc/src/notes.xml b/lib/et/doc/src/notes.xml index 815dfada5e..e122187cf3 100644 --- a/lib/et/doc/src/notes.xml +++ b/lib/et/doc/src/notes.xml @@ -36,6 +36,51 @@ one section in this document. The title of each section is the version number of <c>Event Tracer (ET)</c>.</p> +<section><title>ET 1.5</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Application upgrade (appup) files are corrected for the + following applications: </p> + <p> + <c>asn1, common_test, compiler, crypto, debugger, + dialyzer, edoc, eldap, erl_docgen, et, eunit, gs, hipe, + inets, observer, odbc, os_mon, otp_mibs, parsetools, + percept, public_key, reltool, runtime_tools, ssh, + syntax_tools, test_server, tools, typer, webtool, wx, + xmerl</c></p> + <p> + A new test utility for testing appup files is added to + test_server. This is now used by most applications in + OTP.</p> + <p> + (Thanks to Tobias Schlager)</p> + <p> + Own Id: OTP-11744</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Removed gs based applications and gs based backends. The + <c>observer</c> application replaces the removed + applications.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-10915</p> + </item> + </list> + </section> + +</section> + <section><title>ET 1.4.4.5</title> <section><title>Improvements and New Features</title> diff --git a/lib/eunit/doc/src/notes.xml b/lib/eunit/doc/src/notes.xml index c52f12dcc8..e5a190e3e9 100644 --- a/lib/eunit/doc/src/notes.xml +++ b/lib/eunit/doc/src/notes.xml @@ -32,6 +32,50 @@ </header> <p>This document describes the changes made to the EUnit application.</p> +<section><title>Eunit 2.2.8</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Minor refactoring.</p> + <p> + Own Id: OTP-12051</p> + </item> + </list> + </section> + +</section> + +<section><title>Eunit 2.2.7</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Application upgrade (appup) files are corrected for the + following applications: </p> + <p> + <c>asn1, common_test, compiler, crypto, debugger, + dialyzer, edoc, eldap, erl_docgen, et, eunit, gs, hipe, + inets, observer, odbc, os_mon, otp_mibs, parsetools, + percept, public_key, reltool, runtime_tools, ssh, + syntax_tools, test_server, tools, typer, webtool, wx, + xmerl</c></p> + <p> + A new test utility for testing appup files is added to + test_server. This is now used by most applications in + OTP.</p> + <p> + (Thanks to Tobias Schlager)</p> + <p> + Own Id: OTP-11744</p> + </item> + </list> + </section> + +</section> + <section><title>Eunit 2.2.6</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/eunit/src/Makefile b/lib/eunit/src/Makefile index e6dab67363..82bd64c914 100644 --- a/lib/eunit/src/Makefile +++ b/lib/eunit/src/Makefile @@ -46,6 +46,8 @@ SOURCES= \ INCLUDE_FILES = eunit.hrl +INTERNAL_HRL_FILES= eunit_internal.hrl + PARSE_TRANSFORM_BIN = $(PARSE_TRANSFORM:%.erl=$(EBIN)/%.$(EMULATOR)) TARGET_FILES= $(SOURCES:%.erl=$(EBIN)/%.$(EMULATOR)) @@ -119,6 +121,7 @@ release_spec: opt $(INSTALL_DATA) $(PARSE_TRANSFORM_BIN) $(OBJECTS) "$(RELSYSDIR)/ebin" $(INSTALL_DIR) "$(RELSYSDIR)/src" $(INSTALL_DATA) $(PARSE_TRANSFORM) $(SOURCES) "$(RELSYSDIR)/src" + $(INSTALL_DATA) $(INTERNAL_HRL_FILES) "$(RELSYSDIR)/src" $(INSTALL_DIR) "$(RELSYSDIR)/include" $(INSTALL_DATA) $(INCLUDE_DELIVERABLES) "$(RELSYSDIR)/include" diff --git a/lib/eunit/src/eunit_data.erl b/lib/eunit/src/eunit_data.erl index 0350f9bf6e..cbbc6fbc15 100644 --- a/lib/eunit/src/eunit_data.erl +++ b/lib/eunit/src/eunit_data.erl @@ -440,13 +440,8 @@ parse_function({M, F}) when is_atom(M), is_atom(F) -> parse_function(F) -> bad_test(F). -check_arity(F, N, T) when is_function(F) -> - case erlang:fun_info(F, arity) of - {arity, N} -> - ok; - _ -> - bad_test(T) - end; +check_arity(F, N, _) when is_function(F, N) -> + ok; check_arity(_, _, T) -> bad_test(T). diff --git a/lib/eunit/vsn.mk b/lib/eunit/vsn.mk index f04c0536fe..855e1dac88 100644 --- a/lib/eunit/vsn.mk +++ b/lib/eunit/vsn.mk @@ -1 +1 @@ -EUNIT_VSN = 2.2.7 +EUNIT_VSN = 2.2.8 diff --git a/lib/gs/doc/src/notes.xml b/lib/gs/doc/src/notes.xml index f5117cb9ba..e40310d36e 100644 --- a/lib/gs/doc/src/notes.xml +++ b/lib/gs/doc/src/notes.xml @@ -30,7 +30,36 @@ </header> <p>This document describes the changes made to the GS application.</p> - <section><title>GS 1.5.15.2</title> + <section><title>GS 1.5.16</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Application upgrade (appup) files are corrected for the + following applications: </p> + <p> + <c>asn1, common_test, compiler, crypto, debugger, + dialyzer, edoc, eldap, erl_docgen, et, eunit, gs, hipe, + inets, observer, odbc, os_mon, otp_mibs, parsetools, + percept, public_key, reltool, runtime_tools, ssh, + syntax_tools, test_server, tools, typer, webtool, wx, + xmerl</c></p> + <p> + A new test utility for testing appup files is added to + test_server. This is now used by most applications in + OTP.</p> + <p> + (Thanks to Tobias Schlager)</p> + <p> + Own Id: OTP-11744</p> + </item> + </list> + </section> + +</section> + +<section><title>GS 1.5.15.2</title> <section><title>Improvements and New Features</title> <list> diff --git a/lib/hipe/cerl/cerl_prettypr.erl b/lib/hipe/cerl/cerl_prettypr.erl index 22f5b8945a..f4a67439d6 100644 --- a/lib/hipe/cerl/cerl_prettypr.erl +++ b/lib/hipe/cerl/cerl_prettypr.erl @@ -1,7 +1,7 @@ %% ===================================================================== %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2009. All Rights Reserved. +%% Copyright Ericsson AB 2004-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -63,7 +63,8 @@ seq_arg/1, seq_body/1, string_lit/1, try_arg/1, try_body/1, try_vars/1, try_evars/1, try_handler/1, tuple_es/1, type/1, values_es/1, var_name/1, - map_es/1, map_pair_key/1, map_pair_val/1, map_pair_op/1 + c_map/1, map_arg/1, map_es/1, is_c_map_empty/1, + c_map_pair/2, map_pair_key/1, map_pair_val/1, map_pair_op/1 ]). -define(PAPER, 76). @@ -475,13 +476,20 @@ lay_literal(Node, Ctxt) -> %% that could represent printable characters - we %% always print an integer. text(int_lit(Node)); - V when is_binary(V) -> - lay_binary(c_binary([c_bitstr(abstract(B), - abstract(8), + V when is_bitstring(V) -> + Val = fun(I) when is_integer(I) -> I; + (B) when is_bitstring(B) -> + BZ = bit_size(B), <<BV:BZ>> = B, BV + end, + Sz = fun(I) when is_integer(I) -> 8; + (B) when is_bitstring(B) -> bit_size(B) + end, + lay_binary(c_binary([c_bitstr(abstract(Val(B)), + abstract(Sz(B)), abstract(1), abstract(integer), abstract([unsigned, big])) - || B <- binary_to_list(V)]), + || B <- bitstring_to_list(V)]), Ctxt); [] -> text("[]"); @@ -489,7 +497,13 @@ lay_literal(Node, Ctxt) -> %% `lay_cons' will check for strings. lay_cons(Node, Ctxt); V when is_tuple(V) -> - lay_tuple(Node, Ctxt) + lay_tuple(Node, Ctxt); + M when is_map(M), map_size(M) =:= 0 -> + text("~{}~"); + M when is_map(M) -> + lay_map(c_map([c_map_pair(abstract(K),abstract(V)) + || {K,V} <- maps:to_list(M)]), + Ctxt) end. lay_var(Node, Ctxt) -> @@ -596,10 +610,17 @@ lay_tuple(Node, Ctxt) -> floating(text("}")))). lay_map(Node, Ctxt) -> + Arg = map_arg(Node), + After = case is_c_map_empty(Arg) of + true -> floating(text("}~")); + false -> + beside(floating(text(" | ")), + beside(lay(Arg,Ctxt), + floating(text("}~")))) + end, beside(floating(text("~{")), - beside(par(seq(map_es(Node), floating(text(",")), - Ctxt, fun lay/2)), - floating(text("}~")))). + beside(par(seq(map_es(Node), floating(text(",")), Ctxt, fun lay/2)), + After)). lay_map_pair(Node, Ctxt) -> K = map_pair_key(Node), diff --git a/lib/hipe/cerl/erl_bif_types.erl b/lib/hipe/cerl/erl_bif_types.erl index a460f16272..74e93bf098 100644 --- a/lib/hipe/cerl/erl_bif_types.erl +++ b/lib/hipe/cerl/erl_bif_types.erl @@ -2,7 +2,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2003-2013. All Rights Reserved. +%% Copyright Ericsson AB 2003-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -1891,7 +1891,11 @@ infinity_add(neg_inf, _Number) -> neg_inf; infinity_add(_Number, pos_inf) -> pos_inf; infinity_add(_Number, neg_inf) -> neg_inf; infinity_add(Number1, Number2) when is_integer(Number1), is_integer(Number2) -> - Number1 + Number2. + try Number1 + Number2 + catch + error:system_limit when Number1 < 0 -> neg_inf; + error:system_limit -> pos_inf + end. infinity_mult(neg_inf, Number) -> Greater = infinity_geq(Number, 0), @@ -1902,7 +1906,13 @@ infinity_mult(pos_inf, Number) -> infinity_inv(infinity_mult(neg_inf, Number)); infinity_mult(Number, pos_inf) -> infinity_inv(infinity_mult(neg_inf, Number)); infinity_mult(Number, neg_inf) -> infinity_mult(neg_inf, Number); infinity_mult(Number1, Number2) when is_integer(Number1), is_integer(Number2)-> - Number1 * Number2. + try Number1 * Number2 + catch + error:system_limit -> + if (Number1 >= 0) =:= (Number2 >= 0) -> pos_inf; + true -> neg_inf + end + end. width({Min, Max}) -> infinity_max([width(Min), width(Max)]); width(pos_inf) -> pos_inf; @@ -2633,7 +2643,9 @@ opaque_args(M, F, A, Xs, Opaques) -> true -> case t_tuple_subtypes(X, Opaques) of unknown -> false; - List when length(List) >= 1 -> opaque_recargs(List, Y, Opaques) + List when length(List) >= 1 -> + (t_is_atom(Y, Opaques) andalso + opaque_recargs(List, Y, Opaques)) end; false -> t_has_opaque_subtype(X, Opaques) end]; diff --git a/lib/hipe/cerl/erl_types.erl b/lib/hipe/cerl/erl_types.erl index 5938d94e65..4b2bec5fa8 100644 --- a/lib/hipe/cerl/erl_types.erl +++ b/lib/hipe/cerl/erl_types.erl @@ -72,7 +72,7 @@ t_contains_opaque/1, t_contains_opaque/2, t_decorate_with_opaque/3, t_elements/1, - t_find_opaque_mismatch/2, + t_find_opaque_mismatch/3, t_find_unknown_opaque/3, t_fixnum/0, t_map/2, @@ -126,6 +126,8 @@ t_is_instance/2, t_is_integer/1, t_is_integer/2, t_is_list/1, + t_is_map/1, + t_is_map/2, t_is_matchstate/1, t_is_nil/1, t_is_nil/2, t_is_non_neg_integer/1, @@ -148,6 +150,8 @@ t_list/1, t_list_elements/1, t_list_elements/2, t_list_termination/1, + t_map/0, + t_map/1, t_matchstate/0, t_matchstate/2, t_matchstate_present/1, @@ -205,23 +209,16 @@ type_is_defined/4, record_field_diffs_to_string/2, subst_all_vars_to_any/1, + subst_all_remote/2, lift_list_to_pos_empty/1, is_opaque_type/2, is_erl_type/1, - atom_to_string/1, - - t_is_map/2, - t_map/1, - t_map/0 + atom_to_string/1 ]). %%-define(DO_ERL_TYPES_TEST, true). -compile({no_auto_import,[min/2,max/2]}). -%% HiPE does not understand Maps -%% (guard function is_map/1 in t_from_term/1) --compile(no_native). - -ifdef(DO_ERL_TYPES_TEST). -export([test/0]). -else. @@ -265,6 +262,8 @@ -define(TAG_IMMED1_SIZE, 4). -define(BITS, (erlang:system_info(wordsize) * 8) - ?TAG_IMMED1_SIZE). +-define(MAX_TUPLE_SIZE, (1 bsl 10)). + %%----------------------------------------------------------------------------- %% Type tags and qualifiers %% @@ -533,39 +532,51 @@ list_contains_opaque(List, Opaques) -> %% The first argument of the function is the pattern and its second %% argument the type we are matching against the pattern. --spec t_find_opaque_mismatch(erl_type(), erl_type()) -> 'error' | {'ok', erl_type(), erl_type()}. +-spec t_find_opaque_mismatch(erl_type(), erl_type(), [erl_type()]) -> + 'error' | {'ok', erl_type(), erl_type()}. -t_find_opaque_mismatch(T1, T2) -> - t_find_opaque_mismatch(T1, T2, T2). +t_find_opaque_mismatch(T1, T2, Opaques) -> + t_find_opaque_mismatch(T1, T2, T2, Opaques). -t_find_opaque_mismatch(?any, _Type, _TopType) -> error; -t_find_opaque_mismatch(?none, _Type, _TopType) -> error; -t_find_opaque_mismatch(?list(T1, Tl1, _), ?list(T2, Tl2, _), TopType) -> - t_find_opaque_mismatch_ordlists([T1, Tl1], [T2, Tl2], TopType); -t_find_opaque_mismatch(_T1, ?opaque(_) = T2, TopType) -> {ok, TopType, T2}; -t_find_opaque_mismatch(?opaque(_) = T1, _T2, TopType) -> +t_find_opaque_mismatch(?any, _Type, _TopType, _Opaques) -> error; +t_find_opaque_mismatch(?none, _Type, _TopType, _Opaques) -> error; +t_find_opaque_mismatch(?list(T1, Tl1, _), ?list(T2, Tl2, _), TopType, Opaques) -> + t_find_opaque_mismatch_ordlists([T1, Tl1], [T2, Tl2], TopType, Opaques); +t_find_opaque_mismatch(T1, ?opaque(_) = T2, TopType, Opaques) -> + case is_opaque_type(T2, Opaques) of + false -> {ok, TopType, T2}; + true -> + t_find_opaque_mismatch(T1, t_opaque_structure(T2), TopType, Opaques) + end; +t_find_opaque_mismatch(?opaque(_) = T1, T2, TopType, Opaques) -> %% The generated message is somewhat misleading: - {ok, TopType, T1}; -t_find_opaque_mismatch(?product(T1), ?product(T2), TopType) -> - t_find_opaque_mismatch_ordlists(T1, T2, TopType); -t_find_opaque_mismatch(?tuple(T1, Arity, _), ?tuple(T2, Arity, _), TopType) -> - t_find_opaque_mismatch_ordlists(T1, T2, TopType); -t_find_opaque_mismatch(?tuple(_, _, _) = T1, ?tuple_set(_) = T2, TopType) -> + case is_opaque_type(T1, Opaques) of + false -> {ok, TopType, T1}; + true -> + t_find_opaque_mismatch(t_opaque_structure(T1), T2, TopType, Opaques) + end; +t_find_opaque_mismatch(?product(T1), ?product(T2), TopType, Opaques) -> + t_find_opaque_mismatch_ordlists(T1, T2, TopType, Opaques); +t_find_opaque_mismatch(?tuple(T1, Arity, _), ?tuple(T2, Arity, _), + TopType, Opaques) -> + t_find_opaque_mismatch_ordlists(T1, T2, TopType, Opaques); +t_find_opaque_mismatch(?tuple(_, _, _) = T1, ?tuple_set(_) = T2, + TopType, Opaques) -> Tuples1 = t_tuple_subtypes(T1), Tuples2 = t_tuple_subtypes(T2), - t_find_opaque_mismatch_lists(Tuples1, Tuples2, TopType); -t_find_opaque_mismatch(T1, ?union(U2), TopType) -> - t_find_opaque_mismatch_lists([T1], U2, TopType); -t_find_opaque_mismatch(_T1, _T2, _TopType) -> error. + t_find_opaque_mismatch_lists(Tuples1, Tuples2, TopType, Opaques); +t_find_opaque_mismatch(T1, ?union(U2), TopType, Opaques) -> + t_find_opaque_mismatch_lists([T1], U2, TopType, Opaques); +t_find_opaque_mismatch(_T1, _T2, _TopType, _Opaques) -> error. -t_find_opaque_mismatch_ordlists(L1, L2, TopType) -> +t_find_opaque_mismatch_ordlists(L1, L2, TopType, Opaques) -> List = lists:zipwith(fun(T1, T2) -> - t_find_opaque_mismatch(T1, T2, TopType) + t_find_opaque_mismatch(T1, T2, TopType, Opaques) end, L1, L2), t_find_opaque_mismatch_list(List). -t_find_opaque_mismatch_lists(L1, L2, _TopType) -> - List = [t_find_opaque_mismatch(T1, T2, T2) || T1 <- L1, T2 <- L2], +t_find_opaque_mismatch_lists(L1, L2, _TopType, Opaques) -> + List = [t_find_opaque_mismatch(T1, T2, T2, Opaques) || T1 <- L1, T2 <- L2], t_find_opaque_mismatch_list(List). t_find_opaque_mismatch_list([]) -> error; @@ -618,7 +629,7 @@ t_decorate_with_opaque(T1, T2, Opaques) -> end end. -decorate(?none=Type, _, _Opaques) -> Type; +decorate(Type, ?none, _Opaques) -> Type; decorate(?function(Domain, Range), ?function(D, R), Opaques) -> ?function(decorate(Domain, D, Opaques), decorate(Range, R, Opaques)); decorate(?list(Types, Tail, Size), ?list(Ts, Tl, _Sz), Opaques) -> @@ -684,6 +695,7 @@ union_decorate(U1, U2, Opaques) -> List = [A,B,F,I,L,N,T,M,Map], DecList = [Dec || E <- List, + not t_is_none(E), not t_is_none(Dec = decorate(E, Opaque, Opaques))], t_sup([Union|DecList]). @@ -1425,7 +1437,6 @@ t_number_vals(Type) -> t_number_vals(Type, Opaques) -> do_opaque(Type, Opaques, fun number_vals/1). -number_vals(?int_set(?any)) -> unknown; number_vals(?int_set(Set)) -> set_to_list(Set); number_vals(?number(_, _)) -> unknown; number_vals(?opaque(_)) -> unknown; @@ -1732,11 +1743,16 @@ lift_list_to_pos_empty(?list(Content, Termination, _)) -> t_map() -> ?map([]). --spec t_map([{erl_type(),erl_type()}]) -> erl_type(). +-spec t_map([{erl_type(), erl_type()}]) -> erl_type(). t_map(_) -> ?map([]). +-spec t_is_map(erl_type()) -> boolean(). + +t_is_map(Type) -> + t_is_map(Type, 'universe'). + -spec t_is_map(erl_type(), opaques()) -> boolean(). t_is_map(Type, Opaques) -> @@ -1745,7 +1761,6 @@ t_is_map(Type, Opaques) -> is_map1(?map(_)) -> true; is_map1(_) -> false. - %%----------------------------------------------------------------------------- %% Tuples %% @@ -1757,6 +1772,8 @@ t_tuple() -> -spec t_tuple(non_neg_integer() | [erl_type()]) -> erl_type(). +t_tuple(N) when is_integer(N), N > ?MAX_TUPLE_SIZE -> + t_tuple(); t_tuple(N) when is_integer(N) -> ?tuple(lists:duplicate(N, ?any), N, ?any); t_tuple(List) -> @@ -2166,10 +2183,10 @@ t_from_term(T) when is_function(T) -> {arity, Arity} = erlang:fun_info(T, arity), t_fun(Arity, t_any()); t_from_term(T) when is_integer(T) -> t_integer(T); +t_from_term(T) when is_map(T) -> t_map(); t_from_term(T) when is_pid(T) -> t_pid(); t_from_term(T) when is_port(T) -> t_port(); t_from_term(T) when is_reference(T) -> t_reference(); -t_from_term(T) when is_map(T) -> t_map(); t_from_term(T) when is_tuple(T) -> t_tuple([t_from_term(E) || E <- tuple_to_list(T)]). @@ -2569,7 +2586,7 @@ force_union(T = ?function(_, _)) -> ?function_union(T); force_union(T = ?identifier(_)) -> ?identifier_union(T); force_union(T = ?list(_, _, _)) -> ?list_union(T); force_union(T = ?nil) -> ?list_union(T); -force_union(T = ?number(_,_)) -> ?number_union(T); +force_union(T = ?number(_, _)) -> ?number_union(T); force_union(T = ?opaque(_)) -> ?opaque_union(T); force_union(T = ?remote(_)) -> ?remote_union(T); force_union(T = ?map(_)) -> ?map_union(T); @@ -2984,16 +3001,19 @@ inf_union(U1, U2, Opaques) -> List = [A,B,F,I,L,N,T,M,Map], inf_union_collect(List, Opaque, InfFun, [], []) end, - O1 = OpaqueFun(U1, U2, fun(E, Opaque) -> t_inf(Opaque, E, Opaques) end), - O2 = OpaqueFun(U2, U1, fun(E, Opaque) -> t_inf(E, Opaque, Opaques) end), - Union = inf_union(U1, U2, 0, [], Opaques), - t_sup([O1, O2, Union]). + {O1, ThrowList1} = + OpaqueFun(U1, U2, fun(E, Opaque) -> t_inf(Opaque, E, Opaques) end), + {O2, ThrowList2} + = OpaqueFun(U2, U1, fun(E, Opaque) -> t_inf(E, Opaque, Opaques) end), + {Union, ThrowList3} = inf_union(U1, U2, 0, [], [], Opaques), + ThrowList = lists:merge3(ThrowList1, ThrowList2, ThrowList3), + case t_sup([O1, O2, Union]) of + ?none when ThrowList =/= [] -> throw(hd(ThrowList)); + Sup -> Sup + end. inf_union_collect([], _Opaque, _InfFun, InfList, ThrowList) -> - case t_sup(InfList) of - ?none when ThrowList =/= [] -> throw(hd(lists:flatten(ThrowList))); - Sup -> Sup - end; + {t_sup(InfList), lists:usort(ThrowList)}; inf_union_collect([?none|L], Opaque, InfFun, InfList, ThrowList) -> inf_union_collect(L, Opaque, InfFun, [?none|InfList], ThrowList); inf_union_collect([E|L], Opaque, InfFun, InfList, ThrowList) -> @@ -3004,19 +3024,21 @@ inf_union_collect([E|L], Opaque, InfFun, InfList, ThrowList) -> inf_union_collect(L, Opaque, InfFun, InfList, [N|ThrowList]) end. -inf_union([?none|Left1], [?none|Left2], N, Acc, Opaques) -> - inf_union(Left1, Left2, N, [?none|Acc], Opaques); -inf_union([T1|Left1], [T2|Left2], N, Acc, Opaques) -> - case t_inf(T1, T2, Opaques) of - ?none -> inf_union(Left1, Left2, N, [?none|Acc], Opaques); - T -> inf_union(Left1, Left2, N+1, [T|Acc], Opaques) +inf_union([?none|Left1], [?none|Left2], N, Acc, ThrowList, Opaques) -> + inf_union(Left1, Left2, N, [?none|Acc], ThrowList, Opaques); +inf_union([T1|Left1], [T2|Left2], N, Acc, ThrowList, Opaques) -> + try t_inf(T1, T2, Opaques) of + ?none -> inf_union(Left1, Left2, N, [?none|Acc], ThrowList, Opaques); + T -> inf_union(Left1, Left2, N+1, [T|Acc], ThrowList, Opaques) + catch throw:N when is_integer(N) -> + inf_union(Left1, Left2, N, [?none|Acc], [N|ThrowList], Opaques) end; -inf_union([], [], N, Acc, _Opaques) -> - if N =:= 0 -> ?none; +inf_union([], [], N, Acc, ThrowList, _Opaques) -> + if N =:= 0 -> {?none, ThrowList}; N =:= 1 -> [Type] = [T || T <- Acc, T =/= ?none], - Type; - N >= 2 -> ?union(lists:reverse(Acc)) + {Type, ThrowList}; + N >= 2 -> {?union(lists:reverse(Acc)), ThrowList} end. inf_bitstr(U1, B1, U2, B2) -> @@ -3155,6 +3177,18 @@ t_subst_aux(?union(List), VarMap) -> t_subst_aux(T, _VarMap) -> T. +-spec subst_all_remote(erl_type(), erl_type()) -> erl_type(). + +subst_all_remote(Type0, Substitute) -> + Map = + fun(Type) -> + case erl_types:t_is_remote(Type) of + true -> Substitute; + false -> Type + end + end, + erl_types:t_map(Map, Type0). + %%----------------------------------------------------------------------------- %% Unification %% @@ -3580,6 +3614,8 @@ t_subtract(?product(Elements1) = T1, ?product(Elements2)) -> _ -> T1 end end; +t_subtract(?map(_) = T, _) -> % XXX: very crude; will probably need refinement + T; t_subtract(?product(P1), _) -> ?product(P1); t_subtract(T, ?product(_)) -> @@ -4466,7 +4502,9 @@ get_mod_record([{FieldName, DeclType}|Left1], [{FieldName, ModType}|Left2], Acc) -> ModTypeNoVars = subst_all_vars_to_any(ModType), case - t_is_remote(ModTypeNoVars) orelse t_is_subtype(ModTypeNoVars, DeclType) + contains_remote(ModTypeNoVars) + orelse contains_remote(DeclType) + orelse t_is_subtype(ModTypeNoVars, DeclType) of false -> {error, FieldName}; true -> get_mod_record(Left1, Left2, [{FieldName, ModType}|Acc]) @@ -4480,6 +4518,10 @@ get_mod_record(DeclFields, [], Acc) -> get_mod_record(_, [{FieldName2, _ModType}|_], _Acc) -> {error, FieldName2}. +contains_remote(Type) -> + TypeNoRemote = subst_all_remote(Type, t_none()), + not t_is_equal(Type, TypeNoRemote). + fields_from_form([], _TypeNames, _RecDict, _VarDict) -> {[], []}; fields_from_form([{Name, Type}|Tail], TypeNames, RecDict, diff --git a/lib/hipe/doc/src/notes.xml b/lib/hipe/doc/src/notes.xml index e0b1622d19..2962e4a9ac 100644 --- a/lib/hipe/doc/src/notes.xml +++ b/lib/hipe/doc/src/notes.xml @@ -30,6 +30,175 @@ </header> <p>This document describes the changes made to HiPE.</p> +<section><title>Hipe 3.11.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> The pretty-printing of bitstrings has been corrected. + </p> + <p> + Own Id: OTP-12015</p> + </item> + <item> + <p> A bug concerning <c>is_record/2,3</c> has been fixed, + as well as some cases where Dialyzer could crash due to + reaching system limits. </p> + <p> + Own Id: OTP-12018</p> + </item> + </list> + </section> + +</section> + +<section><title>Hipe 3.11</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + A Dialyzer crash involving analysis of Map types has now + been fixed.</p> + <p> + Own Id: OTP-11947</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Handle Maps instructions get_map_elements, put_map_assoc, + put_map_exact in HiPE compiler.</p> + <p> + Own Id: OTP-11900</p> + </item> + </list> + </section> + +</section> + +<section><title>Hipe 3.10.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fix compilation with 'no_remove_comments' (Thanks to + Johannes Weißl)</p> + <p> + Own Id: OTP-11564</p> + </item> + <item> + <p> + Application upgrade (appup) files are corrected for the + following applications: </p> + <p> + <c>asn1, common_test, compiler, crypto, debugger, + dialyzer, edoc, eldap, erl_docgen, et, eunit, gs, hipe, + inets, observer, odbc, os_mon, otp_mibs, parsetools, + percept, public_key, reltool, runtime_tools, ssh, + syntax_tools, test_server, tools, typer, webtool, wx, + xmerl</c></p> + <p> + A new test utility for testing appup files is added to + test_server. This is now used by most applications in + OTP.</p> + <p> + (Thanks to Tobias Schlager)</p> + <p> + Own Id: OTP-11744</p> + </item> + <item> + <p> + There is now a test suite for the Hipe application</p> + <p> + Own Id: OTP-11748</p> + </item> + <item> + <p> + Support for a LLVM backend has been added in HiPE</p> + <p> + Own Id: OTP-11801</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> The pre-defined types <c>array/0</c>, <c>dict/0</c>, + <c>digraph/0</c>, <c>gb_set/0</c>, <c>gb_tree/0</c>, + <c>queue/0</c>, <c>set/0</c>, and <c>tid/0</c> have been + deprecated. They will be removed in Erlang/OTP 18.0. </p> + <p> Instead the types <c>array:array/0</c>, + <c>dict:dict/0</c>, <c>digraph:graph/0</c>, + <c>gb_set:set/0</c>, <c>gb_tree:tree/0</c>, + <c>queue:queue/0</c>, <c>sets:set/0</c>, and + <c>ets:tid/0</c> can be used. (Note: it has always been + necessary to use <c>ets:tid/0</c>.) </p> <p> It is + allowed in Erlang/OTP 17.0 to locally re-define the types + <c>array/0</c>, <c>dict/0</c>, and so on. </p> <p> New + types <c>array:array/1</c>, <c>dict:dict/2</c>, + <c>gb_sets:set/1</c>, <c>gb_trees:tree/2</c>, + <c>queue:queue/1</c>, and <c>sets:set/1</c> have been + added. </p> <p> A compiler option, + <c>nowarn_deprecated_type</c>, has been introduced. By + including the attribute </p> <c> + -compile(nowarn_deprecated_type).</c> <p> in an Erlang + source file, warnings about deprecated types can be + avoided in Erlang/OTP 17.0. </p> <p> The option can also + be given as a compiler flag: </p> <c> erlc + +nowarn_deprecated_type file.erl</c> + <p> + Own Id: OTP-10342</p> + </item> + <item> + <p> + EEP43: New data type - Maps</p> + <p> + With Maps you may for instance: <taglist> <item><c>M0 = + #{ a => 1, b => 2}, % create + associations</c></item> <item><c>M1 = M0#{ a := 10 }, % + update values</c></item> <item><c>M2 = M1#{ "hi" => + "hello"}, % add new associations</c></item> <item><c>#{ + "hi" := V1, a := V2, b := V3} = M2. % match keys with + values</c></item> </taglist></p> + <p> + For information on how to use Maps please see Map Expressions in the + <seealso marker="doc/reference_manual:expressions#map_expressions"> + Reference Manual</seealso>.</p> + <p> + The current implementation is without the following + features: <taglist> <item>No variable keys</item> + <item>No single value access</item> <item>No map + comprehensions</item> </taglist></p> + <p> + Note that Maps is <em>experimental</em> during OTP 17.0.</p> + <p> + Own Id: OTP-11616</p> + </item> + <item> + <p> Parameterized opaque types have been introduced. </p> + <p> + Own Id: OTP-11625</p> + </item> + <item> + <p> + Add support for the compilation of the is_map/1 and + map_size/1 guards to native code.</p> + <p> + Own Id: OTP-11831</p> + </item> + </list> + </section> + +</section> + <section><title>Hipe 3.10.2.2</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/hipe/icode/hipe_beam_to_icode.erl b/lib/hipe/icode/hipe_beam_to_icode.erl index 81249c958e..4691662f9f 100644 --- a/lib/hipe/icode/hipe_beam_to_icode.erl +++ b/lib/hipe/icode/hipe_beam_to_icode.erl @@ -509,6 +509,10 @@ trans_fun([{test,test_arity,{f,Lbl},[Reg,N]}|Instructions], Env) -> I = hipe_icode:mk_type([trans_arg(Reg)],{tuple,N}, hipe_icode:label_name(True),map_label(Lbl)), [I,True | trans_fun(Instructions,Env)]; +%%--- is_map --- +trans_fun([{test,is_map,{f,Lbl},[Arg]}|Instructions], Env) -> + {Code,Env1} = trans_type_test(map,Lbl,Arg,Env), + [Code | trans_fun(Instructions,Env1)]; %%-------------------------------------------------------------------- %%--- select_val --- trans_fun([{select_val,Reg,{f,Lbl},{list,Cases}}|Instructions], Env) -> @@ -1121,6 +1125,49 @@ trans_fun([{trim,N,NY}|Instructions], Env) -> trans_fun([{line,_}|Instructions], Env) -> trans_fun(Instructions,Env); %%-------------------------------------------------------------------- +%% Map instructions added in Spring 2014 (17.0). +%%-------------------------------------------------------------------- +trans_fun([{test,has_map_fields,{f,Lbl},Map,{list,Keys}}|Instructions], Env) -> + {MapMove, MapVar, Env1} = mk_move_and_var(Map, Env), + %% We assume that hipe_icode:mk_call has no side-effects, and reuse + %% the help function of get_map_elements below, discarding the value + %% assignment instruction list. + {TestInstructions, _GetInstructions, Env2} = + trans_map_query(MapVar, map_label(Lbl), Env1, + lists:flatten([[K, {r, 0}] || K <- Keys])), + [MapMove, TestInstructions | trans_fun(Instructions, Env2)]; +trans_fun([{get_map_elements,{f,Lbl},Map,{list,KVPs}}|Instructions], Env) -> + {MapMove, MapVar, Env1} = mk_move_and_var(Map, Env), + {TestInstructions, GetInstructions, Env2} = + trans_map_query(MapVar, map_label(Lbl), Env1, KVPs), + [MapMove, TestInstructions, GetInstructions | trans_fun(Instructions, Env2)]; +%%--- put_map_assoc --- +trans_fun([{put_map_assoc,{f,Lbl},Map,Dst,_N,{list,Pairs}}|Instructions], Env) -> + {MapMove, MapVar, Env1} = mk_move_and_var(Map, Env), + TempMapVar = mk_var(new), + TempMapMove = hipe_icode:mk_move(TempMapVar, MapVar), + {PutInstructions, Env2} + = case Lbl > 0 of + true -> + gen_put_map_instrs(exists, assoc, TempMapVar, Dst, Lbl, Pairs, Env1); + false -> + gen_put_map_instrs(new, assoc, TempMapVar, Dst, new, Pairs, Env1) + end, + [MapMove, TempMapMove, PutInstructions | trans_fun(Instructions, Env2)]; +%%--- put_map_exact --- +trans_fun([{put_map_exact,{f,Lbl},Map,Dst,_N,{list,Pairs}}|Instructions], Env) -> + {MapMove, MapVar, Env1} = mk_move_and_var(Map, Env), + TempMapVar = mk_var(new), + TempMapMove = hipe_icode:mk_move(TempMapVar, MapVar), + {PutInstructions, Env2} + = case Lbl > 0 of + true -> + gen_put_map_instrs(exists, exact, TempMapVar, Dst, Lbl, Pairs, Env1); + false -> + gen_put_map_instrs(new, exact, TempMapVar, Dst, new, Pairs, Env1) + end, + [MapMove, TempMapMove, PutInstructions | trans_fun(Instructions, Env2)]; +%%-------------------------------------------------------------------- %%--- ERROR HANDLING --- %%-------------------------------------------------------------------- trans_fun([X|_], _) -> @@ -1500,6 +1547,102 @@ trans_type_test2(function2, Lbl, Arg, Arity, Env) -> hipe_icode:label_name(True), map_label(Lbl)), {[Move1,Move2,I,True],Env2}. +%% +%% Handles the get_map_elements instruction and the has_map_fields +%% test instruction. +%% +trans_map_query(_MapVar, _FailLabel, Env, []) -> + {[], [], Env}; +trans_map_query(MapVar, FailLabel, Env, [Key,Val|KVPs]) -> + {Move,KeyVar,Env1} = mk_move_and_var(Key,Env), + PassLabel = mk_label(new), + BoolVar = hipe_icode:mk_new_var(), + ValVar = mk_var(Val), + IsKeyCall = hipe_icode:mk_call([BoolVar], maps, is_key, [KeyVar, MapVar], + remote), + TrueTest = hipe_icode:mk_if('=:=', [BoolVar, hipe_icode:mk_const(true)], + hipe_icode:label_name(PassLabel), FailLabel), + GetCall = hipe_icode:mk_call([ValVar], maps, get, [KeyVar, MapVar], remote), + {TestList, GetList, Env2} = trans_map_query(MapVar, FailLabel, Env1, KVPs), + {[Move, IsKeyCall, TrueTest, PassLabel|TestList], [GetCall|GetList], Env2}. + +%% +%% Generates a fail label if necessary when translating put_map_* instructions. +%% +gen_put_map_instrs(exists, Op, TempMapVar, Dst, FailLbl, Pairs, Env) -> + TrueLabel = mk_label(new), + IsMapCode = hipe_icode:mk_type([TempMapVar], map, + hipe_icode:label_name(TrueLabel), map_label(FailLbl)), + DstMapVar = mk_var(Dst), + {ReturnLbl, PutInstructions, Env1} + = case Op of + assoc -> + trans_put_map_assoc(TempMapVar, DstMapVar, Pairs, Env, []); + exact -> + trans_put_map_exact(TempMapVar, DstMapVar, + map_label(FailLbl), Pairs, Env, []) + end, + {[IsMapCode, TrueLabel, PutInstructions, ReturnLbl], Env1}; +gen_put_map_instrs(new, Op, TempMapVar, Dst, new, Pairs, Env) -> + TrueLabel = mk_label(new), + FailLbl = mk_label(new), + IsMapCode = hipe_icode:mk_type([TempMapVar], map, + hipe_icode:label_name(TrueLabel), + hipe_icode:label_name(FailLbl)), + DstMapVar = mk_var(Dst), + {ReturnLbl, PutInstructions, Env1} + = case Op of + assoc -> + trans_put_map_assoc(TempMapVar, DstMapVar, Pairs, Env, []); + exact -> + trans_put_map_exact(TempMapVar, DstMapVar, + hipe_icode:label_name(FailLbl), Pairs, Env, []) + end, + Fail = hipe_icode:mk_fail([hipe_icode:mk_const(badarg)], error), + {[IsMapCode, TrueLabel, PutInstructions, FailLbl, Fail, ReturnLbl], Env1}. + +%%----------------------------------------------------------------------- +%% This function generates the instructions needed to insert several +%% (Key, Value) pairs into an existing map, each recursive call inserts +%% one (Key, Value) pair. +%%----------------------------------------------------------------------- +trans_put_map_assoc(MapVar, DestMapVar, [], Env, Acc) -> + MoveToReturnVar = hipe_icode:mk_move(DestMapVar, MapVar), + ReturnLbl = mk_label(new), + GotoReturn = hipe_icode:mk_goto(hipe_icode:label_name(ReturnLbl)), + {ReturnLbl, lists:reverse([GotoReturn, MoveToReturnVar | Acc]), Env}; +trans_put_map_assoc(MapVar, DestMapVar, [Key, Value | Rest], Env, Acc) -> + {MoveKey, KeyVar, Env1} = mk_move_and_var(Key, Env), + {MoveVal, ValVar, Env2} = mk_move_and_var(Value, Env1), + BifCall = hipe_icode:mk_call([MapVar], maps, put, + [KeyVar, ValVar, MapVar], remote), + trans_put_map_assoc(MapVar, DestMapVar, Rest, Env2, + [BifCall, MoveVal, MoveKey | Acc]). + +%%----------------------------------------------------------------------- +%% This function generates the instructions needed to update several +%% (Key, Value) pairs in an existing map, each recursive call inserts +%% one (Key, Value) pair. +%%----------------------------------------------------------------------- +trans_put_map_exact(MapVar, DestMapVar, _FLbl, [], Env, Acc) -> + MoveToReturnVar = hipe_icode:mk_move(DestMapVar, MapVar), + ReturnLbl = mk_label(new), + GotoReturn = hipe_icode:mk_goto(hipe_icode:label_name(ReturnLbl)), + {ReturnLbl, lists:reverse([GotoReturn, MoveToReturnVar | Acc]), Env}; +trans_put_map_exact(MapVar, DestMapVar, FLbl, [Key, Value | Rest], Env, Acc) -> + SuccLbl = mk_label(new), + {MoveKey, KeyVar, Env1} = mk_move_and_var(Key, Env), + {MoveVal, ValVar, Env2} = mk_move_and_var(Value, Env1), + IsKey = hipe_icode:mk_new_var(), + BifCallIsKey = hipe_icode:mk_call([IsKey], maps, is_key, + [KeyVar, MapVar], remote), + IsKeyTest = hipe_icode:mk_if('=:=', [IsKey, hipe_icode:mk_const(true)], + hipe_icode:label_name(SuccLbl), FLbl), + BifCallPut = hipe_icode:mk_call([MapVar], maps, put, + [KeyVar, ValVar, MapVar], remote), + Acc1 = [BifCallPut, SuccLbl, IsKeyTest, BifCallIsKey, MoveVal, MoveKey | Acc], + trans_put_map_exact(MapVar, DestMapVar, FLbl, Rest, Env2, Acc1). + %%----------------------------------------------------------------------- %% trans_puts(Code, Environment) -> %% {Movs, Code, Vars, NewEnv} diff --git a/lib/hipe/icode/hipe_icode.hrl b/lib/hipe/icode/hipe_icode.hrl index 25deac5152..46c04beb40 100644 --- a/lib/hipe/icode/hipe_icode.hrl +++ b/lib/hipe/icode/hipe_icode.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2013. All Rights Reserved. +%% Copyright Ericsson AB 2004-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -61,8 +61,8 @@ | 'op_exact_eqeq_2' | 'suspend_msg_timeout'. -type icode_type_test() :: 'atom' | 'bignum' | 'binary' | 'bitstr' | 'boolean' - | 'cons' | 'fixnum' | 'float' - | 'function' | 'function2' | 'integer' | 'list' | 'nil' + | 'cons' | 'fixnum' | 'float' | 'function' + | 'function2' | 'integer' | 'list' | 'map' | 'nil' | 'number' | 'pid' | 'port' | 'reference' | 'tuple' | {'atom', atom()} | {'integer', integer()} | {'record', atom(), non_neg_integer()} @@ -108,7 +108,6 @@ length :: non_neg_integer(), cases :: [icode_switch_case()]}). - -record(icode_type, {test :: icode_type_test(), args :: [icode_term_arg()], true_label :: icode_lbl(), diff --git a/lib/hipe/icode/hipe_icode_type.erl b/lib/hipe/icode/hipe_icode_type.erl index 65876b83ea..ebeb5e2c10 100644 --- a/lib/hipe/icode/hipe_icode_type.erl +++ b/lib/hipe/icode/hipe_icode_type.erl @@ -84,15 +84,15 @@ t_fun/0, t_fun/1, t_fun/2, t_fun_args/1, t_fun_arity/1, t_inf/2, t_inf_lists/2, t_integer/0, t_integer/1, t_is_atom/1, t_is_any/1, - t_is_binary/1, t_is_bitstr/1, t_is_bitwidth/1, t_is_boolean/1, - t_is_fixnum/1, t_is_cons/1, + t_is_binary/1, t_is_bitstr/1, t_is_bitwidth/1, + t_is_boolean/1, t_is_fixnum/1, t_is_cons/1, t_is_map/1, t_is_maybe_improper_list/1, t_is_equal/2, t_is_float/1, t_is_fun/1, t_is_integer/1, t_is_non_neg_integer/1, t_is_number/1, t_is_matchstate/1, - t_is_nil/1, t_is_none/1, t_is_port/1, t_is_pid/1, + t_is_none/1, t_is_port/1, t_is_pid/1, t_is_reference/1, t_is_subtype/2, t_is_tuple/1, t_limit/2, t_matchstate_present/1, t_matchstate/0, - t_matchstate_slots/1, t_maybe_improper_list/0, + t_matchstate_slots/1, t_maybe_improper_list/0, t_map/0, t_nil/0, t_none/0, t_number/0, t_number/1, t_number_vals/1, t_pid/0, t_port/0, t_reference/0, t_subtract/2, t_sup/2, t_to_tlist/1, t_tuple/0, t_tuple/1, t_tuple_sizes/1]). @@ -213,7 +213,7 @@ analyse_blocks(Work, State, MFA) -> {NewState, NewLabels} = try analyse_block(Label, Info, State) catch throw:none_type -> - %% io:format("received none type at label: ~p~n",[Label]), + %% io:format("received none type at label: ~p~n", [Label]), {State,[]} end, NewWork2 = add_work(NewWork, NewLabels), @@ -265,7 +265,7 @@ analyse_insn(I, Info, LookupFun) -> do_move(I, Info); #icode_call{} -> NewInfo = do_call(I, Info, LookupFun), - %%io:format("Analysing Call: ~w~n~w~n", [I,NewInfo]), + %% io:format("Analysing Call: ~w~n~w~n", [I, NewInfo]), update_call_arguments(I, NewInfo); #icode_phi{} -> Type = t_limit(join_list(hipe_icode:args(I), Info), ?TYPE_DEPTH), @@ -788,16 +788,16 @@ test_record(Atom, Size, Var, VarInfo, TrueLab, FalseLab, Info) -> end. test_type(Test, Type) -> - %%io:format("Test is: ~w\n", [Test]), - %%io:format("Type is: ~s\n", [format_type(Type)]), + %% io:format("Test is: ~w\n", [Test]), + %% io:format("Type is: ~s\n", [format_type(Type)]), Ans = case t_is_any(Type) of true -> maybe; false -> TrueTest = true_branch_info(Test), Inf = t_inf(TrueTest, Type), - %%io:format("TrueTest is: ~s\n", [format_type(TrueTest)]), - %%io:format("Inf is: ~s\n", [format_type(Inf)]), + %% io:format("TrueTest is: ~s\n", [format_type(TrueTest)]), + %% io:format("Inf is: ~s\n", [format_type(Inf)]), case t_is_equal(Type, Inf) of true -> not t_is_none(Type); @@ -895,11 +895,12 @@ test_type0(boolean, T) -> t_is_boolean(T); test_type0(list, T) -> t_is_maybe_improper_list(T); -test_type0(cons, T) -> - t_is_cons(T); -test_type0(nil, T) -> - t_is_nil(T). - +%% test_type0(cons, T) -> +%% t_is_cons(T); +%% test_type0(nil, T) -> +%% t_is_nil(T). +test_type0(map, T) -> + t_is_map(T). true_branch_info(integer) -> t_integer(); @@ -931,22 +932,24 @@ true_branch_info(reference) -> t_reference(); true_branch_info(function) -> t_fun(); -true_branch_info(cons) -> - t_cons(); -true_branch_info(nil) -> - t_nil(); +%% true_branch_info(cons) -> +%% t_cons(); +%% true_branch_info(nil) -> +%% t_nil(); true_branch_info(boolean) -> t_boolean(); +true_branch_info(map) -> + t_map(); true_branch_info(T) -> - exit({?MODULE,unknown_typetest,T}). + exit({?MODULE, unknown_typetest, T}). %% _________________________________________________________________ %% %% Remove the redundant type tests. If a test is removed, the trace -%% that isn't taken is explicitly removed from the CFG to simpilify +%% that isn't taken is explicitly removed from the CFG to simplify %% the handling of Phi nodes. If a Phi node is left and at least one -%% branch into it has disappeared, the SSA propagation pass can't +%% branch into it has disappeared, the SSA propagation pass cannot %% handle it. %% %% If the CFG has changed at the end of this pass, the analysis is diff --git a/lib/hipe/llvm/hipe_llvm_main.erl b/lib/hipe/llvm/hipe_llvm_main.erl index e911fb89c9..0e50c9539b 100644 --- a/lib/hipe/llvm/hipe_llvm_main.erl +++ b/lib/hipe/llvm/hipe_llvm_main.erl @@ -342,8 +342,8 @@ create_sdesc_list([{ExnLbl, SPOff} | MoreExnAndSPOffs], %% (thus, some of their arguments are passed to the stack). Because of the %% Reserved Call Frame feature that the LLVM uses, the stack descriptors %% are not correct since at the point of call the frame size is reduced -%% proportionally to the number of arguments that are passed on the stack. -%% Also the offsets of the roots need to be re-adjusted. +%% by the number of arguments that are passed on the stack. Also, the +%% offsets of the roots need to be re-adjusted. fix_stack_descriptors(_, _, [], _) -> []; fix_stack_descriptors(RelocsDict, Relocs, SDescs, ExposedClosures) -> @@ -427,30 +427,22 @@ find_offsets([{Off,Arity}|Rest], Offsets, Acc) -> [I | RestOffsets] = lists:dropwhile(fun (Y) -> Y<Off end, Offsets), find_offsets(Rest, RestOffsets, [{I, Arity}|Acc]). -%% The functions below correct the arity of calls, that are identified -%% by offset, in the stack descriptors. +%% The function below corrects the stack descriptors of calls with arguments +%% that are passed on the stack (more than NR_ARG_REGS) by subtracting the +%% number of stacked arguments from the frame size and from the offset of the +%% roots. fix_sdescs([], SDescs) -> SDescs; fix_sdescs([{Offset, Arity} | Rest], SDescs) -> case lists:keyfind(Offset, 2, SDescs) of false -> fix_sdescs(Rest, SDescs); - {?SDESC, Offset, SDesc} -> - {ExnHandler, FrameSize, StkArity, Roots} = SDesc, - DecRoot = fun(X) -> X-Arity end, - NewRootsList = lists:map(DecRoot, tuple_to_list(Roots)), - NewSDesc = - case length(NewRootsList) > 0 andalso hd(NewRootsList) >= 0 of - true -> - {?SDESC, Offset, {ExnHandler, FrameSize-Arity, StkArity, - list_to_tuple(NewRootsList)}}; - false -> - {?SDESC, Offset, {ExnHandler, FrameSize, StkArity, Roots}} - end, - RestSDescs = lists:keydelete(Offset, 2, SDescs), - fix_sdescs(Rest, [NewSDesc | RestSDescs]) + {?SDESC, Offset, {ExnHandler, FrameSize, StkArity, Roots}} -> + FixedRoots = list_to_tuple([Ri - Arity || Ri <- tuple_to_list(Roots)]), + FixedSDesc = + {?SDESC, Offset, {ExnHandler, FrameSize - Arity, StkArity, FixedRoots}}, + fix_sdescs(Rest, [FixedSDesc | lists:keydelete(Offset, 2, SDescs)]) end. - %%------------------------------------------------------------------------------ %% Miscellaneous functions %%------------------------------------------------------------------------------ diff --git a/lib/hipe/llvm/hipe_rtl_to_llvm.erl b/lib/hipe/llvm/hipe_rtl_to_llvm.erl index ba76e1d815..d7d8d1b049 100644 --- a/lib/hipe/llvm/hipe_rtl_to_llvm.erl +++ b/lib/hipe/llvm/hipe_rtl_to_llvm.erl @@ -430,12 +430,13 @@ trans_call_name(RtlCallName, Relocs, CallArgs, FinalArgs) -> case RtlCallName of PrimOp when is_atom(PrimOp) -> LlvmName = trans_prim_op(PrimOp), - Relocs1 = relocs_store(LlvmName, {call, {bif, PrimOp, length(CallArgs)}}, - Relocs), + Relocs1 = + relocs_store(LlvmName, {call, {bif, PrimOp, length(CallArgs)}}, Relocs), {"@" ++ LlvmName, [], Relocs1}; {M, F, A} when is_atom(M), is_atom(F), is_integer(A) -> - LlvmName = trans_mfa_name({M,F,A}), - Relocs1 = relocs_store(LlvmName, {call, {M,F,A}}, Relocs), + LlvmName = trans_mfa_name({M, F, A}), + Relocs1 = + relocs_store(LlvmName, {call, {M, F, length(CallArgs)}}, Relocs), {"@" ++ LlvmName, [], Relocs1}; Reg -> case hipe_rtl:is_reg(Reg) of diff --git a/lib/hipe/main/hipe.erl b/lib/hipe/main/hipe.erl index d47eced6d8..539ce883c0 100644 --- a/lib/hipe/main/hipe.erl +++ b/lib/hipe/main/hipe.erl @@ -200,6 +200,7 @@ compile_core/4, file/1, file/2, + llvm_support_available/0, load/1, help/0, help_hiper/0, @@ -648,7 +649,18 @@ run_compiler_1(DisasmFun, IcodeFun, Options) -> %% The full option expansion is not done %% until the DisasmFun returns. {Code, CompOpts} = DisasmFun(Options), - Opts = expand_options(Options ++ CompOpts), + Opts0 = expand_options(Options ++ CompOpts), + Opts = + case proplists:get_bool(to_llvm, Opts0) andalso + not llvm_support_available() of + true -> + ?error_msg("No LLVM version 3.4 or greater " + "found in $PATH; aborting " + "native code compilation.\n", []), + ?EXIT(cant_find_required_llvm_version); + false -> + Opts0 + end, check_options(Opts), ?when_option(verbose, Options, ?debug_msg("Options: ~p.\n",[Opts])), @@ -1537,4 +1549,22 @@ check_options(Opts) -> ok end. +-spec llvm_support_available() -> boolean(). + +llvm_support_available() -> + get_llvm_version() >= 3.4. + +get_llvm_version() -> + OptStr = os:cmd("opt -version"), + SubStr = "LLVM version ", N = length(SubStr), + case string:str(OptStr, SubStr) of + 0 -> % No opt available + 0.0; + S -> + case string:to_float(string:sub_string(OptStr, S + N)) of + {error, _} -> 0.0; %XXX: Assumes no revision numbers in versioning + {Float, _} -> Float + end + end. + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% diff --git a/lib/hipe/rtl/hipe_icode2rtl.erl b/lib/hipe/rtl/hipe_icode2rtl.erl index 6ab40adcc8..483d0b37f7 100644 --- a/lib/hipe/rtl/hipe_icode2rtl.erl +++ b/lib/hipe/rtl/hipe_icode2rtl.erl @@ -437,6 +437,8 @@ gen_type_test([X], Type, TrueLbl, FalseLbl, Pred, ConstTab) -> {hipe_tagscheme:test_integer(X, TrueLbl, FalseLbl, Pred), ConstTab}; list -> {hipe_tagscheme:test_list(X, TrueLbl, FalseLbl, Pred), ConstTab}; + map -> + {hipe_tagscheme:test_map(X, TrueLbl, FalseLbl, Pred), ConstTab}; nil -> {hipe_tagscheme:test_nil(X, TrueLbl, FalseLbl, Pred), ConstTab}; number -> diff --git a/lib/hipe/rtl/hipe_tagscheme.erl b/lib/hipe/rtl/hipe_tagscheme.erl index 4725889d8d..c27c682915 100644 --- a/lib/hipe/rtl/hipe_tagscheme.erl +++ b/lib/hipe/rtl/hipe_tagscheme.erl @@ -39,7 +39,7 @@ test_tuple/4, test_atom/4, test_bignum/4, test_pos_bignum/4, test_any_pid/4, test_any_port/4, test_ref/4, test_fun/4, test_fun2/5, test_matchstate/4, - test_binary/4, test_bitstr/4, test_list/4, + test_binary/4, test_bitstr/4, test_list/4, test_map/4, test_integer/4, test_number/4, test_tuple_N/5]). -export([realtag_fixnum/2, tag_fixnum/2, realuntag_fixnum/2, untag_fixnum/2]). -export([test_two_fixnums/3, test_fixnums/4, unsafe_fixnum_add/3, @@ -112,13 +112,15 @@ -define(TAG_HEADER_EXTERNAL_PID, ((16#C bsl ?TAG_PRIMARY_SIZE) bor ?TAG_PRIMARY_HEADER)). -define(TAG_HEADER_EXTERNAL_PORT,((16#D bsl ?TAG_PRIMARY_SIZE) bor ?TAG_PRIMARY_HEADER)). -define(TAG_HEADER_EXTERNAL_REF, ((16#E bsl ?TAG_PRIMARY_SIZE) bor ?TAG_PRIMARY_HEADER)). +-define(TAG_HEADER_MAP, ((16#F bsl ?TAG_PRIMARY_SIZE) bor ?TAG_PRIMARY_HEADER)). -define(TAG_HEADER_MASK, 16#3F). -define(HEADER_ARITY_OFFS, 6). %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -mk_header(SZ,TAG) -> (SZ bsl ?HEADER_ARITY_OFFS) + TAG. +mk_header(SZ, TAG) -> (SZ bsl ?HEADER_ARITY_OFFS) + TAG. + mk_arityval(SZ) -> mk_header(SZ, ?TAG_HEADER_ARITYVAL). size_from_header(Sz, Header) -> @@ -132,9 +134,9 @@ mk_var_header(Header, Size, Tag) -> mk_fixnum(X) -> (X bsl ?TAG_IMMED1_SIZE) + ?TAG_IMMED1_SMALL. -define(NIL, ((-1 bsl ?TAG_IMMED2_SIZE) bor ?TAG_IMMED2_NIL)). -mk_nil() -> ?NIL. -%% mk_atom(X) -> (X bsl ?TAG_IMMED2_SIZE) + ?TAG_IMMED2_ATOM. -mk_non_value() -> ?THE_NON_VALUE. +mk_nil() -> ?NIL. +%% mk_atom(X) -> (X bsl ?TAG_IMMED2_SIZE) + ?TAG_IMMED2_ATOM. +mk_non_value() -> ?THE_NON_VALUE. -spec is_fixnum(integer()) -> boolean(). is_fixnum(N) when is_integer(N) -> @@ -252,6 +254,15 @@ test_tuple_N(X, N, TrueLab, FalseLab, Pred) -> hipe_rtl:mk_branch(Tmp, 'eq', hipe_rtl:mk_imm(mk_arityval(N)), TrueLab, FalseLab, Pred)]. +test_map(X, TrueLab, FalseLab, Pred) -> + Tmp = hipe_rtl:mk_new_reg_gcsafe(), + HalfTrueLab = hipe_rtl:mk_new_label(), + MapMask = ?TAG_HEADER_MASK, + [test_is_boxed(X, hipe_rtl:label_name(HalfTrueLab), FalseLab, Pred), + HalfTrueLab, + get_header(Tmp, X), + mask_and_compare(Tmp, MapMask, ?TAG_HEADER_MAP, TrueLab, FalseLab, Pred)]. + test_ref(X, TrueLab, FalseLab, Pred) -> Hdr = hipe_rtl:mk_new_reg_gcsafe(), Tag = hipe_rtl:mk_new_reg_gcsafe(), diff --git a/lib/hipe/test/Makefile b/lib/hipe/test/Makefile index cedb150b5d..009f503abb 100644 --- a/lib/hipe/test/Makefile +++ b/lib/hipe/test/Makefile @@ -8,6 +8,11 @@ include $(ERL_TOP)/make/$(TARGET)/otp.mk MODULES= \ hipe_SUITE +# .erl files for these modules are automatically generated +GEN_MODULES= \ + bs_SUITE \ + maps_SUITE + ERL_FILES= $(MODULES:%=%.erl) TARGET_FILES= $(MODULES:%=$(EBIN)/%.$(EMULATOR)) @@ -40,6 +45,8 @@ EBIN = . make_emakefile: $(ERL_TOP)/make/make_emakefile $(ERL_COMPILE_FLAGS) -o$(EBIN) $(MODULES) \ > $(EMAKEFILE) + $(ERL_TOP)/make/make_emakefile $(ERL_COMPILE_FLAGS) -o$(EBIN) $(GEN_MODULES) \ + >> $(EMAKEFILE) $(ERL_TOP)/make/make_emakefile $(ERL_COMPILE_FLAGS) -o$(EBIN) '*_SUITE_make' \ >> $(EMAKEFILE) diff --git a/lib/hipe/test/hipe_testsuite_driver.erl b/lib/hipe/test/hipe_testsuite_driver.erl index c8fdf1600c..5f05a716bc 100644 --- a/lib/hipe/test/hipe_testsuite_driver.erl +++ b/lib/hipe/test/hipe_testsuite_driver.erl @@ -85,7 +85,7 @@ list_testcases(Dirname) -> list_dir(Dir, Extension, Dirs) -> case file:list_dir(Dir) of - {error, _} = Error-> Error; + {error, _} = Error -> Error; {ok, Filenames} -> FullFilenames = [filename:join(Dir, F) || F <- Filenames], Matches1 = case Dirs of @@ -173,10 +173,29 @@ run(TestCase, Dir, _OutDir) -> %% end, DataFiles), %% try ok = TestCase:test(), - HiPEOpts = try TestCase:hipe_options() catch _:_ -> [] end, + HiPEOpts = try TestCase:hipe_options() catch error:undef -> [] end, {ok, TestCase} = hipe:c(TestCase, HiPEOpts), - ok = TestCase:test(). + ok = TestCase:test(), + case is_llvm_opt_available() of + true -> + {ok, TestCase} = hipe:c(TestCase, [to_llvm|HiPEOpts]), + ok = TestCase:test(); + false -> ok + end. %% after %% lists:foreach(fun (DF) -> ok end, % = file:delete(DF) end, %% [filename:join(OutDir, D) || D <- DataFiles]) %% end. + + +%% This function, which is supposed to check whether the right LLVM +%% infrastructure is available, should be probably written in a better +%% and more portable way and moved to the hipe application. + +is_llvm_opt_available() -> + OptStr = os:cmd("opt -version"), + SubStr = "LLVM version ", N = length(SubStr), + case string:str(OptStr, SubStr) of + 0 -> false; + S -> P = S + N, string:sub_string(OptStr, P, P + 2) >= "3.4" + end. diff --git a/lib/hipe/test/maps_SUITE_data/maps_build_and_match_aliasing.erl b/lib/hipe/test/maps_SUITE_data/maps_build_and_match_aliasing.erl new file mode 100644 index 0000000000..14d8320cdf --- /dev/null +++ b/lib/hipe/test/maps_SUITE_data/maps_build_and_match_aliasing.erl @@ -0,0 +1,20 @@ +-module(maps_build_and_match_aliasing). +-export([test/0]). + +test() -> + M1 = id(#{a=>1,b=>2,c=>3,d=>4}), + #{c:=C1=_=_=C2} = M1, + true = C1 =:= C2, + #{a:=A,a:=A,a:=A,b:=B,b:=B} = M1, + #{a:=A,a:=A,a:=A,b:=B,b:=B,b:=2} = M1, + #{a:=A=1,a:=A,a:=A,b:=B=2,b:=B,b:=2} = M1, + #{c:=C1, c:=_, c:=3, c:=_, c:=C2} = M1, + #{c:=C=_=3=_=C} = M1, + + M2 = id(#{"a"=>1,"b"=>2,"c"=>3,"d"=>4}), + #{"a":=A2,"a":=A2,"a":=A2,"b":=B2,"b":=B2,"b":=2} = M2, + #{"a":=_,"a":=_,"a":=_,"b":=_,"b":=_,"b":=2} = M2, + ok. + +%% Use this function to avoid compile-time evaluation of an expression. +id(I) -> I. diff --git a/lib/hipe/test/maps_SUITE_data/maps_build_and_match_empty_val.erl b/lib/hipe/test/maps_SUITE_data/maps_build_and_match_empty_val.erl new file mode 100644 index 0000000000..2abfa4e5b3 --- /dev/null +++ b/lib/hipe/test/maps_SUITE_data/maps_build_and_match_empty_val.erl @@ -0,0 +1,17 @@ +-module(maps_build_and_match_empty_val). +-export([test/0]). + +test() -> + F = fun(#{ "hi":=_,{1,2}:=_,1337:=_}) -> ok end, + ok = F(id(#{"hi"=>ok,{1,2}=>ok,1337=>ok})), + + %% error case + case (catch (F(id(#{"hi"=>ok})))) of + {'EXIT',{function_clause,_}} -> ok; + {'EXIT', {{case_clause,_},_}} -> {comment,inlined}; + Other -> + test_server:fail({no_match, Other}) + end. + +%% Use this function to avoid compile-time evaluation of an expression. +id(I) -> I. diff --git a/lib/hipe/test/maps_SUITE_data/maps_build_and_match_literals.erl b/lib/hipe/test/maps_SUITE_data/maps_build_and_match_literals.erl new file mode 100644 index 0000000000..dc2c63fab2 --- /dev/null +++ b/lib/hipe/test/maps_SUITE_data/maps_build_and_match_literals.erl @@ -0,0 +1,40 @@ +-module(maps_build_and_match_literals). +-export([test/0]). + +test() -> + #{} = id(#{}), + #{1:=a} = id(#{1=>a}), + #{1:=a,2:=b} = id(#{1=>a,2=>b}), + #{1:=a,2:=b,3:="c"} = id(#{1=>a,2=>b,3=>"c"}), + #{1:=a,2:=b,3:="c","4":="d"} = id(#{1=>a,2=>b,3=>"c","4"=>"d"}), + #{1:=a,2:=b,3:="c","4":="d",<<"5">>:=<<"e">>} = + id(#{1=>a,2=>b,3=>"c","4"=>"d",<<"5">>=><<"e">>}), + #{1:=a,2:=b,3:="c","4":="d",<<"5">>:=<<"e">>,{"6",7}:="f"} = + id(#{1=>a,2=>b,3=>"c","4"=>"d",<<"5">>=><<"e">>,{"6",7}=>"f"}), + #{1:=a,2:=b,3:="c","4":="d",<<"5">>:=<<"e">>,{"6",7}:="f",8:=g} = + id(#{1=>a,2=>b,3=>"c","4"=>"d",<<"5">>=><<"e">>,{"6",7}=>"f",8=>g}), + + #{<<"hi all">> := 1} = id(#{<<"hi",32,"all">> => 1}), + + #{a:=X,a:=X=3,b:=4} = id(#{a=>3,b=>4}), % weird but ok =) + + #{ a:=#{ b:=#{c := third, b:=second}}, b:=first} = + id(#{ b=>first, a=>#{ b=>#{c => third, b=> second}}}), + + M = #{ map_1=>#{ map_2=>#{value_3 => third}, value_2=> second}, value_1=>first}, + M = #{ map_1:=#{ map_2:=#{value_3 := third}, value_2:= second}, value_1:=first} = + id(#{ map_1=>#{ map_2=>#{value_3 => third}, value_2=> second}, value_1=>first}), + + %% nil key + #{[]:=ok,1:=2} = id(#{[]=>ok,1=>2}), + + %% error case + {'EXIT',{{badmatch,_},_}} = (catch (#{x:=3,x:=2} = id(#{x=>3}))), + {'EXIT',{{badmatch,_},_}} = (catch (#{x:=2} = id(#{x=>3}))), + {'EXIT',{{badmatch,_},_}} = (catch (#{x:=3} = id({a,b,c}))), + {'EXIT',{{badmatch,_},_}} = (catch (#{x:=3} = id(#{y=>3}))), + {'EXIT',{{badmatch,_},_}} = (catch (#{x:=3} = id(#{x=>"three"}))), + ok. + +%% Use this function to avoid compile-time evaluation of an expression. +id(I) -> I. diff --git a/lib/hipe/test/maps_SUITE_data/maps_build_and_match_over_alloc.erl b/lib/hipe/test/maps_SUITE_data/maps_build_and_match_over_alloc.erl new file mode 100644 index 0000000000..dae6f64e5f --- /dev/null +++ b/lib/hipe/test/maps_SUITE_data/maps_build_and_match_over_alloc.erl @@ -0,0 +1,16 @@ +-module(maps_build_and_match_over_alloc). +-export([test/0]). + +test() -> + Ls = id([1,2,3]), + V0 = [a|Ls], + M0 = id(#{ "a" => V0 }), + #{ "a" := V1 } = M0, + V2 = id([c|Ls]), + M2 = id(#{ "a" => V2 }), + #{ "a" := V3 } = M2, + {[a,1,2,3],[c,1,2,3]} = id({V1,V3}), + ok. + +%% Use this function to avoid compile-time evaluation of an expression. +id(I) -> I. diff --git a/lib/hipe/test/maps_SUITE_data/maps_build_and_match_val.erl b/lib/hipe/test/maps_SUITE_data/maps_build_and_match_val.erl new file mode 100644 index 0000000000..284f69e06c --- /dev/null +++ b/lib/hipe/test/maps_SUITE_data/maps_build_and_match_val.erl @@ -0,0 +1,23 @@ +-module(maps_build_and_match_val). +-export([test/0]). + +test() -> + F = fun + (#{ "hi" := first, v := V}) -> {1,V}; + (#{ "hi" := second, v := V}) -> {2,V} + end, + + + {1,"hello"} = F(id(#{"hi"=>first,v=>"hello"})), + {2,"second"} = F(id(#{"hi"=>second,v=>"second"})), + + %% error case + case (catch (F(id(#{"hi"=>ok})))) of + {'EXIT',{function_clause,_}} -> ok; + {'EXIT', {{case_clause,_},_}} -> {comment,inlined}; + Other -> + test_server:fail({no_match, Other}) + end. + +%% Use this function to avoid compile-time evaluation of an expression. +id(I) -> I. diff --git a/lib/hipe/test/maps_SUITE_data/maps_expand_map_update.erl b/lib/hipe/test/maps_SUITE_data/maps_expand_map_update.erl new file mode 100644 index 0000000000..df0f77ea47 --- /dev/null +++ b/lib/hipe/test/maps_SUITE_data/maps_expand_map_update.erl @@ -0,0 +1,7 @@ +-module(maps_expand_map_update). +-export([test/0]). + +test() -> + M = #{<<"hello">> => <<"world">>}#{<<"hello">> := <<"les gens">>}, + #{<<"hello">> := <<"les gens">>} = M, + ok. diff --git a/lib/hipe/test/maps_SUITE_data/maps_export.erl b/lib/hipe/test/maps_SUITE_data/maps_export.erl new file mode 100644 index 0000000000..4d43fc96ed --- /dev/null +++ b/lib/hipe/test/maps_SUITE_data/maps_export.erl @@ -0,0 +1,11 @@ +-module(maps_export). +-export([test/0]). + +test() -> + Raclette = id(#{}), + case brie of brie -> Fromage = Raclette end, + Raclette = Fromage#{}, + ok. + +%% Use this function to avoid compile-time evaluation of an expression. +id(I) -> I. diff --git a/lib/hipe/test/maps_SUITE_data/maps_get_map_elements.erl b/lib/hipe/test/maps_SUITE_data/maps_get_map_elements.erl new file mode 100644 index 0000000000..b2d749796a --- /dev/null +++ b/lib/hipe/test/maps_SUITE_data/maps_get_map_elements.erl @@ -0,0 +1,23 @@ +%% -*- erlang-indent-level: 2 -*- +%%------------------------------------------------------------------------- +-module(maps_get_map_elements). + +-export([test/0]). + +test() -> + {A, B} = id({"hej", <<123>>}), + Map = maps:from_list([{a, A}, {b, B}]), + #{a := A, b := B} = id(Map), + false = test_pattern(Map), + true = test_pattern(#{b => 1, a => "hej"}), + case Map of + #{a := C, b := <<124>>} -> yay; + _ -> C = B, nay + end, + C = id(B), + ok. + +id(X) -> X. + +test_pattern(#{a := _, b := 1}) -> true; +test_pattern(#{}) -> false. diff --git a/lib/hipe/test/maps_SUITE_data/maps_guard_bifs.erl b/lib/hipe/test/maps_SUITE_data/maps_guard_bifs.erl new file mode 100644 index 0000000000..61a0eaa1e7 --- /dev/null +++ b/lib/hipe/test/maps_SUITE_data/maps_guard_bifs.erl @@ -0,0 +1,31 @@ +-module(maps_guard_bifs). +-export([test/0]). + +test() -> + true = map_guard_empty(), + true = map_guard_empty_2(), + true = map_guard_head(#{a=>1}), + false = map_guard_head([]), + true = map_guard_body(#{a=>1}), + false = map_guard_body({}), + true = map_guard_pattern(#{a=>1, <<"hi">> => "hi" }), + false = map_guard_pattern("list"), + true = map_guard_tautology(), + true = map_guard_ill_map_size(), + ok. + +map_guard_empty() when is_map(#{}); false -> true. + +map_guard_empty_2() when true; #{} andalso false -> true. + +map_guard_head(M) when is_map(M) -> true; +map_guard_head(_) -> false. + +map_guard_body(M) -> is_map(M). + +map_guard_pattern(#{}) -> true; +map_guard_pattern(_) -> false. + +map_guard_tautology() when #{} =:= #{}; true -> true. + +map_guard_ill_map_size() when true; map_size(0) -> true. diff --git a/lib/hipe/test/maps_SUITE_data/maps_guard_fun.erl b/lib/hipe/test/maps_SUITE_data/maps_guard_fun.erl new file mode 100644 index 0000000000..9f6eb3a04e --- /dev/null +++ b/lib/hipe/test/maps_SUITE_data/maps_guard_fun.erl @@ -0,0 +1,36 @@ +-module(maps_guard_fun). +-export([test/0]). + +test() -> + F1 = fun + (#{s:=v,v:=V}) -> {v,V}; + (#{s:=t,v:={V,V}}) -> {t,V}; + (#{s:=l,v:=[V,V]}) -> {l,V} + end, + + F2 = fun + (#{s:=T,v:={V,V}}) -> {T,V}; + (#{s:=T,v:=[V,V]}) -> {T,V}; + (#{s:=T,v:=V}) -> {T,V} + end, + V = <<"hi">>, + + {v,V} = F1(#{s=>v,v=>V}), + {t,V} = F1(#{s=>t,v=>{V,V}}), + {l,V} = F1(#{s=>l,v=>[V,V]}), + + {v,V} = F2(#{s=>v,v=>V}), + {t,V} = F2(#{s=>t,v=>{V,V}}), + {l,V} = F2(#{s=>l,v=>[V,V]}), + + %% error case + case (catch F1(#{s=>none,v=>none})) of + {'EXIT', {function_clause,[{?MODULE,_,[#{s:=none,v:=none}],_}|_]}} -> ok; + {'EXIT', {function_clause,[{?MODULE,_,1,[#{s:=none,v:=none}]}|_]}} -> ok; + {'EXIT', {function_clause,[Frame|_]}} + when is_tuple(Frame), element(1, Frame) =:= ?MODULE -> + test_server:comment("Unexpected trace format, probably using HiPE"); + {'EXIT', {{case_clause,_},_}} -> {comment,inlined}; + Other -> + test_server:fail({no_match, Other}) + end. diff --git a/lib/hipe/test/maps_SUITE_data/maps_guard_receive.erl b/lib/hipe/test/maps_SUITE_data/maps_guard_receive.erl new file mode 100644 index 0000000000..f84ba19c86 --- /dev/null +++ b/lib/hipe/test/maps_SUITE_data/maps_guard_receive.erl @@ -0,0 +1,54 @@ +-module(maps_guard_receive). +-export([test/0]). + +test() -> + M0 = #{ id => 0 }, + Pid = spawn_link(fun() -> guard_receive_loop() end), + Big = 36893488147419103229, + B1 = <<"some text">>, + B2 = <<"was appended">>, + B3 = <<B1/binary, B2/binary>>, + + #{id:=1, res:=Big} = M1 = call(Pid, M0#{op=>sub,in=>{1 bsl 65, 3}}), + #{id:=2, res:=26} = M2 = call(Pid, M1#{op=>idiv,in=>{53,2}}), + #{id:=3, res:=832} = M3 = call(Pid, M2#{op=>imul,in=>{26,32}}), + #{id:=4, res:=4} = M4 = call(Pid, M3#{op=>add,in=>{1,3}}), + #{id:=5, res:=Big} = M5 = call(Pid, M4#{op=>sub,in=>{1 bsl 65, 3}}), + #{id:=6, res:=B3} = M6 = call(Pid, M5#{op=>"append",in=>{B1,B2}}), + #{id:=7, res:=4} = _ = call(Pid, M6#{op=>add,in=>{1,3}}), + + + %% update old maps and check id update + #{id:=2, res:=B3} = call(Pid, M1#{op=>"append",in=>{B1,B2}}), + #{id:=5, res:=99} = call(Pid, M4#{op=>add,in=>{33, 66}}), + + %% cleanup + done = call(Pid, done), + ok. + +call(Pid, M) -> + Pid ! {self(), M}, receive {Pid, Res} -> Res end. + +guard_receive_loop() -> + receive + {Pid, #{ id:=Id, op:="append", in:={X,Y}}=M} when is_binary(X), is_binary(Y) -> + Pid ! {self(), M#{ id=>Id+1, res=><<X/binary,Y/binary>>}}, + guard_receive_loop(); + {Pid, #{ id:=Id, op:=add, in:={X,Y}}} -> + Pid ! {self(), #{ id=>Id+1, res=>X+Y}}, + guard_receive_loop(); + {Pid, #{ id:=Id, op:=sub, in:={X,Y}}=M} -> + Pid ! {self(), M#{ id=>Id+1, res=>X-Y}}, + guard_receive_loop(); + {Pid, #{ id:=Id, op:=idiv, in:={X,Y}}=M} -> + Pid ! {self(), M#{ id=>Id+1, res=>X div Y}}, + guard_receive_loop(); + {Pid, #{ id:=Id, op:=imul, in:={X,Y}}=M} -> + Pid ! {self(), M#{ id=>Id+1, res=>X * Y}}, + guard_receive_loop(); + {Pid, done} -> + Pid ! {self(), done}; + {Pid, Other} -> + Pid ! {error, Other}, + guard_receive_loop() + end. diff --git a/lib/hipe/test/maps_SUITE_data/maps_guard_sequence.erl b/lib/hipe/test/maps_SUITE_data/maps_guard_sequence.erl new file mode 100644 index 0000000000..4eb18dcea1 --- /dev/null +++ b/lib/hipe/test/maps_SUITE_data/maps_guard_sequence.erl @@ -0,0 +1,35 @@ +-module(maps_guard_sequence). +-export([test/0]). + +test() -> + {1, "a"} = map_guard_sequence_1(#{seq=>1,val=>id("a")}), + {2, "b"} = map_guard_sequence_1(#{seq=>2,val=>id("b")}), + {3, "c"} = map_guard_sequence_1(#{seq=>3,val=>id("c")}), + {4, "d"} = map_guard_sequence_1(#{seq=>4,val=>id("d")}), + {5, "e"} = map_guard_sequence_1(#{seq=>5,val=>id("e")}), + + {1,M1} = map_guard_sequence_2(M1 = id(#{a=>3})), + {2,M2} = map_guard_sequence_2(M2 = id(#{a=>4, b=>4})), + {3,gg,M3} = map_guard_sequence_2(M3 = id(#{a=>gg, b=>4})), + {4,sc,sc,M4} = map_guard_sequence_2(M4 = id(#{a=>sc, b=>3, c=>sc2})), + {5,kk,kk,M5} = map_guard_sequence_2(M5 = id(#{a=>kk, b=>other, c=>sc2})), + + %% error case + {'EXIT',{function_clause,_}} = (catch map_guard_sequence_1(#{seq=>6,val=>id("e")})), + {'EXIT',{function_clause,_}} = (catch map_guard_sequence_2(#{b=>5})), + ok. + +map_guard_sequence_1(#{seq:=1=Seq, val:=Val}) -> {Seq,Val}; +map_guard_sequence_1(#{seq:=2=Seq, val:=Val}) -> {Seq,Val}; +map_guard_sequence_1(#{seq:=3=Seq, val:=Val}) -> {Seq,Val}; +map_guard_sequence_1(#{seq:=4=Seq, val:=Val}) -> {Seq,Val}; +map_guard_sequence_1(#{seq:=5=Seq, val:=Val}) -> {Seq,Val}. + +map_guard_sequence_2(#{ a:=3 }=M) -> {1, M}; +map_guard_sequence_2(#{ a:=4 }=M) -> {2, M}; +map_guard_sequence_2(#{ a:=X, a:=X, b:=4 }=M) -> {3,X,M}; +map_guard_sequence_2(#{ a:=X, a:=Y, b:=3 }=M) when X =:= Y -> {4,X,Y,M}; +map_guard_sequence_2(#{ a:=X, a:=Y }=M) when X =:= Y -> {5,X,Y,M}. + +%% Use this function to avoid compile-time evaluation of an expression. +id(I) -> I. diff --git a/lib/hipe/test/maps_SUITE_data/maps_guard_update.erl b/lib/hipe/test/maps_SUITE_data/maps_guard_update.erl new file mode 100644 index 0000000000..254c1c2984 --- /dev/null +++ b/lib/hipe/test/maps_SUITE_data/maps_guard_update.erl @@ -0,0 +1,14 @@ +-module(maps_guard_update). +-export([test/0]). + +test() -> + error = map_guard_update(#{},#{}), + first = map_guard_update(#{}, #{x=>first}), + second = map_guard_update(#{y=>old}, #{x=>second,y=>old}), + third = map_guard_update(#{x=>old,y=>old}, #{x=>third,y=>old}), + ok. + +map_guard_update(M1, M2) when M1#{x=>first} =:= M2 -> first; +map_guard_update(M1, M2) when M1#{x=>second} =:= M2 -> second; +map_guard_update(M1, M2) when M1#{x:=third} =:= M2 -> third; +map_guard_update(_, _) -> error. diff --git a/lib/hipe/test/maps_SUITE_data/maps_has_map_fields.erl b/lib/hipe/test/maps_SUITE_data/maps_has_map_fields.erl new file mode 100644 index 0000000000..61653aa519 --- /dev/null +++ b/lib/hipe/test/maps_SUITE_data/maps_has_map_fields.erl @@ -0,0 +1,46 @@ +%% -*- erlang-indent-level: 2 -*- +%%------------------------------------------------------------------------- +-module(maps_has_map_fields). + +-export([test/0]). + +test() -> + false = has_a_field(#{}), + false = has_a_field(#{b => 2}), + true = has_a_field(#{a => 3}), + true = has_a_field(#{b => c, a => false}), + + false = has_a_b_field(#{a => true}), + false = has_a_b_field(#{b => a}), + true = has_a_b_field(#{a => 1, b => 2}), + true = has_a_b_field(#{b => 3, a => 4}), + + false = has_binary_field(#{}), + false = has_binary_field(#{#{} => yay}), + true = has_binary_field(#{<<"true">> => false}), + + false = has_binary_but_no_map_field(#{}), + false = has_map_but_no_binary_field(#{}), + false = has_binary_but_no_map_field(#{#{} => 1}), + false = has_map_but_no_binary_field(#{<<"true">> => true}), + true = has_binary_but_no_map_field(#{<<"true">> => false}), + true = has_map_but_no_binary_field(#{#{} => 1}), + false = has_binary_but_no_map_field(#{<<"true">> => true, #{} => 1}), + false = has_map_but_no_binary_field(#{<<"true">> => true, #{} => 1}), + ok. + +has_a_field(#{a := _}) -> true; +has_a_field(#{}) -> false. + +has_a_b_field(#{a := _, b := _}) -> true; +has_a_b_field(#{}) -> false. + +has_binary_field(#{<<"true">> := _}) -> true; +has_binary_field(#{}) -> false. + +has_map_but_no_binary_field(#{<<"true">> := _}) -> false; +has_map_but_no_binary_field(#{} = M) -> maps:is_key(#{}, M). + +has_binary_but_no_map_field(#{<<"true">> := _} = M) -> + not maps:is_key(#{}, M); +has_binary_but_no_map_field(#{}) -> false. diff --git a/lib/hipe/test/maps_SUITE_data/maps_is_map.erl b/lib/hipe/test/maps_SUITE_data/maps_is_map.erl new file mode 100644 index 0000000000..e84f4b8c44 --- /dev/null +++ b/lib/hipe/test/maps_SUITE_data/maps_is_map.erl @@ -0,0 +1,24 @@ +%% -*- erlang-indent-level: 2 -*- +%%------------------------------------------------------------------------- +-module(maps_is_map). + +-export([test/0]). + +test() -> + true = test_is_map(#{}), + false = test_is_map(<<"hej">>), + true = test_is_map_guard(#{a => b}), + false = test_is_map_guard(3), + true = test_is_map_with_binary_guard(#{"a" => <<"b">>}), + false = test_is_map_with_binary_guard(12), + ok. + +test_is_map(X) -> + is_map(X). + +test_is_map_guard(Map) when is_map(Map) -> true; +test_is_map_guard(_) -> false. + +test_is_map_with_binary_guard(B) when is_binary(B) -> false; +test_is_map_with_binary_guard(#{}) -> true; +test_is_map_with_binary_guard(_) -> false. diff --git a/lib/hipe/test/maps_SUITE_data/maps_list_comprehension.erl b/lib/hipe/test/maps_SUITE_data/maps_list_comprehension.erl new file mode 100644 index 0000000000..ad2c726d65 --- /dev/null +++ b/lib/hipe/test/maps_SUITE_data/maps_list_comprehension.erl @@ -0,0 +1,6 @@ +-module(maps_list_comprehension). +-export([test/0]). + +test() -> + [#{k:=1},#{k:=2},#{k:=3}] = [#{k=>I} || I <- [1,2,3]], + ok. diff --git a/lib/hipe/test/maps_SUITE_data/maps_map_size.erl b/lib/hipe/test/maps_SUITE_data/maps_map_size.erl new file mode 100644 index 0000000000..25c8e5d4c7 --- /dev/null +++ b/lib/hipe/test/maps_SUITE_data/maps_map_size.erl @@ -0,0 +1,29 @@ +-module(maps_map_size). +-export([test/0]). + +test() -> + 0 = map_size(id(#{})), + 1 = map_size(id(#{a=>1})), + 1 = map_size(id(#{a=>"wat"})), + 2 = map_size(id(#{a=>1, b=>2})), + 3 = map_size(id(#{a=>1, b=>2, b=>"3","33"=><<"n">>})), + + true = map_is_size(#{a=>1}, 1), + true = map_is_size(#{a=>1, a=>2}, 1), + M = #{ "a" => 1, "b" => 2}, + true = map_is_size(M, 2), + false = map_is_size(M, 3), + true = map_is_size(M#{ "a" => 2}, 2), + false = map_is_size(M#{ "c" => 2}, 2), + + %% Error cases. + {'EXIT',{badarg,_}} = (catch map_size([])), + {'EXIT',{badarg,_}} = (catch map_size(<<1,2,3>>)), + {'EXIT',{badarg,_}} = (catch map_size(1)), + ok. + +map_is_size(M,N) when map_size(M) =:= N -> true; +map_is_size(_,_) -> false. + +%% Use this function to avoid compile-time evaluation of an expression. +id(I) -> I. diff --git a/lib/hipe/test/maps_SUITE_data/maps_map_sort_literals.erl b/lib/hipe/test/maps_SUITE_data/maps_map_sort_literals.erl new file mode 100644 index 0000000000..31abf15d49 --- /dev/null +++ b/lib/hipe/test/maps_SUITE_data/maps_map_sort_literals.erl @@ -0,0 +1,41 @@ +-module(maps_map_sort_literals). +-export([test/0]). + +test() -> + % test relation + + %% size order + true = #{ a => 1, b => 2} < id(#{ a => 1, b => 1, c => 1}), + true = #{ b => 1, a => 1} < id(#{ c => 1, a => 1, b => 1}), + false = #{ c => 1, b => 1, a => 1} < id(#{ c => 1, a => 1}), + + %% key order + true = id(#{ a => 1 }) < id(#{ b => 1}), + false = id(#{ b => 1 }) < id(#{ a => 1}), + true = id(#{ a => 1, b => 1, c => 1 }) < id(#{ b => 1, c => 1, d => 1}), + true = id(#{ b => 1, c => 1, d => 1 }) > id(#{ a => 1, b => 1, c => 1}), + true = id(#{ c => 1, b => 1, a => 1 }) < id(#{ b => 1, c => 1, d => 1}), + true = id(#{ "a" => 1 }) < id(#{ <<"a">> => 1}), + false = id(#{ <<"a">> => 1 }) < id(#{ "a" => 1}), + false = id(#{ 1 => 1 }) < id(#{ 1.0 => 1}), + false = id(#{ 1.0 => 1 }) < id(#{ 1 => 1}), + + %% value order + true = id(#{ a => 1 }) < id(#{ a => 2}), + false = id(#{ a => 2 }) < id(#{ a => 1}), + false = id(#{ a => 2, b => 1 }) < id(#{ a => 1, b => 3}), + true = id(#{ a => 1, b => 1 }) < id(#{ a => 1, b => 3}), + + true = id(#{ "a" => "hi", b => 134 }) == id(#{ b => 134,"a" => "hi"}), + + %% lists:sort + + SortVs = [#{"a"=>1},#{a=>2},#{1=>3},#{<<"a">>=>4}], + [#{1:=ok},#{a:=ok},#{"a":=ok},#{<<"a">>:=ok}] = lists:sort([#{"a"=>ok},#{a=>ok},#{1=>ok},#{<<"a">>=>ok}]), + [#{1:=3},#{a:=2},#{"a":=1},#{<<"a">>:=4}] = lists:sort(SortVs), + [#{1:=3},#{a:=2},#{"a":=1},#{<<"a">>:=4}] = lists:sort(lists:reverse(SortVs)), + + ok. + +%% Use this function to avoid compile-time evaluation of an expression. +id(I) -> I. diff --git a/lib/hipe/test/maps_SUITE_data/maps_match_and_update_literals.erl b/lib/hipe/test/maps_SUITE_data/maps_match_and_update_literals.erl new file mode 100644 index 0000000000..29a6a29290 --- /dev/null +++ b/lib/hipe/test/maps_SUITE_data/maps_match_and_update_literals.erl @@ -0,0 +1,24 @@ +-module(maps_match_and_update_literals). +-export([test/0]). + +test() -> + Map = #{x=>0,y=>"untouched",z=>"also untouched",q=>1}, + #{x:=16,q:=21,y:="untouched",z:="also untouched"} = loop_match_and_update_literals_x_q(Map, [ + {1,2},{3,4},{5,6},{7,8} + ]), + M0 = id(#{ "hi" => "hello", int => 3, <<"key">> => <<"value">>, + 4 => number, 18446744073709551629 => wat}), + M1 = id(#{}), + M2 = M1#{ "hi" => "hello", int => 3, <<"key">> => <<"value">>, + 4 => number, 18446744073709551629 => wat}, + M0 = M2, + + #{ 4 := another_number, int := 3 } = M2#{ 4 => another_number }, + ok. + +loop_match_and_update_literals_x_q(Map, []) -> Map; +loop_match_and_update_literals_x_q(#{q:=Q0,x:=X0} = Map, [{X,Q}|Vs]) -> + loop_match_and_update_literals_x_q(Map#{q=>Q0+Q,x=>X0+X},Vs). + +%% Use this function to avoid compile-time evaluation of an expression. +id(I) -> I. diff --git a/lib/hipe/test/maps_SUITE_data/maps_put_map_assoc.erl b/lib/hipe/test/maps_SUITE_data/maps_put_map_assoc.erl new file mode 100644 index 0000000000..72ac9ce078 --- /dev/null +++ b/lib/hipe/test/maps_SUITE_data/maps_put_map_assoc.erl @@ -0,0 +1,23 @@ +%% -*- erlang-indent-level: 2 -*- +%%------------------------------------------------------------------------- +-module(maps_put_map_assoc). + +-export([test/0]). + +test() -> + true = assoc_guard(#{}), + false = assoc_guard(not_a_map), + #{a := true} = assoc_update(#{}), + {'EXIT', {badarg, [{?MODULE, assoc_update, 1, _}|_]}} + = (catch assoc_update(not_a_map)), + ok = assoc_guard_clause(#{}), + {'EXIT', {function_clause, [{?MODULE, assoc_guard_clause, _, _}|_]}} + = (catch assoc_guard_clause(not_a_map)), + ok. + +assoc_guard(M) when is_map(M#{a => b}) -> true; +assoc_guard(_) -> false. + +assoc_update(M) -> M#{a => true}. + +assoc_guard_clause(M) when is_map(M#{a => 3}) -> ok. diff --git a/lib/hipe/test/maps_SUITE_data/maps_put_map_exact.erl b/lib/hipe/test/maps_SUITE_data/maps_put_map_exact.erl new file mode 100644 index 0000000000..1cfcd80180 --- /dev/null +++ b/lib/hipe/test/maps_SUITE_data/maps_put_map_exact.erl @@ -0,0 +1,28 @@ +%% -*- erlang-indent-level: 2 -*- +%%------------------------------------------------------------------------- +-module(maps_put_map_exact). + +-export([test/0]). + +test() -> + false = exact_guard(#{b => a}), + false = exact_guard(not_a_map), + true = exact_guard(#{a => false}), + #{a := true} = exact_update(#{a => false}), + {'EXIT', {badarg, [{?MODULE, exact_update, 1, _}|_]}} + = (catch exact_update(not_a_map)), + {'EXIT', {badarg, [{?MODULE, exact_update, 1, _}|_]}} + = (catch exact_update(#{})), + ok = exact_guard_clause(#{a => yes}), + {'EXIT', {function_clause, [{?MODULE, exact_guard_clause, _, _}|_]}} + = (catch exact_guard_clause(#{})), + {'EXIT', {function_clause, [{?MODULE, exact_guard_clause, _, _}|_]}} + = (catch exact_guard_clause(not_a_map)), + ok. + +exact_guard(M) when is_map(M#{a := b}) -> true; +exact_guard(_) -> false. + +exact_update(M) -> M#{a := true}. + +exact_guard_clause(M) when is_map(M#{a := 42}) -> ok. diff --git a/lib/hipe/test/maps_SUITE_data/maps_update_assoc.erl b/lib/hipe/test/maps_SUITE_data/maps_update_assoc.erl new file mode 100644 index 0000000000..cc7c1353de --- /dev/null +++ b/lib/hipe/test/maps_SUITE_data/maps_update_assoc.erl @@ -0,0 +1,22 @@ +-module(maps_update_assoc). +-export([test/0]). + +test() -> + M0 = id(#{1=>a,2=>b,3.0=>c,4=>d,5=>e}), + + M1 = M0#{1=>42,2=>100,4=>[a,b,c]}, + #{1:=42,2:=100,3.0:=c,4:=[a,b,c],5:=e} = M1, + #{1:=42,2:=b,4:=d,5:=e,2.0:=100,3.0:=c,4.0:=[a,b,c]} = M0#{1.0=>float,1:=42,2.0=>wrong,2.0=>100,4.0=>[a,b,c]}, + + M2 = M0#{3.0=>new}, + #{1:=a,2:=b,3.0:=new,4:=d,5:=e} = M2, + M2 = M0#{3.0:=wrong,3.0=>new}, + + %% Errors cases. + BadMap = id(badmap), + {'EXIT',{badarg,_}} = (catch BadMap#{nonexisting=>val}), + + ok. + +%% Use this function to avoid compile-time evaluation of an expression. +id(I) -> I. diff --git a/lib/hipe/test/maps_SUITE_data/maps_update_exact.erl b/lib/hipe/test/maps_SUITE_data/maps_update_exact.erl new file mode 100644 index 0000000000..6e5acb3283 --- /dev/null +++ b/lib/hipe/test/maps_SUITE_data/maps_update_exact.erl @@ -0,0 +1,32 @@ +-module(maps_update_exact). +-export([test/0]). + +test() -> + M0 = id(#{1=>a,2=>b,3.0=>c,4=>d,5=>e}), + + M1 = M0#{1:=42,2:=100,4:=[a,b,c]}, + #{1:=42,2:=100,3.0:=c,4:=[a,b,c],5:=e} = M1, + M1 = M0#{1:=wrong,1=>42,2=>wrong,2:=100,4:=[a,b,c]}, + + M2 = M0#{3.0:=new}, + #{1:=a,2:=b,3.0:=new,4:=d,5:=e} = M2, + M2 = M0#{3.0=>wrong,3.0:=new}, + true = M2 =/= M0#{3=>right,3.0:=new}, + #{ 3 := right, 3.0 := new } = M0#{3=>right,3.0:=new}, + + M3 = id(#{ 1 => val}), + #{1 := update2,1.0 := new_val4} = M3#{ + 1.0 => new_val1, 1 := update, 1=> update3, + 1 := update2, 1.0 := new_val2, 1.0 => new_val3, + 1.0 => new_val4 }, + + %% Errors cases. + {'EXIT',{badarg,_}} = (catch ((id(nil))#{ a := b })), + {'EXIT',{badarg,_}} = (catch M0#{nonexisting:=val}), + {'EXIT',{badarg,_}} = (catch M0#{1.0:=v,1.0=>v2}), + {'EXIT',{badarg,_}} = (catch M0#{42.0:=v,42:=v2}), + {'EXIT',{badarg,_}} = (catch M0#{42=>v1,42.0:=v2,42:=v3}), + ok. + +%% Use this function to avoid compile-time evaluation of an expression. +id(I) -> I. diff --git a/lib/hipe/test/maps_SUITE_data/maps_update_literals.erl b/lib/hipe/test/maps_SUITE_data/maps_update_literals.erl new file mode 100644 index 0000000000..87aea3d8e1 --- /dev/null +++ b/lib/hipe/test/maps_SUITE_data/maps_update_literals.erl @@ -0,0 +1,13 @@ +-module(maps_update_literals). +-export([test/0]). + +test() -> + Map = #{x=>1,y=>2,z=>3,q=>4}, + #{x:="d",q:="4"} = loop_update_literals_x_q(Map, [ + {"a","1"},{"b","2"},{"c","3"},{"d","4"} + ]), + ok. + +loop_update_literals_x_q(Map, []) -> Map; +loop_update_literals_x_q(Map, [{X,Q}|Vs]) -> + loop_update_literals_x_q(Map#{q=>Q,x=>X},Vs). diff --git a/lib/hipe/test/maps_SUITE_data/maps_update_map_expressions.erl b/lib/hipe/test/maps_SUITE_data/maps_update_map_expressions.erl new file mode 100644 index 0000000000..181e3f18f7 --- /dev/null +++ b/lib/hipe/test/maps_SUITE_data/maps_update_map_expressions.erl @@ -0,0 +1,32 @@ +-module(maps_update_map_expressions). +-export([test/0]). + +test() -> + M = maps:new(), + X = id(fondue), + M1 = #{ a := 1 } = M#{a => 1}, + #{ b := {X} } = M1#{ a := 1, b => {X} }, + + #{ b := 2 } = (maps:new())#{ b => 2 }, + + #{ a :=42, b:=42, c:=42 } = (maps:from_list([{a,1},{b,2},{c,3}]))#{ a := 42, b := 42, c := 42 }, + #{ "a" :=1, "b":=42, "c":=42 } = (maps:from_list([{"a",1},{"b",2}]))#{ "b" := 42, "c" => 42 }, + + %% Test need to be in a fun. + %% This tests that let expr optimisation in sys_core_fold + %% covers maps correctly. + F = fun() -> + M0 = id(#{ "a" => [1,2,3] }), + #{ "a" := _ } = M0, + M0#{ "a" := b } + end, + + #{ "a" := b } = F(), + + %% Error cases, FIXME: should be 'badmap'? + {'EXIT',{badarg,_}} = (catch (id(<<>>))#{ a := 42, b => 2 }), + {'EXIT',{badarg,_}} = (catch (id([]))#{ a := 42, b => 2 }), + ok. + +%% Use this function to avoid compile-time evaluation of an expression. +id(I) -> I. diff --git a/lib/hipe/test/maps_SUITE_data/maps_update_values.erl b/lib/hipe/test/maps_SUITE_data/maps_update_values.erl new file mode 100644 index 0000000000..bbad5ac19e --- /dev/null +++ b/lib/hipe/test/maps_SUITE_data/maps_update_values.erl @@ -0,0 +1,28 @@ +-module(maps_update_values). +-export([test/0]). + +test() -> + V0 = id(1337), + M0 = #{ a => 1, val => V0}, + V1 = get_val(M0), + M1 = M0#{ val := [V0,V1], "wazzup" => 42 }, + [1337, {some_val, 1337}] = get_val(M1), + + N = 110, + List = [{[I,1,2,3,I],{1,2,3,"wat",I}}|| I <- lists:seq(1,N)], + + {_,_,#{val2 := {1,2,3,"wat",N}, val1 := [N,1,2,3,N]}} = lists:foldl(fun + ({V2,V3},{Old2,Old3,Mi}) -> + ok = check_val(Mi,Old2,Old3), + #{ val1 := Old2, val2 := Old3 } = Mi, + {V2,V3, Mi#{ val1 := id(V2), val2 := V1, val2 => id(V3)}} + end, {none, none, #{val1=>none,val2=>none}},List), + ok. + +get_val(#{ "wazzup" := _, val := V}) -> V; +get_val(#{ val := V }) -> {some_val, V}. + +check_val(#{val1:=V1, val2:=V2},V1,V2) -> ok. + +%% Use this function to avoid compile-time evaluation of an expression. +id(I) -> I. diff --git a/lib/hipe/test/maps_SUITE_data/maps_warn_pair_key_overloaded.erl b/lib/hipe/test/maps_SUITE_data/maps_warn_pair_key_overloaded.erl new file mode 100644 index 0000000000..76b2a91f94 --- /dev/null +++ b/lib/hipe/test/maps_SUITE_data/maps_warn_pair_key_overloaded.erl @@ -0,0 +1,27 @@ +-module(maps_warn_pair_key_overloaded). +-export([test/0]). + +test() -> + #{ "hi1" := 42 } = id(#{ "hi1" => 1, "hi1" => 42 }), + + #{ "hi1" := 1337, "hi2" := [2], "hi3" := 3 } = id(#{ + "hi1" => erlang:atom_to_binary(?MODULE,utf8), + "hi1" => erlang:binary_to_atom(<<"wazzup">>,utf8), + "hi1" => erlang:binary_to_float(<<"3.1416">>), + "hi1" => erlang:float_to_binary(3.1416), + "hi2" => erlang:pid_to_list(self()), + "hi3" => erlang:float_to_binary(3.1416), + "hi2" => lists:subtract([1,2],[1]), + "hi3" => +3, + "hi1" => erlang:min(1,2), + "hi1" => erlang:hash({1,2},35), + "hi1" => erlang:phash({1,2},33), + "hi1" => erlang:phash2({1,2},34), + "hi1" => erlang:integer_to_binary(1337), + "hi1" => erlang:binary_to_integer(<<"1337">>), + "hi4" => erlang:float_to_binary(3.1416) + }), + ok. + +%% Use this function to avoid compile-time evaluation of an expression. +id(I) -> I. diff --git a/lib/hipe/test/maps_SUITE_data/maps_warn_useless_build.erl b/lib/hipe/test/maps_SUITE_data/maps_warn_useless_build.erl new file mode 100644 index 0000000000..6cb0366314 --- /dev/null +++ b/lib/hipe/test/maps_SUITE_data/maps_warn_useless_build.erl @@ -0,0 +1,9 @@ +-module(maps_warn_useless_build). +-export([test/0]). + +test() -> + [#{ a => id(I)} || I <- [1,2,3]], + ok. + +%% Use this function to avoid compile-time evaluation of an expression. +id(I) -> I. diff --git a/lib/hipe/vsn.mk b/lib/hipe/vsn.mk index fb7e4b91a0..cf1976d8d6 100644 --- a/lib/hipe/vsn.mk +++ b/lib/hipe/vsn.mk @@ -1 +1 @@ -HIPE_VSN = 3.10.3 +HIPE_VSN = 3.11.1 diff --git a/lib/ic/c_src/oe_ei_encode_pid.c b/lib/ic/c_src/oe_ei_encode_pid.c index b7083f84a0..609f441cf8 100644 --- a/lib/ic/c_src/oe_ei_encode_pid.c +++ b/lib/ic/c_src/oe_ei_encode_pid.c @@ -23,7 +23,7 @@ int oe_ei_encode_pid(CORBA_Environment *ev, const erlang_pid *p) { int size = ev->_iout; - (int) ei_encode_pid(NULL, &size, p); + ei_encode_pid(NULL, &size, p); if (size >= ev->_outbufsz) { char *buf = ev->_outbuf; diff --git a/lib/ic/doc/src/notes.xml b/lib/ic/doc/src/notes.xml index 9ae464e028..bacac09f11 100644 --- a/lib/ic/doc/src/notes.xml +++ b/lib/ic/doc/src/notes.xml @@ -30,7 +30,38 @@ <file>notes.xml</file> </header> - <section><title>IC 4.3.4</title> + <section><title>IC 4.3.6</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fix compiler warnings reported by LLVM</p> + <p> + Own Id: OTP-12138</p> + </item> + </list> + </section> + +</section> + +<section><title>IC 4.3.5</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> Added Latin-1 code directive in the generated files + to keep old behaviour. Updated IC so it can handle + Unicode characters in the path. </p> + <p> + Own Id: OTP-11783</p> + </item> + </list> + </section> + +</section> + +<section><title>IC 4.3.4</title> <section><title>Fixed Bugs and Malfunctions</title> <list> diff --git a/lib/ic/src/ic.erl b/lib/ic/src/ic.erl index c0742cf7bd..4f3e8d180c 100644 --- a/lib/ic/src/ic.erl +++ b/lib/ic/src/ic.erl @@ -250,7 +250,7 @@ make_erl_options(Opts) -> Optimize = Opts#options.optimize, PreProc = lists:flatten( - lists:map(fun(D) -> io_lib:format("-I\"~s\" ", [ic_util:to_list(D)]) end, + lists:map(fun(D) -> io_lib:format("-I\"~ts\" ", [ic_util:to_list(D)]) end, Includes1)++ lists:map( fun ({Name, Value}) -> diff --git a/lib/ic/src/ic_codegen.erl b/lib/ic/src/ic_codegen.erl index f611c69bea..82431b79a2 100644 --- a/lib/ic/src/ic_codegen.erl +++ b/lib/ic/src/ic_codegen.erl @@ -216,12 +216,14 @@ exp_to_string({F,N}) -> io_lib:format("~p/~p", [ic_util:to_atom(F), N]). %%-------------------------------------------------------------------- emit_stub_head(_G, ignore, _Name, _) -> ignore; emit_stub_head(G, F1, Name, erlang) -> + comment(F1, " coding: latin-1", []), mcomment(F1, stub_header(G, Name)), nl(F1), emit(F1, "-module(~p).\n", [list_to_atom(Name)]), emit(F1, "-ic_compiled(~p).\n", [compiler_vsn(?COMPILERVSN)]), emit(F1, "\n\n"), F1; emit_stub_head(G, F1, Name, erlang_template) -> + comment(F1, " coding: latin-1", []), ic_erl_template:emit_header(G, F1, Name), F1; emit_stub_head(_G, F1, _Name, erlang_template_no_gen) -> @@ -259,6 +261,7 @@ compiler_vsn(Vsn) -> %% Name is Fully scoped (undescore) name of interface or module emit_hrl_head(_G, ignore, _Name, _) -> ignore; emit_hrl_head(G, Fd, Name, erlang) -> + comment(Fd, " coding: latin-1", []), mcomment(Fd, ["Erlang header file" | hrl_header(G, Name)]), nl(Fd), diff --git a/lib/ic/src/ic_pp.erl b/lib/ic/src/ic_pp.erl index 54701f7438..e54304ebaa 100644 --- a/lib/ic/src/ic_pp.erl +++ b/lib/ic/src/ic_pp.erl @@ -257,15 +257,15 @@ run_include(FileName, FileList, _Out, Defs, Err, War, IncLine, IncFile, IncDir, tokenise(File, FileName) -> {Result, _L} = token(File, 2, [], not_set, 0), - FI_start = lists:reverse(lists:flatten(io_lib:format("# 1 ~p~n",[FileName]))), + FI_start = lists:reverse(lists:flatten(io_lib:format("# 1 \"~ts\"~n",[FileName]))), FileInfoStart = {file_info, FI_start}, [FileInfoStart | Result]. tokenise(File, FileName, IncLine, PrevFile) -> {Result, _L} = token(File, 2, [], not_set, 0), - FI_start = lists:reverse(lists:flatten(io_lib:format("# 1 ~p 1~n",[FileName]))), + FI_start = lists:reverse(lists:flatten(io_lib:format("# 1 \"~ts\" 1~n",[FileName]))), FileInfoStart = {file_info, FI_start}, - FI_end = lists:reverse(lists:flatten(io_lib:format("# ~p ~p 2~n~n",[IncLine-1,PrevFile]))), + FI_end = lists:reverse(lists:flatten(io_lib:format("# ~p \"~ts\" 2~n~n",[IncLine-1,PrevFile]))), FileInfoEnd = [{file_info, FI_end}], {Result, FileInfoStart, FileInfoEnd}. % [FileInfoStart | Result] ++ FileInfoEnd. @@ -1942,7 +1942,7 @@ read_inc_file(FileName, IncDir, Mio) -> FileList = binary_to_list(Bin), {ok, AbsFile, FileList}; {error, Text} -> - {error, Text} + {error, Text} end; true -> skip diff --git a/lib/ic/test/java_client_erl_server_SUITE.erl b/lib/ic/test/java_client_erl_server_SUITE.erl index 9e49305c3c..cbcf32515e 100644 --- a/lib/ic/test/java_client_erl_server_SUITE.erl +++ b/lib/ic/test/java_client_erl_server_SUITE.erl @@ -288,7 +288,7 @@ classpath(Dir) -> cmd(Cmd) -> PortOpts = [{line,80},eof,exit_status,stderr_to_stdout], - io:format("<cmd> ~s~n", [Cmd]), + io:format("<cmd> ~ts~n", [Cmd]), case catch open_port({spawn,Cmd}, PortOpts) of Port when is_port(Port) -> Result = cmd_loop(Port, []), diff --git a/lib/ic/vsn.mk b/lib/ic/vsn.mk index 2ffbbad444..bb273c7b57 100644 --- a/lib/ic/vsn.mk +++ b/lib/ic/vsn.mk @@ -1 +1 @@ -IC_VSN = 4.3.5 +IC_VSN = 4.3.6 diff --git a/lib/inets/doc/src/httpc.xml b/lib/inets/doc/src/httpc.xml index 37eb7ba718..06cb035370 100644 --- a/lib/inets/doc/src/httpc.xml +++ b/lib/inets/doc/src/httpc.xml @@ -332,7 +332,7 @@ filename() = string() <p>Defaults to <c>true</c>. </p> </item> - <tag><c><![CDATA[header_as_is]]></c></tag> + <tag><c><![CDATA[headers_as_is]]></c></tag> <item> <p>Shall the headers provided by the user be made lower case or be regarded as case sensitive. </p> diff --git a/lib/inets/doc/src/httpd.xml b/lib/inets/doc/src/httpd.xml index 3830b2e5ab..4ca038cc99 100644 --- a/lib/inets/doc/src/httpd.xml +++ b/lib/inets/doc/src/httpd.xml @@ -139,7 +139,7 @@ <marker id="prop_server_root"></marker> <tag>{server_root, path()} </tag> <item> - <p>Defines the servers home directory where log files etc can + <p>Defines the server's home directory where log files etc can be stored. Relative paths specified in other properties refer to this directory. </p> </item> @@ -904,7 +904,7 @@ bytes <p>Fetches information about the HTTP server. When called with only the pid all properties are fetched, when called with a list of specific properties they are fetched. - Available properties are the same as the servers start options. + Available properties are the same as the server's start options. </p> <note><p>Pid is the pid returned from inets:start/[2,3]. @@ -930,7 +930,7 @@ bytes <p>Fetches information about the HTTP server. When called with only the Address and Port all properties are fetched, when called with a list of specific properties they are fetched. - Available properties are the same as the servers start + Available properties are the same as the server's start options. </p> @@ -956,7 +956,7 @@ bytes server. Incoming requests will be answered with a temporary down message during the time the it takes to reload.</p> - <note><p>Available properties are the same as the servers + <note><p>Available properties are the same as the server's start options, although the properties bind_address and port can not be changed.</p></note> @@ -1068,7 +1068,7 @@ bytes <type> <v>OldData = list()</v> <v>NewData = [{response,{StatusCode,Body}}] | [{response,{response,Head,Body}}] | [{response,{already_sent,Statuscode,Size}}] </v> - <v>StausCode = integer()</v> + <v>StatusCode = integer()</v> <v>Body = io_list() | nobody | {Fun, Arg}</v> <v>Head = [HeaderOption]</v> <v>HeaderOption = {Option, Value} | {code, StatusCode}</v> diff --git a/lib/inets/doc/src/notes.xml b/lib/inets/doc/src/notes.xml index f77214c589..921de8e490 100644 --- a/lib/inets/doc/src/notes.xml +++ b/lib/inets/doc/src/notes.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2002</year><year>2013</year> + <year>2002</year><year>2014</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -32,7 +32,194 @@ <file>notes.xml</file> </header> - <section><title>Inets 5.9.8</title> + <section><title>Inets 5.10.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fix some spelling mistakes in documentation</p> + <p> + Own Id: OTP-12152</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + httpd: Seperate timeout for TLS/SSL handshake from + keepalive timeout</p> + <p> + Own Id: OTP-12013</p> + </item> + <item> + <p> + Warning: this is experimental and may disappear or change + without previous warning.</p> + <p> + Experimental support for running Quickcheck and PropEr + tests from common_test suites is added to common_test. + See the reference manual for the new module + <c>ct_property_testing</c>.</p> + <p> + Experimental property tests are added under + <c>lib/{inet,ssh}/test/property_test</c>. They can be run + directly or from the commont_test suites + <c>inet/ftp_property_test_SUITE.erl</c> and + <c>ssh/test/ssh_property_test_SUITE.erl</c>.</p> + <p> + See the code in the <c>test</c> directories and the man + page for details.</p> + <p> + (Thanks to Tuncer Ayaz for a patch adding Triq)</p> + <p> + Own Id: OTP-12119</p> + </item> + </list> + </section> + +</section> + +<section><title>Inets 5.10.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + httpc: Fix streaming bugs when handling small responses</p> + <p> + Own Id: OTP-11992</p> + </item> + </list> + </section> + +</section> + +<section><title>Inets 5.10.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Correct distirbing mode for httpd:reload_config/2</p> + <p> + Own Id: OTP-11914</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Improved handling of invalid strings in the HTTP request + line.</p> + <p> + Impact: May improve memory consumption</p> + <p> + Own Id: OTP-11925 Aux Id: Sequence 12601 </p> + </item> + </list> + </section> + +</section> + +<section><title>Inets 5.10</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fixed a spelling mistake in httpc doc (Thanks to Wasif + Riaz Malik)</p> + <p> + Own Id: OTP-11538</p> + </item> + <item> + <p> + Application upgrade (appup) files are corrected for the + following applications: </p> + <p> + <c>asn1, common_test, compiler, crypto, debugger, + dialyzer, edoc, eldap, erl_docgen, et, eunit, gs, hipe, + inets, observer, odbc, os_mon, otp_mibs, parsetools, + percept, public_key, reltool, runtime_tools, ssh, + syntax_tools, test_server, tools, typer, webtool, wx, + xmerl</c></p> + <p> + A new test utility for testing appup files is added to + test_server. This is now used by most applications in + OTP.</p> + <p> + (Thanks to Tobias Schlager)</p> + <p> + Own Id: OTP-11744</p> + </item> + <item> + <p> + ftp now sanitize file name, user name and passwords from + <CR> and <LF> tags (Thanks to Sergei Golovan)</p> + <p> + Own Id: OTP-11750</p> + </item> + <item> + <p> + Corrected error handling in the HTTP client, making it + behave more graceful.</p> + <p> + Thanks to Kirilll Zaborsky</p> + <p> + Own Id: OTP-11794</p> + </item> + <item> + <p> + Support identity transfer-encoding in httpc.</p> + <p> + Thanks to Anthony Ramine</p> + <p> + Own Id: OTP-11802</p> + </item> + <item> + <p> + Ignore empty Set-Cookie headers to increase + interoperability with servers that violate the RFC. </p> + <p> + Thanks to Kirilll Zaborsky</p> + <p> + Own Id: OTP-11803</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + The commit 6189bc07 "inets: httpc improve pipelining" has + been reverted, as it turned out to break things rather + than improve pipelining utilization. It is instead up to + the user to configure httpc and use it wisely to be able + to get the most out of pipelining.</p> + <p> + Own Id: OTP-11756</p> + </item> + <item> + <p> + Handle all response codes in httpd_util:message/3</p> + <p> + Own Id: OTP-11838</p> + </item> + </list> + </section> + +</section> + +<section><title>Inets 5.9.8</title> <section><title>Improvements and New Features</title> <list> diff --git a/lib/inets/src/http_client/httpc_handler.erl b/lib/inets/src/http_client/httpc_handler.erl index 88e08be789..d152d9f0be 100644 --- a/lib/inets/src/http_client/httpc_handler.erl +++ b/lib/inets/src/http_client/httpc_handler.erl @@ -1116,8 +1116,16 @@ handle_http_body(Body, #state{headers = Headers, {new_body, NewBody}]), NewHeaders = http_chunk:handle_headers(Headers, ChunkedHeaders), - handle_response(State#state{headers = NewHeaders, - body = NewBody}) + case Body of + <<>> -> + handle_response(State#state{headers = NewHeaders, + body = NewBody}); + _ -> + {NewBody2, NewRequest} = + stream(NewBody, Request, Code), + handle_response(State#state{headers = NewHeaders, + body = NewBody2}) + end end; Enc when Enc =:= "identity"; Enc =:= undefined -> ?hcrt("handle_http_body - identity", []), @@ -1218,6 +1226,7 @@ handle_response(#state{request = Request, handle_queue(State#state{request = undefined}, Data); {ok, Msg, Data} -> ?hcrd("handle response - ok", []), + stream_remaining_body(Body, Request, StatusLine), end_stream(StatusLine, Request), NewState = maybe_send_answer(Request, Msg, State), handle_queue(NewState, Data); @@ -1648,6 +1657,10 @@ start_stream(_StatusLine, _Headers, Request) -> ?hcrt("start stream - no op", []), {ok, Request}. +stream_remaining_body(<<>>, _, _) -> + ok; +stream_remaining_body(Body, Request, {_, Code, _}) -> + stream(Body, Request, Code). %% Note the end stream message is handled by httpc_response and will %% be sent by answer_request @@ -1780,7 +1793,7 @@ tls_tunnel_request(#request{headers = Headers, host_header(#http_request_h{host = Host}, _) -> Host; -%% Handles header_as_is +%% Handles headers_as_is host_header(_, URI) -> {ok, {_, _, Host, _, _, _}} = http_uri:parse(URI), Host. diff --git a/lib/inets/src/http_lib/http_internal.hrl b/lib/inets/src/http_lib/http_internal.hrl index 97cf474ab9..53b776c4e7 100644 --- a/lib/inets/src/http_lib/http_internal.hrl +++ b/lib/inets/src/http_lib/http_internal.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2002-2011. All Rights Reserved. +%% Copyright Ericsson AB 2002-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -26,6 +26,8 @@ -define(HTTP_MAX_BODY_SIZE, nolimit). -define(HTTP_MAX_HEADER_SIZE, 10240). -define(HTTP_MAX_URI_SIZE, nolimit). +-define(HTTP_MAX_VERSION_STRING, 8). +-define(HTTP_MAX_METHOD_STRING, 20). -ifndef(HTTP_DEFAULT_SSL_KIND). -define(HTTP_DEFAULT_SSL_KIND, essl). diff --git a/lib/inets/src/http_server/httpd.erl b/lib/inets/src/http_server/httpd.erl index 6052ae9022..e8148ea362 100644 --- a/lib/inets/src/http_server/httpd.erl +++ b/lib/inets/src/http_server/httpd.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2013. All Rights Reserved. +%% Copyright Ericsson AB 1997-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -294,9 +294,13 @@ do_reload_config(ConfigList, Mode) -> {ok, Config} -> Address = proplists:get_value(bind_address, Config, any), Port = proplists:get_value(port, Config, 80), - block(Address, Port, Mode), - reload(Config, Address, Port), - unblock(Address, Port); + case block(Address, Port, Mode) of + ok -> + reload(Config, Address, Port), + unblock(Address, Port); + Error -> + Error + end; Error -> Error end. diff --git a/lib/inets/src/http_server/httpd_manager.erl b/lib/inets/src/http_server/httpd_manager.erl index e155498bb8..3da0343401 100644 --- a/lib/inets/src/http_server/httpd_manager.erl +++ b/lib/inets/src/http_server/httpd_manager.erl @@ -210,9 +210,10 @@ handle_call({block , Blocker, Mode, Timeout}, From, handle_call({block , _, _, _}, _, State) -> {reply, {error, blocked}, State}; -handle_call({unblock, Blocker}, _, #state{blocker_ref = {Blocker,_}, +handle_call({unblock, Blocker}, _, #state{blocker_ref = {Blocker, Monitor}, admin_state = blocked} = State) -> - + + erlang:demonitor(Monitor), {reply, ok, State#state{admin_state = unblocked, blocker_ref = undefined}}; @@ -247,37 +248,36 @@ handle_cast(Message, State) -> handle_info(connections_terminated, #state{admin_state = shutting_down, blocking_from = From} = State) -> gen_server:reply(From, ok), - {noreply, State#state{admin_state = blocked, blocking_from = undefined, - blocker_ref = undefined}}; + {noreply, State#state{admin_state = blocked, blocking_from = undefined}}; handle_info(connections_terminated, State) -> {noreply, State}; -handle_info({block_timeout, non_disturbing}, +handle_info({block_timeout, non_disturbing, Blocker}, #state{admin_state = shutting_down, blocking_from = From, - blocker_ref = {_, Monitor}} = State) -> + blocker_ref = {_, Monitor} = Blocker} = State) -> erlang:demonitor(Monitor), gen_server:reply(From, {error, timeout}), {noreply, State#state{admin_state = unblocked, blocking_from = undefined, blocker_ref = undefined}}; -handle_info({block_timeout, disturbing}, +handle_info({block_timeout, disturbing, Blocker}, #state{admin_state = shutting_down, blocking_from = From, - blocker_ref = {_, Monitor}, + blocker_ref = Blocker, connection_sup = Sup} = State) -> SupPid = whereis(Sup), shutdown_connections(SupPid), - erlang:demonitor(Monitor), gen_server:reply(From, ok), - {noreply, State#state{admin_state = blocked, blocker_ref = undefined, + {noreply, State#state{admin_state = blocked, blocking_from = undefined}}; handle_info({block_timeout, _, _}, State) -> {noreply, State}; handle_info({'DOWN', _, process, Pid, _Info}, #state{admin_state = Admin, - blocker_ref = {Pid, _}} = State) when + blocker_ref = {Pid, Monitor}} = State) when Admin =/= unblocked -> + erlang:demonitor(Monitor), {noreply, State#state{admin_state = unblocked, blocking_from = undefined, blocker_ref = undefined}}; @@ -333,18 +333,16 @@ handle_new_connection(_UsageState, _AdminState, State, _Handler) -> handle_block(disturbing, infinity, #state{connection_sup = CSup, - blocking_from = From, - blocker_ref = {_, Monitor}} = State) -> + blocking_from = From} = State) -> SupPid = whereis(CSup), shutdown_connections(SupPid), - erlang:demonitor(Monitor), gen_server:reply(From, ok), - {noreply, State#state{admin_state = blocked, blocker_ref = undefined, + {noreply, State#state{admin_state = blocked, blocking_from = undefined}}; -handle_block(disturbing, Timeout, #state{connection_sup = CSup} = State) -> +handle_block(disturbing, Timeout, #state{connection_sup = CSup, blocker_ref = Blocker} = State) -> Manager = self(), spawn_link(fun() -> wait_for_shutdown(CSup, Manager) end), - erlang:send_after(Timeout, self(), {block_timeout, disturbing}), + erlang:send_after(Timeout, self(), {block_timeout, disturbing, Blocker}), {noreply, State#state{admin_state = shutting_down}}; handle_block(non_disturbing, infinity, @@ -354,10 +352,10 @@ handle_block(non_disturbing, infinity, {noreply, State#state{admin_state = shutting_down}}; handle_block(non_disturbing, Timeout, - #state{connection_sup = CSup} = State) -> + #state{connection_sup = CSup, blocker_ref = Blocker} = State) -> Manager = self(), spawn_link(fun() -> wait_for_shutdown(CSup, Manager) end), - erlang:send_after(Timeout, self(), {block_timeout, non_disturbing}), + erlang:send_after(Timeout, self(), {block_timeout, non_disturbing, Blocker}), {noreply, State#state{admin_state = shutting_down}}. handle_reload(undefined, #state{config_file = undefined} = State) -> diff --git a/lib/inets/src/http_server/httpd_request.erl b/lib/inets/src/http_server/httpd_request.erl index 5ba79b2706..712c73599f 100644 --- a/lib/inets/src/http_server/httpd_request.erl +++ b/lib/inets/src/http_server/httpd_request.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2011. All Rights Reserved. +%% Copyright Ericsson AB 2005-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -44,26 +44,26 @@ %%%========================================================================= parse([Bin, MaxSizes]) -> ?hdrt("parse", [{bin, Bin}, {max_sizes, MaxSizes}]), - parse_method(Bin, [], MaxSizes, []); + parse_method(Bin, [], 0, proplists:get_value(max_method, MaxSizes), MaxSizes, []); parse(Unknown) -> ?hdrt("parse", [{unknown, Unknown}]), exit({bad_args, Unknown}). %% Functions that may be returned during the decoding process %% if the input data is incompleate. -parse_method([Bin, Method, MaxSizes, Result]) -> - parse_method(Bin, Method, MaxSizes, Result). +parse_method([Bin, Method, Current, Max, MaxSizes, Result]) -> + parse_method(Bin, Method, Current, Max, MaxSizes, Result). -parse_uri([Bin, URI, CurrSize, MaxSizes, Result]) -> - parse_uri(Bin, URI, CurrSize, MaxSizes, Result). +parse_uri([Bin, URI, Current, Max, MaxSizes, Result]) -> + parse_uri(Bin, URI, Current, Max, MaxSizes, Result). -parse_version([Bin, Rest, Version, MaxSizes, Result]) -> - parse_version(<<Rest/binary, Bin/binary>>, Version, MaxSizes, +parse_version([Bin, Rest, Version, Current, Max, MaxSizes, Result]) -> + parse_version(<<Rest/binary, Bin/binary>>, Version, Current, Max, MaxSizes, Result). -parse_headers([Bin, Rest, Header, Headers, CurrSize, MaxSizes, Result]) -> +parse_headers([Bin, Rest, Header, Headers, Current, Max, MaxSizes, Result]) -> parse_headers(<<Rest/binary, Bin/binary>>, - Header, Headers, CurrSize, MaxSizes, Result). + Header, Headers, Current, Max, MaxSizes, Result). whole_body([Bin, Body, Length]) -> whole_body(<<Body/binary, Bin/binary>>, Length). @@ -107,8 +107,12 @@ validate("POST", Uri, "HTTP/1." ++ _N) -> validate("TRACE", Uri, "HTTP/1." ++ N) when hd(N) >= $1 -> validate_uri(Uri); validate(Method, Uri, Version) -> - {error, {not_supported, {Method, Uri, Version}}}. - + case validate_version(Version) of + true -> + {error, {not_supported, {Method, Uri, Version}}}; + false -> + {error, {bad_version, Version}} + end. %%---------------------------------------------------------------------- %% The request is passed through the server as a record of type mod %% create it. @@ -131,104 +135,75 @@ update_mod_data(ModData, Method, RequestURI, HTTPVersion, Headers)-> %%%======================================================================== %%% Internal functions %%%======================================================================== -parse_method(<<>>, Method, MaxSizes, Result) -> - ?hdrt("parse_method - empty bin", - [{method, Method}, {max_sizes, MaxSizes}, {result, Result}]), - {?MODULE, parse_method, [Method, MaxSizes, Result]}; -parse_method(<<?SP, Rest/binary>>, Method, MaxSizes, Result) -> - ?hdrt("parse_method - SP begin", - [{rest, Rest}, - {method, Method}, - {max_sizes, MaxSizes}, - {result, Result}]), - parse_uri(Rest, [], 0, MaxSizes, +parse_method(<<>>, Method, Current, Max, MaxSizes, Result) -> + {?MODULE, parse_method, [Method, Current, Max, MaxSizes, Result]}; +parse_method(<<?SP, Rest/binary>>, Method, _Current, _Max, MaxSizes, Result) -> + parse_uri(Rest, [], 0, proplists:get_value(max_uri, MaxSizes), MaxSizes, [string:strip(lists:reverse(Method)) | Result]); -parse_method(<<Octet, Rest/binary>>, Method, MaxSizes, Result) -> - ?hdrt("parse_method", - [{octet, Octet}, - {rest, Rest}, - {method, Method}, - {max_sizes, MaxSizes}, - {result, Result}]), - parse_method(Rest, [Octet | Method], MaxSizes, Result). - -parse_uri(_, _, CurrSize, {MaxURI, _}, _) - when (CurrSize > MaxURI) andalso (MaxURI =/= nolimit) -> - ?hdrt("parse_uri", - [{current_size, CurrSize}, - {max_uri, MaxURI}]), +parse_method(<<Octet, Rest/binary>>, Method, Current, Max, MaxSizes, Result) when Current =< Max -> + parse_method(Rest, [Octet | Method], Current + 1, Max, MaxSizes, Result); +parse_method(_, _, _, Max, _, _) -> + %% We do not know the version of the client as it comes after the + %% method send the lowest version in the response so that the client + %% will be able to handle it. + {error, {too_long, Max, 413, "Method unreasonably long"}, lowest_version()}. + +parse_uri(_, _, Current, MaxURI, _, _) + when (Current > MaxURI) andalso (MaxURI =/= nolimit) -> %% We do not know the version of the client as it comes after the %% uri send the lowest version in the response so that the client %% will be able to handle it. - HttpVersion = "HTTP/0.9", - {error, {uri_too_long, MaxURI}, HttpVersion}; -parse_uri(<<>>, URI, CurrSize, MaxSizes, Result) -> - ?hdrt("parse_uri - empty bin", - [{uri, URI}, - {current_size, CurrSize}, - {max_sz, MaxSizes}, - {result, Result}]), - {?MODULE, parse_uri, [URI, CurrSize, MaxSizes, Result]}; -parse_uri(<<?SP, Rest/binary>>, URI, _, MaxSizes, Result) -> - ?hdrt("parse_uri - SP begin", - [{uri, URI}, - {max_sz, MaxSizes}, - {result, Result}]), - parse_version(Rest, [], MaxSizes, + {error, {too_long, MaxURI, 414, "URI unreasonably long"},lowest_version()}; +parse_uri(<<>>, URI, Current, Max, MaxSizes, Result) -> + {?MODULE, parse_uri, [URI, Current, Max, MaxSizes, Result]}; +parse_uri(<<?SP, Rest/binary>>, URI, _, _, MaxSizes, Result) -> + parse_version(Rest, [], 0, proplists:get_value(max_version, MaxSizes), MaxSizes, [string:strip(lists:reverse(URI)) | Result]); %% Can happen if it is a simple HTTP/0.9 request e.i "GET /\r\n\r\n" -parse_uri(<<?CR, _Rest/binary>> = Data, URI, _, MaxSizes, Result) -> - ?hdrt("parse_uri - CR begin", - [{uri, URI}, - {max_sz, MaxSizes}, - {result, Result}]), - parse_version(Data, [], MaxSizes, +parse_uri(<<?CR, _Rest/binary>> = Data, URI, _, _, MaxSizes, Result) -> + parse_version(Data, [], 0, proplists:get_value(max_version, MaxSizes), MaxSizes, [string:strip(lists:reverse(URI)) | Result]); -parse_uri(<<Octet, Rest/binary>>, URI, CurrSize, MaxSizes, Result) -> - ?hdrt("parse_uri", - [{octet, Octet}, - {uri, URI}, - {curr_sz, CurrSize}, - {max_sz, MaxSizes}, - {result, Result}]), - parse_uri(Rest, [Octet | URI], CurrSize + 1, MaxSizes, Result). - -parse_version(<<>>, Version, MaxSizes, Result) -> - {?MODULE, parse_version, [<<>>, Version, MaxSizes, Result]}; -parse_version(<<?LF, Rest/binary>>, Version, MaxSizes, Result) -> +parse_uri(<<Octet, Rest/binary>>, URI, Current, Max, MaxSizes, Result) -> + parse_uri(Rest, [Octet | URI], Current + 1, Max, MaxSizes, Result). + +parse_version(<<>>, Version, Current, Max, MaxSizes, Result) -> + {?MODULE, parse_version, [<<>>, Version, Current, Max, MaxSizes, Result]}; +parse_version(<<?LF, Rest/binary>>, Version, Current, Max, MaxSizes, Result) -> %% If ?CR is is missing RFC2616 section-19.3 - parse_version(<<?CR, ?LF, Rest/binary>>, Version, MaxSizes, Result); -parse_version(<<?CR, ?LF, Rest/binary>>, Version, MaxSizes, Result) -> - parse_headers(Rest, [], [], 0, MaxSizes, + parse_version(<<?CR, ?LF, Rest/binary>>, Version, Current, Max, MaxSizes, Result); +parse_version(<<?CR, ?LF, Rest/binary>>, Version, _, _, MaxSizes, Result) -> + parse_headers(Rest, [], [], 0, proplists:get_value(max_header, MaxSizes), MaxSizes, [string:strip(lists:reverse(Version)) | Result]); -parse_version(<<?CR>> = Data, Version, MaxSizes, Result) -> - {?MODULE, parse_version, [Data, Version, MaxSizes, Result]}; -parse_version(<<Octet, Rest/binary>>, Version, MaxSizes, Result) -> - parse_version(Rest, [Octet | Version], MaxSizes, Result). - -parse_headers(_, _, _, CurrSize, {_, MaxHeaderSize}, Result) - when CurrSize > MaxHeaderSize, MaxHeaderSize =/= nolimit -> +parse_version(<<?CR>> = Data, Version, Current, Max, MaxSizes, Result) -> + {?MODULE, parse_version, [Data, Version, Current, Max, MaxSizes, Result]}; +parse_version(<<Octet, Rest/binary>>, Version, Current, Max, MaxSizes, Result) when Current =< Max -> + parse_version(Rest, [Octet | Version], Current + 1, Max, MaxSizes, Result); +parse_version(_, _, _, Max,_,_) -> + {error, {too_long, Max, 413, "Version string unreasonably long"}, lowest_version()}. + +parse_headers(_, _, _, Current, Max, _, Result) + when Max =/= nolimit andalso Current > Max -> HttpVersion = lists:nth(3, lists:reverse(Result)), - {error, {header_too_long, MaxHeaderSize}, HttpVersion}; + {error, {too_long, Max, 413, "Headers unreasonably long"}, HttpVersion}; -parse_headers(<<>>, Header, Headers, CurrSize, MaxSizes, Result) -> - {?MODULE, parse_headers, [<<>>, Header, Headers, CurrSize, +parse_headers(<<>>, Header, Headers, Current, Max, MaxSizes, Result) -> + {?MODULE, parse_headers, [<<>>, Header, Headers, Current, Max, MaxSizes, Result]}; -parse_headers(<<?CR,?LF,?LF,Body/binary>>, [], [], CurrSize, MaxSizes, Result) -> +parse_headers(<<?CR,?LF,?LF,Body/binary>>, [], [], Current, Max, MaxSizes, Result) -> %% If ?CR is is missing RFC2616 section-19.3 - parse_headers(<<?CR,?LF,?CR,?LF,Body/binary>>, [], [], CurrSize, + parse_headers(<<?CR,?LF,?CR,?LF,Body/binary>>, [], [], Current, Max, MaxSizes, Result); -parse_headers(<<?LF,?LF,Body/binary>>, [], [], CurrSize, MaxSizes, Result) -> +parse_headers(<<?LF,?LF,Body/binary>>, [], [], Current, Max, MaxSizes, Result) -> %% If ?CR is is missing RFC2616 section-19.3 - parse_headers(<<?CR,?LF,?CR,?LF,Body/binary>>, [], [], CurrSize, + parse_headers(<<?CR,?LF,?CR,?LF,Body/binary>>, [], [], Current, Max, MaxSizes, Result); -parse_headers(<<?CR,?LF,?CR,?LF,Body/binary>>, [], [], _, _, Result) -> +parse_headers(<<?CR,?LF,?CR,?LF,Body/binary>>, [], [], _, _, _, Result) -> NewResult = list_to_tuple(lists:reverse([Body, {#http_request_h{}, []} | Result])), {ok, NewResult}; -parse_headers(<<?CR,?LF,?CR,?LF,Body/binary>>, Header, Headers, _, +parse_headers(<<?CR,?LF,?CR,?LF,Body/binary>>, Header, Headers, _, _, _, Result) -> HTTPHeaders = [lists:reverse(Header) | Headers], RequestHeaderRcord = @@ -238,52 +213,51 @@ parse_headers(<<?CR,?LF,?CR,?LF,Body/binary>>, Header, Headers, _, HTTPHeaders} | Result])), {ok, NewResult}; -parse_headers(<<?CR,?LF,?CR>> = Data, Header, Headers, CurrSize, +parse_headers(<<?CR,?LF,?CR>> = Data, Header, Headers, Current, Max, MaxSizes, Result) -> - {?MODULE, parse_headers, [Data, Header, Headers, CurrSize, + {?MODULE, parse_headers, [Data, Header, Headers, Current, Max, MaxSizes, Result]}; -parse_headers(<<?LF>>, [], [], CurrSize, MaxSizes, Result) -> +parse_headers(<<?LF>>, [], [], Current, Max, MaxSizes, Result) -> %% If ?CR is is missing RFC2616 section-19.3 - parse_headers(<<?CR,?LF>>, [], [], CurrSize, MaxSizes, Result); + parse_headers(<<?CR,?LF>>, [], [], Current, Max, MaxSizes, Result); %% There where no headers, which is unlikely to happen. -parse_headers(<<?CR,?LF>>, [], [], _, _, Result) -> +parse_headers(<<?CR,?LF>>, [], [], _, _, _, Result) -> NewResult = list_to_tuple(lists:reverse([<<>>, {#http_request_h{}, []} | Result])), {ok, NewResult}; -parse_headers(<<?LF>>, Header, Headers, CurrSize, +parse_headers(<<?LF>>, Header, Headers, Current, Max, MaxSizes, Result) -> %% If ?CR is is missing RFC2616 section-19.3 - parse_headers(<<?CR,?LF>>, Header, Headers, CurrSize, MaxSizes, Result); + parse_headers(<<?CR,?LF>>, Header, Headers, Current, Max, MaxSizes, Result); -parse_headers(<<?CR,?LF>> = Data, Header, Headers, CurrSize, +parse_headers(<<?CR,?LF>> = Data, Header, Headers, Current, Max, MaxSizes, Result) -> - {?MODULE, parse_headers, [Data, Header, Headers, CurrSize, + {?MODULE, parse_headers, [Data, Header, Headers, Current, Max, MaxSizes, Result]}; -parse_headers(<<?LF, Octet, Rest/binary>>, Header, Headers, CurrSize, +parse_headers(<<?LF, Octet, Rest/binary>>, Header, Headers, Current, Max, MaxSizes, Result) -> %% If ?CR is is missing RFC2616 section-19.3 - parse_headers(<<?CR,?LF, Octet, Rest/binary>>, Header, Headers, CurrSize, + parse_headers(<<?CR,?LF, Octet, Rest/binary>>, Header, Headers, Current, Max, MaxSizes, Result); -parse_headers(<<?CR,?LF, Octet, Rest/binary>>, Header, Headers, CurrSize, +parse_headers(<<?CR,?LF, Octet, Rest/binary>>, Header, Headers, _, Max, MaxSizes, Result) -> parse_headers(Rest, [Octet], [lists:reverse(Header) | Headers], - CurrSize + 1, MaxSizes, Result); - -parse_headers(<<?CR>> = Data, Header, Headers, CurrSize, + 0, Max, MaxSizes, Result); +parse_headers(<<?CR>> = Data, Header, Headers, Current, Max, MaxSizes, Result) -> - {?MODULE, parse_headers, [Data, Header, Headers, CurrSize, + {?MODULE, parse_headers, [Data, Header, Headers, Current, Max, MaxSizes, Result]}; -parse_headers(<<?LF>>, Header, Headers, CurrSize, +parse_headers(<<?LF>>, Header, Headers, Current, Max, MaxSizes, Result) -> %% If ?CR is is missing RFC2616 section-19.3 - parse_headers(<<?CR, ?LF>>, Header, Headers, CurrSize, + parse_headers(<<?CR, ?LF>>, Header, Headers, Current, Max, MaxSizes, Result); -parse_headers(<<Octet, Rest/binary>>, Header, Headers, - CurrSize, MaxSizes, Result) -> - parse_headers(Rest, [Octet | Header], Headers, CurrSize + 1, +parse_headers(<<Octet, Rest/binary>>, Header, Headers, Current, + Max, MaxSizes, Result) -> + parse_headers(Rest, [Octet | Header], Headers, Current + 1, Max, MaxSizes, Result). whole_body(Body, Length) -> @@ -326,6 +300,14 @@ validate_path([".." | Rest], N, RequestURI) -> validate_path([_ | Rest], N, RequestURI) -> validate_path(Rest, N + 1, RequestURI). +validate_version("HTTP/1.1") -> + true; +validate_version("HTTP/1.0") -> + true; +validate_version("HTTP/0.9") -> + true; +validate_version(_) -> + false. %%---------------------------------------------------------------------- %% There are 3 possible forms of the reuqest URI %% @@ -430,3 +412,5 @@ tag([$:|Rest], Tag) -> tag([Chr|Rest], Tag) -> tag(Rest, [Chr|Tag]). +lowest_version()-> + "HTTP/0.9". diff --git a/lib/inets/src/http_server/httpd_request_handler.erl b/lib/inets/src/http_server/httpd_request_handler.erl index bd37066ff6..9bea58cc9e 100644 --- a/lib/inets/src/http_server/httpd_request_handler.erl +++ b/lib/inets/src/http_server/httpd_request_handler.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2013. All Rights Reserved. +%% Copyright Ericsson AB 1997-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -35,6 +35,7 @@ -include("http_internal.hrl"). -include("httpd_internal.hrl"). +-define(HANDSHAKE_TIMEOUT, 5000). -record(state, {mod, %% #mod{} manager, %% pid() status, %% accept | busy | blocked @@ -96,15 +97,13 @@ init([Manager, ConfigDB, AcceptTimeout]) -> {SocketType, Socket} = await_socket_ownership_transfer(AcceptTimeout), - TimeOut = httpd_util:lookup(ConfigDB, keep_alive_timeout, 150000), - Then = erlang:now(), + KeepAliveTimeOut = httpd_util:lookup(ConfigDB, keep_alive_timeout, 150000), - case http_transport:negotiate(SocketType, Socket, TimeOut) of + case http_transport:negotiate(SocketType, Socket, ?HANDSHAKE_TIMEOUT) of {error, _Error} -> exit(shutdown); %% Can be 'normal'. ok -> - NewTimeout = TimeOut - timer:now_diff(now(),Then) div 1000, - continue_init(Manager, ConfigDB, SocketType, Socket, NewTimeout) + continue_init(Manager, ConfigDB, SocketType, Socket, KeepAliveTimeOut) end. continue_init(Manager, ConfigDB, SocketType, Socket, TimeOut) -> @@ -123,7 +122,8 @@ continue_init(Manager, ConfigDB, SocketType, Socket, TimeOut) -> {_, Status} = httpd_manager:new_connection(Manager), - MFA = {httpd_request, parse, [{MaxURISize, MaxHeaderSize}]}, + MFA = {httpd_request, parse, [[{max_uri, MaxURISize}, {max_header, MaxHeaderSize}, + {max_version, ?HTTP_MAX_VERSION_STRING}, {max_method, ?HTTP_MAX_METHOD_STRING}]]}, State = #state{mod = Mod, manager = Manager, @@ -207,23 +207,15 @@ handle_info({Proto, Socket, Data}, set_new_data_size(cancel_request_timeout(State), NewDataSize) end, handle_http_msg(Result, NewState); - - {error, {uri_too_long, MaxSize}, Version} -> - NewModData = ModData#mod{http_version = Version}, - httpd_response:send_status(NewModData, 414, "URI too long"), - Reason = io_lib:format("Uri too long, max size is ~p~n", - [MaxSize]), - error_log(Reason, NewModData), - {stop, normal, State#state{response_sent = true, - mod = NewModData}}; - {error, {header_too_long, MaxSize}, Version} -> + {error, {too_long, MaxSize, ErrCode, ErrStr}, Version} -> NewModData = ModData#mod{http_version = Version}, - httpd_response:send_status(NewModData, 413, "Header too long"), - Reason = io_lib:format("Header too long, max size is ~p~n", - [MaxSize]), + httpd_response:send_status(NewModData, ErrCode, ErrStr), + Reason = io_lib:format("~p: ~p max size is ~p~n", + [ErrCode, ErrStr, MaxSize]), error_log(Reason, NewModData), {stop, normal, State#state{response_sent = true, mod = NewModData}}; + NewMFA -> http_transport:setopts(SockType, Socket, [{active, once}]), case NewDataSize of @@ -382,6 +374,11 @@ handle_http_msg({Method, Uri, Version, {RecordHeaders, Headers}, Body}, 400, URI), Reason = io_lib:format("Malformed syntax in URI: ~p~n", [URI]), error_log(Reason, ModData), + {stop, normal, State#state{response_sent = true}}; + {error, {bad_version, Ver}} -> + httpd_response:send_status(ModData#mod{http_version = "HTTP/0.9"}, 400, Ver), + Reason = io_lib:format("Malformed syntax version: ~p~n", [Ver]), + error_log(Reason, ModData), {stop, normal, State#state{response_sent = true}} end; handle_http_msg({ChunkedHeaders, Body}, @@ -549,7 +546,8 @@ handle_next_request(#state{mod = #mod{connection = true} = ModData, MaxHeaderSize = max_header_size(ModData#mod.config_db), MaxURISize = max_uri_size(ModData#mod.config_db), - MFA = {httpd_request, parse, [{MaxURISize, MaxHeaderSize}]}, + MFA = {httpd_request, parse, [[{max_uri, MaxURISize}, {max_header, MaxHeaderSize}, + {max_version, ?HTTP_MAX_VERSION_STRING}, {max_method, ?HTTP_MAX_METHOD_STRING}]]}, TmpState = State#state{mod = NewModData, mfa = MFA, max_keep_alive_request = decrease(Max), diff --git a/lib/inets/src/http_server/httpd_util.erl b/lib/inets/src/http_server/httpd_util.erl index b0b18b9c3d..0d04a75205 100644 --- a/lib/inets/src/http_server/httpd_util.erl +++ b/lib/inets/src/http_server/httpd_util.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2011. All Rights Reserved. +%% Copyright Ericsson AB 1997-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -206,9 +206,6 @@ message(413, Reason,_) -> "Entity: " ++ html_encode(Reason); message(414,ReasonPhrase,_) -> "Message " ++ html_encode(ReasonPhrase) ++ "."; -message(416,ReasonPhrase,_) -> - html_encode(ReasonPhrase); - message(500,_,ConfigDB) -> ServerAdmin=lookup(ConfigDB,server_admin,"unknown@unknown"), "The server encountered an internal error or " @@ -233,7 +230,9 @@ message(501,{Method, RequestURI, HTTPVersion}, _ConfigDB) -> end; message(503, String, _ConfigDB) -> - "This service in unavailable due to: " ++ html_encode(String). + "This service in unavailable due to: " ++ html_encode(String); +message(_, ReasonPhrase, _) -> + html_encode(ReasonPhrase). maybe_encode(URI) -> Decoded = try http_uri:decode(URI) of diff --git a/lib/inets/src/inets_app/inets.appup.src b/lib/inets/src/inets_app/inets.appup.src index dd081962cc..4bc49e1e67 100644 --- a/lib/inets/src/inets_app/inets.appup.src +++ b/lib/inets/src/inets_app/inets.appup.src @@ -17,11 +17,44 @@ %% %CopyrightEnd% {"%VSN%", [ - {"5.9.8", [{load_module, ftp, soft_purge, soft_purge, []}]}, + {"5.10.2", + [ + {load_module, httpd_request_handler, soft_purge, soft_purge, + []}]}, + {"5.10.1", + [{load_module, httpc_handler, soft_purge, soft_purge, []}, + {load_module, httpd, soft_purge, soft_purge, []}, + {load_module, httpd_manager, soft_purge, soft_purge, []}, + {load_module, httpd_request, soft_purge, soft_purge, []}, + {load_module, httpd_request_handler, soft_purge, soft_purge, + []}]}, + {"5.10", + [{load_module, httpc_handler, soft_purge, soft_purge, []}, + {load_module, httpd, soft_purge, soft_purge, []}, + {load_module, httpd_manager, soft_purge, soft_purge, []}, + {load_module, httpd_request, soft_purge, soft_purge, []}, + {load_module, httpd_request_handler, soft_purge, soft_purge, + []}]}, {<<"5\\..*">>,[{restart_application, inets}]} ], [ - {"5.9.8", [{load_module, ftp, soft_purge, soft_purge, []}]}, + {"5.10.2", + [ + {load_module, httpd_request_handler, soft_purge, soft_purge, + []}]}, + {"5.10.1", + [{load_module, httpc_handler, soft_purge, soft_purge, []}, + {load_module, httpd, soft_purge, soft_purge, []}, + {load_module, httpd_manager, soft_purge, soft_purge, []}, + {load_module, httpd_request, soft_purge, soft_purge, []}, + {load_module, httpd_request_handler, soft_purge, soft_purge, + []}]}, + {"5.10", + [{load_module, httpc_handler, soft_purge, soft_purge, []}, + {load_module, httpd, soft_purge, soft_purge, []}, + {load_module, httpd_manager, soft_purge, soft_purge, []}, + {load_module, httpd_request, soft_purge, soft_purge, []}, + {load_module, httpd_request_handler, soft_purge, soft_purge, []}]}, {<<"5\\..*">>,[{restart_application, inets}]} ] }. diff --git a/lib/inets/test/ftp_property_test_SUITE.erl b/lib/inets/test/ftp_property_test_SUITE.erl new file mode 100644 index 0000000000..c7077421f4 --- /dev/null +++ b/lib/inets/test/ftp_property_test_SUITE.erl @@ -0,0 +1,52 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2004-2014. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% + +%%% Run like this: +%%% ct:run_test([{suite,"ftp_property_test_SUITE"}, {logdir,"/ldisk/OTP/LOG"}]). + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%%% %%% +%%% WARNING %%% +%%% %%% +%%% This is experimental code which may be changed or removed %%% +%%% anytime without any warning. %%% +%%% %%% +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +-module(ftp_property_test_SUITE). + +-compile(export_all). + +-include_lib("common_test/include/ct.hrl"). + +all() -> [prop_ftp_case]. + + +init_per_suite(Config) -> + inets:start(), + ct_property_test:init_per_suite(Config). + + +%%%---- test case +prop_ftp_case(Config) -> + ct_property_test:quickcheck( + ftp_simple_client_server:prop_ftp(Config), + Config + ). diff --git a/lib/inets/test/http_format_SUITE.erl b/lib/inets/test/http_format_SUITE.erl index c5920a3968..d4a3f28f38 100644 --- a/lib/inets/test/http_format_SUITE.erl +++ b/lib/inets/test/http_format_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2013. All Rights Reserved. +%% Copyright Ericsson AB 2004-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -356,7 +356,10 @@ http_request(Config) when is_list(Config) -> "HTTP/1.1", {#http_request_h{host = "www.erlang.org", te = []}, ["te: ","host:www.erlang.org"]}, <<>>} = - parse(httpd_request, parse, [?HTTP_MAX_HEADER_SIZE], HttpHead), + parse(httpd_request, parse, [[{max_header, ?HTTP_MAX_HEADER_SIZE}, + {max_version, ?HTTP_MAX_VERSION_STRING}, + {max_method, ?HTTP_MAX_METHOD_STRING}]], + HttpHead), HttpHead1 = ["GET http://www.erlang.org HTTP/1.1" ++ [?CR], [?LF, ?CR, ?LF]], @@ -364,7 +367,9 @@ http_request(Config) when is_list(Config) -> "http://www.erlang.org", "HTTP/1.1", {#http_request_h{}, []}, <<>>} = - parse(httpd_request, parse, [?HTTP_MAX_HEADER_SIZE], HttpHead1), + parse(httpd_request, parse, [[{max_header, ?HTTP_MAX_HEADER_SIZE}, + {max_version, ?HTTP_MAX_VERSION_STRING}, + {max_method, ?HTTP_MAX_METHOD_STRING}]], HttpHead1), HttpHead2 = ["GET http://www.erlang.org HTTP/1.1" ++ @@ -373,7 +378,9 @@ http_request(Config) when is_list(Config) -> "http://www.erlang.org", "HTTP/1.1", {#http_request_h{}, []}, <<>>} = - parse(httpd_request, parse, [?HTTP_MAX_HEADER_SIZE], HttpHead2), + parse(httpd_request, parse, [[{max_header, ?HTTP_MAX_HEADER_SIZE}, + {max_version, ?HTTP_MAX_VERSION_STRING}, + {max_method, ?HTTP_MAX_METHOD_STRING}]], HttpHead2), %% Note the following body is not related to the headers above HttpBody = ["<HTML>\n<HEAD>\n<TITLE> dummy </TITLE>\n</HEAD>\n<BODY>\n", diff --git a/lib/inets/test/httpc_SUITE.erl b/lib/inets/test/httpc_SUITE.erl index b1b799c953..c535d59b9f 100644 --- a/lib/inets/test/httpc_SUITE.erl +++ b/lib/inets/test/httpc_SUITE.erl @@ -27,15 +27,14 @@ -include_lib("kernel/include/file.hrl"). -include_lib("common_test/include/ct.hrl"). -include("inets_test_lib.hrl"). - +-include("http_internal.hrl"). %% Note: This directive should only be used in test suites. -compile(export_all). -define(URL_START, "http://"). -define(TLS_URL_START, "https://"). -define(NOT_IN_USE_PORT, 8997). --define(LF, $\n). --define(HTTP_MAX_HEADER_SIZE, 10240). + -record(sslsocket, {fd = nil, pid = nil}). %%-------------------------------------------------------------------- %% Common Test interface functions ----------------------------------- @@ -94,6 +93,8 @@ only_simulated() -> empty_set_cookie, trace, stream_once, + stream_single_chunk, + stream_no_length, no_content_204, tolerate_missing_CR, userinfo, @@ -387,6 +388,22 @@ stream_once(Config) when is_list(Config) -> Request2 = {url(group_name(Config), "/once_chunked.html", Config), []}, stream_test(Request2, {stream, {self, once}}). +%%------------------------------------------------------------------------- +stream_single_chunk() -> + [{doc, "Test the option stream for asynchrony requests"}]. +stream_single_chunk(Config) when is_list(Config) -> + Request = {url(group_name(Config), "/single_chunk.html", Config), []}, + stream_test(Request, {stream, self}). +%%------------------------------------------------------------------------- +stream_no_length() -> + [{doc, "Test the option stream for asynchrony requests with HTTP 1.0 " + "body end on closed connection" }]. +stream_no_length(Config) when is_list(Config) -> + Request1 = {url(group_name(Config), "/http_1_0_no_length_single.html", Config), []}, + stream_test(Request1, {stream, self}), + Request2 = {url(group_name(Config), "/http_1_0_no_length_multiple.html", Config), []}, + stream_test(Request2, {stream, self}). + %%------------------------------------------------------------------------- redirect_multiple_choises() -> @@ -1047,7 +1064,7 @@ stream_test(Request, To) -> ct:fail(Msg) end, - Body == binary_to_list(StreamedBody). + Body = binary_to_list(StreamedBody). url(http, End, Config) -> Port = ?config(port, Config), @@ -1226,7 +1243,10 @@ dummy_server_init(Caller, ip_comm, Inet, _) -> {ok, ListenSocket} = gen_tcp:listen(0, [Inet | BaseOpts]), {ok, Port} = inet:port(ListenSocket), Caller ! {port, Port}, - dummy_ipcomm_server_loop({httpd_request, parse, [?HTTP_MAX_HEADER_SIZE]}, + dummy_ipcomm_server_loop({httpd_request, parse, [[{max_uri, ?HTTP_MAX_URI_SIZE}, + {max_header, ?HTTP_MAX_HEADER_SIZE}, + {max_version,?HTTP_MAX_VERSION_STRING}, + {max_method, ?HTTP_MAX_METHOD_STRING}]]}, [], ListenSocket); dummy_server_init(Caller, ssl, Inet, SSLOptions) -> @@ -1238,7 +1258,10 @@ dummy_ssl_server_init(Caller, BaseOpts, Inet) -> {ok, ListenSocket} = ssl:listen(0, [Inet | BaseOpts]), {ok, {_, Port}} = ssl:sockname(ListenSocket), Caller ! {port, Port}, - dummy_ssl_server_loop({httpd_request, parse, [?HTTP_MAX_HEADER_SIZE]}, + dummy_ssl_server_loop({httpd_request, parse, [[{max_uri, ?HTTP_MAX_URI_SIZE}, + {max_method, ?HTTP_MAX_METHOD_STRING}, + {max_version,?HTTP_MAX_VERSION_STRING}, + {max_method, ?HTTP_MAX_METHOD_STRING}]]}, [], ListenSocket). dummy_ipcomm_server_loop(MFA, Handlers, ListenSocket) -> @@ -1268,6 +1291,7 @@ dummy_ssl_server_loop(MFA, Handlers, ListenSocket) -> From ! {stopped, self()} after 0 -> {ok, Socket} = ssl:transport_accept(ListenSocket), + ok = ssl:ssl_accept(Socket, infinity), HandlerPid = dummy_request_handler(MFA, Socket), ssl:controlling_process(Socket, HandlerPid), HandlerPid ! ssl_controller, @@ -1314,10 +1338,16 @@ handle_request(Module, Function, Args, Socket) -> stop -> stop; <<>> -> - {httpd_request, parse, [[<<>>, ?HTTP_MAX_HEADER_SIZE]]}; + {httpd_request, parse, [[<<>>, [{max_uri, ?HTTP_MAX_URI_SIZE}, + {max_header, ?HTTP_MAX_HEADER_SIZE}, + {max_version,?HTTP_MAX_VERSION_STRING}, + {max_method, ?HTTP_MAX_METHOD_STRING}]]]}; Data -> handle_request(httpd_request, parse, - [Data |[?HTTP_MAX_HEADER_SIZE]], Socket) + [Data, [{max_uri, ?HTTP_MAX_URI_SIZE}, + {max_header, ?HTTP_MAX_HEADER_SIZE}, + {max_version,?HTTP_MAX_VERSION_STRING}, + {max_method, ?HTTP_MAX_METHOD_STRING}]], Socket) end; NewMFA -> NewMFA @@ -1675,6 +1705,30 @@ handle_uri(_,"/once_chunked.html",_,_,Socket,_) -> http_chunk:encode("obar</BODY></HTML>")), http_chunk:encode_last(); +handle_uri(_,"/single_chunk.html",_,_,Socket,_) -> + Chunk = "HTTP/1.1 200 ok\r\n" ++ + "Transfer-Encoding:Chunked\r\n\r\n" ++ + http_chunk:encode("<HTML><BODY>fo") ++ + http_chunk:encode("obar</BODY></HTML>") ++ + http_chunk:encode_last(), + send(Socket, Chunk); + +handle_uri(_,"/http_1_0_no_length_single.html",_,_,Socket,_) -> + Body = "HTTP/1.0 200 ok\r\n" + "Content-type:text/plain\r\n\r\n" + "single packet", + send(Socket, Body), + close(Socket); + +handle_uri(_,"/http_1_0_no_length_multiple.html",_,_,Socket,_) -> + Head = "HTTP/1.0 200 ok\r\n" + "Content-type:text/plain\r\n\r\n" + "multiple packets, ", + send(Socket, Head), + %% long body to make sure it will be sent in multiple tcp packets + send(Socket, string:copies("other multiple packets ", 200)), + close(Socket); + handle_uri(_,"/once.html",_,_,Socket,_) -> Head = "HTTP/1.1 200 ok\r\n" ++ "Content-Length:32\r\n\r\n", diff --git a/lib/inets/test/httpd_SUITE.erl b/lib/inets/test/httpd_SUITE.erl index 3eb8a0818f..4010597657 100644 --- a/lib/inets/test/httpd_SUITE.erl +++ b/lib/inets/test/httpd_SUITE.erl @@ -39,6 +39,7 @@ -define(FAIL_EXPIRE_TIME,1). %% Seconds before successful auths timeout. -define(AUTH_TIMEOUT,5). +-define(URL_START, "http://"). %%-------------------------------------------------------------------- %% Common Test interface functions ----------------------------------- @@ -63,7 +64,9 @@ all() -> {group, http_htaccess}, {group, https_htaccess}, {group, http_security}, - {group, https_security} + {group, https_security}, + {group, http_reload}, + {group, https_reload} ]. groups() -> @@ -84,7 +87,18 @@ groups() -> {https_htaccess, [], [{group, htaccess}]}, {http_security, [], [{group, security}]}, {https_security, [], [{group, security}]}, + {http_reload, [], [{group, reload}]}, + {https_reload, [], [{group, reload}]}, {limit, [], [max_clients_1_1, max_clients_1_0, max_clients_0_9]}, + {reload, [], [non_disturbing_reconfiger_dies, + disturbing_reconfiger_dies, + non_disturbing_1_1, + non_disturbing_1_0, + non_disturbing_0_9, + disturbing_1_1, + disturbing_1_0, + disturbing_0_9 + ]}, {basic_auth, [], [basic_auth_1_1, basic_auth_1_0, basic_auth_0_9]}, {auth_api, [], [auth_api_1_1, auth_api_1_0, auth_api_0_9 ]}, @@ -134,8 +148,24 @@ init_per_suite(Config) -> inets_test_lib:del_dirs(ServerRoot), DocRoot = filename:join(ServerRoot, "htdocs"), setup_server_dirs(ServerRoot, DocRoot, DataDir), + {ok, Hostname0} = inet:gethostname(), + Inet = + case (catch ct:get_config(ipv6_hosts)) of + undefined -> + inet; + Hosts when is_list(Hosts) -> + case lists:member(list_to_atom(Hostname0), Hosts) of + true -> + inet6; + false -> + inet + end; + _ -> + inet + end, [{server_root, ServerRoot}, {doc_root, DocRoot}, + {ipfamily, Inet}, {node, node()}, {host, inets_test_lib:hostname()}, {address, getaddr()} | Config]. @@ -150,7 +180,8 @@ init_per_group(Group, Config0) when Group == https_basic; Group == https_auth_api; Group == https_auth_api_dets; Group == https_auth_api_mnesia; - Group == https_security + Group == https_security; + Group == https_reload -> init_ssl(Group, Config0); init_per_group(Group, Config0) when Group == http_basic; @@ -159,7 +190,8 @@ init_per_group(Group, Config0) when Group == http_basic; Group == http_auth_api; Group == http_auth_api_dets; Group == http_auth_api_mnesia; - Group == http_security + Group == http_security; + Group == http_reload -> ok = start_apps(Group), init_httpd(Group, [{type, ip_comm} | Config0]); @@ -168,7 +200,12 @@ init_per_group(http_1_1, Config) -> init_per_group(http_1_0, Config) -> [{http_version, "HTTP/1.0"} | Config]; init_per_group(http_0_9, Config) -> - [{http_version, "HTTP/0.9"} | Config]; + case {os:type(), os:version()} of + {{win32, _}, {5,1,2600}} -> + {skip, "eaddrinuse XP problem"}; + _ -> + [{http_version, "HTTP/0.9"} | Config] + end; init_per_group(http_htaccess = Group, Config) -> Path = ?config(doc_root, Config), catch remove_htaccess(Path), @@ -197,17 +234,19 @@ end_per_group(Group, _Config) when Group == http_basic; Group == http_auth_api_dets; Group == http_auth_api_mnesia; Group == http_htaccess; - Group == http_security + Group == http_security; + Group == http_reload -> inets:stop(); end_per_group(Group, _Config) when Group == https_basic; Group == https_limit; Group == https_basic_auth; Group == https_auth_api; - Group == http_auth_api_dets; - Group == http_auth_api_mnesia; + Group == https_auth_api_dets; + Group == https_auth_api_mnesia; Group == https_htaccess; - Group == http_security + Group == https_security; + Group == https_reload -> ssl:stop(), inets:stop(); @@ -501,7 +540,7 @@ ipv6(Config) when is_list(Config) -> true -> Version = ?config(http_version, Config), Host = ?config(host, Config), - URI = http_request("GET /", Version, Host), + URI = http_request("GET / ", Version, Host), httpd_test_lib:verify_request(?config(type, Config), Host, ?config(port, Config), [inet6], ?config(code, Config), @@ -1083,12 +1122,114 @@ security(Config) -> [{statuscode, 401}]), true = unblock_user(Node, "two", Port, OpenDir). + +%%------------------------------------------------------------------------- +non_disturbing_reconfiger_dies(Config) when is_list(Config) -> + do_reconfiger_dies([{http_version, "HTTP/1.1"} | Config], non_disturbing). +disturbing_reconfiger_dies(Config) when is_list(Config) -> + do_reconfiger_dies([{http_version, "HTTP/1.1"} | Config], disturbing). + +do_reconfiger_dies(Config, DisturbingType) -> + Server = ?config(server_pid, Config), + Version = ?config(http_version, Config), + Host = ?config(host, Config), + Port = ?config(port, Config), + Type = ?config(type, Config), + + HttpdConfig = httpd:info(Server), + BlockRequest = http_request("GET /eval?httpd_example:delay(2000) ", Version, Host), + {ok, Socket} = inets_test_lib:connect_bin(Type, Host, Port, transport_opts(Type, Config)), + inets_test_lib:send(Type, Socket, BlockRequest), + ct:sleep(100), %% Avoid possible timing issues + Pid = spawn(fun() -> httpd:reload_config([{server_name, "httpd_kill_" ++ Version}, + {port, Port}| + proplists:delete(server_name, HttpdConfig)], DisturbingType) + end), + monitor(process, Pid), + exit(Pid, kill), + receive + {'DOWN', _, _, _, _} -> + ok + end, + inets_test_lib:close(Type, Socket), + [{server_name, "httpd_test"}] = httpd:info(Server, [server_name]). +%%------------------------------------------------------------------------- +disturbing_1_1(Config) when is_list(Config) -> + disturbing([{http_version, "HTTP/1.1"} | Config]). + +disturbing_1_0(Config) when is_list(Config) -> + disturbing([{http_version, "HTTP/1.0"} | Config]). + +disturbing_0_9(Config) when is_list(Config) -> + disturbing([{http_version, "HTTP/0.9"} | Config]). + +disturbing(Config) when is_list(Config)-> + Server = ?config(server_pid, Config), + Version = ?config(http_version, Config), + Host = ?config(host, Config), + Port = ?config(port, Config), + Type = ?config(type, Config), + HttpdConfig = httpd:info(Server), + BlockRequest = http_request("GET /eval?httpd_example:delay(2000) ", Version, Host), + {ok, Socket} = inets_test_lib:connect_bin(Type, Host, Port, transport_opts(Type, Config)), + inets_test_lib:send(Type, Socket, BlockRequest), + ct:sleep(100), %% Avoid possible timing issues + ok = httpd:reload_config([{server_name, "httpd_disturbing_" ++ Version}, {port, Port}| + proplists:delete(server_name, HttpdConfig)], disturbing), + Close = list_to_atom((typestr(Type)) ++ "_closed"), + receive + {Close, Socket} -> + ok; + Msg -> + ct:fail({{expected, {Close, Socket}}, {got, Msg}}) + end, + inets_test_lib:close(Type, Socket), + [{server_name, "httpd_disturbing_" ++ Version}] = httpd:info(Server, [server_name]). +%%------------------------------------------------------------------------- +non_disturbing_1_1(Config) when is_list(Config) -> + non_disturbing([{http_version, "HTTP/1.1"} | Config]). + +non_disturbing_1_0(Config) when is_list(Config) -> + non_disturbing([{http_version, "HTTP/1.0"} | Config]). + +non_disturbing_0_9(Config) when is_list(Config) -> + non_disturbing([{http_version, "HTTP/0.9"} | Config]). + +non_disturbing(Config) when is_list(Config)-> + Server = ?config(server_pid, Config), + Version = ?config(http_version, Config), + Host = ?config(host, Config), + Port = ?config(port, Config), + Type = ?config(type, Config), + + HttpdConfig = httpd:info(Server), + BlockRequest = http_request("GET /eval?httpd_example:delay(2000) ", Version, Host), + {ok, Socket} = inets_test_lib:connect_bin(Type, Host, Port, transport_opts(Type, Config)), + inets_test_lib:send(Type, Socket, BlockRequest), + ct:sleep(100), %% Avoid possible timing issues + ok = httpd:reload_config([{server_name, "httpd_non_disturbing_" ++ Version}, {port, Port}| + proplists:delete(server_name, HttpdConfig)], non_disturbing), + Transport = type(Type), + receive + {Transport, Socket, Msg} -> + ct:pal("Received message ~p~n", [Msg]), + ok + after 2000 -> + ct:fail(timeout) + end, + inets_test_lib:close(Type, Socket), + [{server_name, "httpd_non_disturbing_" ++ Version}] = httpd:info(Server, [server_name]). %%-------------------------------------------------------------------- %% Internal functions ----------------------------------- %%-------------------------------------------------------------------- +url(http, End, Config) -> + Port = ?config(port, Config), + {ok,Host} = inet:gethostname(), + ?URL_START ++ Host ++ ":" ++ integer_to_list(Port) ++ End. + do_max_clients(Config) -> Version = ?config(http_version, Config), Host = ?config(host, Config), @@ -1099,7 +1240,7 @@ do_max_clients(Config) -> BlockRequest = http_request("GET /eval?httpd_example:delay(2000) ", Version, Host), {ok, Socket} = inets_test_lib:connect_bin(Type, Host, Port, transport_opts(Type, Config)), inets_test_lib:send(Type, Socket, BlockRequest), - ct:sleep(100), + ct:sleep(100), %% Avoid possible timing issues ok = httpd_test_lib:verify_request(Type, Host, Port, transport_opts(Type, Config), @@ -1112,6 +1253,7 @@ do_max_clients(Config) -> ok end, inets_test_lib:close(Type, Socket), + ct:sleep(100), %% Avoid possible timing issues ok = httpd_test_lib:verify_request(Type, Host, Port, transport_opts(Type, Config), @@ -1165,7 +1307,9 @@ start_apps(Group) when Group == https_basic; Group == https_auth_api_dets; Group == https_auth_api_mnesia; Group == http_htaccess; - Group == http_security -> + Group == http_security; + Group == http_reload + -> inets_test_lib:start_apps([inets, asn1, crypto, public_key, ssl]); start_apps(Group) when Group == http_basic; Group == http_limit; @@ -1174,7 +1318,8 @@ start_apps(Group) when Group == http_basic; Group == http_auth_api_dets; Group == http_auth_api_mnesia; Group == https_htaccess; - Group == https_security -> + Group == https_security; + Group == https_reload-> inets_test_lib:start_apps([inets]). server_start(_, HttpdConfig) -> @@ -1218,6 +1363,10 @@ server_config(http_basic, Config) -> basic_conf() ++ server_config(http, Config); server_config(https_basic, Config) -> basic_conf() ++ server_config(https, Config); +server_config(http_reload, Config) -> + [{keep_alive_timeout, 2}] ++ server_config(http, Config); +server_config(https_reload, Config) -> + [{keep_alive_timeout, 2}] ++ server_config(https, Config); server_config(http_limit, Config) -> [{max_clients, 1}] ++ server_config(http, Config); server_config(https_limit, Config) -> @@ -1264,7 +1413,7 @@ server_config(http, Config) -> {server_root, ServerRoot}, {document_root, ?config(doc_root, Config)}, {bind_address, any}, - {ipfamily, inet}, + {ipfamily, ?config(ipfamily, Config)}, {max_header_size, 256}, {max_header_action, close}, {directory_index, ["index.html", "welcome.html"]}, @@ -1533,9 +1682,10 @@ cleanup_mnesia() -> transport_opts(ssl, Config) -> PrivDir = ?config(priv_dir, Config), - [{cacertfile, filename:join(PrivDir, "public_key_cacert.pem")}]; -transport_opts(_, _) -> - []. + [?config(ipfamily, Config), + {cacertfile, filename:join(PrivDir, "public_key_cacert.pem")}]; +transport_opts(_, Config) -> + [?config(ipfamily, Config)]. %%% mod_range @@ -1786,3 +1936,12 @@ event(What, Port, Dir, Data) -> global:send(mod_security_test, Msg) end. +type(ip_comm) -> + tcp; +type(_) -> + ssl. + +typestr(ip_comm) -> + "tcp"; +typestr(_) -> + "ssl". diff --git a/lib/inets/test/httpd_basic_SUITE.erl b/lib/inets/test/httpd_basic_SUITE.erl index fbe65145dc..baef699629 100644 --- a/lib/inets/test/httpd_basic_SUITE.erl +++ b/lib/inets/test/httpd_basic_SUITE.erl @@ -32,9 +32,9 @@ suite() -> [{ct_hooks,[ts_install_cth]}]. all() -> - [ - uri_too_long_414, + [uri_too_long_414, header_too_long_413, + entity_too_long, erl_script_nocache_opt, script_nocache, escaped_url_in_error_body, @@ -63,15 +63,13 @@ end_per_group(_GroupName, Config) -> %% variable, but should NOT alter/remove any existing entries. %%-------------------------------------------------------------------- init_per_suite(Config) -> - tsp("init_per_suite -> entry with" - "~n Config: ~p", [Config]), inets_test_lib:stop_apps([inets]), inets_test_lib:start_apps([inets]), PrivDir = ?config(priv_dir, Config), DataDir = ?config(data_dir, Config), - + Dummy = -"<HTML> + "<HTML> <HEAD> <TITLE>/index.html</TITLE> </HEAD> @@ -79,7 +77,7 @@ init_per_suite(Config) -> DUMMY </BODY> </HTML>", - + DummyFile = filename:join([PrivDir,"dummy.html"]), CgiDir = filename:join(PrivDir, "cgi-bin"), ok = file:make_dir(CgiDir), @@ -116,8 +114,6 @@ DUMMY %% Description: Cleanup after the whole suite %%-------------------------------------------------------------------- end_per_suite(_Config) -> - tsp("end_per_suite -> entry with" - "~n Config: ~p", [_Config]), inets:stop(), ok. @@ -133,9 +129,7 @@ end_per_suite(_Config) -> %% Note: This function is free to add any key/value pairs to the Config %% variable, but should NOT alter/remove any existing entries. %%-------------------------------------------------------------------- -init_per_testcase(Case, Config) -> - tsp("init_per_testcase(~w) -> entry with" - "~n Config: ~p", [Case, Config]), +init_per_testcase(_Case, Config) -> Config. @@ -147,22 +141,18 @@ init_per_testcase(Case, Config) -> %% A list of key/value pairs, holding the test case configuration. %% Description: Cleanup after each test case %%-------------------------------------------------------------------- -end_per_testcase(Case, Config) -> - tsp("end_per_testcase(~w) -> entry with" - "~n Config: ~p", [Case, Config]), +end_per_testcase(_Case, Config) -> Config. %%------------------------------------------------------------------------- %% Test cases starts here. %%------------------------------------------------------------------------- -uri_too_long_414(doc) -> - ["Test that too long uri's get 414 HTTP code"]; -uri_too_long_414(suite) -> - []; +uri_too_long_414() -> + [{doc, "Test that too long uri's get 414 HTTP code"}]. uri_too_long_414(Config) when is_list(Config) -> HttpdConf = ?config(httpd_conf, Config), - {ok, Pid} = inets:start(httpd, [{port, 0}, {max_uri_size, 10} + {ok, Pid} = inets:start(httpd, [{max_uri_size, 10} | HttpdConf]), Info = httpd:info(Pid), Port = proplists:get_value(port, Info), @@ -178,17 +168,12 @@ uri_too_long_414(Config) when is_list(Config) -> {version, "HTTP/0.9"}]), inets:stop(httpd, Pid). - -%%------------------------------------------------------------------------- %%------------------------------------------------------------------------- - -header_too_long_413(doc) -> - ["Test that too long headers's get 413 HTTP code"]; -header_too_long_413(suite) -> - []; +header_too_long_413() -> + [{doc,"Test that too long headers's get 413 HTTP code"}]. header_too_long_413(Config) when is_list(Config) -> HttpdConf = ?config(httpd_conf, Config), - {ok, Pid} = inets:start(httpd, [{port, 0}, {max_header_size, 10} + {ok, Pid} = inets:start(httpd, [{max_header_size, 10} | HttpdConf]), Info = httpd:info(Pid), Port = proplists:get_value(port, Info), @@ -202,8 +187,72 @@ header_too_long_413(Config) when is_list(Config) -> inets:stop(httpd, Pid). %%------------------------------------------------------------------------- + +entity_too_long() -> + [{doc, "Test that too long versions and method strings are rejected"}]. +entity_too_long(Config) when is_list(Config) -> + HttpdConf = ?config(httpd_conf, Config), + {ok, Pid} = inets:start(httpd, HttpdConf), + Info = httpd:info(Pid), + Port = proplists:get_value(port, Info), + Address = proplists:get_value(bind_address, Info), + + %% Not so long but wrong + ok = httpd_test_lib:verify_request(ip_comm, Address, Port, node(), + "GET / " ++ + lists:duplicate(5, $A) ++ "\r\n\r\n", + [{statuscode, 400}, + %% Server will send lowest version + %% as it will not get to the + %% client version + %% before aborting + {version, "HTTP/0.9"}]), + + %% Too long + ok = httpd_test_lib:verify_request(ip_comm, Address, Port, node(), + "GET / " ++ + lists:duplicate(100, $A) ++ "\r\n\r\n", + [{statuscode, 413}, + %% Server will send lowest version + %% as it will not get to the + %% client version + %% before aborting + {version, "HTTP/0.9"}]), + %% Not so long but wrong + ok = httpd_test_lib:verify_request(ip_comm, Address, Port, node(), + lists:duplicate(5, $A) ++ " / " + "HTTP/1.1\r\n\r\n", + [{statuscode, 501}, + %% Server will send lowest version + %% as it will not get to the + %% client version + %% before aborting + {version, "HTTP/1.1"}]), + %% Too long + ok = httpd_test_lib:verify_request(ip_comm, Address, Port, node(), + lists:duplicate(100, $A) ++ " / " + "HTTP/1.1\r\n\r\n", + [{statuscode, 413}, + %% Server will send lowest version + %% as it will not get to the + %% client version + %% before aborting + {version, "HTTP/0.9"}]), + inets:stop(httpd, Pid). + %%------------------------------------------------------------------------- +script_nocache() -> + [{doc,"Test nocache option for mod_cgi and mod_esi"}]. +script_nocache(Config) when is_list(Config) -> + Normal = {no_header, "cache-control"}, + NoCache = {header, "cache-control", "no-cache"}, + verify_script_nocache(Config, false, false, Normal, Normal), + verify_script_nocache(Config, true, false, NoCache, Normal), + verify_script_nocache(Config, false, true, Normal, NoCache), + verify_script_nocache(Config, true, true, NoCache, NoCache). + +%%------------------------------------------------------------------------- erl_script_nocache_opt(doc) -> ["Test that too long headers's get 413 HTTP code"]; erl_script_nocache_opt(suite) -> @@ -225,155 +274,49 @@ erl_script_nocache_opt(Config) when is_list(Config) -> inets:stop(httpd, Pid). %%------------------------------------------------------------------------- -%%------------------------------------------------------------------------- -script_nocache(doc) -> - ["Test nocache option for mod_cgi and mod_esi"]; -script_nocache(suite) -> - []; -script_nocache(Config) when is_list(Config) -> - Normal = {no_header, "cache-control"}, - NoCache = {header, "cache-control", "no-cache"}, - verify_script_nocache(Config, false, false, Normal, Normal), - verify_script_nocache(Config, true, false, NoCache, Normal), - verify_script_nocache(Config, false, true, Normal, NoCache), - verify_script_nocache(Config, true, true, NoCache, NoCache), - ok. -verify_script_nocache(Config, CgiNoCache, EsiNoCache, CgiOption, EsiOption) -> - HttpdConf = ?config(httpd_conf, Config), - CgiScript = ?config(cgi_printenv, Config), - CgiDir = ?config(cgi_dir, Config), - {ok, Pid} = inets:start(httpd, [{port, 0}, - {script_alias, - {"/cgi-bin/", CgiDir ++ "/"}}, - {script_nocache, CgiNoCache}, - {erl_script_alias, - {"/cgi-bin/erl", [httpd_example,io]}}, - {erl_script_nocache, EsiNoCache} - | HttpdConf]), - Info = httpd:info(Pid), - Port = proplists:get_value(port, Info), - Address = proplists:get_value(bind_address, Info), - ok = httpd_test_lib:verify_request(ip_comm, Address, Port, node(), - "GET /cgi-bin/" ++ CgiScript ++ - " HTTP/1.0\r\n\r\n", - [{statuscode, 200}, - CgiOption, - {version, "HTTP/1.0"}]), - ok = httpd_test_lib:verify_request(ip_comm, Address, Port, node(), - "GET /cgi-bin/erl/httpd_example:get " - "HTTP/1.0\r\n\r\n", - [{statuscode, 200}, - EsiOption, - {version, "HTTP/1.0"}]), - inets:stop(httpd, Pid). - - -%%------------------------------------------------------------------------- %%------------------------------------------------------------------------- -escaped_url_in_error_body(doc) -> - ["Test Url-encoding see OTP-8940"]; -escaped_url_in_error_body(suite) -> - []; -escaped_url_in_error_body(Config) when is_list(Config) -> - %% <CONDITIONAL-SKIP> - %% This skip is due to a problem on windows with long path's - %% If a path is too long file:open fails with, for example, eio. - %% Until that problem is fixed, we skip this case... - Skippable = [win32], - Condition = fun() -> ?OS_BASED_SKIP(Skippable) end, - ?NON_PC_TC_MAYBE_SKIP(Config, Condition), - %% </CONDITIONAL-SKIP> - - tsp("escaped_url_in_error_body -> entry"), +escaped_url_in_error_body() -> + [{doc, "Test Url-encoding see OTP-8940"}]. +escaped_url_in_error_body(Config) when is_list(Config) -> HttpdConf = ?config(httpd_conf, Config), {ok, Pid} = inets:start(httpd, [{port, 0} | HttpdConf]), Info = httpd:info(Pid), Port = proplists:get_value(port, Info), - _Address = proplists:get_value(bind_address, Info), - - %% Request 1 - tss(1000), - tsp("escaped_url_in_error_body -> request 1"), URL1 = ?URL_START ++ integer_to_list(Port), - %% Make sure the server is ok, by making a request for a valid page - case httpc:request(get, {URL1 ++ "/dummy.html", []}, - [{url_encode, false}, - {version, "HTTP/1.0"}], - [{full_result, false}]) of - {ok, {200, _}} -> - %% Don't care about the the body, just that we get a ok response - ok; - {ok, {StatusCode1, Body1}} -> - tsf({unexpected_ok_1, StatusCode1, Body1}) - end, - - %% Request 2 - tss(1000), - tsp("escaped_url_in_error_body -> request 2"), - %% Make sure the server is ok, by making a request for a valid page - case httpc:request(get, {URL1 ++ "/dummy.html", []}, - [{url_encode, true}, - {version, "HTTP/1.0"}], - [{full_result, false}]) of - {ok, {200, _}} -> - %% Don't care about the the body, just that we get a ok response - ok; - {ok, {StatusCode2, Body2}} -> - tsf({unexpected_ok_2, StatusCode2, Body2}) - end, - - %% Request 3 - tss(1000), - tsp("escaped_url_in_error_body -> request 3"), + + %% Sanity check + {ok, {200, _}} = httpc:request(get, {URL1 ++ "/dummy.html", []}, + [{url_encode, false}, + {version, "HTTP/1.0"}], + [{full_result, false}]), + {ok, {200, _}} = httpc:request(get, {URL1 ++ "/dummy.html", []}, + [{url_encode, true}, + {version, "HTTP/1.0"}], + [{full_result, false}]), + %% Ask for a non-existing page(1) Path = "/<b>this_is_bold<b>", HTMLEncodedPath = http_util:html_encode(Path), URL2 = URL1 ++ Path, - case httpc:request(get, {URL2, []}, - [{url_encode, true}, - {version, "HTTP/1.0"}], - [{full_result, false}]) of - {ok, {404, Body3}} -> - case find_URL_path(string:tokens(Body3, " ")) of - HTMLEncodedPath -> - ok; - BadPath3 -> - tsf({unexpected_path_3, HTMLEncodedPath, BadPath3}) - end; - {ok, UnexpectedOK3} -> - tsf({unexpected_ok_3, UnexpectedOK3}) - end, + {ok, {404, Body3}} = httpc:request(get, {URL2, []}, + [{url_encode, true}, + {version, "HTTP/1.0"}], + [{full_result, false}]), - %% Request 4 - tss(1000), - tsp("escaped_url_in_error_body -> request 4"), - %% Ask for a non-existing page(2) - case httpc:request(get, {URL2, []}, - [{url_encode, false}, - {version, "HTTP/1.0"}], - [{full_result, false}]) of - {ok, {404, Body4}} -> - case find_URL_path(string:tokens(Body4, " ")) of - HTMLEncodedPath -> - ok; - BadPath4 -> - tsf({unexpected_path_4, HTMLEncodedPath, BadPath4}) - end; - {ok, UnexpectedOK4} -> - tsf({unexpected_ok_4, UnexpectedOK4}) - end, - tss(1000), - tsp("escaped_url_in_error_body -> stop inets"), - inets:stop(httpd, Pid), - tsp("escaped_url_in_error_body -> done"), - ok. + HTMLEncodedPath = find_URL_path(string:tokens(Body3, " ")), + {ok, {404, Body4}} = httpc:request(get, {URL2, []}, + [{url_encode, false}, + {version, "HTTP/1.0"}], + [{full_result, false}]), + + HTMLEncodedPath = find_URL_path(string:tokens(Body4, " ")), + inets:stop(httpd, Pid). %%------------------------------------------------------------------------- -%%------------------------------------------------------------------------- keep_alive_timeout(doc) -> ["Test the keep_alive_timeout option"]; @@ -393,7 +336,6 @@ keep_alive_timeout(Config) when is_list(Config) -> inets:stop(httpd, Pid). %%------------------------------------------------------------------------- -%%------------------------------------------------------------------------- script_timeout(doc) -> ["Test the httpd script_timeout option"]; @@ -423,12 +365,10 @@ verify_script_timeout(Config, ScriptTimeout, StatusCode) -> {version, "HTTP/1.0"}]), inets:stop(httpd, Pid). - -%%------------------------------------------------------------------------- %%------------------------------------------------------------------------- -slowdose(doc) -> - ["Testing minimum bytes per second option"]; +slowdose() -> + [{doc, "Testing minimum bytes per second option"}]. slowdose(Config) when is_list(Config) -> HttpdConf = ?config(httpd_conf, Config), {ok, Pid} = inets:start(httpd, [{port, 0}, {minimum_bytes_per_second, 200}|HttpdConf]), @@ -439,6 +379,40 @@ slowdose(Config) when is_list(Config) -> after 6000 -> {error, closed} = gen_tcp:send(Socket, "Hey") end. + +%%------------------------------------------------------------------------- +%% Internal functions +%%------------------------------------------------------------------------- + +verify_script_nocache(Config, CgiNoCache, EsiNoCache, CgiOption, EsiOption) -> + HttpdConf = ?config(httpd_conf, Config), + CgiScript = ?config(cgi_printenv, Config), + CgiDir = ?config(cgi_dir, Config), + {ok, Pid} = inets:start(httpd, [{port, 0}, + {script_alias, + {"/cgi-bin/", CgiDir ++ "/"}}, + {script_nocache, CgiNoCache}, + {erl_script_alias, + {"/cgi-bin/erl", [httpd_example,io]}}, + {erl_script_nocache, EsiNoCache} + | HttpdConf]), + Info = httpd:info(Pid), + Port = proplists:get_value(port, Info), + Address = proplists:get_value(bind_address, Info), + ok = httpd_test_lib:verify_request(ip_comm, Address, Port, node(), + "GET /cgi-bin/" ++ CgiScript ++ + " HTTP/1.0\r\n\r\n", + [{statuscode, 200}, + CgiOption, + {version, "HTTP/1.0"}]), + ok = httpd_test_lib:verify_request(ip_comm, Address, Port, node(), + "GET /cgi-bin/erl/httpd_example:get " + "HTTP/1.0\r\n\r\n", + [{statuscode, 200}, + EsiOption, + {version, "HTTP/1.0"}]), + inets:stop(httpd, Pid). + find_URL_path([]) -> ""; find_URL_path(["URL", URL | _]) -> @@ -446,21 +420,6 @@ find_URL_path(["URL", URL | _]) -> find_URL_path([_ | Rest]) -> find_URL_path(Rest). - -tsp(F) -> - inets_test_lib:tsp(F). -tsp(F, A) -> - inets_test_lib:tsp(F, A). - -tsf(Reason) -> - inets_test_lib:tsf(Reason). - -tss(Time) -> - inets_test_lib:tss(Time). - - - - skip(Reason) -> {skip, Reason}. diff --git a/lib/inets/test/httpd_block.erl b/lib/inets/test/httpd_block.erl index 706d014bda..9790623b6f 100644 --- a/lib/inets/test/httpd_block.erl +++ b/lib/inets/test/httpd_block.erl @@ -111,8 +111,7 @@ block_disturbing_active_timeout_not_released(Type, Port, Host, Node) -> process_flag(trap_exit, true), Poller = long_poll(Type, Host, Port, Node, 200, 60000), ct:sleep(15000), - Blocker = blocker(Node, Host, Port, 50000), - await_normal_process_exit(Blocker, "blocker", 50000), + ok = httpd_block(undefined, Port, disturbing, 50000), await_normal_process_exit(Poller, "poller", 30000), blocked = get_admin_state(Node, Host, Port), process_flag(trap_exit, false), @@ -123,8 +122,7 @@ block_disturbing_active_timeout_released(Type, Port, Host, Node) -> process_flag(trap_exit, true), Poller = long_poll(Type, Host, Port, Node, 200, 40000), ct:sleep(5000), - Blocker = blocker(Node, Host, Port, 10000), - await_normal_process_exit(Blocker, "blocker", 15000), + ok = httpd_block(undefined, Port, disturbing, 10000), await_suite_failed_process_exit(Poller, "poller", 40000, connection_closed), blocked = get_admin_state(Node, Host, Port), diff --git a/lib/inets/test/httpd_test_lib.erl b/lib/inets/test/httpd_test_lib.erl index ed466fd727..647fa6f6c1 100644 --- a/lib/inets/test/httpd_test_lib.erl +++ b/lib/inets/test/httpd_test_lib.erl @@ -91,19 +91,10 @@ verify_request(SocketType, Host, Port, Node, RequestStr, Options, TimeOut) when (is_integer(TimeOut) orelse (TimeOut =:= infinity)) -> verify_request(SocketType, Host, Port, [], Node, RequestStr, Options, TimeOut). -verify_request(SocketType, Host, Port, TranspOpts0, Node, RequestStr, Options, TimeOut) -> - %% For now, until we modernize the httpd tests - TranspOpts = - case lists:member(inet6, TranspOpts0) of - true -> - TranspOpts0; - false -> - [inet | TranspOpts0] - end, - +verify_request(SocketType, Host, Port, TranspOpts, Node, RequestStr, Options, TimeOut) -> try inets_test_lib:connect_bin(SocketType, Host, Port, TranspOpts) of {ok, Socket} -> - SendRes = inets_test_lib:send(SocketType, Socket, RequestStr), + ok = inets_test_lib:send(SocketType, Socket, RequestStr), State = case inets_regexp:match(RequestStr, "printenv") of nomatch -> #state{}; diff --git a/lib/inets/test/inets_sup_SUITE.erl b/lib/inets/test/inets_sup_SUITE.erl index 12b85a816f..60979278fc 100644 --- a/lib/inets/test/inets_sup_SUITE.erl +++ b/lib/inets/test/inets_sup_SUITE.erl @@ -77,75 +77,32 @@ end_per_suite(_) -> %% variable, but should NOT alter/remove any existing entries. %%-------------------------------------------------------------------- init_per_testcase(httpd_subtree, Config) -> - io:format("init_per_testcase(httpd_subtree) -> entry with" - "~n Config: ~p" - "~n", [Config]), Dog = test_server:timetrap(?t:minutes(1)), NewConfig = lists:keydelete(watchdog, 1, Config), - - DataDir = ?config(data_dir, Config), PrivDir = ?config(priv_dir, Config), - ServerROOT = filename:join(PrivDir, "server_root"), - DocROOT = filename:join(PrivDir, "htdocs"), - ConfDir = filename:join(ServerROOT, "conf"), - - io:format("init_per_testcase(httpd_subtree) -> create dir(s)" - "~n", []), - file:make_dir(ServerROOT), %% until http_test is cleaned up! - ok = file:make_dir(DocROOT), - ok = file:make_dir(ConfDir), - - io:format("init_per_testcase(httpd_subtree) -> copy file(s)" - "~n", []), - {ok, _} = inets_test_lib:copy_file("simple.conf", DataDir, PrivDir), - {ok, _} = inets_test_lib:copy_file("mime.types", DataDir, ConfDir), - - io:format("init_per_testcase(httpd_subtree) -> write file(s)" - "~n", []), - ConfFile = filename:join(PrivDir, "simple.conf"), - {ok, Fd} = file:open(ConfFile, [append]), - ok = file:write(Fd, "ServerRoot " ++ ServerROOT ++ "\n"), - ok = file:write(Fd, "DocumentRoot " ++ DocROOT ++ "\n"), - ok = file:close(Fd), - - %% To make sure application:set_env is not overwritten by any - %% app-file settings. - io:format("init_per_testcase(httpd_subtree) -> load inets app" - "~n", []), - application:load(inets), - io:format("init_per_testcase(httpd_subtree) -> update inets env" - "~n", []), - ok = application:set_env(inets, services, [{httpd, ConfFile}]), - + + SimpleConfig = [{port, 0}, + {server_name,"www.test"}, + {modules, [mod_get]}, + {server_root, PrivDir}, + {document_root, PrivDir}, + {bind_address, any}, + {ipfamily, inet}], try - io:format("init_per_testcase(httpd_subtree) -> start inets app" - "~n", []), - ok = inets:start(), - io:format("init_per_testcase(httpd_subtree) -> done" - "~n", []), - [{watchdog, Dog}, {server_root, ServerROOT}, {doc_root, DocROOT}, - {conf_dir, ConfDir}| NewConfig] + inets:start(), + inets:start(httpd, SimpleConfig), + [{watchdog, Dog} | NewConfig] catch _:Reason -> - io:format("init_per_testcase(httpd_subtree) -> " - "failed starting inets - cleanup" - "~n Reason: ~p" - "~n", [Reason]), - application:unset_env(inets, services), - application:unload(inets), + inets:stop(), exit({failed_starting_inets, Reason}) end; -init_per_testcase(Case, Config) -> - io:format("init_per_testcase(~p) -> entry with" - "~n Config: ~p" - "~n", [Case, Config]), +init_per_testcase(_Case, Config) -> Dog = test_server:timetrap(?t:minutes(5)), NewConfig = lists:keydelete(watchdog, 1, Config), - Stop = inets:stop(), - io:format("init_per_testcase(~p) -> Stop: ~p" - "~n", [Case, Stop]), + inets:stop(), ok = inets:start(), [{watchdog, Dog} | NewConfig]. @@ -260,10 +217,10 @@ tftpd_worker(suite) -> []; tftpd_worker(Config) when is_list(Config) -> [] = supervisor:which_children(tftp_sup), - {ok, Pid0} = inets:start(tftpd, [{host, "localhost"}, - {port, inet_port()}]), - {ok, _Pid1} = inets:start(tftpd, [{host, "localhost"}, - {port, inet_port()}], stand_alone), + {ok, Pid0} = inets:start(tftpd, [{host, inets_test_lib:hostname()}, + {port, 0}]), + {ok, _Pid1} = inets:start(tftpd, [{host, inets_test_lib:hostname()}, + {port, 0}], stand_alone), [{_,Pid0, worker, _}] = supervisor:which_children(tftp_sup), inets:stop(tftpd, Pid0), @@ -280,30 +237,21 @@ httpd_subtree(doc) -> httpd_subtree(suite) -> []; httpd_subtree(Config) when is_list(Config) -> - io:format("httpd_subtree -> entry with" - "~n Config: ~p" - "~n", [Config]), - %% Check that we have the httpd top supervisor - io:format("httpd_subtree -> verify inets~n", []), {ok, _} = verify_child(inets_sup, httpd_sup, supervisor), %% Check that we have the httpd instance supervisor - io:format("httpd_subtree -> verify httpd~n", []), {ok, Id} = verify_child(httpd_sup, httpd_instance_sup, supervisor), {httpd_instance_sup, Addr, Port} = Id, Instance = httpd_util:make_name("httpd_instance_sup", Addr, Port), %% Check that we have the expected httpd instance children - io:format("httpd_subtree -> verify httpd instance children " - "(acceptor, misc and manager)~n", []), {ok, _} = verify_child(Instance, httpd_connection_sup, supervisor), {ok, _} = verify_child(Instance, httpd_acceptor_sup, supervisor), {ok, _} = verify_child(Instance, httpd_misc_sup, supervisor), {ok, _} = verify_child(Instance, httpd_manager, worker), %% Check that the httpd instance acc supervisor has children - io:format("httpd_subtree -> verify acc~n", []), InstanceAcc = httpd_util:make_name("httpd_acceptor_sup", Addr, Port), case supervisor:which_children(InstanceAcc) of [_ | _] -> @@ -328,15 +276,7 @@ httpd_subtree(Config) when is_list(Config) -> verify_child(Parent, Child, Type) -> -%% io:format("verify_child -> entry with" -%% "~n Parent: ~p" -%% "~n Child: ~p" -%% "~n Type: ~p" -%% "~n", [Parent, Child, Type]), Children = supervisor:which_children(Parent), -%% io:format("verify_child -> which children" -%% "~n Children: ~p" -%% "~n", [Children]), verify_child(Children, Parent, Child, Type). verify_child([], Parent, Child, _Type) -> @@ -344,21 +284,12 @@ verify_child([], Parent, Child, _Type) -> verify_child([{Id, _Pid, Type2, Mods}|Children], Parent, Child, Type) -> case lists:member(Child, Mods) of true when (Type2 =:= Type) -> -%% io:format("verify_child -> found with expected type" -%% "~n Id: ~p" -%% "~n", [Id]), {ok, Id}; true when (Type2 =/= Type) -> -%% io:format("verify_child -> found with unexpected type" -%% "~n Type2: ~p" -%% "~n Id: ~p" -%% "~n", [Type2, Id]), {error, {wrong_type, Type2, Child, Parent}}; false -> verify_child(Children, Parent, Child, Type) end. - - %%------------------------------------------------------------------------- %% httpc_subtree @@ -368,47 +299,19 @@ httpc_subtree(doc) -> httpc_subtree(suite) -> []; httpc_subtree(Config) when is_list(Config) -> - tsp("httpc_subtree -> entry with" - "~n Config: ~p", [Config]), - - tsp("httpc_subtree -> start inets service httpc with profile foo"), - {ok, _Foo} = inets:start(httpc, [{profile, foo}]), + {ok, Foo} = inets:start(httpc, [{profile, foo}]), - tsp("httpc_subtree -> " - "start stand-alone inets service httpc with profile bar"), - {ok, _Bar} = inets:start(httpc, [{profile, bar}], stand_alone), + {ok, Bar} = inets:start(httpc, [{profile, bar}], stand_alone), - tsp("httpc_subtree -> retreive list of httpc instances"), HttpcChildren = supervisor:which_children(httpc_profile_sup), - tsp("httpc_subtree -> HttpcChildren: ~n~p", [HttpcChildren]), - - tsp("httpc_subtree -> verify httpc stand-alone instances"), + {value, {httpc_manager, _, worker, [httpc_manager]}} = lists:keysearch(httpc_manager, 1, HttpcChildren), - tsp("httpc_subtree -> verify httpc (named) instances"), - {value,{{httpc,foo}, Pid, worker, [httpc_manager]}} = + {value,{{httpc,foo}, _Pid, worker, [httpc_manager]}} = lists:keysearch({httpc, foo}, 1, HttpcChildren), false = lists:keysearch({httpc, bar}, 1, HttpcChildren), - tsp("httpc_subtree -> stop inets"), - inets:stop(httpc, Pid), - - tsp("httpc_subtree -> done"), - ok. - -inet_port() -> - {ok, Socket} = gen_tcp:listen(0, [{reuseaddr, true}]), - {ok, Port} = inet:port(Socket), - gen_tcp:close(Socket), - Port. - - -tsp(F) -> - tsp(F, []). -tsp(F, A) -> - test_server:format("~p ~p:" ++ F ++ "~n", [self(), ?MODULE | A]). - -tsf(Reason) -> - test_server:fail(Reason). + inets:stop(httpc, Foo), + exit(Bar, normal). diff --git a/lib/inets/test/inets_sup_SUITE_data/mime.types b/lib/inets/test/inets_sup_SUITE_data/mime.types deleted file mode 100644 index e52d345ff7..0000000000 --- a/lib/inets/test/inets_sup_SUITE_data/mime.types +++ /dev/null @@ -1,3 +0,0 @@ -# MIME type Extension -text/html html htm -text/plain asc txt diff --git a/lib/inets/test/inets_sup_SUITE_data/simple.conf b/lib/inets/test/inets_sup_SUITE_data/simple.conf deleted file mode 100644 index e1429b4a28..0000000000 --- a/lib/inets/test/inets_sup_SUITE_data/simple.conf +++ /dev/null @@ -1,6 +0,0 @@ -Port 8888 -ServerName www.test -SocketType ip_comm -Modules mod_get -ServerAdmin [email protected] - diff --git a/lib/inets/test/old_httpd_SUITE.erl b/lib/inets/test/old_httpd_SUITE.erl index 3e1a1a3845..74c11f71ba 100644 --- a/lib/inets/test/old_httpd_SUITE.erl +++ b/lib/inets/test/old_httpd_SUITE.erl @@ -182,22 +182,27 @@ groups() -> %% ip_load_medium, %% ip_load_heavy, %%ip_dos_hostname, - ip_time_test - %% Replaced by load_config - %% ip_restart_no_block, - %% ip_restart_disturbing_block, - %% ip_restart_non_disturbing_block, - %% ip_block_disturbing_idle, - %% ip_block_non_disturbing_idle, - %% ip_block_503, - %% ip_block_disturbing_active, - %% ip_block_non_disturbing_active, - %% ip_block_disturbing_active_timeout_not_released, - %% ip_block_disturbing_active_timeout_released, - %% ip_block_non_disturbing_active_timeout_not_released, - %% ip_block_non_disturbing_active_timeout_released, - %% ip_block_disturbing_blocker_dies, - %% ip_block_non_disturbing_blocker_dies + ip_time_test, + %% Only used through load_config + %% but we still need these tests + %% should be cleaned up and moved to new test suite + %%ip_restart_no_block, + %%ip_restart_disturbing_block, + %%ip_restart_non_disturbing_block, + %% Tested in inets_SUITE + %%ip_block_disturbing_idle, + %%ip_block_non_disturbing_idle, + ip_block_503 + %% Tested in new httpd_SUITE + %%ip_block_disturbing_active, + %%ip_block_non_disturbing_active, + %%ip_block_disturbing_blocker_dies, + %%ip_block_non_disturbing_blocker_dies + %% No longer relevant + %%ip_block_disturbing_active_timeout_not_released, + %%ip_block_disturbing_active_timeout_released, + %%ip_block_non_disturbing_active_timeout_not_released, + %%ip_block_non_disturbing_active_timeout_released, ]}, {ssl, [], [{group, essl}]}, {essl, [], diff --git a/lib/inets/test/property_test/README b/lib/inets/test/property_test/README new file mode 100644 index 0000000000..57602bf719 --- /dev/null +++ b/lib/inets/test/property_test/README @@ -0,0 +1,12 @@ + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%%% %%% +%%% WARNING %%% +%%% %%% +%%% This is experimental code which may be changed or removed %%% +%%% anytime without any warning. %%% +%%% %%% +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +The test in this directory are written assuming that the user has a QuickCheck license. They are to be run manually. Some may be possible to be run with other tools, e.g. PropEr. + diff --git a/lib/inets/test/property_test/ftp_simple_client_server.erl b/lib/inets/test/property_test/ftp_simple_client_server.erl new file mode 100644 index 0000000000..40e630ee5c --- /dev/null +++ b/lib/inets/test/property_test/ftp_simple_client_server.erl @@ -0,0 +1,306 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2004-2014. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% + +-module(ftp_simple_client_server). + +-compile(export_all). + +-ifndef(EQC). +-ifndef(PROPER). +-define(EQC,true). +%%-define(PROPER,true). +-endif. +-endif. + + +-ifdef(EQC). + +-include_lib("eqc/include/eqc.hrl"). +-include_lib("eqc/include/eqc_statem.hrl"). +-define(MOD_eqc, eqc). +-define(MOD_eqc_gen, eqc_gen). +-define(MOD_eqc_statem, eqc_statem). + +-else. +-ifdef(PROPER). + +-include_lib("proper/include/proper.hrl"). +-define(MOD_eqc, proper). +-define(MOD_eqc_gen, proper_gen). +-define(MOD_eqc_statem, proper_statem). + +-endif. +-endif. + +-record(state, { + initialized = false, + priv_dir, + data_dir, + servers = [], % [ {IP,Port,Userid,Pwd} ] + clients = [], % [ client_ref() ] + store = [] % [ {Name,Contents} ] + }). + +-define(fmt(F,A), io:format(F,A)). +%%-define(fmt(F,A), ok). + +-define(v(K,L), proplists:get_value(K,L)). + +%%%================================================================ +%%% +%%% Properties +%%% + +%% This function is for normal eqc calls: +prop_ftp() -> + {ok,PWD} = file:get_cwd(), + prop_ftp(filename:join([PWD,?MODULE_STRING++"_data"]), + filename:join([PWD,?MODULE_STRING,"_files"])). + +%% This function is for calls from common_test test cases: +prop_ftp(Config) -> + prop_ftp(filename:join([?v(property_dir,Config), ?MODULE_STRING++"_data"]), + ?v(priv_dir,Config) ). + + +prop_ftp(DataDir, PrivDir) -> + S0 = #state{data_dir = DataDir, + priv_dir = PrivDir}, + ?FORALL(Cmds, more_commands(10,commands(?MODULE,S0)), + aggregate(command_names(Cmds), + begin {_H,S,Result} = run_commands(?MODULE,Cmds), + % io:format('**** Result=~p~n',[Result]), + % io:format('**** S=~p~n',[S]), + % io:format('**** _H=~p~n',[_H]), + % io:format('**** Cmds=~p~n',[Cmds]), + [cmnd_stop_server(X) || X <- S#state.servers], + [inets:stop(ftpc,X) || {ok,X} <- S#state.clients], + Result==ok + end) + ). + +%%%================================================================ +%%% +%%% State model +%%% + +%% @doc Returns the state in which each test case starts. (Unless a different +%% initial state is supplied explicitly to, e.g. commands/2.) +-spec initial_state() ->?MOD_eqc_statem:symbolic_state(). +initial_state() -> + ?fmt("Initial_state()~n",[]), + #state{}. + +%% @doc Command generator, S is the current state +-spec command(S :: ?MOD_eqc_statem:symbolic_state()) -> ?MOD_eqc_gen:gen(eqc_statem:call()). + +command(#state{initialized=false, + priv_dir=PrivDir}) -> + {call,?MODULE,cmnd_init,[PrivDir]}; + +command(#state{servers=[], + priv_dir=PrivDir, + data_dir=DataDir}) -> + {call,?MODULE,cmnd_start_server,[PrivDir,DataDir]}; + +command(#state{servers=Ss=[_|_], + clients=[]}) -> + {call,?MODULE,cmnd_start_client,[oneof(Ss)]}; + +command(#state{servers=Ss=[_|_], + clients=Cs=[_|_], + store=Store=[_|_] + }) -> + frequency([ + { 5, {call,?MODULE,cmnd_start_client,[oneof(Ss)]}}, + { 5, {call,?MODULE,cmnd_stop_client,[oneof(Cs)]}}, + {10, {call,?MODULE,cmnd_put,[oneof(Cs),file_path(),file_contents()]}}, + {20, {call,?MODULE,cmnd_get,[oneof(Cs),oneof(Store)]}}, + {10, {call,?MODULE,cmnd_delete,[oneof(Cs),oneof(Store)]}} + ]); + +command(#state{servers=Ss=[_|_], + clients=Cs=[_|_], + store=[] + }) -> + frequency([ + {5, {call,?MODULE,cmnd_start_client,[oneof(Ss)]}}, + {5, {call,?MODULE,cmnd_stop_client,[oneof(Cs)]}}, + {10, {call,?MODULE,cmnd_put,[oneof(Cs),file_path(),file_contents()]}} + ]). + +%% @doc Precondition, checked before command is added to the command sequence. +-spec precondition(S :: ?MOD_eqc_statem:symbolic_state(), C :: ?MOD_eqc_statem:call()) -> boolean(). + +precondition(#state{clients=Cs}, {call, _, cmnd_put, [C,_,_]}) -> lists:member(C,Cs); + +precondition(#state{clients=Cs, store=Store}, + {call, _, cmnd_get, [C,X]}) -> lists:member(C,Cs) andalso lists:member(X,Store); + +precondition(#state{clients=Cs, store=Store}, + {call, _, cmnd_delete, [C,X]}) -> lists:member(C,Cs) andalso lists:member(X,Store); + +precondition(#state{servers=Ss}, {call, _, cmnd_start_client, _}) -> Ss =/= []; + +precondition(#state{clients=Cs}, {call, _, cmnd_stop_client, [C]}) -> lists:member(C,Cs); + +precondition(#state{initialized=IsInit}, {call, _, cmnd_init, _}) -> IsInit==false; + +precondition(_S, {call, _, _, _}) -> true. + + +%% @doc Postcondition, checked after command has been evaluated +%% Note: S is the state before next_state(S,_,C) +-spec postcondition(S :: ?MOD_eqc_statem:dynamic_state(), C :: ?MOD_eqc_statem:call(), + Res :: term()) -> boolean(). + +postcondition(_S, {call, _, cmnd_get, [_,{_Name,Expected}]}, {ok,Value}) -> + Value == Expected; + +postcondition(S, {call, _, cmnd_delete, [_,{Name,_Expected}]}, ok) -> + ?fmt("file:read_file(..) = ~p~n",[file:read_file(filename:join(S#state.priv_dir,Name))]), + {error,enoent} == file:read_file(filename:join(S#state.priv_dir,Name)); + +postcondition(S, {call, _, cmnd_put, [_,Name,Value]}, ok) -> + {ok,Bin} = file:read_file(filename:join(S#state.priv_dir,Name)), + Bin == unicode:characters_to_binary(Value); + +postcondition(_S, {call, _, cmnd_stop_client, _}, ok) -> true; + +postcondition(_S, {call, _, cmnd_start_client, _}, {ok,_}) -> true; + +postcondition(_S, {call, _, cmnd_init, _}, ok) -> true; + +postcondition(_S, {call, _, cmnd_start_server, _}, {ok,_}) -> true. + + +%% @doc Next state transformation, S is the current state. Returns next state. +-spec next_state(S :: ?MOD_eqc_statem:symbolic_state(), + V :: ?MOD_eqc_statem:var(), + C :: ?MOD_eqc_statem:call()) -> ?MOD_eqc_statem:symbolic_state(). + +next_state(S, _V, {call, _, cmnd_put, [_,Name,Val]}) -> + S#state{store = [{Name,Val} | lists:keydelete(Name,1,S#state.store)]}; + +next_state(S, _V, {call, _, cmnd_delete, [_,{Name,_Val}]}) -> + S#state{store = lists:keydelete(Name,1,S#state.store)}; + +next_state(S, V, {call, _, cmnd_start_client, _}) -> + S#state{clients = [V | S#state.clients]}; + +next_state(S, V, {call, _, cmnd_start_server, _}) -> + S#state{servers = [V | S#state.servers]}; + +next_state(S, _V, {call, _, cmnd_stop_client, [C]}) -> + S#state{clients = S#state.clients -- [C]}; + +next_state(S, _V, {call, _, cmnd_init, _}) -> + S#state{initialized=true}; + +next_state(S, _V, {call, _, _, _}) -> + S. + +%%%================================================================ +%%% +%%% Data model +%%% + +file_path() -> non_empty(list(alphanum_char())). +%%file_path() -> non_empty( list(oneof([alphanum_char(), utf8_char()])) ). + +%%file_contents() -> list(alphanum_char()). +file_contents() -> list(oneof([alphanum_char(), utf8_char()])). + +alphanum_char() -> oneof(lists:seq($a,$z) ++ lists:seq($A,$Z) ++ lists:seq($0,$9)). + +utf8_char() -> oneof("åäöÅÄÖ話话カタカナひらがな"). + +%%%================================================================ +%%% +%%% Commands doing something with the System Under Test +%%% + +cmnd_init(PrivDir) -> + ?fmt('Call cmnd_init(~p)~n',[PrivDir]), + os:cmd("killall vsftpd"), + clear_files(PrivDir), + ok. + +cmnd_start_server(PrivDir, DataDir) -> + ?fmt('Call cmnd_start_server(~p, ~p)~n',[PrivDir,DataDir]), + Cmnd = ["vsftpd ", filename:join(DataDir,"vsftpd.conf"), + " -oftpd_banner=erlang_otp_testing" + " -oanon_root=",PrivDir + ], + ?fmt("Cmnd=~s~n",[Cmnd]), + case os:cmd(Cmnd) of + [] -> + {ok,{"localhost",9999,"ftp","[email protected]"}}; + Other -> + {error,Other} + end. + +cmnd_stop_server({ok,{_Host,Port,_Usr,_Pwd}}) -> + os:cmd("kill `netstat -tpln | grep "++integer_to_list(Port)++" | awk '{print $7}' | awk -F/ '{print $1}'`"). + +cmnd_start_client({ok,{Host,Port,Usr,Pwd}}) -> + ?fmt('Call cmnd_start_client(~p)...',[{Host,Port,Usr,Pwd}]), + case inets:start(ftpc, [{host,Host},{port,Port}]) of + {ok,Client} -> + ?fmt("~p...",[{ok,Client}]), + case ftp:user(Client, Usr, Pwd) of + ok -> + ?fmt("OK!~n",[]), + {ok,Client}; + Other -> + ?fmt("Other1=~p~n",[Other]), + inets:stop(ftpc,Client), Other + end; + Other -> + ?fmt("Other2=~p~n",[Other]), + Other + end. + +cmnd_stop_client({ok,Client}) -> + ?fmt('Call cmnd_stop_client(~p)~n',[Client]), + inets:stop(ftpc, Client). %% -> ok | Other + +cmnd_delete({ok,Client}, {Name,_ExpectedValue}) -> + ?fmt('Call cmnd_delete(~p, ~p)~n',[Client,Name]), + R=ftp:delete(Client, Name), + ?fmt("R=~p~n",[R]), + R. + +cmnd_put({ok,Client}, Name, Value) -> + ?fmt('Call cmnd_put(~p, ~p, ~p)...',[Client, Name, Value]), + R = ftp:send_bin(Client, unicode:characters_to_binary(Value), Name), % ok | {error,Error} + ?fmt('~p~n',[R]), + R. + +cmnd_get({ok,Client}, {Name,_ExpectedValue}) -> + ?fmt('Call cmnd_get(~p, ~p)~n',[Client,Name]), + case ftp:recv_bin(Client, Name) of + {ok,Bin} -> {ok, unicode:characters_to_list(Bin)}; + Other -> Other + end. + + +clear_files(Dir) -> + os:cmd(["rm -fr ",filename:join(Dir,"*")]). diff --git a/lib/inets/test/property_test/ftp_simple_client_server_data/vsftpd.conf b/lib/inets/test/property_test/ftp_simple_client_server_data/vsftpd.conf new file mode 100644 index 0000000000..fd48e2abf0 --- /dev/null +++ b/lib/inets/test/property_test/ftp_simple_client_server_data/vsftpd.conf @@ -0,0 +1,26 @@ + +### +### Some parameters are given in the vsftpd start command. +### +### Typical command-line paramters are such that has a file path +### component like cert files. +### + + +listen=YES +listen_port=9999 +run_as_launching_user=YES +ssl_enable=NO +#allow_anon_ssl=YES + +background=YES + +write_enable=YES +anonymous_enable=YES +anon_upload_enable=YES +anon_mkdir_write_enable=YES +anon_other_write_enable=YES +anon_world_readable_only=NO + +### Shouldn't be necessary.... +require_ssl_reuse=NO diff --git a/lib/inets/vsn.mk b/lib/inets/vsn.mk index cbcf0362c9..029f6ac4d2 100644 --- a/lib/inets/vsn.mk +++ b/lib/inets/vsn.mk @@ -18,6 +18,6 @@ # %CopyrightEnd% APPLICATION = inets -INETS_VSN = 5.10 +INETS_VSN = 5.10.3 PRE_VSN = APP_VSN = "$(APPLICATION)-$(INETS_VSN)$(PRE_VSN)" diff --git a/lib/jinterface/doc/src/notes.xml b/lib/jinterface/doc/src/notes.xml index 8c45d187bc..8dc7dac64a 100644 --- a/lib/jinterface/doc/src/notes.xml +++ b/lib/jinterface/doc/src/notes.xml @@ -30,6 +30,60 @@ </header> <p>This document describes the changes made to the Jinterface application.</p> +<section><title>Jinterface 1.5.10</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Array now show meaningful values in exceptions.</p> + <p> + Own Id: OTP-12049</p> + </item> + <item> + <p> + Documentation improvements.</p> + <p> + Own Id: OTP-12050</p> + </item> + <item> + <p> + Include the cause when raising a new IOException, which + should make the reason for the exception clearer.</p> + <p> + Own Id: OTP-12075</p> + </item> + <item> + <p> + Arrays (here: md5 and freeVars) must not be compared with + equals, which is broken (compares identity).</p> + <p> + Own Id: OTP-12121</p> + </item> + </list> + </section> + +</section> + +<section><title>Jinterface 1.5.9</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Implement support for Maps</p> + <p> + The API and implementation are simplistic, like for lists + and tuples, using arrays and without any connection to + java.util.Map. (Thanks to Vlad Dumitrescu)</p> + <p> + Own Id: OTP-11703</p> + </item> + </list> + </section> + +</section> + <section><title>Jinterface 1.5.8</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/jinterface/java_src/com/ericsson/otp/erlang/AbstractConnection.java b/lib/jinterface/java_src/com/ericsson/otp/erlang/AbstractConnection.java index 9ba6a4a0ab..85d303689f 100644 --- a/lib/jinterface/java_src/com/ericsson/otp/erlang/AbstractConnection.java +++ b/lib/jinterface/java_src/com/ericsson/otp/erlang/AbstractConnection.java @@ -266,7 +266,7 @@ public abstract class AbstractConnection extends Thread { * * @param dest * the Erlang PID of the remote process. - * @param msg + * @param payload * the encoded message to send. * * @exception java.io.IOException @@ -959,7 +959,9 @@ public abstract class AbstractConnection extends Thread { } catch (final Exception e) { final String nn = peer.node(); close(); - throw new IOException("Error accepting connection from " + nn); + IOException ioe = new IOException("Error accepting connection from " + nn); + ioe.initCause(e); + throw ioe; } if (traceLevel >= handshakeThreshold) { System.out.println("<- MD5 ACCEPTED " + peer.host()); @@ -990,7 +992,9 @@ public abstract class AbstractConnection extends Thread { throw ae; } catch (final Exception e) { close(); - throw new IOException("Cannot connect to peer node"); + IOException ioe = new IOException("Cannot connect to peer node"); + ioe.initCause(e); + throw ioe; } } diff --git a/lib/jinterface/java_src/com/ericsson/otp/erlang/Makefile b/lib/jinterface/java_src/com/ericsson/otp/erlang/Makefile index f476d4594d..5a73fdb8a7 100644 --- a/lib/jinterface/java_src/com/ericsson/otp/erlang/Makefile +++ b/lib/jinterface/java_src/com/ericsson/otp/erlang/Makefile @@ -66,7 +66,7 @@ ifneq ($(V),0) JARFLAGS=-cfv endif -JAVA_OPTIONS = +JAVA_OPTIONS = -Xlint ifeq ($(TESTROOT),) RELEASE_PATH="$(ERL_TOP)/release/$(TARGET)" @@ -112,5 +112,4 @@ release_docs_spec: -# ---------------------------------------------------- - +# ----------------------------------------------------
\ No newline at end of file diff --git a/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpConnection.java b/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpConnection.java index 8e8bd473c8..e7a9d1092c 100644 --- a/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpConnection.java +++ b/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpConnection.java @@ -404,7 +404,7 @@ public class OtpConnection extends AbstractConnection { * * @param dest * the Erlang PID of the remote process. - * @param msg + * @param payload * the encoded message to send. * * @exception java.io.IOException diff --git a/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpErlangFun.java b/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpErlangFun.java index fc104e9564..c52909acc5 100644 --- a/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpErlangFun.java +++ b/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpErlangFun.java @@ -94,7 +94,7 @@ public class OtpErlangFun extends OtpErlangObject implements Serializable { return false; } } else { - if (!md5.equals(f.md5)) { + if (!Arrays.equals(md5, f.md5)) { return false; } } @@ -104,7 +104,7 @@ public class OtpErlangFun extends OtpErlangObject implements Serializable { if (freeVars == null) { return f.freeVars == null; } - return freeVars.equals(f.freeVars); + return Arrays.equals(freeVars, f.freeVars); } @Override diff --git a/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpErlangLong.java b/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpErlangLong.java index 7e3e2a7296..84b1355c54 100644 --- a/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpErlangLong.java +++ b/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpErlangLong.java @@ -51,8 +51,8 @@ public class OtpErlangLong extends OtpErlangObject implements Serializable, /** * Create an Erlang integer from the given value. * - * @param val - * the long value to use. + * @param v + * the big integer value to use. */ public OtpErlangLong(final BigInteger v) { if (v == null) { diff --git a/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpErlangMap.java b/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpErlangMap.java index 03c18e55a2..0254edd5da 100644 --- a/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpErlangMap.java +++ b/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpErlangMap.java @@ -58,15 +58,23 @@ public class OtpErlangMap extends OtpErlangObject implements Serializable, /** * Create a map from an array of terms. * - * @param elems + * @param keys * the array of terms to create the map from. - * @param start - * the offset of the first term to insert. + * @param kstart + * the offset of the first key to insert. + * @param kcount + * the number of keys to insert. + * @param values + * the array of values to create the map from. + * @param vstart + * the offset of the first value to insert. * @param vcount - * the number of terms to insert. + * the number of values to insert. * * @exception java.lang.IllegalArgumentException * if any array is empty (null) or contains null elements. + * @exception java.lang.IllegalArgumentException + * if kcount and vcount differ. */ public OtpErlangMap(final OtpErlangObject[] keys, final int kstart, final int kcount, final OtpErlangObject[] values, final int vstart, diff --git a/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpErlangPid.java b/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpErlangPid.java index fe81ce302d..f75e4353d0 100644 --- a/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpErlangPid.java +++ b/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpErlangPid.java @@ -162,7 +162,7 @@ public class OtpErlangPid extends OtpErlangObject implements Serializable, * Determine if two PIDs are equal. PIDs are equal if their components are * equal. * - * @param port + * @param o * the other PID to compare to. * * @return true if the PIDs are equal, false otherwise. diff --git a/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpErlangString.java b/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpErlangString.java index 6766b52ce5..a5e202c473 100644 --- a/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpErlangString.java +++ b/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpErlangString.java @@ -41,8 +41,6 @@ public class OtpErlangString extends OtpErlangObject implements Serializable, /** * Create an Erlang string from a list of integers. - * - * @return an Erlang string with Unicode code units. * * @throws OtpErlangException * for non-proper and non-integer lists. diff --git a/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpInputStream.java b/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpInputStream.java index 0d1342d796..f813594541 100644 --- a/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpInputStream.java +++ b/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpInputStream.java @@ -21,6 +21,7 @@ package com.ericsson.otp.erlang; import java.io.ByteArrayInputStream; import java.io.IOException; import java.math.BigDecimal; +import java.util.Arrays; /** * Provides a stream for decoding Erlang terms from external format. @@ -819,7 +820,7 @@ public class OtpInputStream extends ByteArrayInputStream { if (unsigned) { if (c < 0) { throw new OtpErlangDecodeException("Value not unsigned: " - + b); + + Arrays.toString(b)); } while (b[i] == 0) { i++; // Skip leading zero sign bytes @@ -844,7 +845,7 @@ public class OtpInputStream extends ByteArrayInputStream { if (b.length - i > 8) { // More than 64 bits of value throw new OtpErlangDecodeException( - "Value does not fit in long: " + b); + "Value does not fit in long: " + Arrays.toString(b)); } // Convert the necessary bytes for (v = c < 0 ? -1 : 0; i < b.length; i++) { diff --git a/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpMbox.java b/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpMbox.java index 0fd93b09f4..4a4a1e7f8f 100644 --- a/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpMbox.java +++ b/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpMbox.java @@ -69,6 +69,7 @@ package com.ericsson.otp.erlang; * notify other parties in a timely manner. * </p> * + * <p> * When retrieving messages from a mailbox that has received an exit signal, an * {@link OtpErlangExit OtpErlangExit} exception will be raised. Note that the * exception is queued in the mailbox along with other messages, and will not be @@ -420,7 +421,6 @@ public class OtpMbox { /** * Equivalent to <code>exit(new OtpErlangAtom(reason))</code>. - * </p> * * @see #exit(OtpErlangObject) */ diff --git a/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpMsg.java b/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpMsg.java index 6f507bf4bb..31a5d0fb8f 100644 --- a/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpMsg.java +++ b/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpMsg.java @@ -30,14 +30,14 @@ package com.ericsson.otp.erlang; * </p> * * <p> - * The header information that is available is as follows: <lu> + * The header information that is available is as follows: <ul> * <li> a tag indicating the type of message * <li> the intended recipient of the message, either as a * {@link OtpErlangPid pid} or as a String, but never both. * <li> (sometimes) the sender of the message. Due to some eccentric * characteristics of the Erlang distribution protocol, not all messages have * information about the sending process. In particular, only messages whose tag - * is {@link OtpMsg#regSendTag regSendTag} contain sender information. </lu> + * is {@link OtpMsg#regSendTag regSendTag} contain sender information. </ul> * * <p> * Message are sent using the Erlang external format (see separate diff --git a/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpOutputStream.java b/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpOutputStream.java index a78423db44..c98790bbd4 100644 --- a/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpOutputStream.java +++ b/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpOutputStream.java @@ -202,7 +202,7 @@ public class OtpOutputStream extends ByteArrayOutputStream { /** * Write an array of bytes to the stream. * - * @param buf + * @param bytes * the array of bytes to write. * */ @@ -637,7 +637,7 @@ public class OtpOutputStream extends ByteArrayOutputStream { * Write a positive short to the stream. The short is interpreted as a two's * complement unsigned short even if it is negative. * - * @param s + * @param us * the short to use. */ public void write_ushort(final short us) { diff --git a/lib/jinterface/test/jinterface_SUITE.erl b/lib/jinterface/test/jinterface_SUITE.erl index cb725164cd..00abc97ff5 100644 --- a/lib/jinterface/test/jinterface_SUITE.erl +++ b/lib/jinterface/test/jinterface_SUITE.erl @@ -38,7 +38,8 @@ java_exit_with_reason_any_term/1, status_handler_localStatus/1, status_handler_remoteStatus/1, status_handler_connAttempt/1, - maps/1 + maps/1, + fun_equals/1 ]). -include_lib("common_test/include/ct.hrl"). @@ -106,7 +107,8 @@ fundamental() -> register_and_whereis, % RegisterAndWhereis.java get_names, % GetNames.java boolean_atom, % BooleanAtom.java - maps % Maps.java + maps, % Maps.java + fun_equals % FunEquals.java ]. ping() -> @@ -691,6 +693,18 @@ maps(Config) when is_list(Config) -> []). %%%----------------------------------------------------------------- +fun_equals(doc) -> + ["FunEquals.java: " + "Test OtpErlangFun.equals()"]; +fun_equals(suite) -> + []; +fun_equals(Config) when is_list(Config) -> + ok = jitu:java(?config(java, Config), + ?config(data_dir, Config), + "FunEquals", + []). + +%%%----------------------------------------------------------------- %%% INTERNAL FUNCTIONS %%%----------------------------------------------------------------- send_receive(TestCaseTag,Fun,Config) -> diff --git a/lib/jinterface/test/jinterface_SUITE_data/FunEquals.java b/lib/jinterface/test/jinterface_SUITE_data/FunEquals.java new file mode 100644 index 0000000000..14f884cee7 --- /dev/null +++ b/lib/jinterface/test/jinterface_SUITE_data/FunEquals.java @@ -0,0 +1,71 @@ +/* + * %CopyrightBegin% + * + * Copyright Ericsson AB 2004-2010. All Rights Reserved. + * + * The contents of this file are subject to the Erlang Public License, + * Version 1.1, (the "License"); you may not use this file except in + * compliance with the License. You should have received a copy of the + * Erlang Public License along with this software. If not, it can be + * retrieved online at http://www.erlang.org/. + * + * Software distributed under the License is distributed on an "AS IS" + * basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + * the License for the specific language governing rights and limitations + * under the License. + * + * %CopyrightEnd% + */ + +import java.util.Arrays; + +import com.ericsson.otp.erlang.*; + +public class FunEquals { + + /* + Implements test case jinterface_SUITE:fun_equals/1 + + Test the function OtpErlangFun.equals() + */ + + public static void main(String argv[]) { + + OtpErlangPid pid = new OtpErlangPid("here", 4, 5, 0); + String module = "mod"; + int arity = 2; + byte[] md5 = new byte[]{3,5,7}; + int index = 2; + long old_index = 1; + long uniq= 2; + OtpErlangObject[] freeVars = new OtpErlangObject[]{ + new OtpErlangAtom("hej"), new OtpErlangLong(9) + }; + + OtpErlangFun f1 = new OtpErlangFun(pid, module, arity, md5, + index, old_index, uniq, freeVars); + OtpErlangFun f2 = new OtpErlangFun(pid, module, arity, copyArray(md5), + index, old_index, uniq, copyArray(freeVars)); + + if(!f1.equals(f2)) + fail(1); + + } + + private static void fail(int reason) { + System.exit(reason); + } + + private static byte[] copyArray(byte[] source) { + byte[] result = new byte[source.length]; + System.arraycopy(source, 0, result, 0, source.length); + return result; + } + + private static OtpErlangObject[] copyArray(OtpErlangObject[] source) { + OtpErlangObject[] result = new OtpErlangObject[source.length]; + System.arraycopy(source, 0, result, 0, source.length); + return result; + } + +} diff --git a/lib/jinterface/test/jinterface_SUITE_data/Makefile.src b/lib/jinterface/test/jinterface_SUITE_data/Makefile.src index a15ed1aa63..cd68f1ead5 100644 --- a/lib/jinterface/test/jinterface_SUITE_data/Makefile.src +++ b/lib/jinterface/test/jinterface_SUITE_data/Makefile.src @@ -47,7 +47,8 @@ JAVA_FILES = \ MboxSendReceive.java \ MboxLinkUnlink.java \ NodeStatusHandler.java \ - Maps.java + Maps.java \ + FunEquals.java CLASS_FILES = $(JAVA_FILES:.java=.class) diff --git a/lib/jinterface/vsn.mk b/lib/jinterface/vsn.mk index c50200fab6..7df92024bc 100644 --- a/lib/jinterface/vsn.mk +++ b/lib/jinterface/vsn.mk @@ -1 +1 @@ -JINTERFACE_VSN = 1.5.9 +JINTERFACE_VSN = 1.5.10 diff --git a/lib/kernel/doc/src/application.xml b/lib/kernel/doc/src/application.xml index 016151891c..7664fda4db 100644 --- a/lib/kernel/doc/src/application.xml +++ b/lib/kernel/doc/src/application.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1996</year><year>2013</year> + <year>1996</year><year>2014</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -459,8 +459,7 @@ Nodes = [cp1@cave, {cp2@cave, cp3@cave}]</code> <name>Module:start(StartType, StartArgs) -> {ok, Pid} | {ok, Pid, State} | {error, Reason}</name> <fsummary>Start an application</fsummary> <type> - <v>StartType = normal | {takeover,Node} | {failover,Node}</v> - <v> Node = node()</v> + <v>StartType = <seealso marker="#type-start_type">start_type()</seealso></v> <v>StartArgs = term()</v> <v>Pid = pid()</v> <v>State = term()</v> diff --git a/lib/kernel/doc/src/error_logger.xml b/lib/kernel/doc/src/error_logger.xml index 3815b0877c..df2f0b01ee 100644 --- a/lib/kernel/doc/src/error_logger.xml +++ b/lib/kernel/doc/src/error_logger.xml @@ -58,7 +58,7 @@ specific events. (<c>add_report_handler/1,2</c>). Also, there is a useful event handler in STDLIB for multi-file logging of events, see <c>log_mf_h(3)</c>.</p> - <p>Warning events was introduced in Erlang/OTP R9C. To retain + <p>Warning events were introduced in Erlang/OTP R9C. To retain backwards compatibility, these are by default tagged as errors, thus showing up as error reports in the logs. By using the command line flag <c><![CDATA[+W <w | i>]]></c>, they can instead diff --git a/lib/kernel/doc/src/file.xml b/lib/kernel/doc/src/file.xml index 8dae34431b..1c03efe7fd 100644 --- a/lib/kernel/doc/src/file.xml +++ b/lib/kernel/doc/src/file.xml @@ -428,14 +428,6 @@ </desc> </func> <func> - <name name="file_info" arity="1"/> - <fsummary>Get information about a file (deprecated)</fsummary> - <desc> - <p>This function is obsolete. Use <c>read_file_info/1,2</c> - instead.</p> - </desc> - </func> - <func> <name name="format_error" arity="1"/> <fsummary>Return a descriptive string for an error reason</fsummary> <desc> diff --git a/lib/kernel/doc/src/gen_tcp.xml b/lib/kernel/doc/src/gen_tcp.xml index dbd0d3c815..820ecd1e30 100644 --- a/lib/kernel/doc/src/gen_tcp.xml +++ b/lib/kernel/doc/src/gen_tcp.xml @@ -112,7 +112,12 @@ do_recv(Sock, Bs) -> <item> <p>If a socket has somehow been connected without using <c>gen_tcp</c>, use this option to pass the file - descriptor for it.</p> + descriptor for it. If <c>{ip, ip_address()}</c> + and/or <c>{port, port_number()}</c> is combined with + this option the fd will be bound to the given interface + and port before connecting. If these options are not given + it is assumed that the fd is already bound appropriately. + </p> </item> <tag><c>inet</c></tag> diff --git a/lib/kernel/doc/src/gen_udp.xml b/lib/kernel/doc/src/gen_udp.xml index 503725fe18..291d1b0da7 100644 --- a/lib/kernel/doc/src/gen_udp.xml +++ b/lib/kernel/doc/src/gen_udp.xml @@ -84,7 +84,12 @@ <item> <p>If a socket has somehow been opened without using <c>gen_udp</c>, use this option to pass the file - descriptor for it.</p> + descriptor for it. If <c><anno>Port</anno></c> is not set to 0 + and/or <c>{ip, ip_address()}</c> is combined with this option + the fd will be bound to the given interface and port after being + opened. If these options are not given it is assumed that the fd + is already bound appropriately. + </p> </item> <tag><c>inet6</c></tag> <item> diff --git a/lib/kernel/doc/src/inet.xml b/lib/kernel/doc/src/inet.xml index 4a48a5c3d8..50e1cc290c 100644 --- a/lib/kernel/doc/src/inet.xml +++ b/lib/kernel/doc/src/inet.xml @@ -361,7 +361,7 @@ fe80::204:acff:fe17:bf38 </item> <tag><c>send_dvi</c></tag> <item> - <p>Average packet size deviation in bytes received sent from the socket.</p> + <p>Average packet size deviation in bytes sent from the socket.</p> </item> <tag><c>send_max</c></tag> <item> diff --git a/lib/kernel/doc/src/kernel_app.xml b/lib/kernel/doc/src/kernel_app.xml index 49a93d2c70..00c6bc33d6 100644 --- a/lib/kernel/doc/src/kernel_app.xml +++ b/lib/kernel/doc/src/kernel_app.xml @@ -4,7 +4,7 @@ <appref> <header> <copyright> - <year>1996</year><year>2013</year> + <year>1996</year><year>2014</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -35,7 +35,7 @@ Erlang/OTP consists of Kernel and STDLIB. The Kernel application contains the following services:</p> <list type="bulleted"> - <item>application controller, see <c>application(3)</c></item> + <item>application controller, see <seealso marker="application">application(3)</seealso></item> <item><c>code</c></item> <item><c>disk_log</c></item> <item><c>dist_ac</c>, distributed application controller</item> @@ -66,8 +66,8 @@ <section> <title>Configuration</title> <p>The following configuration parameters are defined for the Kernel - application. See <c>app(3)</c> for more information about - configuration parameters.</p> + application. See <seealso marker="app">app(4)</seealso> for more + information about configuration parameters.</p> <taglist> <tag><c>browser_cmd = string() | {M,F,A}</c></tag> <item> @@ -93,7 +93,8 @@ <item><c>Time = integer()>0</c></item> <item><c>Nodes = [node() | {node(),...,node()}]</c></item> </list> - <p>The parameter is described in <c>application(3)</c>, function + <p>The parameter is described in + <seealso marker="application">application(3)</seealso>, function <c>load/2</c>.</p> </item> <tag><c>dist_auto_connect = Value</c></tag> @@ -105,11 +106,13 @@ <taglist> <tag><c>never</c></tag> <item>Connections are never automatically established, they - must be explicitly connected. See <c>net_kernel(3)</c>.</item> + must be explicitly connected. See + <seealso marker="net_kernel">net_kernel(3)</seealso>.</item> <tag><c>once</c></tag> <item>Connections will be established automatically, but only once per node. If a node goes down, it must thereafter be - explicitly connected. See <c>net_kernel(3)</c>.</item> + explicitly connected. See + <seealso marker="net_kernel">net_kernel(3)</seealso>.</item> </taglist> </item> <tag><c>permissions = [Perm]</c></tag> @@ -121,7 +124,8 @@ <item><c>ApplName = atom()</c></item> <item><c>Bool = boolean()</c></item> </list> - <p>Permissions are described in <c>application(3)</c>, function + <p>Permissions are described in + <seealso marker="application">application(3)</seealso>, function <c>permit/2</c>.</p> </item> <tag><c>error_logger = Value</c></tag> @@ -149,7 +153,8 @@ </item> <tag><c>global_groups = [GroupTuple]</c></tag> <item> - <p>Defines global groups, see <c>global_group(3)</c>.</p> + <p>Defines global groups, see + <seealso marker="global_group">global_group(3)</seealso>.</p> <list type="bulleted"> <item><c>GroupTuple = {GroupName, [Node]} | {GroupName, PublishType, [Node]}</c></item> <item><c>GroupName = atom()</c></item> @@ -160,18 +165,19 @@ <tag><c>inet_default_connect_options = [{Opt, Val}]</c></tag> <item> <p>Specifies default options for <c>connect</c> sockets, - see <c>inet(3)</c>.</p> + see <seealso marker="inet">inet(3)</seealso>.</p> </item> <tag><c>inet_default_listen_options = [{Opt, Val}]</c></tag> <item> <p>Specifies default options for <c>listen</c> (and - <c>accept</c>) sockets, see <c>inet(3)</c>.</p> + <c>accept</c>) sockets, see <seealso marker="inet">inet(3)</seealso>.</p> </item> <tag><c>{inet_dist_use_interface, ip_address()}</c></tag> <item> <p>If the host of an Erlang node has several network interfaces, this parameter specifies which one to listen on. See - <c>inet(3)</c> for the type definition of <c>ip_address()</c>.</p> + <seealso marker="inet">inet(3)</seealso> for the type definition + of <c>ip_address()</c>.</p> </item> <tag><c>{inet_dist_listen_min, First}</c></tag> <item> @@ -276,7 +282,8 @@ MaxT = TickTime + TickTime / 4</code> <tag><c>start_boot_server = true | false</c></tag> <item> <p>Starts the <c>boot_server</c> if the parameter is <c>true</c> - (see <c>erl_boot_server(3)</c>). This parameter should be + (see <seealso marker="erl_boot_server">erl_boot_server(3)</seealso>). + This parameter should be set to <c>true</c> in an embedded system which uses this service.</p> <p>The default value is <c>false</c>.</p> @@ -296,13 +303,15 @@ MaxT = TickTime + TickTime / 4</code> <tag><c>start_disk_log = true | false</c></tag> <item> <p>Starts the <c>disk_log_server</c> if the parameter is - <c>true</c> (see <c>disk_log(3)</c>). This parameter should be + <c>true</c> (see <seealso marker="disk_log">disk_log(3)</seealso>). + This parameter should be set to true in an embedded system which uses this service.</p> <p>The default value is <c>false</c>.</p> </item> <tag><c>start_pg2 = true | false</c></tag> <item> - <p>Starts the <c>pg2</c> server (see <c>pg2(3)</c>) if + <p>Starts the <c>pg2</c> server (see + <seealso marker="pg2">pg2(3)</seealso>) if the parameter is <c>true</c>. This parameter should be set to <c>true</c> in an embedded system which uses this service.</p> <p>The default value is <c>false</c>.</p> @@ -310,7 +319,8 @@ MaxT = TickTime + TickTime / 4</code> <tag><c>start_timer = true | false</c></tag> <item> <p>Starts the <c>timer_server</c> if the parameter is - <c>true</c> (see <c>timer(3)</c>). This parameter should be + <c>true</c> (see <seealso marker="stdlib:timer">timer(3)</seealso>). + This parameter should be set to <c>true</c> in an embedded system which uses this service.</p> <p>The default value is <c>false</c>.</p> @@ -351,6 +361,7 @@ MaxT = TickTime + TickTime / 4</code> <seealso marker="pg2">pg2(3)</seealso>, <seealso marker="rpc">rpc(3)</seealso>, <seealso marker="seq_trace">seq_trace(3)</seealso>, + <seealso marker="stdlib:timer">timer(3)</seealso>, <seealso marker="user">user(3)</seealso></p> </section> </appref> diff --git a/lib/kernel/doc/src/notes.xml b/lib/kernel/doc/src/notes.xml index b2e89ea850..7eaf2d4a44 100644 --- a/lib/kernel/doc/src/notes.xml +++ b/lib/kernel/doc/src/notes.xml @@ -30,6 +30,307 @@ </header> <p>This document describes the changes made to the Kernel application.</p> +<section><title>Kernel 3.0.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Accept inet:ip_address() in net_adm:names/1</p> + <p> + Own Id: OTP-12154</p> + </item> + </list> + </section> + +</section> + +<section><title>Kernel 3.0.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + OTP-11850 fixed filelib:wildcard/1 to work with broken + symlinks. This correction, however, introduced problems + since symlinks were no longer followed for functions like + filelib:ensure_dir/1, filelib:is_dir/1, + filelib:file_size/1, etc. This is now corrected.</p> + <p> + Own Id: OTP-12054 Aux Id: seq12660 </p> + </item> + </list> + </section> + +</section> + +<section><title>Kernel 3.0.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + If the Config given to + application_controller:change_application_data included + other config files, it was only expanded for already + existing (loaded) applications. If an upgrade added a new + application which had config data in an included config + file, the new application did not get correct config + data.</p> + <p> + This is now changed so config data will be expanded for + all applications.</p> + <p> + Own Id: OTP-11864</p> + </item> + <item> + <p>It was allowed to re-load pre-loaded modules such as + <c>erlang</c>, but that could cause strange and unwanted + things to happen, such as call <c>apply/3</c> to loop. + Pre-loaded modules are now sticky by default. (Thanks to + Loïc Hoguin for reporting this bug.)</p> + <p><c>code:add_path("/ending/in/slash/")</c> removes the + trailing slash, adding <c>/ending/in/slash</c> to the + code path. However, + <c>code:del_path("/ending/in/slash/")</c> would fail to + remove the path since it did not remove the trailing + slash. This has been fixed.</p> + <p> + Own Id: OTP-11913</p> + </item> + <item> + <p> + Fix erts_debug:size/1 to handle Map sizes</p> + <p> + Own Id: OTP-11923</p> + </item> + <item> + <p>The documentation for <c>file:file_info/1</c> has been + removed. The function itself was removed a long time + ago.</p> + <p> + Own Id: OTP-11982</p> + </item> + </list> + </section> + +</section> + +<section><title>Kernel 3.0</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fixed a deadlock possibility in terminate application</p> + <p> + Own Id: OTP-11171</p> + </item> + <item> + <p> + Fixed bug where sendfile would return the wrong error + code for a remotely closed socket if the socket was in + passive mode. (Thanks to Vincent Siliakus for reporting + the bug.)</p> + <p> + Own Id: OTP-11614</p> + </item> + <item> + <p> + The new option <c>persistent</c> is added to + <c>application:set_env/4</c> and + <c>application:unset_env/3</c>. An environment key set + with the <c>persistent</c> option will not be overridden + by the ones configured in the application resource file + on load. This means persistent values will stick after + the application is loaded and also on application reload. + (Thanks to José Valim)</p> + <p> + Own Id: OTP-11708</p> + </item> + <item> + <p> + The spec for file:set_cwd/1 is modified to also accept + binaries as arguments. This has always been allowed in + the code, but it was not reflected in the spec since + binaries are mostly used for raw file names. Raw file + names are names that are not encoded according to + file:native_name_encoding(), and these are not allowed in + file:set_cwd/1. The spec is now, however, more allowing + in order to avoid unnecessary dialyzer warnings. Raw file + names will still fail in runtime with reason + 'no_translation'. (Thanks to José Valim)</p> + <p> + Own Id: OTP-11787</p> + </item> + </list> + </section> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + heart:set_cmd/1 is updated to allow unicode code points > + 255 in the given heart command</p> + <p> + Own Id: OTP-10843</p> + </item> + <item> + <p> Dialyzer's <c>unmatched_return</c> warnings have been + corrected. </p> + <p> + Own Id: OTP-10908</p> + </item> + <item> + <p> + Make erlang:open_port/2 spawn and spawn_executable handle + unicode.</p> + <p> + Own Id: OTP-11105</p> + </item> + <item> + <p> + Erlang/OTP has been ported to the realtime operating + system OSE. The port supports both smp and non-smp + emulator. For details around the port and how to started + see the User's Guide in the <seealso + marker="ose:ose_intro">ose</seealso> application. </p> + <p> + Note that not all parts of Erlang/OTP has been ported. </p> + <p> + Notable things that work are: non-smp and smp emulators, + OSE signal interaction, crypto, asn1, run_erl/to_erl, + tcp, epmd, distribution and most if not all non-os + specific functionality of Erlang.</p> + <p> + Notable things that does not work are: udp/sctp, os_mon, + erl_interface, binding of schedulers.</p> + <p> + Own Id: OTP-11334</p> + </item> + <item> + <p> + Add the {active,N} socket option for TCP, UDP, and SCTP, + where N is an integer in the range -32768..32767, to + allow a caller to specify the number of data messages to + be delivered to the controlling process. Once the + socket's delivered message count either reaches 0 or is + explicitly set to 0 with inet:setopts/2 or by including + {active,0} as an option when the socket is created, the + socket transitions to passive ({active, false}) mode and + the socket's controlling process receives a message to + inform it of the transition. TCP sockets receive + {tcp_passive,Socket}, UDP sockets receive + {udp_passive,Socket} and SCTP sockets receive + {sctp_passive,Socket}. </p> + <p> + The socket's delivered message counter defaults to 0, but + it can be set using {active,N} via any gen_tcp, gen_udp, + or gen_sctp function that takes socket options as + arguments, or via inet:setopts/2. New N values are added + to the socket's current counter value, and negative + numbers can be used to reduce the counter value. + Specifying a number that would cause the socket's counter + value to go above 32767 causes an einval error. If a + negative number is specified such that the counter value + would become negative, the socket's counter value is set + to 0 and the socket transitions to passive mode. If the + counter value is already 0 and inet:setopts(Socket, + [{active,0}]) is specified, the counter value remains at + 0 but the appropriate passive mode transition message is + generated for the socket.</p> + <p> + Thanks to Steve Vinoski</p> + <p> + Own Id: OTP-11368</p> + </item> + <item> + <p> + A call to either the <c>garbage_collect/1</c> BIF or the + <c>check_process_code/2</c> BIF may trigger garbage + collection of another processes than the process calling + the BIF. The previous implementations performed these + kinds of garbage collections without considering the + internal state of the process being garbage collected. In + order to be able to more easily and more efficiently + implement yielding native code, these types of garbage + collections have been rewritten. A garbage collection + like this is now triggered by an asynchronous request + signal, the actual garbage collection is performed by the + process being garbage collected itself, and finalized by + a reply signal to the process issuing the request. Using + this approach processes can disable garbage collection + and yield without having to set up the heap in a state + that can be garbage collected.</p> + <p> + The <seealso + marker="erts:erlang#garbage_collect/2"><c>garbage_collect/2</c></seealso>, + and <seealso + marker="erts:erlang#check_process_code/3"><c>check_process_code/3</c></seealso> + BIFs have been introduced. Both taking an option list as + last argument. Using these, one can issue asynchronous + requests.</p> + <p> + <c>code:purge/1</c> and <c>code:soft_purge/1</c> have + been rewritten to utilize asynchronous + <c>check_process_code</c> requests in order to + parallelize work.</p> + <p> + Characteristics impact: A call to the + <c>garbage_collect/1</c> BIF or the + <c>check_process_code/2</c> BIF will normally take longer + time to complete while the system as a whole wont be as + much negatively effected by the operation as before. A + call to <c>code:purge/1</c> and <c>code:soft_purge/1</c> + may complete faster or slower depending on the state of + the system while the system as a whole wont be as much + negatively effected by the operation as before.</p> + <p> + Own Id: OTP-11388 Aux Id: OTP-11535, OTP-11648 </p> + </item> + <item> + <p> + Add sync option to file:open/2.</p> + <p> + The sync option adds the POSIX O_SYNC flag to the open + system call on platforms that support the flag or its + equivalent, e.g., FILE_FLAG_WRITE_THROUGH on Windows. For + platforms that don't support it, file:open/2 returns + {error, enotsup} if the sync option is passed in. Thank + to Steve Vinoski and Joseph Blomstedt</p> + <p> + Own Id: OTP-11498</p> + </item> + <item> + <p> The contract of <c>inet:ntoa/1</c> has been + corrected. </p> <p> Thanks to Max Treskin. </p> + <p> + Own Id: OTP-11730</p> + </item> + </list> + </section> +</section> + +<section><title>Kernel 2.16.4.1</title> + + <section><title>Known Bugs and Problems</title> + <list> + <item> + <p> + When using gen_tcp:connect and the <c>fd</c> option with + <c>port</c> and/or <c>ip</c>, the <c>port</c> and + <c>ip</c> options were ignored. This has been fixed so + that if <c>port</c> and/or <c>ip</c> is specified + together with <c>fd</c> a bind is requested for that + <c>fd</c>. If <c>port</c> and/or <c>ip</c> is not + specified bind will not be called.</p> + <p> + Own Id: OTP-12061</p> + </item> + </list> + </section> +</section> + <section><title>Kernel 2.16.4</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/kernel/src/Makefile b/lib/kernel/src/Makefile index cb3c0a49f4..c7c70ad257 100644 --- a/lib/kernel/src/Makefile +++ b/lib/kernel/src/Makefile @@ -122,6 +122,7 @@ HRL_FILES= ../include/file.hrl ../include/inet.hrl ../include/inet_sctp.hrl \ ../include/net_address.hrl INTERNAL_HRL_FILES= application_master.hrl disk_log.hrl \ + erl_epmd.hrl hipe_ext_format.hrl \ inet_dns.hrl inet_res.hrl \ inet_boot.hrl inet_config.hrl inet_int.hrl \ inet_dns_record_adts.hrl diff --git a/lib/kernel/src/application.erl b/lib/kernel/src/application.erl index 76a80553b0..c4bef5188a 100644 --- a/lib/kernel/src/application.erl +++ b/lib/kernel/src/application.erl @@ -30,6 +30,8 @@ -export([get_application/0, get_application/1, info/0]). -export([start_type/0]). +-export_type([start_type/0]). + %%%----------------------------------------------------------------- -type start_type() :: 'normal' @@ -58,8 +60,7 @@ %%------------------------------------------------------------------ --callback start(StartType :: normal | {takeover, node()} | {failover, node()}, - StartArgs :: term()) -> +-callback start(StartType :: start_type(), StartArgs :: term()) -> {'ok', pid()} | {'ok', pid(), State :: term()} | {'error', Reason :: term()}. -callback stop(State :: term()) -> diff --git a/lib/kernel/src/application_controller.erl b/lib/kernel/src/application_controller.erl index ed13035104..daad45b6c2 100644 --- a/lib/kernel/src/application_controller.erl +++ b/lib/kernel/src/application_controller.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2013. All Rights Reserved. +%% Copyright Ericsson AB 1996-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -829,12 +829,12 @@ handle_call({change_application_data, Applications, Config}, _From, S) -> {reply, Error, S}; {'EXIT', R} -> {reply, {error, R}, S}; - NewAppls -> + {NewAppls, NewConfig} -> lists:foreach(fun(Appl) -> ets:insert(ac_tab, {{loaded, Appl#appl.name}, Appl}) end, NewAppls), - {reply, ok, S#state{conf_data = Config}} + {reply, ok, S#state{conf_data = NewConfig}} end; handle_call(prep_config_change, _From, S) -> @@ -1550,18 +1550,19 @@ do_change_apps(Applications, Config, OldAppls) -> end, Errors), - map(fun(Appl) -> - AppName = Appl#appl.name, - case is_loaded_app(AppName, Applications) of - {true, Application} -> - do_change_appl(make_appl(Application), - Appl, SysConfig); - - %% ignored removed apps - handled elsewhere - false -> - Appl - end - end, OldAppls). + {map(fun(Appl) -> + AppName = Appl#appl.name, + case is_loaded_app(AppName, Applications) of + {true, Application} -> + do_change_appl(make_appl(Application), + Appl, SysConfig); + + %% ignored removed apps - handled elsewhere + false -> + Appl + end + end, OldAppls), + SysConfig}. is_loaded_app(AppName, [{application, AppName, App} | _]) -> {true, {application, AppName, App}}; diff --git a/lib/kernel/src/code_server.erl b/lib/kernel/src/code_server.erl index fc7ac08699..819554ce74 100644 --- a/lib/kernel/src/code_server.erl +++ b/lib/kernel/src/code_server.erl @@ -63,7 +63,10 @@ init(Ref, Parent, [Root,Mode0]) -> process_flag(trap_exit, true), Db = ets:new(code, [private]), - foreach(fun (M) -> ets:insert(Db, {M,preloaded}) end, erlang:pre_loaded()), + foreach(fun (M) -> + %% Pre-loaded modules are always sticky. + ets:insert(Db, [{M,preloaded},{{sticky,M},true}]) + end, erlang:pre_loaded()), ets:insert(Db, init:fetch_loaded()), Mode = @@ -988,7 +991,7 @@ try_archive_subdirs(_Archive, Base, []) -> %% the complete directory name. %% del_path(Name0,Path,NameDb) -> - case catch to_list(Name0)of + case catch filename:join([to_list(Name0)]) of {'EXIT',_} -> {{error,bad_name},Path}; Name -> diff --git a/lib/kernel/src/erl_boot_server.erl b/lib/kernel/src/erl_boot_server.erl index 9a49655a9f..ef09d86ca4 100644 --- a/lib/kernel/src/erl_boot_server.erl +++ b/lib/kernel/src/erl_boot_server.erl @@ -341,9 +341,13 @@ handle_command(S, PS, Msg) -> send_file_result(S, list_dir, Res), PS2; {read_file_info,File} -> - {Res, PS2} = erl_prim_loader:prim_read_file_info(PS, File), + {Res, PS2} = erl_prim_loader:prim_read_file_info(PS, File, true), send_file_result(S, read_file_info, Res), PS2; + {read_link_info,File} -> + {Res, PS2} = erl_prim_loader:prim_read_file_info(PS, File, false), + send_file_result(S, read_link_info, Res), + PS2; get_cwd -> {Res, PS2} = erl_prim_loader:prim_get_cwd(PS, []), send_file_result(S, get_cwd, Res), diff --git a/lib/kernel/src/erl_epmd.erl b/lib/kernel/src/erl_epmd.erl index b4fae24ef3..f6e2ca0954 100644 --- a/lib/kernel/src/erl_epmd.erl +++ b/lib/kernel/src/erl_epmd.erl @@ -85,24 +85,19 @@ port_please1(Node,HostName, Timeout) -> Else end. -names() -> +names() -> {ok, H} = inet:gethostname(), names(H). -names(HostName) when is_atom(HostName) -> - names1(atom_to_list(HostName)); -names(HostName) when is_list(HostName) -> - names1(HostName); -names(EpmdAddr) -> - get_names(EpmdAddr). - -names1(HostName) -> +names(HostName) when is_atom(HostName); is_list(HostName) -> case inet:gethostbyname(HostName) of {ok,{hostent, _Name, _ , _Af, _Size, [EpmdAddr | _]}} -> get_names(EpmdAddr); Else -> Else - end. + end; +names(EpmdAddr) -> + get_names(EpmdAddr). register_node(Name, PortNo) -> diff --git a/lib/kernel/src/erts_debug.erl b/lib/kernel/src/erts_debug.erl index f7a815882b..ef605d0bfe 100644 --- a/lib/kernel/src/erts_debug.erl +++ b/lib/kernel/src/erts_debug.erl @@ -182,6 +182,11 @@ size(Tuple, Seen0, Sum0) when is_tuple(Tuple) -> Sum = Sum0 + 1 + tuple_size(Tuple), tuple_size(1, tuple_size(Tuple), Tuple, Seen, Sum) end; +size(Map, Seen0, Sum) when is_map(Map) -> + case remember_term(Map, Seen0) of + seen -> {Sum,Seen0}; + Seen -> map_size(Map, Seen, Sum) + end; size(Fun, Seen0, Sum) when is_function(Fun) -> case remember_term(Fun, Seen0) of seen -> {Sum,Seen0}; @@ -203,6 +208,12 @@ tuple_size(I, Sz, Tuple, Seen0, Sum0) -> {Sum,Seen} = size(element(I, Tuple), Seen0, Sum0), tuple_size(I+1, Sz, Tuple, Seen, Sum). +map_size(Map,Seen0,Sum0) -> + Kt = erts_internal:map_to_tuple_keys(Map), + Vs = maps:values(Map), + {Sum1,Seen1} = size(Kt,Seen0,Sum0), + fold_size(Vs,Seen1,Sum1+length(Vs)+3). + fun_size(Fun, Seen, Sum) -> case erlang:fun_info(Fun, type) of {type,external} -> @@ -210,14 +221,14 @@ fun_size(Fun, Seen, Sum) -> {type,local} -> Sz = erts_debug:flat_size(fun() -> ok end), {env,Env} = erlang:fun_info(Fun, env), - fun_size_1(Env, Seen, Sum+Sz+length(Env)) + fold_size(Env, Seen, Sum+Sz+length(Env)) end. -fun_size_1([H|T], Seen0, Sum0) -> +fold_size([H|T], Seen0, Sum0) -> {Sum,Seen} = size(H, Seen0, Sum0), - fun_size_1(T, Seen, Sum); -fun_size_1([], Seen, Sum) -> {Sum,Seen}. - + fold_size(T, Seen, Sum); +fold_size([], Seen, Sum) -> {Sum,Seen}. + remember_term(Term, Seen) -> case gb_trees:lookup(Term, Seen) of none -> gb_trees:insert(Term, [Term], Seen); diff --git a/lib/kernel/src/file.erl b/lib/kernel/src/file.erl index 20b703e084..ee2fb85de2 100644 --- a/lib/kernel/src/file.erl +++ b/lib/kernel/src/file.erl @@ -117,15 +117,7 @@ | {'time', 'posix'}. %%% BIFs --export([file_info/1, native_name_encoding/0]). - --spec file_info(Filename) -> {ok, FileInfo} | {error, Reason} when - Filename :: name_all(), - FileInfo :: file_info(), - Reason :: posix() | badarg. - -file_info(_) -> - erlang:nif_error(undef). +-export([native_name_encoding/0]). -spec native_name_encoding() -> latin1 | utf8. diff --git a/lib/kernel/src/inet.erl b/lib/kernel/src/inet.erl index 41d422d7d4..d17da2d329 100644 --- a/lib/kernel/src/inet.erl +++ b/lib/kernel/src/inet.erl @@ -1257,9 +1257,9 @@ open(FdO, Addr, Port, Opts, Protocol, Family, Type, Module) Error -> Error end; -open(Fd, _Addr, _Port, Opts, Protocol, Family, Type, Module) +open(Fd, Addr, Port, Opts, Protocol, Family, Type, Module) when is_integer(Fd) -> - fdopen(Fd, Opts, Protocol, Family, Type, Module). + fdopen(Fd, Addr, Port, Opts, Protocol, Family, Type, Module). bindx(S, [Addr], Port0) -> {IP, Port} = set_bindx_port(Addr, Port0), @@ -1298,12 +1298,35 @@ change_bindx_0_port({_IP, _Port}=Addr, _AssignedPort) -> {'ok', socket()} | {'error', posix()}. fdopen(Fd, Opts, Protocol, Family, Type, Module) -> - case prim_inet:fdopen(Protocol, Family, Type, Fd) of + fdopen(Fd, any, 0, Opts, Protocol, Family, Type, Module). + +fdopen(Fd, Addr, Port, Opts, Protocol, Family, Type, Module) -> + IsAnyAddr = (Addr == {0,0,0,0} orelse Addr == {0,0,0,0,0,0,0,0} + orelse Addr == any), + Bound = Port == 0 andalso IsAnyAddr, + case prim_inet:fdopen(Protocol, Family, Type, Fd, Bound) of {ok, S} -> case prim_inet:setopts(S, Opts) of ok -> - inet_db:register_socket(S, Module), - {ok, S}; + case if + Bound -> + %% We do not do any binding if default + %% port+addr options where given in order + %% to keep backwards compatability with + %% pre Erlang/TOP 17 + {ok, ok}; + is_list(Addr) -> + bindx(S, Addr, Port); + true -> + prim_inet:bind(S, Addr, Port) + end of + {ok, _} -> + inet_db:register_socket(S, Module), + {ok, S}; + Error -> + prim_inet:close(S), + Error + end; Error -> prim_inet:close(S), Error end; diff --git a/lib/kernel/src/inet_config.erl b/lib/kernel/src/inet_config.erl index 2461f3ff25..fdc244f959 100644 --- a/lib/kernel/src/inet_config.erl +++ b/lib/kernel/src/inet_config.erl @@ -197,6 +197,9 @@ do_load_resolv({win32,Type}, longnames) -> win32_load_from_registry(Type), inet_db:set_lookup([native]); +do_load_resolv({ose,_}, _) -> + inet_db:set_lookup([file]); + do_load_resolv(_, _) -> inet_db:set_lookup([native]). diff --git a/lib/kernel/src/kernel.app.src b/lib/kernel/src/kernel.app.src index 5658c6b6cf..9f6c0f4624 100644 --- a/lib/kernel/src/kernel.app.src +++ b/lib/kernel/src/kernel.app.src @@ -115,6 +115,6 @@ {applications, []}, {env, [{error_logger, tty}]}, {mod, {kernel, []}}, - {runtime_dependencies, ["erts-6.0", "stdlib-2.0", "sasl-2.4"]} + {runtime_dependencies, ["erts-6.1.2", "stdlib-2.0", "sasl-2.4"]} ] }. diff --git a/lib/kernel/src/net_adm.erl b/lib/kernel/src/net_adm.erl index 3f5eac7822..2cdfb76417 100644 --- a/lib/kernel/src/net_adm.erl +++ b/lib/kernel/src/net_adm.erl @@ -89,18 +89,13 @@ names() -> -spec names(Host) -> {ok, [{Name, Port}]} | {error, Reason} when - Host :: atom() | string(), + Host :: atom() | string() | inet:ip_address(), Name :: string(), Port :: non_neg_integer(), Reason :: address | file:posix(). names(Hostname) -> - case inet:gethostbyname(Hostname) of - {ok, {hostent, _Name, _ , _Af, _Size, [Addr | _]}} -> - erl_epmd:names(Addr); - Else -> - Else - end. + erl_epmd:names(Hostname). -spec dns_hostname(Host) -> {ok, Name} | {error, Host} when Host :: atom() | string(), diff --git a/lib/kernel/src/os.erl b/lib/kernel/src/os.erl index 3bda391b8e..187fd0001b 100644 --- a/lib/kernel/src/os.erl +++ b/lib/kernel/src/os.erl @@ -67,7 +67,7 @@ unsetenv(_) -> %%% End of BIFs -spec type() -> {Osfamily, Osname} when - Osfamily :: unix | win32, + Osfamily :: unix | win32 | ose, Osname :: atom(). type() -> diff --git a/lib/kernel/test/application_SUITE.erl b/lib/kernel/test/application_SUITE.erl index c6cbd1a0ef..4901206c8e 100644 --- a/lib/kernel/test/application_SUITE.erl +++ b/lib/kernel/test/application_SUITE.erl @@ -1076,10 +1076,13 @@ otp_1586(Conf) when is_list(Conf) -> {ok, Fd} = file:open(filename:join(Dir, "app5.app"), [write]), w_app5(Fd), file:close(Fd), - code:add_patha(Dir), - ok = application:load(app4()), - ok = application:unload(app4), - ok. + try + true = code:add_patha(Dir), + ok = application:load(app4()), + ok = application:unload(app4) + after + _ = code:del_path(Dir) + end. %%----------------------------------------------------------------- %% Ticket: OTP-2078 @@ -1949,14 +1952,22 @@ config_change(Conf) when is_list(Conf) -> %% Find out application data from boot script Boot = filename:join([code:root_dir(), "bin", "start.boot"]), {ok, Bin} = file:read_file(Boot), - Appls = get_appls(binary_to_term(Bin)), + Appls0 = get_appls(binary_to_term(Bin)), + + %% And add app1 in order to test OTP-11864 - included config files + %% not read for new (not already loaded) applications + Appls = [app1() | Appls0], %% Simulate contents of "sys.config" Config = [{stdlib, [{par1,sys},{par2,sys}]}, "t1", "t2.config", filename:join([DataDir, "subdir", "t3"]), - {stdlib, [{par6,sys}]}], + {stdlib, [{par6,sys}]}, + "t4.config"], + + %% Check that app1 is not loaded + false = lists:keymember(app1,1,application:loaded_applications()), %% Order application_controller to update configuration ok = application_controller:change_application_data(Appls, @@ -1971,6 +1982,13 @@ config_change(Conf) when is_list(Conf) -> {value, {par5,t3}} = lists:keysearch(par5, 1, Env), {value, {par6,sys}} = lists:keysearch(par6, 1, Env), + %% Check that app1 parameters are correctly set after loading + [] = application:get_all_env(app1), + application:load(app1()), + App1Env = application:get_all_env(app1), + {value, {par1,t4}} = lists:keysearch(par1, 1, App1Env), + application:unload(app1), + ok = file:set_cwd(CWD). %% This function is stolen from SASL module release_handler, OTP R10B @@ -1989,6 +2007,7 @@ get_appls([_ | T], Res) -> get_appls([], Res) -> Res. + persistent_env(suite) -> []; persistent_env(doc) -> diff --git a/lib/kernel/test/application_SUITE_data/t4.config b/lib/kernel/test/application_SUITE_data/t4.config new file mode 100644 index 0000000000..8b2bc52c01 --- /dev/null +++ b/lib/kernel/test/application_SUITE_data/t4.config @@ -0,0 +1 @@ +[{app1, [{par1,t4}]}].
\ No newline at end of file diff --git a/lib/kernel/test/code_SUITE.erl b/lib/kernel/test/code_SUITE.erl index 42b81d16b3..afedc17e57 100644 --- a/lib/kernel/test/code_SUITE.erl +++ b/lib/kernel/test/code_SUITE.erl @@ -37,8 +37,7 @@ native_early_modules/1, get_mode/1]). -export([init_per_testcase/2, end_per_testcase/2, - init_per_suite/1, end_per_suite/1, - sticky_compiler/1]). + init_per_suite/1, end_per_suite/1]). %% error_logger -export([init/1, @@ -55,7 +54,7 @@ all() -> delete, purge, purge_many_exits, soft_purge, is_loaded, all_loaded, load_binary, dir_req, object_code, set_path_file, upgrade, - pa_pz_option, add_del_path, dir_disappeared, + sticky_dir, pa_pz_option, add_del_path, dir_disappeared, ext_mod_dep, clash, load_cached, start_node_with_cache, add_and_rehash, where_is_file_no_cache, where_is_file_cached, purge_stacktrace, mult_lib_roots, @@ -219,6 +218,13 @@ del_path(suite) -> []; del_path(doc) -> []; del_path(Config) when is_list(Config) -> P = code:get_path(), + try + del_path_1(P) + after + code:set_path(P) + end. + +del_path_1(P) -> test_server:format("Initial code:get_path()=~p~n",[P]), {'EXIT',_} = (catch code:del_path(3)), false = code:del_path(my_dummy_name), @@ -226,19 +232,22 @@ del_path(Config) when is_list(Config) -> Dir = filename:join([code:lib_dir(kernel),"ebin"]), test_server:format("kernel dir: ~p~n",[Dir]), - true = code:del_path(kernel), NewP = code:get_path(), test_server:format("Path after removing 'kernel':~p~n",[NewP]), ReferenceP = lists:delete(Dir,P), test_server:format("Reference path:~p~n",[ReferenceP]), NewP = ReferenceP, % check that dir is deleted + code:set_path(P), + %% An superfluous "/" should also work. + true = code:del_path("kernel/"), + NewP = ReferenceP, % check that dir is deleted code:set_path(P), + true = code:del_path(Dir), NewP1 = code:get_path(), NewP1 = lists:delete(Dir,P), % check that dir is deleted - code:set_path(P), ok. replace_path(suite) -> []; @@ -577,35 +586,42 @@ sticky_dir(suite) -> []; sticky_dir(doc) -> ["Test that a module with the same name as a module in ", "a sticky directory cannot be loaded."]; sticky_dir(Config) when is_list(Config) -> - MyDir=filename:dirname(code:which(?MODULE)), - {ok, Node}=?t:start_node(sticky_dir, slave,[{args, "-pa \""++MyDir++"\""}]), - File=filename:join([?config(data_dir, Config), "calendar"]), - Ret=rpc:call(Node, ?MODULE, sticky_compiler, [File]), + Pa = filename:dirname(code:which(?MODULE)), + {ok,Node} = ?t:start_node(sticky_dir, slave, [{args,"-pa "++Pa}]), + Mods = [code,lists,erlang,init], + OutDir = filename:join(?config(priv_dir, Config), sticky_dir), + _ = file:make_dir(OutDir), + Ret = rpc:call(Node, erlang, apply, + [fun sticky_compiler/2,[Mods,OutDir]]), case Ret of - fail -> - ?t:fail("c:c allowed a sticky module to be compiled and loaded."); - ok -> + [] -> ok; Other -> - test_server:format("Other: ~p",[Other]) + io:format("~p\n", [Other]), + ?t:fail() end, - ?t:stop_node(Node). + ?t:stop_node(Node), + ok. -sticky_compiler(File) -> - Compiled=File++code:objfile_extension(), - Dir=filename:dirname(File), - code:add_patha(Dir), - file:delete(Compiled), - case c:c(File, [{outdir, Dir}]) of - {ok, Module} -> - case catch Module:test(apa) of - {error, _} -> - fail; - {'EXIT', _} -> - ok - end; - Other -> - test_server:format("c:c(~p) returned: ~p",[File, Other]), +sticky_compiler(Files, PrivDir) -> + code:add_patha(PrivDir), + Rets = [do_sticky_compile(F, PrivDir) || F <- Files], + [R || R <- Rets, R =/= ok]. + +do_sticky_compile(Mod, Dir) -> + %% Make sure that the module is loaded. A module being sticky + %% only prevents it from begin reloaded, not from being loaded + %% from the wrong place to begin with. + Mod = Mod:module_info(module), + File = filename:append(Dir, atom_to_list(Mod)), + Src = io_lib:format("-module(~s).\n" + "-export([test/1]).\n" + "test(me) -> fail.\n", [Mod]), + ok = file:write_file(File++".erl", Src), + case c:c(File, [{outdir,Dir}]) of + {ok,Module} -> + Module:test(me); + {error,sticky_directory} -> ok end. diff --git a/lib/kernel/test/code_SUITE_data/calendar.erl b/lib/kernel/test/code_SUITE_data/calendar.erl deleted file mode 100644 index c1a4a1c12a..0000000000 --- a/lib/kernel/test/code_SUITE_data/calendar.erl +++ /dev/null @@ -1,23 +0,0 @@ -%% -%% %CopyrightBegin% -%% -%% Copyright Ericsson AB 1997-2009. All Rights Reserved. -%% -%% The contents of this file are subject to the Erlang Public License, -%% Version 1.1, (the "License"); you may not use this file except in -%% compliance with the License. You should have received a copy of the -%% Erlang Public License along with this software. If not, it can be -%% retrieved online at http://www.erlang.org/. -%% -%% Software distributed under the License is distributed on an "AS IS" -%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See -%% the License for the specific language governing rights and limitations -%% under the License. -%% -%% %CopyrightEnd% -%% --module(calendar). --export([test/1]). - -test(apa) -> - {error, this_function_should_not_be_called}. diff --git a/lib/kernel/test/erl_prim_loader_SUITE.erl b/lib/kernel/test/erl_prim_loader_SUITE.erl index b2ca3bdbc2..658c31c14d 100644 --- a/lib/kernel/test/erl_prim_loader_SUITE.erl +++ b/lib/kernel/test/erl_prim_loader_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2013. All Rights Reserved. +%% Copyright Ericsson AB 1996-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -328,6 +328,30 @@ file_requests(Config) when is_list(Config) -> {ok,Info} = file:read_file_info(code:which(test_server)), ?line {ok,Info} = rpc:call(Node, erl_prim_loader, read_file_info, [code:which(test_server)]), + + PrivDir = ?config(priv_dir,Config), + Dir = filename:join(PrivDir,?MODULE_STRING++"_file_requests"), + ok = file:make_dir(Dir), + Alias = filename:join(Dir,"symlink"), + case file:make_symlink(code:which(test_server), Alias) of + {error, enotsup} -> + %% Links not supported on this platform + ok; + {error, eperm} -> + {win32,_} = os:type(), + %% Windows user not privileged to create symlinks" + ok; + ok -> + %% Reading file info for link should return file info for + %% link target + {ok,Info} = rpc:call(Node, erl_prim_loader, read_file_info, + [Alias]), + #file_info{type=regular} = Info, + {ok,#file_info{type=symlink}} = + rpc:call(Node, erl_prim_loader, read_link_info, + [Alias]) + end, + {ok,Cwd} = file:get_cwd(), ?line {ok,Cwd} = rpc:call(Node, erl_prim_loader, get_cwd, []), case file:get_cwd("C:") of diff --git a/lib/kernel/test/gen_tcp_api_SUITE.erl b/lib/kernel/test/gen_tcp_api_SUITE.erl index a7af00c12a..c27d265550 100644 --- a/lib/kernel/test/gen_tcp_api_SUITE.erl +++ b/lib/kernel/test/gen_tcp_api_SUITE.erl @@ -32,14 +32,16 @@ t_connect_bad/1, t_recv_timeout/1, t_recv_eof/1, t_shutdown_write/1, t_shutdown_both/1, t_shutdown_error/1, - t_fdopen/1, t_implicit_inet6/1]). + t_fdopen/1, t_fdconnect/1, t_implicit_inet6/1]). + +-export([getsockfd/0,closesockfd/1]). suite() -> [{ct_hooks,[ts_install_cth]}]. all() -> [{group, t_accept}, {group, t_connect}, {group, t_recv}, t_shutdown_write, t_shutdown_both, t_shutdown_error, - t_fdopen, t_implicit_inet6]. + t_fdopen, t_fdconnect, t_implicit_inet6]. groups() -> [{t_accept, [], [t_accept_timeout]}, @@ -185,6 +187,37 @@ t_fdopen(Config) when is_list(Config) -> ?line ok = gen_tcp:close(L), ok. +t_fdconnect(Config) when is_list(Config) -> + Question = "Aaaa... Long time ago in a small town in Germany,", + Question1 = list_to_binary(Question), + Question2 = [<<"Aaaa">>, "... ", $L, <<>>, $o, "ng time ago ", + ["in ", [], <<"a small town">>, [" in Germany,", <<>>]]], + Question1 = iolist_to_binary(Question2), + Answer = "there was a shoemaker, Schumacher was his name.", + Path = ?config(data_dir, Config), + Lib = "gen_tcp_api_SUITE", + ok = erlang:load_nif(filename:join(Path,Lib), []), + {ok, L} = gen_tcp:listen(0, [{active, false}]), + {ok, Port} = inet:port(L), + FD = gen_tcp_api_SUITE:getsockfd(), + {ok, Client} = gen_tcp:connect(localhost, Port, [{fd,FD},{port,20002}, + {active,false}]), + {ok, Server} = gen_tcp:accept(L), + ok = gen_tcp:send(Client, Question), + {ok, Question} = gen_tcp:recv(Server, length(Question), 2000), + ok = gen_tcp:send(Client, Question1), + {ok, Question} = gen_tcp:recv(Server, length(Question), 2000), + ok = gen_tcp:send(Client, Question2), + {ok, Question} = gen_tcp:recv(Server, length(Question), 2000), + ok = gen_tcp:send(Server, Answer), + {ok, Answer} = gen_tcp:recv(Client, length(Answer), 2000), + ok = gen_tcp:close(Client), + FD = gen_tcp_api_SUITE:closesockfd(FD), + {error,closed} = gen_tcp:recv(Server, 1, 2000), + ok = gen_tcp:close(Server), + ok = gen_tcp:close(L), + ok. + %%% implicit inet6 option to api functions @@ -300,3 +333,7 @@ unused_ip(A, B, C, D) -> end. ok({ok,V}) -> V. + + +getsockfd() -> undefined. +closesockfd(_FD) -> undefined. diff --git a/lib/kernel/test/gen_tcp_api_SUITE_data/Makefile.src b/lib/kernel/test/gen_tcp_api_SUITE_data/Makefile.src new file mode 100644 index 0000000000..5477598160 --- /dev/null +++ b/lib/kernel/test/gen_tcp_api_SUITE_data/Makefile.src @@ -0,0 +1,9 @@ + +NIF_LIBS = gen_tcp_api_SUITE@dll@ +SHLIB_EXTRA_LDLIBS = @LIBS@ + +all: $(NIF_LIBS) + +@SHLIB_RULES@ + +$(NIF_LIBS): gen_tcp_api_SUITE.c diff --git a/lib/kernel/test/gen_tcp_api_SUITE_data/gen_tcp_api_SUITE.c b/lib/kernel/test/gen_tcp_api_SUITE_data/gen_tcp_api_SUITE.c new file mode 100644 index 0000000000..73a6568b30 --- /dev/null +++ b/lib/kernel/test/gen_tcp_api_SUITE_data/gen_tcp_api_SUITE.c @@ -0,0 +1,60 @@ +/* + * %CopyrightBegin% + * + * Copyright Ericsson AB 2009-2013. All Rights Reserved. + * + * The contents of this file are subject to the Erlang Public License, + * Version 1.1, (the "License"); you may not use this file except in + * compliance with the License. You should have received a copy of the + * Erlang Public License along with this software. If not, it can be + * retrieved online at http://www.erlang.org/. + * + * Software distributed under the License is distributed on an "AS IS" + * basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + * the License for the specific language governing rights and limitations + * under the License. + * + * %CopyrightEnd% + */ +#include "erl_nif.h" + +#include <stdio.h> +#include <string.h> +#include <assert.h> +#include <limits.h> +#include <sys/types.h> + +#ifdef __WIN32__ +#include <winsock2.h> +#else +#include <sys/socket.h> +#endif + +#define sock_open(af, type, proto) socket((af), (type), (proto)) + +static ERL_NIF_TERM getsockfd(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) +{ + int fd; + + fd = sock_open(AF_INET, SOCK_STREAM, 0); + return enif_make_int(env, fd); +} + +static ERL_NIF_TERM closesockfd(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) +{ + int fd; + + enif_get_int(env, argv[0], &fd); + + close(fd); + + return enif_make_int(env, fd); +} + +static ErlNifFunc nif_funcs[] = +{ + {"getsockfd", 0, getsockfd}, + {"closesockfd", 1, closesockfd} +}; + +ERL_NIF_INIT(gen_tcp_api_SUITE,nif_funcs,NULL,NULL,NULL,NULL) diff --git a/lib/kernel/test/gen_tcp_misc_SUITE.erl b/lib/kernel/test/gen_tcp_misc_SUITE.erl index ee8bfcceb1..4e4aeb67e2 100644 --- a/lib/kernel/test/gen_tcp_misc_SUITE.erl +++ b/lib/kernel/test/gen_tcp_misc_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1998-2013. All Rights Reserved. +%% Copyright Ericsson AB 1998-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -50,6 +50,14 @@ oct_acceptor/1, otp_7731_server/1, zombie_server/2, do_iter_max_socks/2]). +init_per_testcase(wrapping_oct, Config) when is_list(Config) -> + Dog = case os:type() of + {ose,_} -> + test_server:timetrap(test_server:minutes(20)); + _Else -> + test_server:timetrap(test_server:seconds(600)) + end, + [{watchdog, Dog}|Config]; init_per_testcase(iter_max_socks, Config) when is_list(Config) -> Dog = case os:type() of {win32,_} -> @@ -58,6 +66,17 @@ init_per_testcase(iter_max_socks, Config) when is_list(Config) -> test_server:timetrap(test_server:seconds(240)) end, [{watchdog, Dog}|Config]; +init_per_testcase(accept_system_limit, Config) when is_list(Config) -> + case os:type() of + {ose,_} -> + {skip,"Skip in OSE"}; + _ -> + Dog = test_server:timetrap(test_server:seconds(240)), + [{watchdog,Dog}|Config] + end; +init_per_testcase(wrapping_oct, Config) when is_list(Config) -> + Dog = test_server:timetrap(test_server:seconds(600)), + [{watchdog, Dog}|Config]; init_per_testcase(_Func, Config) when is_list(Config) -> Dog = test_server:timetrap(test_server:seconds(240)), [{watchdog, Dog}|Config]. @@ -537,7 +556,6 @@ otp_3924(Config) when is_list(Config) -> otp_3924_1(MaxDelay). otp_3924_1(MaxDelay) -> - Dog = test_server:timetrap(test_server:seconds(240)), ?line {ok, Node} = start_node(otp_3924), ?line DataLen = 100*1024, ?line Data = otp_3924_data(DataLen), @@ -548,7 +566,6 @@ otp_3924_1(MaxDelay) -> ?line ok = otp_3924(MaxDelay, Node, Data, DataLen, N) end), ?line test_server:stop_node(Node), - test_server:timetrap_cancel(Dog), ok. otp_3924(MaxDelay, Node, Data, DataLen, N) -> @@ -865,7 +882,7 @@ passive_sockets_server_send(Socket, X) -> accept_closed_by_other_process(doc) -> ["Tests the return value from gen_tcp:accept when ", - "the socket is closed from an other process. (OTP-3817)"]; + "the socket is closed from another process. (OTP-3817)"]; accept_closed_by_other_process(Config) when is_list(Config) -> ?line Parent = self(), ?line {ok, ListenSocket} = gen_tcp:listen(0, []), @@ -1953,7 +1970,9 @@ accept_system_limit(doc) -> accept_system_limit(Config) when is_list(Config) -> ?line {ok, LS} = gen_tcp:listen(0, []), ?line {ok, TcpPort} = inet:port(LS), - ?line Connector = spawn_link(fun () -> connector(TcpPort) end), + Me = self(), + ?line Connector = spawn_link(fun () -> connector(TcpPort, Me) end), + receive {Connector, sync} -> Connector ! {self(), continue} end, ?line ok = acceptor(LS, false, []), ?line Connector ! stop, ok. @@ -1970,8 +1989,10 @@ acceptor(LS, GotSL, A) -> error end. -connector(TcpPort) -> +connector(TcpPort, Tester) -> ManyPorts = open_ports([]), + Tester ! {self(), sync}, + receive {Tester, continue} -> timer:sleep(100) end, ConnF = fun (Port) -> case catch gen_tcp:connect({127,0,0,1}, TcpPort, []) of {ok, Sock} -> @@ -2705,13 +2726,11 @@ wrapping_oct(doc) -> wrapping_oct(suite) -> []; wrapping_oct(Config) when is_list(Config) -> - Dog = test_server:timetrap(test_server:seconds(600)), {ok,Sock} = gen_tcp:listen(0,[{active,false},{mode,binary}]), {ok,Port} = inet:port(Sock), spawn_link(?MODULE,oct_acceptor,[Sock]), Res = oct_datapump(Port,16#1FFFFFFFF), gen_tcp:close(Sock), - test_server:timetrap_cancel(Dog), ok = Res, ok. diff --git a/lib/kernel/test/gen_udp_SUITE.erl b/lib/kernel/test/gen_udp_SUITE.erl index 6bb41999c5..8177123332 100644 --- a/lib/kernel/test/gen_udp_SUITE.erl +++ b/lib/kernel/test/gen_udp_SUITE.erl @@ -447,8 +447,8 @@ open_fd(Config) when is_list(Config) -> {ok,S1} = gen_udp:open(0), {ok,P2} = inet:port(S1), {ok,FD} = prim_inet:getfd(S1), - {error,einval} = gen_udp:open(P2, [inet6, {fd,FD}]), - {ok,S2} = gen_udp:open(P2, [{fd,FD}]), + {error,einval} = gen_udp:open(0, [inet6, {fd,FD}]), + {ok,S2} = gen_udp:open(0, [{fd,FD}]), {ok,S3} = gen_udp:open(0), {ok,P3} = inet:port(S3), ok = gen_udp:send(S3, Addr, P2, Msg), diff --git a/lib/kernel/test/heart_SUITE.erl b/lib/kernel/test/heart_SUITE.erl index 320b23bea1..35d3b75b34 100644 --- a/lib/kernel/test/heart_SUITE.erl +++ b/lib/kernel/test/heart_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2012. All Rights Reserved. +%% Copyright Ericsson AB 1996-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -93,10 +93,10 @@ start_check(Type, Name) -> start_check(Type, Name, []). start_check(Type, Name, Envs) -> Args = case ?t:os_type() of - {win32,_} -> - "-heart " ++ env_encode([{"HEART_COMMAND", no_reboot}|Envs]); + {win32,_} -> + "+t50000 -heart " ++ env_encode([{"HEART_COMMAND", no_reboot}|Envs]); _ -> - "-heart " ++ env_encode(Envs) + "+t50000 -heart " ++ env_encode(Envs) end, {ok, Node} = case Type of loose -> diff --git a/lib/kernel/test/kernel_SUITE.erl b/lib/kernel/test/kernel_SUITE.erl index 78f5e93fc3..1884e8cf58 100644 --- a/lib/kernel/test/kernel_SUITE.erl +++ b/lib/kernel/test/kernel_SUITE.erl @@ -95,10 +95,10 @@ appup_tests(App,{OkVsns,NokVsns}) -> ok. create_test_vsns(App) -> - This = erlang:system_info(otp_release), - FirstMajor = previous_major(This), + ThisMajor = erlang:system_info(otp_release), + FirstMajor = previous_major(ThisMajor), SecondMajor = previous_major(FirstMajor), - Ok = app_vsn(App,[FirstMajor]), + Ok = app_vsn(App,[ThisMajor,FirstMajor]), Nok0 = app_vsn(App,[SecondMajor]), Nok = case Ok of [Ok1|_] -> @@ -109,9 +109,9 @@ create_test_vsns(App) -> {Ok,Nok}. previous_major("17") -> - "r16"; -previous_major("r"++Rel) -> - "r"++previous_major(Rel); + "r16b"; +previous_major("r16b") -> + "r15b"; previous_major(Rel) -> integer_to_list(list_to_integer(Rel)-1). diff --git a/lib/kernel/test/sendfile_SUITE.erl b/lib/kernel/test/sendfile_SUITE.erl index 2c741232c4..123e849ccb 100644 --- a/lib/kernel/test/sendfile_SUITE.erl +++ b/lib/kernel/test/sendfile_SUITE.erl @@ -72,7 +72,12 @@ end_per_suite(Config) -> file:delete(proplists:get_value(big_file, Config)). init_per_group(async_threads,Config) -> - [{sendfile_opts,[{use_threads,true}]}|Config]; + case erlang:system_info(thread_pool_size) of + 0 -> + {skip,"No async threads"}; + _ -> + [{sendfile_opts,[{use_threads,true}]}|Config] + end; init_per_group(no_async_threads,Config) -> [{sendfile_opts,[{use_threads,false}]}|Config]. diff --git a/lib/kernel/vsn.mk b/lib/kernel/vsn.mk index dd5316b825..be633a304a 100644 --- a/lib/kernel/vsn.mk +++ b/lib/kernel/vsn.mk @@ -1 +1 @@ -KERNEL_VSN = 3.0 +KERNEL_VSN = 3.0.3 diff --git a/lib/megaco/aclocal.m4 b/lib/megaco/aclocal.m4 index c51c26794a..ed492d55ff 100644 --- a/lib/megaco/aclocal.m4 +++ b/lib/megaco/aclocal.m4 @@ -83,8 +83,10 @@ AC_ARG_VAR(erl_xcomp_ose_LM_POST_LINK, [OSE postlink tool (only used when cross AC_ARG_VAR(erl_xcomp_ose_LM_SET_CONF, [Sets the configuration for an OSE load module (only used when cross compiling for OSE)]) AC_ARG_VAR(erl_xcomp_ose_LM_ELF_SIZE, [Prints the section size information for an OSE load module (only used when cross compiling for OSE)]) AC_ARG_VAR(erl_xcomp_ose_LM_LCF, [OSE load module linker configuration file (only used when cross compiling for OSE)]) -AC_ARG_VAR(erl_xcomp_ose_LM_CONF, [OSE load module default configuration file (only used when cross compiling for OSE)]) -AC_ARG_VAR(erl_xcomp_ose_CONFD, [OSE OSE confd source file]) +AC_ARG_VAR(erl_xcomp_ose_BEAM_LM_CONF, [BEAM OSE load module default configuration file (only used when cross compiling for OSE)]) +AC_ARG_VAR(erl_xcomp_ose_EPMD_LM_CONF, [EPMD OSE load module default configuration file (only used when cross compiling for OSE)]) +AC_ARG_VAR(erl_xcomp_ose_RUN_ERL_LM_CONF, [run_erl_lm OSE load module default configuration file (only used when cross compiling for OSE)]) +AC_ARG_VAR(erl_xcomp_ose_CONFD, [OSE confd source file]) AC_ARG_VAR(erl_xcomp_ose_CRT0_LM, [OSE crt0 lm source file]) ]) @@ -1116,7 +1118,7 @@ case "$THR_LIB_NAME" in [Define if you have the "ose_spi/ose_spi.h" header file.])) ;; esac - if test "x$THR_LIB_NAME" == "xpthread"; then + if test "x$THR_LIB_NAME" = "xpthread"; then case $host_os in openbsd*) # The default stack size is insufficient for our needs @@ -1220,7 +1222,7 @@ case "$THR_LIB_NAME" in dnl dnl Check for functions dnl - if test "x$THR_LIB_NAME" == "xpthread"; then + if test "x$THR_LIB_NAME" = "xpthread"; then AC_CHECK_FUNC(pthread_spin_lock, \ [ethr_have_native_spinlock=yes \ AC_DEFINE(ETHR_HAVE_PTHREAD_SPIN_LOCK, 1, \ diff --git a/lib/megaco/configure.in b/lib/megaco/configure.in index 64daa959b5..e3c24a58b8 100644 --- a/lib/megaco/configure.in +++ b/lib/megaco/configure.in @@ -167,6 +167,26 @@ if test "x$GCC" = xyes; then LM_TRY_ENABLE_CFLAG([-Werror=return-type], [CFLAGS]) fi +dnl ---------------------------------------------------------------------- +dnl Enable -fsanitize= flags. +dnl ---------------------------------------------------------------------- + +m4_define(DEFAULT_SANITIZERS, [address,undefined]) +AC_ARG_ENABLE( + sanitizers, + AS_HELP_STRING( + [--enable-sanitizers@<:@=comma-separated list of sanitizers@:>@], + [Default=DEFAULT_SANITIZERS]), +[ +case "$enableval" in + no) sanitizers= ;; + yes) sanitizers="-fsanitize=DEFAULT_SANITIZERS" ;; + *) sanitizers="-fsanitize=$enableval" ;; +esac +CFLAGS="$CFLAGS $sanitizers" +LDFLAGS="$LDFLAGS $sanitizers" +]) + dnl dnl If ${ERL_TOP}/make/otp_ded.mk.in exists and contains DED_MK_VSN > 0, dnl every thing releted to compiling Dynamic Erlang Drivers can be found diff --git a/lib/megaco/doc/src/notes.xml b/lib/megaco/doc/src/notes.xml index f71166b1b1..9a260c3878 100644 --- a/lib/megaco/doc/src/notes.xml +++ b/lib/megaco/doc/src/notes.xml @@ -36,7 +36,40 @@ section is the version number of Megaco.</p> - <section><title>Megaco 3.17.0.3</title> + <section><title>Megaco 3.17.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Implement --enable-sanitizers[=sanitizers]. Similar to + debugging with Valgrind, it's very useful to enable + -fsanitize= switches to catch bugs at runtime.</p> + <p> + Own Id: OTP-12153</p> + </item> + </list> + </section> + +</section> + +<section><title>Megaco 3.17.1</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> The default encoding of Erlang files has been changed + from ISO-8859-1 to UTF-8. </p> <p> The encoding of XML + files has also been changed to UTF-8. </p> + <p> + Own Id: OTP-10907</p> + </item> + </list> + </section> + +</section> + +<section><title>Megaco 3.17.0.3</title> <section><title>Improvements and New Features</title> <list> diff --git a/lib/megaco/src/app/megaco.appup.src b/lib/megaco/src/app/megaco.appup.src index db59f55b55..a3a2e2ea9c 100644 --- a/lib/megaco/src/app/megaco.appup.src +++ b/lib/megaco/src/app/megaco.appup.src @@ -174,11 +174,19 @@ %% | %% v %% 3.17.0.3 +%% | +%% v +%% 3.17.1 +%% | +%% v +%% 3.17.2 %% %% {"%VSN%", [ + {"3.17.1", [{restart_application,megaco}]}, + {"3.17.0.3", [{restart_application,megaco}]}, {"3.17.0.2", []}, {"3.17.0.1", []}, {"3.17", []}, @@ -190,6 +198,8 @@ } ], [ + {"3.17.1", [{restart_application,megaco}]}, + {"3.17.0.3", [{restart_application,megaco}]}, {"3.17.0.2", []}, {"3.17.0.1", []}, {"3.17", []}, diff --git a/lib/megaco/vsn.mk b/lib/megaco/vsn.mk index 373f5199bf..1f4e3b8e95 100644 --- a/lib/megaco/vsn.mk +++ b/lib/megaco/vsn.mk @@ -18,6 +18,6 @@ # %CopyrightEnd% APPLICATION = megaco -MEGACO_VSN = 3.17.1 +MEGACO_VSN = 3.17.2 PRE_VSN = APP_VSN = "$(APPLICATION)-$(MEGACO_VSN)$(PRE_VSN)" diff --git a/lib/mnesia/doc/src/notes.xml b/lib/mnesia/doc/src/notes.xml index 0e9131190d..e5c7d87f52 100644 --- a/lib/mnesia/doc/src/notes.xml +++ b/lib/mnesia/doc/src/notes.xml @@ -38,7 +38,127 @@ thus constitutes one section in this document. The title of each section is the version number of Mnesia.</p> - <section><title>Mnesia 4.11</title> + <section><title>Mnesia 4.12.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Various logging fixes, including: Add run queue index to + the process dump in crash dumps.<br/> Add thread index to + enomem slogan when crashing.<br/> Remove error logger + message for sending messages to old instances of the same + node.</p> + <p> + Own Id: OTP-12115</p> + </item> + </list> + </section> + +</section> + +<section><title>Mnesia 4.12.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fixed a race which could make create_table fail if a node + was going down during the transaction.</p> + <p> + Own Id: OTP-12124 Aux Id: seq12694 </p> + </item> + </list> + </section> + +</section> + +<section><title>Mnesia 4.12.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Force load table could hang when a node went away during + start up.</p> + <p> + Own Id: OTP-11948 Aux Id: seq12585 </p> + </item> + <item> + <p> + The time for inserting locks for a transaction with large + number of locks is reduced significantly.</p> + <p> + Own Id: OTP-11981</p> + </item> + </list> + </section> + +</section> + +<section><title>Mnesia 4.12</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Some local implementations of removing the last element + from a list are replaced by <c>lists:droplast/1</c>. Note + that this requires at least <c>stdlib-2.0</c>, which is + the stdlib version delivered in OTP 17.0. (Thanks to Hans + Svensson)</p> + <p> + Own Id: OTP-11678</p> + </item> + <item> + <p> + Application upgrade (appup) files are corrected for the + following applications: </p> + <p> + <c>asn1, common_test, compiler, crypto, debugger, + dialyzer, edoc, eldap, erl_docgen, et, eunit, gs, hipe, + inets, observer, odbc, os_mon, otp_mibs, parsetools, + percept, public_key, reltool, runtime_tools, ssh, + syntax_tools, test_server, tools, typer, webtool, wx, + xmerl</c></p> + <p> + A new test utility for testing appup files is added to + test_server. This is now used by most applications in + OTP.</p> + <p> + (Thanks to Tobias Schlager)</p> + <p> + Own Id: OTP-11744</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + To prevent a race condition if there is a short + communication problem when node-down and node-up events + are received. They are now stored and later checked if + the node came up just before mnesia flagged the node as + down. (Thanks to Jonas Falkevik )</p> + <p> + Own Id: OTP-11497</p> + </item> + <item> + <p> + Added <c>mnesia:sync_log/0</c> to explicit sync mnesias + transaction log.</p> + <p> + Own Id: OTP-11729</p> + </item> + </list> + </section> + +</section> + +<section><title>Mnesia 4.11</title> <section><title>Fixed Bugs and Malfunctions</title> <list> diff --git a/lib/mnesia/src/mnesia.hrl b/lib/mnesia/src/mnesia.hrl index 2855792646..c8010d5466 100644 --- a/lib/mnesia/src/mnesia.hrl +++ b/lib/mnesia/src/mnesia.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2011. All Rights Reserved. +%% Copyright Ericsson AB 1996-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -35,7 +35,7 @@ -define(ets_last(Tab), ets:last(Tab)). -define(ets_prev(Tab, Key), ets:prev(Tab, Key)). -define(ets_slot(Tab, Pos), ets:slot(Tab, Pos)). --define(ets_new_table(Tab, Props), ets:new(Tab, Props)). +-define(ets_new_table(Tab, Props), _ = ets:new(Tab, Props)). -define(ets_delete_table(Tab), ets:delete(Tab)). -define(ets_fixtable(Tab, Bool), ets:fixtable(Tab, Bool)). diff --git a/lib/mnesia/src/mnesia_controller.erl b/lib/mnesia/src/mnesia_controller.erl index a83e55ac62..5a9bae54da 100644 --- a/lib/mnesia/src/mnesia_controller.erl +++ b/lib/mnesia/src/mnesia_controller.erl @@ -300,8 +300,13 @@ mnesia_down(Node) -> end. wait_for_schema_commit_lock() -> - link(whereis(?SERVER_NAME)), - unsafe_call(wait_for_schema_commit_lock). + try + Pid = whereis(?SERVER_NAME), + link(Pid), %% Keep the link until release_schema_commit_lock + gen_server:call(Pid, wait_for_schema_commit_lock, infinity) + catch _:_ -> + mnesia:abort({node_not_running, node()}) + end. block_controller() -> call(block_controller). @@ -557,12 +562,6 @@ cast(Msg) -> abcast(Nodes, Msg) -> gen_server:abcast(Nodes, ?SERVER_NAME, Msg). -unsafe_call(Msg) -> - case whereis(?SERVER_NAME) of - undefined -> {error, {node_not_running, node()}}; - Pid -> gen_server:call(Pid, Msg, infinity) - end. - call(Msg) -> case whereis(?SERVER_NAME) of undefined -> @@ -1208,7 +1207,14 @@ handle_info(Done = #loader_done{worker_pid=WPid, table_name=Tab}, State0) -> {value,{_,Worker}} = lists:keysearch(WPid,1,get_loaders(State0)), add_loader(Tab,Worker,State1); _ -> - State1 + DelState = State1#state{late_loader_queue=gb_trees:delete_any(Tab, LateQueue0)}, + case ?catch_val({Tab, storage_type}) of + ram_copies -> + cast({disc_load, Tab, ram_only}), + DelState; + _ -> + DelState + end end end, State3 = opt_start_worker(State2), diff --git a/lib/mnesia/src/mnesia_frag.erl b/lib/mnesia/src/mnesia_frag.erl index 4a1616e054..66fc20913c 100644 --- a/lib/mnesia/src/mnesia_frag.erl +++ b/lib/mnesia/src/mnesia_frag.erl @@ -939,7 +939,7 @@ do_split(_FH, _OldN, _FragNames, [], Ops) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% Delete a fragment from a fragmented table -%% and merge its records with an other fragment +%% and merge its records with another fragment make_multi_del_frag(Tab) -> verify_multi(Tab), diff --git a/lib/mnesia/src/mnesia_index.erl b/lib/mnesia/src/mnesia_index.erl index 8fef611a48..87cb58dae1 100644 --- a/lib/mnesia/src/mnesia_index.erl +++ b/lib/mnesia/src/mnesia_index.erl @@ -52,7 +52,11 @@ val(Var) -> case ?catch_val(Var) of - {'EXIT', _ReASoN_} -> mnesia_lib:other_val(Var, _ReASoN_); + {'EXIT', _ReASoN_} -> + case mnesia_lib:other_val(Var) of + error -> mnesia_lib:pr_other(Var, _ReASoN_); + Val -> Val + end; _VaLuE_ -> _VaLuE_ end. diff --git a/lib/mnesia/src/mnesia_lib.erl b/lib/mnesia/src/mnesia_lib.erl index 109e924971..a32c69c59e 100644 --- a/lib/mnesia/src/mnesia_lib.erl +++ b/lib/mnesia/src/mnesia_lib.erl @@ -115,6 +115,8 @@ mkcore/1, not_active_here/1, other_val/2, + other_val/1, + pr_other/2, overload_read/0, overload_read/1, overload_set/2, @@ -389,16 +391,19 @@ unset(Var) -> ?ets_delete(mnesia_gvar, Var). other_val(Var, Other) -> + case other_val(Var) of + error -> pr_other(Var, Other); + Val -> Val + end. + +other_val(Var) -> case Var of {_, where_to_read} -> nowhere; {_, where_to_write} -> []; {_, active_replicas} -> []; - _ -> - pr_other(Var, Other) + _ -> error end. --spec pr_other(_,_) -> no_return(). - pr_other(Var, Other) -> Why = case is_running() of diff --git a/lib/mnesia/src/mnesia_loader.erl b/lib/mnesia/src/mnesia_loader.erl index 4afbea1cc2..530317bcdd 100644 --- a/lib/mnesia/src/mnesia_loader.erl +++ b/lib/mnesia/src/mnesia_loader.erl @@ -208,7 +208,8 @@ do_get_network_copy(Tab, Reason, Ns, Storage, Cs) -> set({Tab, load_node}, Node), set({Tab, load_reason}, Reason), mnesia_controller:i_have_tab(Tab), - dbg_out("Table ~p copied from ~p to ~p~n", [Tab, Node, node()]), + dbg_out("Table ~p copied from ~p to ~p (~b entries)~n", + [Tab, Node, node(), mnesia:table_info(Tab, size)]), {loaded, ok}; Err = {error, _} when element(1, Reason) == dumper -> {not_loaded,Err}; diff --git a/lib/mnesia/src/mnesia_locker.erl b/lib/mnesia/src/mnesia_locker.erl index 32cea903c9..e27396731f 100644 --- a/lib/mnesia/src/mnesia_locker.erl +++ b/lib/mnesia/src/mnesia_locker.erl @@ -84,7 +84,7 @@ init(Parent) -> register(?MODULE, self()), process_flag(trap_exit, true), ?ets_new_table(mnesia_held_locks, [ordered_set, private, named_table]), - ?ets_new_table(mnesia_tid_locks, [bag, private, named_table]), + ?ets_new_table(mnesia_tid_locks, [ordered_set, private, named_table]), ?ets_new_table(mnesia_sticky_locks, [set, private, named_table]), ?ets_new_table(mnesia_lock_queue, [bag, private, named_table, {keypos, 2}]), @@ -131,9 +131,14 @@ send_release_tid(Nodes, Tid) -> receive_release_tid_acc([Node | Nodes], Tid) -> receive {?MODULE, Node, {tid_released, Tid}} -> - receive_release_tid_acc(Nodes, Tid); - {mnesia_down, Node} -> receive_release_tid_acc(Nodes, Tid) + after 0 -> + receive + {?MODULE, Node, {tid_released, Tid}} -> + receive_release_tid_acc(Nodes, Tid); + {mnesia_down, Node} -> + receive_release_tid_acc(Nodes, Tid) + end end; receive_release_tid_acc([], _Tid) -> ok. @@ -248,13 +253,13 @@ loop(State) -> end. set_lock(Tid, Oid, Op, []) -> - ?ets_insert(mnesia_tid_locks, {Tid, Oid, Op}), + ?ets_insert(mnesia_tid_locks, {{Tid, Oid, Op}}), ?ets_insert(mnesia_held_locks, {Oid, Op, [{Op, Tid}]}); set_lock(Tid, Oid, read, [{Oid, Prev, Items}]) -> - ?ets_insert(mnesia_tid_locks, {Tid, Oid, read}), + ?ets_insert(mnesia_tid_locks, {{Tid, Oid, read}}), ?ets_insert(mnesia_held_locks, {Oid, Prev, [{read, Tid}|Items]}); set_lock(Tid, Oid, write, [{Oid, _Prev, Items}]) -> - ?ets_insert(mnesia_tid_locks, {Tid, Oid, write}), + ?ets_insert(mnesia_tid_locks, {{Tid, Oid, write}}), ?ets_insert(mnesia_held_locks, {Oid, write, [{write, Tid}|Items]}); set_lock(Tid, Oid, Op, undefined) -> set_lock(Tid, Oid, Op, ?ets_lookup(mnesia_held_locks, Oid)). @@ -270,7 +275,8 @@ try_sticky_lock(Tid, Op, Pid, {Tab, _} = Oid) -> try_lock(Tid, Op, Pid, Oid); [{_,N}] -> Req = {Pid, {Op, Tid, Oid}}, - Pid ! {?MODULE, node(), {switch, N, Req}} + Pid ! {?MODULE, node(), {switch, N, Req}}, + true end. try_lock(Tid, read_write, Pid, Oid) -> @@ -293,7 +299,7 @@ try_lock(Tid, Op, SimpleOp, Lock, Pid, Oid) -> ?ets_insert(mnesia_lock_queue, #queue{oid = Oid, tid = Tid, op = Op, pid = Pid, lucky = Lucky}), - ?ets_insert(mnesia_tid_locks, {Tid, Oid, {queued, Op}}) + ?ets_insert(mnesia_tid_locks, {{Tid, Oid, {queued, Op}}}) end. grant_lock(Tid, read, Lock, Oid = {Tab, Key}, Default) @@ -492,7 +498,7 @@ set_read_lock_on_all_keys(Tid, From, Tab, IxKey, Pos) -> ?ets_insert(mnesia_lock_queue, #queue{oid = Oid, tid = Tid, op = Op, pid = From, lucky = Lucky}), - ?ets_insert(mnesia_tid_locks, {Tid, Oid, {queued, Op}}) + ?ets_insert(mnesia_tid_locks, {{Tid, Oid, {queued, Op}}}) end. %%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -508,7 +514,8 @@ release_remote_non_pending(Node, Pending) -> %% running at the failed node and also simply remove all %% queue'd requests back to the failed node - AllTids = ?ets_match(mnesia_tid_locks, {'$1', '_', '_'}), + AllTids0 = ?ets_match(mnesia_tid_locks, {{'$1', '_', '_'}}), + AllTids = lists:usort(AllTids0), Tids = [T || [T] <- AllTids, Node == node(T#tid.pid), not lists:member(T, Pending)], do_release_tids(Tids). @@ -519,9 +526,10 @@ do_release_tids([]) -> ok. do_release_tid(Tid) -> - Locks = ?ets_lookup(mnesia_tid_locks, Tid), + Objects = ets:select(mnesia_tid_locks, [{{{Tid, '_', '_'}}, [], ['$_']}]), + Locks = lists:map(fun({L}) -> L end, Objects), ?dbg("Release ~p ~p ~n", [Tid, Locks]), - ?ets_delete(mnesia_tid_locks, Tid), + [?ets_delete(mnesia_tid_locks, L) || L <- Locks], release_locks(Locks), %% Removed queued locks which has had locks UniqueLocks = keyunique(lists:sort(Locks),[]), diff --git a/lib/mnesia/src/mnesia_recover.erl b/lib/mnesia/src/mnesia_recover.erl index 0548a25ebf..b6492707e2 100644 --- a/lib/mnesia/src/mnesia_recover.erl +++ b/lib/mnesia/src/mnesia_recover.erl @@ -178,7 +178,11 @@ log_decision(D) -> val(Var) -> case ?catch_val(Var) of - {'EXIT', Reason} -> mnesia_lib:other_val(Var, Reason); + {'EXIT', Reason} -> + case mnesia_lib:other_val(Var) of + error -> mnesia_lib:pr_other(Var, Reason); + Val -> Val + end; Value -> Value end. diff --git a/lib/mnesia/src/mnesia_snmp_hook.erl b/lib/mnesia/src/mnesia_snmp_hook.erl index 893b39f3c0..256f83b029 100644 --- a/lib/mnesia/src/mnesia_snmp_hook.erl +++ b/lib/mnesia/src/mnesia_snmp_hook.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2011. All Rights Reserved. +%% Copyright Ericsson AB 1996-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -32,7 +32,11 @@ val(Var) -> case ?catch_val(Var) of - {'EXIT', _ReASoN_} -> mnesia_lib:other_val(Var, _ReASoN_); + {'EXIT', _ReASoN_} -> + case mnesia_lib:other_val(Var) of + error -> mnesia_lib:pr_other(Var, _ReASoN_); + Val -> Val + end; _VaLuE_ -> _VaLuE_ end. diff --git a/lib/mnesia/test/mnesia_config_test.erl b/lib/mnesia/test/mnesia_config_test.erl index 6baf86a4a5..c495bce63f 100644 --- a/lib/mnesia/test/mnesia_config_test.erl +++ b/lib/mnesia/test/mnesia_config_test.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2013. All Rights Reserved. +%% Copyright Ericsson AB 1997-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -639,10 +639,10 @@ send_compressed(Config) -> end, ?match([], mnesia_test_lib:kill_mnesia([N2])), - + sys:get_status(mnesia_monitor), %% sync N1 ?match([], mnesia_test_lib:kill_mnesia([N1])), ?match(ok, mnesia:start([{send_compressed, 9}])), - ?match(ok, mnesia:wait_for_tables([t0,t1,t2], 5000)), + ?match(ok, mnesia:wait_for_tables([t0,t1,t2], 25000)), ?match({atomic, ok}, mnesia:transaction(Create, [t0])), ?match({atomic, ok}, mnesia:transaction(Create, [t1])), @@ -1158,6 +1158,7 @@ dynamic_basic(Config) when is_list(Config) -> %%% SYNC!!! timer:sleep(1000), + sys:get_status(mnesia_monitor), ?match([N3,N1], sort(rpc:call(N1, mnesia, system_info, [running_db_nodes]))), ?match([N3,N1], sort(rpc:call(N3, mnesia, system_info, [running_db_nodes]))), diff --git a/lib/mnesia/test/mnesia_evil_coverage_test.erl b/lib/mnesia/test/mnesia_evil_coverage_test.erl index 91820238e5..2d1623b6ca 100644 --- a/lib/mnesia/test/mnesia_evil_coverage_test.erl +++ b/lib/mnesia/test/mnesia_evil_coverage_test.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2013. All Rights Reserved. +%% Copyright Ericsson AB 1996-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -671,13 +671,16 @@ add_copy_when_going_down(Config) -> ?acquire_nodes(2, Config ++ [{tc_timeout, timer:minutes(2)}]), ?match({atomic, ok}, mnesia:create_table(a, [{ram_copies, [Node1]}])), %% Grab a write lock + Tester = self(), WriteAndWait = fun() -> mnesia:write({a,1,1}), - receive continue -> ok + Tester ! {self(), got_lock}, + receive continue -> ok end end, - _Lock = spawn(fun() -> mnesia:transaction(WriteAndWait) end), - Tester = self(), + Locker = spawn(fun() -> mnesia:transaction(WriteAndWait) end), + receive {Locker, got_lock} -> ok end, + spawn_link(fun() -> Res = rpc:call(Node2, mnesia, add_table_copy, [a, Node2, ram_copies]), Tester ! {test, Res} diff --git a/lib/mnesia/test/mnesia_qlc_test.erl b/lib/mnesia/test/mnesia_qlc_test.erl index 5f46840ae9..9886754710 100644 --- a/lib/mnesia/test/mnesia_qlc_test.erl +++ b/lib/mnesia/test/mnesia_qlc_test.erl @@ -264,7 +264,7 @@ atomic_eval(Config) -> ?match({1,[{a,{a,9},91}]}, ok(Restart,[Pid3, Cursor])), QC1 = ok(fun() -> qlc:cursor(Q1) end, []), - ?match({'EXIT', _}, qlc:next_answers(QC1)), + ?match({'EXIT', _}, (catch qlc:next_answers(QC1))), ?match({aborted,_}, ok(fun()->qlc:next_answers(QC1)end,[])), ?verify_mnesia(Ns, []). diff --git a/lib/mnesia/test/mnesia_test_lib.hrl b/lib/mnesia/test/mnesia_test_lib.hrl index 281634c239..94a195f01f 100644 --- a/lib/mnesia/test/mnesia_test_lib.hrl +++ b/lib/mnesia/test/mnesia_test_lib.hrl @@ -46,15 +46,32 @@ -define(match(ExpectedRes,Expr), fun() -> - AcTuAlReS = (catch (Expr)), - case AcTuAlReS of - ExpectedRes -> - ?verbose("ok, ~n Result as expected:~p~n",[AcTuAlReS]), - {success,AcTuAlReS}; - _ -> - ?error("Not Matching Actual result was:~n ~p~n", - [AcTuAlReS]), - {fail,AcTuAlReS} + try Expr of + _AR_0 = ExpectedRes -> + ?verbose("ok, ~n Result as expected:~p~n",[_AR_0]), + {success,_AR_0}; + _AR_0 -> + ?error("Not Matching Actual result was:~n ~p~n",[_AR_0]), + {fail,_AR_0} + catch + exit:{aborted, _ER_1} when + element(1, _ER_1) =:= node_not_running; + element(1, _ER_1) =:= bad_commit; + element(1, _ER_1) =:= cyclic -> + %% Need to re-raise these to restart transaction + erlang:raise(exit, {aborted, _ER_1}, erlang:get_stacktrace()); + exit:_AR_1 -> + case fun(_AR_EXIT_) -> {'EXIT', _AR_EXIT_} end(_AR_1) of + _AR_2 = ExpectedRes -> + ?verbose("ok, ~n Result as expected:~p~n",[_AR_2]), + {success,_AR_2}; + _AR_2 -> + ?error("Not Matching Actual result was:~n ~p~n", [_AR_2]), + {fail,_AR_2} + end; + _:_AR_1 -> + ?error("Not Matching Actual result was:~n ~p~n", [_AR_1]), + {fail,_AR_1} end end()). diff --git a/lib/mnesia/test/mnesia_trans_access_test.erl b/lib/mnesia/test/mnesia_trans_access_test.erl index 157e441b27..237984978e 100644 --- a/lib/mnesia/test/mnesia_trans_access_test.erl +++ b/lib/mnesia/test/mnesia_trans_access_test.erl @@ -677,7 +677,7 @@ check_res(sync_dirty, Res) when is_list(Res) -> check_res(ets, Res) when is_list(Res) -> Res; check_res(Type,Res) -> - ?match(bug,{Type,Res}). + ?match({bug, bug},{Type,Res}). read_op(Oid) -> case lists:reverse(mnesia:read(Oid)) of @@ -1118,10 +1118,7 @@ create_live_table_index(Config, Storage) -> ValPos = 3, mnesia:dirty_write({Tab, 1, 2}), - Fun = fun() -> - ?match(ok, mnesia:write({Tab, 2, 2})), - ok - end, + Fun = fun() -> mnesia:write({Tab, 2, 2}) end, ?match({atomic, ok}, mnesia:transaction(Fun)), ?match({atomic, ok}, mnesia:add_table_index(Tab, ValPos)), IRead = fun() -> lists:sort(mnesia:index_read(Tab, 2, ValPos)) end, diff --git a/lib/mnesia/vsn.mk b/lib/mnesia/vsn.mk index c596f98c81..d5b96c5c76 100644 --- a/lib/mnesia/vsn.mk +++ b/lib/mnesia/vsn.mk @@ -1 +1 @@ -MNESIA_VSN = 4.12 +MNESIA_VSN = 4.12.3 diff --git a/lib/observer/doc/src/notes.xml b/lib/observer/doc/src/notes.xml index 9de00b8c16..658ac2c7cf 100644 --- a/lib/observer/doc/src/notes.xml +++ b/lib/observer/doc/src/notes.xml @@ -31,6 +31,106 @@ <p>This document describes the changes made to the Observer application.</p> +<section><title>Observer 2.0.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fixed statusbar on Windows</p> + <p> + Own Id: OTP-12162</p> + </item> + </list> + </section> + +</section> + +<section><title>Observer 2.0.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + crashdump_viewer would crash if the owner of a timer was + specified as the process' registered name. This has been + corrected.</p> + <p> + Own Id: OTP-11919</p> + </item> + <item> + <p> + Fix crash and minor updates.</p> + <p> + Own Id: OTP-11949</p> + </item> + </list> + </section> + +</section> + +<section><title>Observer 2.0</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + etop trace handler now works in smp environment (Thanks + to Péter Gömöri)</p> + <p> + Own Id: OTP-11633</p> + </item> + <item> + <p> + Application upgrade (appup) files are corrected for the + following applications: </p> + <p> + <c>asn1, common_test, compiler, crypto, debugger, + dialyzer, edoc, eldap, erl_docgen, et, eunit, gs, hipe, + inets, observer, odbc, os_mon, otp_mibs, parsetools, + percept, public_key, reltool, runtime_tools, ssh, + syntax_tools, test_server, tools, typer, webtool, wx, + xmerl</c></p> + <p> + A new test utility for testing appup files is added to + test_server. This is now used by most applications in + OTP.</p> + <p> + (Thanks to Tobias Schlager)</p> + <p> + Own Id: OTP-11744</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Removed gs based applications and gs based backends. The + <c>observer</c> application replaces the removed + applications.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-10915</p> + </item> + <item> + <p> + The <c>crashdump_viewer</c> is re-written using + <c>wx</c>. The old <c>webtool</c> interface for + <c>crashdump_viewer</c> does no longer exist.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-11179</p> + </item> + </list> + </section> + +</section> + <section><title>Observer 1.3.1.2</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/observer/src/cdv_dist_cb.erl b/lib/observer/src/cdv_dist_cb.erl index 3860324d6f..f7e6c9aded 100644 --- a/lib/observer/src/cdv_dist_cb.erl +++ b/lib/observer/src/cdv_dist_cb.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2013. All Rights Reserved. +%% Copyright Ericsson AB 2013-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -22,7 +22,8 @@ get_info/1, get_detail_cols/1, get_details/1, - detail_pages/0]). + detail_pages/0, + format/1]). -include_lib("wx/include/wx.hrl"). -include("crashdump_viewer.hrl"). @@ -75,6 +76,11 @@ init_gen_page(Parent, Info) -> Fields = info_fields(), cdv_info_wx:start_link(Parent,{Fields,Info,[]}). +format({creations,Creations}) -> + string:join([integer_to_list(C) || C <- Creations],","); +format(D) -> + D. + %%%----------------------------------------------------------------- %%% Internal info_fields() -> @@ -83,7 +89,7 @@ info_fields() -> {"Type", conn_type}, {"Channel", channel}, {"Controller", {click,controller}}, - {"Creation", creation}, + {"Creation", {{format,fun format/1},creation}}, {"Extra Info", error}]}, {scroll_boxes, [{"Remote Links",1,{click,remote_links}}, diff --git a/lib/observer/src/cdv_timer_cb.erl b/lib/observer/src/cdv_timer_cb.erl index 9cdbfa05a9..d44592cf18 100644 --- a/lib/observer/src/cdv_timer_cb.erl +++ b/lib/observer/src/cdv_timer_cb.erl @@ -27,18 +27,21 @@ %% Defines -define(COL_OWNER, 0). --define(COL_MSG, ?COL_OWNER+1). +-define(COL_NAME, ?COL_OWNER+1). +-define(COL_MSG, ?COL_NAME+1). -define(COL_TIME, ?COL_MSG+1). %% Callbacks for cdv_virtual_list_wx col_to_elem(id) -> col_to_elem(?COL_OWNER); col_to_elem(?COL_OWNER) -> #timer.pid; +col_to_elem(?COL_NAME) -> #timer.name; col_to_elem(?COL_MSG) -> #timer.msg; col_to_elem(?COL_TIME) -> #timer.time. col_spec() -> [{"Owner", ?wxLIST_FORMAT_LEFT, 110}, - {"Message", ?wxLIST_FORMAT_LEFT, 400}, + {"Owner name", ?wxLIST_FORMAT_LEFT, 150}, + {"Message", ?wxLIST_FORMAT_LEFT, 300}, {"Time left (ms)", ?wxLIST_FORMAT_RIGHT, 80}]. get_info(Owner) -> diff --git a/lib/observer/src/cdv_virtual_list_wx.erl b/lib/observer/src/cdv_virtual_list_wx.erl index c5a7d9a2e5..bfe115a42e 100644 --- a/lib/observer/src/cdv_virtual_list_wx.erl +++ b/lib/observer/src/cdv_virtual_list_wx.erl @@ -269,7 +269,7 @@ handle_event(#wx{event=#wxList{type=command_list_item_right_click, MenuId = ?ID_DETAILS + Col, ColText = call(Holder, {get_row, self(), Row, Col}), case ColText of - "[]" -> []; + Empty when Empty=="[]"; Empty=="" -> []; _ -> What = case catch list_to_integer(ColText) of @@ -284,8 +284,13 @@ handle_event(#wx{event=#wxList{type=command_list_item_right_click, end end, MenuCols), - wxWindow:popupMenu(Panel, Menu), - wxMenu:destroy(Menu), + case MenuItems of + [] -> + wxMenu:destroy(Menu); + _ -> + wxWindow:popupMenu(Panel, Menu), + wxMenu:destroy(Menu) + end, {noreply,State#state{menu_items=MenuItems}}; handle_event(#wx{event=#wxList{type=command_list_col_click, col=Col}}, diff --git a/lib/observer/src/crashdump_viewer.erl b/lib/observer/src/crashdump_viewer.erl index a17efbccb0..99329b94e2 100644 --- a/lib/observer/src/crashdump_viewer.erl +++ b/lib/observer/src/crashdump_viewer.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2003-2013. All Rights Reserved. +%% Copyright Ericsson AB 2003-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -298,6 +298,7 @@ expand_binary(Pos) -> %%-------------------------------------------------------------------- init([]) -> ets:new(cdv_dump_index_table,[ordered_set,named_table,public]), + ets:new(cdv_reg_proc_table,[ordered_set,named_table,public]), {ok, #state{}}. %%-------------------------------------------------------------------- @@ -978,9 +979,20 @@ count() -> %%----------------------------------------------------------------- %% Page with all processes procs_summary(File,WS) -> - ParseFun = fun(Fd,Pid) -> + ParseFun = fun(Fd,Pid0) -> + Pid = list_to_pid(Pid0), Proc = get_procinfo(Fd,fun main_procinfo/5, - #proc{pid=list_to_pid(Pid)},WS), + #proc{pid=Pid},WS), + case Proc#proc.name of + undefined -> + true; + Name -> + %% Registered process - store to allow + %% lookup for timers connected to + %% registered name instead of pid. + ets:insert(cdv_reg_proc_table,{Name,Pid}), + ets:insert(cdv_reg_proc_table,{Pid0,Name}) + end, case Proc#proc.memory of undefined -> Proc#proc{memory=Proc#proc.stack_heap}; _ -> Proc @@ -1495,8 +1507,28 @@ get_internal_ets_tables(File,WS) -> %%----------------------------------------------------------------- %% Page with list of all timers get_timers(File,Pid) -> - ParseFun = fun(Fd,Id) -> get_timerinfo_1(Fd,#timer{pid=list_to_pid(Id)}) end, - lookup_and_parse_index(File,{?timer,Pid},ParseFun,"timers"). + ParseFun = fun(Fd,Id) -> get_timerinfo(Fd,Id) end, + T1 = lookup_and_parse_index(File,{?timer,Pid},ParseFun,"timers"), + T2 = case ets:lookup(cdv_reg_proc_table,Pid) of + [{_,Name}] -> + lookup_and_parse_index(File,{?timer,Name},ParseFun,"timers"); + _ -> + [] + end, + T1 ++ T2. + +get_timerinfo(Fd,Id) -> + case catch list_to_pid(Id) of + Pid when is_pid(Pid) -> + get_timerinfo_1(Fd,#timer{pid=Pid}); + _ -> + case ets:lookup(cdv_reg_proc_table,Id) of + [{_,Pid}] when is_pid(Pid) -> + get_timerinfo_1(Fd,#timer{pid=Pid,name=Id}); + [] -> + get_timerinfo_1(Fd,#timer{name=Id}) + end + end. get_timerinfo_1(Fd,Timer) -> case line_head(Fd) of @@ -1576,7 +1608,13 @@ get_nodeinfo(Fd,Nod) -> "Controller" -> get_nodeinfo(Fd,Nod#nod{controller=val(Fd)}); "Creation" -> - get_nodeinfo(Fd,Nod#nod{creation=list_to_integer(val(Fd))}); + %% Throwing away elements like "(refc=1)", which might be + %% printed from a debug compiled emulator. + Creations = lists:flatmap(fun(C) -> try [list_to_integer(C)] + catch error:badarg -> [] + end + end, string:tokens(val(Fd)," ")), + get_nodeinfo(Fd,Nod#nod{creation={creations,Creations}}); "Remote link" -> Procs = val(Fd), % e.g. "<0.31.0> <4322.54.0>" {Local,Remote} = split(Procs), @@ -2559,11 +2597,11 @@ progress_pmap(Report,File,Fun,List) -> {L1,L2} = if length(L)>=NPerProc -> lists:split(NPerProc,L); true -> {L,[]} % last chunk end, - P = spawn( + {P,_Ref} = + spawn_monitor( fun() -> progress_map(Collector,ReportInterval,File,Fun,L1) end), - erlang:monitor(process,P), {L2,[P|Ps]} end, {List,[]}, diff --git a/lib/observer/src/crashdump_viewer.hrl b/lib/observer/src/crashdump_viewer.hrl index ae288ed573..0e2eba6dee 100644 --- a/lib/observer/src/crashdump_viewer.hrl +++ b/lib/observer/src/crashdump_viewer.hrl @@ -108,6 +108,7 @@ -record(timer, {pid, + name, msg, time}). diff --git a/lib/observer/src/observer.erl b/lib/observer/src/observer.erl index 098100e8ee..a30ceecc63 100644 --- a/lib/observer/src/observer.erl +++ b/lib/observer/src/observer.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2011. All Rights Reserved. +%% Copyright Ericsson AB 2011-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -18,8 +18,11 @@ -module(observer). --export([start/0]). +-export([start/0, stop/0]). start() -> observer_wx:start(). + +stop() -> + observer_wx:stop(). diff --git a/lib/observer/src/observer_lib.erl b/lib/observer/src/observer_lib.erl index cedaf7d2b8..34c7b127ff 100644 --- a/lib/observer/src/observer_lib.erl +++ b/lib/observer/src/observer_lib.erl @@ -249,6 +249,8 @@ to_str({func, {F,A}}) when is_atom(F), is_integer(A) -> lists:concat([F, "/", A]); to_str({func, {F,'_'}}) when is_atom(F) -> atom_to_list(F); +to_str({{format,Fun},Value}) when is_function(Fun) -> + Fun(Value); to_str({A, B}) when is_atom(A), is_atom(B) -> lists:concat([A, ":", B]); to_str({M,F,A}) when is_atom(M), is_atom(F), is_integer(A) -> diff --git a/lib/observer/src/observer_procinfo.erl b/lib/observer/src/observer_procinfo.erl index 3ffa5fc77d..8e8a37fc93 100644 --- a/lib/observer/src/observer_procinfo.erl +++ b/lib/observer/src/observer_procinfo.erl @@ -34,6 +34,7 @@ -record(state, {parent, frame, + notebook, pid, pages=[], expand_table, @@ -76,6 +77,7 @@ init([Pid, ParentFrame, Parent]) -> {Frame, #state{parent=Parent, pid=Pid, frame=Frame, + notebook=Notebook, pages=[ProcessPage,MessagePage,DictPage,StackPage,StatePage], expand_table=Table }} @@ -158,6 +160,9 @@ handle_event(#wx{event=#wxHtmlLink{linkInfo=#wxHtmlLinkInfo{href=Info}}}, State) handle_event(Event, _State) -> error({unhandled_event, Event}). +handle_info({get_debug_info, From}, State = #state{notebook=Notebook}) -> + From ! {procinfo_debug, Notebook}, + {noreply, State}; handle_info(_Info, State) -> %% io:format("~p: ~p, Handle info: ~p~n", [?MODULE, ?LINE, Info]), {noreply, State}. diff --git a/lib/observer/src/observer_tv_table.erl b/lib/observer/src/observer_tv_table.erl index 59fe5b5670..7757dfea53 100644 --- a/lib/observer/src/observer_tv_table.erl +++ b/lib/observer/src/observer_tv_table.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2011-2013. All Rights Reserved. +%% Copyright Ericsson AB 2011-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -409,7 +409,7 @@ handle_info({refresh, Min, Min}, State = #state{grid=Grid}) -> wxListCtrl:refreshItem(Grid, Min), %% Avoid assert in wx below if Max is 0 {noreply, State}; handle_info({refresh, Min, Max}, State = #state{grid=Grid}) -> - wxListCtrl:refreshItems(Grid, Min, Max), + Max > 0 andalso wxListCtrl:refreshItems(Grid, Min, Max), {noreply, State}; handle_info(refresh_interval, State = #state{pid=Pid}) -> diff --git a/lib/observer/src/observer_tv_wx.erl b/lib/observer/src/observer_tv_wx.erl index b276965f83..da4cb8e041 100644 --- a/lib/observer/src/observer_tv_wx.erl +++ b/lib/observer/src/observer_tv_wx.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2011-2012. All Rights Reserved. +%% Copyright Ericsson AB 2011-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -132,8 +132,8 @@ handle_event(#wx{event=#wxSize{size={W,_}}}, State=#state{grid=Grid}) -> observer_lib:set_listctrl_col_size(Grid, W), {noreply, State}; -handle_event(#wx{obj=Grid, event=#wxList{type=command_list_item_activated, - itemIndex=Index}}, +handle_event(#wx{event=#wxList{type=command_list_item_activated, + itemIndex=Index}}, State=#state{grid=Grid, node=Node, opt=#opt{type=Type}, tabs=Tabs}) -> Table = lists:nth(Index+1, Tabs), case Table#tab.protection of diff --git a/lib/observer/src/observer_wx.erl b/lib/observer/src/observer_wx.erl index ecb8e132fe..c86f5ea916 100644 --- a/lib/observer/src/observer_wx.erl +++ b/lib/observer/src/observer_wx.erl @@ -19,7 +19,7 @@ -behaviour(wx_object). --export([start/0]). +-export([start/0, stop/0]). -export([create_menus/2, get_attrib/1, get_tracer/0, set_status/1, create_txt_dialog/4, try_rpc/4, return_to_localnode/2]). @@ -69,6 +69,9 @@ start() -> _Obj -> ok end. +stop() -> + wx_object:call(observer, stop). + create_menus(Object, Menus) when is_list(Menus) -> wx_object:call(Object, {create_menus, Menus}). @@ -109,9 +112,6 @@ setup(#state{frame = Frame} = State) -> observer_lib:create_menus(DefMenus, MenuBar, default), wxFrame:setMenuBar(Frame, MenuBar), - StatusBar = wxFrame:createStatusBar(Frame, []), - wxFrame:setTitle(Frame, atom_to_list(node())), - wxStatusBar:setStatusText(StatusBar, atom_to_list(node())), %% Setup panels Panel = wxPanel:new(Frame, []), @@ -127,6 +127,11 @@ setup(#state{frame = Frame} = State) -> wxSizer:add(MainSizer, Notebook, [{proportion, 1}, {flag, ?wxEXPAND}]), wxPanel:setSizer(Panel, MainSizer), + StatusBar = wxStatusBar:new(Frame), + wxFrame:setStatusBar(Frame, StatusBar), + wxFrame:setTitle(Frame, atom_to_list(node())), + wxStatusBar:setStatusText(StatusBar, atom_to_list(node())), + wxNotebook:connect(Notebook, command_notebook_page_changing), wxFrame:connect(Frame, close_window, [{skip, true}]), wxMenu:connect(Frame, command_menu_selected), @@ -331,6 +336,10 @@ handle_call({get_attrib, Attrib}, _From, State) -> handle_call(get_tracer, _From, State=#state{trace_panel=TraceP}) -> {reply, TraceP, State}; +handle_call(stop, _, State = #state{frame = Frame}) -> + wxFrame:destroy(Frame), + {stop, normal, ok, State}; + handle_call(_Msg, _From, State) -> {reply, ok, State}. @@ -367,6 +376,10 @@ handle_info({open_link, Pid0}, State = #state{pro_panel=ProcViewer, frame=Frame} end, {noreply, State}; +handle_info({get_debug_info, From}, State = #state{notebook=Notebook, active_tab=Pid}) -> + From ! {observer_debug, wx:get_env(), Notebook, Pid}, + {noreply, State}; + handle_info({'EXIT', Pid, _Reason}, State) -> io:format("Child (~s) crashed exiting: ~p ~p~n", [pid2panel(Pid, State), Pid,_Reason]), @@ -377,6 +390,7 @@ handle_info(_Info, State) -> terminate(_Reason, #state{frame = Frame}) -> wxFrame:destroy(Frame), + wx:destroy(), ok. code_change(_, _, State) -> diff --git a/lib/observer/test/crashdump_helper.erl b/lib/observer/test/crashdump_helper.erl index 40dbe28d46..0eb4a92c53 100644 --- a/lib/observer/test/crashdump_helper.erl +++ b/lib/observer/test/crashdump_helper.erl @@ -35,7 +35,9 @@ n1_proc(Creator,_N2,Pid2,Port2,_L) -> register(aaaaaaaa,self()), process_flag(save_calls,3), ets:new(cdv_test_ordset_table,[ordered_set]), - erlang:send_after(1000000,self(),cdv_test_timer_message), + erlang:send_after(1000000,self(),cdv_test_timer_message1), + erlang:send_after(1000000,aaaaaaaa,cdv_test_timer_message2), + erlang:send_after(1000000,noexistproc,cdv_test_timer_message3), Port = hd(erlang:ports()), Fun = fun() -> ok end, Ref = make_ref(), diff --git a/lib/observer/test/crashdump_viewer_SUITE.erl b/lib/observer/test/crashdump_viewer_SUITE.erl index 7a582436b4..03ab0c20e1 100644 --- a/lib/observer/test/crashdump_viewer_SUITE.erl +++ b/lib/observer/test/crashdump_viewer_SUITE.erl @@ -120,27 +120,63 @@ start_stop(Config) when is_list(Config) -> Dump = hd(?config(dumps,Config)), timer:sleep(1000), - ProcsBefore = length(processes()), + ProcsBefore = processes(), + NumProcsBefore = length(ProcsBefore), ok = crashdump_viewer:start(Dump), - true = is_pid(whereis(crashdump_viewer_server)), - true = is_pid(whereis(cdv_wx)), - true = is_pid(whereis(cdv_proc_cb)), - true = is_pid(whereis(cdv_port_cb)), - true = is_pid(whereis(cdv_ets_cb)), - true = is_pid(whereis(cdv_timer_cb)), - true = is_pid(whereis(cdv_fun_cb)), - true = is_pid(whereis(cdv_atom_cb)), - true = is_pid(whereis(cdv_dist_cb)), - true = is_pid(whereis(cdv_mod_cb)), + ExpectedRegistered = [crashdump_viewer_server, + cdv_wx, + cdv_proc_cb, + cdv_proc_cb__holder, + cdv_port_cb, + cdv_port_cb__holder, + cdv_ets_cb, + cdv_ets_cb__holder, + cdv_timer_cb, + cdv_timer_cb__holder, + cdv_fun_cb, + cdv_fun_cb__holder, + cdv_atom_cb, + cdv_atom_cb__holder, + cdv_dist_cb, + cdv_dist_cb__holder, + cdv_mod_cb, + cdv_mod_cb__holder], + Regs=[begin + P=whereis(N), + {P,N,erlang:monitor(process,P)} + end || N <- ExpectedRegistered], + ct:log("CDV procs: ~n~p~n",[Regs]), + [true=is_pid(P) || {P,_,_} <- Regs], timer:sleep(5000), % give some time to live ok = crashdump_viewer:stop(), - timer:sleep(1000), % give some time to stop - undefined = whereis(crashdump_viewer_server), - undefined = whereis(cdv_wx), - ProcsAfter=length(processes()), - ProcsAfter=ProcsBefore, + recv_downs(Regs), + timer:sleep(2000), + ProcsAfter = processes(), + NumProcsAfter = length(ProcsAfter), + if NumProcsAfter=/=NumProcsBefore -> + ct:log("Before but not after:~n~p~n", + [[{P,process_info(P)} || P <- ProcsBefore -- ProcsAfter]]), + ct:log("After but not before:~n~p~n", + [[{P,process_info(P)} || P <- ProcsAfter -- ProcsBefore]]), + ct:fail("leaking processes"); + true -> + ok + end, ok. +recv_downs([]) -> + ok; +recv_downs(Regs) -> + receive + {'DOWN',Ref,process,_Pid,_} -> + ct:log("Got 'DOWN' for process ~n~p~n",[_Pid]), + recv_downs(lists:keydelete(Ref,3,Regs)) + after 30000 -> + ct:log("Timeout waiting for down:~n~p~n", + [[{Reg,process_info(P)} || {P,_,_}=Reg <- Regs]]), + ct:log("Message queue:~n~p~n",[process_info(self(),messages)]) + end. + %% Try to load nonexisting file non_existing(Config) when is_list(Config) -> ExpectedReason = "non-existing-file is not an Erlang crash dump\n", @@ -265,10 +301,12 @@ wait_for_progress_done() -> %%%----------------------------------------------------------------- %%% General check of what is displayed for a dump browse_file(File) -> - io:format("Browsing file: ~s~n",[File]), + io:format("~nBrowsing file: ~s",[File]), ok = start_backend(File), + io:format(" backend started",[]), + {ok,_GI=#general_info{},_GenTW} = crashdump_viewer:general_info(), {ok,Procs,_ProcsTW} = crashdump_viewer:processes(), {ok,Ports,_PortsTW} = crashdump_viewer:ports(), @@ -285,10 +323,16 @@ browse_file(File) -> {ok,_HashTabs,_HashTabsTW} = crashdump_viewer:hash_tables(), {ok,_IndexTabs,_IndexTabsTW} = crashdump_viewer:index_tables(), + io:format(" info read",[]), + lookat_all_pids(Procs), + io:format(" pids ok",[]), lookat_all_ports(Ports), + io:format(" ports ok",[]), lookat_all_mods(Mods), + io:format(" mods ok",[]), lookat_all_nodes(Nodes), + io:format(" nodes ok",[]), Procs. % used as second arg to special/2 @@ -303,6 +347,7 @@ special(File,Procs) -> [#proc{pid=Pid0}|_Rest] = lists:keysort(#proc.name,Procs), Pid = pid_to_list(Pid0), {ok,ProcDetails=#proc{},[]} = crashdump_viewer:proc_details(Pid), + io:format(" process details ok",[]), #proc{dict=Dict} = ProcDetails, @@ -314,6 +359,7 @@ special(File,Procs) -> ['#CDVBin',SOffset,SSize,SPos] = proplists:get_value(sub_bin,Dict), {ok,<<_:SSize/binary>>} = crashdump_viewer:expand_binary({SOffset,SSize,SPos}), + io:format(" expand binary ok",[]), ['#CDVPid',X1,Y1,Z1] = proplists:get_value(ext_pid,Dict), ChannelStr1 = integer_to_list(X1), @@ -323,22 +369,34 @@ special(File,Procs) -> integer_to_list(Z1) ++ ">", {error,{other_node,ChannelStr1}} = crashdump_viewer:proc_details(ExtPid), + io:format(" process details external ok",[]), ['#CDVPort',X2,Y2] = proplists:get_value(port,Dict), ChannelStr2 = integer_to_list(X2), Port = "#Port<"++ChannelStr2++"."++integer_to_list(Y2)++">", {ok,_PortDetails=#port{},[]} = crashdump_viewer:port(Port), + io:format(" port details ok",[]), ['#CDVPort',X3,Y3] = proplists:get_value(ext_port,Dict), ChannelStr3 = integer_to_list(X3), ExtPort = "#Port<"++ChannelStr3++"."++integer_to_list(Y3)++">", {error,{other_node,ChannelStr3}} = crashdump_viewer:port(ExtPort), + io:format(" port details external ok",[]), {ok,[_Ets=#ets_table{}],[]} = crashdump_viewer:ets_tables(Pid), - {ok,[_Timer=#timer{}],[]} = crashdump_viewer:timers(Pid), + io:format(" ets tables ok",[]), + + {ok,[#timer{pid=Pid0,name=undefined}, + #timer{pid=Pid0,name="aaaaaaaa"}],[]} = + crashdump_viewer:timers(Pid), + {ok,AllTimers,_TimersTW} = crashdump_viewer:timers(all), + #timer{name="noexistproc"} = + lists:keyfind(undefined,#timer.pid,AllTimers), + io:format(" timers ok:",[]), {ok,Mod1=#loaded_mod{},[]} = crashdump_viewer:loaded_mod_details(atom_to_list(?helper_mod)), + io:format(" modules ok",[]), #loaded_mod{current_size=CS, old_size=OS, old_attrib=A,old_comp_info=C}=Mod1, true = is_integer(CS), @@ -347,6 +405,7 @@ special(File,Procs) -> true = (C=/=undefined), {ok,Mod2=#loaded_mod{},[]} = crashdump_viewer:loaded_mod_details("application"), + io:format(" module details ok",[]), #loaded_mod{old_size="No old code exists", old_attrib=undefined, old_comp_info=undefined}=Mod2, @@ -428,8 +487,10 @@ do_create_dumps(DataDir,Rel) -> end. -%% Create a dump which has two visible nodes, one hidden and one +%% Create a dump which has three visible nodes, one hidden and one %% not connected node, and with monitors and links between nodes. +%% One of the visible nodes is stopped and started again in order to +%% get multiple creations. full_dist_dump(DataDir,Rel) -> Opt = rel_opt(Rel), Pz = "-pz \"" ++ filename:dirname(code:which(?MODULE)) ++ "\"", @@ -450,6 +511,15 @@ full_dist_dump(DataDir,Rel) -> get_response(P4), get_response(P1), + %% start, stop and start a node in order to get multiple 'creations' + {ok,N5} = ?t:start_node(n5,peer,Opt ++ PzOpt), + P51 = rpc:call(N5,?helper_mod,remote_proc,[P1,Creator]), + get_response(P51), + ?t:stop_node(N5), + {ok,N5} = ?t:start_node(n5,peer,Opt ++ PzOpt), + P52 = rpc:call(N5,?helper_mod,remote_proc,[P1,Creator]), + get_response(P52), + {aaaaaaaa,N1} ! {hello,from,other,node}, % distribution message ?t:stop_node(N3), @@ -458,6 +528,7 @@ full_dist_dump(DataDir,Rel) -> ?t:stop_node(N2), ?t:stop_node(N4), + ?t:stop_node(N5), CD. get_response(P) -> diff --git a/lib/observer/test/observer_SUITE.erl b/lib/observer/test/observer_SUITE.erl index c076c5e81e..5cf719acb1 100644 --- a/lib/observer/test/observer_SUITE.erl +++ b/lib/observer/test/observer_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2006-2013. All Rights Reserved. +%% Copyright Ericsson AB 2006-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -19,34 +19,35 @@ -module(observer_SUITE). -include_lib("test_server/include/test_server.hrl"). +-include_lib("wx/include/wx.hrl"). +-include_lib("observer/src/observer_tv.hrl"). %% Test server specific exports --export([all/0, suite/0,groups/0,init_per_suite/1, end_per_suite/1, - init_per_group/2,end_per_group/2]). --export([init_per_testcase/2, end_per_testcase/2]). +-export([all/0, suite/0,groups/0]). +-export([init_per_testcase/2, end_per_testcase/2, + init_per_group/2, end_per_group/2, + init_per_suite/1, end_per_suite/1 + ]). %% Test cases --export([app_file/1, appup_file/1]). +-export([app_file/1, appup_file/1, + basic/1, process_win/1, table_win/1 + ]). %% Default timetrap timeout (set in init_per_testcase) -define(default_timeout, ?t:minutes(1)). -init_per_testcase(_Case, Config) -> - Dog = ?t:timetrap(?default_timeout), - [{watchdog, Dog} | Config]. - -end_per_testcase(_Case, Config) -> - Dog = ?config(watchdog, Config), - ?t:timetrap_cancel(Dog), - ok. - suite() -> [{ct_hooks,[ts_install_cth]}]. -all() -> - [app_file, appup_file]. +all() -> + [app_file, appup_file, {group, gui}]. -groups() -> - []. +groups() -> + [{gui, [], + [basic + , process_win, table_win + ] + }]. init_per_suite(Config) -> Config. @@ -54,12 +55,40 @@ init_per_suite(Config) -> end_per_suite(_Config) -> ok. -init_per_group(_GroupName, Config) -> - Config. +init_per_testcase(_Case, Config) -> + Dog = ?t:timetrap(?default_timeout), + [{watchdog, Dog} | Config]. -end_per_group(_GroupName, Config) -> - Config. +end_per_testcase(_Case, Config) -> + Dog = ?config(watchdog, Config), + ?t:timetrap_cancel(Dog), + ok. +init_per_group(gui, Config) -> + try + case os:type() of + {unix,darwin} -> + exit("Can not test on MacOSX"); + {unix, _} -> + io:format("DISPLAY ~s~n", [os:getenv("DISPLAY")]), + case ct:get_config(xserver, none) of + none -> ignore; + Server -> os:putenv("DISPLAY", Server) + end; + _ -> ignore + end, + wx:new(), + wx:destroy(), + Config + catch + _:undef -> + {skipped, "No wx compiled for this platform"}; + _:Reason -> + SkipReason = io_lib:format("Start wx failed: ~p", [Reason]), + {skipped, lists:flatten(SkipReason)} + end. +end_per_group(_, _) -> + ok. app_file(suite) -> []; @@ -72,3 +101,178 @@ app_file(Config) when is_list(Config) -> %% Testing .appup file appup_file(Config) when is_list(Config) -> ok = ?t:appup_test(observer). + +-define(DBG(Foo), io:format("~p: ~p~n",[?LINE, catch Foo])). + +basic(suite) -> []; +basic(doc) -> [""]; +basic(Config) when is_list(Config) -> + timer:send_after(100, "foobar"), %% Otherwise the timer sever gets added to procs + ProcsBefore = processes(), + NumProcsBefore = length(ProcsBefore), + + ok = observer:start(), + Notebook = setup_whitebox_testing(), + + io:format("Notebook ~p~n",[Notebook]), + Count = wxNotebook:getPageCount(Notebook), + true = Count >= 6, + 0 = wxNotebook:getSelection(Notebook), + timer:sleep(500), + Check = fun(N, TestMore) -> + TestMore andalso + test_page(wxNotebook:getPageText(Notebook, N), + wxNotebook:getCurrentPage(Notebook)), + timer:sleep(200), + ok = wxNotebook:advanceSelection(Notebook) + end, + %% Just verify that we can toogle trough all pages + [_|_] = [Check(N, false) || N <- lists:seq(1, Count)], + %% Cause it to resize + Frame = get_top_level_parent(Notebook), + {W,H} = wxWindow:getSize(Frame), + wxWindow:setSize(Frame, W+10, H+10), + [_|_] = [Check(N, true) || N <- lists:seq(0, Count-1)], + + ok = observer:stop(), + timer:sleep(2000), %% stop is async + ProcsAfter = processes(), + NumProcsAfter = length(ProcsAfter), + if NumProcsAfter=/=NumProcsBefore -> + ct:log("Before but not after:~n~p~n", + [[{P,process_info(P)} || P <- ProcsBefore -- ProcsAfter]]), + ct:log("After but not before:~n~p~n", + [[{P,process_info(P)} || P <- ProcsAfter -- ProcsBefore]]), + ct:fail("leaking processes"); + true -> + ok + end, + ok. + +test_page("Load Charts" ++ _, _Window) -> + %% Just let it display some info and hopefully it doesn't crash + timer:sleep(2000), + ok; +test_page("Applications" ++ _, _Window) -> + ok = application:start(mnesia), + timer:sleep(1000), %% Give it time to refresh + Active = get_active(), + FakeEv = #wx{event=#wxCommand{type=command_listbox_selected, cmdString="mnesia"}}, + Active ! FakeEv, + timer:sleep(1000), %% Give it time to refresh + ok = application:stop(mnesia), + timer:sleep(1000), %% Give it time to refresh + ok; + +test_page("Processes" ++ _, _Window) -> + timer:sleep(500), %% Give it time to refresh + Active = get_active(), + ChangeSort = fun(N) -> + FakeEv = #wx{event=#wxList{type=command_list_col_click, col=N}}, + Active ! FakeEv, + timer:sleep(200) + end, + [ChangeSort(N) || N <- lists:seq(1,5) ++ [0]], + Focus = #wx{event=#wxList{type=command_list_item_focused, itemIndex=2}}, + Active ! Focus, + Activate = #wx{event=#wxList{type=command_list_item_activated}}, + Active ! Activate, + timer:sleep(1000), %% Give it time to refresh + ok; + +test_page(_Title = "Table" ++ _, _Window) -> + Tables = [ets:new(list_to_atom("Test-" ++ [C]), [public]) || C <- lists:seq($A, $Z)], + Table = lists:nth(3, Tables), + ets:insert(Table, [{N,100-N} || N <- lists:seq(1,100)]), + + Active = get_active(), + Active ! refresh_interval, + ChangeSort = fun(N) -> + FakeEv = #wx{event=#wxList{type=command_list_col_click, col=N}}, + Active ! FakeEv, + timer:sleep(200) + end, + [ChangeSort(N) || N <- lists:seq(1,5) ++ [0]], + timer:sleep(1000), + Focus = #wx{event=#wxList{type=command_list_item_selected, itemIndex=2}}, + Active ! Focus, + Activate = #wx{event=#wxList{type=command_list_item_activated, itemIndex=2}}, + Active ! Activate, + + Info = 407, %% whitebox... + Active ! #wx{id=Info}, + timer:sleep(1000), + ok; + +test_page(Title, Window) -> + io:format("Page ~p: ~p~n", [Title, Window]), + %% Just let it display some info and hopefully it doesn't crash + timer:sleep(1000), + ok. + + +process_win(suite) -> []; +process_win(doc) -> [""]; +process_win(Config) when is_list(Config) -> + ok = observer:start(), + ObserverNB = setup_whitebox_testing(), + Parent = get_top_level_parent(ObserverNB), + Frame = observer_procinfo:start(self(), Parent, self()), + PIPid = wx_object:get_pid(Frame), + PIPid ! {get_debug_info, self()}, + Notebook = receive {procinfo_debug, NB} -> NB end, + Count = wxNotebook:getPageCount(Notebook), + Check = fun(_N) -> + ok = wxNotebook:advanceSelection(Notebook), + timer:sleep(400) + end, + [_|_] = [Check(N) || N <- lists:seq(1, Count)], + PIPid ! #wx{event=#wxClose{type=close_window}}, + observer:stop(), + ok. + +table_win(suite) -> []; +table_win(doc) -> [""]; +table_win(Config) when is_list(Config) -> + Tables = [ets:new(list_to_atom("Test-" ++ [C]), [public]) || C <- lists:seq($A, $Z)], + Table = lists:nth(3, Tables), + ets:insert(Table, [{N,100-N} || N <- lists:seq(1,100)]), + ok = observer:start(), + Notebook = setup_whitebox_testing(), + Parent = get_top_level_parent(Notebook), + TObj = observer_tv_table:start_link(Parent, [{node,node()}, {type,ets}, {table,#tab{name=foo, id=Table}}]), + %% Modal can not test edit.. + %% TPid = wx_object:get_pid(TObj), + %% TPid ! #wx{event=#wxList{type=command_list_item_activated, itemIndex=12}}, + timer:sleep(3000), + wx_object:get_pid(TObj) ! #wx{event=#wxClose{type=close_window}}, + observer:stop(), + ok. + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +get_top_level_parent(Window) -> + Parent = wxWindow:getParent(Window), + case wx:is_null(Parent) of + true -> Window; + false -> get_top_level_parent(Parent) + end. + +setup_whitebox_testing() -> + %% So that if we die observer exists + link(whereis(observer)), + {Env, Notebook, _Active} = get_observer_debug(), + wx:set_env(Env), + Notebook. + +get_active() -> + {_, _, Active} = get_observer_debug(), + Active. + +get_observer_debug() -> + observer ! {get_debug_info, self()}, + receive + {observer_debug, Env, Notebook, Active} -> + {Env, Notebook, Active} + end. diff --git a/lib/observer/vsn.mk b/lib/observer/vsn.mk index a6300eeb18..dbbbde1467 100644 --- a/lib/observer/vsn.mk +++ b/lib/observer/vsn.mk @@ -1 +1 @@ -OBSERVER_VSN = 2.0 +OBSERVER_VSN = 2.0.2 diff --git a/lib/odbc/aclocal.m4 b/lib/odbc/aclocal.m4 index c51c26794a..ed492d55ff 100644 --- a/lib/odbc/aclocal.m4 +++ b/lib/odbc/aclocal.m4 @@ -83,8 +83,10 @@ AC_ARG_VAR(erl_xcomp_ose_LM_POST_LINK, [OSE postlink tool (only used when cross AC_ARG_VAR(erl_xcomp_ose_LM_SET_CONF, [Sets the configuration for an OSE load module (only used when cross compiling for OSE)]) AC_ARG_VAR(erl_xcomp_ose_LM_ELF_SIZE, [Prints the section size information for an OSE load module (only used when cross compiling for OSE)]) AC_ARG_VAR(erl_xcomp_ose_LM_LCF, [OSE load module linker configuration file (only used when cross compiling for OSE)]) -AC_ARG_VAR(erl_xcomp_ose_LM_CONF, [OSE load module default configuration file (only used when cross compiling for OSE)]) -AC_ARG_VAR(erl_xcomp_ose_CONFD, [OSE OSE confd source file]) +AC_ARG_VAR(erl_xcomp_ose_BEAM_LM_CONF, [BEAM OSE load module default configuration file (only used when cross compiling for OSE)]) +AC_ARG_VAR(erl_xcomp_ose_EPMD_LM_CONF, [EPMD OSE load module default configuration file (only used when cross compiling for OSE)]) +AC_ARG_VAR(erl_xcomp_ose_RUN_ERL_LM_CONF, [run_erl_lm OSE load module default configuration file (only used when cross compiling for OSE)]) +AC_ARG_VAR(erl_xcomp_ose_CONFD, [OSE confd source file]) AC_ARG_VAR(erl_xcomp_ose_CRT0_LM, [OSE crt0 lm source file]) ]) @@ -1116,7 +1118,7 @@ case "$THR_LIB_NAME" in [Define if you have the "ose_spi/ose_spi.h" header file.])) ;; esac - if test "x$THR_LIB_NAME" == "xpthread"; then + if test "x$THR_LIB_NAME" = "xpthread"; then case $host_os in openbsd*) # The default stack size is insufficient for our needs @@ -1220,7 +1222,7 @@ case "$THR_LIB_NAME" in dnl dnl Check for functions dnl - if test "x$THR_LIB_NAME" == "xpthread"; then + if test "x$THR_LIB_NAME" = "xpthread"; then AC_CHECK_FUNC(pthread_spin_lock, \ [ethr_have_native_spinlock=yes \ AC_DEFINE(ETHR_HAVE_PTHREAD_SPIN_LOCK, 1, \ diff --git a/lib/odbc/c_src/odbcserver.c b/lib/odbc/c_src/odbcserver.c index b4655ce373..84c201a656 100644 --- a/lib/odbc/c_src/odbcserver.c +++ b/lib/odbc/c_src/odbcserver.c @@ -389,6 +389,9 @@ DWORD WINAPI database_handler(const char *port) close_socket(socket); clean_socket_lib(); /* Exit will be done by suervisor thread */ +#ifdef WIN32 + return (DWORD)0; +#endif } /* Description: Calls the appropriate function to handle the database @@ -631,7 +634,7 @@ static db_result_msg db_query(byte *sql, db_state *state) &statement_handle(state)))) DO_EXIT(EXIT_ALLOC); - result = SQLExecDirect(statement_handle(state), sql, SQL_NTS); + result = SQLExecDirect(statement_handle(state), (SQLCHAR *)sql, SQL_NTS); /* SQL_SUCCESS_WITH_INFO at this point may indicate an error in user input. */ if (result != SQL_SUCCESS && result != SQL_NO_DATA_FOUND) { @@ -723,7 +726,7 @@ static db_result_msg db_select_count(byte *sql, db_state *state) (SQLPOINTER)SQL_SCROLLABLE, (SQLINTEGER)0); } - if(!sql_success(SQLExecDirect(statement_handle(state), sql, SQL_NTS))) { + if(!sql_success(SQLExecDirect(statement_handle(state), (SQLCHAR *)sql, SQL_NTS))) { diagnos = get_diagnos(SQL_HANDLE_STMT, statement_handle(state), extended_errors(state)); clean_state(state); return encode_error_message(diagnos.error_msg, extended_error(state, diagnos.sqlState), diagnos.nativeError); @@ -864,7 +867,7 @@ static db_result_msg db_param_query(byte *buffer, db_state *state) if(params != NULL) { - result = SQLExecDirect(statement_handle(state), sql, SQL_NTS); + result = SQLExecDirect(statement_handle(state), (SQLCHAR *)sql, SQL_NTS); if (!sql_success(result) || result == SQL_NO_DATA) { diagnos = get_diagnos(SQL_HANDLE_STMT, statement_handle(state), extended_errors(state)); } @@ -955,7 +958,7 @@ static db_result_msg db_describe_table(byte *sql, db_state *state) &statement_handle(state)))) DO_EXIT(EXIT_ALLOC); - if (!sql_success(SQLPrepare(statement_handle(state), sql, SQL_NTS))){ + if (!sql_success(SQLPrepare(statement_handle(state), (SQLCHAR *)sql, SQL_NTS))){ diagnos = get_diagnos(SQL_HANDLE_STMT, statement_handle(state), extended_errors(state)); msg = encode_error_message(diagnos.error_msg, extended_error(state, diagnos.sqlState), diagnos.nativeError); clean_state(state); @@ -1324,7 +1327,7 @@ static db_result_msg encode_column_name_list(SQLSMALLINT num_of_columns, if (columns(state)[i].type.c == SQL_C_BINARY) { /* retrived later by retrive_binary_data */ - }else { + } else { if(!sql_success( SQLBindCol (statement_handle(state), @@ -1336,7 +1339,7 @@ static db_result_msg encode_column_name_list(SQLSMALLINT num_of_columns, DO_EXIT(EXIT_BIND); } ei_x_encode_string_len(&dynamic_buffer(state), - name, name_len); + (char *)name, name_len); } else { columns(state)[i].type.len = 0; @@ -2739,8 +2742,8 @@ static diagnos get_diagnos(SQLSMALLINT handleType, SQLHANDLE handle, Boolean ext the error message is obtained */ for(record_nr = 1; ;record_nr++) { result = SQLGetDiagRec(handleType, handle, record_nr, current_sql_state, - &nativeError, current_errmsg_pos, - (SQLSMALLINT)errmsg_buffer_size, &errmsg_size); + &nativeError, (SQLCHAR *)current_errmsg_pos, + (SQLSMALLINT)errmsg_buffer_size, &errmsg_size); if(result == SQL_SUCCESS) { /* update the sqlstate in the diagnos record, because the SQLGetDiagRec call succeeded */ diff --git a/lib/odbc/c_src/odbcserver.h b/lib/odbc/c_src/odbcserver.h index 916a7cb31d..7112fd2d47 100644 --- a/lib/odbc/c_src/odbcserver.h +++ b/lib/odbc/c_src/odbcserver.h @@ -119,7 +119,7 @@ /*------------------------ TYPDEFS ----------------------------------*/ -typedef unsigned char byte; +typedef char byte; typedef int Boolean; typedef struct { @@ -201,4 +201,4 @@ typedef enum { #define param_query(db_state) (db_state -> param_query) #define out_params(db_state) (db_state -> out_params) #define extended_errors(db_state) (db_state -> extended_errors) -#define extended_error(db_state, errorcode) ( extended_errors(state) ? errorcode : NULL ) +#define extended_error(db_state, errorcode) ( extended_errors(state) ? ((char *)errorcode) : NULL ) diff --git a/lib/odbc/configure.in b/lib/odbc/configure.in index f86146759c..ea5c51965f 100644 --- a/lib/odbc/configure.in +++ b/lib/odbc/configure.in @@ -228,4 +228,24 @@ if test "x$GCC" = xyes; then LM_TRY_ENABLE_CFLAG([-Werror=return-type], [CFLAGS]) fi +dnl ---------------------------------------------------------------------- +dnl Enable -fsanitize= flags. +dnl ---------------------------------------------------------------------- + +m4_define(DEFAULT_SANITIZERS, [address,undefined]) +AC_ARG_ENABLE( + sanitizers, + AS_HELP_STRING( + [--enable-sanitizers@<:@=comma-separated list of sanitizers@:>@], + [Default=DEFAULT_SANITIZERS]), +[ +case "$enableval" in + no) sanitizers= ;; + yes) sanitizers="-fsanitize=DEFAULT_SANITIZERS" ;; + *) sanitizers="-fsanitize=$enableval" ;; +esac +CFLAGS="$CFLAGS $sanitizers" +LDFLAGS="$LDFLAGS $sanitizers" +]) + AC_OUTPUT(c_src/$host/Makefile:c_src/Makefile.in) diff --git a/lib/odbc/doc/src/error_handling.xml b/lib/odbc/doc/src/error_handling.xml index b255865263..0b6179409d 100644 --- a/lib/odbc/doc/src/error_handling.xml +++ b/lib/odbc/doc/src/error_handling.xml @@ -88,7 +88,7 @@ <section> <title>The whole picture </title> <p>As the Erlang ODBC application relies on third party products - and communicates with a database that probably runs on an other + and communicates with a database that probably runs on another computer in the network there are plenty of things that might go wrong. To fully understand the things that might happen it facilitate to know the design of the Erlang ODBC application, diff --git a/lib/odbc/doc/src/notes.xml b/lib/odbc/doc/src/notes.xml index b254ca3bc9..495a675631 100644 --- a/lib/odbc/doc/src/notes.xml +++ b/lib/odbc/doc/src/notes.xml @@ -31,7 +31,80 @@ <p>This document describes the changes made to the odbc application. </p> - <section><title>ODBC 2.10.19</title> + <section><title>ODBC 2.10.21</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fix compiler warnings reported by LLVM</p> + <p> + Own Id: OTP-12138</p> + </item> + <item> + <p> + Implement --enable-sanitizers[=sanitizers]. Similar to + debugging with Valgrind, it's very useful to enable + -fsanitize= switches to catch bugs at runtime.</p> + <p> + Own Id: OTP-12153</p> + </item> + </list> + </section> + +</section> + +<section><title>ODBC 2.10.20</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Application upgrade (appup) files are corrected for the + following applications: </p> + <p> + <c>asn1, common_test, compiler, crypto, debugger, + dialyzer, edoc, eldap, erl_docgen, et, eunit, gs, hipe, + inets, observer, odbc, os_mon, otp_mibs, parsetools, + percept, public_key, reltool, runtime_tools, ssh, + syntax_tools, test_server, tools, typer, webtool, wx, + xmerl</c></p> + <p> + A new test utility for testing appup files is added to + test_server. This is now used by most applications in + OTP.</p> + <p> + (Thanks to Tobias Schlager)</p> + <p> + Own Id: OTP-11744</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Removed warnings at compile time by adding missing + include file (Thanks to Anthony Ramine)</p> + <p> + Own Id: OTP-11569</p> + </item> + <item> + <p> + Apple has removed iODBC in OS X 10.9 Mavericks, but + forgot to remove all binaries, adopt configure so that + will be possible to build odbc with own installation.</p> + <p> + Own Id: OTP-11630</p> + </item> + </list> + </section> + +</section> + +<section><title>ODBC 2.10.19</title> <section><title>Fixed Bugs and Malfunctions</title> <list> diff --git a/lib/odbc/test/odbc_connect_SUITE.erl b/lib/odbc/test/odbc_connect_SUITE.erl index 2a16388929..1907069726 100644 --- a/lib/odbc/test/odbc_connect_SUITE.erl +++ b/lib/odbc/test/odbc_connect_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2002-2013. All Rights Reserved. +%% Copyright Ericsson AB 2002-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -47,7 +47,7 @@ all() -> case odbc_test_lib:odbc_check() of ok -> [not_exist_db, commit, rollback, not_explicit_commit, - no_c_node, port_dies, control_process_dies, + no_c_executable, port_dies, control_process_dies, {group, client_dies}, connect_timeout, timeout, many_timeouts, timeout_reset, disconnect_on_timeout, connection_closed, disable_scrollable_cursors, @@ -248,28 +248,31 @@ not_exist_db(_Config) -> test_server:sleep(100). %%------------------------------------------------------------------------- -no_c_node(doc) -> +no_c_executable(doc) -> "Test what happens if the port-program can not be found"; -no_c_node(suite) -> []; -no_c_node(_Config) -> +no_c_executable(suite) -> []; +no_c_executable(_Config) -> process_flag(trap_exit, true), Dir = filename:nativename(filename:join(code:priv_dir(odbc), "bin")), FileName1 = filename:nativename(os:find_executable("odbcserver", Dir)), FileName2 = filename:nativename(filename:join(Dir, "odbcsrv")), - ok = file:rename(FileName1, FileName2), - Result = - case catch odbc:connect(?RDBMS:connection_string(), - odbc_test_lib:platform_options()) of - {error, port_program_executable_not_found} -> - ok; - Else -> - Else - end, - - ok = file:rename(FileName2, FileName1), - ok = Result. + case file:rename(FileName1, FileName2) of + ok -> + Result = + case catch odbc:connect(?RDBMS:connection_string(), + odbc_test_lib:platform_options()) of + {error, port_program_executable_not_found} -> + ok; + Else -> + Else + end, + ok = file:rename(FileName2, FileName1), + ok = Result; + _ -> + {skip, "File permission issues"} + end. %%------------------------------------------------------------------------ port_dies(doc) -> diff --git a/lib/odbc/vsn.mk b/lib/odbc/vsn.mk index 1af4751248..b374e42d15 100644 --- a/lib/odbc/vsn.mk +++ b/lib/odbc/vsn.mk @@ -1 +1 @@ -ODBC_VSN = 2.10.20 +ODBC_VSN = 2.10.21 diff --git a/lib/orber/COSS/CosNaming/orber_cosnaming_utils.erl b/lib/orber/COSS/CosNaming/orber_cosnaming_utils.erl index 5ec0c084e3..c665e1fbc8 100644 --- a/lib/orber/COSS/CosNaming/orber_cosnaming_utils.erl +++ b/lib/orber/COSS/CosNaming/orber_cosnaming_utils.erl @@ -176,7 +176,6 @@ addresses({false, Adr, Rest}, Addresses) -> {lists:reverse([Adr|Addresses]), Rest}; addresses({true, Adr, Rest}, Addresses) -> addresses(address(protocol, Rest, [], []), [Adr|Addresses]). - %% Which protocol. address(protocol, [$:|T], [], []) -> address(version, T, [], [iiop]); @@ -192,6 +191,7 @@ address(protocol, What, _, _) -> [?LINE, What], ?DEBUG_LEVEL), corba:raise(#'CosNaming_NamingContextExt_InvalidAddress'{}); + %% Parsed one address, no version found or port found. address(version, [$,|T], Acc, Previous) -> {true, lists:reverse([?DEF_PORT, lists:reverse(Acc), ?DEF_VERS|Previous]), T}; @@ -208,15 +208,26 @@ address(version, [$@|T], Acc, Previous) -> [?LINE, What], ?DEBUG_LEVEL), corba:raise(#'CosNaming_NamingContextExt_InvalidAddress'{}) end; + +%% Found no iiop version, switch to ipv6. +address(version, [$[|T], [], Previous) -> + address(ipv6, T, [], [?DEF_VERS|Previous]); + %% Found no iiop version, switch to port. In this case Acc contains the %% Host. address(version, [$:|T], Acc, Previous) -> - case check_ip_version(T, [$:|Acc]) of - false -> - address(port, T, [], [lists:reverse(Acc), ?DEF_VERS|Previous]); - {ok, NewAcc, NewT, Type} -> - address(Type, NewT, [], [lists:reverse(NewAcc), ?DEF_VERS|Previous]) - end; + address(port, T, [], [lists:reverse(Acc), ?DEF_VERS|Previous]); + +%% Found IPv6 +address(address, [$[|T], [], Previous) -> + address(ipv6, T, [], Previous); + +%% Found port +address(address, [$:|T], Acc, Previous) -> + address(port, T, [], [lists:reverse(Acc)|Previous]); + +address(ipv6, [$]|T], Acc, Previous) -> + address(address, T, Acc, Previous); %% Parsed one address, port not found. address(address, [$,|T], [], Previous) -> @@ -267,68 +278,9 @@ address(address, [], [], Previous) -> address(address, [], Acc, Previous) -> {false, lists:reverse([?DEF_PORT, lists:reverse(Acc)|Previous]), []}; -%% Found port -address(address, [$:|T], Acc, Previous) -> - case check_ip_version(T, [$:|Acc]) of - false -> - address(port, T, [], [lists:reverse(Acc)|Previous]); - {ok, NewAcc, NewT, Type} -> - address(Type, NewT, [], [lists:reverse(NewAcc)|Previous]) - end; - address(Type, [H|T], Acc, Previous) -> address(Type, T, [H|Acc], Previous). - -check_ip_version(T, Acc) -> - case orber_env:ip_version() of - inet -> - false; - inet6 -> - case search_for_delimiter(1, T, Acc, $:) of - {ok, NewAcc, NewT, Type} -> - {ok, NewAcc, NewT, Type}; - _ -> - false - end - end. - -%% An IPv6 address may look like (x == hex, d == dec): -%% * "0:0:0:0:0:0:10.1.1.1" - x:x:x:x:x:x:d.d.d.d -%% * "0:0:0:0:8:800:200C:417A" - x:x:x:x:x:x:x:x -%% We cannot allow compressed addresses (::10.1.1.1) since we it is not -%% possible to know if the last part is a port number or part of the address. -search_for_delimiter(7, [], Acc, $:) -> - {ok, Acc, [], address}; -search_for_delimiter(9, [], Acc, $.) -> - {ok, Acc, [], address}; -search_for_delimiter(_, [], _, _) -> - false; -search_for_delimiter(7, [$/|T], Acc, $:) -> - {ok, Acc, [$/|T], address}; -search_for_delimiter(9, [$/|T], Acc, $.) -> - {ok, Acc, [$/|T], address}; -search_for_delimiter(_, [$/|_T], _Acc, _) -> - false; -search_for_delimiter(7, [$,|T], Acc, $:) -> - {ok, Acc, [$,|T], address}; -search_for_delimiter(9, [$,|T], Acc, $.) -> - {ok, Acc, [$,|T], address}; -search_for_delimiter(_, [$,|_T], _Acc, _) -> - false; -search_for_delimiter(7, [$:|T], Acc, $:) -> - {ok, Acc, T, port}; -search_for_delimiter(9, [$:|T], Acc, $.) -> - {ok, Acc, T, port}; -search_for_delimiter(N, [$:|T], Acc, $:) -> - search_for_delimiter(N+1, T, [$:|Acc], $:); -search_for_delimiter(N, [$.|T], Acc, $.) when N > 6, N < 9 -> - search_for_delimiter(N+1, T, [$.|Acc], $.); -search_for_delimiter(6, [$.|T], Acc, $:) -> - search_for_delimiter(7, T, [$.|Acc], $.); -search_for_delimiter(N, [H|T], Acc, LookingFor) -> - search_for_delimiter(N, T, [H|Acc], LookingFor). - %%---------------------------------------------------------------------- %% Function : key %% Arguments : A string which contain a Key we want to use and, if defined, diff --git a/lib/orber/doc/src/ch_naming_service.xml b/lib/orber/doc/src/ch_naming_service.xml index b735629a14..e355db2edb 100644 --- a/lib/orber/doc/src/ch_naming_service.xml +++ b/lib/orber/doc/src/ch_naming_service.xml @@ -273,25 +273,28 @@ lists:foreach(fun({{Id, Kind},BindingType}) -> case BindingType of <p>The notation of this scheme is similar to the more well known URL <c>HTTP</c>, and the full <c>corbaloc</c> BNF is:</p> <code type="none"><![CDATA[ -<corbaloc> = "corbaloc:"<obj_addr_list>["/"<key_string>] -<obj_addr_list> = [<obj_addr>","]*<obj_addr> -<obj_addr> = <prot_addr> | <future_prot_addr> -<prot_addr> = <rir_prot_addr> | <iiop_prot_addr> -<rir_prot_addr> = <rir_prot_token>":" -<rir_prot_token> = rir -<future_prot_addr> = <future_prot_id><future_prot_addr> -<future_prot_id> = <future_prot_token>":" -<iiop_prot_addr> = <iiop_id><iiop_addr> -<iiop_id> = <iiop_default> | <iiop_prot_token>":" -<iiop_default> = ":" -<iiop_prot_token> = "iiop" -<iiop_addr> = <version><host>[":"<port>] -<host> = DNS-style Host Name | ip_address -<version> = <major>"."<minor>"@" | empty_string -<port> = number -<major> = number -<minor> = number -<key_string> = for example NameService +<corbaloc> = "corbaloc:"<obj_addr_list>["/"<key_string>] +<obj_addr_list> = [<obj_addr>","]*<obj_addr> +<obj_addr> = <prot_addr> | <future_prot_addr> +<prot_addr> = <rir_prot_addr> | <iiop_prot_addr> +<rir_prot_addr> = <rir_prot_token>":" +<rir_prot_token> = rir +<future_prot_addr> = <future_prot_id><future_prot_addr> +<future_prot_id> = <future_prot_token>":" +<iiop_prot_addr> = <iiop_id><iiop_addr> +<iiop_id> = <iiop_default> | <iiop_prot_token>":" +<iiop_default> = ":" +<iiop_prot_token> = "iiop" +<iiop_addr> = <version><host>[":"<port>] +<host> = <DNS-style Host Name> | <ip_v4_address> | "["<ip_v6_address>"]" +<version> = <major>"."<minor>"@" | empty_string +<port> = number +<major> = number +<minor> = number +<DNS-style Host Name> = string +<ip_v4_address> = string +<ip_v6_address> = string +<key_string> = for example NameService ]]></code> <p>The <c>corbaloc</c> scheme consists of 3 parts:</p> <list type="bulleted"> diff --git a/lib/orber/doc/src/corba.xml b/lib/orber/doc/src/corba.xml index 004c7fb9b0..4a11b271b4 100644 --- a/lib/orber/doc/src/corba.xml +++ b/lib/orber/doc/src/corba.xml @@ -294,7 +294,9 @@ Example: <p>This function returns the object reference for the object id asked for. The remote modifier string has the following format: - <c>"iiop://host:port"</c>.</p> + <c>"iiop://"<host>":"<port></c> where <c><host> = <DNS hostname> | + <IPv4 address> | "["<IPv6 address>"]"</c>. + </p> <p>The <em>configuration</em> context is used to override the global SSL client side <seealso marker="ch_install#config">configuration</seealso>.</p> @@ -322,8 +324,11 @@ Example: <v>ObjectId = string()</v> </type> <desc> - <p>This function returns a list of allowed object id's. The remote modifier - string has the following format: <c>"iiop://host:port"</c>.</p> + <p>This function returns a list of allowed object id's. + The remote modifier string has the following format: + <c>"iiop://"<host>":"<port></c> where <c><host> = <DNS hostname> | + <IPv4 address> | "["<IPv6 address>"]"</c>. + </p> <p>The <em>configuration</em> context is used to override the global SSL client side <seealso marker="ch_install#config">configuration</seealso>.</p> @@ -365,9 +370,11 @@ Example: <p>This function takes a <c>corbaname</c>, <c>corbaloc</c> or an IOR on the external string representation and returns the object reference.</p> <p>To lookup the NameService reference, simply use - <c>"corbaloc:iiop:[email protected]:4001/NameService"</c></p> + <c>"corbaloc:iiop:[email protected]:4001/NameService"</c></p> <p>We can also resolve an object from the NameService by using - <c>"corbaname:iiop:[email protected]:4001/NameService#org/Erlang/MyObj"</c></p> + <c>"corbaname:iiop:[email protected]:4001/NameService#org/Erlang/MyObj"</c></p> + <p>To lookup the NameService reference with an IPv6 address, simply use + <c>"corbaloc:iiop:1.2@[FEC1:0:3:0:0312:44AF:FAB1:3D01]:4001/NameService"</c></p> <p>For more information about <c>corbaname</c> and <c>corbaloc</c>, see the User's Guide (Interoperable Naming Service).</p> <p>The <em>configuration</em> context is used to override the global diff --git a/lib/orber/doc/src/notes.xml b/lib/orber/doc/src/notes.xml index c4d13a5a2f..141d306740 100644 --- a/lib/orber/doc/src/notes.xml +++ b/lib/orber/doc/src/notes.xml @@ -33,7 +33,65 @@ </header> - <section><title>Orber 3.6.26.1</title> + <section><title>Orber 3.7</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> The following functions have been corrected so they + work properly with IPv6 addresses. </p> <list> + <item><c>corba:resolve_initial_references_remote/2/3</c></item> + <item><c>corba:list_initial_references_remote/1/2</c></item> + <item><c>corba:string_to_object/1/2</c></item> </list> + <p> + Own Id: OTP-12016</p> + </item> + <item> + <p> A couple of macros were malformed, missing commas: + PROFILEBODY_1_1_TYPEDEF and PROFILEBODY_1_2_TYPEDEF. + Thanks to Vlad Dumitrescu. </p> + <p> + Own Id: OTP-12062</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> It is now possible to add listen interfaces for IPv6 + when orber is default configured for IPv4 and the other + way around. For more information, consult the User's + Guide and the orber module Reference Manual. </p> + <p> + Own Id: OTP-12007</p> + </item> + </list> + </section> + +</section> + +<section><title>Orber 3.6.27</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Some local implementations of removing the last element + from a list are replaced by <c>lists:droplast/1</c>. Note + that this requires at least <c>stdlib-2.0</c>, which is + the stdlib version delivered in OTP 17.0. (Thanks to Hans + Svensson)</p> + <p> + Own Id: OTP-11678</p> + </item> + </list> + </section> + +</section> + +<section><title>Orber 3.6.26.1</title> <section><title>Improvements and New Features</title> <list> diff --git a/lib/orber/doc/src/orber.xml b/lib/orber/doc/src/orber.xml index 16781059c7..a182a56972 100644 --- a/lib/orber/doc/src/orber.xml +++ b/lib/orber/doc/src/orber.xml @@ -356,8 +356,8 @@ <v>Type = normal | ssl</v> <v>Port = integer() > 0</v> <v>ConfigurationParameters = [{Key, Value}]</v> - <v>Key = flags | iiop_in_connection_timeout | iiop_max_fragments | iiop_max_in_requests | interceptors | iiop_port | iiop_ssl_port | ssl_server_options</v> - <v>Value = as described in the User's Guide</v> + <v>Key = flags | ip_family | iiop_in_connection_timeout | iiop_max_fragments | iiop_max_in_requests | interceptors | iiop_port | iiop_ssl_port | ssl_server_options</v> + <v>Value = as described in the User's Guide or below</v> <v>Result = {ok, Ref} | {error, Reason} | {'EXCEPTION', #'BAD_PARAM'{}}</v> <v>Ref = #Ref</v> <v>Reason = string()</v> @@ -383,6 +383,9 @@ <item><em>flags</em> - currently it is only possible to override the global setting for the <c>Use Current Interface in IOR</c> and <c>Exclude CodeSet Component</c> flags.</item> + <item><em>ip_family</em> - can be set to <c>inet</c> or <c>inet6</c> and is + used to get a listen interface that uses another IP version than the default + set with flags at startup.</item> <item><em>iiop_port</em> - requires that <c>Use Current Interface in IOR</c> is activated and the supplied <c>Type</c> is <c>normal</c>. If so, exported IOR:s will contain the IIOP port defined by this configuration @@ -390,7 +393,7 @@ <item><em>iiop_ssl_port</em> - almost equivalent to <c>iiop_port</c>. The difference is that <c>Type</c> shall be <c>ssl</c> and that exported IOR:s will contain the IIOP via SSL port defined by this configuration - parameter.</item> + parameter.</item> </list> <p>If it is not possible to add a listener based on the supplied interface and port, the error message is one of the ones described in <c>inet</c> diff --git a/lib/orber/src/Makefile b/lib/orber/src/Makefile index d96350f4d5..398e481138 100644 --- a/lib/orber/src/Makefile +++ b/lib/orber/src/Makefile @@ -194,8 +194,7 @@ ERL_IDL_FLAGS += -pa $(ERL_TOP)/lib/orber/ebin ERL_COMPILE_FLAGS += $(ERL_IDL_FLAGS) \ -I$(ERL_TOP)/lib/orber/include \ +'{parse_transform,sys_pre_attributes}' \ - +'{attribute,insert,app_vsn,"orber_$(ORBER_VSN)"}' \ - -D'ORBVSN="$(ORBER_VSN)"' + +'{attribute,insert,app_vsn,"orber_$(ORBER_VSN)"}' ASN_FLAGS = -bber +der +compact_bit_string +nowarn_unused_record diff --git a/lib/orber/src/corba.erl b/lib/orber/src/corba.erl index 989e84f581..586a02d540 100644 --- a/lib/orber/src/corba.erl +++ b/lib/orber/src/corba.erl @@ -311,19 +311,18 @@ resolve_initial_references_remote(_ObjectId, [], _Ctx) -> raise(#'BAD_PARAM'{completion_status=?COMPLETED_NO}); resolve_initial_references_remote(ObjectId, [RemoteModifier| Rest], Ctx) when is_list(RemoteModifier) -> - case lists:prefix("iiop://", RemoteModifier) of - true -> - [_, Host, Port] = string:tokens(RemoteModifier, ":/"), + case parse_remote_modifier(RemoteModifier) of + {error, _} -> + resolve_initial_references_remote(ObjectId, Rest, Ctx); + {ok, Host, Port} -> IOR = iop_ior:create_external(orber:giop_version(), "", - Host, list_to_integer(Port), "INIT"), + Host, list_to_integer(Port), "INIT"), %% We know it's an external referens. Hence, no need to check. {_, Key} = iop_ior:get_key(IOR), orber_iiop:request(Key, 'get', [ObjectId], {{'tk_objref', 12, "object"}, [{'tk_string', 0}], - []}, 'true', infinity, IOR, Ctx); - false -> - resolve_initial_references_remote(ObjectId, Rest, Ctx) + []}, 'true', infinity, IOR, Ctx) end. list_initial_services_remote(Address) -> @@ -332,24 +331,44 @@ list_initial_services_remote(Address) -> list_initial_services_remote([], _Ctx) -> raise(#'BAD_PARAM'{completion_status=?COMPLETED_NO}); list_initial_services_remote([RemoteModifier| Rest], Ctx) when is_list(RemoteModifier) -> - case lists:prefix("iiop://", RemoteModifier) of - true -> - [_, Host, Port] = string:tokens(RemoteModifier, ":/"), + case parse_remote_modifier(RemoteModifier) of + {error, _} -> + resolve_initial_references_remote(Rest, Ctx); + {ok, Host, Port} -> IOR = iop_ior:create_external(orber:giop_version(), "", Host, list_to_integer(Port), "INIT"), %% We know it's an external referens. Hence, no need to check. {_, Key} = iop_ior:get_key(IOR), orber_iiop:request(Key, 'list', [], {{'tk_sequence', {'tk_string',0},0}, - [], []}, 'true', infinity, IOR, Ctx); - false -> - list_initial_services_remote(Rest, Ctx) + [], []}, 'true', infinity, IOR, Ctx) end; list_initial_services_remote(_, _) -> raise(#'BAD_PARAM'{completion_status=?COMPLETED_NO}). +parse_remote_modifier("iiop://" ++ Rest) -> + parse_host_version(Rest); +parse_remote_modifier(_RemoteModifier) -> + {error, not_supported}. + +parse_host_version("[" ++ Rest) -> + parse_ipv6(Rest, []); +parse_host_version(Rest) -> + parse_ipv4_or_dnsname(Rest, []). + + +parse_ipv4_or_dnsname([$: |Rest], Acc) -> + {ok, lists:reverse(Acc), Rest}; +parse_ipv4_or_dnsname([C |Rest], Acc) -> + parse_ipv4_or_dnsname(Rest, [C |Acc]). + +parse_ipv6("]:" ++ Rest, Acc) -> + {ok, lists:reverse(Acc), Rest}; +parse_ipv6([C |Rest], Acc) -> + parse_ipv6(Rest, [C |Acc]). + %%----------------------------------------------------------------- %% Objectreference convertions %%----------------------------------------------------------------- diff --git a/lib/orber/src/corba_request.erl b/lib/orber/src/corba_request.erl deleted file mode 100644 index c4226739a9..0000000000 --- a/lib/orber/src/corba_request.erl +++ /dev/null @@ -1,384 +0,0 @@ -%%-------------------------------------------------------------------- -%% -%% %CopyrightBegin% -%% -%% Copyright Ericsson AB 1998-2009. All Rights Reserved. -%% -%% The contents of this file are subject to the Erlang Public License, -%% Version 1.1, (the "License"); you may not use this file except in -%% compliance with the License. You should have received a copy of the -%% Erlang Public License along with this software. If not, it can be -%% retrieved online at http://www.erlang.org/. -%% -%% Software distributed under the License is distributed on an "AS IS" -%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See -%% the License for the specific language governing rights and limitations -%% under the License. -%% -%% %CopyrightEnd% -%% -%% -%%----------------------------------------------------------------- -%% File: corba_request.erl -%% Description: -%% This file contains an corba request server for Orber -%% -%%----------------------------------------------------------------- --module(corba_request). - --behaviour(gen_server). - --include_lib("orber/include/corba.hrl"). - -%%----------------------------------------------------------------- -%% External exports -%%----------------------------------------------------------------- --export([start/1, stop/0, stop_all/0, create/1, - create_schema/1]). - -%%----------------------------------------------------------------- -%% Internal exports -%%----------------------------------------------------------------- --export([init/1, terminate/2, install/2, handle_call/3, handle_info/2]). --export([handle_cast/2, dump/0, get_key_from_pid/1]). - -%%----------------------------------------------------------------- -%% Standard interface CORBA::Request -%%----------------------------------------------------------------- --export([add_arg/6, - invoke/2, - delete/1, - send/2, - get_response/2]). - -%%----------------------------------------------------------------- -%% Mnesia table definition -%%----------------------------------------------------------------- --record('corba_request', {reqid, ctx, operation, arg_list, result, req_flags, pid}). - - -%%----------------------------------------------------------------- -%% Macros -%%----------------------------------------------------------------- --define(dirty_query_context, true). - -%% This macro returns a read fun suitable for evaluation in a transaction --define(read_function(ReqId), - fun() -> - mnesia:dirty_read(ReqId) - end). - -%% This macro returns a write fun suitable for evaluation in a transaction --define(write_function(R), - fun() -> - mnesia:dirty_write(R) - end). - -%% This macro returns a write fun suitable for evaluation in a transaction --define(update(R), - fun() -> - mnesia:dirty_write(R) - end). - -%% This macro returns a delete fun suitable for evaluation in a transaction --define(delete_function(R), - fun() -> - mnesia:delete(R) - end). - --ifdef(dirty_query_context). --define(query_check(Q_res), Q_res). --else. --define(query_check(Q_res), {atomic, Q_res}). --endif. - - --define(CHECK_EXCEPTION(Res), case Res of - {'EXCEPTION', E} -> - corba:raise(E); - R -> - R - end). - -%%----------------------------------------------------------------- -%% Debugging function -%%----------------------------------------------------------------- -dump() -> - case catch mnesia:dirty_first('orber_request') of - {'EXIT', R} -> - io:format("Exited with ~p\n",[R]); - Key -> - dump_print(Key), - dump_loop(Key) - end. - -dump_loop(PreviousKey) -> - case catch mnesia:dirty_next('orber_request', PreviousKey) of - {'EXIT', R} -> - io:format("Exited with ~p\n",[R]); - '$end_of_table' -> - ok; - Key -> - dump_print(Key), - dump_loop(Key) - end. - -dump_print(Key) -> - case catch mnesia:dirty_read({'orber_request', Key}) of - {'EXIT', R} -> - io:format("Exited with ~p\n",[R]); - [X] -> - io:format("Req Id: ~p, op: ~p\n",[binary_to_term(X#orber_request.object_key), - X#orber_request.pid]); - _ -> - ok - end. - - -%%----------------------------------------------------------------- -%% External interface functions -%%----------------------------------------------------------------- -start(Opts) -> - gen_server:start_link({local, orber_requestserver}, orber_request, Opts, []). - -stop() -> - gen_server:call(orber_requestserver, stop, infinity). - - -stop_all() -> - Fun = fun() -> - mnesia:match_object({orber_request, '_', '_', '_', '_', '_', '_', '_'}) - end, - case catch mnesia:transaction(Fun) of - {atomic, Objects} -> - lists:foreach(fun({orber_request, _, _, _, _, _, _, _ }) -> - ok %gen_server:call(Pid, stop, infinity) - end, - Objects); - R -> - R - end. - -create() -> - ?CHECK_EXCEPTION(gen_server:call(orber_requestserver, - create, infinity)). - -create(Ctx, OP, Args, Flags) -> - ?CHECK_EXCEPTION(gen_server:call(orber_requestserver, - {create, Ctx, OP, Args, Flags}, infinity)). - -delete(ReqId) -> - ?CHECK_EXCEPTION(gen_server:call(orber_requestserver, - {delete, ReqId}, infinity)). - -%%------------------------------------------------------------ -%% Implementation of standard interface -%%------------------------------------------------------------ -add_arg(ReqId, ArgumentName, TC, Value, Len, ArgFlags) -> - Request = ets:lookup_element(orber_request, ReqId), - case Request of - [] -> - ok; - R -> - Args = Request#orber_request.arg_list, - NewArgs = lists:append(Args, []), - ets:insert(orber_request, NewArgs), - ok - end. - -invoke(ReqId, InvokeFlags) -> - ok. - - - -send(ReqId, InvokeFlags) -> - ok. - -get_response(ReqId, ResponseFlags) -> - [{_, Val}] = ets:lookup_element(orber_request, ReqId), - Val#'orber_request'.result. - -%%----------------------------------------------------------------- -%% Server functions -%%----------------------------------------------------------------- -init(Env) -> - case mnesia:wait_for_tables(['orber_request'], infinity) of - ok -> - process_flag(trap_exit, true), - {ok, []}; - StopReason -> - {stop, StopReason} - end. - -terminate(From, Reason) -> - ok. - - - -install(Timeout, Options) -> - %% check if there already exists a database. If not, create one. - %% DB_initialized = perhaps_create_schema(Nodelist), - %% check if mnesia is running. If not, start mnesia. - DB_started = perhaps_start_mnesia(), - - %% Do we have a complete set of IFR tables? If not, create them. - AllTabs = mnesia:system_info(tables), - - DB_Result = case lists:member(orber_request, AllTabs) of - true -> - case lists:member({local_content, true}, - Options) of - true-> - mnesia:add_table_copy(orber_request, - node(), - ram_copies); - _ -> - mnesia:create_table(orber_request, - [{attributes, - record_info(fields, - orber_objkeys)} - |Options]) - end; - _ -> - mnesia:create_table(orber_request, - [{attributes, - record_info(fields, - orber_objkeys)} - |Options]) - end, - - Wait = mnesia:wait_for_tables([orber_request], Timeout), - %% Check if any error has occured yet. If there are errors, return them. - if - DB_Result == {atomic, ok}, - Wait == ok -> - ok; - true -> - {error, {DB_Result, Wait}} - end. - -%%----------------------------------------------------------------- -%% Func: handle_call/3 -%% -%% Comment: -%% In objectkey gen_server all exceptions are tupples and corba:raise -%% may not be used. It is too time consuming to add catches in every -%% function before returning. On the client side there is a case which -%% maps every tupple on the format {'exception', E} to corba:raise(E). -%%----------------------------------------------------------------- -handle_call(stop, From, State) -> - {stop, normal, [], State}; -handle_call(create, From, State) -> - ReqId = term_to_binary({node(), now()}), - _F = ?write_function(#'corba_request'{reqid=ReqId}), - R = write_result(mnesia:transaction(_F)), - - ReqId - - ?query_check(Qres) = mnesia:dirty_read({orber_request, Objkey}), - case Qres of - [] -> - _F = ?write_function(#orber_requests{object_key=Objkey, pid=Pid}), - R = write_result(mnesia:transaction(_F)), - if - R == ok, pid(Pid) -> - link(Pid); - true -> - true - end, - {reply, R, State}; - X -> - {reply, {'EXCEPTION', #'INTERNAL'{completion_status=?COMPLETED_NO}}, - State} - end; -handle_call({delete, ReqId}, From, State) -> - ?query_check(Qres) = mnesia:dirty_read({orber_request, ReqId}), - case Qres of - [] -> - true; - [X] when pid(X#orber_request.pid) -> - unlink(X#orber_request.pid); - _ -> - true - end, - _F = ?delete_function({orber_request, ReqId}), - R = write_result(mnesia:transaction(_F)), - {reply, R, State}. - -handle_info({'EXIT', Pid, Reason}, State) when pid(Pid) -> - _MF = fun() -> - mnesia:match_object({orber_request, '_', '_', '_', '_', '_', '_', Pid}) - end, - ?query_check(Qres) = mnesia:ets(_MF), - case Qres of - [] -> - true; - X -> - remove_requests(X), - unlink(Pid); - _ -> - true - end, - {noreply, State}. - -%%----------------------------------------------------------------- -%% Internal Functions -%%----------------------------------------------------------------- -get_reqids_from_pid(Pid) -> - case mnesia:dirty_match_object({orber_request, '_', '_', '_', '_', '_', '_', Pid}) of - Keys -> - [Keys] - _ -> - corba:raise(#'OBJECT_NOT_EXIST'{completion_status=?COMPLETED_NO}) - end. - -remove_requests([]) -> - ok; -remove_requests([H|T]) -> - _F = ?delete_function({orber_request, H#orber_request.reqid}), - write_result(mnesia:transaction(_F)), - remove_requests(T). - -%%----------------------------------------------------------------- -%% Check a read transaction -query_result(?query_check(Qres)) -> - case Qres of - [Hres] -> - Hres#orber_request.pid; - [] -> - {'excpetion', #'OBJECT_NOT_EXIST'{completion_status=?COMPLETED_NO}}; - Other -> - {'excpetion', #'INTERNAL'{completion_status=?COMPLETED_NO}} - end. - -%%----------------------------------------------------------------- -%% Check a write transaction -write_result({atomic,ok}) -> ok; -write_result(Foo) -> - {'excpetion', #'INTERNAL'{completion_status=?COMPLETED_NO}}. - - -create_schema(Nodes) -> - case mnesia:system_info(use_dir) of - false -> - mnesia:create_schema(Nodes); - _ -> - ok - end. - -perhaps_start_mnesia() -> - case mnesia:system_info(is_running) of - no -> - mnesia:start(); - _ -> - ok - end. - - -%%------------------------------------------------------------ -%% Standard gen_server cast handle -%% -handle_cast(_, State) -> - {noreply, State}. - - diff --git a/lib/orber/src/orber_env.erl b/lib/orber/src/orber_env.erl index 1c8a90bc81..16dbb74253 100644 --- a/lib/orber/src/orber_env.erl +++ b/lib/orber/src/orber_env.erl @@ -204,9 +204,9 @@ info(IoDevice) -> _ -> lists:flatten( io_lib:format("======= Orber Execution Environment ======~n" - " *** Orber-~s is not running ***~n" + " *** Orber is not running ***~n" "==========================================~n", - [?ORBVSN])) + [])) end, case IoDevice of info_msg -> @@ -223,6 +223,7 @@ info(IoDevice) -> create_main_info() -> {Major, Minor} = giop_version(), + {orber, _, OrberVsn} = lists:keyfind(orber, 1, application:loaded_applications()), [io_lib:format("======= Orber Execution Environment ======~n" "Orber version.................: ~s~n" "Orber domain..................: ~s~n" @@ -257,7 +258,7 @@ create_main_info() -> "Debug Level...................: ~p~n" "orbInitRef....................: ~p~n" "orbDefaultInitRef.............: ~p~n", - [?ORBVSN, domain(), iiop_port(), nat_iiop_port(), host(), + [OrberVsn, domain(), iiop_port(), nat_iiop_port(), host(), nat_host(), ip_address_local(), orber:orber_nodes(), Major, Minor, iiop_timeout(), iiop_connection_timeout(), diff --git a/lib/orber/src/orber_iiop.hrl b/lib/orber/src/orber_iiop.hrl index 7a30af63e4..b2e970b30d 100644 --- a/lib/orber/src/orber_iiop.hrl +++ b/lib/orber/src/orber_iiop.hrl @@ -467,14 +467,14 @@ [{"iiop_version",?IIOP_VERSION }, {"host", {'tk_string', 0}}, {"port", 'tk_ushort'}, - {"object_key", {'tk_sequence', 'tk_octet', 0}} + {"object_key", {'tk_sequence', 'tk_octet', 0}}, {"components", ?IOP_TAGGEDCOMPONENT_SEQ}]}). -define(PROFILEBODY_1_2_TYPEDEF, {'tk_struct', ?SYSTEM_TYPE, 'IIOP_ProfileBody_1_1', [{"iiop_version",?IIOP_VERSION }, {"host", {'tk_string', 0}}, {"port", 'tk_ushort'}, - {"object_key", {'tk_sequence', 'tk_octet', 0}} + {"object_key", {'tk_sequence', 'tk_octet', 0}}, {"components", ?IOP_TAGGEDCOMPONENT_SEQ}]}). -define(SSLIOP_SSL, {'tk_struct', ?SYSTEM_TYPE, 'SSLIOP_SSL', diff --git a/lib/orber/src/orber_iiop_net.erl b/lib/orber/src/orber_iiop_net.erl index 33e02e3c04..1bfc6b7d58 100644 --- a/lib/orber/src/orber_iiop_net.erl +++ b/lib/orber/src/orber_iiop_net.erl @@ -157,7 +157,7 @@ terminate(_Reason, _State) -> ok. %%----------------------------------------------------------------- -%% Func: parse_options/2 +%% Func: get_options/2 %%----------------------------------------------------------------- get_options(normal, _Options) -> []; @@ -212,14 +212,21 @@ get_options(ssl, Options) -> %%----------------------------------------------------------------- parse_options([{port, Type, Port} | Rest], State) -> Options = get_options(Type, []), - Options2 = case orber_env:ip_address_variable_defined() of - false -> - Options; - Host -> - IPVersion = orber:ip_version(), - {ok, IP} = inet:getaddr(Host, IPVersion), - [{ip, IP} | Options] - end, + Family = orber_env:ip_version(), + IPFamilyOptions = + case Family of + inet -> [inet]; + inet6 -> [inet6, {ipv6_v6only, true}] + end, + Options2 = + case orber_env:ip_address_variable_defined() of + false -> + IPFamilyOptions ++ Options; + Host -> + {ok, IP} = inet:getaddr(Host, Family), + IPFamilyOptions ++ [{ip, IP} |Options] + end, + {ok, Listen, NewPort} = orber_socket:listen(Type, Port, Options2, true), {ok, Pid} = orber_iiop_socketsup:start_accept(Type, Listen, 0), link(Pid), @@ -283,28 +290,33 @@ handle_call({remove, Ref}, _From, State) -> {reply, ok, State} end; handle_call({add, IP, Type, Port, AllOptions}, _From, State) -> - Family = orber_env:ip_version(), + Family = orber_tb:keysearch(ip_family, AllOptions, orber_env:ip_version()), + IPFamilyOptions = + case Family of + inet -> [inet]; + inet6 -> [inet6, {ipv6_v6only, true}] + end, case inet:getaddr(IP, Family) of {ok, IPTuple} -> - try [{ip, IPTuple} |get_options(Type, AllOptions)] of - Options -> - Ref = make_ref(), - ProxyOptions = filter_options(AllOptions, []), - case orber_socket:listen(Type, Port, Options, false) of - {ok, Listen, NewPort} -> - {ok, Pid} = orber_iiop_socketsup:start_accept(Type, Listen, Ref, - ProxyOptions), - link(Pid), - ets:insert(?CONNECTION_DB, #listen{pid = Pid, - socket = Listen, - port = NewPort, - type = Type, ref = Ref, - options = Options, - proxy_options = ProxyOptions}), - {reply, {ok, Ref}, State}; - Error -> - {reply, Error, State} - end + try + Options = IPFamilyOptions ++ [{ip, IPTuple} |get_options(Type, AllOptions)], + Ref = make_ref(), + ProxyOptions = filter_options(AllOptions, []), + case orber_socket:listen(Type, Port, Options, false) of + {ok, Listen, NewPort} -> + {ok, Pid} = orber_iiop_socketsup:start_accept(Type, Listen, Ref, + ProxyOptions), + link(Pid), + ets:insert(?CONNECTION_DB, #listen{pid = Pid, + socket = Listen, + port = NewPort, + type = Type, ref = Ref, + options = Options, + proxy_options = ProxyOptions}), + {reply, {ok, Ref}, State}; + Error -> + {reply, Error, State} + end catch error:Reason -> {reply, {error, Reason}, State} diff --git a/lib/orber/src/orber_iiop_outproxy.erl b/lib/orber/src/orber_iiop_outproxy.erl index 8319d89088..4ba5b05995 100644 --- a/lib/orber/src/orber_iiop_outproxy.erl +++ b/lib/orber/src/orber_iiop_outproxy.erl @@ -112,7 +112,8 @@ stop(Pid) -> %%----------------------------------------------------------------- init({connect, Host, Port, SocketType, SocketOptions, Parent, Key, NewKey}) -> process_flag(trap_exit, true), - case catch orber_socket:connect(SocketType, Host, Port, SocketOptions) of + case catch orber_socket:connect(SocketType, Host, Port, + get_ip_family_opts(Host) ++ SocketOptions) of {'EXCEPTION', _E} -> ignore; %% We used to reply the below but since this would generate a CRASH REPORT @@ -507,3 +508,38 @@ clear_queue(Proxy, RequestId, MRef) -> end end. +get_ip_family_opts(Host) -> + case inet:parse_address(Host) of + {ok, {_,_,_,_}} -> + [inet]; + {ok, {_,_,_,_,_,_,_,_}} -> + [inet6]; + {error, einval} -> + check_family_for_name(Host, orber_env:ip_version()) + end. + +check_family_for_name(Host, inet) -> + case inet:getaddr(Host, inet) of + {ok, _Address} -> + [inet]; + {error, _} -> + case inet:getaddrs(Host, inet6) of + {ok, _Address} -> + [inet6]; + {error, _} -> + [inet] + end + end; +check_family_for_name(Host, inet6) -> + case inet:getaddr(Host, inet6) of + {ok, _Address} -> + [inet6]; + {error, _} -> + case inet:getaddr(Host, inet) of + {ok, _Address} -> + [inet]; + {error, _} -> + [inet6] + end + end. + diff --git a/lib/orber/src/orber_socket.erl b/lib/orber/src/orber_socket.erl index 07a0e09ccc..c8d2f0636b 100644 --- a/lib/orber/src/orber_socket.erl +++ b/lib/orber/src/orber_socket.erl @@ -205,29 +205,27 @@ listen(normal, Port, Options, Exception) -> MaxSize -> [{packet_size, MaxSize}|Options2] end, - case catch gen_tcp:listen(Port, [binary, {packet,cdr}, {keepalive, Keepalive}, + Options4 = [binary, {packet,cdr}, {keepalive, Keepalive}, {reuseaddr,true}, {backlog, Backlog} | - Options3]) of + Options3], + + case catch gen_tcp:listen(Port, Options4) of {ok, ListenSocket} -> {ok, ListenSocket, check_port(Port, normal, ListenSocket)}; {error, Reason} when Exception == false -> {error, Reason}; {error, eaddrinuse} -> - AllOpts = [binary, {packet,cdr}, - {reuseaddr,true} | Options3], orber:dbg("[~p] orber_socket:listen(normal, ~p, ~p);~n" "Looks like the listen port is already in use.~n" "Check if another Orber is started~n" "on the same node and uses the same listen port (iiop_port). But it may also~n" "be used by any other application; confirm with 'netstat'.", - [?LINE, Port, AllOpts], ?DEBUG_LEVEL), + [?LINE, Port, Options4], ?DEBUG_LEVEL), corba:raise(#'COMM_FAILURE'{completion_status=?COMPLETED_NO}); Error -> - AllOpts = [binary, {packet,cdr}, - {reuseaddr,true} | Options3], orber:dbg("[~p] orber_socket:listen(normal, ~p, ~p);~n" "Failed with reason: ~p", - [?LINE, Port, AllOpts, Error], ?DEBUG_LEVEL), + [?LINE, Port, Options4, Error], ?DEBUG_LEVEL), corba:raise(#'COMM_FAILURE'{completion_status=?COMPLETED_NO}) end; listen(ssl, Port, Options, Exception) -> @@ -252,26 +250,24 @@ listen(ssl, Port, Options, Exception) -> true -> Options3 end, - case catch ssl:listen(Port, [binary, {packet,cdr}, - {backlog, Backlog} | Options4]) of + Options5 = [binary, {packet,cdr}, {backlog, Backlog} | Options4], + case catch ssl:listen(Port, Options5) of {ok, ListenSocket} -> {ok, ListenSocket, check_port(Port, ssl, ListenSocket)}; {error, Reason} when Exception == false -> {error, Reason}; {error, eaddrinuse} -> - AllOpts = [binary, {packet,cdr} | Options4], orber:dbg("[~p] orber_socket:listen(ssl, ~p, ~p);~n" "Looks like the listen port is already in use. Check if~n" "another Orber is started on the same node and uses the~n" "same listen port (iiop_port). But it may also~n" "be used by any other application; confirm with 'netstat'.", - [?LINE, Port, AllOpts], ?DEBUG_LEVEL), + [?LINE, Port, Options5], ?DEBUG_LEVEL), corba:raise(#'COMM_FAILURE'{completion_status=?COMPLETED_NO}); Error -> - AllOpts = [binary, {packet,cdr} | Options4], orber:dbg("[~p] orber_socket:listen(ssl, ~p, ~p);~n" "Failed with reason: ~p", - [?LINE, Port, AllOpts, Error], ?DEBUG_LEVEL), + [?LINE, Port, Options5, Error], ?DEBUG_LEVEL), corba:raise(#'COMM_FAILURE'{completion_status=?COMPLETED_NO}) end. @@ -485,16 +481,14 @@ check_port(Port, _, _) -> %%----------------------------------------------------------------- %% Check Options. check_options(normal, Options, _Generation) -> - [orber:ip_version()|Options]; + Options; check_options(ssl, Options, Generation) -> - case orber:ip_version() of - inet when Generation > 2 -> + if + Generation > 2 -> [{ssl_imp, new}|Options]; - inet -> - [{ssl_imp, old}|Options]; - inet6 when Generation > 2 -> - [{ssl_imp, new}, inet6|Options]; - inet6 -> - [{ssl_imp, old}, inet6|Options] + true -> + [{ssl_imp, old}|Options] end. + + diff --git a/lib/orber/test/Makefile b/lib/orber/test/Makefile index 2f8777c773..50be14c24a 100644 --- a/lib/orber/test/Makefile +++ b/lib/orber/test/Makefile @@ -73,7 +73,8 @@ MODULES = \ orber_firewall_ipv6_in_SUITE \ orber_firewall_ipv4_out_SUITE \ orber_firewall_ipv6_out_SUITE \ - orber_nat_SUITE + orber_nat_SUITE \ + ip_v4v6_interop_SUITE GEN_MOD_ORBER = \ oe_orber_test \ diff --git a/lib/orber/test/ip_v4v6_interop_SUITE.erl b/lib/orber/test/ip_v4v6_interop_SUITE.erl new file mode 100644 index 0000000000..5eee5a29c2 --- /dev/null +++ b/lib/orber/test/ip_v4v6_interop_SUITE.erl @@ -0,0 +1,199 @@ +%%---------------------------------------------------------------------- +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 1999-2011. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%%---------------------------------------------------------------------- +%% File : ip_v4v6_interop_SUITE.erl +%% Description : +%% +%%---------------------------------------------------------------------- +-module(ip_v4v6_interop_SUITE). + +-compile(export_all). +%%---------------------------------------------------------------------- +%% External exports +%%---------------------------------------------------------------------- +-export([ + all/0, + init_per_suite/1, + end_per_suite/1, + init_per_testcase/2, + end_per_testcase/2, + groups/0, + init_per_group/2, + end_per_group/2 + ]). +%%----------------------------------------------------------------- +%% Internal exports +%%----------------------------------------------------------------- +-export([]). + +%%---------------------------------------------------------------------- +%% Include files +%%---------------------------------------------------------------------- +-include_lib("test_server/include/test_server.hrl"). +-include_lib("orber/include/corba.hrl"). +-include_lib("orber/COSS/CosNaming/CosNaming.hrl"). +-include_lib("orber/src/orber_iiop.hrl"). +-include_lib("orber/src/ifr_objects.hrl"). +-include("idl_output/orber_test_server.hrl"). +-include_lib("orber/COSS/CosNaming/CosNaming_NamingContextExt.hrl"). +-include_lib("orber/COSS/CosNaming/CosNaming_NamingContext.hrl"). + +%%---------------------------------------------------------------------- +%% Macros +%%---------------------------------------------------------------------- +-define(default_timeout, ?t:minutes(15)). + +-define(match(ExpectedRes,Expr), + fun() -> + AcTuAlReS = (catch (Expr)), + case AcTuAlReS of + ExpectedRes -> + io:format("------ CORRECT RESULT ------~n~p~n", + [AcTuAlReS]), + AcTuAlReS; + _ -> + io:format("###### ERROR ERROR ######~nRESULT: ~p~n", + [AcTuAlReS]), + ?line exit(AcTuAlReS) + end + end()). +%%---------------------------------------------------------------------- +%% Records +%%---------------------------------------------------------------------- + +%%====================================================================== +%% Initialization functions. +%%====================================================================== + +init_per_testcase(_Case, Config) -> + %% Starting dual configured ORB + orber:jump_start([{iiop_port, 10001}, {flags, 16#1000}]), + orber:info(), + Dog=test_server:timetrap(?default_timeout), + [{watchdog, Dog}|Config]. + + +end_per_testcase(_Case, Config) -> + orber:jump_stop(), + Dog = ?config(watchdog, Config), + test_server:timetrap_cancel(Dog), + ok. + +init_per_suite(Config) -> + Config. + +end_per_suite(Config) -> + Config. + +%%==================================================================== +%% SUITE specification +%%==================================================================== +all() -> + [ + dual_ipv4v6 + ]. + +suite() -> [{ct_hooks,[ts_install_cth]}]. + + +groups() -> + []. + +init_per_group(_GroupName, Config) -> + Config. + +end_per_group(_GroupName, Config) -> + Config. + +%%==================================================================== +%% Test Cases +%%==================================================================== +dual_ipv4v6(doc) -> + ["ORB configured for supporting both IPv4 and IPv6"]; +dual_ipv4v6(_Config) -> + + %% Starting slave node with ipv4 configured ORB + {ok, Ipv4Node, _Ipv4Host} = + ?match({ok,_,_}, orber_test_lib:js_node([{iiop_port, 4001}])), + Ipv4NS = orber_test_lib:remote_apply(Ipv4Node, corba, resolve_initial_references, ["NameService"]), + + %% Starting slave node with ipv6 configured ORB + {ok, Ipv6Node, _Ipv6Host} = + ?match({ok,_,_}, orber_test_lib:js_node([{iiop_port, 6001}, {flags, 16#0100}])), + Ipv6NS = orber_test_lib:remote_apply(Ipv6Node, corba, resolve_initial_references, ["NameService"]), + + %% Add the ipv6 interface in the dual configured ORB + ?match({ok, _}, orber:add_listen_interface("::1", normal, + [{ip_family, inet6}, {iiop_port, 10002}])), + DualNS = corba:resolve_initial_references("NameService"), + + %% Bind IPv4 NameServer to a name in the dual stack orbs NameServer + NSDual4 = orber_test_lib:remote_apply(Ipv4Node, corba, resolve_initial_references_remote, + ["NameService", ["iiop://127.0.0.1:10001"]]), + ?match(ok, orber_test_lib:remote_apply(Ipv4Node, 'CosNaming_NamingContext', bind, + [NSDual4, lname:new(["ns4"]), Ipv4NS])), + 'CosNaming_NamingContext':resolve(DualNS, lname:new(["ns4"])), + + %% Bind IPv6 NameServer to a name in the dual stack orbs NameServer + NSDual6 = orber_test_lib:remote_apply(Ipv6Node, corba, resolve_initial_references_remote, + ["NameService", ["iiop://[::1]:10002"]]), + ?match(ok, orber_test_lib:remote_apply(Ipv6Node, 'CosNaming_NamingContext', bind, + [NSDual6, lname:new(["ns6"]), Ipv6NS])), + 'CosNaming_NamingContext':resolve(DualNS, lname:new(["ns6"])), + + %% IPv4: Fetch IPv6 NS reference from dual stack orber and register own NameServer in that + Ipv4NSO = orber_test_lib:remote_apply(Ipv4Node, 'CosNaming_NamingContext', resolve, + [NSDual4, lname:new(["ns6"])]), + ?match(ok, orber_test_lib:remote_apply(Ipv4Node, 'CosNaming_NamingContext', bind, + [Ipv4NSO, lname:new(["nso"]), Ipv4NS])), + + %% IPv6: Fetch IPv4 NS reference from dual stack orber and register own NameServer in that + Ipv6NSO = orber_test_lib:remote_apply(Ipv6Node, 'CosNaming_NamingContext', resolve, + [NSDual6, lname:new(["ns4"])]), + ?match(ok, orber_test_lib:remote_apply(Ipv6Node, 'CosNaming_NamingContext', bind, + [Ipv6NSO, lname:new(["nso"]), Ipv6NS])), + + + %% IPv4: Fetch own NS reference from IPv6 NameServer and add a context then check that everything went well + Ipv4NSFromIpv6 = orber_test_lib:remote_apply(Ipv6Node, 'CosNaming_NamingContext', resolve, + [Ipv6NS, lname:new(["nso"])]), + _Ipv4NC = orber_test_lib:remote_apply(Ipv4Node, 'CosNaming_NamingContext', bind_new_context, + [Ipv4NSFromIpv6, lname:new(["test_context4"])]), + + %% IPv6: Fetch own NS reference from IPv4 NameServer and add a context then check that everything went well + Ipv6NSFromIpv4 = orber_test_lib:remote_apply(Ipv4Node, 'CosNaming_NamingContext', resolve, + [Ipv4NS, lname:new(["nso"])]), + _Ipv6NC = orber_test_lib:remote_apply(Ipv6Node, 'CosNaming_NamingContext', bind_new_context, + [Ipv6NSFromIpv4, lname:new(["test_context6"])]), + + %% Check that all the names are register correctly + {ok,DualNames,_} = 'CosNaming_NamingContext':list(DualNS, 100), + {ok,Ipv4Names,_} = orber_test_lib:remote_apply(Ipv4Node, 'CosNaming_NamingContext', list, [Ipv4NS, 100]), + {ok,Ipv6Names,_} = orber_test_lib:remote_apply(Ipv6Node, 'CosNaming_NamingContext', list, [Ipv6NS, 100]), + + io:format("\nNames in Dual NS: ~p\n", [DualNames]), + ?match(2, length(DualNames)), + io:format("\nNames in IPv4 NS: ~p\n", [Ipv4Names]), + ?match(2, length(Ipv4Names)), + io:format("\nNames in IPv6 NS: ~p\n", [Ipv6Names]), + ?match(2, length(Ipv6Names)), + + ok. + diff --git a/lib/orber/test/multi_ORB_SUITE.erl b/lib/orber/test/multi_ORB_SUITE.erl index 41a309ff16..40d8846e0f 100644 --- a/lib/orber/test/multi_ORB_SUITE.erl +++ b/lib/orber/test/multi_ORB_SUITE.erl @@ -582,12 +582,12 @@ proxy_interface_ipv6_api2() -> IOR1 = ?match(#'IOP_IOR'{}, orber_test_lib:remote_apply(ClientNode, corba, string_to_object, - ["corbaloc::1.2@"++IP++":"++integer_to_list(ServerPort)++"/NameService"])), + ["corbaloc::1.2@["++IP++"]:"++integer_to_list(ServerPort)++"/NameService"])), ?match({'external', {IP, ServerPort, _ObjectKey, _Counter, _TP, _NewHD}}, orber_test_lib:remote_apply(ClientNode, iop_ior, get_key, [IOR1])), IOR2 = ?match(#'IOP_IOR'{}, orber_test_lib:remote_apply(ClientNode, corba, string_to_object, - ["corbaloc::1.2@"++Loopback++":"++integer_to_list(ServerPort)++"/NameService"])), + ["corbaloc::1.2@["++Loopback++"]:"++integer_to_list(ServerPort)++"/NameService"])), ?match({'external', {Loopback, ServerPort, _ObjectKey, _Counter, _TP, _NewHD}}, orber_test_lib:remote_apply(ClientNode, iop_ior, get_key, [IOR2])), ok. diff --git a/lib/orber/test/orber_firewall_ipv6_in_SUITE.erl b/lib/orber/test/orber_firewall_ipv6_in_SUITE.erl index 10827b6ef5..2853949a49 100644 --- a/lib/orber/test/orber_firewall_ipv6_in_SUITE.erl +++ b/lib/orber/test/orber_firewall_ipv6_in_SUITE.erl @@ -188,6 +188,7 @@ allow_port_range_api(_Config) -> ?ORB_ENV_USE_ACL_INCOMING)}, {iiop_acl, [{tcp_in, IP++"/128#5980/6000"}]}])), ServerPort = orber_test_lib:remote_apply(ServerNode, orber, iiop_port, []), + io:format("ServerNode: ~p\nServerHost: ~p\n", [ServerNode, ServerHost]), IOR = ?match({'IOP_IOR',_,_}, corba:string_to_object("corbaloc::1.2@"++ServerHost++":"++integer_to_list(ServerPort)++"/NameService")), @@ -243,34 +244,34 @@ check_address_api(doc) -> ["Test corbaloc strings"]; check_address_api(suite) -> []; check_address_api(_Config) -> ?match({[[iiop,{1,0},"0:0:0:0:0:FFFF:C02A:2A2A",2809]],"NameService"}, - orber_cosnaming_utils:addresses(":0:0:0:0:0:FFFF:C02A:2A2A/NameService")), + orber_cosnaming_utils:addresses(":[0:0:0:0:0:FFFF:C02A:2A2A]/NameService")), ?match({[[iiop,{1,0},"0:0:0:0:0:FFFF:C02A:2A2A",2809]],[]}, - orber_cosnaming_utils:addresses(":0:0:0:0:0:FFFF:C02A:2A2A")), + orber_cosnaming_utils:addresses(":[0:0:0:0:0:FFFF:C02A:2A2A]")), ?match({[[iiop,{1,2},"0:0:0:0:0:FFFF:C02A:2A2A",2809]],"NameService"}, - orber_cosnaming_utils:addresses(":1.2@0:0:0:0:0:FFFF:C02A:2A2A/NameService")), + orber_cosnaming_utils:addresses(":1.2@[0:0:0:0:0:FFFF:C02A:2A2A]/NameService")), ?match({[[iiop,{1,0},"0:0:0:0:0:FFFF:C02A:2A2A",4001]],"NameService"}, - orber_cosnaming_utils:addresses(":0:0:0:0:0:FFFF:C02A:2A2A:4001/NameService")), + orber_cosnaming_utils:addresses(":[0:0:0:0:0:FFFF:C02A:2A2A]:4001/NameService")), ?match({[[iiop,{1,1},"0:0:0:0:0:FFFF:C02A:2A2A",4001]],"NameService"}, - orber_cosnaming_utils:addresses(":1.1@0:0:0:0:0:FFFF:C02A:2A2A:4001/NameService")), + orber_cosnaming_utils:addresses(":1.1@[0:0:0:0:0:FFFF:C02A:2A2A]:4001/NameService")), ?match({[[iiop,{1,1},"0:0:0:0:0:FFFF:C02A:2A2A",4001]],[]}, - orber_cosnaming_utils:addresses(":1.1@0:0:0:0:0:FFFF:C02A:2A2A:4001")), + orber_cosnaming_utils:addresses(":1.1@[0:0:0:0:0:FFFF:C02A:2A2A]:4001")), ?match({[[iiop,{1,1},"0:0:0:0:0:FFFF:C02A:2A2A",4001]],[]}, - orber_cosnaming_utils:addresses("iiop:1.1@0:0:0:0:0:FFFF:C02A:2A2A:4001")), + orber_cosnaming_utils:addresses("iiop:1.1@[0:0:0:0:0:FFFF:C02A:2A2A]:4001")), ?match({[[iiop,{1,0},"0:0:0:0:0:FFFF:10.11.11.11",2809]],"NameService"}, - orber_cosnaming_utils:addresses(":0:0:0:0:0:FFFF:10.11.11.11/NameService")), + orber_cosnaming_utils:addresses(":[0:0:0:0:0:FFFF:10.11.11.11]/NameService")), ?match({[[iiop,{1,0},"0:0:0:0:0:FFFF:10.11.11.11",2809]],[]}, - orber_cosnaming_utils:addresses(":0:0:0:0:0:FFFF:10.11.11.11")), + orber_cosnaming_utils:addresses(":[0:0:0:0:0:FFFF:10.11.11.11]")), ?match({[[iiop,{1,2},"0:0:0:0:0:FFFF:10.11.11.11",2809]],"NameService"}, - orber_cosnaming_utils:addresses(":1.2@0:0:0:0:0:FFFF:10.11.11.11/NameService")), + orber_cosnaming_utils:addresses(":1.2@[0:0:0:0:0:FFFF:10.11.11.11]/NameService")), ?match({[[iiop,{1,0},"0:0:0:0:0:FFFF:10.11.11.11",4001]],"NameService"}, - orber_cosnaming_utils:addresses(":0:0:0:0:0:FFFF:10.11.11.11:4001/NameService")), + orber_cosnaming_utils:addresses(":[0:0:0:0:0:FFFF:10.11.11.11]:4001/NameService")), ?match({[[iiop,{1,1},"0:0:0:0:0:FFFF:10.11.11.11",4001]],"NameService"}, - orber_cosnaming_utils:addresses(":1.1@0:0:0:0:0:FFFF:10.11.11.11:4001/NameService")), + orber_cosnaming_utils:addresses(":1.1@[0:0:0:0:0:FFFF:10.11.11.11]:4001/NameService")), ?match({[[iiop,{1,1},"0:0:0:0:0:FFFF:10.11.11.11",4001]],[]}, - orber_cosnaming_utils:addresses(":1.1@0:0:0:0:0:FFFF:10.11.11.11:4001/")), + orber_cosnaming_utils:addresses(":1.1@[0:0:0:0:0:FFFF:10.11.11.11]:4001/")), ?match({[[iiop,{1,1},"0:0:0:0:0:FFFF:10.11.11.11",4001]],[]}, - orber_cosnaming_utils:addresses("iiop:1.1@0:0:0:0:0:FFFF:10.11.11.11:4001/")), + orber_cosnaming_utils:addresses("iiop:1.1@[0:0:0:0:0:FFFF:10.11.11.11]:4001/")), ?match({[[iiop,{1,1},"myhost",4001]],[]}, orber_cosnaming_utils:addresses("iiop:1.1@myhost:4001")), @@ -282,31 +283,31 @@ check_address_api(_Config) -> ?match({[[iiop,{1,1},"0:0:0:0:0:FFFF:10.11.11.11",4001], [iiop,{1,1},"0:0:0:0:0:FFFF:C02A:2A2A",4001]], "NameService"}, - orber_cosnaming_utils:addresses(":1.1@0:0:0:0:0:FFFF:10.11.11.11:4001,:1.1@0:0:0:0:0:FFFF:C02A:2A2A:4001/NameService")), + orber_cosnaming_utils:addresses(":1.1@[0:0:0:0:0:FFFF:10.11.11.11]:4001,:1.1@[0:0:0:0:0:FFFF:C02A:2A2A]:4001/NameService")), ?match({[[iiop,{1,1},"0:0:0:0:0:FFFF:10.11.11.11",4001], [iiop,{1,1},"0:0:0:0:0:FFFF:C02A:2A2A",4001]], []}, - orber_cosnaming_utils:addresses(":1.1@0:0:0:0:0:FFFF:10.11.11.11:4001,:1.1@0:0:0:0:0:FFFF:C02A:2A2A:4001")), + orber_cosnaming_utils:addresses(":1.1@[0:0:0:0:0:FFFF:10.11.11.11]:4001,:1.1@[0:0:0:0:0:FFFF:C02A:2A2A]:4001")), ?match({[[iiop,{1,0},"0:0:0:0:0:FFFF:10.11.11.11",4001], [iiop,{1,1},"0:0:0:0:0:FFFF:C02A:2A2A",4001]], "NameService"}, - orber_cosnaming_utils:addresses(":0:0:0:0:0:FFFF:10.11.11.11:4001,:1.1@0:0:0:0:0:FFFF:C02A:2A2A:4001/NameService")), + orber_cosnaming_utils:addresses(":[0:0:0:0:0:FFFF:10.11.11.11]:4001,:1.1@[0:0:0:0:0:FFFF:C02A:2A2A]:4001/NameService")), ?match({[[iiop,{1,1},"0:0:0:0:0:FFFF:10.11.11.11",4001], [iiop,{1,0},"0:0:0:0:0:FFFF:C02A:2A2A",4001]], "NameService"}, - orber_cosnaming_utils:addresses(":1.1@0:0:0:0:0:FFFF:10.11.11.11:4001,:0:0:0:0:0:FFFF:C02A:2A2A:4001/NameService")), + orber_cosnaming_utils:addresses(":1.1@[0:0:0:0:0:FFFF:10.11.11.11]:4001,:[0:0:0:0:0:FFFF:C02A:2A2A]:4001/NameService")), ?match({[[iiop,{1,1},"0:0:0:0:0:FFFF:10.11.11.11",2809], [iiop,{1,1},"0:0:0:0:0:FFFF:C02A:2A2A",4001]], "NameService"}, - orber_cosnaming_utils:addresses(":1.1@0:0:0:0:0:FFFF:10.11.11.11,:1.1@0:0:0:0:0:FFFF:C02A:2A2A:4001/NameService")), + orber_cosnaming_utils:addresses(":1.1@[0:0:0:0:0:FFFF:10.11.11.11],:1.1@[0:0:0:0:0:FFFF:C02A:2A2A]:4001/NameService")), ?match({[[iiop,{1,1},"0:0:0:0:0:FFFF:10.11.11.11",4001], [iiop,{1,1},"0:0:0:0:0:FFFF:C02A:2A2A",2809]], "NameService"}, - orber_cosnaming_utils:addresses(":1.1@0:0:0:0:0:FFFF:10.11.11.11:4001,:1.1@0:0:0:0:0:FFFF:C02A:2A2A/NameService")), + orber_cosnaming_utils:addresses(":1.1@[0:0:0:0:0:FFFF:10.11.11.11]:4001,:1.1@[0:0:0:0:0:FFFF:C02A:2A2A]/NameService")), ?match({[[iiop,{1,0},"0:0:0:0:0:FFFF:10.11.11.11",2809], [iiop,{1,0},"0:0:0:0:0:FFFF:C02A:2A2A",2809]], "NameService"}, - orber_cosnaming_utils:addresses(":0:0:0:0:0:FFFF:10.11.11.11,:0:0:0:0:0:FFFF:C02A:2A2A/NameService")), + orber_cosnaming_utils:addresses(":[0:0:0:0:0:FFFF:10.11.11.11],:[0:0:0:0:0:FFFF:C02A:2A2A]/NameService")), ?match({[[iiop,{1,0},"0:0:0:0:0:FFFF:10.11.11.11",2809], [iiop,{1,0},"0:0:0:0:0:FFFF:C02A:2A2A",2809]], []}, - orber_cosnaming_utils:addresses(":0:0:0:0:0:FFFF:10.11.11.11,:0:0:0:0:0:FFFF:C02A:2A2A/")), + orber_cosnaming_utils:addresses(":[0:0:0:0:0:FFFF:10.11.11.11],:[0:0:0:0:0:FFFF:C02A:2A2A]/")), ?match({[[iiop,{1,0},"0:0:0:0:0:FFFF:10.11.11.11",2809], [iiop,{1,0},"0:0:0:0:0:FFFF:C02A:2A2A",2809]], []}, - orber_cosnaming_utils:addresses("iiop:0:0:0:0:0:FFFF:10.11.11.11,:0:0:0:0:0:FFFF:C02A:2A2A/")), + orber_cosnaming_utils:addresses("iiop:[0:0:0:0:0:FFFF:10.11.11.11],:[0:0:0:0:0:FFFF:C02A:2A2A]/")), [IP] = ?match([_], orber:host()), {ok, ServerNode, _ServerHost} = @@ -315,7 +316,7 @@ check_address_api(_Config) -> {iiop_acl, [{tcp_in, IP++"/128"}]}])), ServerPort = orber_test_lib:remote_apply(ServerNode, orber, iiop_port, []), ?match({'IOP_IOR',_,_}, - corba:string_to_object("corbaloc::1.2@"++IP++":"++integer_to_list(ServerPort)++"/NameService")), + corba:string_to_object("corbaloc::1.2@["++IP++"]:"++integer_to_list(ServerPort)++"/NameService")), % ?line catch orber_test_lib:destroy_node(ServerNode, timeout), ok. diff --git a/lib/orber/test/orber_test_lib.erl b/lib/orber/test/orber_test_lib.erl index 6824d25aef..46ed26f210 100644 --- a/lib/orber/test/orber_test_lib.erl +++ b/lib/orber/test/orber_test_lib.erl @@ -166,7 +166,7 @@ get_host(Family) -> {6, _, _} when Family == inet -> "127.0.0.1"; {6, _, _} -> - "0:0:0:0:0:FFFF:7F00:0001"; + "0:0:0:0:0:0:0:0001"; _ -> [IP] = ?match([_], orber:host()), IP @@ -192,16 +192,16 @@ get_loopback_interface(Family) -> {6, _, _} when Family == inet -> "127.0.0.2"; {6, _, _} -> - "0:0:0:0:0:FFFF:7F00:0002"; + "0:0:0:0:0:0:0:0002"; _ when Family == inet -> "127.0.0.1"; _ -> - "0:0:0:0:0:FFFF:7F00:0001" + "0:0:0:0:0:0:0:0001" end; _ when Family == inet -> "127.0.0.1"; _ -> - "0:0:0:0:0:FFFF:7F00:0001" + "0:0:0:0:0:0:0:0001" end. %%------------------------------------------------------------ diff --git a/lib/orber/vsn.mk b/lib/orber/vsn.mk index 3ea64b1ff6..13bdf55c07 100644 --- a/lib/orber/vsn.mk +++ b/lib/orber/vsn.mk @@ -1,2 +1,2 @@ -ORBER_VSN = 3.6.27 +ORBER_VSN = 3.7 diff --git a/lib/os_mon/doc/src/disksup.xml b/lib/os_mon/doc/src/disksup.xml index dbcfd65095..0e76178edb 100644 --- a/lib/os_mon/doc/src/disksup.xml +++ b/lib/os_mon/doc/src/disksup.xml @@ -73,6 +73,17 @@ much disk can be utilized before the <c>disk_almost_full</c> alarm is set. The default is 0.80 (80%).</p> </item> + <tag><c>disksup_posix_only = bool()</c></tag> + <item> + <p>Specifies whether the <c>disksup</c> helper process should only + use POSIX conformant commands (<c>true</c>) or not. The default is + <c>false</c>. Setting this parameter to <c>true</c> can be + necessary on embedded systems with stripped-down versions + of Unix tools like <c>df</c>. The returned disk data and alarms + can be different when using this option.</p> + <p>The parameter is ignored on platforms that are known to not be + posix compatible (Windows and SunOS).</p> + </item> </taglist> <p>See <seealso marker="kernel:config">config(4)</seealso> for information about how to change the value of configuration diff --git a/lib/os_mon/doc/src/notes.xml b/lib/os_mon/doc/src/notes.xml index 3a7d7793d4..6bc0cf7d43 100644 --- a/lib/os_mon/doc/src/notes.xml +++ b/lib/os_mon/doc/src/notes.xml @@ -30,6 +30,65 @@ </header> <p>This document describes the changes made to the OS_Mon application.</p> +<section><title>Os_Mon 2.3</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Adds a new application parameter 'disksup_posix_only', to + make diskup use only options defined in the POSIX + standard.</p> + <p> + Own Id: OTP-12053</p> + </item> + </list> + </section> + +</section> + +<section><title>Os_Mon 2.2.15</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Application upgrade (appup) files are corrected for the + following applications: </p> + <p> + <c>asn1, common_test, compiler, crypto, debugger, + dialyzer, edoc, eldap, erl_docgen, et, eunit, gs, hipe, + inets, observer, odbc, os_mon, otp_mibs, parsetools, + percept, public_key, reltool, runtime_tools, ssh, + syntax_tools, test_server, tools, typer, webtool, wx, + xmerl</c></p> + <p> + A new test utility for testing appup files is added to + test_server. This is now used by most applications in + OTP.</p> + <p> + (Thanks to Tobias Schlager)</p> + <p> + Own Id: OTP-11744</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Calls to erlang:open_port/2 with 'spawn' are updated to + handle space in the command path.</p> + <p> + Own Id: OTP-10842</p> + </item> + </list> + </section> + +</section> + <section><title>Os_Mon 2.2.14</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/os_mon/src/cpu_sup.erl b/lib/os_mon/src/cpu_sup.erl index 34251178ee..1f088ecbde 100644 --- a/lib/os_mon/src/cpu_sup.erl +++ b/lib/os_mon/src/cpu_sup.erl @@ -543,7 +543,8 @@ measurement_server_init() -> Server = case OS of {unix, Flavor} when Flavor==sunos; Flavor==linux -> - port_server_start(); + {ok, Pid} = port_server_start_link(), + Pid; {unix, Flavor} when Flavor==darwin; Flavor==freebsd; Flavor==dragonfly; @@ -588,8 +589,9 @@ measurement_server_loop(State) -> Error -> Pid ! {error, Error} end, measurement_server_loop(State); - {'EXIT', Pid, _n} when State#internal.port == Pid -> - measurement_server_loop(State#internal{port = port_server_start()}); + {'EXIT', OldPid, _n} when State#internal.port == OldPid -> + {ok, NewPid} = port_server_start_link(), + measurement_server_loop(State#internal{port = NewPid}); _Other -> measurement_server_loop(State) end. @@ -605,12 +607,12 @@ port_server_call(Pid, Command) -> {Pid, {error, Reason}} -> {error, Reason} end. -port_server_start() -> +port_server_start_link() -> Timeout = 6000, Pid = spawn_link(fun() -> port_server_init(Timeout) end), Pid ! {self(), ?ping}, receive - {Pid, {data,4711}} -> Pid; + {Pid, {data,4711}} -> {ok, Pid}; {error,Reason} -> {error, Reason} after Timeout -> {error, timeout} diff --git a/lib/os_mon/src/disksup.erl b/lib/os_mon/src/disksup.erl index 278da26a20..af5bcc6fe8 100644 --- a/lib/os_mon/src/disksup.erl +++ b/lib/os_mon/src/disksup.erl @@ -81,10 +81,12 @@ param_type(disk_space_check_interval, Val) when is_integer(Val), param_type(disk_almost_full_threshold, Val) when is_number(Val), 0=<Val, Val=<1 -> true; +param_type(disksup_posix_only, Val) when Val==true; Val==false -> true; param_type(_Param, _Val) -> false. param_default(disk_space_check_interval) -> 30; -param_default(disk_almost_full_threshold) -> 0.80. +param_default(disk_almost_full_threshold) -> 0.80; +param_default(disksup_posix_only) -> false. %%---------------------------------------------------------------------- %% gen_server callbacks @@ -94,7 +96,8 @@ init([]) -> process_flag(trap_exit, true), process_flag(priority, low), - OS = get_os(), + PosixOnly = os_mon:get_env(disksup, disksup_posix_only), + OS = get_os(PosixOnly), Port = case OS of {unix, Flavor} when Flavor==sunos4; Flavor==solaris; @@ -102,6 +105,7 @@ init([]) -> Flavor==dragonfly; Flavor==darwin; Flavor==linux; + Flavor==posix; Flavor==openbsd; Flavor==netbsd; Flavor==irix64; @@ -205,14 +209,16 @@ format_status(_Opt, [_PDict, #state{os = OS, threshold = Threshold, %% Internal functions %%---------------------------------------------------------------------- -get_os() -> +get_os(PosixOnly) -> case os:type() of {unix, sunos} -> - case os:version() of + case os:version() of {5,_,_} -> {unix, solaris}; {4,_,_} -> {unix, sunos4}; V -> exit({unknown_os_version, V}) - end; + end; + {unix, _} when PosixOnly -> + {unix, posix}; {unix, irix64} -> {unix, irix}; OS -> OS @@ -259,6 +265,9 @@ check_disk_space({unix, irix}, Port, Threshold) -> check_disk_space({unix, linux}, Port, Threshold) -> Result = my_cmd("/bin/df -lk", Port), check_disks_solaris(skip_to_eol(Result), Threshold); +check_disk_space({unix, posix}, Port, Threshold) -> + Result = my_cmd("df -k -P", Port), + check_disks_solaris(skip_to_eol(Result), Threshold); check_disk_space({unix, dragonfly}, Port, Threshold) -> Result = my_cmd("/bin/df -k -t ufs,hammer", Port), check_disks_solaris(skip_to_eol(Result), Threshold); diff --git a/lib/os_mon/test/cpu_sup_SUITE.erl b/lib/os_mon/test/cpu_sup_SUITE.erl index e0382cb0c7..9f58e043db 100644 --- a/lib/os_mon/test/cpu_sup_SUITE.erl +++ b/lib/os_mon/test/cpu_sup_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2002-2011. All Rights Reserved. +%% Copyright Ericsson AB 2002-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -257,8 +257,8 @@ port(Config) when is_list(Config) -> terminate(suite) -> []; terminate(Config) when is_list(Config) -> - ?line ok = application:set_env(os_mon, start_cpu_sup, false), - ?line ok = supervisor:terminate_child(os_mon_sup, cpu_sup), + ok = application:set_env(os_mon, start_cpu_sup, false), + _ = supervisor:terminate_child(os_mon_sup, cpu_sup), ok. unavailable(suite) -> diff --git a/lib/os_mon/test/disksup_SUITE.erl b/lib/os_mon/test/disksup_SUITE.erl index 9c65d8b692..f9addd96cf 100644 --- a/lib/os_mon/test/disksup_SUITE.erl +++ b/lib/os_mon/test/disksup_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2011. All Rights Reserved. +%% Copyright Ericsson AB 1996-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -29,6 +29,7 @@ -export([port/1]). -export([terminate/1, unavailable/1, restart/1]). -export([otp_5910/1]). +-export([posix_only/1]). %% Default timetrap timeout (set in init_per_testcase) -define(default_timeout, ?t:minutes(1)). @@ -48,7 +49,8 @@ init_per_testcase(_Case, Config) -> Dog = ?t:timetrap(?default_timeout), [{watchdog,Dog} | Config]. -end_per_testcase(unavailable, Config) -> +end_per_testcase(TC, Config) when TC =:= unavailable; + TC =:= posix_only -> restart(Config), end_per_testcase(dummy, Config); end_per_testcase(_Case, Config) -> @@ -60,11 +62,10 @@ suite() -> [{ct_hooks,[ts_install_cth]}]. all() -> Bugs = [otp_5910], + Always = [api, config, alarm, port, posix_only, unavailable] ++ Bugs, case test_server:os_type() of - {unix, sunos} -> - [api, config, alarm, port, unavailable] ++ Bugs; - {unix, _OSname} -> [api, alarm] ++ Bugs; - {win32, _OSname} -> [api, alarm] ++ Bugs; + {unix, _OSname} -> Always; + {win32, _OSname} -> Always; _OS -> [unavailable] end. @@ -83,12 +84,7 @@ api(doc) -> ["Test of API functions"]; api(Config) when is_list(Config) -> %% get_disk_data() - [{Id,KByte,Capacity}|_] = get_disk_data(), - true = io_lib:printable_list(Id), - true = is_integer(KByte), - true = is_integer(Capacity), - true = Capacity>0, - true = KByte>0, + ok = check_get_disk_data(), %% get_check_interval() 1800000 = disksup:get_check_interval(), @@ -340,6 +336,7 @@ restart(suite) -> []; restart(Config) when is_list(Config) -> ok = application:set_env(os_mon, start_disksup, true), + ok = application:set_env(os_mon, disksup_posix_only, false), {ok, _Pid} = supervisor:restart_child(os_mon_sup, disksup), ok. @@ -350,15 +347,16 @@ otp_5910(doc) -> otp_5910(Config) when is_list(Config) -> %% Make sure disksup sets at least one alarm - Data = disksup:get_disk_data(), + Data = lists:sort(disksup:get_disk_data()), Threshold0 = disksup:get_almost_full_threshold(), Threshold = case over_threshold(Data, Threshold0) of - 0 -> - [{_Id,_Kbyte,Cap}|_] = Data, - ok = disksup:set_almost_full_threshold((Cap-1)/100), - Cap-1; - _N -> Threshold0 - end, + 0 -> + [{_Id,_Kbyte,Cap}|_] = Data, + io:format("Data ~p Threshold ~p ~n",[Data, Cap-1]), + ok = disksup:set_almost_full_threshold((Cap-1)/100), + Cap-1; + _N -> Threshold0 + end, ok = application:set_env(os_mon, disk_almost_full_threshold, Threshold/100), disksup ! timeout, % force a disk check Data2 = disksup:get_disk_data(), @@ -404,9 +402,28 @@ otp_5910(Config) when is_list(Config) -> ok = application:start(os_mon), ok. +posix_only(suite) -> []; +posix_only(doc) -> ["Test disksup_posix_only option"]; +posix_only(Config) when is_list(Config) -> + %% Set option and restart disksup + ok = application:set_env(os_mon, disksup_posix_only, true), + ok = supervisor:terminate_child(os_mon_sup, disksup), + {ok, _Child1} = supervisor:restart_child(os_mon_sup, disksup), + + ok = check_get_disk_data(). + dump_info() -> io:format("Status: ~p~n", [sys:get_status(disksup)]). +check_get_disk_data() -> + [{Id,KByte,Capacity}|_] = get_disk_data(), + true = io_lib:printable_list(Id), + true = is_integer(KByte), + true = is_integer(Capacity), + true = Capacity>0, + true = KByte>0, + ok. + % filter get_disk_data and remove entriew with zero capacity % "non-normal" filesystems report zero capacity % - Perhaps errorneous 'df -k -l'? diff --git a/lib/os_mon/vsn.mk b/lib/os_mon/vsn.mk index 74397c2bc6..f90cc306f0 100644 --- a/lib/os_mon/vsn.mk +++ b/lib/os_mon/vsn.mk @@ -1 +1 @@ -OS_MON_VSN = 2.2.15 +OS_MON_VSN = 2.3 diff --git a/lib/ose/doc/src/ose_intro.xml b/lib/ose/doc/src/ose_intro.xml index 0dd3ec409e..0ed470890b 100644 --- a/lib/ose/doc/src/ose_intro.xml +++ b/lib/ose/doc/src/ose_intro.xml @@ -65,7 +65,7 @@ erl /home/erlang --</code> <p> - The arguments are printed on seperate lines to make it possible to know + The arguments are printed on separate lines to make it possible to know what has to be quoted with ". Each line is one quotable unit. So taking the arguments above you can supply them to pm_create or just execute directly on the command line. For example:</p> @@ -75,10 +75,10 @@ pid: 0x110059 rtose@acp3400> pm_start 0x110059</code> <p> Also note that since we are running erl to figure out the arguments on a - seperate machine the paths have to be updated. In the example above + separate machine the paths have to be updated. In the example above <c>/usr/local/lib/erlang</c> was replaced by <c>/mst/erlang/</c>. The goal is to in future releases not have to do the special argument handling - but for now (17.0-rc2) you have to do it. + but for now (OTP 17.0) you have to do it. </p> <note> Because of a limitation in the way the OSE handles stdio when starting @@ -146,7 +146,7 @@ os:type(). It is possible to set the priorities you want for the OSE processes that thr emulator creates in the lmconf. An example of how to do it can be found in the default lmconf file in - $ERL_TOP/erts/emulator/sys/ose/default.lmconf. + $ERL_TOP/erts/emulator/sys/ose/beam.lmconf. </p> </section> diff --git a/lib/ose/vsn.mk b/lib/ose/vsn.mk index 78ffa4d496..ef754cf593 100644 --- a/lib/ose/vsn.mk +++ b/lib/ose/vsn.mk @@ -1 +1 @@ -OSE_VSN = 1.0 +OSE_VSN = 1.0.1 diff --git a/lib/otp_mibs/doc/src/notes.xml b/lib/otp_mibs/doc/src/notes.xml index 0c13fb2349..391c82b1c5 100644 --- a/lib/otp_mibs/doc/src/notes.xml +++ b/lib/otp_mibs/doc/src/notes.xml @@ -31,6 +31,55 @@ <p>This document describes the changes made to the OTP_Mibs application.</p> +<section><title>Otp_Mibs 1.0.9</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Add type based integer value truncation/reset.</p> + <p> + This fixes errors when querying e.g. the + erlNodeReductions, erlNodeInBytes and erlNodeOutBytes + objects in long-running Erlang/OTP systems.</p> + <p> + Update types of applicable MIB objects to 64bit based + types.</p> + <p> + Potential incompatibility: Type change of Counter32 to + Counter64 in OTP-MIB.mib</p> + <p> + (Thanks to Tobias Schlager)</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-11203</p> + </item> + <item> + <p> + Application upgrade (appup) files are corrected for the + following applications: </p> + <p> + <c>asn1, common_test, compiler, crypto, debugger, + dialyzer, edoc, eldap, erl_docgen, et, eunit, gs, hipe, + inets, observer, odbc, os_mon, otp_mibs, parsetools, + percept, public_key, reltool, runtime_tools, ssh, + syntax_tools, test_server, tools, typer, webtool, wx, + xmerl</c></p> + <p> + A new test utility for testing appup files is added to + test_server. This is now used by most applications in + OTP.</p> + <p> + (Thanks to Tobias Schlager)</p> + <p> + Own Id: OTP-11744</p> + </item> + </list> + </section> + +</section> + <section><title>Otp_Mibs 1.0.8</title> <section><title>Improvements and New Features</title> diff --git a/lib/parsetools/doc/src/notes.xml b/lib/parsetools/doc/src/notes.xml index 929daedf74..a8368740da 100644 --- a/lib/parsetools/doc/src/notes.xml +++ b/lib/parsetools/doc/src/notes.xml @@ -30,6 +30,42 @@ </header> <p>This document describes the changes made to the Parsetools application.</p> +<section><title>Parsetools 2.0.11</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Application upgrade (appup) files are corrected for the + following applications: </p> + <p> + <c>asn1, common_test, compiler, crypto, debugger, + dialyzer, edoc, eldap, erl_docgen, et, eunit, gs, hipe, + inets, observer, odbc, os_mon, otp_mibs, parsetools, + percept, public_key, reltool, runtime_tools, ssh, + syntax_tools, test_server, tools, typer, webtool, wx, + xmerl</c></p> + <p> + A new test utility for testing appup files is added to + test_server. This is now used by most applications in + OTP.</p> + <p> + (Thanks to Tobias Schlager)</p> + <p> + Own Id: OTP-11744</p> + </item> + <item> + <p> + A Yecc example has been updated in the documentation + (Thanks to Pierre Fenoll.)</p> + <p> + Own Id: OTP-11749</p> + </item> + </list> + </section> + +</section> + <section><title>Parsetools 2.0.10</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/percept/doc/src/notes.xml b/lib/percept/doc/src/notes.xml index 3cead186c4..bae999ed1a 100644 --- a/lib/percept/doc/src/notes.xml +++ b/lib/percept/doc/src/notes.xml @@ -32,6 +32,35 @@ </header> <p>This document describes the changes made to the Percept application.</p> +<section><title>Percept 0.8.9</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Application upgrade (appup) files are corrected for the + following applications: </p> + <p> + <c>asn1, common_test, compiler, crypto, debugger, + dialyzer, edoc, eldap, erl_docgen, et, eunit, gs, hipe, + inets, observer, odbc, os_mon, otp_mibs, parsetools, + percept, public_key, reltool, runtime_tools, ssh, + syntax_tools, test_server, tools, typer, webtool, wx, + xmerl</c></p> + <p> + A new test utility for testing appup files is added to + test_server. This is now used by most applications in + OTP.</p> + <p> + (Thanks to Tobias Schlager)</p> + <p> + Own Id: OTP-11744</p> + </item> + </list> + </section> + +</section> + <section><title>Percept 0.8.8.2</title> <section><title>Improvements and New Features</title> diff --git a/lib/percept/src/Makefile b/lib/percept/src/Makefile index 6bf0af9dc6..0282d6346a 100644 --- a/lib/percept/src/Makefile +++ b/lib/percept/src/Makefile @@ -50,6 +50,8 @@ MODULES= \ #HRL_FILES= ../include/ +INTERNAL_HRL_FILES= egd.hrl percept.hrl + ERL_FILES= $(MODULES:%=%.erl) TARGET_FILES= $(MODULES:%=$(EBIN)/%.$(EMULATOR)) $(APP_TARGET) $(APPUP_TARGET) @@ -95,6 +97,7 @@ include $(ERL_TOP)/make/otp_release_targets.mk release_spec: opt $(INSTALL_DIR) "$(RELSYSDIR)/src" $(INSTALL_DATA) $(ERL_FILES) "$(RELSYSDIR)/src" + $(INSTALL_DATA) $(INTERNAL_HRL_FILES) "$(RELSYSDIR)/src" # $(INSTALL_DIR) "$(RELSYSDIR)/include" # $(INSTALL_DATA) $(HRL_FILES) "$(RELSYSDIR)/include" $(INSTALL_DIR) "$(RELSYSDIR)/ebin" diff --git a/lib/public_key/asn1/OTP-PKIX.asn1 b/lib/public_key/asn1/OTP-PKIX.asn1 index 8d3c76adf5..37196bb9bf 100644 --- a/lib/public_key/asn1/OTP-PKIX.asn1 +++ b/lib/public_key/asn1/OTP-PKIX.asn1 @@ -452,23 +452,23 @@ SupportedPublicKeyAlgorithms PUBLIC-KEY-ALGORITHM-CLASS ::= { ecdsa-with-sha1 SIGNATURE-ALGORITHM-CLASS ::= { ID ecdsa-with-SHA1 - TYPE NULL } -- XXX Must be empty and not NULL + TYPE EcpkParameters } -- XXX Must be empty and not NULL ecdsa-with-sha224 SIGNATURE-ALGORITHM-CLASS ::= { ID ecdsa-with-SHA224 - TYPE NULL } -- XXX Must be empty and not NULL + TYPE EcpkParameters } -- XXX Must be empty and not NULL ecdsa-with-sha256 SIGNATURE-ALGORITHM-CLASS ::= { ID ecdsa-with-SHA256 - TYPE NULL } -- XXX Must be empty and not NULL + TYPE EcpkParameters } -- XXX Must be empty and not NULL ecdsa-with-sha384 SIGNATURE-ALGORITHM-CLASS ::= { ID ecdsa-with-SHA384 - TYPE NULL } -- XXX Must be empty and not NULL + TYPE EcpkParameters } -- XXX Must be empty and not NULL ecdsa-with-sha512 SIGNATURE-ALGORITHM-CLASS ::= { ID ecdsa-with-SHA512 - TYPE NULL } -- XXX Must be empty and not NULL + TYPE EcpkParameters } -- XXX Must be empty and not NULL FIELD-ID-CLASS ::= CLASS { &id OBJECT IDENTIFIER UNIQUE, diff --git a/lib/public_key/doc/src/cert_records.xml b/lib/public_key/doc/src/cert_records.xml index 79e5cb219d..b66c66bead 100644 --- a/lib/public_key/doc/src/cert_records.xml +++ b/lib/public_key/doc/src/cert_records.xml @@ -5,7 +5,7 @@ <header> <copyright> <year>2008</year> - <year>2013</year> + <year>2014</year> <holder>Ericsson AB, All Rights Reserved</holder> </copyright> <legalnotice> @@ -36,10 +36,11 @@ <p>This chapter briefly describes erlang records derived from ASN1 specifications used to handle <c> X509 certificates</c> and <c>CertificationRequest</c>. - The intent is to describe the data types and not to specify the meaning of each - component for this we refer you to <url + The intent is to describe the data types +and not to specify the semantics of each component. For information on the +semantics, please see <url href="http://www.ietf.org/rfc/rfc5280.txt">RFC 5280</url> and - <url href="http://www.rsa.com/rsalabs/node.asp?id=2124">PKCS-10</url>. + <url href="http://www.ietf.org/rfc/rfc5967.txt">PKCS-10</url>. </p> <p>Use the following include directive to get access to the @@ -79,7 +80,7 @@ <p><c> special_string() = {teletexString, string()} | {printableString, string()} | - {universalString, string()} | {utf8String, string()} | + {universalString, string()} | {utf8String, binary()} | {bmpString, string()} </c></p> diff --git a/lib/public_key/doc/src/notes.xml b/lib/public_key/doc/src/notes.xml index 1dce718ea3..fe4bf5ce2d 100644 --- a/lib/public_key/doc/src/notes.xml +++ b/lib/public_key/doc/src/notes.xml @@ -34,6 +34,86 @@ <file>notes.xml</file> </header> +<section><title>Public_Key 0.22.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Added missing encoding support for PBES2, and also + completed support for PBES1 that was incomplete.</p> + <p> + Own Id: OTP-11915</p> + </item> + </list> + </section> + +</section> + +<section><title>Public_Key 0.22</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fix incorrect dialyzer spec and types, also enhance + documentation. </p> + <p> + Thanks to Ayaz Tuncer.</p> + <p> + Own Id: OTP-11627</p> + </item> + <item> + <p> + Application upgrade (appup) files are corrected for the + following applications: </p> + <p> + <c>asn1, common_test, compiler, crypto, debugger, + dialyzer, edoc, eldap, erl_docgen, et, eunit, gs, hipe, + inets, observer, odbc, os_mon, otp_mibs, parsetools, + percept, public_key, reltool, runtime_tools, ssh, + syntax_tools, test_server, tools, typer, webtool, wx, + xmerl</c></p> + <p> + A new test utility for testing appup files is added to + test_server. This is now used by most applications in + OTP.</p> + <p> + (Thanks to Tobias Schlager)</p> + <p> + Own Id: OTP-11744</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Moved elliptic curve definition from the crypto + NIF/OpenSSL into Erlang code, adds the RFC-5639 brainpool + curves and makes TLS use them (RFC-7027).</p> + <p> + Thanks to Andreas Schultz</p> + <p> + Own Id: OTP-11578</p> + </item> + <item> + <p> + Handle v1 CRLs, with no extensions and fixes issues with + IDP (Issuing Distribution Point) comparison during CRL + validation. </p> + <p> + Thanks to Andrew Thompson</p> + <p> + Own Id: OTP-11761</p> + </item> + </list> + </section> + +</section> + <section><title>Public_Key 0.21</title> <section><title>Improvements and New Features</title> diff --git a/lib/public_key/doc/src/part.xml b/lib/public_key/doc/src/part.xml index 51e628aa90..73146c8e2a 100644 --- a/lib/public_key/doc/src/part.xml +++ b/lib/public_key/doc/src/part.xml @@ -5,7 +5,7 @@ <header> <copyright> <year>2008</year> - <year>2013</year> + <year>2014</year> <holder>Ericsson AB, All Rights Reserved</holder> </copyright> <legalnotice> @@ -34,7 +34,7 @@ <p> This application provides an API to public key infrastructure from <url href="http://www.ietf.org/rfc/rfc5280.txt">RFC 5280</url> (X.509 certificates) and public key formats defined by - the <url href="http://www.rsa.com/rsalabs/node.asp?id=2124"> + the <url href="http://en.wikipedia.org/wiki/PKCS"> PKCS-standard</url></p> </description> <xi:include href="introduction.xml"/> diff --git a/lib/public_key/doc/src/public_key.xml b/lib/public_key/doc/src/public_key.xml index fc3479cb64..e3473f80d7 100644 --- a/lib/public_key/doc/src/public_key.xml +++ b/lib/public_key/doc/src/public_key.xml @@ -5,7 +5,7 @@ <header> <copyright> <year>2008</year> - <year>2013</year> + <year>2014</year> <holder>Ericsson AB, All Rights Reserved</holder> </copyright> <legalnotice> @@ -48,12 +48,12 @@ <item>Supports <url href="http://www.ietf.org/rfc/rfc5280.txt">RFC 5280 </url> - Internet X.509 Public Key Infrastructure Certificate and Certificate Revocation List (CRL) Profile </item> - <item>Supports <url href="http://www.rsa.com/rsalabs/node.asp?id=2125"> PKCS-1 </url> - RSA Cryptography Standard </item> + <item>Supports <url href="http://www.ietf.org/rfc/rfc3447.txt"> PKCS-1 </url> - RSA Cryptography Standard </item> <item>Supports <url href="http://csrc.nist.gov/publications/fips/fips186-3/fips_186-3.pdf"> DSS</url>- Digital Signature Standard (DSA - Digital Signature Algorithm)</item> - <item>Supports <url href="http://www.rsa.com/rsalabs/node.asp?id=2126"> PKCS-3 </url> - Diffie-Hellman Key Agreement Standard </item> - <item>Supports <url href="http://www.rsa.com/rsalabs/node.asp?id=2127"> PKCS-5</url> - Password-Based Cryptography Standard </item> - <item>Supports <url href="http://www.rsa.com/rsalabs/node.asp?id=2130"> PKCS-8</url> - Private-Key Information Syntax Standard</item> - <item>Supports <url href="http://www.rsa.com/rsalabs/node.asp?id=2132"> PKCS-10</url> - Certification Request Syntax Standard</item> + <item>Supports <url href="http://www.emc.com/emc-plus/rsa-labs/standards-initiatives/pkcs-3-diffie-hellman-key-agreement-standar.htm"> PKCS-3 </url> - Diffie-Hellman Key Agreement Standard </item> + <item>Supports <url href="http://www.ietf.org/rfc/rfc2898.txt"> PKCS-5</url> - Password-Based Cryptography Standard </item> + <item>Supports <url href="http://www.ietf.org/rfc/rfc5208.txt"> PKCS-8</url> - Private-Key Information Syntax Standard</item> + <item>Supports <url href="http://www.ietf.org/rfc/rfc5967.txt"> PKCS-10</url> - Certification Request Syntax Standard</item> </list> </section> @@ -75,7 +75,7 @@ <p><em>Data Types </em></p> - <p><code>oid() - a tuple of integers as generated by the ASN1 compiler.</code></p> + <p><code>oid() - Object Identifier, a tuple of integers as generated by the ASN1 compiler.</code></p> <p><code>boolean() = true | false</code></p> @@ -92,7 +92,7 @@ not_encrypted | cipher_info()}</code></p> <p><code>cipher_info() = {"RC2-CBC | "DES-CBC" | "DES-EDE3-CBC", - crypto:rand_bytes(8)} | 'PBES2-params'}</code></p> + crypto:rand_bytes(8) | {#'PBEParameter{}, digest_type()} |#'PBES2-params'{}}</code></p> <p><code>public_key() = rsa_public_key() | dsa_public_key() | ec_public_key()</code></p> <p><code>private_key() = rsa_private_key() | dsa_private_key() | ec_private_key()</code></p> @@ -113,6 +113,8 @@ <p><code>rsa_padding() = 'rsa_pkcs1_padding' | 'rsa_pkcs1_oaep_padding' | 'rsa_no_padding'</code></p> + + <p><code>digest_type() - Union of below digest types</code></p> <p><code>rsa_digest_type() = 'md5' | 'sha' | 'sha224' | 'sha256' | 'sha384' | 'sha512'</code></p> @@ -429,10 +431,12 @@ <name>pkix_path_validation(TrustedCert, CertChain, Options) -> {ok, {PublicKeyInfo, PolicyTree}} | {error, {bad_cert, Reason}} </name> <fsummary> Performs a basic path validation according to RFC 5280.</fsummary> <type> - <v> TrustedCert = #'OTPCertificate'{} | der_encode() | unknown_ca | selfsigned_peer </v> - <d>Normally a trusted certificate but it can also be one of the path validation - errors <c>unknown_ca </c> or <c>selfsigned_peer </c> that can be discovered while - constructing the input to this function and that should be run through the <c>verify_fun</c>.</d> + <v> TrustedCert = #'OTPCertificate'{} | der_encode() | atom() </v> + <d>Normally a trusted certificate but it can also be a path validation + error that can be discovered while + constructing the input to this function and that should be run through the <c>verify_fun</c>. + For example <c>unknown_ca </c> or <c>selfsigned_peer </c> + </d> <v> CertChain = [der_encode()]</v> <d>A list of DER encoded certificates in trust order ending with the peer certificate.</d> <v> Options = proplists:proplist()</v> @@ -440,8 +444,8 @@ rsa_public_key() | integer(), 'NULL' | 'Dss-Parms'{}}</v> <v> PolicyTree = term() </v> <d>At the moment this will always be an empty list as Policies are not currently supported</d> - <v> Reason = cert_expired | invalid_issuer | invalid_signature | unknown_ca | - selfsigned_peer | name_not_permitted | missing_basic_constraint | invalid_key_usage | crl_reason() + <v> Reason = cert_expired | invalid_issuer | invalid_signature | name_not_permitted | + missing_basic_constraint | invalid_key_usage | {revoked, crl_reason()} | atom() </v> </type> <desc> @@ -449,7 +453,7 @@ Performs a basic path validation according to <url href="http://www.ietf.org/rfc/rfc5280.txt">RFC 5280.</url> However CRL validation is done separately by <seealso - marker="public_key#pkix_crls_validate-3">pkix_crls_validate/3 </seealso> and should be called + marker="#pkix_crls_validate-3">pkix_crls_validate/3 </seealso> and should be called from the supplied <c>verify_fun</c> </p> @@ -461,11 +465,14 @@ <p>The fun should be defined as:</p> <code> -fun(OtpCert :: #'OTPCertificate'{}, Event :: {bad_cert, Reason :: atom()} | - {extension, #'Extension'{}}, +fun(OtpCert :: #'OTPCertificate'{}, + Event :: {bad_cert, Reason :: atom() | {revoked, atom()}} | + {extension, #'Extension'{}}, InitialUserState :: term()) -> - {valid, UserState :: term()} | {valid_peer, UserState :: term()} | - {fail, Reason :: term()} | {unknown, UserState :: term()}. + {valid, UserState :: term()} | + {valid_peer, UserState :: term()} | + {fail, Reason :: term()} | + {unknown, UserState :: term()}. </code> <p>If the verify callback fun returns {fail, Reason}, the @@ -488,6 +495,35 @@ fun(OtpCert :: #'OTPCertificate'{}, Event :: {bad_cert, Reason :: atom()} | on. </item> </taglist> + + <p> Possible reasons for a bad certificate are: </p> + <taglist> + <tag>cert_expired</tag> + <item>The certificate is no longer valid as its expiration date has passed.</item> + + <tag>invalid_issuer</tag> + <item>The certificate issuer name does not match the name of the issuer certificate in the chain.</item> + + <tag>invalid_signature</tag> + <item>The certificate was not signed by its issuer certificate in the chain.</item> + + <tag>name_not_permitted</tag> + <item>Invalid Subject Alternative Name extension.</item> + + <tag>missing_basic_constraint</tag> + <item>Certificate, required to have the basic constraints extension, does not have + a basic constraints extension.</item> + + <tag>invalid_key_usage</tag> + <item>Certificate key is used in an invalid way according to the key usage extension.</item> + + <tag>{revoked, crl_reason()}</tag> + <item>Certificate has been revoked.</item> + + <tag>atom()</tag> + <item>Application specific error reason that should be checked by the verify_fun</item> + </taglist> + </desc> </func> @@ -496,14 +532,14 @@ fun(OtpCert :: #'OTPCertificate'{}, Event :: {bad_cert, Reason :: atom()} | <fsummary> Performs CRL validation.</fsummary> <type> <v> OTPCertificate = #'OTPCertificate'{}</v> - <v> DPAndCRLs = [{DP::#'DistributionPoint'{} ,CRL::#'CertificateList'{}}] </v> + <v> DPAndCRLs = [{DP::#'DistributionPoint'{}, {DerCRL::der_encoded(), CRL::#'CertificateList'{}}}] </v> <v> Options = proplists:proplist()</v> <v> CRLStatus() = valid | {bad_cert, revocation_status_undetermined} | {bad_cert, {revoked, crl_reason()}}</v> </type> <desc> <p> Performs CRL validation. It is intended to be called from - the verify fun of <seealso marker="public_key#pkix_path_validation-3"> pkix_path_validation/3 + the verify fun of <seealso marker="#pkix_path_validation-3"> pkix_path_validation/3 </seealso></p> <taglist> <p> Available options are: </p> @@ -511,7 +547,8 @@ fun(OtpCert :: #'OTPCertificate'{}, Event :: {bad_cert, Reason :: atom()} | <item> <p>The fun has the following type spec:</p> - <code> fun(#'DistributionPoint'{}, #'CertificateList'{}) -> #'CertificateList'{}</code> + <code> fun(#'DistributionPoint'{}, #'CertificateList'{}) -> + #'CertificateList'{}</code> <p>The fun should use the information in the distribution point to acesses the lates possible version of the CRL. If this fun is not specified @@ -519,6 +556,21 @@ fun(OtpCert :: #'OTPCertificate'{}, Event :: {bad_cert, Reason :: atom()} | </p> <code> fun(_DP, CRL) -> CRL end</code> </item> + + <tag>{issuer_fun, fun()}</tag> + <item> + <p>The fun has the following type spec:</p> + + <code> +fun(#'DistributionPoint'{}, #'CertificateList'{}, + {rdnSequence,[#'AttributeTypeAndValue'{}]}, term()) -> + {ok, #'OTPCertificate'{}, [der_encoded]}</code> + + <p>The fun should return the root certificate and certificate chain + that has signed the CRL. + </p> + <code> fun(DP, CRL, Issuer, UserState) -> {ok, RootCert, CertChain}</code> + </item> </taglist> </desc> </func> diff --git a/lib/public_key/doc/src/public_key_records.xml b/lib/public_key/doc/src/public_key_records.xml index 13bb996f7f..d3534846fa 100644 --- a/lib/public_key/doc/src/public_key_records.xml +++ b/lib/public_key/doc/src/public_key_records.xml @@ -5,7 +5,7 @@ <header> <copyright> <year>2008</year> - <year>2013</year> + <year>2014</year> <holder>Ericsson AB, All Rights Reserved</holder> </copyright> <legalnotice> @@ -35,17 +35,27 @@ </header> <p>This chapter briefly describes Erlang records derived from ASN1 - specifications used to handle public and private keys. The intent - is to describe the data types and not to specify the meaning of - each component for this we refer you to the relevant standards and RFCs.</p> + specifications used to handle public and private keys. + The intent is to describe the data types + and not to specify the semantics of each component. For information on the + semantics, please see the relevant standards and RFCs.</p> <p>Use the following include directive to get access to the - records and constant macros used in the following sections.</p> + records and constant macros described in the following sections.</p> <code> -include_lib("public_key/include/public_key.hrl"). </code> + <section> + <title>Common Data Types</title> + + <p>Common non-standard Erlang + data types used to described the record fields in the + below sections are defined in <seealso + marker="public_key">public key reference manual </seealso></p> + </section> + <section> - <title>RSA as defined by the PKCS-1 standard and RFC 3447.</title> + <title>RSA as defined by the PKCS-1 standard and <url href="http://www.ietf.org/rfc/rfc3447.txt"> RFC 3447 </url></title> <code> #'RSAPublicKey'{ @@ -76,7 +86,8 @@ </section> <section> - <title>DSA as defined by Digital Signature Standard (NIST FIPS PUB 186-2) + <title>DSA as defined by + <url href="http://csrc.nist.gov/publications/fips/fips186-3/fips_186-3.pdf"> Digital Signature Standard (NIST FIPS PUB 186-2) </url> </title> <code> @@ -96,4 +107,47 @@ }. </code> </section> + + <section> + <title>ECC (Elliptic Curve) <url href="http://www.ietf.org/rfc/rfc3447.txt"> RFC 5480 </url> + </title> + + <code> +#'ECPrivateKey'{ + version, % integer() + privateKey, % octet_string() + parameters, % der_encoded() - {'EcpkParameters', #'ECParameters'{}} | + {'EcpkParameters', {namedCurve, oid()}} | + {'EcpkParameters', 'NULL'} % Inherited by CA + publicKey % bitstring() + }. + +#'ECParameters'{ + version, % integer() + fieldID, % #'FieldID'{} + curve, % #'Curve'{} + base, % octet_string() + order, % integer() + cofactor % integer() + }. + +#'Curve'{ + a, % octet_string() + b, % octet_string() + seed % bitstring() - optional + + }. + +#'FieldID'{ + fieldType, % oid() + parameters % Depending on fieldType + }. + +#'ECPoint'{ + point % octet_string() - the public key + }. + + </code> + </section> + </chapter> diff --git a/lib/public_key/src/pubkey_cert.erl b/lib/public_key/src/pubkey_cert.erl index dc8d68c78f..ae517ca642 100644 --- a/lib/public_key/src/pubkey_cert.erl +++ b/lib/public_key/src/pubkey_cert.erl @@ -319,6 +319,8 @@ verify_fun(Otpcert, Result, UserState0, VerifyFun) -> %% %% Description: Extracts a specific extension from a list of extensions. %%-------------------------------------------------------------------- +select_extension(_, asn1_NOVALUE) -> + undefined; select_extension(_, []) -> undefined; select_extension(Id, [#'Extension'{extnID = Id} = Extension | _]) -> @@ -342,8 +344,11 @@ match_name(uniformResourceIdentifier, URI, [PermittedName | Rest]) -> incomplete -> false; {_, _, Host, _, _} -> - match_name(fun is_valid_host_or_domain/2, Host, - PermittedName, Rest) + PN = case split_uri(PermittedName) of + {_, _, PNhost, _, _} -> PNhost; + _X -> PermittedName + end, + match_name(fun is_valid_host_or_domain/2, Host, PN, Rest) end; match_name(emailAddress, Name, [PermittedName | Rest]) -> @@ -511,10 +516,10 @@ is_dir_name2(Value, Value) -> true; is_dir_name2({printableString, Value1}, {printableString, Value2}) -> string:to_lower(strip_spaces(Value1)) =:= string:to_lower(strip_spaces(Value2)); -is_dir_name2({utf8String, Value1}, String) -> %% BUGBUG FIX UTF8 conv - is_dir_name2({printableString, binary_to_list(Value1)}, String); -is_dir_name2(String, {utf8String, Value1}) -> %% BUGBUG FIX UTF8 conv - is_dir_name2(String, {printableString, binary_to_list(Value1)}); +is_dir_name2({utf8String, Value1}, String) -> + is_dir_name2({printableString, unicode:characters_to_list(Value1)}, String); +is_dir_name2(String, {utf8String, Value1}) -> + is_dir_name2(String, {printableString, unicode:characters_to_list(Value1)}); is_dir_name2(_, _) -> false. diff --git a/lib/public_key/src/pubkey_crl.erl b/lib/public_key/src/pubkey_crl.erl index eaba5bfa1b..f0df4bc3f2 100644 --- a/lib/public_key/src/pubkey_crl.erl +++ b/lib/public_key/src/pubkey_crl.erl @@ -39,7 +39,13 @@ validate(OtpCert, OtherDPCRLs, DP, {DerCRL, CRL}, {DerDeltaCRL, DeltaCRL}, CertIssuer = TBSCert#'OTPTBSCertificate'.issuer, TBSCRL = CRL#'CertificateList'.tbsCertList, CRLIssuer = TBSCRL#'TBSCertList'.issuer, - AltNames = subject_alt_names(TBSCert#'OTPTBSCertificate'.extensions), + AltNames = case pubkey_cert:select_extension(?'id-ce-subjectAltName', + TBSCert#'OTPTBSCertificate'.extensions) of + undefined -> + []; + Ext -> + Ext#'Extension'.extnValue + end, revoked_status(DP, IDP, {directoryName, CRLIssuer}, [ {directoryName, CertIssuer} | AltNames], SerialNumber, Revoked, DeltaRevoked, RevokedState1); @@ -397,16 +403,18 @@ verify_dp_name(IDPNames, DPorIssuerNames) -> match_one([], _) -> false; match_one([{Type, Name} | Names], CandidateNames) -> - Candidates = [NameName || {NameType, NameName} <- CandidateNames, NameType == Type], + Candidates = [NameName || {NameType, NameName} <- CandidateNames, + NameType == Type], case Candidates of [] -> false; - [_|_] -> case pubkey_cert:match_name(Type, Name, Candidates) of - true -> - true; - false -> - match_one(Names, CandidateNames) - end + [_|_] -> + case pubkey_cert:match_name(Type, Name, Candidates) of + true -> + true; + false -> + match_one(Names, CandidateNames) + end end. verify_dp_bools(TBSCert, IDP) -> @@ -664,6 +672,8 @@ verify_extensions([#'TBSCertList_revokedCertificates_SEQOF'{crlEntryExtensions = verify_extensions(pubkey_cert:extensions_list(Ext)) and verify_extensions(Rest); verify_extensions([]) -> true; +verify_extensions(asn1_NOVALUE) -> + true; verify_extensions([#'Extension'{critical = true, extnID = Id} | Rest]) -> case lists:member(Id, [?'id-ce-authorityKeyIdentifier', ?'id-ce-issuerAltName', @@ -689,13 +699,3 @@ authority_key_identifier(Extensions) -> Enc = extension_value(?'id-ce-authorityKeyIdentifier', 'AuthorityKeyIdentifier', Extensions), pubkey_cert_records:transform(Enc, decode). - -subject_alt_names(Extensions) -> - Enc = extension_value(?'id-ce-subjectAltName', - 'GeneralNames', Extensions), - case Enc of - undefined -> - []; - _ -> - pubkey_cert_records:transform(Enc, decode) - end. diff --git a/lib/public_key/src/pubkey_pbe.erl b/lib/public_key/src/pubkey_pbe.erl index 460624163b..521a32189d 100644 --- a/lib/public_key/src/pubkey_pbe.erl +++ b/lib/public_key/src/pubkey_pbe.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2011-2013. All Rights Reserved. +%% Copyright Ericsson AB 2011-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -22,7 +22,7 @@ -include("public_key.hrl"). --export([encode/4, decode/4, decrypt_parameters/1]). +-export([encode/4, decode/4, decrypt_parameters/1, encrypt_parameters/1]). -export([pbdkdf1/4, pbdkdf2/7]). -define(DEFAULT_SHA_MAC_KEYLEN, 20). @@ -40,16 +40,16 @@ %%-------------------------------------------------------------------- encode(Data, Password, "DES-CBC" = Cipher, KeyDevParams) -> {Key, IV} = password_to_key_and_iv(Password, Cipher, KeyDevParams), - crypto:block_encrypt(des_cbc, Key, IV, Data); + crypto:block_encrypt(des_cbc, Key, IV, pbe_pad(Data, KeyDevParams)); encode(Data, Password, "DES-EDE3-CBC" = Cipher, KeyDevParams) -> {Key, IV} = password_to_key_and_iv(Password, Cipher, KeyDevParams), <<Key1:8/binary, Key2:8/binary, Key3:8/binary>> = Key, - crypto:block_encrypt(des3_cbc, [Key1, Key2, Key3], IV, Data); + crypto:block_encrypt(des3_cbc, [Key1, Key2, Key3], IV, pbe_pad(Data)); encode(Data, Password, "RC2-CBC" = Cipher, KeyDevParams) -> {Key, IV} = password_to_key_and_iv(Password, Cipher, KeyDevParams), - crypto:block_encrypt(rc2_cbc, Key, IV, Data). + crypto:block_encrypt(rc2_cbc, Key, IV, pbe_pad(Data, KeyDevParams)). %%-------------------------------------------------------------------- -spec decode(binary(), string(), string(), term()) -> binary(). %% @@ -108,6 +108,15 @@ decrypt_parameters(#'EncryptedPrivateKeyInfo_encryptionAlgorithm'{ algorithm = Oid, parameters = Param}) -> decrypt_parameters(Oid, Param). + +%%-------------------------------------------------------------------- +-spec encrypt_parameters({Cipher::string(), Params::term()}) -> + #'EncryptedPrivateKeyInfo_encryptionAlgorithm'{}. +%% +%% Description: Performs ANS1-decoding of encryption parameters. +%%-------------------------------------------------------------------- +encrypt_parameters({Cipher, Params}) -> + encrypt_parameters(Cipher, Params). %%-------------------------------------------------------------------- %%% Internal functions %%-------------------------------------------------------------------- @@ -117,14 +126,18 @@ password_to_key_and_iv(Password, _, #'PBES2-params'{} = Params) -> <<Key:KeyLen/binary, _/binary>> = pbdkdf2(Password, Salt, ItrCount, KeyLen, PseudoRandomFunction, PseudoHash, PseudoOtputLen), {Key, IV}; +password_to_key_and_iv(Password, _Cipher, {#'PBEParameter'{salt = Salt, + iterationCount = Count}, Hash}) -> + <<Key:8/binary, IV:8/binary, _/binary>> + = pbdkdf1(Password, erlang:iolist_to_binary(Salt), Count, Hash), + {Key, IV}; password_to_key_and_iv(Password, Cipher, Salt) -> - KeyLen = derived_key_length(Cipher, undefined), + KeyLen = derived_key_length(Cipher, undefined), <<Key:KeyLen/binary, _/binary>> = pem_encrypt(<<>>, Password, Salt, ceiling(KeyLen div 16), <<>>, md5), %% Old PEM encryption does not use standard encryption method - %% pbdkdf1 and uses then salt as IV + %% pbdkdf1 and uses then salt as IV {Key, Salt}. - pem_encrypt(_, _, _, 0, Acc, _) -> Acc; pem_encrypt(Prev, Password, Salt, Count, Acc, Hash) -> @@ -169,7 +182,52 @@ do_xor_sum(Prf, PrfHash, PrfLen, Prev, Password, Count, Acc)-> decrypt_parameters(?'id-PBES2', DekParams) -> {ok, Params} = 'PKCS-FRAME':decode('PBES2-params', DekParams), - {cipher(Params#'PBES2-params'.encryptionScheme), Params}. + {cipher(Params#'PBES2-params'.encryptionScheme), Params}; +decrypt_parameters(?'pbeWithSHA1AndRC2-CBC', DekParams) -> + {ok, Params} = 'PKCS-FRAME':decode('PBEParameter', DekParams), + {"RC2-CBC", {Params, sha}}; +decrypt_parameters(?'pbeWithSHA1AndDES-CBC', DekParams) -> + {ok, Params} = 'PKCS-FRAME':decode('PBEParameter', DekParams), + {"DES-CBC", {Params, sha}}; +decrypt_parameters(?'pbeWithMD5AndRC2-CBC', DekParams) -> + {ok, Params} = 'PKCS-FRAME':decode('PBEParameter', DekParams), + {"RC2-CBC", {Params, md5}}; +decrypt_parameters(?'pbeWithMD5AndDES-CBC', DekParams) -> + {ok, Params} = 'PKCS-FRAME':decode('PBEParameter', DekParams), + {"DES-CBC", {Params, md5}}. + +encrypt_parameters(_Cipher, #'PBES2-params'{} = Params) -> + {ok, Der} ='PKCS-FRAME':encode('PBES2-params', Params), + #'EncryptedPrivateKeyInfo_encryptionAlgorithm'{ + algorithm = ?'id-PBES2', + parameters = Der}; + +encrypt_parameters(Cipher, {#'PBEParameter'{} = Params, Hash}) -> + {ok, Der} ='PKCS-FRAME':encode('PBEParameter', Params), + #'EncryptedPrivateKeyInfo_encryptionAlgorithm'{ + algorithm = pbe1_oid(Cipher, Hash), + parameters = Der}. + +pbe1_oid("RC2-CBC", sha) -> + ?'pbeWithSHA1AndRC2-CBC'; +pbe1_oid("DES-CBC", sha) -> + ?'pbeWithSHA1AndDES-CBC'; +pbe1_oid("RC2-CBC", md5) -> + ?'pbeWithMD5AndRC2-CBC'; +pbe1_oid("DES-CBC", md5) -> + ?'pbeWithMD5AndDES-CBC'. + +pbe_pad(Data, {#'PBEParameter'{}, _}) -> + pbe_pad(Data); +pbe_pad(Data, #'PBES2-params'{}) -> + pbe_pad(Data); +pbe_pad(Data, _) -> + Data. + +pbe_pad(Data) -> + N = 8 - (erlang:byte_size(Data) rem 8), + Pad = list_to_binary(lists:duplicate(N, N)), + <<Data/binary, Pad/binary>>. key_derivation_params(#'PBES2-params'{keyDerivationFunc = KeyDerivationFunc, encryptionScheme = EncScheme}) -> diff --git a/lib/public_key/src/pubkey_pem.erl b/lib/public_key/src/pubkey_pem.erl index 3a1653d989..98881c4a6a 100644 --- a/lib/public_key/src/pubkey_pem.erl +++ b/lib/public_key/src/pubkey_pem.erl @@ -68,7 +68,8 @@ encode(PemEntries) -> %%-------------------------------------------------------------------- -spec decipher({public_key:pki_asn1_type(), DerEncrypted::binary(), - {Cipher :: string(), Salt :: iodata() | #'PBES2-params'{}}}, + {Cipher :: string(), Salt :: iodata() | #'PBES2-params'{} + | {#'PBEParameter'{}, atom()}}}, string()) -> Der::binary(). %% %% Description: Deciphers a decrypted pem entry. @@ -77,7 +78,8 @@ decipher({_, DecryptDer, {Cipher, KeyDevParams}}, Password) -> pubkey_pbe:decode(DecryptDer, Password, Cipher, KeyDevParams). %%-------------------------------------------------------------------- --spec cipher(Der::binary(), {Cipher :: string(), Salt :: iodata() | #'PBES2-params'{}} , +-spec cipher(Der::binary(), {Cipher :: string(), Salt :: iodata() | #'PBES2-params'{} + | {#'PBEParameter'{}, atom()}}, string()) -> binary(). %% %% Description: Ciphers a PEM entry @@ -94,6 +96,10 @@ encode_pem_entries(Entries) -> encode_pem_entry({Type, Der, not_encrypted}) -> StartStr = pem_start(Type), [StartStr, "\n", b64encode_and_split(Der), "\n", pem_end(StartStr) ,"\n\n"]; +encode_pem_entry({'PrivateKeyInfo', Der, EncParams}) -> + EncDer = encode_encrypted_private_keyinfo(Der, EncParams), + StartStr = pem_start('EncryptedPrivateKeyInfo'), + [StartStr, "\n", b64encode_and_split(EncDer), "\n", pem_end(StartStr) ,"\n\n"]; encode_pem_entry({Type, Der, {Cipher, Salt}}) -> StartStr = pem_start(Type), [StartStr,"\n", pem_decrypt(),"\n", pem_decrypt_info(Cipher, Salt),"\n", @@ -139,6 +145,12 @@ decode_encrypted_private_keyinfo(Der) -> DecryptParams = pubkey_pbe:decrypt_parameters(AlgorithmInfo), {'PrivateKeyInfo', iolist_to_binary(Data), DecryptParams}. + +encode_encrypted_private_keyinfo(EncData, EncryptParmams) -> + AlgorithmInfo = pubkey_pbe:encrypt_parameters(EncryptParmams), + public_key:der_encode('EncryptedPrivateKeyInfo', + #'EncryptedPrivateKeyInfo'{encryptionAlgorithm = AlgorithmInfo, + encryptedData = EncData}). split_bin(Bin) -> split_bin(0, Bin). @@ -197,13 +209,15 @@ pem_start('DSAPrivateKey') -> <<"-----BEGIN DSA PRIVATE KEY-----">>; pem_start('DHParameter') -> <<"-----BEGIN DH PARAMETERS-----">>; +pem_start('EncryptedPrivateKeyInfo') -> + <<"-----BEGIN ENCRYPTED PRIVATE KEY-----">>; pem_start('CertificationRequest') -> <<"-----BEGIN CERTIFICATE REQUEST-----">>; pem_start('ContentInfo') -> <<"-----BEGIN PKCS7-----">>; pem_start('CertificateList') -> <<"-----BEGIN X509 CRL-----">>; -pem_start('OTPEcpkParameters') -> +pem_start('EcpkParameters') -> <<"-----BEGIN EC PARAMETERS-----">>; pem_start('ECPrivateKey') -> <<"-----BEGIN EC PRIVATE KEY-----">>. @@ -260,7 +274,7 @@ asn1_type(<<"-----BEGIN PKCS7-----">>) -> asn1_type(<<"-----BEGIN X509 CRL-----">>) -> 'CertificateList'; asn1_type(<<"-----BEGIN EC PARAMETERS-----">>) -> - 'OTPEcpkParameters'; + 'EcpkParameters'; asn1_type(<<"-----BEGIN EC PRIVATE KEY-----">>) -> 'ECPrivateKey'. diff --git a/lib/public_key/src/public_key.erl b/lib/public_key/src/public_key.erl index a732455aa7..1bbf4ef416 100644 --- a/lib/public_key/src/public_key.erl +++ b/lib/public_key/src/public_key.erl @@ -64,9 +64,15 @@ -type der_encoded() :: binary(). -type pki_asn1_type() :: 'Certificate' | 'RSAPrivateKey' | 'RSAPublicKey' | 'DSAPrivateKey' | 'DSAPublicKey' | 'DHParameter' - | 'SubjectPublicKeyInfo' | 'CertificationRequest' | 'CertificateList'. --type pem_entry() :: {pki_asn1_type(), binary(), %% DER or Encrypted DER - not_encrypted | {Cipher :: string(), Salt :: binary()}}. + | 'SubjectPublicKeyInfo' | 'PrivateKeyInfo' | + 'CertificationRequest' | 'CertificateList' | + 'ECPrivateKey' | 'EcpkParameters'. +-type pem_entry() :: {pki_asn1_type(), + binary(), %% DER or Encrypted DER + not_encrypted | {Cipher :: string(), Salt :: binary()} | + {Cipher :: string(), #'PBES2-params'{}} | + {Cipher :: string(), {#'PBEParameter'{}, atom()}} %% hash type + }. -type asn1_type() :: atom(). %% see "OTP-PUB-KEY.hrl -type ssh_file() :: openssh_public_key | rfc4716_public_key | known_hosts | auth_keys. @@ -133,20 +139,19 @@ pem_entry_decode({Asn1Type, CryptDer, {Cipher, #'PBES2-params'{}}} = PemEntry, is_binary(CryptDer) andalso is_list(Cipher) -> do_pem_entry_decode(PemEntry, Password); +pem_entry_decode({Asn1Type, CryptDer, {Cipher, {#'PBEParameter'{},_}}} = PemEntry, + Password) when is_atom(Asn1Type) andalso + is_binary(CryptDer) andalso + is_list(Cipher) -> + do_pem_entry_decode(PemEntry, Password); pem_entry_decode({Asn1Type, CryptDer, {Cipher, Salt}} = PemEntry, Password) when is_atom(Asn1Type) andalso is_binary(CryptDer) andalso is_list(Cipher) andalso is_binary(Salt) andalso - erlang:byte_size(Salt) == 8 -> - do_pem_entry_decode(PemEntry, Password); -pem_entry_decode({Asn1Type, CryptDer, {"AES-128-CBC"=Cipher, IV}} = PemEntry, - Password) when is_atom(Asn1Type) andalso - is_binary(CryptDer) andalso - is_list(Cipher) andalso - is_binary(IV) andalso - erlang:byte_size(IV) == 16 -> - do_pem_entry_decode(PemEntry, Password). + ((erlang:byte_size(Salt) == 8) or (erlang:byte_size(Salt) == 16)) -> + do_pem_entry_decode(PemEntry, Password). + %%-------------------------------------------------------------------- -spec pem_entry_encode(pki_asn1_type(), term()) -> pem_entry(). @@ -174,13 +179,19 @@ pem_entry_encode(Asn1Type, Entity, {{Cipher, #'PBES2-params'{}} = CipherInfo, is_list(Password) andalso is_list(Cipher) -> do_pem_entry_encode(Asn1Type, Entity, CipherInfo, Password); - +pem_entry_encode(Asn1Type, Entity, {{Cipher, + {#'PBEParameter'{}, _}} = CipherInfo, + Password}) when is_atom(Asn1Type) andalso + is_list(Password) andalso + is_list(Cipher) -> + do_pem_entry_encode(Asn1Type, Entity, CipherInfo, Password); pem_entry_encode(Asn1Type, Entity, {{Cipher, Salt} = CipherInfo, Password}) when is_atom(Asn1Type) andalso is_list(Password) andalso is_list(Cipher) andalso is_binary(Salt) andalso - erlang:byte_size(Salt) == 8 -> + ((erlang:byte_size(Salt) == 8) or + (erlang:byte_size(Salt) == 16)) -> do_pem_entry_encode(Asn1Type, Entity, CipherInfo, Password). %%-------------------------------------------------------------------- @@ -615,11 +626,11 @@ pkix_path_validation(#'OTPCertificate'{} = TrustedCert, CertChain, Options) %-------------------------------------------------------------------- -spec pkix_crls_validate(#'OTPCertificate'{}, - [{DP::#'DistributionPoint'{} ,CRL::#'CertificateList'{}}], + [{DP::#'DistributionPoint'{}, {DerCRL::binary(), CRL::#'CertificateList'{}}}], Options :: proplists:proplist()) -> valid | {bad_cert, revocation_status_undetermined} | {bad_cert, {revoked, crl_reason()}}. -%% Description: Performs a basic path validation according to RFC 5280. +%% Description: Performs a CRL validation according to RFC 5280. %%-------------------------------------------------------------------- pkix_crls_validate(OtpCert, [{_,_,_} |_] = DPAndCRLs, Options) -> pkix_crls_validate(OtpCert, DPAndCRLs, DPAndCRLs, diff --git a/lib/public_key/test/pbe_SUITE.erl b/lib/public_key/test/pbe_SUITE.erl index b68ffbd5fd..aa2bbdd24b 100644 --- a/lib/public_key/test/pbe_SUITE.erl +++ b/lib/public_key/test/pbe_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2011-2013. All Rights Reserved. +%% Copyright Ericsson AB 2011-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -35,7 +35,9 @@ all() -> [ pbdkdf1, pbdkdf2, - encrypted_private_key_info]. + old_enc, + pbes1, + pbes2]. groups() -> []. @@ -192,44 +194,48 @@ pbdkdf2(Config) when is_list(Config) -> 16#cc, 16#37, 16#d7, 16#f0, 16#34, 16#25, 16#e0, 16#c3>> = pubkey_pbe:pbdkdf2("pass\0word", "sa\0lt", 4096, 16, fun crypto:hmac/4, sha, 20). - -encrypted_private_key_info() -> - [{doc,"Tests reading a EncryptedPrivateKeyInfo file encrypted with different ciphers"}]. -encrypted_private_key_info(Config) when is_list(Config) -> + +old_enc() -> + [{doc,"Tests encode/decode RSA key encrypted with different ciphers using old PEM encryption scheme"}]. +old_enc(Config) when is_list(Config) -> Datadir = ?config(data_dir, Config), - {ok, PemDes} = file:read_file(filename:join(Datadir, "des_cbc_enc_key.pem")), + %% key generated with ssh-keygen -N hello_aes -f old_aes_128_cbc_enc_key.pem + {ok, PemAesCbc} = file:read_file(filename:join(Datadir, "old_aes_128_cbc_enc_key.pem")), - PemDesEntry = public_key:pem_decode(PemDes), - ct:print("Pem entry: ~p" , [PemDesEntry]), - [{'PrivateKeyInfo', _, {"DES-CBC",_}} = PubEntry0] = PemDesEntry, - KeyInfo = public_key:pem_entry_decode(PubEntry0, "password"), - - {ok, Pem3Des} = file:read_file(filename:join(Datadir, "des_ede3_cbc_enc_key.pem")), - - Pem3DesEntry = public_key:pem_decode(Pem3Des), - ct:print("Pem entry: ~p" , [Pem3DesEntry]), - [{'PrivateKeyInfo', _, {"DES-EDE3-CBC",_}} = PubEntry1] = Pem3DesEntry, - KeyInfo = public_key:pem_entry_decode(PubEntry1, "password"), - - {ok, PemRc2} = file:read_file(filename:join(Datadir, "rc2_cbc_enc_key.pem")), - - PemRc2Entry = public_key:pem_decode(PemRc2), - ct:print("Pem entry: ~p" , [PemRc2Entry]), - [{'PrivateKeyInfo', _, {"RC2-CBC",_}} = PubEntry2] = PemRc2Entry, - KeyInfo = public_key:pem_entry_decode(PubEntry2, "password"), - - %% key generated with ssh-keygen -N hello_aes -f aes_128_cbc_enc_key - {ok, PemAesCbc} = file:read_file(filename:join(Datadir, "aes_128_cbc_enc_key")), - PemAesCbcEntry = public_key:pem_decode(PemAesCbc), ct:print("Pem entry: ~p" , [PemAesCbcEntry]), [{'RSAPrivateKey', _, {"AES-128-CBC",_}} = PubAesCbcEntry] = PemAesCbcEntry, - #'RSAPrivateKey'{} = public_key:pem_entry_decode(PubAesCbcEntry, "hello_aes"), - - check_key_info(KeyInfo). + #'RSAPrivateKey'{} = public_key:pem_entry_decode(PubAesCbcEntry, "hello_aes"). +pbes1() -> + [{doc,"Tests encode/decode EncryptedPrivateKeyInfo encrypted with different ciphers using PBES1"}]. +pbes1(Config) when is_list(Config) -> + decode_encode_key_file("pbes1_des_cbc_md5_enc_key.pem", "password", "DES-CBC", Config). + +pbes2() -> + [{doc,"Tests encode/decode EncryptedPrivateKeyInfo encrypted with different ciphers using PBES2"}]. +pbes2(Config) when is_list(Config) -> + decode_encode_key_file("pbes2_des_cbc_enc_key.pem", "password", "DES-CBC", Config), + decode_encode_key_file("pbes2_des_ede3_cbc_enc_key.pem", "password", "DES-EDE3-CBC", Config), + decode_encode_key_file("pbes2_rc2_cbc_enc_key.pem", "password", "RC2-CBC", Config). check_key_info(#'PrivateKeyInfo'{privateKeyAlgorithm = #'PrivateKeyInfo_privateKeyAlgorithm'{algorithm = ?rsaEncryption}, privateKey = Key}) -> #'RSAPrivateKey'{} = public_key:der_decode('RSAPrivateKey', iolist_to_binary(Key)). + +decode_encode_key_file(File, Password, Cipher, Config) -> + Datadir = ?config(data_dir, Config), + {ok, PemKey} = file:read_file(filename:join(Datadir, File)), + + PemEntry = public_key:pem_decode(PemKey), + ct:print("Pem entry: ~p" , [PemEntry]), + [{Asn1Type, _, {Cipher,_} = CipherInfo} = PubEntry] = PemEntry, + KeyInfo = public_key:pem_entry_decode(PubEntry, Password), + PemKey1 = public_key:pem_encode([public_key:pem_entry_encode(Asn1Type, KeyInfo, {CipherInfo, Password})]), + Pem = strip_ending_newlines(PemKey), + Pem = strip_ending_newlines(PemKey1), + check_key_info(KeyInfo). + +strip_ending_newlines(Bin) -> + string:strip(binary_to_list(Bin), right, 10). diff --git a/lib/public_key/test/pbe_SUITE_data/aes_128_cbc_enc_key b/lib/public_key/test/pbe_SUITE_data/old_aes_128_cbc_enc_key.pem index 34c7543f30..34c7543f30 100644 --- a/lib/public_key/test/pbe_SUITE_data/aes_128_cbc_enc_key +++ b/lib/public_key/test/pbe_SUITE_data/old_aes_128_cbc_enc_key.pem diff --git a/lib/public_key/test/pbe_SUITE_data/pbes1_des_cbc_md5_enc_key.pem b/lib/public_key/test/pbe_SUITE_data/pbes1_des_cbc_md5_enc_key.pem new file mode 100644 index 0000000000..12e860c7a7 --- /dev/null +++ b/lib/public_key/test/pbe_SUITE_data/pbes1_des_cbc_md5_enc_key.pem @@ -0,0 +1,17 @@ +-----BEGIN ENCRYPTED PRIVATE KEY----- +MIICoTAbBgkqhkiG9w0BBQMwDgQIZmB6EGEnIOcCAggABIICgDaaKoCEUowjKt5M +uwMAIf7uugy09OcqR8PcB9ioiuk5NQGXkIBxOOlOuFb6xrP+O2dSppr5k/ZU+NEX +Lf18AdMld1nlE6lwjPytOIqt6Q3YMeny8un1/jopnkQZKthJ5moER5ohp/2osTDV +4Ih8MtHTwE879SHAmj7Y3G7itKHQi17212DVmL+D+P7iRzTCKIyPj5KMXvXN+eor +j0urZXVOeyRTABHQnf6xJn8K+dGowC/AJTQWOgFunlBKzecepqF22OQzIW2R60aM +VgykSd8A5G8o1F+tO2Qrp6KM9Ak709dUX8qRb/C02w5rjg2g0frgFyEGX0pUJbno +dJLKMOT1WvDnsXaS720beyzrOynWiAuaFZwb1/nPSQnzJ4t0mUvQQis5ph3eHSR/ +a9/PER81IDjPtjlTJjaOGuwhIRmGFsLUrQhOnVcI7Z5TCSj7EHdqK3xzjSVzu5DY +BqE2rsigiIOszPdbK4tKCDheIwBhYdptDvG9c+j3Mj0YNOXJxsX0gVoMqtpwryNG +OZy5fLujS4l+cPq64dOh/LE87mrM9St6M6gw2VRW7d0U18Muubp/MK8q9O2i80Nw +ZFrHHE1N09x3aTnty4mwdCHl6w5aJMZg6WbUXJnf0zKa8ADv5wZmAvW3fO4G8434 +3FHj1hdyKPcoVjoFVawyRUflF/jYd1pLpV+iZwDDR4lacb4ay1Lut452ifZ8DqOq +lWYL0uskCn1WI856vtlLV3gnV02xDjAilSY2hASOyoD1wypZefPn5S+U3vkLuzFZ +ycbyIwGYTLWj71u8Vu3JceRI3OIPDuM7zcNHr71eQyiwLEA0iszQQA9xgqmeFtJO +IkpUTAY= +-----END ENCRYPTED PRIVATE KEY----- diff --git a/lib/public_key/test/pbe_SUITE_data/des_cbc_enc_key.pem b/lib/public_key/test/pbe_SUITE_data/pbes2_des_cbc_enc_key.pem index eaa06145aa..eaa06145aa 100644 --- a/lib/public_key/test/pbe_SUITE_data/des_cbc_enc_key.pem +++ b/lib/public_key/test/pbe_SUITE_data/pbes2_des_cbc_enc_key.pem diff --git a/lib/public_key/test/pbe_SUITE_data/des_ede3_cbc_enc_key.pem b/lib/public_key/test/pbe_SUITE_data/pbes2_des_ede3_cbc_enc_key.pem index 22ea46d56f..22ea46d56f 100644 --- a/lib/public_key/test/pbe_SUITE_data/des_ede3_cbc_enc_key.pem +++ b/lib/public_key/test/pbe_SUITE_data/pbes2_des_ede3_cbc_enc_key.pem diff --git a/lib/public_key/test/pbe_SUITE_data/rc2_cbc_enc_key.pem b/lib/public_key/test/pbe_SUITE_data/pbes2_rc2_cbc_enc_key.pem index 618cddcfd7..618cddcfd7 100644 --- a/lib/public_key/test/pbe_SUITE_data/rc2_cbc_enc_key.pem +++ b/lib/public_key/test/pbe_SUITE_data/pbes2_rc2_cbc_enc_key.pem diff --git a/lib/public_key/vsn.mk b/lib/public_key/vsn.mk index f0450918aa..2fa2d725c3 100644 --- a/lib/public_key/vsn.mk +++ b/lib/public_key/vsn.mk @@ -1 +1 @@ -PUBLIC_KEY_VSN = 0.22 +PUBLIC_KEY_VSN = 0.22.1 diff --git a/lib/reltool/doc/src/notes.xml b/lib/reltool/doc/src/notes.xml index df81418677..18b36ff953 100644 --- a/lib/reltool/doc/src/notes.xml +++ b/lib/reltool/doc/src/notes.xml @@ -37,7 +37,69 @@ thus constitutes one section in this document. The title of each section is the version number of Reltool.</p> - <section><title>Reltool 0.6.4.1</title> + <section><title>Reltool 0.6.6</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fixed a minor typo in an error message from + reltool_server.</p> + <p> + Own Id: OTP-11977</p> + </item> + </list> + </section> + +</section> + +<section><title>Reltool 0.6.5</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + When adding a regexp to a filter in reltool using + {add,Regexp}, and the existing regexp was undefined, + reltool would crash since it got an improper list. This + has been corrected. (Thanks to Håkan Mattsson)</p> + <p> + Own Id: OTP-11591</p> + </item> + <item> + <p> + Adapted reltool test server to common test usage of + tc_status. (Note that this code is not used by OTP daily + test runs.) (Thanks to Håkan Mattsson)</p> + <p> + Own Id: OTP-11592</p> + </item> + <item> + <p> + Application upgrade (appup) files are corrected for the + following applications: </p> + <p> + <c>asn1, common_test, compiler, crypto, debugger, + dialyzer, edoc, eldap, erl_docgen, et, eunit, gs, hipe, + inets, observer, odbc, os_mon, otp_mibs, parsetools, + percept, public_key, reltool, runtime_tools, ssh, + syntax_tools, test_server, tools, typer, webtool, wx, + xmerl</c></p> + <p> + A new test utility for testing appup files is added to + test_server. This is now used by most applications in + OTP.</p> + <p> + (Thanks to Tobias Schlager)</p> + <p> + Own Id: OTP-11744</p> + </item> + </list> + </section> + +</section> + +<section><title>Reltool 0.6.4.1</title> <section><title>Improvements and New Features</title> <list> diff --git a/lib/reltool/src/reltool_server.erl b/lib/reltool/src/reltool_server.erl index 98eeed5c27..e7af4bd3f7 100644 --- a/lib/reltool/src/reltool_server.erl +++ b/lib/reltool/src/reltool_server.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009-2013. All Rights Reserved. +%% Copyright Ericsson AB 2009-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -2030,7 +2030,7 @@ ensure_app_info(#app{name = Name, [BadVsn | _] -> reltool_utils:throw_error( "~w: Application version clash. " - "Multiple directories contains version ~tp.", + "Multiple directories contain version ~tp.", [Name,BadVsn]) end, FirstInfo = hd(AllInfo), diff --git a/lib/reltool/test/reltool_server_SUITE.erl b/lib/reltool/test/reltool_server_SUITE.erl index bfe5d39d53..347e80ed7c 100644 --- a/lib/reltool/test/reltool_server_SUITE.erl +++ b/lib/reltool/test/reltool_server_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009-2013. All Rights Reserved. +%% Copyright Ericsson AB 2009-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in diff --git a/lib/reltool/test/reltool_test_lib.erl b/lib/reltool/test/reltool_test_lib.erl index 530d0a9985..fa12f19aa7 100644 --- a/lib/reltool/test/reltool_test_lib.erl +++ b/lib/reltool/test/reltool_test_lib.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009-2013. All Rights Reserved. +%% Copyright Ericsson AB 2009-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -20,12 +20,13 @@ -compile(export_all). -include("reltool_test_lib.hrl"). +-define(timeout, 20). % minutes %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% init_per_suite(Config) when is_list(Config)-> global:register_name(reltool_global_logger, group_leader()), - incr_timetrap(Config, 10). + incr_timetrap(Config, ?timeout). end_per_suite(Config) when is_list(Config)-> global:unregister_name(reltool_global_logger), @@ -51,7 +52,7 @@ set_kill_timer(Config) -> Time = case lookup_config(tc_timeout, Config) of [] -> - timer:minutes(10); + timer:minutes(?timeout); ConfigTime when is_integer(ConfigTime) -> ConfigTime end, diff --git a/lib/reltool/vsn.mk b/lib/reltool/vsn.mk index 163b77dfa0..4fc1534250 100644 --- a/lib/reltool/vsn.mk +++ b/lib/reltool/vsn.mk @@ -1 +1 @@ -RELTOOL_VSN = 0.6.5 +RELTOOL_VSN = 0.6.6 diff --git a/lib/runtime_tools/doc/src/notes.xml b/lib/runtime_tools/doc/src/notes.xml index 32b7d168f5..9b026aee11 100644 --- a/lib/runtime_tools/doc/src/notes.xml +++ b/lib/runtime_tools/doc/src/notes.xml @@ -31,6 +31,95 @@ <p>This document describes the changes made to the Runtime_Tools application.</p> +<section><title>Runtime_Tools 1.8.14</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + The documentation for the return value of + dbg:{stop,stop_clear} functions are now correct (Thanks + to Luca Favatella)</p> + <p> + Own Id: OTP-11603</p> + </item> + <item> + <p> + Fix DTrace build on Illumos. (Thanks to Ryan Zezeski.)</p> + <p> + Own Id: OTP-11622</p> + </item> + <item> + <p> + Do not turn off scheduler_wall_time, as it can interfere + with other applications usage.</p> + <p> + Own Id: OTP-11693 Aux Id: seq12528 </p> + </item> + <item> + <p> + Application upgrade (appup) files are corrected for the + following applications: </p> + <p> + <c>asn1, common_test, compiler, crypto, debugger, + dialyzer, edoc, eldap, erl_docgen, et, eunit, gs, hipe, + inets, observer, odbc, os_mon, otp_mibs, parsetools, + percept, public_key, reltool, runtime_tools, ssh, + syntax_tools, test_server, tools, typer, webtool, wx, + xmerl</c></p> + <p> + A new test utility for testing appup files is added to + test_server. This is now used by most applications in + OTP.</p> + <p> + (Thanks to Tobias Schlager)</p> + <p> + Own Id: OTP-11744</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Allow install path to have unicode characters.</p> + <p> + Own Id: OTP-10877</p> + </item> + <item> + <p> + The <c>erts_alloc_config</c> tool has been updated to + produce configurations that better fit todays SMP support + in the VM.</p> + <p> + Own Id: OTP-11662</p> + </item> + <item> + <p> + The <seealso + marker="kernel:app"><c>app</c></seealso>-file key + <seealso + marker="kernel:app#runtime_dependencies"><c>runtime_dependencies</c></seealso> + has been introduced.</p> + <p> + Runtime dependencies have been added to all app-files in + OTP. Note that these may not be completely correct during + OTP 17, but this is actively being worked on.</p> + <p> + The function <seealso + marker="runtime_tools:system_information#sanity_check/0"><c>system_information:sanity_check/0</c></seealso> + will verify all declared runtime dependencies in the + system when called.</p> + <p> + Own Id: OTP-11773</p> + </item> + </list> + </section> + +</section> + <section><title>Runtime_Tools 1.8.13</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/sasl/doc/src/alarm_handler.xml b/lib/sasl/doc/src/alarm_handler.xml index ab3041137e..e4def7c7f5 100644 --- a/lib/sasl/doc/src/alarm_handler.xml +++ b/lib/sasl/doc/src/alarm_handler.xml @@ -5,7 +5,7 @@ <header> <copyright> <year>1996</year> - <year>2013</year> + <year>2014</year> <holder>Ericsson AB, All Rights Reserved</holder> </copyright> <legalnotice> @@ -87,7 +87,9 @@ <v>AlarmId = term()</v> </type> <desc> - <p>Clears all alarms with id <c>AlarmId</c>. + <p>Sends the <c>clear_alarm</c> event to all event handlers.</p> + <p>When receiving this event, the default simple handler + clears the latest received alarm with id <c>AlarmId</c>. </p> </desc> </func> @@ -109,8 +111,10 @@ <v>AlarmDescription = term()</v> </type> <desc> - <p>Sets an alarm with id <c>AlarmId</c>. This id is used at a - later stage when the alarm is cleared. + <p>Sends the <c>set_alarm</c> event to all event handlers.</p> + <p>When receiving this event, the default simple handler + stores the alarm. The <c>AlarmId</c> identifies the alarm + and is used when the alarm is cleared. </p> </desc> </func> diff --git a/lib/sasl/doc/src/appup.xml b/lib/sasl/doc/src/appup.xml index 85fcbed3ba..95f315d269 100644 --- a/lib/sasl/doc/src/appup.xml +++ b/lib/sasl/doc/src/appup.xml @@ -4,7 +4,7 @@ <fileref> <header> <copyright> - <year>1997</year><year>2013</year> + <year>1997</year><year>2014</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -84,6 +84,9 @@ version identifier must be specified as a binary, e.g.</p> <code type="none"><<"2\\.1\\.[0-9]+">></code> <p>will match all versions <c>2.1.x</c>, where x is any number.</p> + <p>Note that the regular expression must match the complete + version string, so the above example will work for for + e.g. <c>2.1.1</c>, but not for <c>2.1.1.1</c></p> </section> <section> @@ -339,7 +342,7 @@ restart_new_emulator version of erts, kernel, stdlib and sasl are used when the emulator restarts. Only one <c>restart_new_emulator</c> instruction is allowed in the relup, and it shall be placed - first. <seealso marker="systools#make_relup/3">systools:make_relup3,4</seealso> + first. <seealso marker="systools#make_relup/3">systools:make_relup/3,4</seealso> will ensure this when the relup is generated. The rest of the relup script is executed after the restart as a part of the boot script.</p> @@ -347,11 +350,25 @@ restart_new_emulator completed. To programatically find out if the upgrade is complete, call <seealso marker="release_handler#which_releases/0"> - release_handler:which_releases</seealso> and check if the + release_handler:which_releases/0,1</seealso> and check if the expected release has status <c>current</c>.</p> <p>The new release must still be made permanent after the upgrade is completed. Otherwise, the old emulator is started in case of an emulator restart.</p> + <warning> + <p>As stated above, the <c>restart_new_emulator</c> + instruction causes the emulator to be restarted with new + versions of <c>erts</c>, <c>kernel</c>, <c>stdlib</c> and + <c>sasl</c>. All other applications, however, will at startup + be running their old versions in this new emulator. In most + cases this is no problem, but every now and then there will be + incompatible changes to the core applications which may cause + trouble in this setting. Such incompatible changes (when + functions are removed) are normally preceded by a deprecation + over two major releases. To make sure your application is not + crashed by an incompatible change, always remove any call to + deprecated functions as soon as possible.</p> + </warning> <pre> restart_emulator </pre> diff --git a/lib/sasl/doc/src/notes.xml b/lib/sasl/doc/src/notes.xml index 09d97cbe7b..95d7c6fa50 100644 --- a/lib/sasl/doc/src/notes.xml +++ b/lib/sasl/doc/src/notes.xml @@ -30,6 +30,84 @@ </header> <p>This document describes the changes made to the SASL application.</p> +<section><title>SASL 2.4.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + The documentation erroneously specified that + <c>alarm_handler:clear_alarm/1</c> would clear + <em>all</em> alarms with id <c>AlarmId</c>. This is now + corrected according to the implementation - only the + latest received alarm with the given <c>AlarmId</c> is + cleared by the simple default handler.</p> + <p> + Own Id: OTP-12025</p> + </item> + </list> + </section> + +</section> + +<section><title>SASL 2.4</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + The upgrade instruction 'restart_application' would + earlier ignore the restart type configured in the .rel + file and always restart the application as permanent. + This is now changed, and the restart type from the .rel + file is used. If restart type is 'load', the application + will only be loaded and not started. If the restart type + is 'none', the application will not be loaded nor + started, but all modules in the application will be + loaded. (Thanks to Tobias Schlager for reporting this + problem)</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-11716</p> + </item> + <item> + <p> + If <c>systools:make_script/2</c> failed with reason + <c>duplicate_modules</c>, and the <c>silent</c> flag was + not used, a crash with reason <c>function_clause</c> + would occur when <c>systools</c> tried to format the + error message. This has been corrected. (Thanks to + Jean-Sébastien Pédron)</p> + <p> + Own Id: OTP-11819</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Calls to erlang:open_port/2 with 'spawn' are updated to + handle space in the command path.</p> + <p> + Own Id: OTP-10842</p> + </item> + <item> + <p> + Some more documentation is added to explain the behavior + when an upgrade includes new versions of ERTS, Kernel, + STDLIB or SASL.</p> + <p> + Own Id: OTP-11717</p> + </item> + </list> + </section> + +</section> + <section><title>SASL 2.3.4</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/sasl/examples/src/target_system.erl b/lib/sasl/examples/src/target_system.erl index fb9e9aaaaf..a0ae016791 100644 --- a/lib/sasl/examples/src/target_system.erl +++ b/lib/sasl/examples/src/target_system.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2011-2013. All Rights Reserved. +%% Copyright Ericsson AB 2011-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -65,7 +65,7 @@ create(RelFileName,SystoolsOpts) -> [RelFileName, RelFileName]), make_script(RelFileName,SystoolsOpts), - TarFileName = filename:join(Dir,RelFileName ++ ".tar.gz"), + TarFileName = RelFileName ++ ".tar.gz", io:fwrite("Creating tar file ~tp ...~n", [TarFileName]), make_tar(RelFileName,SystoolsOpts), @@ -100,6 +100,12 @@ create(RelFileName,SystoolsOpts) -> copy_file(filename:join([ErtsBinDir, "to_erl"]), filename:join([TmpBinDir, "to_erl"]), [preserve]), + %% This is needed if 'start' script created from 'start.src' shall + %% be used as it points out this directory as log dir for 'run_erl' + TmpLogDir = filename:join([TmpDir, "log"]), + io:fwrite("Creating temporary directory ~tp ...~n", [TmpLogDir]), + ok = file:make_dir(TmpLogDir), + StartErlDataFile = filename:join([TmpDir, "releases", "start_erl.data"]), io:fwrite("Creating ~tp ...~n", [StartErlDataFile]), StartErlData = io_lib:fwrite("~s ~s~n", [ErtsVsn, RelVsn]), @@ -115,6 +121,7 @@ create(RelFileName,SystoolsOpts) -> erl_tar:add(Tar, filename:join(TmpDir,ErtsDir), ErtsDir, []), erl_tar:add(Tar, filename:join(TmpDir,"releases"), "releases", []), erl_tar:add(Tar, filename:join(TmpDir,"lib"), "lib", []), + erl_tar:add(Tar, filename:join(TmpDir,"log"), "log", []), erl_tar:close(Tar), %% file:set_cwd(Cwd), io:fwrite("Removing directory ~tp ...~n",[TmpDir]), @@ -136,6 +143,11 @@ install(RelFileName, RootDir) -> subst_src_scripts(["erl", "start", "start_erl"], ErtsBinDir, BinDir, [{"FINAL_ROOTDIR", RootDir}, {"EMU", "beam"}], [preserve]), + %%! Workaround for pre OTP 17.0: start.src and start_erl.src did + %%! not have correct permissions, so the above 'preserve' option did not help + ok = file:change_mode(filename:join(BinDir,"start"),8#0755), + ok = file:change_mode(filename:join(BinDir,"start_erl"),8#0755), + io:fwrite("Creating the RELEASES file ...\n"), create_RELEASES(RootDir, filename:join([RootDir, "releases", filename:basename(RelFileName)])). diff --git a/lib/sasl/src/systools_make.erl b/lib/sasl/src/systools_make.erl index 3d370a93a5..e5da797efb 100644 --- a/lib/sasl/src/systools_make.erl +++ b/lib/sasl/src/systools_make.erl @@ -2233,7 +2233,7 @@ format_error({undefined_applications,Apps}) -> io_lib:format("Undefined applications: ~p~n",[Apps]); format_error({duplicate_modules,Dups}) -> io_lib:format("Duplicated modules: ~n~ts", - [map(fun({{Mod,_,App1,_,_},{Mod,_,App2,_,_}}) -> + [map(fun({{Mod,App1,_},{Mod,App2,_}}) -> io_lib:format("\t~w specified in ~w and ~w~n", [Mod,App1,App2]) end, Dups)]); diff --git a/lib/sasl/test/release_handler_SUITE.erl b/lib/sasl/test/release_handler_SUITE.erl index ad2a8005b9..bd7414fbb4 100644 --- a/lib/sasl/test/release_handler_SUITE.erl +++ b/lib/sasl/test/release_handler_SUITE.erl @@ -667,6 +667,9 @@ release_handler_which_releases(Conf) -> ok. +release_handler_which_releases(cleanup,_Conf) -> + stop_node(node_name(release_handler_which_releases)). + %%----------------------------------------------------------------- %% Ticket: OTP-2740 %% Slogan: vsn not numeric doesn't work so good in release_handling @@ -1365,6 +1368,9 @@ upgrade_supervisor(Conf) when is_list(Conf) -> ok. +upgrade_supervisor(cleanup,_Condf) -> + stop_node(node_name(upgrade_supervisor)). + %% Check that if the supervisor fails, then the upgrade is rolled back %% and an ok error message is returned upgrade_supervisor_fail(Conf) when is_list(Conf) -> @@ -1404,18 +1410,41 @@ upgrade_supervisor_fail(Conf) when is_list(Conf) -> {error,{code_change_failed,_Pid,a_sup,_Vsn, {error,{invalid_shutdown,brutal_kil}}}} = - rpc:call(Node, release_handler, install_release, [RelVsn2]), - - %% Check that the upgrade is terminated - normally this would mean - %% rollback, but since this testcase is very simplified the node - %% is not started with heart supervision and will therefore not be - %% restarted. So we just check that the node goes down. + rpc:call(Node, release_handler, install_release, + [RelVsn2, [{error_action,reboot}]]), + + %% Check that the upgrade is terminated - normally this would be a + %% rollback, but + %% + %% 1. Default rollback is done with init:restart(), which does not + %% reboot the emulator, it only restarts the system inside the + %% running erlang node. + %% + %% 2. This does not work well on a slave node since, if timing is + %% right (bad), the slave node will get the nodedown from its + %% master (because distribution is terminated as part of + %% init:restart()) and then it will do halt() and thus never be + %% restarted (see slave:wloop/1) + %% + %% 3. Sometimes, though, init:restart() will manage to finish its + %% job before the nodedown is received, making the node + %% actually restart - in which case it might very well confuse + %% the next test case. + %% + %% 4. So, to avoid unstability we use {error_action,reboot} above, + %% to ensure that the node is actually stopped. Of course, in a + %% real system this must be used together with heart + %% supervision, and then the node will be restarted anyway. But + %% here in this simple test case we are satisfied to see that + %% the node terminates. receive {nodedown,Node} -> ok after 10000 -> ct:fail(failed_upgrade_never_restarted_node) end, ok. +upgrade_supervisor_fail(cleanup,_Condf) -> + stop_node(node_name(upgrade_supervisor_fail)). %% Test upgrade and downgrade of applications eval_appup(Conf) when is_list(Conf) -> @@ -2263,8 +2292,8 @@ create_p1g(Conf,TargetDir) -> ok. fix_version(SystemLib,App) -> - FromVsn = vsn(App,current), - ToVsn = vsn(App,old), + FromVsn = re:replace(vsn(App,current),"\\.","\\\\.",[{return,binary}]), + ToVsn = re:replace(vsn(App,old),"\\.","\\\\.",[{return,binary}]), Rootname = filename:join([SystemLib,app_dir(App,old),ebin,atom_to_list(App)]), AppFile = Rootname ++ ".app", @@ -2417,9 +2446,28 @@ check_gg_info(Node,OtherAlive,OtherDead,Synced) -> ?t:format("~ncheck_gg_info failed for ~p: ~p~nwhen GGI was: ~p~n" "and GI was: ~p~n", [Node,E,GGI,GI]), + %% An attempt to find out if it is only a timing issue + %% that makes this fail every now and then: + try_again_check(Node,GGI,GI,1), ?t:fail("check_gg_info failed") end. +try_again_check(_Node,_GGI,_GI,6) -> + ok; +try_again_check(Node,GGI,GI,N) -> + timer:sleep(1000), + case {rpc:call(Node,global_group,info,[]), + rpc:call(Node,global,info,[])} of + {GGI,GI} -> + ?t:format("~nAfter one more sek, GGI and GI are still the same"), + try_again_check(Node,GGI,GI,N+1); + {NewGGI,NewGI} -> + ?t:format("~nAfter one more sek:~nNew GGI: ~p~nNew GI: ~p~n", + [NewGGI,NewGI]), + try_again_check(Node,NewGGI,NewGI,N+1) + end. + + do_check_gg_info(OtherAlive,OtherDead,Synced,GGI,GI) -> {_,gg1} = lists:keyfind(own_group_name,1,GGI), {_,synced} = lists:keyfind(state,1,GGI), @@ -2563,7 +2611,7 @@ start_nodes(Conf,Snames,Tag) -> start_node_unix(Sname,NodeDir) -> Script = filename:join([NodeDir,"bin","start"]), - ?t:format("Starting ~p: ~tp~n", [Sname,Script]), + ?t:format("Starting ~p: ~ts~n", [Sname,Script]), case rh_test_lib:cmd(Script,[],[{"NODENAME",atom_to_list(Sname)}]) of ok -> {ok,node_name(Sname)}; diff --git a/lib/sasl/test/sasl_SUITE.erl b/lib/sasl/test/sasl_SUITE.erl index f4455f7e9b..e91d220daf 100644 --- a/lib/sasl/test/sasl_SUITE.erl +++ b/lib/sasl/test/sasl_SUITE.erl @@ -73,10 +73,10 @@ appup_tests(App,{OkVsns,NokVsns}) -> ok. create_test_vsns(App) -> - This = erlang:system_info(otp_release), - FirstMajor = previous_major(This), + ThisMajor = erlang:system_info(otp_release), + FirstMajor = previous_major(ThisMajor), SecondMajor = previous_major(FirstMajor), - Ok = app_vsn(App,[FirstMajor]), + Ok = app_vsn(App,[ThisMajor,FirstMajor]), Nok0 = app_vsn(App,[SecondMajor]), Nok = case Ok of [Ok1|_] -> @@ -87,9 +87,9 @@ create_test_vsns(App) -> {Ok,Nok}. previous_major("17") -> - "r16"; -previous_major("r"++Rel) -> - "r"++previous_major(Rel); + "r16b"; +previous_major("r16b") -> + "r15b"; previous_major(Rel) -> integer_to_list(list_to_integer(Rel)-1). diff --git a/lib/sasl/test/systools_SUITE.erl b/lib/sasl/test/systools_SUITE.erl index e3f6933476..49a4303e0b 100644 --- a/lib/sasl/test/systools_SUITE.erl +++ b/lib/sasl/test/systools_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2012-2013. All Rights Reserved. +%% Copyright Ericsson AB 2012-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -47,6 +47,7 @@ abnormal_script/1, src_tests_script/1, crazy_script/1, included_script/1, included_override_script/1, included_fail_script/1, included_bug_script/1, exref_script/1, + duplicate_modules_script/1, otp_3065_circular_dependenies/1, included_and_used_sort_script/1]). -export([tar_options/1, normal_tar/1, no_mod_vsn_tar/1, system_files_tar/1, system_files_tar/2, invalid_system_files_tar/1, @@ -84,6 +85,7 @@ groups() -> src_tests_script, crazy_script, included_script, included_override_script, included_fail_script, included_bug_script, exref_script, + duplicate_modules_script, otp_3065_circular_dependenies, included_and_used_sort_script]}, {tar, [], [tar_options, normal_tar, no_mod_vsn_tar, system_files_tar, @@ -822,6 +824,33 @@ no_hipe({ok, Value}) -> {ok, Value} end. +%% duplicate_modules_script: Check that make_script rejects two +%% applications providing the same module. +duplicate_modules_script(Config) when is_list(Config) -> + {ok, OldDir} = file:get_cwd(), + + {LatestDir, LatestName} = create_script(duplicate_modules,Config), + + DataDir = filename:absname(?copydir), + + ok = file:set_cwd(LatestDir), + LibDir = fname([DataDir, d_duplicate_modules, lib]), + P = [fname([LibDir, 'app1-1.0', ebin]), + fname([LibDir, 'app2-1.0', ebin])], + + %% Check wrong app vsn + error = systools:make_script(LatestName, [{path, P}]), + {error, + systools_make, + {duplicate_modules, [ + {{myapp,app1,_}, {myapp,app2,_}} + ] + } + } = systools:make_script(LatestName, [silent, {path, P}]), + + ok = file:set_cwd(OldDir), + ok. + %% tar_options: Check illegal tar options. tar_options(Config) when is_list(Config) -> {'EXIT',{{badarg,[{path,["Path",12,"Another"]}]}, _}} = @@ -1586,9 +1615,19 @@ no_sasl_relup(Config) when is_list(Config) -> %% make_relup: Check that application start type is used in relup app_start_type_relup(Config) when is_list(Config) -> + %% This might fail if some applications are not available, if so + %% skip the test case. + try create_script(latest_app_start_type2,Config) of + {Dir2,Name2} -> + app_start_type_relup(Dir2,Name2,Config) + catch throw:{error,Reason} -> + {skip,Reason} + end. + +app_start_type_relup(Dir2,Name2,Config) -> PrivDir = ?config(priv_dir, Config), {Dir1,Name1} = create_script(latest_app_start_type1,Config), - {Dir2,Name2} = create_script(latest_app_start_type2,Config), + Release1 = filename:join(Dir1,Name1), Release2 = filename:join(Dir2,Name2), @@ -2186,7 +2225,10 @@ create_script(current_all_future_sasl,Config) -> do_create_script(current_all_future_sasl,Config,current,Apps); create_script({unicode,RelVsn},Config) -> Apps = core_apps(current) ++ [{ua,"1.0"}], - do_create_script(unicode,RelVsn,Config,current,Apps). + do_create_script(unicode,RelVsn,Config,current,Apps); +create_script(duplicate_modules,Config) -> + Apps = core_apps(current) ++ [{app1,"1.0"},{app2,"1.0"}], + do_create_script(duplicate_modules,Config,current,Apps). do_create_script(Id,Config,ErtsVsn,AppVsns) -> @@ -2210,9 +2252,13 @@ app_vsns(AppVsns) -> [{App,app_vsn(App,Vsn)} || {App,Vsn} <- AppVsns] ++ [{App,app_vsn(App,Vsn),Type} || {App,Vsn,Type} <- AppVsns]. app_vsn(App,current) -> - application:load(App), - {ok,Vsn} = application:get_key(App,vsn), - Vsn; + case application:load(App) of + Ok when Ok==ok; Ok=={error,{already_loaded,App}} -> + {ok,Vsn} = application:get_key(App,vsn), + Vsn; + Error -> + throw(Error) + end; app_vsn(_App,Vsn) -> Vsn. diff --git a/lib/sasl/test/systools_SUITE_data/d_duplicate_modules/lib/app1-1.0/ebin/app1.app b/lib/sasl/test/systools_SUITE_data/d_duplicate_modules/lib/app1-1.0/ebin/app1.app new file mode 100644 index 0000000000..dea9257f2f --- /dev/null +++ b/lib/sasl/test/systools_SUITE_data/d_duplicate_modules/lib/app1-1.0/ebin/app1.app @@ -0,0 +1,7 @@ +{application, app1, + [{description, "Application 1"}, + {vsn, "1.0"}, + {modules, [myapp]}, + {registered, []}, + {applications, []}, + {env, []}]}. diff --git a/lib/sasl/test/systools_SUITE_data/d_duplicate_modules/lib/app1-1.0/src/myapp.erl b/lib/sasl/test/systools_SUITE_data/d_duplicate_modules/lib/app1-1.0/src/myapp.erl new file mode 100644 index 0000000000..bf2ab7c79c --- /dev/null +++ b/lib/sasl/test/systools_SUITE_data/d_duplicate_modules/lib/app1-1.0/src/myapp.erl @@ -0,0 +1,2 @@ +-module(myapp). +-vsn("1.0"). diff --git a/lib/sasl/test/systools_SUITE_data/d_duplicate_modules/lib/app2-1.0/ebin/app2.app b/lib/sasl/test/systools_SUITE_data/d_duplicate_modules/lib/app2-1.0/ebin/app2.app new file mode 100644 index 0000000000..476750d8b2 --- /dev/null +++ b/lib/sasl/test/systools_SUITE_data/d_duplicate_modules/lib/app2-1.0/ebin/app2.app @@ -0,0 +1,7 @@ +{application, app2, + [{description, "Application 2"}, + {vsn, "1.0"}, + {modules, [myapp]}, + {registered, []}, + {applications, []}, + {env, []}]}. diff --git a/lib/sasl/test/systools_SUITE_data/d_duplicate_modules/lib/app2-1.0/src/myapp.erl b/lib/sasl/test/systools_SUITE_data/d_duplicate_modules/lib/app2-1.0/src/myapp.erl new file mode 100644 index 0000000000..bf2ab7c79c --- /dev/null +++ b/lib/sasl/test/systools_SUITE_data/d_duplicate_modules/lib/app2-1.0/src/myapp.erl @@ -0,0 +1,2 @@ +-module(myapp). +-vsn("1.0"). diff --git a/lib/sasl/vsn.mk b/lib/sasl/vsn.mk index da8cbc5130..4259a2d76c 100644 --- a/lib/sasl/vsn.mk +++ b/lib/sasl/vsn.mk @@ -1 +1 @@ -SASL_VSN = 2.4 +SASL_VSN = 2.4.1 diff --git a/lib/snmp/.gitignore b/lib/snmp/.gitignore index 650c1d6865..aef73491a4 100644 --- a/lib/snmp/.gitignore +++ b/lib/snmp/.gitignore @@ -5,5 +5,4 @@ *.rej doc/index.html - - +mibs/.index diff --git a/lib/snmp/doc/src/notes.xml b/lib/snmp/doc/src/notes.xml index ab5514e550..bbe6438f04 100644 --- a/lib/snmp/doc/src/notes.xml +++ b/lib/snmp/doc/src/notes.xml @@ -33,7 +33,75 @@ </header> - <section> + <section><title>SNMP 5.1</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + The SNMP manager has been enhanced with dual stack + IPv4+IPv6, as the agent just was. The documentation is + also now updated for both the agent and the manager.</p> + <p> + Own Id: OTP-12108 Aux Id: OTP-12020 </p> + </item> + </list> + </section> + +</section> + +<section><title>SNMP 5.0</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + SNMP has been improved to handle IPv6. The agent can + handle dual stack IPv4 + IPv6, but not yet the manager. + The documentation also still lags behind... If you do + such advanced stuff like writing a custom net_if module, + the interface for it has changed, but other than that + SNMP is backwards compatible.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-12020 Aux Id: OTP-11518 </p> + </item> + </list> + </section> + +</section> + +<section><title>SNMP 4.25.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Application upgrade (appup) files are corrected for the + following applications: </p> + <p> + <c>asn1, common_test, compiler, crypto, debugger, + dialyzer, edoc, eldap, erl_docgen, et, eunit, gs, hipe, + inets, observer, odbc, os_mon, otp_mibs, parsetools, + percept, public_key, reltool, runtime_tools, ssh, + syntax_tools, test_server, tools, typer, webtool, wx, + xmerl</c></p> + <p> + A new test utility for testing appup files is added to + test_server. This is now used by most applications in + OTP.</p> + <p> + (Thanks to Tobias Schlager)</p> + <p> + Own Id: OTP-11744</p> + </item> + </list> + </section> + +</section> + +<section> <title>SNMP Development Toolkit 4.25.0.1</title> <p>Version 4.25.0.1 supports code replacement in runtime from/to version 4.25, 4.24.2, 4.24.1 and 4.24. </p> diff --git a/lib/snmp/doc/src/snmp_agent_config_files.xml b/lib/snmp/doc/src/snmp_agent_config_files.xml index 1e8e879814..1e938c0dc8 100644 --- a/lib/snmp/doc/src/snmp_agent_config_files.xml +++ b/lib/snmp/doc/src/snmp_agent_config_files.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>1997</year><year>2013</year> + <year>1997</year><year>2014</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -102,20 +102,41 @@ <p><c>AgentVariable</c> is one of the variables is SNMP-FRAMEWORK-MIB or one of the internal variables <c>intAgentUDPPort</c>, which defines which UDP port the agent - listens to, or <c>intAgentIpAddress</c>, which defines the IP - address of the agent. </p> + listens to, or <c>intAgentTransports</c>, which defines the + transport domains and addresses of the agent. </p> </item> <item> <p><c>Value</c> is the value for the variable.</p> </item> </list> - <p>The following example shows a <c>agent.conf</c> file: </p> + <p>The following example shows an <c>agent.conf</c> file: </p> <pre> {intAgentUDPPort, 4000}. -{intAgentIpAddress,[141,213,11,24]}. +{intAgentTransports, + [{transportDomainUdpIpv4, {141,213,11,24}}, + {transportDomainUdpIpv6, {0,0,0,0,0,0,0,1}}]}. {snmpEngineID, "mbj's engine"}. {snmpEngineMaxPacketSize, 484}. </pre> + <p>The value of <c>intAgentTransports</c> is a list of + <c>{Domain, Addr}</c> tuples, where <c>Domain</c> + is either <c>transportDomainUdpIpv4</c> or <c>transportDomainUdpIpv6</c>, + and <c>Addr</c> is the address in the domain. + <c>Addr</c> can be specified either as an + <c>IpAddr</c> or as an <c>{IpAddr, IpPort}</c> tuple. + <c>IpAddr</c> is either a regular Erlang/OTP + <seealso marker="kernel:inet#type-ip_address"><c>ip_address()</c></seealso> + or a traditional SNMP integer list and <c>IpPort</c> is an integer. + </p> + + <p>When the <c>Addr</c> value does not contain a port number, + the value of <c>intAgentUDPPort</c> is used.</p> + + <p>The legacy and intermediate variables <c>intAgentIpAddress</c> + and <c>intAgentTransportDomain</c> are still supported so old + <c>agent.conf</c> files will work. + </p> + <p>The value of <c>snmpEngineID</c> is a string, which for a deployed agent should have a very specific structure. See RFC 2271/2571 for details.</p> @@ -362,9 +383,9 @@ SNMP-TARGET-MIB and <c>snmpTargetAddrExtTable</c> in the SNMP-COMMUNITY-MIB. </p> <p>Each entry is a term: </p> - <p><c>{TargetName, Ip, Udp, Timeout, RetryCount, TagList, ParamsName, EngineId}.</c> <br></br> or <br></br> -<c>{TargetName, Ip, Udp, Timeout, RetryCount, TagList, ParamsName, EngineId, TMask, MaxMessageSize}.</c> <br></br> or <br></br> -<c>{TargetName, Domain, Ip, Udp, Timeout, RetryCount, TagList, ParamsName, EngineId, TMask, MaxMessageSize}.</c> </p> + <p><c>{TargetName, Domain, Addr, Timeout, RetryCount, TagList, ParamsName, EngineId}.</c> + <br></br> or <br></br> + <c>{TargetName, Domain, Addr, Timeout, RetryCount, TagList, ParamsName, EngineId, TMask, MaxMessageSize}.</c> </p> <list type="bulleted"> <item> <p><c>TargetName</c> is a unique non-empty string. </p> @@ -374,11 +395,14 @@ <c>transportDomainUdpIpv4</c> | <c>transportDomainUdpIpv6</c>. </p> </item> <item> - <p><c>Ip</c> is a list of four or eight integers. </p> - </item> - <item> - <p><c>Udp</c> is an integer. </p> + <p><c>Addr</c> is either an <c>IpAddr</c> or + an <c>{IpAddr, IpPort}</c> tuple. <c>IpAddr</c> is either + a regular Erlang/OTP + <seealso marker="kernel:inet#type-ip_address"><c>ip_address()</c></seealso> + or a traditional SNMP integer list, and <c>IpPort</c> is an integer.</p> + <p>If <c>IpPort</c> is omitted <c>162</c> is used.</p> </item> + <item> <p><c>Timeout</c> is an integer. </p> </item> @@ -395,13 +419,17 @@ <p><c>EngineId</c> is a string or the atom <c>discovery</c>. </p> </item> <item> - <p><c>TMask</c> is a list of integer() of size 0, - size 6 or size 10 (default: []). </p> + <p><c>TMask</c> is specified just as <c>Addr</c> or as <c>[]</c>. + Note in particular that using a list of 6 bytes for IPv4 + or 8 words plus 2 bytes for IPv6 are still valid address formats + so old configurations will work.</p> </item> <item> <p><c>MaxMessageSize</c> is an integer (default: 2048). </p> </item> </list> + <p>The old tuple formats with <c>Ip</c> address and <c>Udp</c> + port number found in old configurations still work.</p> <p>Note that if <c>EngineId</c> has the value <c>discovery</c>, the agent cannot send <c>inform</c> messages to that manager until it has performed the diff --git a/lib/snmp/doc/src/snmp_agent_netif.xml b/lib/snmp/doc/src/snmp_agent_netif.xml index fccfc8857a..a9ce05e757 100644 --- a/lib/snmp/doc/src/snmp_agent_netif.xml +++ b/lib/snmp/doc/src/snmp_agent_netif.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>1997</year><year>2013</year> + <year>1997</year><year>2014</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -51,7 +51,8 @@ </p> <p>It is also possible to write your own Net if process. The default Net if process is implemented in the module <c>snmpa_net_if</c> and - it uses UDP as the transport protocol. + it uses UDP as the transport protocol i.e the transport domains + <c>transportDomainUdpIpv4</c> and/or <c>transportDomainUdpIpv6</c>. </p> <p>This section describes how to write a Net if process. </p> @@ -70,6 +71,12 @@ <p>The section <em>Messages</em> describes mandatory messages, which Net if must send and be able to receive. </p> + <p>In this section an <c>Address</c> field is a + <c>{Domain, Addr}</c> tuple where <c>Domain</c> is + <c>transportDomainUdpIpv4</c> or <c>transportDomainUdpIpv4</c>, + and <c>Addr</c> is an + <c>{<seealso marker="kernel:inet#type-ip_address">IpAddr</seealso>, + IpPort}</c> tuple.</p> <section> <marker id="outgoing_messages"></marker> @@ -96,10 +103,7 @@ MasterAgent ! {snmp_pdu, Vsn, Pdu, PduMS, ACMData, From, Extra} in use. Normally this is returned from <c>snmpa_mpd:process_packet</c> (see Reference Manual). </item> - <item><c>From</c> is the source address. If UDP over IP is - used, this should be a 2-tuple <c>{IP, UDPport}</c>, where - <c>IP</c> is a 4-tuple with the IP address, and <c>UDPport</c> - is an integer. + <item><c>From</c> is the source <c>Address</c>. </item> <item><c>Extra</c> is any term the Net if process wishes to send to the agent. This term can be retrieved by the @@ -127,10 +131,7 @@ Pid ! {snmp_response_received, Vsn, Pdu, From} </item> <item><c>Pdu</c> is the SNMP Pdu received </item> - <item><c>From</c> is the source address. If UDP over IP is - used, this should be a 2-tuple <c>{IP, UDPport}</c>, where - <c>IP</c> is a 4-tuple with the IP address, and <c>UDPport</c> - is an integer. + <item><c>From</c> is the source <c>Address</c>. </item> </list> </section> @@ -168,10 +169,9 @@ Pid ! {snmp_response_received, Vsn, Pdu, From} (see Reference Manual). </p> </item> <item> - <p><c>To</c> is the destination address. If UDP over IP - is used, this should be a 2-tuple <c>{IP, UDPport}</c>, - where <c>IP</c> is a 4-tuple with the IP address, and - <c>UDPport</c> is an integer. </p> + <p><c>To</c> is the destination <c>Address</c> that comes + from the <c>From</c> field in the corresponding <c>snmp_pdu</c> + message previously sent to the MasterAgent.</p> </item> <item> <p><c>Extra</c> is the term that the Net if process @@ -230,7 +230,8 @@ Pid ! {snmp_response_received, Vsn, Pdu, From} SNMPv3, it is the context information. </p> </item> <item> - <p><c>To</c> is a list of the destination addresses and + <p><c>To</c> is a list of <c>{Address, SecData}</c> + tuples i.e the destination addresses and their corresponding security parameters. This value is normally sent to <c>snmpa_mpd:generate_message/4</c>. </p> </item> @@ -268,7 +269,8 @@ Pid ! {snmp_response_received, Vsn, Pdu, From} SNMPv3, it is the context information. </p> </item> <item> - <p><c>To</c> is a list of the destination addresses and + <p><c>To</c> is a list of <c>{Address, SecData}</c> + tuples i.e the destination addresses and their corresponding security parameters. This value is normally sent to <c>snmpa_mpd:generate_message/4</c>. </p> </item> diff --git a/lib/snmp/doc/src/snmp_manager_config_files.xml b/lib/snmp/doc/src/snmp_manager_config_files.xml index 486ef7c170..d8bd4b0f3a 100644 --- a/lib/snmp/doc/src/snmp_manager_config_files.xml +++ b/lib/snmp/doc/src/snmp_manager_config_files.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2004</year><year>2013</year> + <year>2004</year><year>2014</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -64,13 +64,42 @@ <item> <p><c>Variable</c> is one of the following:</p> <list type="bulleted"> - <item> - <p><c>address</c> - which defines the IP address of the - manager. Default is local host.</p> - </item> + <item> + <p><c>transports</c> - which defines the transport domains + and their addresses for the manager. <em>Mandatory</em> + </p> + <p><c>Value</c> is a list of <c>{Domain, Addr}</c> tuples + or <c>Domain</c> atoms. + </p> + <list type="bulleted"> + <item> + <p><c>Domain</c> is one of <c>transportDomainUdpIpv4</c> + or <c>transportDomainUdpIpv6</c>.</p> + </item> + <item> + <p><c>Addr</c> is for the currently supported domains + either an <c>IpAddr</c> or an <c>{IpAddr, IpPort}</c> + tuple.<c>IpAddr</c> is either a regular Erlang/OTP + <seealso marker="kernel:inet#type-ip_address"> + <c>ip_address()</c></seealso> or a traditional SNMP integer list + and <c>IpPort</c> is an integer. + </p> + <p>When <c>Addr</c> does not contain a port number, + the value of <c>port</c> is used. + </p> + <p>When a <c>Addr</c> is not specified i.e by + using only a <c>Domain</c> atom, the host's name + is resolved to find the IP address, and the value of + <c>port</c> is used. + </p> + </item> + </list> + </item> <item> <p><c>port</c> - which defines which UDP port the manager uses - for communicating with agents. <em>Mandatory</em>.</p> + for communicating with agents. + <em>Mandatory</em> if <c>transports</c> does not define + a port number for every transport.</p> </item> <item> <p><c>engine_id</c> - The <c>SnmpEngineID</c> as defined in @@ -87,11 +116,13 @@ </p> </item> </list> + <p>The legacy and intermediate variables <c>address</c> and <c>domain</c> + are still supported so old configurations will work.</p> <p>The following example shows a <c>manager.conf</c> file: </p> <pre> -{address, [141,213,11,24]}. -{port, 5000}. +{transports, [{transportDomainUdpIpv4, {{141,213,11,24}, 5000}}, + {transportDomainUdpIpv6, {{0,0,0,0,0,0,0,1}, 5000}}]}. {engine_id, "mgrEngine"}. {max_message_size, 484}. </pre> @@ -146,7 +177,7 @@ </p> <p>Each entry is a tuple: </p> - <p><c>{UserId, TargetName, Comm, Ip, Port, EngineID, Timeout, MaxMessageSize, Version, SecModel, SecName, SecLevel}.</c></p> + <p><c>{UserId, TargetName, Comm, Domain, Addr, EngineID, Timeout, MaxMessageSize, Version, SecModel, SecName, SecLevel}.</c></p> <list type="bulleted"> <item> <p><c>UserId</c> is the identity of the <em>manager user</em> @@ -160,10 +191,17 @@ <p><c>Comm</c> is the community string (string).</p> </item> <item> - <p><c>Ip</c> is the ip address of the agent (a list of four integers).</p> + <p><c>Domain</c> is the transport domain, either + <c>transportDomainUdpIpv4</c> or <c>transportDomainUdpIpv6</c>.</p> </item> <item> - <p><c>Port</c> is the port number of the agent (integer).</p> + <p><c>Addr</c> is the address in the transport domain, + either an <c>{IpAddr, IpPort}</c> tuple or a traditional SNMP + integer list containing port number. <c>IpAddr</c> is either + a regular Erlang/OTP + <seealso marker="kernel:inet#type-ip_address"><c>ip_address()</c></seealso> + or a traditional SNMP integer list not containing port number, + and <c>IpPort</c> is an integer.</p> </item> <item> <p><c>EngineID</c> is the engine-id of the agent (string).</p> @@ -190,6 +228,9 @@ authPriv).</p> </item> </list> + <p>Legacy configurations using tuples without <c>Domain</c> element, + as well as with all <c>TDomain</c>, <c>Ip</c> and <c>Port</c> elements + still work.</p> </section> <section> diff --git a/lib/snmp/doc/src/snmp_manager_netif.xml b/lib/snmp/doc/src/snmp_manager_netif.xml index 757ed32880..97cedf00c0 100644 --- a/lib/snmp/doc/src/snmp_manager_netif.xml +++ b/lib/snmp/doc/src/snmp_manager_netif.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2004</year><year>2013</year> + <year>2004</year><year>2014</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -50,13 +50,14 @@ <p>The snmp application provides two different modules, <c>snmpm_net_if</c> (the default) and <c>snmpm_net_if_mt</c>, - both uses the UDP as the transport protocol. The difference - between the two modules is that the latter is "multi-threaded", - i.e. for each message/request a new process is created that - process the message/request and then exits. </p> + both uses UDP as the transport protocol i.e the transport domains + <c>transportDomainUdpIpv4</c> and/or <c>transportDomainUdpIpv6</c>. + The difference between the two modules is that the latter is + "multi-threaded", i.e. for each message/request a new process + is created that processes the message/request and then exits. </p> - <p>It is also possible to write your own Net if process, - this section describes how to write a Net if processdo that.</p> + <p>It is also possible to write your own Net if process and + this section describes how to do that.</p> <section> <marker id="mandatory_functions"></marker> @@ -70,11 +71,17 @@ <p>The section <em>Messages</em> describes mandatory messages, which Net if must send to the manager server process. </p> + <p>In this section a <c>Domain</c> field is the transport domain i.e + one of <c>transportDomainUdpIpv4</c> or <c>transportDomainUdpIpv6</c>, + and an <c>Addr</c> field is an + <c>{<seealso marker="kernel:inet#type-ip_address">IpAddr</seealso>, + IpPort}</c> tuple.</p> + <p>Net if must send the following message when it receives an SNMP PDU from the network that is aimed for the MasterAgent: </p> <pre> -Server ! {snmp_pdu, Pdu, Addr, Port} +Server ! {snmp_pdu, Pdu, Domain, Addr} </pre> <list type="bulleted"> <item> @@ -82,14 +89,14 @@ Server ! {snmp_pdu, Pdu, Addr, Port} <c>snmp_types.hrl</c>, with the SNMP request.</p> </item> <item> - <p><c>Addr</c> is the source address. </p> + <p><c>Domain</c> is the source transport domain. </p> </item> <item> - <p><c>Port</c> is port number of the sender.</p> + <p><c>Addr</c> is the source address. </p> </item> </list> <pre> -Server ! {snmp_trap, Trap, Addr, Port} +Server ! {snmp_trap, Trap, Domain, Addr} </pre> <list type="bulleted"> <item> @@ -97,14 +104,14 @@ Server ! {snmp_trap, Trap, Addr, Port} as defined in <c>snmp_types.hrl</c>, with the SNMP request.</p> </item> <item> - <p><c>Addr</c> is the source address. </p> + <p><c>Domain</c> is the source transport domain. </p> </item> <item> - <p><c>Port</c> is port number of the sender.</p> + <p><c>Addr</c> is the source address. </p> </item> </list> <pre> -Server ! {snmp_inform, Ref, Pdu, PduMS, Addr, Port} +Server ! {snmp_inform, Ref, Pdu, PduMS, Domain, Addr} </pre> <list type="bulleted"> <item> @@ -123,14 +130,14 @@ Server ! {snmp_inform, Ref, Pdu, PduMS, Addr, Port} <c>snmp_types.hrl</c>, with the SNMP request.</p> </item> <item> - <p><c>Addr</c> is the source address. </p> + <p><c>Domain</c> is the source transport domain. </p> </item> <item> - <p><c>Port</c> is port number of the sender.</p> + <p><c>Addr</c> is the source address. </p> </item> </list> <pre> -Server ! {snmp_report, Data, Addr, Port} +Server ! {snmp_report, Data, Domain, Addr} </pre> <list type="bulleted"> <item> @@ -152,10 +159,10 @@ Server ! {snmp_report, Data, Addr, Port} <p><c>ReasonInfo</c> is a term().</p> </item> <item> - <p><c>Addr</c> is the source address. </p> + <p><c>Domain</c> is the source transport domain. </p> </item> <item> - <p><c>Port</c> is port number of the sender.</p> + <p><c>Addr</c> is the source address. </p> </item> </list> diff --git a/lib/snmp/doc/src/snmp_target_mib.xml b/lib/snmp/doc/src/snmp_target_mib.xml index be6fa15c73..a076ff2d8e 100644 --- a/lib/snmp/doc/src/snmp_target_mib.xml +++ b/lib/snmp/doc/src/snmp_target_mib.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1998</year><year>2013</year> + <year>1998</year><year>2014</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -38,18 +38,18 @@ functions for the SNMP-TARGET-MIB, and functions for configuring the database. </p> <p>The configuration files are described in the SNMP User's Manual.</p> + <p>Legacy API functions <c>add_addr/10</c> that does not specify + transport domain, and <c>add_addr/11</c> that has got separate + <c>IpAddr</c> and <c>PortNumber</c> arguments still work as before + for backwards compatibility reasons.</p> <marker id="types"></marker> </description> <section> <title>DATA TYPES</title> - <code type="none"><![CDATA[ -transportDomain() = transportDomainUdpIpv4 | transportDomainUdpIpv6 -transportAddressIPv4() = [integer()], length 4 -transportAddressIPv6() = [integer()], length 8 -transportAddressMask() = [integer()], length 0 (default), 6 (IPv4) or 10 (IPv6) - ]]></code> + <p>See the <seealso marker="snmpa_conf#types"> + data types in <c>snmpa_conf</c></seealso>.</p> <marker id="configure"></marker> </section> @@ -129,20 +129,18 @@ transportAddressMask() = [integer()], length 0 (default), 6 (IPv4) or 10 (IPv6) </func> <func> - <name>add_addr(Name, Ip, Port, Timeout, Retry, TagList, Params, EngineId, TMask, MMS) -> Ret</name> - <name>add_addr(Name, Domain, Ip, Port, Timeout, Retry, TagList, Params, EngineId, TMask, MMS) -> Ret</name> + <name>add_addr(Name, Domain, Addr, Timeout, Retry, TagList, Params, EngineId, TMask, MMS) -> Ret</name> <fsummary>Add one target address definition</fsummary> <type> <v>Name = string()</v> <v>Domain = transportDomain()</v> - <v>Ip = transportAddressIPv4() | transportAddressIPv6() (depends on the value of Domain)</v> - <v>Port = integer()</v> + <v>Addr = transportAddress() % Default port is 162</v> <v>Timeout = integer()</v> <v>Retry = integer()</v> <v>TagList = string()</v> <v>ParamsName = string()</v> <v>EngineId = string()</v> - <v>TMask = transportAddressMask() (depends on Domain)</v> + <v>TMask = transportAddressMask() % Depends on Domain</v> <v>MMS = integer()</v> <v>Ret = {ok, Key} | {error, Reason}</v> <v>Key = term()</v> diff --git a/lib/snmp/doc/src/snmpa_conf.xml b/lib/snmp/doc/src/snmpa_conf.xml index 99a56cd601..2780cec156 100644 --- a/lib/snmp/doc/src/snmpa_conf.xml +++ b/lib/snmp/doc/src/snmpa_conf.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>2006</year><year>2013</year> + <year>2006</year><year>2014</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -45,20 +45,56 @@ <title>DATA TYPES</title> <code type="none"><![CDATA[ transportDomain() = transportDomainUdpIpv4 | transportDomainUdpIpv6 -transportAddressIPv4() = [integer()], length 4 -transportAddressIPv6() = [integer()], length 8 -transportAddressMask() = [integer()], length 0 (default), 6 (IPv4) or 10 (IPv6) + +transportAddress() = + transportAddressIPv4() | transportAddressIPv6() + +transportAddressWithPort() = + transportAddressIPv4WithPort() | transportAddressIPv6WithPort() + +transportAddressWithoutPort() = + transportAddressIPv4WithoutPort() | transportAddressIPv6WithoutPort() + +transportAddressIPv4() = + transportAddressIPv4WithPort() | transportAddressIPv4WithoutPort() +transportAddressIPv4WithPort = + {transportAddressIPv4WithoutPort(), inet:port_number()} | + [byte() x 4, byte() x 2] +transportAddressIPv4WithoutPort = + inet:ip4_address() | [byte() x 4] + +transportAddressIPv6() = + transportAddressIPv6WithPort() | transportAddressIPv6WithoutPort() +transportAddressIPv6WithPort = + {transportAddressIPv6WithoutPort(), inet:port_number()} | + [word() x 8, inet:port_number()] | + [word() x 8, byte() x 2] | + {byte() x 16, byte() x 2] +transportAddressIPv6WithoutPort = + inet:ip6_address() | [word() x 8] | [byte() x 16] + +transportAddressMask() = + [] | transportAddressWithPort() + +byte() = 0..255 +word() = 0..65535 ]]></code> + <p>For <c>inet:ip4_address()</c>, <c>inet:ip6_address()</c> + and <c>inet:port_number()</c>, see also + <seealso marker="kernel:inet#type-ip_address"> + <c>inet:ip_address()</c></seealso></p> <marker id="agent_entry"></marker> </section> + + <funcs> <func> <name>agent_entry(Tag, Val) -> agent_entry()</name> <fsummary>Create an agent entry</fsummary> <type> - <v>Tag = intAgentIpAddress | intAgentUDPPort | intAgentMaxPacketSize | snmpEngineMaxMessageSize | snmpEngineID</v> + <v>Tag = intAgentTransports | intAgentUDPPort | intAgentMaxPacketSize | snmpEngineMaxMessageSize | snmpEngineID</v> <v>Val = term()</v> <v>agent_entry() = term()</v> </type> @@ -390,17 +426,15 @@ transportAddressMask() = [integer()], length 0 (default), 6 (IPv4) or 10 (IPv6) </func> <func> - <name>target_addr_entry(Name, Ip, TagList, ParamsName, EngineId) -> target_addr_entry()</name> - <name>target_addr_entry(Name, Ip, TagList, ParamsName, EngineId, TMask) -> target_addr_entry()</name> - <name>target_addr_entry(Name, Ip, Udp, TagList, ParamsName, EngineId, TMask, MaxMessageSize) -> target_addr_entry()</name> - <name>target_addr_entry(Name, Ip, Udp, Timeout, RetryCount, TagList, ParamsName, EngineId, TMask, MaxMessageSize) -> target_addr_entry()</name> - <name>target_addr_entry(Name, Domain, Ip, Udp, Timeout, RetryCount, TagList, ParamsName, EngineId, TMask, MaxMessageSize) -> target_addr_entry()</name> + <name>target_addr_entry(Name, Domain, Addr, TagList, ParamsName, EngineId) -> target_addr_entry()</name> + <name>target_addr_entry(Name, Domain, Addr, TagList, ParamsName, EngineId, TMask) -> target_addr_entry()</name> + <name>target_addr_entry(Name, Domain, Addr, TagList, ParamsName, EngineId, TMask, MaxMessageSize) -> target_addr_entry()</name> + <name>target_addr_entry(Name, Domain, Addr, Timeout, RetryCount, TagList, ParamsName, EngineId, TMask, MaxMessageSize) -> target_addr_entry()</name> <fsummary>Create an target_addr entry</fsummary> <type> <v>Name = string()</v> <v>Domain = transportDomain()</v> - <v>Ip = transportAddressIPv4() | transportAddressIPv6() (depends on Domain)</v> - <v>Udp = integer()</v> + <v>Ip = transportAddress() (depends on Domain)</v> <v>Timeout = integer()</v> <v>RetryCount = integer()</v> <v>TagList = string()</v> @@ -414,12 +448,12 @@ transportAddressMask() = [integer()], length 0 (default), 6 (IPv4) or 10 (IPv6) <p>Create an entry for the agent target_addr config file, <c>target_addr.conf</c>. </p> <p><c>Name</c> must be a <em>non-empty</em> string. </p> - <p><c>target_addr_entry/5</c> translates to the following call: - <c>target_addr_entry(Name, Ip, TagList, ParamsName, EngineId)</c>. </p> <p><c>target_addr_entry/6</c> translates to the following call: - <c>target_addr_entry(Name, Ip, 162, TagList, ParamsName, EngineId, TMask, 2048)</c>. </p> + <c>target_addr_entry(Name, Domain, Addr, TagList, ParamsName, EngineId, [])</c>. </p> + <p><c>target_addr_entry/7</c> translates to the following call: + <c>target_addr_entry(Name, Domain, Addr, TagList, ParamsName, EngineId, TMask, 2048)</c>. </p> <p><c>target_addr_entry/8</c> translates to the following call: - <c>target_addr_entry(Name, Ip, Udp, 1500, 3, TagList, ParamsName, EngineId, TMask, MaxMessageSize)</c>. </p> + <c>target_addr_entry(Name, Domain, Addr, 1500, 3, TagList, ParamsName, EngineId, TMask, MaxMessageSize)</c>. </p> <p>See <seealso marker="snmp_agent_config_files#target_addr">Target Address Definitions</seealso> for more info. </p> diff --git a/lib/snmp/doc/src/snmpa_mpd.xml b/lib/snmp/doc/src/snmpa_mpd.xml index c5ab0a0520..518100d30c 100644 --- a/lib/snmp/doc/src/snmpa_mpd.xml +++ b/lib/snmp/doc/src/snmpa_mpd.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1999</year><year>2013</year> + <year>1999</year><year>2014</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -43,6 +43,12 @@ <marker id="init"></marker> </description> + <section> + <title>DATA TYPES</title> + <p>See the <seealso marker="snmpa_conf#types"> + data types in <c>snmpa_conf</c></seealso>.</p> + </section> + <funcs> <func> <name>init(Vsns) -> mpd_state()</name> @@ -63,16 +69,17 @@ </func> <func> - <name>process_packet(Packet, TDomain, TAddress, State, NoteStore, Log) -> {ok, Vsn, Pdu, PduMS, ACMData} | {discarded, Reason} | {discovery, DiscoPacket}</name> - <name>process_packet(Packet, TDomain, TAddress, LocalEngineID, State, NoteStore, Log) -> {ok, Vsn, Pdu, PduMS, ACMData} | {discarded, Reason} | {discovery, DiscoPacket}</name> + <name>process_packet(Packet, From, State, NoteStore, Log) -> {ok, Vsn, Pdu, PduMS, ACMData} | {discarded, Reason} | {discovery, DiscoPacket}</name> + <name>process_packet(Packet, From, LocalEngineID, State, NoteStore, Log) -> {ok, Vsn, Pdu, PduMS, ACMData} | {discarded, Reason} | {discovery, DiscoPacket}</name> <fsummary>Process a packet received from the network</fsummary> <type> <v>Packet = binary()</v> - <v>TDomain = snmpUDPDomain</v> - <v>TAddress = {Ip, Udp}</v> + <v>From = {TDomain, TAddr}</v> + <v>TDomain = transportDomainUdpIpv4 | transportDomainUdpIpv6</v> + <v>TAddr = {IpAddr, IpPort}</v> <v>LocalEngineID = string()</v> - <v>Ip = {integer(), integer(), integer(), integer()}</v> - <v>Udp = integer()</v> + <v>IpAddr = <seealso marker="kernel:inet#type-ip_address">inet:ip_address()</seealso></v> + <v>IpPort = inet:port_number()</v> <v>State = mpd_state()</v> <v>NoteStore = pid()</v> <v>Log = snmp_log()</v> @@ -85,7 +92,7 @@ </type> <desc> <p>Processes an incoming packet. Performs authentication and - decryption as necessary. The return values should be passed the + decryption as necessary. The return values should be passed to the agent.</p> <note> @@ -150,14 +157,20 @@ network. </p> <p><c>MsgData</c> is the message specific data used in - the SNMP message. This value is received in a <c>send_pdu</c> - or <c>send_pdu_req</c> message from the agent. In SNMPv1 and + the SNMP message. This value is received in a + <seealso marker="snmp_agent_netif#im_send_pdu"><c>send_pdu</c></seealso> + or + <seealso marker="snmp_agent_netif#im_send_pdu_req"> + <c>send_pdu_req</c></seealso> + message from the agent. In SNMPv1 and SNMPv2c, this message data is the community string. In - SNMPv3, it is the context information. - <c>To</c> is a list of the destination addresses and + SNMPv3, it is the context information.</p> + <p> + <c>To</c> is a list of destination addresses and their corresponding security parameters. This value is - also received from the requests mentioned above. - </p> + received in the same message from the agent and then transformed + trough <seealso marker="#process_taddrs"><c>process_taddrs</c></seealso> + before passed to this function.</p> <note> <p>Note that the use of the LocalEngineID argument is only intended @@ -166,6 +179,32 @@ (see SNMP-FRAMEWORK-MIB). </p> </note> + <marker id="process_taddrs"></marker> + </desc> + </func> + + <func> + <name>process_taddrs(TDests) -> Dests</name> + <fsummary>Transform addresses from internal MIB format to a less internal + </fsummary> + <type> + <v>TDests = [TDest]</v> + <v>TDest = {{TDomain, TAddr}, SecData} | {TDomain, TAddr}</v> + <v>TDomain = term() % Not at tuple</v> + <v>TAddr = term()</v> + <v>SecData = term()</v> + <v>Dests = [Dest]</v> + <v>Dest = {{Domain, Addr}, SecData} | {Domain, Addr}</v> + <v>Domain = transportDomain()</v> + <v>Addr = transportAddress() % Depends on Domain</v> + </type> + <desc> + <p>Transforms addresses from internal MIB format to one + more useful to <seealso marker="snmp_agent_netif">Agent Net if</seealso>. + </p> + <p>See also <seealso marker="#generate_msg"><c>generate_msg</c>.</seealso> + </p> + <marker id="discarded_pdu"></marker> </desc> </func> diff --git a/lib/snmp/doc/src/snmpa_network_interface_filter.xml b/lib/snmp/doc/src/snmpa_network_interface_filter.xml index e08a26ed92..eb640e1bc3 100644 --- a/lib/snmp/doc/src/snmpa_network_interface_filter.xml +++ b/lib/snmp/doc/src/snmpa_network_interface_filter.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>2007</year><year>2013</year> + <year>2007</year><year>2014</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -58,10 +58,10 @@ on two levels: </p> <list type="bulleted"> <item> - <p>The first level is at the UDP entry / exit point, i.e. - immediately after the receipt of the message, before any message + <p>The first level is at the transport entry / exit point, i.e. + immediately after the receipt of the message before any message processing is done (accept_recv) and - immediately before sending the message, after all message + immediately before sending the message after all message processing is done (accept_send).</p> </item> <item> @@ -78,6 +78,12 @@ <seealso marker="snmp_app#configuration_params">req_limit</seealso> and the function <seealso marker="snmpa#register_notification_filter">register_notification_filter</seealso>. </p> + <p>Legacy network interface filter modules used arguments on the form + <c>(IpAddr, PortNumber,...)</c> instead of + <c>(Domain, Addr, ...)</c>, and if the SNMP agent is run without + changing the configuration to use transport domains + the network interface filter will still get + the old arguments and work as before.</p> </description> <section> @@ -88,15 +94,17 @@ pdu_type() = 'get-request' | 'get-next-request' | 'get-response' | 'set-request' | trap | 'get-bulk-request' | 'inform-request' | report </code> + <p>See also the <seealso marker="snmpa_conf#types"> + data types in <c>snmpa_conf</c></seealso>.</p> <marker id="accept_recv"></marker> </section> <funcs> <func> - <name>accept_recv(Ip, Port) -> boolean()</name> + <name>accept_recv(Domain, Addr) -> boolean()</name> <fsummary>Shall the received message be accepted</fsummary> <type> - <v>Ip = ip_address()</v> - <v>Port = port()</v> + <v>Domain = transportDomain()</v> + <v>Addr = transportAddressWithPort()</v> </type> <desc> <p>Called at the reception of a message (before <em>any</em> processing @@ -107,11 +115,11 @@ pdu_type() = 'get-request' | 'get-next-request' | 'get-response' | </desc> </func> <func> - <name>accept_send(Ip, Port) -> boolean()</name> + <name>accept_send(Domain, Addr) -> boolean()</name> <fsummary>Shall the message be sent</fsummary> <type> - <v>Ip = ip_address()</v> - <v>Port = port()</v> + <v>Domain = transportDomain()</v> + <v>Addr = transportAddressWithPort()</v> </type> <desc> <p>Called before the sending of a message (after <em>all</em> processing @@ -122,11 +130,11 @@ pdu_type() = 'get-request' | 'get-next-request' | 'get-response' | </desc> </func> <func> - <name>accept_recv_pdu(Ip, Port, PduType) -> boolean()</name> + <name>accept_recv_pdu(Domain, Addr, PduType) -> boolean()</name> <fsummary>Shall the received pdu be accepted</fsummary> <type> - <v>Ip = ip_address()</v> - <v>Port = port()</v> + <v>Domain = transportDomain()</v> + <v>Addr = transportAddressWithPort()</v> <v>PduType = pdu_type()</v> </type> <desc> @@ -144,7 +152,9 @@ pdu_type() = 'get-request' | 'get-next-request' | 'get-response' | <type> <v>Targets = targets()</v> <v>targets() = [target()]</v> - <v>target() = {ip_address(), port()}</v> + <v>target() = {Domain, Addr}</v> + <v>Domain = transportDomain()</v> + <v>Addr = transportAddressWithPort()</v> <v>PduType = pdu_type() > 0</v> <v>Reply = boolean() | NewTargets</v> <v>NewTargets = targets()</v> @@ -155,7 +165,7 @@ pdu_type() = 'get-request' | 'get-next-request' | 'get-response' | <p>For the message to be discarded all together, the function <em>must</em> return <em>false</em>. </p> <p>Note that it is possible for this function to filter out targets - (but <em>not</em> add its own) by returning an updated + (but <em>not</em> to add its own) by returning an updated <c>Targets</c> list (<c>NewTargets</c>). </p> </desc> </func> diff --git a/lib/snmp/doc/src/snmpa_notification_delivery_info_receiver.xml b/lib/snmp/doc/src/snmpa_notification_delivery_info_receiver.xml index aff71688b6..814f02a14c 100644 --- a/lib/snmp/doc/src/snmpa_notification_delivery_info_receiver.xml +++ b/lib/snmp/doc/src/snmpa_notification_delivery_info_receiver.xml @@ -5,7 +5,7 @@ <header> <copyright> <year>2008</year> - <year>2013</year> + <year>2014</year> <holder>Ericsson AB, All Rights Reserved</holder> </copyright> <legalnotice> @@ -45,22 +45,30 @@ must export the following functions: </p> <list type="bulleted"> <item> - <p><seealso marker="#delivery_targets">delivery_targets/3</seealso></p> + <p><seealso marker="#delivery_targets/3">delivery_targets/3</seealso></p> </item> <item> - <p><seealso marker="#delivery_info">delivery_info/4</seealso></p> + <p><seealso marker="#delivery_info/4">delivery_info/4</seealso></p> </item> </list> <p>The semantics of them and their exact signatures are explained below. </p> + <p>Legacy notification delivery information receiver modules + used a target argument on the form + <c>{IpAddr, PortNumber}</c> instead of + <c>{Domain, Addr}</c>, and if the SNMP Agent is run without + changing the configuration to use transport domains + the notification delivery information receiver will still get + the old arguments and work as before.</p> + </description> <section> <title>DATA TYPES</title> - <code type="none"><![CDATA[ -address() = A 4-tuple - ]]></code> + <p>See the <seealso marker="snmpa_conf#types"> + data types in <c>snmpa_conf</c></seealso>.</p> + <marker id="accept_recv"></marker> <marker id="delivery_targets"></marker> </section> @@ -71,10 +79,8 @@ address() = A 4-tuple <fsummary>Inform about target addresses</fsummary> <type> <v>Tag = term()</v> - <v>Targets = [target()]</v> - <v>target() = {Address, Port}</v> - <v>Address = address()</v> - <v>Port = integer()</v> + <v>Targets = [Target]</v> + <v>Target = {transportDomain(), transportAddressWithPort()</v> <v>Extra = term()</v> </type> <desc> @@ -94,10 +100,8 @@ address() = A 4-tuple <fsummary>Inform about delivery result</fsummary> <type> <v>Tag = term()</v> - <v>Target = target()</v> - <v>target() = {Address, Port}</v> - <v>Address = address()</v> - <v>Port = integer()</v> + <v>Targets = [Target]</v> + <v>Target = {transportDomain(), transportAddressWithPort()</v> <v>DeliveryResult = delivery_result()</v> <v>delivery_result() = no_response | got_response</v> <v>Extra = term()</v> diff --git a/lib/snmp/doc/src/snmpm.xml b/lib/snmp/doc/src/snmpm.xml index dc8226bb87..ff90e49968 100644 --- a/lib/snmp/doc/src/snmpm.xml +++ b/lib/snmp/doc/src/snmpm.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>2004</year><year>2013</year> + <year>2004</year><year>2014</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -69,6 +69,9 @@ sec_name() = string() sec_level() = noAuthNoPriv | authNoPriv | authPriv ]]></code> + <p>See also the <seealso marker="snmpa_conf#types"> + data types in <c>snmpa_conf</c></seealso>.</p> + <marker id="monitor"></marker> </section> <funcs> @@ -300,9 +303,9 @@ sec_level = noAuthNoPriv | authNoPriv | authPriv <p>The type of <c>Val</c> depends on <c>Item</c>: </p> <code type="none"><![CDATA[ [mandatory] engine_id = string() -[mandatory] address = ip_address() -[optional] port = integer() -[optional] tdomain = transportDomainUdpIpv4 | transportDomainUdpIpv6 +[mandatory] tadress = transportAddress() % Depends on tdomain +[optional] port = inet:port_number() +[optional] tdomain = transportDomain() [optional] community = string() [optional] timeout = integer() | snmp_timer() [optional] max_message_size = integer() @@ -313,7 +316,8 @@ sec_level = noAuthNoPriv | authNoPriv | authPriv ]]></code> <p>Note that if no <c>tdomain</c> is given, the default value, <c>transportDomainUdpIpv4</c>, is used.</p> - <p>Note that if no <c>port</c> is given, the default value is used.</p> + <p>Note that if no <c>port</c> is given and if <c>taddress</c> does not + contain a port number, the default value is used.</p> <marker id="unregister_agent"></marker> </desc> diff --git a/lib/snmp/doc/src/snmpm_conf.xml b/lib/snmp/doc/src/snmpm_conf.xml index 0cc9ff3379..8635fb705b 100644 --- a/lib/snmp/doc/src/snmpm_conf.xml +++ b/lib/snmp/doc/src/snmpm_conf.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>2006</year><year>2013</year> + <year>2006</year><year>2014</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -195,14 +195,14 @@ </desc> </func> <func> - <name>agents_entry(UserId, TargetName, Comm, Ip, Port, EngineID, Timeout, MaxMessageSize, Version, SecModel, SecName, SecLevel) -> agents_entry()</name> + <name>agents_entry(UserId, TargetName, Comm, Domain, Addr, EngineID, Timeout, MaxMessageSize, Version, SecModel, SecName, SecLevel) -> agents_entry()</name> <fsummary>Create an agents entry</fsummary> <type> <v>UserId = term()</v> <v>TargetName = string()</v> <v>Comm = string()</v> - <v>Ip = string()</v> - <v>Port = integer()</v> + <v>Domain = transportDomain()</v> + <v>Addr = transportAddress()</v> <v>EngineID = string()</v> <v>Timeout = integer()</v> <v>MaxMessageSize = integer()</v> diff --git a/lib/snmp/doc/src/snmpm_mpd.xml b/lib/snmp/doc/src/snmpm_mpd.xml index ad72fd7bc0..c23b2b6833 100644 --- a/lib/snmp/doc/src/snmpm_mpd.xml +++ b/lib/snmp/doc/src/snmpm_mpd.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>2004</year><year>2013</year> + <year>2004</year><year>2014</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -39,7 +39,12 @@ It is supposed to be used from a Network Interface process (<seealso marker="snmp_manager_netif">Definition of Manager Net if</seealso>). </p> + + <p>Legacy API function <c>process_msg/7</c> that has got separate + <c>IpAddr</c> and <c>PortNumber</c> arguments still works as before + for backwards compatibility reasons.</p> </description> + <funcs> <func> <name>init_mpd(Vsns) -> mpd_state()</name> @@ -58,13 +63,12 @@ </desc> </func> <func> - <name>process_msg(Msg, TDomain, Addr, Port, State, NoteStore, Logger) -> {ok, Vsn, Pdu, PduMS, MsgData} | {discarded, Reason}</name> + <name>process_msg(Msg, Domain, Addr, State, NoteStore, Logger) -> {ok, Vsn, Pdu, PduMS, MsgData} | {discarded, Reason}</name> <fsummary>Process a message received from the network</fsummary> <type> <v>Msg = binary()</v> - <v>TDomain = snmpUDPDomain</v> - <v>Addr = {integer(), integer(), integer(), integer()}</v> - <v>Port = integer()</v> + <v>Domain = transportDomainUdpIpv4 | transportDomainUdpIpv6</v> + <v>Addr = {<seealso marker="kernel:inet#type-ip_address">inet:ip_address(), inet:port_number()</seealso>} </v> <v>State = mpd_state()</v> <v>NoteStore = pid()</v> <v>Logger = function()</v> diff --git a/lib/snmp/doc/src/snmpm_network_interface.xml b/lib/snmp/doc/src/snmpm_network_interface.xml index 6cf7bd6ed7..bea6b46dc7 100644 --- a/lib/snmp/doc/src/snmpm_network_interface.xml +++ b/lib/snmp/doc/src/snmpm_network_interface.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>2004</year><year>2013</year> + <year>2004</year><year>2014</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -69,6 +69,10 @@ <p>The semantics of them and their exact signatures are explained below. </p> + <p>Legacy API function <c>send_pdu/7</c> that has got separate + <c>IpAddr</c> and <c>PortNumber</c> arguments still works as before + for backwards compatibility reasons.</p> + <marker id="start_link"></marker> </description> @@ -103,15 +107,15 @@ </func> <func> - <name>send_pdu(Pid, Pdu, Vsn, MsgData, Addr, Port, ExtraInfo) -> void()</name> + <name>send_pdu(Pid, Pdu, Vsn, MsgData, Domain, Addr, ExtraInfo) -> void()</name> <fsummary>Request the network interface process to send this pdu</fsummary> <type> <v>Pid = pid()</v> <v>Pdu = pdu()</v> <v>Vsn = 'version-1' | 'version-2' | 'version-3'</v> <v>MsgData = term()</v> - <v>Addr = address()</v> - <v>Port = integer()</v> + <v>Domain = transportDomainUdpIpv4 | transportDomainUdpIpv6</v> + <v>Addr = {<seealso marker="kernel:inet#type-ip_address">inet:ip_address(), inet:port_number()</seealso>} </v> <v>ExtraInfo = term()</v> </type> <desc> diff --git a/lib/snmp/doc/src/snmpm_network_interface_filter.xml b/lib/snmp/doc/src/snmpm_network_interface_filter.xml index f0526269b3..1ef4f29c0f 100644 --- a/lib/snmp/doc/src/snmpm_network_interface_filter.xml +++ b/lib/snmp/doc/src/snmpm_network_interface_filter.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>2007</year><year>2013</year> + <year>2007</year><year>2014</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -76,6 +76,12 @@ The default filter accepts all messages.</p> <p>A network interface filter can e.g. be used during testing or for load regulation. </p> + <p>Legacy network interface filter modules used arguments on the form + <c>(IpAddr, PortNumber,...)</c> instead of + <c>(Domain, Addr, ...)</c>, and if the SNMP manager is run without + changing the configuration to use transport domains + the network interface filter will still get + the old arguments and work as before.</p> </description> <section> @@ -86,16 +92,18 @@ pdu_type() = 'get-request' | 'get-next-request' | 'get-response' | 'set-request' | trap | 'get-bulk-request' | 'inform-request' | report | trappdu </code> + <p>See also the <seealso marker="snmpa_conf#types"> + data types in <c>snmpa_conf</c></seealso>.</p> <marker id="accept_recv"></marker> </section> <funcs> <func> - <name>accept_recv(Addr, Port) -> boolean()</name> + <name>accept_recv(Domain, Addr) -> boolean()</name> <fsummary>Shall the received message be accepted</fsummary> <type> - <v>Addr = ip_address()</v> - <v>Port = port()</v> + <v>Domain = transportDomain()</v> + <v>Addr = transportAddressWithPort()</v> </type> <desc> <p>Called at the reception of a message (before <em>any</em> processing @@ -107,11 +115,11 @@ pdu_type() = 'get-request' | 'get-next-request' | 'get-response' | </func> <func> - <name>accept_send(Addr, Port) -> boolean()</name> + <name>accept_send(Domain, Addr) -> boolean()</name> <fsummary>Shall the message be sent</fsummary> <type> - <v>Addr = ip_address()</v> - <v>Port = port()</v> + <v>Domain = transportDomain()</v> + <v>Addr = transportAddressWithPort()</v> </type> <desc> <p>Called before the sending of a message (after <em>all</em> processing @@ -123,11 +131,11 @@ pdu_type() = 'get-request' | 'get-next-request' | 'get-response' | </func> <func> - <name>accept_recv_pdu(Addr, Port, PduType) -> boolean()</name> + <name>accept_recv_pdu(Domain, Addr, PduType) -> boolean()</name> <fsummary>Shall the received pdu be accepted</fsummary> <type> - <v>Addr = ip_address()</v> - <v>Port = port()</v> + <v>Domain = transportDomain()</v> + <v>Addr = transportAddressWithPort()</v> <v>PduType = pdu_type()</v> </type> <desc> @@ -141,11 +149,11 @@ pdu_type() = 'get-request' | 'get-next-request' | 'get-response' | </func> <func> - <name>accept_send_pdu(Addr, Port, PduType) -> boolean()</name> + <name>accept_send_pdu(Domain, Addr, PduType) -> boolean()</name> <fsummary>Shall the pdu be sent</fsummary> <type> - <v>Addr = ip_address()</v> - <v>Port = port()</v> + <v>Domain = transportDomain()</v> + <v>Addr = transportAddressWithPort()</v> <v>PduType = pdu_type() > 0</v> </type> <desc> diff --git a/lib/snmp/doc/src/snmpm_user.xml b/lib/snmp/doc/src/snmpm_user.xml index 6f412d90f8..a4492839cd 100644 --- a/lib/snmp/doc/src/snmpm_user.xml +++ b/lib/snmp/doc/src/snmpm_user.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>2004</year><year>2013</year> + <year>2004</year><year>2014</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -63,10 +63,15 @@ <p>The semantics of them and their exact signatures are explained below. </p> <p>Some of the function has no defined return value (<c>void()</c>), - they can ofcourse return anythyng. But the functions that do have + they can of course return anything. But the functions that do have specified return value(s) <em>must</em> adhere to this. None of the functions can use exit of throw to return. </p> + <p>If the manager is not configured to use any particular + transport domain, the behaviour <c>handle_agent/4</c> + will for backwards copmpatibility reasons be called with the old + <c>IpAddr</c> and <c>PortNumber</c> arguments</p> + <marker id="types"></marker> </description> @@ -116,11 +121,11 @@ snmp_v1_trap_info() :: {Enteprise :: snmp:oid(), </func> <func> - <name>handle_agent(Addr, Port, Type, SnmpInfo, UserData) -> Reply</name> + <name>handle_agent(Domain, Addr, Type, SnmpInfo, UserData) -> Reply</name> <fsummary>Handle agent</fsummary> <type> - <v>Addr = ip_address()</v> - <v>Port = integer()</v> + <v>Domain = transportDomainUdpIpv4 | transportDomainUdpIpv6</v> + <v>Addr = {<seealso marker="kernel:inet#type-ip_address">inet:ip_address(), inet:port_number()</seealso>} </v> <v>Type = pdu | trap | report | inform</v> <v>SnmpInfo = SnmpPduInfo | SnmpTrapInfo | SnmpReportInfo | SnmpInformInfo</v> <v>SnmpPduInfo = snmp_gen_info()</v> diff --git a/lib/snmp/src/agent/snmp_community_mib.erl b/lib/snmp/src/agent/snmp_community_mib.erl index 7bdd500727..4546c343b7 100644 --- a/lib/snmp/src/agent/snmp_community_mib.erl +++ b/lib/snmp/src/agent/snmp_community_mib.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2012. All Rights Reserved. +%% Copyright Ericsson AB 1999-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -31,6 +31,7 @@ -include("snmpa_internal.hrl"). -include("SNMP-COMMUNITY-MIB.hrl"). -include("SNMP-TARGET-MIB.hrl"). +-include("SNMPv2-TM.hrl"). -include("SNMPv2-TC.hrl"). -include("snmp_types.hrl"). @@ -129,10 +130,11 @@ read_community_config_files(Dir) -> [FileName, D, Reason]), ok end, - Filter = fun(Comms) -> Comms end, - Check = fun(Entry) -> check_community(Entry) end, + Order = fun snmp_conf:no_order/2, + Filter = fun snmp_conf:no_filter/1, + Check = fun(Entry, State) -> {check_community(Entry), State} end, [Comms] = - snmp_conf:read_files(Dir, [{Gen, Filter, Check, "community.conf"}]), + snmp_conf:read_files(Dir, [{FileName, Gen, Order, Check, Filter}]), Comms. check_community({Index, CommunityName, SecName, CtxName, TransportTag}) -> @@ -192,7 +194,7 @@ add_community(Idx, CommName, SecName, EngineId, CtxName, TransportTag) -> do_add_community(Community). do_add_community(Community) -> - case (catch check_community(Community)) of + try check_community(Community) of {ok, Row} -> Key = element(1, Row), case table_cre_row(snmpCommunityTable, Key, Row) of @@ -201,11 +203,12 @@ do_add_community(Community) -> {ok, Key}; false -> {error, create_failed} - end; + end + catch {error, Reason} -> {error, Reason}; - Error -> - {error, Error} + Class:Reason -> + {error, {Class, Reason, erlang:get_stacktrace()}} end. %% FIXME: does not work with mnesia @@ -243,6 +246,10 @@ gc_tabs() -> %%----------------------------------------------------------------- community2vacm(Community, Addr) -> Idxs = ets:lookup(snmp_community_cache, Community), + ?vtrace("community2vacm ->~n" + " Community: ~p~n" + " Addr: ~p~n" + " Idxs: ~p", [Community, Addr, Idxs]), loop_c2v_rows(lists:keysort(2, Idxs), Addr). loop_c2v_rows([{_, CommunityIndex} | T], Addr) -> @@ -250,6 +257,9 @@ loop_c2v_rows([{_, CommunityIndex} | T], Addr) -> "~n CommunityIndex: ~p", [CommunityIndex]), case get_row(CommunityIndex) of {_Community, VacmParams, Tag} -> + ?vtrace("loop_c2v_rows ->~n" + " VacmParams: ~p~n" + " Tag: ~p", [VacmParams, Tag]), {TDomain, TAddr} = Addr, case snmp_target_mib:is_valid_tag(Tag, TDomain, TAddr) of true -> @@ -506,7 +516,12 @@ snmpTargetAddrExtTable(get_next, RowIndex, Cols) -> NCols = conv1(Cols), conv2(next(snmpTargetAddrExtTable, RowIndex, NCols)); snmpTargetAddrExtTable(set, RowIndex, Cols0) -> - case (catch verify_snmpTargetAddrExtTable_cols(Cols0, [])) of + case + (catch verify_snmpTargetAddrExtTable_cols( + Cols0, + get_snmpTargetAddrTDomain(RowIndex, Cols0), + [])) + of {ok, Cols} -> NCols = conv3(Cols), snmp_generic:table_func(set, RowIndex, NCols, @@ -515,7 +530,11 @@ snmpTargetAddrExtTable(set, RowIndex, Cols0) -> Error end; snmpTargetAddrExtTable(is_set_ok, RowIndex, Cols0) -> - case (catch verify_snmpTargetAddrExtTable_cols(Cols0, [])) of + case (catch verify_snmpTargetAddrExtTable_cols( + Cols0, + get_snmpTargetAddrTDomain(RowIndex, Cols0), + [])) + of {ok, Cols} -> NCols = conv3(Cols), snmp_generic:table_func(is_set_ok, RowIndex, NCols, @@ -525,29 +544,49 @@ snmpTargetAddrExtTable(is_set_ok, RowIndex, Cols0) -> end. -verify_snmpTargetAddrExtTable_cols([], Cols) -> + +get_snmpTargetAddrTDomain(RowIndex, Col) -> + case + get( + snmpTargetAddrTable, RowIndex, + [?snmpTargetAddrRowStatus,?snmpTargetAddrTDomain]) + of + [{value,?snmpTargetAddrRowStatus_active},ValueTDomain] -> + case ValueTDomain of + {value,TDomain} -> + TDomain; + _ -> + ?snmpUDPDomain + end; + _ -> + wrongValue(Col) + end. + + + +verify_snmpTargetAddrExtTable_cols([], _TDomain, Cols) -> {ok, lists:reverse(Cols)}; -verify_snmpTargetAddrExtTable_cols([{Col, Val0}|Cols], Acc) -> - Val = verify_snmpTargetAddrExtTable_col(Col, Val0), - verify_snmpTargetAddrExtTable_cols(Cols, [{Col, Val}|Acc]). +verify_snmpTargetAddrExtTable_cols([{Col, Val0}|Cols], TDomain, Acc) -> + Val = verify_snmpTargetAddrExtTable_col(Col, TDomain, Val0), + verify_snmpTargetAddrExtTable_cols(Cols, TDomain, [{Col, Val}|Acc]). -verify_snmpTargetAddrExtTable_col(?snmpTargetAddrTMask, []) -> +verify_snmpTargetAddrExtTable_col(?snmpTargetAddrTMask, _TDomain, []) -> []; -verify_snmpTargetAddrExtTable_col(?snmpTargetAddrTMask, TMask) -> - case (catch snmp_conf:check_taddress(TMask)) of +verify_snmpTargetAddrExtTable_col(?snmpTargetAddrTMask, TDomain, TMask) -> + case (catch snmp_conf:check_taddress(TDomain, TMask)) of ok -> TMask; _ -> wrongValue(?snmpTargetAddrTMask) end; -verify_snmpTargetAddrExtTable_col(?snmpTargetAddrMMS, MMS) -> +verify_snmpTargetAddrExtTable_col(?snmpTargetAddrMMS, _TDomain, MMS) -> case (catch snmp_conf:check_packet_size(MMS)) of ok -> MMS; _ -> wrongValue(?snmpTargetAddrMMS) end; -verify_snmpTargetAddrExtTable_col(_, Val) -> +verify_snmpTargetAddrExtTable_col(_, _TDomain, Val) -> Val. db(snmpTargetAddrExtTable) -> db(snmpTargetAddrTable); @@ -583,6 +622,7 @@ conv3([{Idx, Val}|T]) -> [{Idx+10, Val} | conv3(T)]; conv3([]) -> []. + get(Name, RowIndex, Cols) -> snmp_generic:handle_table_get(db(Name), RowIndex, Cols, foi(Name)). diff --git a/lib/snmp/src/agent/snmp_framework_mib.erl b/lib/snmp/src/agent/snmp_framework_mib.erl index cc191bd956..6ff9224d34 100644 --- a/lib/snmp/src/agent/snmp_framework_mib.erl +++ b/lib/snmp/src/agent/snmp_framework_mib.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2012. All Rights Reserved. +%% Copyright Ericsson AB 1999-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -41,6 +41,7 @@ -compile({no_auto_import,[error/1]}). -export([init/0, configure/1]). -export([intContextTable/1, intContextTable/3, + intAgentTransportDomain/1, intAgentTransports/1, intAgentUDPPort/1, intAgentIpAddress/1, snmpEngineID/1, snmpEngineBoots/1, @@ -51,7 +52,7 @@ set_engine_boots/1, set_engine_time/1, table_next/2, check_status/3]). -export([add_context/1, delete_context/1]). --export([check_agent/1, check_context/1]). +-export([check_agent/2, check_context/1, order_agent/2]). %%----------------------------------------------------------------- @@ -115,54 +116,46 @@ do_configure(Dir) -> read_internal_config_files(Dir) -> ?vdebug("read context config file",[]), - Gen = fun(D, Reason) -> - convert_context(D, Reason) - end, - Filter = fun(Contexts) -> Contexts end, - Check = fun(Entry) -> check_context(Entry) end, - [Ctxs] = snmp_conf:read_files(Dir, [{Gen, Filter, Check, "context.conf"}]), + Gen = fun gen_context/2, + Order = fun snmp_conf:no_order/2, + Filter = fun snmp_conf:no_filter/1, + Check = fun(Entry, State) -> {check_context(Entry), State} end, + [Ctxs] = + snmp_conf:read_files + (Dir, [{"context.conf", Gen, Order, Check, Filter}]), Ctxs. - read_agent(Dir) -> ?vdebug("read agent config file", []), - FileName = "agent.conf", - Check = fun(Entry) -> check_agent(Entry) end, + FileName = "agent.conf", File = filename:join(Dir, FileName), - Agent = + Conf0 = try - snmp_conf:read(File, Check) + snmp_conf:read(File, fun order_agent/2, fun check_agent/2) catch throw:{error, Reason} -> error({failed_reading_config_file, Dir, FileName, Reason}) end, - sort_agent(Agent). - - -%%----------------------------------------------------------------- -%% Make sure that each mandatory agent attribute is present, and -%% provide default values for the other non-present attributes. -%%----------------------------------------------------------------- -sort_agent(L) -> - Mand = [{intAgentIpAddress, mandatory}, - {intAgentUDPPort, mandatory}, - {snmpEngineMaxMessageSize, mandatory}, - {snmpEngineID, mandatory}], - {ok, L2} = snmp_conf:check_mandatory(L, Mand), - lists:keysort(1, L2). + Mand = + [{intAgentTransports, mandatory}, + {snmpEngineMaxMessageSize, mandatory}, + {snmpEngineID, mandatory}], + {ok, Conf} = snmp_conf:check_mandatory(Conf0, Mand), + Conf. %%----------------------------------------------------------------- %% Generate a context.conf file. %%----------------------------------------------------------------- -convert_context(Dir, _Reason) -> +gen_context(Dir, _Reason) -> config_err("missing context.conf file => generating a default file", []), File = filename:join(Dir, "context.conf"), case file:open(File, [write]) of {ok, Fid} -> ok = io:format(Fid, "~s\n", [context_header()]), ok = io:format(Fid, "%% The default context\n\"\".\n", []), - file:close(Fid); + file:close(Fid), + []; {error, Reason} -> file:delete(File), error({failed_creating_file, File, Reason}) @@ -196,10 +189,50 @@ check_context(Context) -> %% Agent %% {Name, Value}. %%----------------------------------------------------------------- -check_agent({intAgentIpAddress, Value}) -> - snmp_conf:check_ip(Value); -check_agent({intAgentUDPPort, Value}) -> - snmp_conf:check_integer(Value); +check_agent(Entry, undefined) -> + check_agent(Entry, {snmp_target_mib:default_domain(), undefined}); +check_agent({intAgentTransportDomain, Domain}, {_, Port}) -> + {snmp_conf:check_domain(Domain), {Domain, Port}}; +check_agent({intAgentUDPPort, Port}, {Domain, _}) -> + ok = snmp_conf:check_port(Port), + {ok, {Domain, Port}}; +check_agent({intAgentIpAddress, _}, {_, undefined}) -> + error({missing_mandatory, intAgentUDPPort}); +check_agent({intAgentIpAddress = Tag, Ip} = Entry, {Domain, Port} = State) -> + {case snmp_conf:check_ip(Domain, Ip) of + ok -> + [Entry, + {intAgentTransports, [{Domain, {Ip, Port}}]}]; + {ok, FixedIp} -> + [{Tag, FixedIp}, + {intAgentTransports, [{Domain, {FixedIp, Port}}]}] + end, State}; +check_agent({intAgentTransports = Tag, Transports}, {_, Port} = State) + when is_list(Transports) -> + CheckedTransports = + [case Transport of + {Domain, Address} -> + case + case Port of + undefined -> + snmp_conf:check_address(Domain, Address); + _ -> + snmp_conf:check_address(Domain, Address, Port) + end + of + ok -> + Transport; + {ok, FixedAddress} -> + {Domain, FixedAddress} + end; + _ -> + error({bad_transport, Transport}) + end + || Transport <- Transports], + {{ok, {Tag, CheckedTransports}}, State}; +check_agent(Entry, State) -> + {check_agent(Entry), State}. + %% This one is kept for backwards compatibility check_agent({intAgentMaxPacketSize, Value}) -> snmp_conf:check_packet_size(Value); @@ -210,6 +243,14 @@ check_agent({snmpEngineID, Value}) -> check_agent(X) -> error({invalid_agent_attribute, X}). +%% Ordering function to sort intAgentTransportDomain first +%% hence before intAgentIpAddress. Sort other entries on the key. +order_agent(EntryA, EntryB) -> + snmp_conf:keyorder( + 1, EntryA, EntryB, + [intAgentTransportDomain, intAgentUDPPort | sort]). + + maybe_create_table(Name) -> case snmpa_local_db:table_exists(db(Name)) of @@ -382,6 +423,14 @@ intAgentUDPPort(Op) -> intAgentIpAddress(Op) -> snmp_generic:variable_func(Op, db(intAgentIpAddress)). +intAgentTransportDomain(Op) -> + snmp_generic:variable_func(Op, db(intAgentTransportDomain)). + +intAgentTransports(Op) -> + snmp_generic:variable_func(Op, db(intAgentTransports)). + + + snmpEngineID(print) -> VarAndValue = [{snmpEngineID, snmpEngineID(get)}], snmpa_mib_lib:print_variables(VarAndValue); diff --git a/lib/snmp/src/agent/snmp_generic.erl b/lib/snmp/src/agent/snmp_generic.erl index 06afa68d96..3195ca2500 100644 --- a/lib/snmp/src/agent/snmp_generic.erl +++ b/lib/snmp/src/agent/snmp_generic.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2010. All Rights Reserved. +%% Copyright Ericsson AB 1996-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -414,7 +414,12 @@ table_check_status(NameDb, Col, ?'RowStatus_createAndGo', RowIndex, Cols) -> % side effect free and we only use the result temporary. case catch snmpa_local_db:table_construct_row( NameDb, RowIndex, ?'RowStatus_createAndGo', Cols) of - {'EXIT', _} -> + {'EXIT', _Reason} -> + ?vtrace( + "failed construct row (createAndGo): " + " n Reason: ~p" + " n Stack: ~p", + [_Reason, erlang:get_stacktrace()]), {noCreation, Col}; % Bad RowIndex Row -> case lists:member(noinit, tuple_to_list(Row)) of @@ -431,7 +436,12 @@ table_check_status(NameDb, Col, ?'RowStatus_createAndWait', RowIndex, Cols) -> false -> case catch snmpa_local_db:table_construct_row( NameDb, RowIndex, ?'RowStatus_createAndGo', Cols) of - {'EXIT', _} -> + {'EXIT', _Reason} -> + ?vtrace( + "failed construct row (createAndWait): " + " n Reason: ~p" + " n Stack: ~p", + [_Reason, erlang:get_stacktrace()]), {noCreation, Col}; % Bad RowIndex _Row -> {noError, 0} @@ -711,7 +721,19 @@ find_col(Col, [_H | T]) -> find_col(Col, T). try_apply(nofunc, _) -> {noError, 0}; -try_apply(F, Args) -> apply(F, Args). +try_apply(F, Args) -> maybe_verbose_apply(F, Args). + +maybe_verbose_apply(M, Args) -> + case get(verbosity) of + false -> + apply(M, Args); + _ -> + ?vlog("~n apply: ~w,~p~n", [M,Args]), + Res = apply(M,Args), + ?vlog("~n returned: ~p", [Res]), + Res + end. + table_info({Name, _Db}) -> case snmpa_symbolic_store:table_info(Name) of diff --git a/lib/snmp/src/agent/snmp_notification_mib.erl b/lib/snmp/src/agent/snmp_notification_mib.erl index 37e09f5d3e..31c7735226 100644 --- a/lib/snmp/src/agent/snmp_notification_mib.erl +++ b/lib/snmp/src/agent/snmp_notification_mib.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1998-2012. All Rights Reserved. +%% Copyright Ericsson AB 1998-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -106,17 +106,19 @@ do_reconfigure(Dir) -> read_notify_config_files(Dir) -> ?vdebug("read notify config file",[]), FileName = "notify.conf", - Gen = fun(D, Reason) -> - info_msg("failed reading config file ~s" - "~n Config Dir: ~s" - "~n Reason: ~p", - [FileName, D, Reason]), - ok - end, - Filter = fun(Notifs) -> Notifs end, - Check = fun(Entry) -> check_notify(Entry) end, + Gen = + fun (D, Reason) -> + info_msg("failed reading config file ~s" + "~n Config Dir: ~s" + "~n Reason: ~p", + [FileName, D, Reason]), + ok + end, + Order = fun snmp_conf:no_order/2, + Filter = fun snmp_conf:no_filter/1, + Check = fun (Entry, State) -> {check_notify(Entry), State} end, [Notifs] = - snmp_conf:read_files(Dir, [{Gen, Filter, Check, "notify.conf"}]), + snmp_conf:read_files(Dir, [{FileName, Gen, Order, Check, Filter}]), Notifs. check_notify({Name, Tag, Type}) -> diff --git a/lib/snmp/src/agent/snmp_standard_mib.erl b/lib/snmp/src/agent/snmp_standard_mib.erl index 766b75022b..aace3fd413 100644 --- a/lib/snmp/src/agent/snmp_standard_mib.erl +++ b/lib/snmp/src/agent/snmp_standard_mib.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2012. All Rights Reserved. +%% Copyright Ericsson AB 1996-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -152,16 +152,19 @@ do_reconfigure(Dir) -> %%----------------------------------------------------------------- read_standard(Dir) -> ?vdebug("check standard config file",[]), - FileName = "standard.conf", - Gen = fun(D, Reason) -> - throw({error, {failed_reading_config_file, - D, FileName, - list_dir(Dir), Reason}}) - end, - Filter = fun(Standard) -> sort_standard(Standard) end, - Check = fun(Entry) -> check_standard(Entry) end, + FileName = "standard.conf", + Gen = + fun (D, Reason) -> + throw( + {error, + {failed_reading_config_file, + D, FileName, list_dir(Dir), Reason}}) + end, + Order = fun snmp_conf:no_order/2, + Check = fun (Entry, State) -> {check_standard(Entry), State} end, + Filter = fun sort_standard/1, [Standard] = - snmp_conf:read_files(Dir, [{Gen, Filter, Check, FileName}]), + snmp_conf:read_files(Dir, [{FileName, Gen, Order, Check, Filter}]), Standard. list_dir(Dir) -> diff --git a/lib/snmp/src/agent/snmp_target_mib.erl b/lib/snmp/src/agent/snmp_target_mib.erl index b01d536caa..ef9503cda8 100644 --- a/lib/snmp/src/agent/snmp_target_mib.erl +++ b/lib/snmp/src/agent/snmp_target_mib.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1998-2012. All Rights Reserved. +%% Copyright Ericsson AB 1998-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -133,18 +133,22 @@ do_reconfigure(Dir) -> read_target_config_files(Dir) -> ?vdebug("check target address and parameter config file(s)",[]), - TAGen = fun(_D, _Reason) -> ok end, - TAFilter = fun(Addr) -> Addr end, - TACheck = fun(Entry) -> check_target_addr(Entry) end, - TPGen = fun(_D, _Reason) -> ok end, - TPFilter = fun(Params) -> Params end, - TPCheck = fun(Entry) -> check_target_params(Entry) end, + TAName = "target_addr.conf", + TACheck = fun (Entry, State) -> {check_target_addr(Entry), State} end, + + TPName = "target_params.conf", + TPCheck = fun (Entry, State) -> {check_target_params(Entry), State} end, + + NoGen = fun snmp_conf:no_gen/2, + NoOrder = fun snmp_conf:no_order/2, + NoFilter = fun snmp_conf:no_filter/1, [Addrs, Params] = - snmp_conf:read_files(Dir, - [{TAGen, TAFilter, TACheck, "target_addr.conf"}, - {TPGen, TPFilter, TPCheck, "target_params.conf"}]), + snmp_conf:read_files( + Dir, + [{TAName, NoGen, NoOrder, TACheck, NoFilter}, + {TPName, NoGen, NoOrder, TPCheck, NoFilter}]), {Addrs, Params}. @@ -154,79 +158,173 @@ read_target_config_files(Dir) -> %% TMask, MMS} %%----------------------------------------------------------------- -check_target_addr({Name, Domain, Ip, Udp, Timeout, RetryCount, TagList, - Params, EngineId, TMask, MMS}) -> +check_target_addr( + {Name, Domain, Ip, Udp, Timeout, RetryCount, TagList, Params, + EngineId, TMask, MMS}) -> % Arity 11 + Address = {Ip, Udp}, + check_target_addr( + Name, Domain, Address, Timeout, RetryCount, TagList, Params, + EngineId, TMask, MMS); +check_target_addr( + {Name, Domain, Address, Timeout, RetryCount, TagList, Params, + EngineId, TMask, MMS}) % Arity 10 + when is_atom(Domain) -> + check_target_addr( + Name, Domain, Address, Timeout, RetryCount, TagList, Params, + EngineId, TMask, MMS); +check_target_addr( + {Name, Ip, Udp, Timeout, RetryCount, TagList, Params, + EngineId, TMask, MMS}) + when is_integer(Udp) -> % Arity 10 + Domain = default_domain(), + Address = {Ip, Udp}, + check_target_addr( + Name, Domain, Address, Timeout, RetryCount, TagList, Params, + EngineId, TMask, MMS); +check_target_addr( + {_Name, Domain, Address, _Timeout, _RetryCount, _TagList, _Params, + _EngineId, _TMask, _MMS}) -> % Arity 10 + error({bad_address, {Domain, Address}}); +check_target_addr( + {Name, Domain, Address, Timeout, RetryCount, TagList, Params, + EngineId}) % Arity 8 + when is_atom(Domain) -> + check_target_addr( + Name, Domain, Address, Timeout, RetryCount, TagList, Params, + EngineId); +check_target_addr( + {Name, Ip, Udp, Timeout, RetryCount, TagList, Params, + EngineId}) % Arity 8 + when is_integer(Udp) -> + Domain = default_domain(), + Address = {Ip, Udp}, + check_target_addr( + Name, Domain, Address, Timeout, RetryCount, TagList, Params, + EngineId); +check_target_addr( + {_Name, Domain, Address, _Timeout, _RetryCount, _TagList, _Params, + _EngineId}) ->% Arity 8 + error({bad_address, {Domain, Address}}); +%% Use dummy engine id if the old style is found +check_target_addr( + {Name, Domain, Address, Timeout, RetryCount, TagList, Params}) % Arity 7 + when is_atom(Domain) -> + check_target_addr( + Name, Domain, Address, Timeout, RetryCount, TagList, Params); +check_target_addr( + {Name, Ip, Udp, Timeout, RetryCount, TagList, Params}) % Arity 7 + when is_integer(Udp) -> + Domain = default_domain(), + Address = {Ip, Udp}, + check_target_addr( + Name, Domain, Address, Timeout, RetryCount, TagList, Params); +check_target_addr( + {_Name, Domain, Address, _Timeout, _RetryCount, _TagList, _Params}) -> % Arity 7 + error({bad_address, {Domain, Address}}); +%% Use dummy engine id if the old style is found +check_target_addr( + {Name, Domain, Address, Timeout, RetryCount, TagList, Params, + TMask, MMS}) % Arity 9 + when is_atom(Domain) -> + check_target_addr( + Name, Domain, Address, Timeout, RetryCount, TagList, Params, TMask, MMS); +check_target_addr( + {Name, Ip, Udp, Timeout, RetryCount, TagList, Params, + TMask, MMS}) % Arity 9 + when is_integer(Udp) -> + Domain = default_domain(), + Address = {Ip, Udp}, + check_target_addr( + Name, Domain, Address, Timeout, RetryCount, TagList, Params, TMask, MMS); +check_target_addr( + {_Name, Domain, Address, _Timeout, _RetryCount, _TagList, _Params, + _TMask, _MMS}) -> % Arity 9 + error({bad_address, {Domain, Address}}); +check_target_addr(X) -> + error({invalid_target_addr, X}). + +check_target_addr( + Name, Domain, Address, Timeout, RetryCount, TagList, Params) -> % Arity 7 + check_target_addr( + Name, Domain, Address, Timeout, RetryCount, TagList, Params, + "dummy"). +%% +check_target_addr( + Name, Domain, Address, Timeout, RetryCount, TagList, Params, + EngineId) -> % Arity 8 + check_target_addr( + Name, Domain, Address, Timeout, RetryCount, TagList, Params, + EngineId, [], 2048). +%% +check_target_addr( + Name, Domain, Address, Timeout, RetryCount, TagList, Params, + TMask, MMS) -> % Arity 9 + check_target_addr( + Name, Domain, Address, Timeout, RetryCount, TagList, Params, + "dummy", TMask, MMS). +%% +check_target_addr( + Name, Domain, Address, Timeout, RetryCount, TagList, Params, + EngineId, Mask, MMS) -> % Arity 10 ?vtrace("check target address with:" "~n Name: ~s" "~n Domain: ~p" - "~n Ip: ~p" - "~n Udp: ~p" + "~n Address: ~p" "~n Timeout: ~p" "~n RetryCount: ~p" "~n TagList: ~p" "~n Params: ~p" "~n EngineId: ~p" - "~n TMask: ~p" + "~n Mask: ~p" "~n MMS: ~p", - [Name, - Domain, Ip, Udp, + [Name, Domain, Address, Timeout, RetryCount, - TagList, Params, EngineId, TMask, MMS]), + TagList, Params, EngineId, Mask, MMS]), snmp_conf:check_string(Name,{gt,0}), snmp_conf:check_domain(Domain), - snmp_conf:check_ip(Domain, Ip), - snmp_conf:check_integer(Udp, {gt, 0}), + NAddress = check_address(Domain, Address), snmp_conf:check_integer(Timeout, {gte, 0}), snmp_conf:check_integer(RetryCount, {gte,0}), snmp_conf:check_string(TagList), snmp_conf:check_string(Params), check_engine_id(EngineId), - TAddress = snmp_conf:mk_taddress(Domain, Ip, Udp), + NMask = check_mask(Domain, Mask), TDomain = snmp_conf:mk_tdomain(Domain), - check_tmask(TDomain, TMask, TAddress), + TAddress = snmp_conf:mk_taddress(Domain, NAddress), + TMask = snmp_conf:mk_taddress(Domain, NMask), snmp_conf:check_packet_size(MMS), ?vtrace("check target address done",[]), Addr = {Name, TDomain, TAddress, Timeout, RetryCount, TagList, Params, ?'StorageType_nonVolatile', ?'RowStatus_active', EngineId, TMask, MMS}, % Values for Augmenting table in SNMP-COMMUNITY-MIB - {ok, Addr}; -check_target_addr({Name, Ip, Udp, Timeout, RetryCount, TagList, - Params, EngineId, TMask, MMS}) -> - Domain = default_domain(), - check_target_addr({Name, - Domain, Ip, Udp, - Timeout, RetryCount, TagList, - Params, EngineId, TMask, MMS}); -check_target_addr({Name, Ip, Udp, Timeout, RetryCount, TagList, Params, - EngineId}) -> - check_target_addr({Name, Ip, Udp, Timeout, RetryCount, TagList, - Params, EngineId, [], 2048}); -%% Use dummy engine id if the old style is found -check_target_addr({Name, Ip, Udp, Timeout, RetryCount, TagList, Params}) -> - check_target_addr({Name, Ip, Udp, Timeout, RetryCount, TagList, - Params, "dummy", [], 2048}); -%% Use dummy engine id if the old style is found -check_target_addr({Name, Ip, Udp, Timeout, RetryCount, TagList, Params, - TMask, MMS}) -> - check_target_addr({Name, Ip, Udp, Timeout, RetryCount, TagList, - Params, "dummy", TMask, MMS}); -check_target_addr(X) -> - error({invalid_target_addr, X}). - + {ok, Addr}. check_engine_id(discovery) -> ok; check_engine_id(EngineId) -> snmp_conf:check_string(EngineId). +check_address(Domain, Address) -> + case snmp_conf:check_address(Domain, Address, 162) of + ok -> + Address; + {ok, NAddress} -> + NAddress + end. -check_tmask(_TDomain, [], _TAddress) -> - ok; -check_tmask(TDomain, TMask, TAddress) when length(TMask) =:= length(TAddress) -> - snmp_conf:check_taddress(TDomain, TMask); -check_tmask(_TDomain, TMask, _TAddr) -> - throw({error, {invalid_tmask, TMask}}). +check_mask(_Domain, [] = Mask) -> + Mask; +check_mask(Domain, Mask) -> + try snmp_conf:check_address(Domain, Mask) of + ok -> + Mask; + {ok, NMask} -> + NMask + catch + {error, {bad_address, Info}} -> + error({bad_mask, Info}) + end. %%----------------------------------------------------------------- @@ -286,16 +384,21 @@ table_del_row(Tab, Key) -> snmpa_mib_lib:table_del_row(db(Tab), Key). -add_addr(Name, Ip, Port, Timeout, Retry, TagList, - Params, EngineId, TMask, MMS) -> - Domain = default_domain(), - add_addr(Name, Domain, Ip, Port, Timeout, Retry, TagList, - Params, EngineId, TMask, MMS). - -add_addr(Name, Domain, Ip, Port, Timeout, Retry, TagList, - Params, EngineId, TMask, MMS) -> - Addr = {Name, Domain, Ip, Port, Timeout, Retry, TagList, - Params, EngineId, TMask, MMS}, +add_addr( + Name, Domain_or_Ip, Addr_or_Port, Timeout, Retry, TagList, Params, + EngineId, TMask, MMS) -> + add_addr( + {Name, Domain_or_Ip, Addr_or_Port, Timeout, Retry, TagList, Params, + EngineId, TMask, MMS}). +%% +add_addr( + Name, Domain, Ip, Port, Timeout, Retry, TagList, Params, + EngineId, TMask, MMS) -> + add_addr( + {Name, Domain, Ip, Port, Timeout, Retry, TagList, Params, + EngineId, TMask, MMS}). +%% +add_addr(Addr) -> case (catch check_target_addr(Addr)) of {ok, Row} -> Key = element(1, Row), @@ -604,6 +707,20 @@ snmpTargetAddrTable(print) -> FOI = foi(Table), PrintRow = fun(Prefix, Row) -> + TDomain = element(?snmpTargetAddrTDomain, Row), + Domain = + try snmp_conf:tdomain_to_domain(TDomain) + catch + {error, {bad_tdomain, _}} -> + undefined + end, + TAddress = element(?snmpTargetAddrTAddress, Row), + AddrString = + try snmp_conf:mk_addr_string({Domain, TAddress}) + catch + {error, {bad_address, _}} -> + "-" + end, lists:flatten( io_lib:format("~sName: ~p" "~n~sTDomain: ~p (~w)" @@ -618,21 +735,8 @@ snmpTargetAddrTable(print) -> "~n~s[Ext] TMask: ~p" "~n~s[Ext] MMS: ~p", [Prefix, element(?snmpTargetAddrName, Row), - Prefix, element(?snmpTargetAddrTDomain, Row), - case element(?snmpTargetAddrTDomain, Row) of - ?snmpUDPDomain -> snmpUDPDomain; - ?transportDomainUdpIpv4 -> transportDomainUdpIpv4; - ?transportDomainUdpIpv6 -> transportDomainUdpIpv6; - _ -> undefined - end, - Prefix, element(?snmpTargetAddrTAddress, Row), - case element(?snmpTargetAddrTAddress, Row) of - [A,B,C,D,U1,U2] -> - lists:flatten( - io_lib:format("~w.~w.~w.~w:~w", - [A, B, C, D, U1 bsl 8 + U2])); - _ -> "-" - end, + Prefix, TDomain, Domain, + Prefix, TAddress, AddrString, Prefix, element(?snmpTargetAddrTimeout, Row), Prefix, element(?snmpTargetAddrRetryCount, Row), Prefix, element(?snmpTargetAddrTagList, Row), diff --git a/lib/snmp/src/agent/snmp_user_based_sm_mib.erl b/lib/snmp/src/agent/snmp_user_based_sm_mib.erl index 223d3f7218..69dce337ba 100644 --- a/lib/snmp/src/agent/snmp_user_based_sm_mib.erl +++ b/lib/snmp/src/agent/snmp_user_based_sm_mib.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2013. All Rights Reserved. +%% Copyright Ericsson AB 1999-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -137,18 +137,20 @@ do_reconfigure(Dir) -> read_usm_config_files(Dir) -> ?vdebug("read usm config file",[]), - Gen = fun(D, Reason) -> generate_usm(D, Reason) end, - Filter = fun(Usms) -> Usms end, - Check = fun(Entry) -> check_usm(Entry) end, + Gen = fun (D, Reason) -> generate_usm(D, Reason) end, + Order = fun snmp_conf:no_order/2, + Check = fun (Entry, State) -> {check_usm(Entry), State} end, + Filter = fun snmp_conf:no_filter/1, [Usms] = - snmp_conf:read_files(Dir, [{Gen, Filter, Check, "usm.conf"}]), + snmp_conf:read_files(Dir, [{"usm.conf", Gen, Order, Check, Filter}]), Usms. generate_usm(Dir, _Reason) -> info_msg("Incomplete configuration. Generating empty usm.conf.", []), USMFile = filename:join(Dir, "usm.conf"), - ok = file:write_file(USMFile, list_to_binary([])). + ok = file:write_file(USMFile, list_to_binary([])), + []. check_usm({EngineID, Name, SecName, Clone, AuthP, AuthKeyC, OwnAuthKeyC, diff --git a/lib/snmp/src/agent/snmp_view_based_acm_mib.erl b/lib/snmp/src/agent/snmp_view_based_acm_mib.erl index c0177b1cea..722bd7ac5b 100644 --- a/lib/snmp/src/agent/snmp_view_based_acm_mib.erl +++ b/lib/snmp/src/agent/snmp_view_based_acm_mib.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2013. All Rights Reserved. +%% Copyright Ericsson AB 1999-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -123,15 +123,18 @@ do_reconfigure(Dir) -> read_vacm_config_files(Dir) -> ?vdebug("read vacm config file",[]), - Gen = fun(_D, _Reason) -> ok end, - Filter = fun(Vacms) -> - Sec2Group = [X || {vacmSecurityToGroup, X} <- Vacms], - Access = [X || {vacmAccess, X} <- Vacms], - View = [X || {vacmViewTreeFamily, X} <- Vacms], - {Sec2Group, Access, View} - end, - Check = fun(Entry) -> check_vacm(Entry) end, - [Vacms] = snmp_conf:read_files(Dir, [{Gen, Filter, Check, "vacm.conf"}]), + Gen = fun snmp_conf:no_gen/2, + Order = fun snmp_conf:no_order/2, + Check = fun (Entry, State) -> {check_vacm(Entry), State} end, + Filter = + fun (Vacms) -> + Sec2Group = [X || {vacmSecurityToGroup, X} <- Vacms], + Access = [X || {vacmAccess, X} <- Vacms], + View = [X || {vacmViewTreeFamily, X} <- Vacms], + {Sec2Group, Access, View} + end, + [Vacms] = + snmp_conf:read_files(Dir, [{"vacm.conf", Gen, Order, Check, Filter}]), Vacms. %%----------------------------------------------------------------- diff --git a/lib/snmp/src/agent/snmpa_agent.erl b/lib/snmp/src/agent/snmpa_agent.erl index 9bed6e554e..6d3f1cca4a 100644 --- a/lib/snmp/src/agent/snmpa_agent.erl +++ b/lib/snmp/src/agent/snmpa_agent.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2013. All Rights Reserved. +%% Copyright Ericsson AB 1996-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -2512,10 +2512,19 @@ handle_mib_of(MibServer, Oid) -> %% Func: process_msg/7 %% Returns: RePdu %%----------------------------------------------------------------- -process_msg(MibView, Vsn, Pdu, PduMS, Community, {Ip, Udp}, ContextName, - GbMaxVBs) -> +process_msg( + MibView, Vsn, Pdu, PduMS, Community, + SourceAddress, ContextName, GbMaxVBs) -> #pdu{request_id = ReqId} = Pdu, - put(snmp_address, {tuple_to_list(Ip), Udp}), + put( + snmp_address, + case SourceAddress of + {Domain, _} when is_atom(Domain) -> + SourceAddress; + {Ip, Port} when is_integer(Port) -> + %% Legacy transport domain + {tuple_to_list(Ip), Port} + end), put(snmp_request_id, ReqId), put(snmp_community, Community), put(snmp_context, ContextName), diff --git a/lib/snmp/src/agent/snmpa_conf.erl b/lib/snmp/src/agent/snmpa_conf.erl index c17a6abbd7..534d0e447b 100644 --- a/lib/snmp/src/agent/snmpa_conf.erl +++ b/lib/snmp/src/agent/snmpa_conf.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2006-2011. All Rights Reserved. +%% Copyright Ericsson AB 2006-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -47,7 +47,7 @@ read_standard_config/1, %% target_addr.conf - target_addr_entry/5, target_addr_entry/6, + target_addr_entry/5, target_addr_entry/6, target_addr_entry/7, target_addr_entry/8, target_addr_entry/10, target_addr_entry/11, write_target_addr_config/2, write_target_addr_config/3, append_target_addr_config/2, @@ -108,36 +108,25 @@ write_agent_config(Dir, Conf) -> Hdr = header() ++ Comment, write_agent_config(Dir, Hdr, Conf). -write_agent_config(Dir, Hdr, Conf) +write_agent_config(Dir, Hdr, Conf) when is_list(Dir) and is_list(Hdr) and is_list(Conf) -> - Verify = fun() -> verify_agent_conf(Conf) end, - Write = fun(Fd) -> write_agent_conf(Fd, Hdr, Conf) end, - write_config_file(Dir, "agent.conf", Verify, Write). - + Order = fun snmp_framework_mib:order_agent/2, + Check = fun snmp_framework_mib:check_agent/2, + Write = fun (Fd, Entries) -> write_agent_conf(Fd, Hdr, Entries) end, + write_config_file(Dir, "agent.conf", Order, Check, Write, Conf). -append_agent_config(Dir, Conf) +append_agent_config(Dir, Conf) when is_list(Dir) and is_list(Conf) -> - Verify = fun() -> verify_agent_conf(Conf) end, - Write = fun(Fd) -> write_agent_conf(Fd, Conf) end, - append_config_file(Dir, "agent.conf", Verify, Write). - + Order = fun snmp_framework_mib:order_agent/2, + Check = fun snmp_framework_mib:check_agent/2, + Write = fun write_agent_conf/2, + append_config_file(Dir, "agent.conf", Order, Check, Write, Conf). read_agent_config(Dir) -> - Verify = fun(Entry) -> verify_agent_conf_entry(Entry) end, - read_config_file(Dir, "agent.conf", Verify). + Order = fun snmp_framework_mib:order_agent/2, + Check = fun snmp_framework_mib:check_agent/2, + read_config_file(Dir, "agent.conf", Order, Check). - -verify_agent_conf([]) -> - ok; -verify_agent_conf([H|T]) -> - verify_agent_conf_entry(H), - verify_agent_conf(T); -verify_agent_conf(X) -> - error({bad_agent_config, X}). - -verify_agent_conf_entry(Entry) -> - ok = snmp_framework_mib:check_agent(Entry), - ok. write_agent_conf(Fd, "", Conf) -> write_agent_conf(Fd, Conf); @@ -151,6 +140,10 @@ write_agent_conf(Fd, [H|T]) -> do_write_agent_conf(Fd, H), write_agent_conf(Fd, T). +do_write_agent_conf(Fd, {intAgentTransports = Tag, Val}) -> + io:format(Fd, "{~w, ~w}.~n", [Tag, Val]); +do_write_agent_conf(Fd, {intAgentTransportDomain = Tag, Val}) -> + io:format(Fd, "{~w, ~w}.~n", [Tag, Val]); do_write_agent_conf(Fd, {intAgentIpAddress = Tag, Val}) -> io:format(Fd, "{~w, ~w}.~n", [Tag, Val]); do_write_agent_conf(Fd, {intAgentUDPPort = Tag, Val} ) -> @@ -191,34 +184,27 @@ write_context_config(Dir, Conf) -> write_context_config(Dir, Hdr, Conf) when is_list(Dir) and is_list(Hdr) and is_list(Conf) -> - Verify = fun() -> verify_context_conf(Conf) end, - Write = fun(Fd) -> write_context_conf(Fd, Hdr, Conf) end, - write_config_file(Dir, "context.conf", Verify, Write). - + Order = fun snmp_conf:no_order/2, + Check = fun check_context/2, + Write = fun (Fd, Entries) -> write_context_conf(Fd, Hdr, Entries) end, + write_config_file(Dir, "context.conf", Order, Check, Write, Conf). -append_context_config(Dir, Conf) +append_context_config(Dir, Conf) when is_list(Dir) and is_list(Conf) -> - Verify = fun() -> verify_context_conf(Conf) end, - Write = fun(Fd) -> write_context_conf(Fd, Conf) end, - append_config_file(Dir, "context.conf", Verify, Write). - + Order = fun snmp_conf:no_order/2, + Check = fun check_context/2, + Write = fun write_context_conf/2, + append_config_file(Dir, "context.conf", Order, Check, Write, Conf). read_context_config(Dir) -> - Verify = fun(Entry) -> verify_context_conf_entry(Entry) end, - read_config_file(Dir, "context.conf", Verify). + Order = fun snmp_conf:no_order/2, + Check = fun check_context/2, + read_config_file(Dir, "context.conf", Order, Check). - -verify_context_conf([]) -> - ok; -verify_context_conf([H|T]) -> - verify_context_conf_entry(H), - verify_context_conf(T); -verify_context_conf(X) -> - error({error_context_config, X}). - -verify_context_conf_entry(Context) -> - {ok, _} = snmp_framework_mib:check_context(Context), - ok. + +check_context(Entry, State) -> + {check_ok(snmp_framework_mib:check_context(Entry)), + State}. write_context_conf(Fd, "", Conf) -> write_context_conf(Fd, Conf); @@ -271,36 +257,29 @@ write_community_config(Dir, Conf) -> Hdr = header() ++ Comment, write_community_config(Dir, Hdr, Conf). -write_community_config(Dir, Hdr, Conf) +write_community_config(Dir, Hdr, Conf) when is_list(Dir) and is_list(Hdr) and is_list(Conf) -> - Verify = fun() -> verify_community_conf(Conf) end, - Write = fun(Fd) -> write_community_conf(Fd, Hdr, Conf) end, - write_config_file(Dir, "community.conf", Verify, Write). + Order = fun snmp_conf:no_order/2, + Check = fun check_community/2, + Write = fun (Fd, Entries) -> write_community_conf(Fd, Hdr, Entries) end, + write_config_file(Dir, "community.conf", Order, Check, Write, Conf). - -append_community_config(Dir, Conf) +append_community_config(Dir, Conf) when is_list(Dir) and is_list(Conf) -> - Verify = fun() -> verify_community_conf(Conf) end, - Write = fun(Fd) -> write_community_conf(Fd, Conf) end, - append_config_file(Dir, "community.conf", Verify, Write). - + Order = fun snmp_conf:no_order/2, + Check = fun check_community/2, + Write = fun write_community_conf/2, + append_config_file(Dir, "community.conf", Order, Check, Write, Conf). read_community_config(Dir) -> - Verify = fun(Entry) -> verify_community_conf_entry(Entry) end, - read_config_file(Dir, "community.conf", Verify). + Order = fun snmp_conf:no_order/2, + Check = fun check_community/2, + read_config_file(Dir, "community.conf", Order, Check). - -verify_community_conf([]) -> - ok; -verify_community_conf([H|T]) -> - verify_community_conf_entry(H), - verify_community_conf(T); -verify_community_conf(X) -> - error({invalid_community_config, X}). - -verify_community_conf_entry(Context) -> - {ok, _} = snmp_community_mib:check_community(Context), - ok. + +check_community(Entry, State) -> + {check_ok(snmp_community_mib:check_community(Entry)), + State}. write_community_conf(Fd, "", Conf) -> write_community_conf(Fd, Conf); @@ -309,14 +288,17 @@ write_community_conf(Fd, Hdr, Conf) -> write_community_conf(Fd, Conf). write_community_conf(Fd, Conf) -> - Fun = fun({Idx, Name, SecName, CtxName, TranspTag}) -> - io:format(Fd, "{\"~s\", \"~s\", \"~s\", \"~s\", \"~s\"}.~n", - [Idx, Name, SecName, CtxName, TranspTag]); + Fun = + fun({Idx, Name, SecName, CtxName, TranspTag}) -> + io:format( + Fd, + "{\"~s\", \"~s\", \"~s\", \"~s\", \"~s\"}.~n", + [Idx, Name, SecName, CtxName, TranspTag]); (Crap) -> - error({bad_community_config, Crap}) + error({bad_community_config, Crap}) end, lists:foreach(Fun, Conf). - + %% %% ------ standard.conf ------ @@ -343,40 +325,29 @@ write_standard_config(Dir, Conf) -> Hdr = header() ++ Comment, write_standard_config(Dir, Hdr, Conf). -write_standard_config(Dir, Hdr, Conf) +write_standard_config(Dir, Hdr, Conf) when is_list(Dir) and is_list(Hdr) and is_list(Conf) -> - Verify = fun() -> verify_standard_conf(Conf) end, - Write = fun(Fd) -> write_standard_conf(Fd, Hdr, Conf) end, - write_config_file(Dir, "standard.conf", Verify, Write). - + Order = fun snmp_conf:no_order/2, + Check = fun check_standard/2, + Write = fun (Fd, Entries) -> write_standard_conf(Fd, Hdr, Entries) end, + write_config_file(Dir, "standard.conf", Order, Check, Write, Conf). -append_standard_config(Dir, Conf) +append_standard_config(Dir, Conf) when is_list(Dir) and is_list(Conf) -> - Verify = fun() -> verify_standard_conf(Conf) end, - Write = fun(Fd) -> write_standard_conf(Fd, Conf) end, - append_config_file(Dir, "standard.conf", Verify, Write). - + Order = fun snmp_conf:no_order/2, + Check = fun check_standard/2, + Write = fun write_standard_conf/2, + append_config_file(Dir, "standard.conf", Order, Check, Write, Conf). read_standard_config(Dir) -> - Verify = fun(Entry) -> verify_standard_conf_entry(Entry) end, - read_config_file(Dir, "standard.conf", Verify). + Order = fun snmp_conf:no_order/2, + Check = fun check_standard/2, + read_config_file(Dir, "standard.conf", Order, Check). - -verify_standard_conf([]) -> - ok; -verify_standard_conf([H|T]) -> - verify_standard_conf_entry(H), - verify_standard_conf(T); -verify_standard_conf(X) -> - error({bad_standard_config, X}). - -verify_standard_conf_entry(Std) -> - case snmp_standard_mib:check_standard(Std) of - ok -> - ok; - {ok, _} -> - ok - end. + +check_standard(Entry, State) -> + {check_ok(snmp_standard_mib:check_standard(Entry)), + State}. write_standard_conf(Fd, "", Conf) -> write_standard_conf(Fd, Conf); @@ -410,72 +381,48 @@ do_write_standard_conf(_Fd, Tag, Val) -> %% ------ target_addr.conf ------ %% -target_addr_entry(Name, - Ip, - TagList, - ParamsName, - EngineId) -> +target_addr_entry( + Name, Ip, TagList, ParamsName, EngineId) -> target_addr_entry(Name, Ip, TagList, ParamsName, EngineId, []). -target_addr_entry(Name, - Ip, - TagList, - ParamsName, - EngineId, - TMask) -> - target_addr_entry(Name, Ip, 162, TagList, - ParamsName, EngineId, - TMask, 2048). - -target_addr_entry(Name, - Ip, - Udp, - TagList, - ParamsName, - EngineId, - TMask, - MaxMessageSize) -> - target_addr_entry(Name, Ip, Udp, 1500, 3, TagList, - ParamsName, EngineId, - TMask, MaxMessageSize). - -target_addr_entry(Name, - Ip, - Udp, - Timeout, - RetryCount, - TagList, - ParamsName, - EngineId, - TMask, - MaxMessageSize) -> - target_addr_entry(Name, snmp_target_mib:default_domain(), Ip, Udp, - Timeout, RetryCount, TagList, - ParamsName, EngineId, - TMask, MaxMessageSize). - -target_addr_entry(Name, - Domain, - Ip, - Udp, - Timeout, - RetryCount, - TagList, - ParamsName, - EngineId, - TMask, - MaxMessageSize) -> - {Name, - Domain, - Ip, - Udp, - Timeout, - RetryCount, - TagList, - ParamsName, - EngineId, - TMask, - MaxMessageSize}. +target_addr_entry( + Name, Domain, Addr, TagList, + ParamsName, EngineId) when is_atom(Domain) -> + target_addr_entry( + Name, Domain, Addr, TagList, + ParamsName, EngineId, []); +target_addr_entry( + Name, Ip, TagList, ParamsName, + EngineId, TMask) -> + target_addr_entry( + Name, Ip, 162, TagList, ParamsName, + EngineId, TMask, 2048). + +target_addr_entry( + Name, Domain_or_Ip, Addr_or_Port, TagList, + ParamsName, EngineId, TMask) -> + target_addr_entry( + Name, Domain_or_Ip, Addr_or_Port, TagList, + ParamsName, EngineId, TMask, 2048). + +target_addr_entry( + Name, Domain_or_Ip, Addr_or_Port, TagList, + ParamsName, EngineId, TMask, MaxMessageSize) -> + target_addr_entry( + Name, Domain_or_Ip, Addr_or_Port, 1500, 3, TagList, + ParamsName, EngineId, TMask, MaxMessageSize). + +target_addr_entry( + Name, Domain_or_Ip, Addr_or_Port, Timeout, RetryCount, TagList, + ParamsName, EngineId, TMask, MaxMessageSize) -> + {Name, Domain_or_Ip, Addr_or_Port, Timeout, RetryCount, TagList, + ParamsName, EngineId, TMask, MaxMessageSize}. + +target_addr_entry( + Name, Domain, Ip, Udp, Timeout, RetryCount, TagList, + ParamsName, EngineId,TMask, MaxMessageSize) -> + {Name, Domain, Ip, Udp, Timeout, RetryCount, TagList, + ParamsName, EngineId, TMask, MaxMessageSize}. write_target_addr_config(Dir, Conf) -> @@ -504,37 +451,29 @@ write_target_addr_config(Dir, Conf) -> Hdr = header() ++ Comment, write_target_addr_config(Dir, Hdr, Conf). -write_target_addr_config(Dir, Hdr, Conf) +write_target_addr_config(Dir, Hdr, Conf) when is_list(Dir) and is_list(Hdr) and is_list(Conf) -> - Verify = fun() -> verify_target_addr_conf(Conf) end, - Write = fun(Fd) -> write_target_addr_conf(Fd, Hdr, Conf) end, - write_config_file(Dir, "target_addr.conf", Verify, Write). + Order = fun snmp_conf:no_order/2, + Check = fun check_target_addr/2, + Write = fun (Fd, Entries) -> write_target_addr_conf(Fd, Hdr, Entries) end, + write_config_file(Dir, "target_addr.conf", Order, Check, Write, Conf). - - -append_target_addr_config(Dir, Conf) +append_target_addr_config(Dir, Conf) when is_list(Dir) and is_list(Conf) -> - Verify = fun() -> verify_target_addr_conf(Conf) end, - Write = fun(Fd) -> write_target_addr_conf(Fd, Conf) end, - append_config_file(Dir, "target_addr.conf", Verify, Write). - + Order = fun snmp_conf:no_order/2, + Check = fun check_target_addr/2, + Write = fun write_target_addr_conf/2, + append_config_file(Dir, "target_addr.conf", Order, Check, Write, Conf). read_target_addr_config(Dir) -> - Verify = fun(Entry) -> verify_target_addr_conf_entry(Entry) end, - read_config_file(Dir, "target_addr.conf", Verify). + Order = fun snmp_conf:no_order/2, + Check = fun check_target_addr/2, + read_config_file(Dir, "target_addr.conf", Order, Check). - -verify_target_addr_conf([]) -> - ok; -verify_target_addr_conf([H|T]) -> - verify_target_addr_conf_entry(H), - verify_target_addr_conf(T); -verify_target_addr_conf(X) -> - error({bad_target_addr_config, X}). - -verify_target_addr_conf_entry(Entry) -> - {ok, _} = snmp_target_mib:check_target_addr(Entry), - ok. + +check_target_addr(Entry, State) -> + {check_ok(snmp_target_mib:check_target_addr(Entry)), + State}. write_target_addr_conf(Fd, "", Conf) -> write_target_addr_conf(Fd, Conf); @@ -547,29 +486,41 @@ write_target_addr_conf(Fd, Conf) -> lists:foreach(Fun, Conf), ok. -do_write_target_addr_conf(Fd, - {Name, - Ip, Udp, - Timeout, RetryCount, TagList, - ParamsName, EngineId, - TMask, MaxMessageSize}) -> - Domain = snmp_target_mib:default_domain(), - do_write_target_addr_conf(Fd, - {Name, - Domain, Ip, Udp, - Timeout, RetryCount, TagList, - ParamsName, EngineId, - TMask, MaxMessageSize}); -do_write_target_addr_conf(Fd, - {Name, - Domain, Ip, Udp, - Timeout, RetryCount, TagList, - ParamsName, EngineId, - TMask, MaxMessageSize}) -> - io:format(Fd, - "{\"~s\", ~w, ~w, ~w, ~w, ~w, \"~s\", \"~s\", \"~s\", ~w, ~w}.~n", - [Name, Domain, Ip, Udp, Timeout, RetryCount, TagList, - ParamsName, EngineId, TMask, MaxMessageSize]); +do_write_target_addr_conf( + Fd, + {Name, Domain, Address, Timeout, RetryCount, TagList, + ParamsName, EngineId, TMask, MaxMessageSize}) + when is_atom(Domain) -> + io:format( + Fd, + "{\"~s\", ~w, ~w, ~w, ~w, \"~s\", \"~s\", \"~s\", ~w, ~w}.~n", + [Name, Domain, Address, Timeout, RetryCount, TagList, + ParamsName, EngineId, TMask, MaxMessageSize]); +do_write_target_addr_conf( + Fd, + {Name, Ip, Udp, Timeout, RetryCount, TagList, + ParamsName, EngineId, TMask, MaxMessageSize}) + when is_integer(Udp) -> + Domain = snmp_target_mib:default_domain(), + Address = {Ip, Udp}, + do_write_target_addr_conf( + Fd, + {Name, Domain, Address, Timeout, RetryCount, TagList, + ParamsName, EngineId, TMask, MaxMessageSize}); +do_write_target_addr_conf( + _Fd, + {_Name, Domain, Address, _Timeout, _RetryCount, _TagList, + _ParamsName, _EngineId, _TMask, _MaxMessageSize}) -> + error({bad_address, {Domain, Address}}); +do_write_target_addr_conf( + Fd, + {Name, Domain, Ip, Udp, Timeout, RetryCount, TagList, + ParamsName, EngineId, TMask, MaxMessageSize}) -> + Address = {Ip, Udp}, + do_write_target_addr_conf( + Fd, + {Name, Domain, Address, Timeout, RetryCount, TagList, + ParamsName, EngineId, TMask, MaxMessageSize}); do_write_target_addr_conf(_Fd, Crap) -> error({bad_target_addr_config, Crap}). @@ -613,34 +564,27 @@ write_target_params_config(Dir, Conf) -> write_target_params_config(Dir, Hdr, Conf) when is_list(Dir) and is_list(Hdr) and is_list(Conf) -> - Verify = fun() -> verify_target_params_conf(Conf) end, - Write = fun(Fd) -> write_target_params_conf(Fd, Hdr, Conf) end, - write_config_file(Dir, "target_params.conf", Verify, Write). - + Order = fun snmp_conf:no_order/2, + Check = fun check_target_params/2, + Write = fun (Fd, Entries) -> write_target_params_conf(Fd, Hdr, Entries) end, + write_config_file(Dir, "target_params.conf", Order, Check, Write, Conf). append_target_params_config(Dir, Conf) when is_list(Dir) and is_list(Conf) -> - Verify = fun() -> verify_target_params_conf(Conf) end, - Write = fun(Fd) -> write_target_params_conf(Fd, Conf) end, - append_config_file(Dir, "target_params.conf", Verify, Write). - + Order = fun snmp_conf:no_order/2, + Check = fun check_target_params/2, + Write = fun write_target_params_conf/2, + append_config_file(Dir, "target_params.conf", Order, Check, Write, Conf). read_target_params_config(Dir) -> - Verify = fun(Entry) -> verify_target_params_conf_entry(Entry) end, - read_config_file(Dir, "target_params.conf", Verify). + Order = fun snmp_conf:no_order/2, + Check = fun check_target_params/2, + read_config_file(Dir, "target_params.conf", Order, Check). -verify_target_params_conf([]) -> - ok; -verify_target_params_conf([H|T]) -> - verify_target_params_conf_entry(H), - verify_target_params_conf(T); -verify_target_params_conf(X) -> - error({bad_target_params_config, X}). - -verify_target_params_conf_entry(Entry) -> - {ok, _} = snmp_target_mib:check_target_params(Entry), - ok. +check_target_params(Entry, State) -> + {check_ok(snmp_target_mib:check_target_params(Entry)), + State}. write_target_params_conf(Fd, "", Conf) -> write_target_params_conf(Fd, Conf); @@ -685,34 +629,27 @@ write_notify_config(Dir, Conf) -> write_notify_config(Dir, Hdr, Conf) when is_list(Dir) and is_list(Hdr) and is_list(Conf) -> - Verify = fun() -> verify_notify_conf(Conf) end, - Write = fun(Fd) -> write_notify_conf(Fd, Hdr, Conf) end, - write_config_file(Dir, "notify.conf", Verify, Write). - + Order = fun snmp_conf:no_order/2, + Check = fun check_notify/2, + Write = fun (Fd, Entries) -> write_notify_conf(Fd, Hdr, Entries) end, + write_config_file(Dir, "notify.conf", Order, Check, Write, Conf). append_notify_config(Dir, Conf) when is_list(Dir) and is_list(Conf) -> - Verify = fun() -> verify_notify_conf(Conf) end, - Write = fun(Fd) -> write_notify_conf(Fd, Conf) end, - append_config_file(Dir, "notify.conf", Verify, Write). - + Order = fun snmp_conf:no_order/2, + Check = fun check_notify/2, + Write = fun write_notify_conf/2, + append_config_file(Dir, "notify.conf", Order, Check, Write, Conf). read_notify_config(Dir) -> - Verify = fun(Entry) -> verify_notify_conf_entry(Entry) end, - read_config_file(Dir, "notify.conf", Verify). + Order = fun snmp_conf:no_order/2, + Check = fun check_notify/2, + read_config_file(Dir, "notify.conf", Order, Check). -verify_notify_conf([]) -> - ok; -verify_notify_conf([H|T]) -> - verify_notify_conf_entry(H), - verify_notify_conf(T); -verify_notify_conf(X) -> - error({bad_notify_config, X}). - -verify_notify_conf_entry(Entry) -> - {ok, _} = snmp_notification_mib:check_notify(Entry), - ok. +check_notify(Entry, State) -> + {check_ok(snmp_notification_mib:check_notify(Entry)), + State}. write_notify_conf(Fd, "", Conf) -> write_notify_conf(Fd, Conf); @@ -781,34 +718,27 @@ write_usm_config(Dir, Conf) -> write_usm_config(Dir, Hdr, Conf) when is_list(Dir) and is_list(Hdr) and is_list(Conf) -> - Verify = fun() -> verify_usm_conf(Conf) end, - Write = fun(Fd) -> write_usm_conf(Fd, Hdr, Conf) end, - write_config_file(Dir, "usm.conf", Verify, Write). - + Order = fun snmp_conf:no_order/2, + Check = fun check_usm/2, + Write = fun (Fd, Entries) -> write_usm_conf(Fd, Hdr, Entries) end, + write_config_file(Dir, "usm.conf", Order, Check, Write, Conf). append_usm_config(Dir, Conf) when is_list(Dir) and is_list(Conf) -> - Verify = fun() -> verify_usm_conf(Conf) end, - Write = fun(Fd) -> write_usm_conf(Fd, Conf) end, - append_config_file(Dir, "usm.conf", Verify, Write). - + Order = fun snmp_conf:no_order/2, + Check = fun check_usm/2, + Write = fun write_usm_conf/2, + append_config_file(Dir, "usm.conf", Order, Check, Write, Conf). read_usm_config(Dir) -> - Verify = fun(Entry) -> verify_usm_conf_entry(Entry) end, - read_config_file(Dir, "usm.conf", Verify). + Order = fun snmp_conf:no_order/2, + Check = fun check_usm/2, + read_config_file(Dir, "usm.conf", Order, Check). -verify_usm_conf([]) -> - ok; -verify_usm_conf([H|T]) -> - verify_usm_conf_entry(H), - verify_usm_conf(T); -verify_usm_conf(X) -> - error({bad_usm_conf, X}). - -verify_usm_conf_entry(Entry) -> - {ok, _} = snmp_user_based_sm_mib:check_usm(Entry), - ok. +check_usm(Entry, State) -> + {check_ok(snmp_user_based_sm_mib:check_usm(Entry)), + State}. write_usm_conf(Fd, "", Conf) -> write_usm_conf(Fd, Conf); @@ -820,11 +750,12 @@ write_usm_conf(Fd, Conf) -> Fun = fun(Entry) -> do_write_usm_conf(Fd, Entry) end, lists:foreach(Fun, Conf). -do_write_usm_conf(Fd, - {EngineID, UserName, SecName, Clone, - AuthP, AuthKeyC, OwnAuthKeyC, - PrivP, PrivKeyC, OwnPrivKeyC, - Public, AuthKey, PrivKey}) -> +do_write_usm_conf( + Fd, + {EngineID, UserName, SecName, Clone, + AuthP, AuthKeyC, OwnAuthKeyC, + PrivP, PrivKeyC, OwnPrivKeyC, + Public, AuthKey, PrivKey}) -> io:format(Fd, "{", []), io:format(Fd, "\"~s\", ", [EngineID]), io:format(Fd, "\"~s\", ", [UserName]), @@ -890,34 +821,27 @@ write_vacm_config(Dir, Conf) -> write_vacm_config(Dir, Hdr, Conf) when is_list(Dir) and is_list(Hdr) and is_list(Conf) -> - Verify = fun() -> verify_vacm_conf(Conf) end, - Write = fun(Fd) -> write_vacm_conf(Fd, Hdr, Conf) end, - write_config_file(Dir, "vacm.conf", Verify, Write). - + Order = fun snmp_conf:no_order/2, + Check = fun check_vacm/2, + Write = fun (Fd, Entries) -> write_vacm_conf(Fd, Hdr, Entries) end, + write_config_file(Dir, "vacm.conf", Order, Check, Write, Conf). append_vacm_config(Dir, Conf) when is_list(Dir) and is_list(Conf) -> - Verify = fun() -> verify_vacm_conf(Conf) end, - Write = fun(Fd) -> write_vacm_conf(Fd, Conf) end, - append_config_file(Dir, "vacm.conf", Verify, Write). - + Order = fun snmp_conf:no_order/2, + Check = fun check_vacm/2, + Write = fun write_vacm_conf/2, + append_config_file(Dir, "vacm.conf", Order, Check, Write, Conf). read_vacm_config(Dir) -> - Verify = fun(Entry) -> verify_vacm_conf_entry(Entry) end, - read_config_file(Dir, "vacm.conf", Verify). + Order = fun snmp_conf:no_order/2, + Check = fun check_vacm/2, + read_config_file(Dir, "vacm.conf", Order, Check). -verify_vacm_conf([]) -> - ok; -verify_vacm_conf([H|T]) -> - verify_vacm_conf_entry(H), - verify_vacm_conf(T); -verify_vacm_conf(X) -> - error({bad_vacm_conf, X}). - -verify_vacm_conf_entry(Entry) -> - {ok, _} = snmp_view_based_acm_mib:check_vacm(Entry), - ok. +check_vacm(Entry, State) -> + {check_ok(snmp_view_based_acm_mib:check_vacm(Entry)), + State}. write_vacm_conf(Fd, "", Conf) -> write_vacm_conf(Fd, Conf); @@ -929,49 +853,60 @@ write_vacm_conf(Fd, Conf) -> Fun = fun(Entry) -> do_write_vacm_conf(Fd, Entry) end, lists:foreach(Fun, Conf). -do_write_vacm_conf(Fd, - {vacmSecurityToGroup, - SecModel, SecName, GroupName}) -> - io:format(Fd, "{vacmSecurityToGroup, ~w, \"~s\", \"~s\"}.~n", - [SecModel, SecName, GroupName]); -do_write_vacm_conf(Fd, - {vacmAccess, - GroupName, Prefix, SecModel, SecLevel, Match, RV, WV, NV}) -> - io:format(Fd, "{vacmAccess, \"~s\", \"~s\", ~w, ~w, ~w, " - "\"~s\", \"~s\", \"~s\"}.~n", - [GroupName, Prefix, SecModel, SecLevel, - Match, RV, WV, NV]); -do_write_vacm_conf(Fd, - {vacmViewTreeFamily, - ViewIndex, ViewSubtree, ViewStatus, ViewMask}) -> - io:format(Fd, "{vacmViewTreeFamily, \"~s\", ~w, ~w, ~w}.~n", - [ViewIndex, ViewSubtree, ViewStatus, ViewMask]); +do_write_vacm_conf( + Fd, + {vacmSecurityToGroup, + SecModel, SecName, GroupName}) -> + io:format( + Fd, "{vacmSecurityToGroup, ~w, \"~s\", \"~s\"}.~n", + [SecModel, SecName, GroupName]); +do_write_vacm_conf( + Fd, + {vacmAccess, + GroupName, Prefix, SecModel, SecLevel, Match, RV, WV, NV}) -> + io:format( + Fd, "{vacmAccess, \"~s\", \"~s\", ~w, ~w, ~w, " + "\"~s\", \"~s\", \"~s\"}.~n", + [GroupName, Prefix, SecModel, SecLevel, + Match, RV, WV, NV]); +do_write_vacm_conf( + Fd, + {vacmViewTreeFamily, + ViewIndex, ViewSubtree, ViewStatus, ViewMask}) -> + io:format( + Fd, "{vacmViewTreeFamily, \"~s\", ~w, ~w, ~w}.~n", + [ViewIndex, ViewSubtree, ViewStatus, ViewMask]); do_write_vacm_conf(_Fd, Crap) -> error({bad_vacm_config, Crap}). %% ---- config file wrapper functions ---- -write_config_file(Dir, File, Verify, Write) -> - snmp_config:write_config_file(Dir, File, Verify, Write). +write_config_file(Dir, File, Order, Check, Write, Conf) -> + snmp_config:write_config_file(Dir, File, Order, Check, Write, Conf). -append_config_file(Dir, File, Verify, Write) -> - snmp_config:append_config_file(Dir, File, Verify, Write). +append_config_file(Dir, File, Order, Check, Write, Conf) -> + snmp_config:append_config_file(Dir, File, Order, Check, Write, Conf). -read_config_file(Dir, File, Verify) -> - snmp_config:read_config_file(Dir, File, Verify). +read_config_file(Dir, File, Order, Check) -> + snmp_config:read_config_file(Dir, File, Order, Check). %% ---- config file utility functions ---- +check_ok(ok) -> + ok; +check_ok({ok, _}) -> + ok. + header() -> {Y,Mo,D} = date(), {H,Mi,S} = time(), - io_lib:format("%% This file was generated by " - "~w (version-~s) ~w-~2.2.0w-~2.2.0w " - "~2.2.0w:~2.2.0w:~2.2.0w\n", - [?MODULE, ?version, Y, Mo, D, H, Mi, S]). - + io_lib:format( + "%% This file was generated by " + "~w (version-~s) ~w-~2.2.0w-~2.2.0w " + "~2.2.0w:~2.2.0w:~2.2.0w\n", + [?MODULE, ?version, Y, Mo, D, H, Mi, S]). error(R) -> throw({error, R}). diff --git a/lib/snmp/src/agent/snmpa_local_db.erl b/lib/snmp/src/agent/snmpa_local_db.erl index f991244287..292c370d51 100644 --- a/lib/snmp/src/agent/snmpa_local_db.erl +++ b/lib/snmp/src/agent/snmpa_local_db.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2013. All Rights Reserved. +%% Copyright Ericsson AB 1996-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -1011,6 +1011,10 @@ table_construct_row(Name, RowIndex, Status, Cols) -> defvals = Defs, status_col = StatusCol, first_own_index = FirstOwnIndex, not_accessible = NoAccs} = snmp_generic:table_info(Name), + ?vtrace( + "table_construct_row Indexes: ~p~n" + " RowIndex: ~p", + [Indexes, RowIndex]), Keys = snmp_generic:split_index_to_keys(Indexes, RowIndex), OwnKeys = snmp_generic:get_own_indexes(FirstOwnIndex, Keys), Row = OwnKeys ++ snmp_generic:table_create_rest(length(OwnKeys) + 1, diff --git a/lib/snmp/src/agent/snmpa_mpd.erl b/lib/snmp/src/agent/snmpa_mpd.erl index 11ae806866..642b1f7fc5 100644 --- a/lib/snmp/src/agent/snmpa_mpd.erl +++ b/lib/snmp/src/agent/snmpa_mpd.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2013. All Rights Reserved. +%% Copyright Ericsson AB 1997-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -20,7 +20,7 @@ -export([init/1, reset/0, inc/1, counters/0, discarded_pdu/1, - process_packet/6, process_packet/7, + process_packet/5, process_packet/6, process_packet/7, generate_response_msg/5, generate_response_msg/6, generate_msg/5, generate_msg/6, generate_discovery_msg/4, @@ -113,22 +113,30 @@ reset() -> % length(snmp_pdus:enc_message(M)) + 4. %%----------------------------------------------------------------- -%% Func: process_packet(Packet, TDomain, TAddress, State, Log) -> +%% Func: process_packet(Packet, Domain, Address, State, Log) -> %% {ok, SnmpVsn, Pdu, PduMS, ACMData} | {discarded, Reason} %% Types: Packet = binary() -%% TDomain = snmpUDPDomain | transportDomain() -%% TAddress = {Ip, Udp} (*but* depends on TDomain) +%% Domain = snmpUDPDomain | transportDomain() +%% Address = {Ip, Udp} (*but* depends on Domain) %% State = #state %% Purpose: This is the main Message Dispatching function. (see %% section 4.2.1 in rfc2272) %%----------------------------------------------------------------- -process_packet(Packet, TDomain, TAddress, State, NoteStore, Log) -> +process_packet(Packet, From, State, NoteStore, Log) -> LocalEngineID = ?DEFAULT_LOCAL_ENGINE_ID, - process_packet(Packet, TDomain, TAddress, LocalEngineID, - State, NoteStore, Log). - -process_packet(Packet, TDomain, TAddress, LocalEngineID, - State, NoteStore, Log) -> + process_packet(Packet, From, LocalEngineID, State, NoteStore, Log). + +process_packet( + Packet, Domain, Address, LocalEngineID, State, NoteStore, Log) -> + From = {Domain, Address}, + process_packet(Packet, From, LocalEngineID, State, NoteStore, Log). + +process_packet(Packet, Domain, Address, State, NoteStore, Log) + when is_atom(Domain) -> + LocalEngineID = ?DEFAULT_LOCAL_ENGINE_ID, + From = {Domain, Address}, + process_packet(Packet, From, LocalEngineID, State, NoteStore, Log); +process_packet(Packet, From, LocalEngineID, State, NoteStore, Log) -> inc(snmpInPkts), case catch snmp_pdus:dec_message_only(binary_to_list(Packet)) of @@ -136,17 +144,17 @@ process_packet(Packet, TDomain, TAddress, LocalEngineID, when State#state.v1 =:= true -> ?vlog("v1, community: ~s", [Community]), HS = ?empty_msg_size + length(Community), - v1_v2c_proc('version-1', NoteStore, Community, - TDomain, TAddress, - LocalEngineID, Data, HS, Log, Packet); + v1_v2c_proc( + 'version-1', NoteStore, Community, From, + LocalEngineID, Data, HS, Log, Packet); #message{version = 'version-2', vsn_hdr = Community, data = Data} when State#state.v2c =:= true -> ?vlog("v2c, community: ~s", [Community]), HS = ?empty_msg_size + length(Community), - v1_v2c_proc('version-2', NoteStore, Community, - TDomain, TAddress, - LocalEngineID, Data, HS, Log, Packet); + v1_v2c_proc( + 'version-2', NoteStore, Community, From, + LocalEngineID, Data, HS, Log, Packet); #message{version = 'version-3', vsn_hdr = V3Hdr, data = Data} when State#state.v3 =:= true -> @@ -154,9 +162,9 @@ process_packet(Packet, TDomain, TAddress, LocalEngineID, [V3Hdr#v3_hdr.msgID, V3Hdr#v3_hdr.msgFlags, V3Hdr#v3_hdr.msgSecurityModel]), - v3_proc(NoteStore, Packet, - TDomain, TAddress, - LocalEngineID, V3Hdr, Data, Log); + v3_proc( + NoteStore, Packet, From, + LocalEngineID, V3Hdr, Data, Log); {'EXIT', {bad_version, Vsn}} -> ?vtrace("exit: bad version: ~p",[Vsn]), @@ -183,11 +191,32 @@ discarded_pdu(Variable) -> inc(Variable). %%----------------------------------------------------------------- %% Handles a Community based message (v1 or v2c). %%----------------------------------------------------------------- -v1_v2c_proc(Vsn, NoteStore, Community, Domain, - {Ip, Udp}, LocalEngineID, - Data, HS, Log, Packet) -> - TDomain = snmp_conf:mk_tdomain(Domain), - TAddress = snmp_conf:mk_taddress(Domain, Ip, Udp), +v1_v2c_proc( + Vsn, NoteStore, Community, From, + LocalEngineID, Data, HS, Log, Packet) -> + try + case From of + {D, A} when is_atom(D) -> + {snmp_conf:mk_tdomain(D), + snmp_conf:mk_taddress(D, A)}; + {_, P} = A when is_integer(P) -> + {snmp_conf:mk_tdomain(), + snmp_conf:mk_taddress(A)} + end + of + {TDomain, TAddress} -> + v1_v2c_proc_dec( + Vsn, NoteStore, Community, TDomain, TAddress, + LocalEngineID, Data, HS, Log, Packet) + catch + _ -> + {discarded, {badarg, From}} + end. + + +v1_v2c_proc_dec( + Vsn, NoteStore, Community, TDomain, TAddress, + LocalEngineID, Data, HS, Log, Packet) -> AgentMS = get_engine_max_message_size(LocalEngineID), MgrMS = snmp_community_mib:get_target_addr_ext_mms(TDomain, TAddress), PduMS = case MgrMS of @@ -214,7 +243,7 @@ v1_v2c_proc(Vsn, NoteStore, Community, Domain, case Pdu#pdu.type of 'set-request' -> %% Check if this message has already been processed - Key = {agent, Ip, ReqId}, + Key = {agent, {TDomain, TAddress}, ReqId}, case snmp_note_store:get_note(NoteStore, Key) of undefined -> %% Set the processed note _after_ pdu processing. @@ -236,13 +265,7 @@ v1_v2c_proc(Vsn, NoteStore, Community, Domain, {discarded, Reason}; _TrapPdu -> {discarded, trap_pdu} - end; -v1_v2c_proc(_Vsn, _NoteStore, _Community, snmpUDPDomain, TAddress, - _LocalEngineID, _Data, _HS, _Log, _Packet) -> - {discarded, {badarg, TAddress}}; -v1_v2c_proc(_Vsn, _NoteStore, _Community, TDomain, _TAddress, - _LocalEngineID, _Data, _HS, _Log, _Packet) -> - {discarded, {badarg, TDomain}}. + end. sec_model('version-1') -> ?SEC_V1; sec_model('version-2') -> ?SEC_V2C. @@ -252,8 +275,7 @@ sec_model('version-2') -> ?SEC_V2C. %% Handles a SNMPv3 Message, following the procedures in rfc2272, %% section 4.2 and 7.2 %%----------------------------------------------------------------- -v3_proc(NoteStore, Packet, _TDomain, _TAddress, LocalEngineID, - V3Hdr, Data, Log) -> +v3_proc(NoteStore, Packet, _From, LocalEngineID, V3Hdr, Data, Log) -> case (catch v3_proc(NoteStore, Packet, LocalEngineID, V3Hdr, Data, Log)) of {'EXIT', Reason} -> exit(Reason); @@ -999,7 +1021,7 @@ generate_discovery_msg(NoteStore, {TDomain, TAddress}, InitialUserName, ContextName, Timeout) -> - {ok, {_Domain, Address}} = transform_taddr(TDomain, TAddress), + {ok, {Domain, Address}} = transform_taddr(TDomain, TAddress), %% 7.1.7 ?vdebug("generate_discovery_msg -> 7.1.7 (~w)", [ManagerEngineID]), @@ -1041,7 +1063,7 @@ generate_discovery_msg(NoteStore, {TDomain, TAddress}, %% Log(Packet), inc_snmp_out_vars(Pdu), ?vdebug("generate_discovery_msg -> done", []), - {Packet, Address}; + {Domain, Address, Packet}; Error -> throw(Error) @@ -1081,8 +1103,22 @@ transform_taddr(?transportDomainUdpIpv4, [A, B, C, D, P1, P2]) -> {ok, {Domain, Address}}; transform_taddr(?transportDomainUdpIpv4, BadAddr) -> {error, {bad_transportDomainUdpIpv4_address, BadAddr}}; -transform_taddr(?transportDomainUdpIpv6, - [A1, A2, A3, A4, A5, A6, A7, A8, P1, P2]) -> +transform_taddr( + ?transportDomainUdpIpv6, + [A1, A2, A3, A4, A5, A6, A7, A8, A9, A10, A11, A12, A13, A14, A15, A16, + P1, P2]) -> + Domain = transportDomainUdpIpv6, + Addr = + {(A1 bsl 8) bor A2, (A3 bsl 8) bor A4, + (A5 bsl 8) bor A6, (A7 bsl 8) bor A8, + (A9 bsl 8) bor A10, (A11 bsl 8) bor A12, + (A13 bsl 8) bor A14, (A15 bsl 8) bor A16}, + Port = P1 bsl 8 + P2, + Address = {Addr, Port}, + {ok, {Domain, Address}}; +transform_taddr( + ?transportDomainUdpIpv6, + [A1, A2, A3, A4, A5, A6, A7, A8, P1, P2]) -> Domain = transportDomainUdpIpv6, Addr = {A1, A2, A3, A4, A5, A6, A7, A8}, Port = P1 bsl 8 + P2, @@ -1171,6 +1207,9 @@ mk_v1_v2_packet_list([{Domain, Addr} | T], %% Sending from default UDP port inc_snmp_out_vars(Pdu), Entry = {Domain, Addr, Packet}, + %% It would be cleaner to return {To, Packet} to not + %% break the abstraction for an address on the + %% {Domain, Address} format. mk_v1_v2_packet_list(T, Packet, Len, Pdu, [Entry | Acc]). @@ -1277,6 +1316,9 @@ mk_v3_packet_entry(NoteStore, Domain, Addr, req_id = Pdu#pdu.request_id}, snmp_note_store:set_note(NoteStore, 1500, CacheKey, CacheVal), inc_snmp_out_vars(Pdu), + %% It would be cleaner to return {To, Packet} to not + %% break the abstraction for an address on the + %% {Domain, Address} format. {ok, {Domain, Addr, Data}} end. diff --git a/lib/snmp/src/agent/snmpa_net_if.erl b/lib/snmp/src/agent/snmpa_net_if.erl index 79c85a6e4e..840d56d563 100644 --- a/lib/snmp/src/agent/snmpa_net_if.erl +++ b/lib/snmp/src/agent/snmpa_net_if.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2011. All Rights Reserved. +%% Copyright Ericsson AB 2004-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -35,18 +35,27 @@ -include("snmp_debug.hrl"). -include("snmp_verbosity.hrl"). --record(state, {parent, - note_store, - master_agent, - usock, - usock_opts, - mpd_state, - log, - reqs = [], - debug = false, - limit = infinity, - rcnt = [], - filter}). +-record(state, + {parent, + note_store, + master_agent, + transports = [], +%% usock, +%% usock_opts, + mpd_state, + log, + reqs = [], + debug = false, + limit = infinity, +%% rcnt = [], + filter}). +%% domain = snmpUDPDomain}). + +-record(transport, + {socket, + domain = snmpUDPDomain, + opts = [], + req_refs = []}). -ifndef(default_verbosity). -define(default_verbosity,silence). @@ -104,13 +113,9 @@ get_request_limit(Pid) -> set_request_limit(Pid, NewLimit) -> call(Pid, {set_request_limit, NewLimit}). -get_port() -> - {value, UDPPort} = snmp_framework_mib:intAgentUDPPort(get), - UDPPort. - -get_address() -> - {value, IPAddress} = snmp_framework_mib:intAgentIpAddress(get), - IPAddress. +get_transports() -> + {value, Transports} = snmp_framework_mib:intAgentTransports(get), + Transports. filter_reset(Pid) -> Pid ! filter_reset. @@ -129,7 +134,22 @@ init(Prio, NoteStore, MasterAgent, Parent, Opts) -> case (catch do_init(Prio, NoteStore, MasterAgent, Parent, Opts)) of {ok, State} -> proc_lib:init_ack({ok, self()}), - loop(State); + try loop(State) + catch + C:E when C =/= exit, E =/= shutdown -> + Fmt = + "loop/1 EXCEPTION ~w:~w~n" + " ~p", + S = erlang:get_stacktrace(), + case C of + exit -> + %% Externally killed, root cause is elsewhere + info_msg(Fmt, [C, E, S]); + _ -> + error_msg(Fmt, [C, E, S]) + end, + erlang:raise(C, E, S) + end; {error, Reason} -> config_err("failed starting net-if: ~n~p", [Reason]), proc_lib:init_ack({error, Reason}); @@ -147,12 +167,6 @@ do_init(Prio, NoteStore, MasterAgent, Parent, Opts) -> put(verbosity,get_verbosity(Opts)), ?vlog("starting",[]), - %% -- Port and address -- - UDPPort = get_port(), - ?vdebug("port: ~w",[UDPPort]), - IPAddress = get_address(), - ?vdebug("addr: ~w",[IPAddress]), - %% -- Versions -- Vsns = get_vsns(Opts), ?vdebug("vsns: ~w",[Vsns]), @@ -162,39 +176,45 @@ do_init(Prio, NoteStore, MasterAgent, Parent, Opts) -> ?vdebug("Limit: ~w", [Limit]), FilterOpts = get_filter_opts(Opts), FilterMod = create_filter(FilterOpts), - ?vdebug("FilterMod: ~w", [FilterMod]), + ?vdebug("FilterMod: ~w FilterOpts: ~p", [FilterMod,FilterOpts]), %% -- Audit trail log Log = create_log(), ?vdebug("Log: ~w",[Log]), - - %% -- Socket -- - IPOpts1 = ip_opt_bind_to_ip_address(Opts, IPAddress), - IPOpts2 = ip_opt_no_reuse_address(Opts), - IPOpts3 = ip_opt_recbuf(Opts), - IPOpts4 = ip_opt_sndbuf(Opts), - IPOpts = [binary | IPOpts1 ++ IPOpts2 ++ IPOpts3 ++ IPOpts4], - ?vdebug("open socket with options: ~w",[IPOpts]), - case gen_udp_open(UDPPort, IPOpts) of - {ok, Sock} -> + DomainAddresses = get_transports(), + ?vdebug("DomainAddresses: ~w",[DomainAddresses]), + try + [begin + SocketOpts = socket_opts(Domain, Address, Opts), + Socket = socket_open(Domain, SocketOpts), + active_once(Socket), + #transport{ + socket = Socket, + domain = Domain, + opts = SocketOpts} + end || {Domain, Address} <- DomainAddresses] + of + [] -> + ?vinfo("No transports configured: ~p", [DomainAddresses]), + {error, {no_transports,DomainAddresses}}; + Transports -> MpdState = snmpa_mpd:init(Vsns), - init_counters(), - active_once(Sock), + init_counters(), S = #state{parent = Parent, note_store = NoteStore, master_agent = MasterAgent, mpd_state = MpdState, - usock = Sock, - usock_opts = IPOpts, + transports = Transports, log = Log, limit = Limit, filter = FilterMod}, ?vdebug("started with MpdState: ~p", [MpdState]), - {ok, S}; - {error, Reason} -> - ?vinfo("Failed to open UDP socket: ~p", [Reason]), - {error, {udp_open, UDPPort, Reason}} + {ok, S} + catch + Error -> + ?vinfo("Failed to initialize socket(s): ~p", [Error]), + {error, Error} end. @@ -250,48 +270,84 @@ create_filter(BadOpts) -> throw({error, {bad_filter_opts, BadOpts}}). -log({_, []}, _, _, _, _) -> +log({_, []}, _, _, _) -> ok; -log({Log, Types}, 'set-request', Packet, Addr, Port) -> +log({Log, Types}, 'set-request', Packet, Address) -> case lists:member(write, Types) of true -> - snmp_log:log(Log, Packet, Addr, Port); + snmp_log:log(Log, Packet, format_address(Address)); false -> ok end; -log({Log, Types}, _, Packet, Addr, Port) -> +log({Log, Types}, _, Packet, Address) -> case lists:member(read, Types) of true -> - snmp_log:log(Log, Packet, Addr, Port); + snmp_log:log(Log, Packet, format_address(Address)); false -> ok - end; -log(_, _, _, _, _) -> - ok. - - -gen_udp_open(Port, Opts) -> + end. + +format_address(Address) -> + iolist_to_binary(snmp_conf:mk_addr_string(Address)). + + +socket_open(snmpUDPDomain = Domain, [IpPort | Opts]) -> case init:get_argument(snmp_fd) of {ok, [[FdStr]]} -> Fd = list_to_integer(FdStr), - gen_udp:open(0, [{fd, Fd}|Opts]); + ?vdebug("socket_open(~p, [~p | ~p]) Fd: ~p", + [Domain, IpPort, Opts, Fd]), + gen_udp_open(IpPort, [{fd, Fd} | Opts]); error -> case init:get_argument(snmpa_fd) of {ok, [[FdStr]]} -> Fd = list_to_integer(FdStr), - gen_udp:open(0, [{fd, Fd}|Opts]); + ?vdebug("socket_open(~p, [~p | ~p]) Fd: ~p", + [Domain, IpPort, Opts, Fd]), + gen_udp_open(IpPort, [{fd, Fd} | Opts]); error -> - gen_udp:open(Port, Opts) + ?vdebug("socket_open(~p, [~p | ~p])", + [Domain, IpPort, Opts]), + gen_udp_open(IpPort, Opts) end + end; +socket_open(Domain, [IpPort | Opts]) + when Domain =:= transportDomainUdpIpv4; + Domain =:= transportDomainUdpIpv6 -> + ?vdebug("socket_open(~p, [~p | ~p])", [Domain, IpPort, Opts]), + gen_udp_open(IpPort, Opts); +socket_open(Domain, Opts) -> + throw({socket_open, Domain, Opts}). + +gen_udp_open(IpPort, Opts) -> + case gen_udp:open(IpPort, Opts) of + {ok, Socket} -> + Socket; + {error, Reason} -> + throw({udp_open, IpPort, Reason}) end. -loop(S) -> + +loop(#state{transports = Transports, limit = Limit, parent = Parent} = S) -> + ?vdebug("loop(~p)", [S]), receive - {udp, _UdpId, Ip, Port, Packet} -> - ?vlog("got paket from ~w:~w",[Ip,Port]), - NewS = maybe_handle_recv(S, Ip, Port, Packet), - loop(NewS); + {udp, Socket, IpAddr, IpPort, Packet} = Msg when is_port(Socket) -> + ?vlog("got paket from ~w:~w on ~w", [IpAddr, IpPort, Socket]), + case lists:keyfind(Socket, #transport.socket, Transports) of + #transport{socket = Socket, domain = Domain} = Transport -> + From = + case Domain of + snmpUDPDomain -> + {IpAddr, IpPort}; + _ -> + {Domain, {IpAddr, IpPort}} + end, + loop(maybe_handle_recv(S, Transport, From, Packet)); + false -> + error_msg("Packet on unknown port: ~p", [Msg]), + loop(S) + end; {info, ReplyRef, Pid} -> Info = get_info(S), @@ -299,47 +355,78 @@ loop(S) -> loop(S); %% response (to get/get_next/get_bulk/set requests) - {snmp_response, Vsn, RePdu, Type, ACMData, Dest, []} -> + {snmp_response, Vsn, RePdu, Type, ACMData, To, Extra} -> ?vlog("reply pdu: " "~n ~s", [?vapply(snmp_misc, format, [256, "~w", [RePdu]])]), - NewS = maybe_handle_reply_pdu(S, Vsn, RePdu, Type, ACMData, Dest), - loop(NewS); + {_, ReqRef} = lists:keyfind(request_ref, 1, Extra), + case + case + (Limit =/= infinity) andalso + select_transport_from_req_ref(ReqRef, Transports) + of + false -> + select_transport_from_domain( + address_to_domain(To), + Transports); + T -> + T + end + of + false -> + error_msg( + "Can not find transport for response PDU to: ~s", + [format_address(To)]), + loop(S); + Transport -> + NewS = update_req_counter_outgoing(S, Transport, ReqRef), + maybe_handle_reply_pdu( + NewS, Transport, Vsn, RePdu, Type, ACMData, To), + loop(NewS) + end; %% Traps/notification - {send_pdu, Vsn, Pdu, MsgData, To} -> - ?vdebug("send pdu: " - "~n Pdu: ~p" - "~n To: ~p", [Pdu, To]), - NewS = maybe_handle_send_pdu(S, Vsn, Pdu, MsgData, To, undefined), + {send_pdu, Vsn, Pdu, MsgData, TDomAddrs} -> + ?vdebug("send pdu:~n" + " Pdu: ~p~n" + " TDomAddrs: ~p", [Pdu, TDomAddrs]), + NewS = + maybe_handle_send_pdu( + S, Vsn, Pdu, MsgData, TDomAddrs, undefined), loop(NewS); %% We dont use the extra-info at this time, ... - {send_pdu, Vsn, Pdu, MsgData, To, _ExtraInfo} -> - ?vdebug("send pdu: " - "~n Pdu: ~p" - "~n To: ~p", [Pdu, To]), - NewS = maybe_handle_send_pdu(S, Vsn, Pdu, MsgData, To, undefined), + {send_pdu, Vsn, Pdu, MsgData, TDomAddrs, _ExtraInfo} -> + ?vdebug("send pdu:~n" + " Pdu: ~p~n" + " TDomAddrs: ~p", [Pdu, TDomAddrs]), + NewS = + maybe_handle_send_pdu( + S, Vsn, Pdu, MsgData, TDomAddrs, undefined), loop(NewS); %% Informs - {send_pdu_req, Vsn, Pdu, MsgData, To, From} -> - ?vdebug("send pdu request: " - "~n Pdu: ~p" - "~n To: ~p" - "~n From: ~p", - [Pdu, To, toname(From)]), - NewS = maybe_handle_send_pdu(S, Vsn, Pdu, MsgData, To, From), + {send_pdu_req, Vsn, Pdu, MsgData, TDomAddrs, From} -> + ?vdebug("send pdu request:~n" + " Pdu: ~p~n" + " TDomAddrs: ~p~n" + " From: ~p", + [Pdu, TDomAddrs, toname(From)]), + NewS = + maybe_handle_send_pdu( + S, Vsn, Pdu, MsgData, TDomAddrs, From), loop(NewS); %% We dont use the extra-info at this time, ... - {send_pdu_req, Vsn, Pdu, MsgData, To, From, _ExtraInfo} -> - ?vdebug("send pdu request: " - "~n Pdu: ~p" - "~n To: ~p" - "~n From: ~p", - [Pdu, To, toname(From)]), - NewS = maybe_handle_send_pdu(S, Vsn, Pdu, MsgData, To, From), + {send_pdu_req, Vsn, Pdu, MsgData, TDomAddrs, From, _ExtraInfo} -> + ?vdebug("send pdu request:~n" + " Pdu: ~p~n" + " TDomAddrs: ~p~n" + " From: ~p", + [Pdu, TDomAddrs, toname(From)]), + NewS = + maybe_handle_send_pdu( + S, Vsn, Pdu, MsgData, TDomAddrs, From), loop(NewS); %% Discovery Inform @@ -365,15 +452,34 @@ loop(S) -> NewS = handle_send_discovery(S, Pdu, MsgData, To, From), loop(NewS); - {discarded_pdu, _Vsn, ReqId, _ACMData, Variable, _Extra} -> - ?vdebug("discard PDU: ~p", [Variable]), + {discarded_pdu, _Vsn, ReqId, _ACMData, Variable, Extra} -> + ?vdebug("discard PDU: ~p - ~p - ~p", + [Variable, Extra, Transports]), snmpa_mpd:discarded_pdu(Variable), - NewS = update_req_counter_outgoing(S, ReqId), - loop(NewS); + {_, ReqRef} = lists:keyfind(request_ref, 1, Extra), + if + Limit =:= infinity -> + %% The incoming PDU was not registered + loop(update_req_counter_outgoing(S, false, ReqRef)); + true -> + case + select_transport_from_req_ref(ReqRef, Transports) + of + false -> + error_msg( + "Can not find transport for discarded PDU: ~p", + [ReqId]), + loop(S); + Transport -> + loop( + update_req_counter_outgoing( + S, Transport, ReqRef)) + end + end; {get_log_type, ReplyRef, Pid} -> ?vdebug("get log type: ~p", []), - #state{log = {_, LogType}} = S, + {_, LogType} = S#state.log, Pid ! {ReplyRef, {ok, LogType}, self()}, loop(S); @@ -385,7 +491,6 @@ loop(S) -> {get_request_limit, ReplyRef, Pid} -> ?vdebug("get request limit: ~p", []), - #state{limit = Limit} = S, Pid ! {ReplyRef, {ok, Limit}, self()}, loop(S); @@ -409,36 +514,50 @@ loop(S) -> reset_counters(), loop(S); - {'EXIT', Parent, Reason} when Parent == S#state.parent -> + {'EXIT', Parent, Reason} -> ?vlog("parent (~p) exited: " "~n ~p", [Parent, Reason]), exit(Reason); - {'EXIT', Port, Reason} when Port == S#state.usock -> - UDPPort = get_port(), - NewS = - case gen_udp_open(UDPPort, S#state.usock_opts) of - {ok, Id} -> - error_msg("Port ~p exited for reason" - "~n ~p" - "~n Re-opened (~p)", [Port, Reason, Id]), - S#state{usock = Id}; - {error, ReopenReason} -> - error_msg("Port ~p exited for reason" - "~n ~p" - "~n Re-open failed with reason" - "~n ~p", - [Port, Reason, ReopenReason]), - ok - end, - loop(NewS); - - {'EXIT', Port, Reason} when is_port(Port) -> - error_msg("Exit message from port ~p for reason ~p~n", - [Port, Reason]), - loop(S); - - {'EXIT', Pid, Reason} -> + {'EXIT', Socket, Reason} when is_port(Socket) -> + case lists:keyfind(Socket, #transport.socket, Transports) of + #transport{ + socket = Socket, + domain = Domain, + opts = SocketOpts, + req_refs = ReqRefs} = Transport -> + try socket_open(Domain, SocketOpts) of + NewSocket -> + error_msg( + "Socket ~p exited for reason" + "~n ~p" + "~n Re-opened (~p)", + [Socket, Reason, NewSocket]), + (length(ReqRefs) < Limit) andalso + active_once(NewSocket), + S#state{ + transports = + lists:keyreplace( + Socket, #transport.socket, Transports, + Transport#transport{socket = NewSocket})} + catch + ReopenReason -> + error_msg( + "Socket ~p exited for reason" + "~n ~p" + "~n Re-open failed with reason" + "~n ~p", + [Socket, Reason, ReopenReason]), + exit(ReopenReason) + end; + false -> + error_msg( + "Exit message from port ~p for reason ~p~n", + [Socket, Reason]), + loop(S) + end; + + {'EXIT', Pid, Reason} when is_pid(Pid) -> ?vlog("~p exited: " "~n ~p", [Pid, Reason]), NewS = clear_reqs(Pid, S), @@ -446,205 +565,321 @@ loop(S) -> {system, From, Msg} -> ?vdebug("system event ~p from ~p", [Msg, From]), - sys:handle_system_msg(Msg, From, S#state.parent, ?MODULE, [], S); + sys:handle_system_msg(Msg, From, Parent, ?MODULE, [], S); _ -> loop(S) end. -update_req_counter_incomming(#state{limit = infinity, usock = Sock} = S, _) -> - active_once(Sock), %% No limit so activate directly +update_req_counter_incoming( + #state{limit = infinity} = S, + #transport{socket = Socket}, + _ReqRef) -> + active_once(Socket), %% No limit so activate directly S; -update_req_counter_incomming(#state{limit = Limit, - rcnt = RCnt, - usock = Sock} = S, Rid) - when length(RCnt) + 1 == Limit -> +update_req_counter_incoming( + #state{limit = Limit} = S, + #transport{socket = Socket, req_refs = ReqRefs} = T, + ReqRef) when length(ReqRefs) + 1 >= Limit -> %% Ok, one more and we are at the limit. %% Just make sure we are not already processing this one... - case lists:member(Rid, RCnt) of + case lists:member(ReqRef, ReqRefs) of false -> %% We are at the limit, do _not_ activate socket - S#state{rcnt = [Rid|RCnt]}; + update_transport_req_refs(S, T, [ReqRef | ReqRefs]); true -> - active_once(Sock), + active_once(Socket), S end; -update_req_counter_incomming(#state{rcnt = RCnt, - usock = Sock} = S, Rid) -> - active_once(Sock), - case lists:member(Rid, RCnt) of +update_req_counter_incoming( + #state{} = S, + #transport{socket = Socket, req_refs = ReqRefs} = T, + ReqRef) -> + active_once(Socket), + case lists:member(ReqRef, ReqRefs) of false -> - S#state{rcnt = [Rid|RCnt]}; + update_transport_req_refs(S, T, [ReqRef | ReqRefs]); true -> S end. - -update_req_counter_outgoing(#state{limit = infinity} = S, _Rid) -> +update_req_counter_outgoing( + #state{limit = infinity} = S, + _Transport, _ReqRef) -> %% Already activated (in the incoming function) S; -update_req_counter_outgoing(#state{limit = Limit, - rcnt = RCnt, - usock = Sock} = S, Rid) - when length(RCnt) == Limit -> - ?vtrace("handle_req_counter_outgoing(~w) -> entry with" - "~n Rid: ~w" - "~n length(RCnt): ~w", [Limit, Rid, length(RCnt)]), - case lists:delete(Rid, RCnt) of - NewRCnt when length(NewRCnt) < Limit -> +update_req_counter_outgoing( + #state{limit = Limit} = S, + #transport{socket = Socket, req_refs = ReqRefs} = Transport, + ReqRef) -> + LengthReqRefs = length(ReqRefs), + ?vtrace("update_req_counter_outgoing() -> entry with~n" + " Limit: ~w~n" + " ReqRef: ~w~n" + " length(ReqRefs): ~w", [Limit, ReqRef, LengthReqRefs]), + NewReqRefs = lists:delete(ReqRef, ReqRefs), + (LengthReqRefs >= Limit) andalso (length(NewReqRefs) < Limit) andalso + begin ?vtrace("update_req_counter_outgoing -> " - "passed below limit: activate", []), - active_once(Sock), - S#state{rcnt = NewRCnt}; - _ -> - S - end; -update_req_counter_outgoing(#state{limit = Limit, rcnt = RCnt} = S, - Rid) -> - ?vtrace("handle_req_counter_outgoing(~w) -> entry with" - "~n Rid: ~w" - "~n length(RCnt): ~w", [Limit, Rid, length(RCnt)]), - NewRCnt = lists:delete(Rid, RCnt), - S#state{rcnt = NewRCnt}. - - -maybe_handle_recv(#state{usock = Sock, filter = FilterMod} = S, - Ip, Port, Packet) -> - case (catch FilterMod:accept_recv(Ip, Port)) of + "passed below limit: activate", []), + active_once(Socket) + end, + update_transport_req_refs(S, Transport, NewReqRefs). + +update_transport_req_refs( + #state{transports = Transports} = S, + #transport{socket = Socket} = T, + ReqRefs) -> + S#state{ + transports = + lists:keyreplace( + Socket, #transport.socket, Transports, + T#transport{req_refs = ReqRefs})}. + + +maybe_handle_recv( + #state{filter = FilterMod} = S, + #transport{socket = Socket} = Transport, + From, Packet) -> + {From_1, From_2} = From, + case + try FilterMod:accept_recv(From_1, From_2) + catch + Class:Exception -> + error_msg( + "FilterMod:accept_recv(~p, ~p) crashed: ~w:~w~n ~p", + [From_1,From_2,Class,Exception,erlang:get_stacktrace()]), + true + end + of false -> %% Drop the received packet inc(netIfMsgInDrops), - active_once(Sock), + active_once(Socket), S; - _ -> - handle_recv(S, Ip, Port, Packet) + Other -> + case Other of + true -> + ok; + _ -> + error_msg( + "FilterMod:accept_recv(~p, ~p) returned: ~p", + [From_1,From_2,Other]) + end, + handle_recv(S, Transport, From, Packet) end. -handle_discovery_response(_Ip, _Port, #pdu{request_id = ReqId} = Pdu, - ManagerEngineId, - #state{usock = Sock, reqs = Reqs} = S) -> - case lists:keysearch(ReqId, 1, S#state.reqs) of - {value, {_, Pid}} -> - active_once(Sock), - Pid ! {snmp_discovery_response_received, Pdu, ManagerEngineId}, - NReqs = lists:keydelete(ReqId, 1, Reqs), - S#state{reqs = NReqs}; - _ -> - %% Ouch, timeout? resend? - S - end. - -handle_recv(#state{usock = Sock, - mpd_state = MpdState, - note_store = NS, - log = Log} = S, Ip, Port, Packet) -> +handle_recv( + #state{mpd_state = MpdState, note_store = NS, log = Log} = S, + #transport{socket = Socket} = Transport, + From, Packet) -> put(n1, erlang:now()), - LogF = fun(Type, Data) -> - log(Log, Type, Data, Ip, Port) - end, - Domain = snmp_conf:which_domain(Ip), % What the ****... - case (catch snmpa_mpd:process_packet(Packet, - Domain, {Ip, Port}, - MpdState, NS, LogF)) of + LogF = + fun(Type, Data) -> + log(Log, Type, Data, From) + end, + case (catch snmpa_mpd:process_packet( + Packet, From, MpdState, NS, LogF)) of {ok, _Vsn, Pdu, _PduMS, {discovery, ManagerEngineId}} -> - handle_discovery_response(Ip, Port, Pdu, ManagerEngineId, S); + handle_discovery_response( + S, Transport, From, Pdu, ManagerEngineId); {ok, _Vsn, Pdu, _PduMS, discovery} -> - handle_discovery_response(Ip, Port, Pdu, undefined, S); + handle_discovery_response( + S, Transport, From, Pdu, undefined); {ok, Vsn, Pdu, PduMS, ACMData} -> ?vlog("got pdu ~s", [?vapply(snmp_misc, format, [256, "~w", [Pdu]])]), - %% handle_recv_pdu(Ip, Port, Vsn, Pdu, PduMS, ACMData, S); - maybe_handle_recv_pdu(Ip, Port, Vsn, Pdu, PduMS, ACMData, S); + %% handle_recv_pdu(S, Transport, From, Vsn, Pdu, PduMS, ACMData); + maybe_handle_recv_pdu( + S, Transport, From, Vsn, Pdu, PduMS, ACMData); {discarded, Reason} -> ?vlog("packet discarded for reason: ~s", [?vapply(snmp_misc, format, [256, "~w", [Reason]])]), - active_once(Sock), + active_once(Socket), S; {discarded, Reason, ReportPacket} -> ?vlog("sending report for reason: " "~n ~s", [?vapply(snmp_misc, format, [256, "~w", [Reason]])]), - (catch udp_send(S#state.usock, Ip, Port, ReportPacket)), - active_once(Sock), + (catch udp_send(Socket, From, ReportPacket)), + active_once(Socket), S; {discovery, ReportPacket} -> ?vlog("sending discovery report", []), - (catch udp_send(S#state.usock, Ip, Port, ReportPacket)), - active_once(Sock), + (catch udp_send(Socket, From, ReportPacket)), + active_once(Socket), S; Error -> error_msg("processing of received message failed: " "~n ~p", [Error]), - active_once(Sock), + active_once(Socket), S end. - -maybe_handle_recv_pdu(Ip, Port, Vsn, #pdu{type = Type} = Pdu, PduMS, ACMData, - #state{usock = Sock, filter = FilterMod} = S) -> - case (catch FilterMod:accept_recv_pdu(Ip, Port, Type)) of + +handle_discovery_response( + #state{reqs = Reqs} = S, + #transport{socket = Socket}, + _From, + #pdu{request_id = ReqId} = Pdu, + ManagerEngineId) -> + case lists:keyfind(ReqId, 1, S#state.reqs) of + {ReqId, Pid} -> + active_once(Socket), + Pid ! {snmp_discovery_response_received, Pdu, ManagerEngineId}, + %% XXX Strange... Reqs from this Pid should be reaped + %% at process exit by clear_reqs/2 so the following + %% should be redundant. + NReqs = lists:keydelete(ReqId, 1, Reqs), + S#state{reqs = NReqs}; + false -> + %% Ouch, timeout? resend? + S + end. + +maybe_handle_recv_pdu( + #state{filter = FilterMod} = S, + #transport{socket = Socket} = Transport, + From, Vsn, + #pdu{type = Type} = Pdu, PduMS, ACMData) -> + {From_1, From_2} = From, + case + try FilterMod:accept_recv_pdu(From_1, From_2, Type) + catch + Class:Exception -> + error_msg( + "FilterMod:accept_recv_pdu(~p, ~p, ~p) crashed: ~w:~w~n" + " ~p", + [From_1,From_2,Type,Class,Exception, + erlang:get_stacktrace()]), + true + end + of false -> inc(netIfPduInDrops), - active_once(Sock), - ok; - _ -> - handle_recv_pdu(Ip, Port, Vsn, Pdu, PduMS, ACMData, S) - end; -maybe_handle_recv_pdu(Ip, Port, Vsn, Pdu, PduMS, ACMData, S) -> - handle_recv_pdu(Ip, Port, Vsn, Pdu, PduMS, ACMData, S). + active_once(Socket), + S; + Other -> + case Other of + true -> + ok; + _ -> + error_msg( + "FilterMod:accept_recv_pdu(~p, ~p, ~p) returned: ~p", + [From_1,From_2,Type,Other]) + end, + handle_recv_pdu(S, Transport, From, Vsn, Pdu, PduMS, ACMData) + end. -handle_recv_pdu(Ip, Port, Vsn, #pdu{type = 'get-response'} = Pdu, - _PduMS, _ACMData, #state{usock = Sock} = S) -> - active_once(Sock), - handle_response(Vsn, Pdu, {Ip, Port}, S), +handle_recv_pdu( + #state{reqs = Reqs} = S, + #transport{socket = Socket}, + From, Vsn, + #pdu{type = 'get-response', request_id = ReqId} = Pdu, + _PduMS, _ACMData) -> + active_once(Socket), + case lists:keyfind(ReqId, 1, Reqs) of + {ReqId, Pid} -> + ?vdebug("handle_recv_pdu -> " + "~n send response to receiver ~p", [Pid]), + Pid ! {snmp_response_received, Vsn, Pdu, From}; + false -> + ?vdebug("handle_recv_pdu -> " + "~n No receiver available for response pdu", []) + end, S; -handle_recv_pdu(Ip, Port, Vsn, #pdu{request_id = Rid, type = Type} = Pdu, - PduMS, ACMData, #state{master_agent = Pid} = S) - when ((Type =:= 'get-request') orelse - (Type =:= 'get-next-request') orelse - (Type =:= 'get-bulk-request')) -> - ?vtrace("handle_recv_pdu -> received get (~w)", [Type]), - Pid ! {snmp_pdu, Vsn, Pdu, PduMS, ACMData, {Ip, Port}, []}, - update_req_counter_incomming(S, Rid); -handle_recv_pdu(Ip, Port, Vsn, Pdu, PduMS, ACMData, - #state{usock = Sock, master_agent = Pid} = S) -> +handle_recv_pdu( + #state{master_agent = Pid} = S, + #transport{} = Transport, + From, Vsn, + #pdu{type = Type} = Pdu, + PduMS, ACMData) + when Type =:= 'set-request'; + Type =:= 'get-request'; + Type =:= 'get-next-request'; + Type =:= 'get-bulk-request' -> + ?vtrace("handle_recv_pdu -> received request (~w)", [Type]), + ReqRef = make_ref(), + Extra = [{request_ref, ReqRef}], + Pid ! {snmp_pdu, Vsn, Pdu, PduMS, ACMData, From, Extra}, + NewS = update_req_counter_incoming(S, Transport, ReqRef), + ?vdebug("handle_recv_pdu -> ~p", [NewS]), + NewS; +handle_recv_pdu( + #state{master_agent = Pid} = S, + #transport{socket = Socket}, + From, Vsn, Pdu, PduMS, ACMData) -> ?vtrace("handle_recv_pdu -> received other request", []), - active_once(Sock), - Pid ! {snmp_pdu, Vsn, Pdu, PduMS, ACMData, {Ip, Port}, []}, + active_once(Socket), + Pid ! {snmp_pdu, Vsn, Pdu, PduMS, ACMData, From, []}, S. -maybe_handle_reply_pdu(#state{filter = FilterMod} = S, Vsn, - #pdu{request_id = Rid} = Pdu, - Type, ACMData, {Ip, Port} = Dest) -> - S1 = update_req_counter_outgoing(S, Rid), - case (catch FilterMod:accept_send_pdu([{Ip, Port}], Type)) of +maybe_handle_reply_pdu( + #state{filter = FilterMod, transports = Transports} = S, + #transport{} = Transport, + Vsn, + #pdu{} = Pdu, + Type, ACMData, To) -> + %% + Addresses = [fix_filter_address(Transports, To)], + case + try + FilterMod:accept_send_pdu(Addresses, Type) + catch + Class:Exception -> + error_msg( + "FilterMod:accept_send_pdu(~p, ~p) crashed: ~w:~w~n ~p", + [Addresses, Type, Class, Exception, + erlang:get_stacktrace()]), + true + end + of false -> inc(netIfPduOutDrops), ok; - _ -> - handle_reply_pdu(S1, Vsn, Pdu, Type, ACMData, Dest) - end, - S1. - -handle_reply_pdu(#state{log = Log, - usock = Sock, - filter = FilterMod}, - Vsn, Pdu, Type, ACMData, {Ip, Port}) -> - LogF = fun(Type2, Data) -> - log(Log, Type2, Data, Ip, Port) - end, + Other -> + case Other of + true -> + ok; + _ -> + error_msg( + "FilterMod:accept_send_pdu(~p, ~p) returned: ~p", + [Addresses,Type,Other]) + end, + handle_reply_pdu(S, Transport, Vsn, Pdu, Type, ACMData, To) + end. + +handle_reply_pdu( + #state{log = Log} = S, + #transport{} = Transport, + Vsn, + #pdu{} = Pdu, + Type, ACMData, To) -> + %% + LogF = + fun(Type2, Data) -> + log(Log, Type2, Data, To) + end, case (catch snmpa_mpd:generate_response_msg(Vsn, Pdu, Type, ACMData, LogF)) of {ok, Packet} -> ?vinfo("time in agent: ~w mysec", [time_in_agent()]), - maybe_udp_send(FilterMod, Sock, Ip, Port, Packet); + try maybe_udp_send(S, Transport, To, Packet) + catch + {Reason, Sz} -> + error_msg("Cannot send message " + "~n size: ~p" + "~n reason: ~p" + "~n pdu: ~p", + [Sz, Reason, Pdu]) + end; {discarded, Reason} -> ?vlog("handle_reply_pdu -> " "~n reply discarded for reason: ~s", @@ -652,105 +887,116 @@ handle_reply_pdu(#state{log = Log, ok; {'EXIT', Reason} -> user_err("failed generating response message: " - "~nPDU: ~w~n~w", [Pdu, Reason]) + "~nPDU: ~p~n~p", [Pdu, Reason]) end. - -process_taddrs(To) -> - process_taddrs(To, []). -process_taddrs([], Acc) -> - lists:reverse(Acc); -%% v3 -process_taddrs([{{_Domain, AddrAndPort}, _SecData}|T], Acc) -> - process_taddrs(T, [AddrAndPort|Acc]); -%% v1 & v2 -process_taddrs([{_Domain, AddrAndPort}|T], Acc) -> - process_taddrs(T, [AddrAndPort|Acc]). -merge_taddrs(To1, To2) -> - merge_taddrs(To1, To2, []). +maybe_handle_send_pdu( + #state{filter = FilterMod, transports = Transports} = S, + Vsn, Pdu, MsgData, TDomAddrSecs, From) -> -merge_taddrs([], _To2, Acc) -> - lists:reverse(Acc); -%% v3 -merge_taddrs([{{_, AddrAndPort}, _} = H|To1], To2, Acc) -> - case lists:member(AddrAndPort, To2) of - true -> - merge_taddrs(To1, To2, [H|Acc]); - false -> - merge_taddrs(To1, To2, Acc) - end; -%% v1 & v2 -merge_taddrs([{_, AddrAndPort} = H|To1], To2, Acc) -> - case lists:member(AddrAndPort, To2) of - true -> - merge_taddrs(To1, To2, [H|Acc]); - false -> - merge_taddrs(To1, To2, Acc) - end; -merge_taddrs([_Crap|To1], To2, Acc) -> - merge_taddrs(To1, To2, Acc). - - -maybe_handle_send_pdu(#state{filter = FilterMod} = S, - Vsn, Pdu, MsgData, To0, From) -> + ?vtrace("maybe_handle_send_pdu -> entry with~n" + " FilterMod: ~p~n" + " TDomAddrSecs: ~p", [FilterMod, TDomAddrSecs]), - ?vtrace("maybe_handle_send_pdu -> entry with" - "~n FilterMod: ~p" - "~n To0: ~p", [FilterMod, To0]), - - To1 = snmpa_mpd:process_taddrs(To0), - To2 = process_taddrs(To1), + DomAddrSecs = snmpa_mpd:process_taddrs(TDomAddrSecs), + AddressesToFilter = + case is_legacy_transports(Transports) of + true -> + [fix_filter_legacy_mpd_address(DAS) + || DAS <- DomAddrSecs]; + false -> + [fix_filter_mpd_address(DAS) + || DAS <- DomAddrSecs] + end, - case (catch FilterMod:accept_send_pdu(To2, pdu_type_of(Pdu))) of + Type = pdu_type_of(Pdu), + + case + try FilterMod:accept_send_pdu(AddressesToFilter, Type) + catch + Class:Exception -> + error_msg( + "FilterMod:accept_send_pdu(~p, ~p) crashed: ~w:~w~n ~p", + [AddressesToFilter,Type, + Class,Exception,erlang:get_stacktrace()]), + true + end + of false -> inc(netIfPduOutDrops), ok; true -> - handle_send_pdu(S, Vsn, Pdu, MsgData, To1, From); - To3 when is_list(To3) -> - To4 = merge_taddrs(To1, To3), - ?vtrace("maybe_handle_send_pdu -> To4: " - "~n ~p", [To4]), - handle_send_pdu(S, Vsn, Pdu, MsgData, To4, From); - _ -> - handle_send_pdu(S, Vsn, Pdu, MsgData, To1, From) + handle_send_pdu(S, Vsn, Pdu, MsgData, DomAddrSecs, From); + FilteredAddresses when is_list(FilteredAddresses) -> + FilteredDomAddrSecs = + case is_legacy_transports(Transports) of + true -> + [DAS || + DAS <- DomAddrSecs, + lists:member( + fix_filter_legacy_mpd_address(DAS), + FilteredAddresses)]; + false -> + [DAS || + DAS <- DomAddrSecs, + lists:member( + fix_filter_mpd_address(DAS), + FilteredAddresses)] + end, + ?vtrace("maybe_handle_send_pdu -> FilteredDomAddrSecs:~n" + " ~p", [FilteredDomAddrSecs]), + handle_send_pdu(S, Vsn, Pdu, MsgData, FilteredDomAddrSecs, From); + Other -> + error_msg( + "FilterMod:accept_send_pdu(~p, ~p) returned: ~p", + [AddressesToFilter,Type,Other]), + handle_send_pdu(S, Vsn, Pdu, MsgData, DomAddrSecs, From) end. -handle_send_pdu(#state{note_store = NS} = S, Vsn, Pdu, MsgData, To, From) -> - - ?vtrace("handle_send_pdu -> entry with" - "~n Pdu: ~p" - "~n To: ~p", [Pdu, To]), +handle_send_pdu( + #state{note_store = NS} = S, + Vsn, Pdu, MsgData, DomAddrSecs, From) -> + %% + ?vtrace("handle_send_pdu -> entry with~n" + " Pdu: ~p~n" + " DomAddrSecs: ~p", [Pdu, DomAddrSecs]), - case (catch snmpa_mpd:generate_msg(Vsn, NS, Pdu, MsgData, To)) of + case (catch snmpa_mpd:generate_msg( + Vsn, NS, Pdu, MsgData, DomAddrSecs)) of {ok, Addresses} -> - handle_send_pdu(S, Pdu, Addresses); + do_handle_send_pdu(S, Pdu, Addresses); {discarded, Reason} -> ?vlog("handle_send_pdu -> " "~n PDU ~p not sent due to ~p", [Pdu, Reason]), ok; {'EXIT', Reason} -> user_err("failed generating message: " - "~nPDU: ~w~n~w", [Pdu, Reason]), + "~nPDU: ~p~n~p", [Pdu, Reason]), ok end, case From of undefined -> S; - Pid -> + Pid when is_pid(Pid) -> ?vtrace("link to ~p and add to request list", [Pid]), link(Pid), - NReqs = snmp_misc:keyreplaceadd(Pid, 2, S#state.reqs, - {Pdu#pdu.request_id, From}), + NReqs = + snmp_misc:keyreplaceadd( + Pid, 2, S#state.reqs, {Pdu#pdu.request_id, From}), S#state{reqs = NReqs} end. -handle_send_discovery(#state{note_store = NS} = S, - Pdu, MsgData, - To, From) -> +handle_send_discovery( + #state{ + note_store = NS, + log = Log, + reqs = Reqs, + transports = Transports} = S, + #pdu{type = Type, request_id = ReqId} = Pdu, + MsgData, To, From) when is_pid(From) -> ?vtrace("handle_send_discovery -> entry with" "~n Pdu: ~p" @@ -759,155 +1005,173 @@ handle_send_discovery(#state{note_store = NS} = S, "~n From: ~p", [Pdu, MsgData, To, From]), case (catch snmpa_mpd:generate_discovery_msg(NS, Pdu, MsgData, To)) of - {ok, {Packet, {Ip, Port}}} -> - handle_send_discovery(S, Pdu, Packet, Ip, Port, From); + {ok, {Domain, Address, Packet}} -> + case select_transport_from_domain(Domain, Transports) of + false -> + error_msg( + "Can not find transport to: ~s", + [format_address(To)]), + S; + #transport{socket = Socket} -> + log(Log, Type, Packet, {Domain, Address}), + udp_send(Socket, {Domain, Address}, Packet), + ?vtrace("handle_send_discovery -> sent (~w)", [ReqId]), + NReqs = snmp_misc:keyreplaceadd(From, 2, Reqs, {ReqId, From}), + S#state{reqs = NReqs} + end; {discarded, Reason} -> ?vlog("handle_send_discovery -> " "~n Discovery PDU ~p not sent due to ~p", [Pdu, Reason]), - ok; + S; {'EXIT', Reason} -> user_err("failed generating discovery message: " - "~n PDU: ~w" - "~n Reason: ~w", [Pdu, Reason]), - ok + "~n PDU: ~p" + "~n Reason: ~p", [Pdu, Reason]), + S end. -handle_send_discovery(#state{log = Log, - usock = Sock, - reqs = Reqs} = S, - #pdu{type = Type, - request_id = ReqId}, - Packet, Ip, Port, From) - when is_binary(Packet) -> - log(Log, Type, Packet, Ip, Port), - udp_send(Sock, Ip, Port, Packet), - ?vtrace("handle_send_discovery -> sent (~w)", [ReqId]), - NReqs = snmp_misc:keyreplaceadd(From, 2, Reqs, {ReqId, From}), - S#state{reqs = NReqs}. +do_handle_send_pdu(S, #pdu{type = Type} = Pdu, Addresses) -> + do_handle_send_pdu(S, Type, Pdu, Addresses); +do_handle_send_pdu(S, Trap, Addresses) -> + do_handle_send_pdu(S, trappdu, Trap, Addresses). -handle_send_pdu(S, #pdu{type = Type} = Pdu, Addresses) -> - handle_send_pdu(S, Type, Pdu, Addresses); -handle_send_pdu(S, Trap, Addresses) -> - handle_send_pdu(S, trappdu, Trap, Addresses). - -handle_send_pdu(S, Type, Pdu, Addresses) -> - case (catch handle_send_pdu1(S, Type, Addresses)) of +do_handle_send_pdu(S, Type, Pdu, Addresses) -> + try do_handle_send_pdu1(S, Type, Addresses) + catch {Reason, Sz} -> - error_msg("Cannot send message " - "~n size: ~p" - "~n reason: ~p" - "~n pdu: ~p", - [Sz, Reason, Pdu]); - _ -> - ok - end. - -handle_send_pdu1(#state{log = Log, - usock = Sock, - filter = FilterMod}, Type, Addresses) -> - SendFun = - fun({snmpUDPDomain, {Ip, Port}, Packet}) - when is_binary(Packet) -> - ?vdebug("[snmpUDPDomain] sending packet:" - "~n size: ~p" - "~n to: ~p:~p", - [sz(Packet), Ip, Port]), - maybe_udp_send(FilterMod, Log, Type, Sock, Ip, Port, Packet); - - ({snmpUDPDomain, {Ip, Port}, {Packet, _LogData}}) - when is_binary(Packet) -> - ?vdebug("[snmpUDPDomain] sending encrypted packet:" - "~n size: ~p" - "~n to: ~p:~p", - [sz(Packet), Ip, Port]), - maybe_udp_send(FilterMod, Log, Type, Sock, Ip, Port, Packet); - - ({transportDomainUdpIpv4, {Ip, Port}, Packet}) - when is_binary(Packet) -> - ?vdebug("[transportDomainUdpIpv4] sending packet:" - "~n size: ~p" - "~n to: ~p:~p", - [sz(Packet), Ip, Port]), - maybe_udp_send(FilterMod, Log, Type, Sock, Ip, Port, Packet); - - ({transportDomainUdpIpv4, {Ip, Port}, {Packet, _LogData}}) - when is_binary(Packet) -> - ?vdebug("[transportDomainUdpIpv4] sending encrypted packet:" - "~n size: ~p" - "~n to: ~p:~p", - [sz(Packet), Ip, Port]), - maybe_udp_send(FilterMod, Log, Type, Sock, Ip, Port, Packet); - - ({transportDomainUdpIpv6, {Ip, Port}, Packet}) - when is_binary(Packet) -> - ?vdebug("[transportDomainUdpIpv6] sending packet:" - "~n size: ~p" - "~n to: ~p:~p", - [sz(Packet), Ip, Port]), - maybe_udp_send(FilterMod, Log, Type, Sock, Ip, Port, Packet); - - ({transportDomainUdpIpv6, {Ip, Port}, {Packet, _LogData}}) - when is_binary(Packet) -> - ?vdebug("[transportDomainUdpIpv6] sending encrypted packet:" - "~n size: ~p" - "~n to: ~p:~p", - [sz(Packet), Ip, Port]), - maybe_udp_send(FilterMod, Log, Type, Sock, Ip, Port, Packet); - - (_X) -> - ?vlog("** bad res: ~p", [_X]), - ok - end, - lists:foreach(SendFun, Addresses). - - -handle_response(Vsn, Pdu, From, S) -> - case lists:keysearch(Pdu#pdu.request_id, 1, S#state.reqs) of - {value, {_, Pid}} -> - ?vdebug("handle_response -> " - "~n send response to receiver ~p", [Pid]), - Pid ! {snmp_response_received, Vsn, Pdu, From}; - _ -> - ?vdebug("handle_response -> " - "~n No receiver available for response pdu", []) + error_msg( + "Can not send message~n" + " size: ~p~n" + " reason: ~p~n" + " pdu: ~p", + [Sz, Reason, Pdu]) end. -maybe_udp_send(FilterMod, Sock, Ip, Port, Packet) -> - case (catch FilterMod:accept_send(Ip, Port)) of +do_handle_send_pdu1( + #state{transports = Transports} = S, + Type, Addresses) -> + lists:foreach( + fun ({Domain, Address, Packet}) when is_binary(Packet) -> + ?vdebug( + "[~w] sending packet:~n" + " size: ~p~n" + " to: ~p", [Domain, sz(Packet), Address]), + To = {Domain, Address}, + case select_transport_from_domain(Domain, Transports) of + false -> + error_msg( + "Can not find transport~n" + " size: ~p~n" + " to: ~s", + [sz(Packet), format_address(To)]); + Transport -> + maybe_udp_send(S, Transport, To, Packet) + end; + ({Domain, Address, {Packet, LogData}}) when is_binary(Packet) -> + ?vdebug( + "[~w] sending encrypted packet:~n" + " size: ~p~n" + " to: ~p", [Domain, sz(Packet), Address]), + To = {Domain, Address}, + case select_transport_from_domain(Domain, Transports) of + false -> + error_msg( + "Can not find transport~n" + " size: ~p~n" + " to: ~s", + [sz(Packet), format_address(To)]); + Transport -> + maybe_udp_send(S, Transport, To, Packet, Type, LogData) + end + end, + Addresses). + +maybe_udp_send( + #state{filter = FilterMod, transports = Transports}, + #transport{socket = Socket}, + To, Packet) -> + {To_1, To_2} = fix_filter_address(Transports, To), + case + try FilterMod:accept_send(To_1, To_2) + catch + Class:Exception -> + error_msg( + "FilterMod:accept_send(~p, ~p) crashed: ~w:~w~n ~p", + [To_1,To_2,Class,Exception,erlang:get_stacktrace()]), + true + end + of false -> inc(netIfMsgOutDrops), ok; - _ -> - (catch udp_send(Sock, Ip, Port, Packet)) + Other -> + case Other of + true -> + ok; + _ -> + error_msg( + "FilterMod:accept_send(~p, ~p) returned: ~p", + [To_1,To_2,Other]) + end, + udp_send(Socket, To, Packet) end. -maybe_udp_send(FilterMod, AtLog, Type, Sock, Ip, Port, Packet) -> - case (catch FilterMod:accept_send(Ip, Port)) of +maybe_udp_send( + #state{log = Log, filter = FilterMod, transports = Transports}, + #transport{socket = Socket}, + To, Packet, Type, _LogData) -> + {To_1, To_2} = fix_filter_address(Transports, To), + case + try FilterMod:accept_send(To_1, To_2) + catch + Class:Exception -> + error_msg( + "FilterMod:accept_send(~p, ~p) crashed for: ~w:~w~n ~p", + [To_1, To_2, Class, Exception, erlang:get_stacktrace()]), + true + end + of false -> inc(netIfMsgOutDrops), ok; - _ -> - log(AtLog, Type, Packet, Ip, Port), - (catch udp_send(Sock, Ip, Port, Packet)) + Other -> + case Other of + true -> + ok; + _ -> + error_msg( + "FilterMod:accept_send(~p, ~p) returned: ~p", + [To_1,To_2,Other]) + end, + log(Log, Type, Packet, To), + udp_send(Socket, To, Packet) end. - -udp_send(UdpId, AgentIp, UdpPort, B) -> - case (catch gen_udp:send(UdpId, AgentIp, UdpPort, B)) of +udp_send(Socket, To, B) -> + {IpAddr, IpPort} = + case To of + {Domain, Addr} when is_atom(Domain) -> + Addr; + {_, P} = Addr when is_integer(P) -> + Addr + end, + try gen_udp:send(Socket, IpAddr, IpPort, B) of {error, emsgsize} -> %% From this message we cannot recover, so exit sending loop throw({emsgsize, sz(B)}); {error, ErrorReason} -> error_msg("[error] cannot send message " "(destination: ~p:~p, size: ~p, reason: ~p)", - [AgentIp, UdpPort, sz(B), ErrorReason]); - {'EXIT', ExitReason} -> - error_msg("[exit] cannot send message " - "(destination: ~p:~p, size: ~p, reason: ~p)", - [AgentIp, UdpPort, sz(B), ExitReason]); - _ -> + [IpAddr, IpPort, sz(B), ErrorReason]); + ok -> ok + catch + error:ExitReason -> + error_msg("[exit] cannot send message " + "(destination: ~p:~p, size: ~p, reason: ~p, at: ~p)", + [IpAddr, IpPort, sz(B), ExitReason, + erlang:get_stacktrace()]) end. sz(L) when is_list(L) -> length(L); @@ -957,6 +1221,75 @@ active_once(Sock) -> inet:setopts(Sock, [{active, once}]). +select_transport_from_req_ref(_, []) -> + false; +select_transport_from_req_ref( + ReqRef, + [#transport{req_refs = ReqRefs} = Transport | Transports]) -> + case lists:member(ReqRef, ReqRefs) of + true -> + Transport; + false -> + select_transport_from_req_ref(ReqRef, Transports) + end. + +select_transport_from_domain(Domain, Transports) when is_atom(Domain) -> + Pos = #transport.domain, + case lists:keyfind(Domain, Pos, Transports) of + #transport{domain = Domain} = Transport -> + Transport; + false when Domain == snmpUDPDomain -> + lists:keyfind(transportDomainUdpIpv4, Pos, Transports); + false when Domain == transportDomainUdpIpv4 -> + lists:keyfind(snmpUDPDomain, Pos, Transports); + false -> + false + end. + +address_to_domain({Domain, _Addr}) when is_atom(Domain) -> + Domain; +address_to_domain({_Ip, Port}) when is_integer(Port) -> + snmpUDPDomain. + +%% If the agent uses legacy snmpUDPDomain e.g has not set +%% intAgentTransportDomain, then make sure +%% snmpa_network_interface_filter gets legacy arguments +%% to not break backwards compatibility. +%% +fix_filter_address(Transports, Address) -> + case is_legacy_transports(Transports) of + true -> + case Address of + {Domain, Addr} when is_atom(Domain) -> + Addr; + {_, IpPort} = Addr when is_integer(IpPort) -> + Addr + end; + false -> + Address + end. + +is_legacy_transports([#transport{domain = snmpUDPDomain}]) -> + true; +is_legacy_transports([#transport{} | _]) -> + false. + +fix_filter_legacy_mpd_address(Domain_Address_SecData) -> + case Domain_Address_SecData of + {{Domain, Addr}, _SecData} when is_atom(Domain) -> % v3 + Addr; + {Domain, Addr} when is_atom(Domain) -> % v1 & v2 + Addr + end. + +fix_filter_mpd_address(Domain_Address_SecData) -> + case Domain_Address_SecData of + {{Domain, _Addr} = Address, _SecData} when is_atom(Domain) -> % v3 + Address; + {Domain, _Addr} = Address when is_atom(Domain) -> % v1 & v2 + Address + end. + %%%----------------------------------------------------------------- handle_set_log_type(#state{log = {Log, OldValue}} = State, NewType) @@ -1113,37 +1446,39 @@ get_counters([Counter|Counters], Acc) -> %% ---------------------------------------------------------------- -ip_opt_bind_to_ip_address(Opts, Ip) -> - case get_bind_to_ip_address(Opts) of - true -> - [{ip, list_to_tuple(Ip)}]; - _ -> - [] - end. - -ip_opt_no_reuse_address(Opts) -> - case get_no_reuse_address(Opts) of - false -> - [{reuseaddr, true}]; - _ -> - [] - end. - -ip_opt_recbuf(Opts) -> - case get_recbuf(Opts) of - use_default -> - []; - Sz -> - [{recbuf, Sz}] - end. - -ip_opt_sndbuf(Opts) -> - case get_sndbuf(Opts) of - use_default -> - []; - Sz -> - [{sndbuf, Sz}] - end. +socket_opts(Domain, {IpAddr, IpPort}, Opts) -> + [IpPort, % Picked off at socket open, separate argument + binary + | case snmp_conf:tdomain_to_family(Domain) of + inet6 = Family -> + [Family, {ipv6_v6only, true}]; + Family -> + [Family] + end ++ + case get_bind_to_ip_address(Opts) of + true -> + [{ip, IpAddr}]; + _ -> + [] + end ++ + case get_no_reuse_address(Opts) of + false -> + [{reuseaddr, true}]; + _ -> + [] + end ++ + case get_recbuf(Opts) of + use_default -> + []; + Sz -> + [{recbuf, Sz}] + end ++ + case get_sndbuf(Opts) of + use_default -> + []; + Sz -> + [{sndbuf, Sz}] + end]. %% ---------------------------------------------------------------- @@ -1203,6 +1538,9 @@ get_bind_to_ip_address(Opts) -> error_msg(F,A) -> ?snmpa_error("NET-IF server: " ++ F, A). +info_msg(F,A) -> + ?snmpa_info("NET-IF server: " ++ F, A). + %% --- user_err(F, A) -> @@ -1227,14 +1565,14 @@ call(Pid, Req) -> %% ---------------------------------------------------------------- -get_info(#state{usock = Id, reqs = Reqs}) -> +get_info(#state{transports = Transports, reqs = Reqs}) -> ProcSize = proc_mem(self()), - PortInfo = get_port_info(Id), Counters = get_counters(), - [{reqs, Reqs}, - {counters, Counters}, - {process_memory, ProcSize}, - {port_info, PortInfo}]. + [{reqs, Reqs}, + {counters, Counters}, + {process_memory, ProcSize} + | [{port_info, get_port_info(Socket)} + || #transport{socket = Socket} <- Transports]]. proc_mem(P) when is_pid(P) -> case (catch erlang:process_info(P, memory)) of diff --git a/lib/snmp/src/agent/snmpa_net_if_filter.erl b/lib/snmp/src/agent/snmpa_net_if_filter.erl index 989f7c95b3..dd77b143d0 100644 --- a/lib/snmp/src/agent/snmpa_net_if_filter.erl +++ b/lib/snmp/src/agent/snmpa_net_if_filter.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2009. All Rights Reserved. +%% Copyright Ericsson AB 2007-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -18,35 +18,49 @@ %% -module(snmpa_net_if_filter). --export([accept_recv/2, - accept_send/2, - accept_recv_pdu/3, - accept_send_pdu/2]). +%% Behaviour +-export([accept_recv/2, accept_send/2, accept_recv_pdu/3, accept_send_pdu/2]). -include("snmp_debug.hrl"). -accept_recv(_Addr, _Port) -> - ?d("accept_recv -> entry with" - "~n Addr: ~p" - "~n Port: ~p", [_Addr, _Port]), +accept_recv(Domain, _Address) when is_atom(Domain) -> + ?d("accept_recv -> entry with~n" + " Domain: ~p~n" + " Address: ~p", [Domain, _Address]), + true; +accept_recv(_Addr, Port) when is_integer(Port) -> + ?d("accept_recv -> entry with~n" + " Addr: ~p~n" + " Port: ~p", [_Addr, Port]), true. -accept_send(_Addr, _Port) -> - ?d("accept_send -> entry with" - "~n Addr: ~p" - "~n Port: ~p", [_Addr, _Port]), +accept_send(Domain, _Address) when is_atom(Domain) -> + ?d("accept_send -> entry with~n" + " Domain: ~p~n" + " Address: ~p", [Domain, _Address]), + true; +accept_send(_Addr, Port) when is_integer(Port) -> + ?d("accept_send -> entry with~n" + " Addr: ~p~n" + " Port: ~p", [_Addr, Port]), true. -accept_recv_pdu(_Addr, _Port, _PduType) -> - ?d("accept_recv_pdu -> entry with" - "~n Addr: ~p" - "~n Port: ~p" - "~n PduType: ~p", [_Addr, _Port, _PduType]), +accept_recv_pdu(Domain, _Address, _PduType) when is_atom(Domain) -> + ?d("accept_recv -> entry with~n" + " Domain: ~p~n" + " Address: ~p~n" + " PduType: ~p", [Domain, _Address, _PduType]), + true; +accept_recv_pdu(_Addr, Port, _PduType) when is_integer(Port) -> + ?d("accept_recv_pdu -> entry with~n" + " Addr: ~p~n" + " Port: ~p~n" + " PduType: ~p", [_Addr, Port, _PduType]), true. accept_send_pdu(_Targets, _PduType) -> - ?d("accept_send_pdu -> entry with" - "~n Targets: ~p" - "~n PduType: ~p", [_Targets, _PduType]), + ?d("accept_send_pdu -> entry with~n" + " Targets: ~p~n" + " PduType: ~p", [_Targets, _PduType]), true. diff --git a/lib/snmp/src/agent/snmpa_network_interface_filter.erl b/lib/snmp/src/agent/snmpa_network_interface_filter.erl index 6fa131beee..90aa54a271 100644 --- a/lib/snmp/src/agent/snmpa_network_interface_filter.erl +++ b/lib/snmp/src/agent/snmpa_network_interface_filter.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2009. All Rights Reserved. +%% Copyright Ericsson AB 2007-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -23,7 +23,7 @@ behaviour_info(callbacks) -> - [{accept_recv, 2}, + [{accept_recv, 2}, {accept_send, 2}, {accept_recv_pdu, 3}, {accept_send_pdu, 2}]; @@ -31,20 +31,21 @@ behaviour_info(_) -> undefined. -%% accept_recv(address(), port()) -> boolean() +%% accept_recv({domain(), address()}) -> boolean() %% Called at the receiption of a message %% (before *any* processing has been done). %% -%% accept_send(address(), port()) -> boolean() +%% accept_send({domain(), address()}) -> boolean() %% Called before the sending of a message %% (after *all* processing has been done). %% -%% accept_recv_pdu(Addr, Port, pdu_type()) -> boolean() +%% accept_recv_pdu({domain(), address()}, pdu_type()) -> boolean() %% Called after the basic message processing (MPD) has been done, %% but before the pdu is handed over to the master-agent for %% primary processing. %% -%% accept_send_pdu(Targets, pdu_type()) -> boolean() | NewTargets +%% accept_send_pdu([{domain(), address()}, ...] = Targets, pdu_type()) -> +%% boolean() | NewTargets %% Called before the basic message processing (MPD) is done, %% when a pdu has been received from the master-agent. %% diff --git a/lib/snmp/src/agent/snmpa_trap.erl b/lib/snmp/src/agent/snmpa_trap.erl index b9a2496341..a79b150f57 100644 --- a/lib/snmp/src/agent/snmpa_trap.erl +++ b/lib/snmp/src/agent/snmpa_trap.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2012. All Rights Reserved. +%% Copyright Ericsson AB 1996-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -270,12 +270,13 @@ localise_type({VariableOid, Value}, Mib) when is_list(VariableOid) -> localise_type(X, _) -> X. %%----------------------------------------------------------------- -%% Func: make_v1_trap_pdu/4 +%% Func: make_v1_trap_pdu/5 %% Args: Enterprise = oid() %% Specific = integer() %% Varbinds is as returned from initiate_vars %% (but only {Oid, Type[, Value} permitted) %% SysUpTime = integer() +%% AgentIp = {A, B, C, D} %% Purpose: Make a #trappdu %% Checks the Varbinds to see that no symbolic names are %% present, and that each var has a type. Performs a get @@ -284,7 +285,7 @@ localise_type(X, _) -> X. %% Fails: yes %% NOTE: Executed at the MA %%----------------------------------------------------------------- -make_v1_trap_pdu(Enterprise, Specific, VarbindList, SysUpTime) -> +make_v1_trap_pdu(Enterprise, Specific, VarbindList, SysUpTime, AgentIp) -> {Enterp,Generic,Spec} = case Enterprise of ?snmp -> @@ -292,7 +293,6 @@ make_v1_trap_pdu(Enterprise, Specific, VarbindList, SysUpTime) -> _ -> {Enterprise,?enterpriseSpecific,Specific} end, - {value, AgentIp} = snmp_framework_mib:intAgentIpAddress(get), #trappdu{enterprise = Enterp, agent_addr = AgentIp, generic_trap = Generic, @@ -369,6 +369,7 @@ send_trap(TrapRec, NotifyName, ContextName, Recv, Vbs, LocalEngineID, {tag, T}, {err, E}, {stacktrace, erlang:get_stacktrace()}], + ?vlog("snmpa_trap:send_trap exception: ~p", [Info]), {error, {failed_sending_trap, Info}} end. @@ -789,23 +790,19 @@ send_trap_pdus([], ContextName, {TrapRec, Vbs}, send_v1_trap(_TrapRec, [], _Vbs, _ExtraInfo, _NetIf, _SysUpTime) -> ok; -send_v1_trap(#trap{enterpriseoid = Enter, specificcode = Spec}, - V1Res, Vbs, ExtraInfo, NetIf, SysUpTime) -> +send_v1_trap( + #trap{enterpriseoid = Enter, specificcode = Spec}, + V1Res, Vbs, ExtraInfo, NetIf, SysUpTime) -> ?vdebug("prepare to send v1 trap " "~n '~p'" "~n with" "~n ~p" "~n to" "~n ~p", [Enter, Spec, V1Res]), - TrapPdu = make_v1_trap_pdu(Enter, Spec, Vbs, SysUpTime), - AddrCommunities = mk_addr_communities(V1Res), - lists:foreach(fun({Community, Addrs}) -> - ?vtrace("send v1 trap pdu to ~p",[Addrs]), - NetIf ! {send_pdu, 'version-1', TrapPdu, - {community, Community}, Addrs, ExtraInfo} - end, AddrCommunities); -send_v1_trap(#notification{oid = Oid}, V1Res, Vbs, ExtraInfo, NetIf, - SysUpTime) -> + do_send_v1_trap(Enter, Spec, V1Res, Vbs, ExtraInfo, NetIf, SysUpTime); +send_v1_trap( + #notification{oid = Oid}, + V1Res, Vbs, ExtraInfo, NetIf, SysUpTime) -> %% Use alg. in rfc2089 to map a v2 trap to a v1 trap % delete Counter64 objects from vbs ?vdebug("prepare to send v1 trap '~p'",[Oid]), @@ -822,14 +819,38 @@ send_v1_trap(#notification{oid = Oid}, V1Res, Vbs, ExtraInfo, NetIf, {lists:reverse(First),Last} end end, - TrapPdu = make_v1_trap_pdu(Enter, Spec, NVbs, SysUpTime), + do_send_v1_trap(Enter, Spec, V1Res, NVbs, ExtraInfo, NetIf, SysUpTime). + +do_send_v1_trap(Enter, Spec, V1Res, NVbs, ExtraInfo, NetIf, SysUpTime) -> + {value, Transports} = snmp_framework_mib:intAgentTransports(get), + {_Domain, {AgentIp, _AgentPort}} = + case lists:keyfind(snmpUDPDomain, 1, Transports) of + false -> + case lists:keyfind(transportDomainUdpIpv4, 1, Transports) of + false -> + ?vtrace( + "snmpa_trap: can not send v1 trap " + "without IPv4 domain: ~p", + [Transports]), + user_err( + "snmpa_trap: can not send v1 trap " + "without IPv4 domain: ~p", + [Transports]); + DomainAddr -> + DomainAddr + end; + DomainAddr -> + DomainAddr + end, + TrapPdu = make_v1_trap_pdu(Enter, Spec, NVbs, SysUpTime, AgentIp), AddrCommunities = mk_addr_communities(V1Res), - lists:foreach(fun({Community, Addrs}) -> - ?vtrace("send v1 trap to ~p",[Addrs]), - NetIf ! {send_pdu, 'version-1', TrapPdu, - {community, Community}, Addrs, ExtraInfo} - end, AddrCommunities). - + lists:foreach( + fun ({Community, Addrs}) -> + ?vtrace("send v1 trap to ~p",[Addrs]), + NetIf ! {send_pdu, 'version-1', TrapPdu, + {community, Community}, Addrs, ExtraInfo} + end, AddrCommunities). + send_v2_trap(_TrapRec, [], _Vbs, _Recv, _ExtraInfo, _NetIf, _SysUpTime) -> ok; send_v2_trap(TrapRec, V2Res, Vbs, Recv, ExtraInfo, NetIf, SysUpTime) -> @@ -1045,7 +1066,7 @@ deliver_recv(#snmpa_notification_delivery_info{tag = Tag, "~n DeliveryResult: ~p" "~n TAddr: ~p" "", [Tag, Mod, Extra, DeliveryResult, TAddr]), - Addr = transform_taddr(TAddr), + [Addr] = transform_taddrs([TAddr]), (catch Mod:delivery_info(Tag, Addr, DeliveryResult, Extra)); deliver_recv({Tag, Receiver}, MsgId, Result) -> ?vtrace("deliver_recv -> entry with" @@ -1072,35 +1093,90 @@ deliver_recv(Else, _MsgId, _Result) -> [Else]), user_err("snmpa: bad receiver, ~w\n", [Else]). -transform_taddrs(Addrs) -> - [transform_taddr(Addr) || Addr <- Addrs]. +transform_taddrs(TAddrs) -> + UseTDomain = + case snmp_framework_mib:intAgentTransportDomain(get) of + {value,snmpUDPDomain} -> + false; + {value,_} -> + true; + genErr -> + false + end, + DomAddrs = [transform_taddr(TAddr) || TAddr <- TAddrs], + case UseTDomain of + true -> + DomAddrs; + false -> + [Addr || {_Domain, Addr} <- DomAddrs] + end. -transform_taddr({?snmpUDPDomain, [A1, A2, A3, A4, P1, P2]}) -> % v2 - Addr = {A1, A2, A3, A4}, - Port = P1 bsl 8 + P2, - {Addr, Port}; -transform_taddr({?transportDomainUdpIpv4, [A1, A2, A3, A4, P1, P2]}) -> % v2 - Addr = {A1, A2, A3, A4}, +%% v2 +transform_taddr({?snmpUDPDomain, Addr}) -> + transform_taddr(transportDomainIdpIpv4, Addr); +transform_taddr({?transportDomainUdpIpv4, Addr}) -> + transform_taddr(transportDomainUdpIpv4, Addr); +transform_taddr({?transportDomainUdpIpv6, Addr}) -> + transform_taddr(transportDomainUdpIpv6, Addr); +%% v3 +transform_taddr({{?snmpUDPDomain, Addr}, _MsgData}) -> + transform_taddr(transportDomainUdpIpv4, Addr); +transform_taddr({{?transportDomainUdpIpv4, Addr}, _MsgData}) -> + transform_taddr(transportDomainUdpIpv4, Addr); +transform_taddr({{?transportDomainUdpIpv6, Addr}, _MsgData}) -> + transform_taddr(transportDomainUdpIpv6, Addr). + +transform_taddr( + transportDomainUdpIpv4 = Domain, + [A1,A2,A3,A4,P1,P2]) -> + Ip = {A1, A2, A3, A4}, Port = P1 bsl 8 + P2, - {Addr, Port}; -transform_taddr({?transportDomainUdpIpv6, - [A1, A2, A3, A4, A5, A6, A7, A8, P1, P2]}) -> % v2 - Addr = {A1, A2, A3, A4, A5, A6, A7, A8}, + {Domain, {Ip, Port}}; +transform_taddr( + transportDomainUdpIpv6 = Domain, + [A1, A2, A3, A4, A5, A6, A7, A8, P1, P2]) -> + Ip = {A1, A2, A3, A4, A5, A6, A7, A8}, Port = P1 bsl 8 + P2, - {Addr, Port}; -transform_taddr({{?snmpUDPDomain, [A1, A2, A3, A4, P1, P2]}, _MsgData}) -> % v3 - Addr = {A1, A2, A3, A4}, + {Domain, {Ip, Port}}; +transform_taddr( + transportDomainUdpIpv6 = Domain, + [A1, A2, A3, A4, A5, A6, A7, A8, A9, A10, A11, A12, A13, A14, A15, A16, + P1, P2]) -> + Ip = + {(A1 bsl 8) bor A2, (A3 bsl 8) bor A4, + (A5 bsl 8) bor A6, (A7 bsl 8) bor A8, + (A9 bsl 8) bor A10, (A11 bsl 8) bor A12, + (A13 bsl 8) bor A14, (A15 bsl 8) bor A16}, Port = P1 bsl 8 + P2, - {Addr, Port}; -transform_taddr({{?transportDomainUdpIpv4, [A1, A2, A3, A4, P1, P2]}, _MsgData}) -> % v3 - Addr = {A1, A2, A3, A4}, - Port = P1 bsl 8 + P2, - {Addr, Port}; -transform_taddr({{?transportDomainUdpIpv6, - [A1, A2, A3, A4, A5, A6, A7, A8, P1, P2]}, _MsgData}) -> % v3 - Addr = {A1, A2, A3, A4, A5, A6, A7, A8}, - Port = P1 bsl 8 + P2, - {Addr, Port}. + {Domain, {Ip, Port}}. + +%% transform_taddr({?snmpUDPDomain, [A1, A2, A3, A4, P1, P2]}) -> % v2 +%% Addr = {A1, A2, A3, A4}, +%% Port = P1 bsl 8 + P2, +%% {Addr, Port}; +%% transform_taddr({?transportDomainUdpIpv4, [A1, A2, A3, A4, P1, P2]}) -> % v2 +%% Addr = {A1, A2, A3, A4}, +%% Port = P1 bsl 8 + P2, +%% {Addr, Port}; +%% transform_taddr({?transportDomainUdpIpv6, +%% [A1, A2, A3, A4, A5, A6, A7, A8, P1, P2]}) -> % v2 +%% Addr = {A1, A2, A3, A4, A5, A6, A7, A8}, +%% Port = P1 bsl 8 + P2, +%% {Addr, Port}; +%% transform_taddr({{?snmpUDPDomain, [A1, A2, A3, A4, P1, P2]}, _MsgData}) -> % v3 +%% Addr = {A1, A2, A3, A4}, +%% Port = P1 bsl 8 + P2, +%% {Addr, Port}; +%% transform_taddr({{?transportDomainUdpIpv4, [A1, A2, A3, A4, P1, P2]}, _MsgData}) -> % v3 +%% Addr = {A1, A2, A3, A4}, +%% Port = P1 bsl 8 + P2, +%% {Addr, Port}; +%% transform_taddr({{?transportDomainUdpIpv6, +%% [A1, A2, A3, A4, A5, A6, A7, A8, P1, P2]}, _MsgData}) -> % v3 +%% Addr = {A1, A2, A3, A4, A5, A6, A7, A8}, +%% Port = P1 bsl 8 + P2, +%% {Addr, Port}. + check_all_varbinds(#notification{oid = Oid}, Vbs, MibView) -> diff --git a/lib/snmp/src/app/snmp.appup.src b/lib/snmp/src/app/snmp.appup.src index babc33e6a5..1cc1a17b1d 100644 --- a/lib/snmp/src/app/snmp.appup.src +++ b/lib/snmp/src/app/snmp.appup.src @@ -28,6 +28,9 @@ %% {update, snmpa_local_db, soft, soft_purge, soft_purge, []} %% {add_module, snmpm_net_if_mt} [ + {"5.0", [{restart_application, snmp}]}, + {"4.25.1", [{restart_application, snmp}]}, + {"4.25.0.1", [{restart_application, snmp}]}, {"4.25", [{restart_application, snmp}]}, {"4.24.2", [{restart_application, snmp}]}, {"4.24.1", [{restart_application, snmp}]}, @@ -40,6 +43,9 @@ %% {remove, {snmpm_net_if_mt, soft_purge, soft_purge}} [ + {"5.0", [{restart_application, snmp}]}, + {"4.25.1", [{restart_application, snmp}]}, + {"4.25.0.1", [{restart_application, snmp}]}, {"4.25", [{restart_application, snmp}]}, {"4.24.2", [{restart_application, snmp}]}, {"4.24.1", [{restart_application, snmp}]}, diff --git a/lib/snmp/src/manager/depend.mk b/lib/snmp/src/manager/depend.mk index 2e7783c8ed..60f61b0d3b 100644 --- a/lib/snmp/src/manager/depend.mk +++ b/lib/snmp/src/manager/depend.mk @@ -2,7 +2,7 @@ # %CopyrightBegin% # -# Copyright Ericsson AB 2004-2012. All Rights Reserved. +# Copyright Ericsson AB 2004-2014. All Rights Reserved. # # The contents of this file are subject to the Erlang Public License, # Version 1.1, (the "License"); you may not use this file except in @@ -50,10 +50,11 @@ $(EBIN)/snmpm_net_if.$(EMULATOR): \ snmpm_network_interface.erl $(EBIN)/snmpm_net_if_mt.$(EMULATOR): \ + snmpm_net_if_mt.erl \ ../../include/snmp_types.hrl \ ../misc/snmp_debug.hrl \ ../misc/snmp_verbosity.hrl \ - snmpm_net_if_mt.erl \ + snmpm_net_if.erl \ snmpm_network_interface.erl $(EBIN)/snmpm_server.$(EMULATOR): \ diff --git a/lib/snmp/src/manager/snmpm.erl b/lib/snmp/src/manager/snmpm.erl index c97b635fc6..8976322c4e 100644 --- a/lib/snmp/src/manager/snmpm.erl +++ b/lib/snmp/src/manager/snmpm.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2013. All Rights Reserved. +%% Copyright Ericsson AB 2004-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -379,21 +379,33 @@ register_agent(UserId, Addr) -> register_agent(UserId, Addr, ?DEFAULT_AGENT_PORT, []). %% Backward compatibility -register_agent(UserId, Addr, Port, Config0) -> +register_agent(UserId, Domain, Addr, Config0) when is_atom(Domain) -> case lists:keymember(target_name, 1, Config0) of false -> - TargetName = mk_target_name(Addr, Port, Config0), - Config = [{reg_type, addr_port}, - {address, Addr}, {port, Port} | Config0], + TargetName = mk_target_name(Domain, Addr, Config0), + Config = + [{reg_type, addr_port}, + {tdomain, Domain}, {taddress, Addr} | Config0], do_register_agent(UserId, TargetName, ensure_engine_id(Config)); true -> {value, {_, TargetName}} = lists:keysearch(target_name, 1, Config0), Config1 = lists:keydelete(target_name, 1, Config0), - Config2 = [{reg_type, addr_port}, - {address, Addr}, {port, Port} | Config1], + Config2 = + [{reg_type, addr_port}, + {tdomain, Domain}, {taddress, Addr} | Config1], register_agent(UserId, TargetName, ensure_engine_id(Config2)) - end. + end; +register_agent(UserId, Ip, Port, Config) when is_integer(Port) -> + Domain = snmpm_config:default_transport_domain(), + Addr = + case snmp_conf:check_address(Domain, {Ip, Port}) of + ok -> + {Ip, Port}; + {ok, FixedAddr} -> + FixedAddr + end, + register_agent(UserId, Domain, Addr, Config). unregister_agent(UserId, TargetName) when is_list(TargetName) -> snmpm_config:unregister_agent(UserId, TargetName); @@ -402,8 +414,8 @@ unregister_agent(UserId, TargetName) when is_list(TargetName) -> unregister_agent(UserId, Addr) -> unregister_agent(UserId, Addr, ?DEFAULT_AGENT_PORT). -unregister_agent(UserId, Addr, Port) -> - case target_name(Addr, Port) of +unregister_agent(UserId, DomainIp, AddressPort) -> + case target_name(DomainIp, AddressPort) of {ok, TargetName} -> unregister_agent(UserId, TargetName); Error -> @@ -1264,14 +1276,17 @@ format_vb_value(Prefix, _Type, Val) -> %% --- Internal utility functions --- %% -target_name(Addr) -> - target_name(Addr, ?DEFAULT_AGENT_PORT). +target_name(Ip) -> + target_name(Ip, ?DEFAULT_AGENT_PORT). -target_name(Addr, Port) -> - snmpm_config:agent_info(Addr, Port, target_name). +target_name(DomainIp, AddressPort) -> + snmpm_config:agent_info(DomainIp, AddressPort, target_name). mk_target_name(Addr, Port, Config) -> - snmpm_config:mk_target_name(Addr, Port, Config). + R = snmpm_config:mk_target_name(Addr, Port, Config), + p(?MODULE_STRING":mk_target_name(~p, ~p, ~p) -> ~p.~n", + [Addr, Port, Config, R]), + R. ensure_engine_id(Config) -> case lists:keymember(engine_id, 1, Config) of @@ -1287,5 +1302,5 @@ ensure_engine_id(Config) -> %% p(F) -> %% p(F, []). -%% p(F, A) -> -%% io:format("~w:" ++ F ++ "~n", [?MODULE | A]). +p(F, A) -> + io:format("~w:" ++ F ++ "~n", [?MODULE | A]). diff --git a/lib/snmp/src/manager/snmpm_conf.erl b/lib/snmp/src/manager/snmpm_conf.erl index 5e2d9fdbf6..087ef6c6ea 100644 --- a/lib/snmp/src/manager/snmpm_conf.erl +++ b/lib/snmp/src/manager/snmpm_conf.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2006-2013. All Rights Reserved. +%% Copyright Ericsson AB 2006-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -78,43 +78,26 @@ write_manager_config(Dir, Conf) -> Hdr = header() ++ Comment, write_manager_config(Dir, Hdr, Conf). -write_manager_config(Dir, Hdr, Conf) - when is_list(Dir) andalso is_list(Hdr) andalso is_list(Conf) -> - Verify = fun() -> verify_manager_conf(Conf) end, - Write = fun(Fid) -> write_manager_conf(Fid, Hdr, Conf) end, - write_config_file(Dir, ?MANAGER_CONF_FILE, Verify, Write). - +write_manager_config(Dir, Hdr, Conf) + when is_list(Dir), is_list(Hdr), is_list(Conf) -> + Order = fun snmpm_config:order_manager_config/2, + Check = fun snmpm_config:check_manager_config/2, + Write = fun (Fd, Entries) -> write_manager_conf(Fd, Hdr, Entries) end, + write_config_file(Dir, ?MANAGER_CONF_FILE, Order, Check, Write, Conf). append_manager_config(Dir, Conf) - when is_list(Dir) andalso is_list(Conf) -> - Verify = fun() -> verify_manager_conf(Conf) end, - Write = fun(Fid) -> write_manager_conf(Fid, Conf) end, - append_config_file(Dir, ?MANAGER_CONF_FILE, Verify, Write). - + when is_list(Dir), is_list(Conf) -> + Order = fun snmpm_config:order_manager_config/2, + Check = fun snmpm_config:check_manager_config/2, + Write = fun write_manager_conf/2, + append_config_file(Dir, ?MANAGER_CONF_FILE, Order, Check, Write, Conf). -read_manager_config(Dir) -> - Verify = fun(Entry) -> verify_manager_conf_entry(Entry) end, - read_config_file(Dir, ?MANAGER_CONF_FILE, Verify). +read_manager_config(Dir) when is_list(Dir) -> + Order = fun snmpm_config:order_manager_config/2, + Check = fun snmpm_config:check_manager_config/2, + read_config_file(Dir, ?MANAGER_CONF_FILE, Order, Check). -verify_manager_conf([]) -> - ok; -verify_manager_conf([H|T]) -> - verify_manager_conf_entry(H), - verify_manager_conf(T); -verify_manager_conf(X) -> - error({bad_manager_config, X}). - -verify_manager_conf_entry(Entry) -> - case snmpm_config:check_manager_config(Entry) of - ok -> - ok; -%% {ok, _} -> -%% ok; - Error -> - throw(Error) - end. - write_manager_conf(Fd, "", Conf) -> write_manager_conf(Fd, Conf); write_manager_conf(Fd, Hdr, Conf) -> @@ -127,14 +110,16 @@ write_manager_conf(Fd, [H|T]) -> do_write_manager_conf(Fd, H), write_manager_conf(Fd, T). -do_write_manager_conf(Fd, {address = Tag, Val}) -> - io:format(Fd, "{~w, ~w}.~n", [Tag, Val]); -do_write_manager_conf(Fd, {port = Tag, Val} ) -> +do_write_manager_conf(Fd, {Tag, Val}) + when Tag =:= domain; + Tag =:= address; + Tag =:= port; + Tag =:= transports; + Tag =:= max_message_size -> io:format(Fd, "{~w, ~w}.~n", [Tag, Val]); -do_write_manager_conf(Fd, {engine_id = Tag, Val} ) -> +do_write_manager_conf(Fd, {Tag, Val}) + when Tag =:= engine_id -> io:format(Fd, "{~w, \"~s\"}.~n", [Tag, Val]); -do_write_manager_conf(Fd, {max_message_size = Tag, Val} ) -> - io:format(Fd, "{~w, ~w}.~n", [Tag, Val]); do_write_manager_conf(_Fd, Crap) -> error({bad_manager_config, Crap}). @@ -167,36 +152,29 @@ write_users_config(Dir, Conf) -> Hdr = header() ++ Comment, write_users_config(Dir, Hdr, Conf). -write_users_config(Dir, Hdr, Conf) +write_users_config(Dir, Hdr, Conf) when is_list(Dir) andalso is_list(Hdr) andalso is_list(Conf) -> - Verify = fun() -> verify_users_conf(Conf) end, - Write = fun(Fd) -> write_users_conf(Fd, Hdr, Conf) end, - write_config_file(Dir, ?USERS_CONF_FILE, Verify, Write). - + Order = fun snmp_conf:no_order/2, + Check = fun check_user_config/2, + Write = fun (Fd, Entries) -> write_users_conf(Fd, Hdr, Entries) end, + write_config_file(Dir, ?USERS_CONF_FILE, Order, Check, Write, Conf). -append_users_config(Dir, Conf) +append_users_config(Dir, Conf) when is_list(Dir) andalso is_list(Conf) -> - Verify = fun() -> verify_users_conf(Conf) end, - Write = fun(Fd) -> write_users_conf(Fd, Conf) end, - append_config_file(Dir, ?USERS_CONF_FILE, Verify, Write). - + Order = fun snmp_conf:no_order/2, + Check = fun check_user_config/2, + Write = fun write_users_conf/2, + append_config_file(Dir, ?USERS_CONF_FILE, Order, Check, Write, Conf). read_users_config(Dir) when is_list(Dir) -> - Verify = fun(Entry) -> verify_users_conf_entry(Entry) end, - read_config_file(Dir, ?USERS_CONF_FILE, Verify). + Order = fun snmp_conf:no_order/2, + Check = fun check_user_config/2, + read_config_file(Dir, ?USERS_CONF_FILE, Order, Check). - -verify_users_conf([]) -> - ok; -verify_users_conf([H|T]) -> - verify_users_conf_entry(H), - verify_users_conf(T); -verify_users_conf(X) -> - error({bad_users_conf, X}). - -verify_users_conf_entry(Entry) -> - {ok, _} = snmpm_config:check_user_config(Entry), - ok. + +check_user_config(Entry, State) -> + {check_ok(snmpm_config:check_user_config(Entry)), + State}. write_users_conf(Fd, "", Conf) -> write_users_conf(Fd, Conf); @@ -222,9 +200,10 @@ do_write_users_conf(_Fd, Crap) -> %% ------ agents.conf ------ %% -agents_entry(UserId, TargetName, Comm, Ip, Port, EngineID, Timeout, - MaxMessageSize, Version, SecModel, SecName, SecLevel) -> - {UserId, TargetName, Comm, Ip, Port, EngineID, Timeout, +agents_entry( + UserId, TargetName, Comm, Domain_or_Ip, Addr_or_Port, EngineID, Timeout, + MaxMessageSize, Version, SecModel, SecName, SecLevel) -> + {UserId, TargetName, Comm, Domain_or_Ip, Addr_or_Port, EngineID, Timeout, MaxMessageSize, Version, SecModel, SecName, SecLevel}. @@ -239,36 +218,29 @@ write_agents_config(Dir, Conf) -> Hdr = header() ++ Comment, write_agents_config(Dir, Hdr, Conf). -write_agents_config(Dir, Hdr, Conf) +write_agents_config(Dir, Hdr, Conf) when is_list(Dir) andalso is_list(Hdr) andalso is_list(Conf) -> - Verify = fun() -> verify_agents_conf(Conf) end, - Write = fun(Fd) -> write_agents_conf(Fd, Hdr, Conf) end, - write_config_file(Dir, ?AGENTS_CONF_FILE, Verify, Write). + Order = fun snmp_conf:no_order/2, + Check = fun check_agent_config/2, + Write = fun (Fd, Entries) -> write_agents_conf(Fd, Hdr, Entries) end, + write_config_file(Dir, ?AGENTS_CONF_FILE, Order, Check, Write, Conf). - -append_agents_config(Dir, Conf) +append_agents_config(Dir, Conf) when is_list(Dir) andalso is_list(Conf) -> - Verify = fun() -> verify_agents_conf(Conf) end, - Write = fun(Fd) -> write_agents_conf(Fd, Conf) end, - append_config_file(Dir, ?AGENTS_CONF_FILE, Verify, Write). - + Order = fun snmp_conf:no_order/2, + Check = fun check_agent_config/2, + Write = fun write_agents_conf/2, + append_config_file(Dir, ?AGENTS_CONF_FILE, Order, Check, Write, Conf). read_agents_config(Dir) -> - Verify = fun(Entry) -> verify_agents_conf_entry(Entry) end, - read_config_file(Dir, ?AGENTS_CONF_FILE, Verify). + Order = fun snmp_conf:no_order/2, + Check = fun check_agent_config/2, + read_config_file(Dir, ?AGENTS_CONF_FILE, Order, Check). -verify_agents_conf([]) -> - ok; -verify_agents_conf([H|T]) -> - verify_agents_conf_entry(H), - verify_agents_conf(T); -verify_agents_conf(X) -> - error({bad_agents_config, X}). - -verify_agents_conf_entry(Entry) -> - {ok, _} = snmpm_config:check_agent_config(Entry), - ok. +check_agent_config(Entry, State) -> + {check_ok(snmpm_config:check_agent_config(Entry)), + State}. write_agents_conf(Fd, "", Conf) -> write_agents_conf(Fd, Conf); @@ -282,13 +254,15 @@ write_agents_conf(Fd, [H|T]) -> do_write_agents_conf(Fd, H), write_agents_conf(Fd, T). -do_write_agents_conf(Fd, - {UserId, - TargetName, Comm, Ip, Port, EngineID, - Timeout, MaxMessageSize, Version, - SecModel, SecName, SecLevel} = _A) -> - io:format(Fd, - "{~w, \"~s\", \"~s\", ~w, ~w, \"~s\", ~w, ~w, ~w, ~w, \"~s\", ~w}.~n", [UserId, TargetName, Comm, Ip, Port, EngineID, Timeout, MaxMessageSize, Version, SecModel, SecName, SecLevel]); +do_write_agents_conf( + Fd, + {UserId, TargetName, Comm, Ip, Port, EngineID, + Timeout, MaxMessageSize, Version, SecModel, SecName, SecLevel} = _A) -> + io:format( + Fd, + "{~w, \"~s\", \"~s\", ~w, ~w, \"~s\", ~w, ~w, ~w, ~w, \"~s\", ~w}.~n", + [UserId, TargetName, Comm, Ip, Port, EngineID, + Timeout, MaxMessageSize, Version, SecModel, SecName, SecLevel]); do_write_agents_conf(_Fd, Crap) -> error({bad_agents_config, Crap}). @@ -314,37 +288,30 @@ write_usm_config(Dir, Conf) -> Hdr = header() ++ Comment, write_usm_config(Dir, Hdr, Conf). -write_usm_config(Dir, Hdr, Conf) +write_usm_config(Dir, Hdr, Conf) when is_list(Dir) andalso is_list(Hdr) andalso is_list(Conf) -> - Verify = fun() -> verify_usm_conf(Conf) end, - Write = fun(Fd) -> write_usm_conf(Fd, Hdr, Conf) end, - write_config_file(Dir, ?USM_USERS_CONF_FILE, Verify, Write). + Order = fun snmp_conf:no_order/2, + Check = fun check_usm_user_config/2, + Write = fun (Fd, Entries) -> write_usm_conf(Fd, Hdr, Entries) end, + write_config_file(Dir, ?USM_USERS_CONF_FILE, Order, Check, Write, Conf). - -append_usm_config(Dir, Conf) +append_usm_config(Dir, Conf) when is_list(Dir) andalso is_list(Conf) -> - Verify = fun() -> verify_usm_conf(Conf) end, - Write = fun(Fd) -> write_usm_conf(Fd, Conf) end, - append_config_file(Dir, ?USM_USERS_CONF_FILE, Verify, Write). - + Order = fun snmp_conf:no_order/2, + Check = fun check_usm_user_config/2, + Write = fun write_usm_conf/2, + append_config_file(Dir, ?USM_USERS_CONF_FILE, Order, Check, Write, Conf). read_usm_config(Dir) when is_list(Dir) -> - Verify = fun(Entry) -> verify_usm_conf_entry(Entry) end, - read_config_file(Dir, ?USM_USERS_CONF_FILE, Verify). + Order = fun snmp_conf:no_order/2, + Check = fun check_usm_user_config/2, + read_config_file(Dir, ?USM_USERS_CONF_FILE, Order, Check). -verify_usm_conf([]) -> - ok; -verify_usm_conf([H|T]) -> - verify_usm_conf_entry(H), - verify_usm_conf(T); -verify_usm_conf(X) -> - error({bad_usm_conf, X}). - -verify_usm_conf_entry(Entry) -> - {ok, _} = snmpm_config:check_usm_user_config(Entry), - ok. +check_usm_user_config(Entry, State) -> + {check_ok(snmpm_config:check_usm_user_config(Entry)), + State}. write_usm_conf(Fd, "", Conf) -> write_usm_conf(Fd, Conf); @@ -358,41 +325,49 @@ write_usm_conf(Fd, [H|T]) -> do_write_usm_conf(Fd, H), write_usm_conf(Fd, T). -do_write_usm_conf(Fd, - {EngineID, UserName, AuthP, AuthKey, PrivP, PrivKey}) -> - io:format(Fd, "{\"~s\", \"~s\", ~w, ~w, ~w, ~w}.~n", - [EngineID, UserName, AuthP, AuthKey, PrivP, PrivKey]); -do_write_usm_conf(Fd, - {EngineID, UserName, SecName, - AuthP, AuthKey, PrivP, PrivKey}) -> - io:format(Fd, "{\"~s\", \"~s\", \"~s\", í~w, ~w, ~w, ~w}.~n", - [EngineID, UserName, SecName, AuthP, AuthKey, PrivP, PrivKey]); +do_write_usm_conf( + Fd, + {EngineID, UserName, AuthP, AuthKey, PrivP, PrivKey}) -> + io:format( + Fd, "{\"~s\", \"~s\", ~w, ~w, ~w, ~w}.~n", + [EngineID, UserName, AuthP, AuthKey, PrivP, PrivKey]); +do_write_usm_conf( + Fd, + {EngineID, UserName, SecName, + AuthP, AuthKey, PrivP, PrivKey}) -> + io:format( + Fd, "{\"~s\", \"~s\", \"~s\", í~w, ~w, ~w, ~w}.~n", + [EngineID, UserName, SecName, AuthP, AuthKey, PrivP, PrivKey]); do_write_usm_conf(_Fd, Crap) -> error({bad_usm_conf, Crap}). %% ---- config file wrapper functions ---- -write_config_file(Dir, File, Verify, Write) -> - snmp_config:write_config_file(Dir, File, Verify, Write). - -append_config_file(Dir, File, Verify, Write) -> - snmp_config:append_config_file(Dir, File, Verify, Write). +write_config_file(Dir, File, Order, Check, Write, Conf) -> + snmp_config:write_config_file(Dir, File, Order, Check, Write, Conf). -read_config_file(Dir, File, Verify) -> - snmp_config:read_config_file(Dir, File, Verify). +append_config_file(Dir, File, Order, Check, Write, Conf) -> + snmp_config:append_config_file(Dir, File, Order, Check, Write, Conf). +read_config_file(Dir, File, Order, Check) -> + snmp_config:read_config_file(Dir, File, Order, Check). %% ---- config file utility functions ---- +check_ok(ok) -> + ok; +check_ok({ok, _}) -> + ok. + header() -> {Y,Mo,D} = date(), {H,Mi,S} = time(), - io_lib:format("%% This file was generated by " - "~w (version-~s) ~w-~2.2.0w-~2.2.0w " - "~2.2.0w:~2.2.0w:~2.2.0w\n", - [?MODULE, ?version, Y, Mo, D, H, Mi, S]). - + io_lib:format( + "%% This file was generated by " + "~w (version-~s) ~w-~2.2.0w-~2.2.0w " + "~2.2.0w:~2.2.0w:~2.2.0w\n", + [?MODULE, ?version, Y, Mo, D, H, Mi, S]). error(R) -> throw({error, R}). diff --git a/lib/snmp/src/manager/snmpm_config.erl b/lib/snmp/src/manager/snmpm_config.erl index 2101ad46e1..5cab81baf6 100644 --- a/lib/snmp/src/manager/snmpm_config.erl +++ b/lib/snmp/src/manager/snmpm_config.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2013. All Rights Reserved. +%% Copyright Ericsson AB 2004-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -103,8 +103,10 @@ get_agent_mp_model/2 ]). --export([check_manager_config/1, - check_user_config/1, +-export([ + order_manager_config/2, + check_manager_config/2, + check_user_config/1, check_agent_config/1, check_usm_user_config/1]). @@ -165,7 +167,7 @@ %%%------------------------------------------------------------------- default_transport_domain() -> - transportDomainUdpIpv4. + snmpUDPDomain. start_link(Opts) -> @@ -190,7 +192,11 @@ register_user(UserId, UserMod, UserData, DefaultAgentConfig) when (UserId =/= ?DEFAULT_USER) andalso is_list(DefaultAgentConfig) -> case (catch verify_user_behaviour(UserMod)) of ok -> - Config = default_agent_config(DefaultAgentConfig), + {ok, SystemDefaultAgentConfig} = agent_info(), + Config = + ensure_config(SystemDefaultAgentConfig, + DefaultAgentConfig), +%% Config = default_agent_config(DefaultAgentConfig), call({register_user, UserId, UserMod, UserData, Config}); Error -> Error @@ -201,19 +207,19 @@ register_user(UserId, _UserMod, _UserData, DefaultAgentConfig) register_user(UserId, _, _, _) -> {error, {bad_user_id, UserId}}. -default_agent_config(DefaultAgentConfig) -> - {ok, SystemDefaultAgentConfig} = agent_info(), - default_agent_config(SystemDefaultAgentConfig, DefaultAgentConfig). +%% default_agent_config(DefaultAgentConfig) -> +%% {ok, SystemDefaultAgentConfig} = agent_info(), +%% default_agent_config(SystemDefaultAgentConfig, DefaultAgentConfig). -default_agent_config([], DefaultAgentConfig) -> - DefaultAgentConfig; -default_agent_config([{Key, _} = Entry|T], DefaultAgentConfig) -> - case lists:keysearch(Key, 1, DefaultAgentConfig) of - {value, _} -> - default_agent_config(T, DefaultAgentConfig); - false -> - default_agent_config(T, [Entry|DefaultAgentConfig]) - end. +%% default_agent_config([], DefaultAgentConfig) -> +%% DefaultAgentConfig; +%% default_agent_config([{Key, _} = Entry|T], DefaultAgentConfig) -> +%% case lists:keymember(Key, 1, DefaultAgentConfig) of +%% true -> +%% default_agent_config(T, DefaultAgentConfig); +%% false -> +%% default_agent_config(T, [Entry|DefaultAgentConfig]) +%% end. verify_user_behaviour(UserMod) -> @@ -280,9 +286,10 @@ do_user_info(_UserId, BadItem) -> %% A target-name constructed in this way is a string with the following: %% <IP-address>:<Port>-<Version> -%% This is intended for backward compatibility and therefor has +%% This is intended for backward compatibility and therefore has %% only support for IPv4 addresses and *no* other transport domain. -mk_target_name(Addr0, Port, Config) when is_list(Config) -> +mk_target_name(Domain, Addr, Config) + when is_atom(Domain), is_list(Config) -> Version = case lists:keysearch(version, 1, Config) of {value, {_, V}} -> @@ -290,18 +297,35 @@ mk_target_name(Addr0, Port, Config) when is_list(Config) -> false -> select_lowest_supported_version() end, - case normalize_address(Addr0) of - {A, B, C, D} -> - lists:flatten( - io_lib:format("~w.~w.~w.~w:~w-~w", [A, B, C, D, Port, Version])); - [A, B, C, D] -> + try + lists:flatten( + io_lib:format( + "~s-~w", [snmp_conf:mk_addr_string({Domain, Addr}), Version])) + catch + _ -> lists:flatten( - io_lib:format("~w.~w.~w.~w:~w-~w", [A, B, C, D, Port, Version])); - _ -> + io_lib:format("~p-~w", [Addr, Version])) + end; +mk_target_name(Ip, Port, Config) + when is_integer(Port), is_list(Config) -> + Domain = default_transport_domain(), + try fix_address(Domain, {Ip, Port}) of + Address -> + mk_target_name(Domain, Address, Config) + catch + _ -> + Version = + case lists:keysearch(version, 1, Config) of + {value, {_, V}} -> + V; + false -> + select_lowest_supported_version() + end, lists:flatten( - io_lib:format("~p:~w-~w", [Addr0, Port, Version])) + io_lib:format("~p:~w-~w", [Ip, Port, Version])) end. - + + select_lowest_supported_version() -> {ok, Versions} = system_info(versions), select_lowest_supported_version([v1, v2, v3], Versions). @@ -335,27 +359,17 @@ register_agent(UserId, TargetName, Config0) %% is not present %% 3) Check that there are no invalid or erroneous configs %% 4) Check that the manager is capable of using the selected version - case verify_agent_config(Config0) of - {ok, Config} -> - call({register_agent, UserId, TargetName, Config}); - Error -> - Error - end. - - -verify_agent_config(Conf0) -> try - begin - verify_mandatory(Conf0, [engine_id, address, reg_type]), - verify_invalid(Conf0, [user_id]), - Conf = verify_agent_config3(Conf0), - Vsns = versions(), - Vsn = which_version(Conf), - verify_version(Vsn, Vsns), - {ok, Conf} - end + verify_mandatory(Config0, [engine_id, reg_type]), + verify_someof(Config0, [address, taddress]), + verify_illegal(Config0, [user_id]), + Config = verify_agent_config(Config0), + Vsns = versions(), + Vsn = which_version(Config), + verify_version(Vsn, Vsns), + call({register_agent, UserId, TargetName, Config}) catch - throw:Error -> + Error -> Error end. @@ -381,52 +395,43 @@ verify_version(Vsn, Vsns) -> ok; false -> Reason = {version_not_supported_by_manager, Vsn, Vsns}, - throw({error, Reason}) - end. - -verify_agent_config3(Conf0) -> - %% Fix (transport) address and domain - {TDomain, Conf1} = - case lists:keysearch(tdomain, 1, Conf0) of - {value, {tdomain, Dom}} -> - {Dom, Conf0}; - false -> - Dom = default_transport_domain(), - {Dom, [{tdomain, Dom} | Conf0]} - end, - Conf2 = case lists:keysearch(address, 1, Conf1) of - {value, {address, Address}} -> - lists:keyreplace(address, 1, Conf1, - {address, {TDomain, Address}}); - false -> - %% This is a mandatory config option, - %% a later test will detect this - Conf1 - end, - case verify_agent2(Conf2) of - {ok, Conf} -> - Conf; - {error, _} = ERROR -> - throw(ERROR) + error(Reason) end. -verify_agent_config2(Conf) -> - verify_agent2(Conf). unregister_agent(UserId, TargetName) -> call({unregister_agent, UserId, TargetName}). %% This is the old style agent unregistration (using Addr and Port). -unregister_agent(UserId, Addr0, Port) -> - Addr = normalize_address(Addr0), - case do_agent_info(Addr, Port, target_name) of +unregister_agent(UserId, Domain, Address) when is_atom(Domain) -> + try fix_address(Domain, Address) of + NAddress -> + do_unregister_agent(UserId, Domain, NAddress) + catch + _ -> + {error, not_found} + end; +unregister_agent(UserId, Ip, Port) when is_integer(Port) -> + Domain = default_transport_domain(), + try fix_address(Domain, {Ip, Port}) of + Address -> + do_unregister_agent(UserId, Domain, Address) + catch + _ -> + {error, not_found} + end. + +do_unregister_agent(UserId, Domain, Address) -> + case do_agent_info(Domain, Address, target_name) of {ok, TargetName} -> unregister_agent(UserId, TargetName); Error -> Error end. + + agent_info() -> agent_info(?DEFAULT_TARGETNAME, all). @@ -437,27 +442,76 @@ agent_info(TargetName, all) -> All -> {ok, [{Item, Val} || {{_, Item}, Val} <- All]} end; +%% Begin backwards compatibility +agent_info(TargetName, address) -> + case agent_info({TargetName, taddress}) of + {ok, Val} -> + {Addr, _} = Val, + {ok, Addr}; + _ -> + %% This should be redundant since 'taddress' should exist + agent_info({TargetName, address}) + end; +agent_info(TargetName, port) -> + case agent_info({TargetName, taddress}) of + {ok, Val} -> + {_, Port} = Val, + {ok, Port}; + _ -> + %% This should be redundant since 'taddress' should exist + agent_info({TargetName, port}) + end; +%% End backwards compatibility agent_info(TargetName, Item) -> - case ets:lookup(snmpm_agent_table, {TargetName, Item}) of + agent_info({TargetName, Item}). + +agent_info(Key) -> + case ets:lookup(snmpm_agent_table, Key) of [{_, Val}] -> {ok, Val}; [] -> {error, not_found} end. - -agent_info(Addr0, Port, Item) -> - Addr = normalize_address(Addr0), - do_agent_info(Addr, Port, Item). -do_agent_info(Addr, Port, target_name = Item) -> - case ets:lookup(snmpm_agent_table, {Addr, Port, Item}) of +agent_info(Domain, Address, Item) when is_atom(Domain) -> + try fix_address(Domain, Address) of + NAddress -> + do_agent_info(Domain, NAddress, Item) + catch + _Thrown -> + %% p(?MODULE_STRING":agent_info(~p, ~p, ~p) throwed ~p at.~n" + %% " ~p", + %% [Domain, Address, Item, _Thrown, erlang:get_stacktrace()]), + {error, not_found} + end; +agent_info(Ip, Port, Item) when is_integer(Port) -> + %% p(?MODULE_STRING":agent_info(~p, ~p, ~p) entry~n", + %% [Ip, Port, Item]), + Domain = default_transport_domain(), + try fix_address(Domain, {Ip, Port}) of + Address -> + do_agent_info(Domain, Address, Item) + catch + _Thrown -> + %% p(?MODULE_STRING":agent_info(~p, ~p, ~p) throwed ~p at.~n" + %% " ~p", + %% [Ip, Port, Item, _Thrown, erlang:get_stacktrace()]), + {error, not_found} + end. + +do_agent_info(Domain, Address, target_name = Item) -> + %% p(?MODULE_STRING":do_agent_info(~p, ~p, ~p) entry~n", + %% [Domain, Address, Item]), + case ets:lookup(snmpm_agent_table, {Domain, Address, Item}) of [{_, Val}] -> {ok, Val}; [] -> {error, not_found} end; -do_agent_info(Addr, Port, Item) -> - case do_agent_info(Addr, Port, target_name) of +do_agent_info(Domain, Address, Item) -> + %% p(?MODULE_STRING":do_agent_info(~p, ~p, ~p) entry~n", + %% [Domain, Address, Item]), + case do_agent_info(Domain, Address, target_name) of {ok, TargetName} -> agent_info(TargetName, Item); Error -> @@ -465,6 +519,19 @@ do_agent_info(Addr, Port, Item) -> end. +ensure_agent_info(_, [], Info) -> + Info; +ensure_agent_info(TargetName, [Item|Items], Info) -> + case lists:keymember(Item, 1, Info) of + true -> + ensure_agent_info(TargetName, Items, Info); + false -> + {ok, Value} = agent_info(TargetName, Item), + ensure_agent_info(TargetName, Items, [{Item, Value}|Info]) + end. + + + which_agents() -> which_agents('_'). @@ -474,38 +541,6 @@ which_agents(UserId) -> [TargetName || [TargetName] <- Agents]. -verify_agent_info(TargetName, Info0) -> - try - begin - verify_invalid(Info0, [user_id]), - %% Check if address is part of the list and - %% if so update it with the domain info. - Info = - case lists:keysearch(address, 1, Info0) of - {value, {address, Addr}} -> - %% If domain is part of the info, then use it. - %% If not, lookup what is already stored for - %% this agent and use that. - Domain = - case lists:keysearch(tdomain, 1, Info0) of - {value, {tdomain, Dom}} -> - Dom; - false -> - {ok, Dom} = - agent_info(TargetName, tdomain), - Dom - end, - Addr2 = {Domain, Addr}, - lists:keyreplace(address, 1, Info0, {address, Addr2}); - false -> - Info0 - end, - verify_agent2(Info) - end - catch - throw:Error -> - Error - end. update_agent_info(UserId, TargetName, Info) -> call({update_agent_info, UserId, TargetName, Info}). @@ -740,7 +775,7 @@ verify_usm_user_config(EngineID, Name, Config) -> try begin verify_mandatory(Config, []), - verify_invalid(Config, [engine_id, name]), + verify_illegal(Config, [engine_id, name]), verify_usm_user_config2(EngineID, Name, Config) end catch @@ -1073,7 +1108,11 @@ do_init(Opts) -> %% -- Prio (optional) -- Prio = get_opt(priority, Opts, normal), ets:insert(snmpm_config_table, {prio, Prio}), - process_flag(priority, Prio), + try process_flag(priority, Prio) + catch + error:badarg -> + error({invalid_priority,Prio}) + end, %% -- Server (optional) -- ServerOpts = get_opt(server, Opts, []), @@ -1271,36 +1310,6 @@ verify_options(Opts, Mandatory) -> verify_mandatory_options(Opts, Mandatory), verify_options(Opts). -%% mandatory() -> [mand()] -%% mand() -> atom() | {atom, [atom()]} -verify_mandatory_options(_Opts, []) -> - ok; -verify_mandatory_options(Opts, [Mand|Mands]) -> - verify_mandatory_option(Opts, Mand), - verify_mandatory_options(Opts, Mands). - -verify_mandatory_option(Opts, {Mand, MandSubOpts}) -> - ?d("verify_mandatory_option -> entry with" - "~n Mand: ~p" - "~n MandSubObjs: ~p", [Mand, MandSubOpts]), - case lists:keysearch(Mand, 1, Opts) of - {value, {Mand, SubOpts}} -> - verify_mandatory_options(SubOpts, MandSubOpts); - false -> - ?d("missing mandatory option: ~w [~p]", [Mand, MandSubOpts]), - error({missing_mandatory, Mand, MandSubOpts}) - end; -verify_mandatory_option(Opts, Mand) -> - ?d("verify_mandatory_option -> entry with" - "~n Mand: ~p", [Mand]), - case lists:keymember(Mand, 1, Opts) of - true -> - ok; - false -> - ?d("missing mandatory option: ~w", [Mand]), - error({missing_mandatory, Mand}) - end. - verify_options([]) -> ?d("verify_options -> done", []), ok; @@ -1506,7 +1515,9 @@ verify_versions([]) -> ok; verify_versions([Vsn|Vsns]) -> verify_version(Vsn), - verify_versions(Vsns). + verify_versions(Vsns); +verify_versions(Vsns) -> + error({invalid_versions, Vsns}). verify_version(v1) -> ok; @@ -1614,7 +1625,38 @@ verify_verbosity(Verbosity) -> _ -> error({invalid_verbosity, Verbosity}) end. + +%% mandatory() -> [mand()] +%% mand() -> atom() | {atom, [atom()]} +verify_mandatory_options(_Opts, []) -> + ok; +verify_mandatory_options(Opts, [Mand|Mands]) -> + verify_mandatory_option(Opts, Mand), + verify_mandatory_options(Opts, Mands). + +verify_mandatory_option(Opts, {Mand, MandSubOpts}) -> + ?d("verify_mandatory_option -> entry with" + "~n Mand: ~p" + "~n MandSubObjs: ~p", [Mand, MandSubOpts]), + case lists:keysearch(Mand, 1, Opts) of + {value, {Mand, SubOpts}} -> + verify_mandatory_options(SubOpts, MandSubOpts); + false -> + ?d("missing mandatory option: ~w [~p]", [Mand, MandSubOpts]), + error({missing_mandatory, Mand, MandSubOpts}) + end; +verify_mandatory_option(Opts, Mand) -> + ?d("verify_mandatory_option -> entry with" + "~n Mand: ~p", [Mand]), + case lists:keymember(Mand, 1, Opts) of + true -> + ok; + false -> + ?d("missing mandatory option: ~w", [Mand]), + error({missing_mandatory, Mand}) + end. + %% ------------------------------------------------------------------------ init_manager_config([]) -> @@ -1629,181 +1671,91 @@ init_agent_default() -> %% The purpose of the default_agent is only to have a place %% to store system wide default values related to agents. %% - - %% Port - init_agent_default(port, ?DEFAULT_AGENT_PORT), - - %% Timeout - init_agent_default(timeout, 10000), - - %% Max message (packet) size - init_agent_default(max_message_size, 484), - - %% MPModel - init_agent_default(version, v2), - - %% SecModel - init_agent_default(sec_model, v2c), - - %% SecName - init_agent_default(sec_name, "initial"), - - %% SecLevel - init_agent_default(sec_level, noAuthNoPriv), - - %% Community - init_agent_default(community, "all-rights"), - ok. - - -init_agent_default(Item, Val) when Item =/= user_id -> - case do_update_agent_info(default_agent, Item, Val) of - ok -> - ok; - {error, Reason} -> - error(Reason) - end. - + AgentDefaultConfig = + [{port, ?DEFAULT_AGENT_PORT}, % Port + {timeout, 10000}, % Timeout + {max_message_size, 484}, % Max message (packet) size + {version, v2}, % MPModel + {sec_model, v2c}, % SecModel + {sec_name, "initial"}, % SecName + {sec_level, noAuthNoPriv}, % SecLevel + {community, "all-rights"}], % Community + do_update_agent_info(default_agent, AgentDefaultConfig). read_agents_config_file(Dir) -> - Check = fun(C) -> check_agent_config2(C) end, - case read_file(Dir, "agents.conf", Check, []) of - {ok, Conf} -> - Conf; - Error -> + Order = fun snmp_conf:no_order/2, + Check = fun check_agent_config/2, + try read_file(Dir, "agents.conf", Order, Check, []) + catch + throw:Error -> ?vlog("agent config error: ~p", [Error]), - throw(Error) + erlang:raise(throw, Error, erlang:get_stacktrace()) end. -check_agent_config2(Agent) -> - case (catch check_agent_config(Agent)) of - {ok, {UserId, TargetName, Conf, Version}} -> - {ok, Vsns} = system_info(versions), - case lists:member(Version, Vsns) of - true -> - {ok, {UserId, TargetName, Conf}}; - false -> - error({version_not_supported_by_manager, - Version, Vsns}) - end; - Err -> - throw(Err) +check_agent_config(Agent, State) -> + {ok, {UserId, TargetName, Conf, Version}} = check_agent_config(Agent), + {ok, Vsns} = system_info(versions), + case lists:member(Version, Vsns) of + true -> + {{ok, {UserId, TargetName, Conf}}, State}; + false -> + error({version_not_supported_by_manager, Version, Vsns}) end. %% For backward compatibility -check_agent_config({UserId, - TargetName, - Community, - Ip, Port, - EngineId, - Timeout, MaxMessageSize, - Version, SecModel, SecName, SecLevel}) -> - TDomain = default_transport_domain(), - check_agent_config({UserId, - TargetName, - Community, - TDomain, Ip, Port, - EngineId, - Timeout, MaxMessageSize, - Version, SecModel, SecName, SecLevel}); - -check_agent_config({UserId, - TargetName, - Community, - TDomain, Ip, Port, - EngineId, - Timeout, MaxMessageSize, - Version, SecModel, SecName, SecLevel}) -> - ?vtrace("check_agent_config -> entry with" - "~n UserId: ~p" - "~n TargetName: ~p" - "~n Community: ~p" - "~n TDomain: ~p" - "~n Ip: ~p" - "~n Port: ~p" - "~n EngineId: ~p" - "~n Timeout: ~p" - "~n MaxMessageSize: ~p" - "~n Version: ~p" - "~n SecModel: ~p" - "~n SecName: ~p" - "~n SecLevel: ~p", - [UserId, TargetName, Community, - TDomain, Ip, Port, - EngineId, Timeout, MaxMessageSize, - Version, SecModel, SecName, SecLevel]), - Addr = normalize_address(TDomain, Ip), - ?vtrace("check_agent_config -> Addr: ~p", [Addr]), - Agent = {UserId, - TargetName, - Community, - TDomain, Addr, Port, - EngineId, - Timeout, MaxMessageSize, - Version, SecModel, SecName, SecLevel}, - {ok, verify_agent(Agent)}; +check_agent_config( + {UserId, TargetName, Community, Domain, Addr, + EngineId, Timeout, MaxMessageSize, + Version, SecModel, SecName, SecLevel}) when is_atom(Domain) -> + check_agent_config( + UserId, TargetName, Community, Domain, Addr, + EngineId, Timeout, MaxMessageSize, + Version, SecModel, SecName, SecLevel); +check_agent_config( + {UserId, TargetName, Community, Ip, Port, + EngineId, Timeout, MaxMessageSize, + Version, SecModel, SecName, SecLevel}) when is_integer(Port) -> + Domain = default_transport_domain(), + Addr = {Ip, Port}, + check_agent_config( + UserId, TargetName, Community, Domain, Addr, + EngineId, Timeout, MaxMessageSize, + Version, SecModel, SecName, SecLevel); +check_agent_config( + {_UserId, _TargetName, _Community, Domain, Addr, + _EngineId, _Timeout, _MaxMessageSize, + _Version, _SecModel, _SecName, _SecLevel}) -> + error({bad_address, {Domain, Addr}}); +check_agent_config( + {UserId, TargetName, Community, Domain, Ip, Port, + EngineId, Timeout, MaxMessageSize, + Version, SecModel, SecName, SecLevel}) -> + Addr = {Ip, Port}, + check_agent_config( + UserId, TargetName, Community, Domain, Addr, + EngineId, Timeout, MaxMessageSize, + Version, SecModel, SecName, SecLevel); check_agent_config(Agent) -> error({bad_agent_config, Agent}). - -init_agents_config([]) -> - ok; -init_agents_config([Agent|Agents]) -> - init_agent_config(Agent), - init_agents_config(Agents). - -init_agent_config({_UserId, ?DEFAULT_TARGETNAME = TargetName, _Config}) -> - throw({error, {invalid_target_name, TargetName}}); -init_agent_config({UserId, TargetName, Config}) -> - case handle_register_agent(UserId, TargetName, Config) of - ok -> - ok; - Error -> - throw(Error) - end. - - -%% For backward compatibility -verify_agent({UserId, - TargetName, - Comm, - Ip, Port, - EngineId, - Timeout, MMS, - Version, SecModel, SecName, SecLevel}) -> - TDomain = default_transport_domain(), - verify_agent({UserId, - TargetName, - Comm, - TDomain, Ip, Port, - EngineId, - Timeout, MMS, - Version, SecModel, SecName, SecLevel}); - -verify_agent({UserId, - TargetName, - Comm, - TDomain, Ip, Port, - EngineId, - Timeout, MMS, - Version, SecModel, SecName, SecLevel}) -> - ?vdebug("verify_agent -> entry with" +check_agent_config( + UserId, TargetName, Comm, Domain, Addr, + EngineId, Timeout, MMS, + Version, SecModel, SecName, SecLevel) -> + ?vdebug("check_agent_config -> entry with" "~n UserId: ~p" "~n TargetName: ~p", [UserId, TargetName]), snmp_conf:check_string(TargetName, {gt, 0}), - snmp_conf:check_integer(Port, {gt, 0}), %% Note that the order of Conf *is* important. %% Some properties may depend on others, so that %% in order to verify one property, another must %% be already verified (and present). An example - %% of this is the property 'address', for which + %% of this is the property 'taddress', for which %% the property tdomain is needed. - Conf0 = + Conf = [{reg_type, target_name}, - {tdomain, TDomain}, - %% This should be taddress, but what the*... - {address, {TDomain, Ip}}, - {port, Port}, + {tdomain, Domain}, + {taddress, fix_address(Domain, Addr)}, {community, Comm}, {engine_id, EngineId}, {timeout, Timeout}, @@ -1813,40 +1765,180 @@ verify_agent({UserId, {sec_name, SecName}, {sec_level, SecLevel} ], - case verify_agent2(Conf0) of - {ok, Conf} -> - {UserId, TargetName, Conf, Version}; - Err -> - throw(Err) + {ok, {UserId, TargetName, verify_agent_config(Conf), Version}}. + + + +init_agents_config([]) -> + ok; +init_agents_config([Agent|Agents]) -> + init_agent_config(Agent), + init_agents_config(Agents). + +init_agent_config({_UserId, ?DEFAULT_TARGETNAME = TargetName, _Config}) -> + error({invalid_target_name, TargetName}); +init_agent_config({UserId, TargetName, Config}) -> + case handle_register_agent(UserId, TargetName, Config) of + ok -> + ok; + Error -> + throw(Error) end. -verify_agent2(Conf) -> - verify_agent2(Conf, []). -verify_agent2([], VerifiedConf) -> - {ok, VerifiedConf}; -verify_agent2([{Item, Val0}|Items], VerifiedConf) -> - case verify_val(Item, Val0) of - {ok, Val} -> - verify_agent2(Items, [{Item, Val} | VerifiedConf]); - Err -> - Err + +%% Sort 'tdomain' first then 'port' to ensure both +%% sorts before 'taddress'. Keep the order of other items. +order_agent(ItemA, ItemB) -> + snmp_conf:keyorder(1, ItemA, ItemB, [tdomain, port]). + +fix_agent_config(Conf) -> + ?vdebug("fix_agent_config -> entry with~n~n" + " Conf: ~p", [Conf]), + fix_agent_config(lists:sort(fun order_agent/2, Conf), []). + +fix_agent_config([], FixedConf) -> + Ret = lists:reverse(FixedConf), + ?vdebug("fix_agent_config -> returns:~n" + " ~p", [Ret]), + Ret; +fix_agent_config([{taddress = Item, Address} = Entry|Conf], FixedConf) -> + {value, {tdomain, TDomain}} = lists:keysearch(tdomain, 1, FixedConf), + {value, {port, DefaultPort}} = lists:keysearch(port, 1, FixedConf), + case snmp_conf:check_address(TDomain, Address, DefaultPort) of + ok -> + fix_agent_config(Conf, [Entry|FixedConf]); + {ok, NAddress} -> + fix_agent_config(Conf, [{Item, NAddress}|FixedConf]) end; -verify_agent2([Bad|_], _VerifiedConf) -> - {error, {bad_agent_config, Bad}}. +fix_agent_config([Entry|Conf], FixedConf) -> + fix_agent_config(Conf, [Entry|FixedConf]). + + + +verify_agent_config(Conf) -> + verify_agent_config(lists:sort(fun order_agent/2, Conf), []). + +verify_agent_config([], VerifiedConf) -> + Ret = lists:reverse(VerifiedConf), + ?vdebug("verify_agent_config -> returns:~n" + " ~p", [Ret]), + Ret; +verify_agent_config([{Item, _} = Entry|Conf], VerifiedConf) -> + verify_illegal(VerifiedConf, [Item]), % Duplicates are hereby illegal + verify_agent_config(Conf, VerifiedConf, Entry); +verify_agent_config([Bad|_], _VerifiedConf) -> + error({bad_agent_config, Bad}). + +verify_agent_config( + Conf, VerifiedConf, {taddress = Item, Address} = Entry) -> + verify_illegal(VerifiedConf, [address]), + {TDomain, VC} = + case lists:keysearch(tdomain, 1, VerifiedConf) of + {value, {tdomain,TD}} -> + {TD, VerifiedConf}; + _ -> + %% Insert tdomain since it is missing + %% Note: not default_transport_domain() since + %% taddress is the new format hence the application + %% should be tdomain aware and therefore addresses + %% on the Domain, Addr format should be used and understood. + TD = transportDomainUdpIpv4, + {TD, [{tdomain, TD}|VerifiedConf]} + end, + case snmp_conf:check_address(TDomain, Address, 0) of + ok -> + verify_agent_config(Conf, [Entry|VC]); + {ok, NAddress} -> + verify_agent_config(Conf, [{Item, NAddress}|VC]) + end; +verify_agent_config(Conf, VerifiedConf, {address, Address}) -> + Item = taddress, + verify_illegal(VerifiedConf, [Item]), + {TDomain, VC} = + case lists:keysearch(tdomain, 1, VerifiedConf) of + {value, {tdomain, TD}} -> + {TD, VerifiedConf}; + _ -> + %% Insert tdomain since it is missing + TD = default_transport_domain(), + {TD, [{tdomain, TD}|VerifiedConf]} + end, + case snmp_conf:check_address(TDomain, Address, 0) of + ok -> + verify_agent_config(Conf, [{Item, Address}|VC]); + {ok, NAddress} -> + verify_agent_config(Conf, [{Item, NAddress}|VC]) + end; +verify_agent_config(Conf, VerifiedConf, {Item, Val} = Entry) -> + case verify_agent_entry(Item, Val) of + ok -> + verify_agent_config(Conf, [Entry|VerifiedConf]); + {ok, NewVal} -> + verify_agent_config(Conf, [{Item, NewVal}|VerifiedConf]) + end. + +verify_agent_entry(user_id, _UserId) -> + ok; +verify_agent_entry(reg_type, RegType) -> + if + RegType =:= addr_port; + RegType =:= target_name -> + ok; + true -> + error({bad_reg_type, RegType}) + end; +verify_agent_entry(tdomain, TDomain) -> + snmp_conf:check_domain(TDomain); +verify_agent_entry(port, Port) -> + snmp_conf:check_port(Port); +verify_agent_entry(community, Comm) -> + snmp_conf:check_string(Comm); +verify_agent_entry(engine_id, EngineId) -> + case EngineId of + discovery -> + ok; + _ -> + snmp_conf:check_string(EngineId) + end; +verify_agent_entry(timeout, Timeout) -> + snmp_conf:check_timer(Timeout); +verify_agent_entry(max_message_size, MMS) -> + snmp_conf:check_packet_size(MMS); +verify_agent_entry(version, V) -> + if + V =:= v1; + V =:= v2; + V =:= v3 -> + ok; + true -> + error({bad_version, V}) + end; +verify_agent_entry(sec_model, Model) -> + snmp_conf:check_sec_model(Model); +verify_agent_entry(sec_name, Name) -> + try snmp_conf:check_string(Name) + catch + _ -> + error({bad_sec_name, Name}) + end; +verify_agent_entry(sec_level, Level) -> + snmp_conf:check_sec_level(Level); +verify_agent_entry(Item, _) -> + error({unknown_item, Item}). + read_users_config_file(Dir) -> - Check = fun(C) -> check_user_config(C) end, - case read_file(Dir, "users.conf", Check, []) of - {ok, Conf} -> - Conf; - Error -> + Order = fun snmp_conf:no_order/2, + Check = fun (User, State) -> {check_user_config(User), State} end, + try read_file(Dir, "users.conf", Order, Check, []) + catch + throw:Error -> ?vlog("failure reading users config file: ~n ~p", [Error]), - throw(Error) + erlang:raise(throw, Error, erlang:get_stacktrace()) end. - check_user_config({Id, Mod, Data}) -> ?vtrace("check_user_config -> entry with" "~n Id: ~p" @@ -1864,21 +1956,18 @@ check_user_config({Id, Mod, Data, DefaultAgentConfig} = _User) case (catch verify_user_behaviour(Mod)) of ok -> ?vtrace("check_user_config -> user behaviour verified", []), - case verify_user_agent_config(DefaultAgentConfig) of - {ok, DefAgentConf} -> - ?vtrace("check_user_config -> " - "user agent (default) config verified", []), - User2 = {Id, Mod, Data, DefAgentConf}, - {ok, User2}; - {error, Reason} -> - error({bad_default_agent_config, Reason}) - end; + DefAgentConf = + verify_default_agent_config(DefaultAgentConfig), + ?vtrace("check_user_config -> " + "user agent (default) config verified", []), + User2 = {Id, Mod, Data, DefAgentConf}, + {ok, User2}; Error -> throw(Error) end; check_user_config({Id, _Mod, _Data, DefaultAgentConfig}) when (Id =/= ?DEFAULT_USER) -> - {error, {bad_default_agent_config, DefaultAgentConfig}}; + error({bad_default_agent_config, DefaultAgentConfig}); check_user_config({Id, _Mod, _Data, _DefaultAgentConfig}) -> error({bad_user_id, Id}); check_user_config(User) -> @@ -1904,7 +1993,7 @@ init_user_config(User) -> error_msg("user config check failed: " "~n~w~n~w", [User, Reason]) end. - + verify_user({Id, UserMod, UserData}) -> verify_user({Id, UserMod, UserData, []}); verify_user({Id, UserMod, UserData, DefaultAgentConfig}) @@ -1917,15 +2006,24 @@ verify_user({Id, UserMod, UserData, DefaultAgentConfig}) [Id, UserMod, UserData, DefaultAgentConfig]), case (catch verify_user_behaviour(UserMod)) of ok -> - case verify_user_agent_config(DefaultAgentConfig) of - {ok, DefAgentConf} -> - Config = default_agent_config(DefAgentConf), - {ok, #user{id = Id, - mod = UserMod, - data = UserData, - default_agent_config = Config}}; - {error, Reason} -> - error({bad_default_agent_config, Reason}) + try + {ok, SystemDefaultAgentConfig} = agent_info(), + Config = + ensure_config( + SystemDefaultAgentConfig, + verify_default_agent_config(DefaultAgentConfig)), +%% Config = +%% default_agent_config( +%% verify_default_agent_config(DefaultAgentConfig)), + {ok, #user{id = Id, + mod = UserMod, + data = UserData, + default_agent_config = Config}} + catch + Error -> + ?vdebug("verify_user default_agent_config -> throw" + "~n Error: ~p", [Error]), + error({bad_default_agent_config, Error}) end; Error -> throw(Error) @@ -1936,27 +2034,24 @@ verify_user({Id, _UserMod, _UserData, DefaultAgentConfig}) verify_user({Id, _, _, _}) -> {error, {bad_user_id, Id}}. -verify_user_agent_config(Conf) -> +verify_default_agent_config(Conf) -> try - begin - verify_invalid(Conf, [user_id, engine_id, address]), - verify_agent_config2(Conf) - end + verify_illegal( + Conf, + [user_id, engine_id, address, tdomain, taddress]), + verify_agent_config(Conf) catch - throw:Error -> - ?vdebug("verify_user_agent_config -> throw" + Error -> + ?vdebug("verify_default_agent_config -> throw" "~n Error: ~p", [Error]), - Error + error({bad_default_agent_config, Error}) end. + read_usm_config_file(Dir) -> - Check = fun(C) -> check_usm_user_config(C) end, - case read_file(Dir, "usm.conf", Check, []) of - {ok, Conf} -> - Conf; - Error -> - throw(Error) - end. + Order = fun snmp_conf:no_order/2, + Check = fun (User, State) -> {check_usm_user_config(User), State} end, + read_file(Dir, "usm.conf", Order, Check, []). %% Identity-function check_usm_user_config({EngineId, Name, @@ -2139,125 +2234,128 @@ is_crypto_supported(Func) -> read_manager_config_file(Dir) -> - Check = fun(Conf) -> check_manager_config(Conf) end, - case read_file(Dir, "manager.conf", Check) of - {ok, Conf} -> - ?d("read_manager_config_file -> ok: " - "~n Conf: ~p", [Conf]), - %% If the address is not specified, then we assume - %% it should be the local host. - %% If the address is not possible to determine - %% that way, then we give up... - check_mandatory_manager_config(Conf), - ensure_manager_config(Conf); - Error -> - throw(Error) - end. + Order = fun order_manager_config/2, + Check = fun check_manager_config/2, + Conf = read_file(Dir, "manager.conf", Order, Check), + ?d("read_manager_config_file -> ok: " + "~n Conf: ~p", [Conf]), + %% If the address is not specified, then we assume + %% it should be the local host. + %% If the address is not possible to determine + %% that way, then we give up... + verify_someof(Conf, [port, transports]), + verify_mandatory(Conf, [engine_id, max_message_size]), + default_manager_config(Conf). + +default_manager_config(Conf) -> + %% Ensure valid transports entry + case lists:keyfind(transports, 1, Conf) of + false -> + {port, Port} = lists:keyfind(port, 1, Conf), + Domain = + case lists:keyfind(domain, 1, Conf) of + false -> + default_transport_domain(); + {_, D} -> + D + end, + Family = snmp_conf:tdomain_to_family(Domain), + {ok, Hostname} = inet:gethostname(), + case inet:getaddr(Hostname, Family) of + {ok, Address} -> + lists:sort( + fun order_manager_config/2, + [{transports, [{Domain, {Address, Port}}]} | Conf]); + {error, _Reason} -> + ?d("default_manager_config -> " + "failed getting ~w address for ~s:~n" + " _Reason: ~p", [Family, Hostname, _Reason]), + Conf + end; + _ -> + Conf + end. + +order_manager_config(EntryA, EntryB) -> + snmp_conf:keyorder(1, EntryA, EntryB, [domain, port]). + +check_manager_config(Entry, undefined) -> + check_manager_config(Entry, {default_transport_domain(), undefined}); +check_manager_config({domain, Domain}, {_, Port}) -> + {snmp_conf:check_domain(Domain), {Domain, Port}}; +check_manager_config({port, Port}, {Domain, _}) -> + {ok = snmp_conf:check_port(Port), {Domain, Port}}; +check_manager_config({address, _}, {_, undefined}) -> + error({missing_mandatory, port}); +check_manager_config({address = Tag, Ip} = Entry, {Domain, Port} = State) -> + {case snmp_conf:check_ip(Domain, Ip) of + ok -> + [Entry, + {transports, [{Domain, {Ip, Port}}]}]; + {ok, FixedIp} -> + [{Tag, FixedIp}, + {transports, [{Domain, {FixedIp, Port}}]}] + end, State}; +check_manager_config({transports = Tag, Transports}, {_, Port} = State) + when is_list(Transports) -> + CheckedTransports = + [case Transport of + {Domain, Address} -> + case + case Port of + undefined -> + snmp_conf:check_address(Domain, Address); + _ -> + snmp_conf:check_address(Domain, Address, Port) + end + of + ok -> + Transport; + {ok, FixedAddress} -> + {Domain, FixedAddress} + end; + _Domain when Port =:= undefined-> + error({missing_mandatory, port}); + Domain -> + Family = snmp_conf:tdomain_to_family(Domain), + {ok, Hostname} = inet:gethostname(), + case inet:getaddr(Hostname, Family) of + {ok, IpAddr} -> + {Domain, {IpAddr, Port}}; + {error, _} -> + error({bad_address, {Domain, Hostname}}) + end + end + || Transport <- Transports], + {{ok, {Tag, CheckedTransports}}, State}; +check_manager_config(Entry, State) -> + {check_manager_config(Entry), State}. -default_manager_config() -> - {ok, HostName} = inet:gethostname(), - case inet:getaddr(HostName, inet) of - {ok, A} -> - [{address, tuple_to_list(A)}]; - {error, _Reason} -> - ?d("default_manager_config -> failed getting address: " - "~n _Reason: ~p", [_Reason]), - [] - end. - -check_manager_config({address, Addr}) -> - snmp_conf:check_ip(Addr); -check_manager_config({port, Port}) -> - snmp_conf:check_integer(Port, {gt, 0}); check_manager_config({engine_id, EngineID}) -> snmp_conf:check_string(EngineID); check_manager_config({max_message_size, Max}) -> snmp_conf:check_integer(Max, {gte, 484}); check_manager_config(Conf) -> - {error, {unknown_config, Conf}}. - - -check_mandatory_manager_config(Conf) -> - Mand = [port, engine_id, max_message_size], - check_mandatory_manager_config(Mand, Conf). - -check_mandatory_manager_config([], _Conf) -> - ok; -check_mandatory_manager_config([Item|Mand], Conf) -> - case lists:keysearch(Item, 1, Conf) of - false -> - error({missing_mandatory_manager_config, Item}); - _ -> - check_mandatory_manager_config(Mand, Conf) - end. - - -ensure_manager_config(Confs) -> - ensure_manager_config(Confs, default_manager_config()). - -ensure_manager_config(Confs, []) -> - Confs; -ensure_manager_config(Confs, [{Key,_} = DefKeyVal|Defs]) -> - case lists:keysearch(Key, 1, Confs) of - false -> - ensure_manager_config([DefKeyVal|Confs], Defs); - {value, _Conf} -> - ensure_manager_config(Confs, Defs) - end. - -% ensure_manager_config([], Defs, Confs) -> -% Confs ++ Defs; -% ensure_manager_config(Confs0, [{Key, DefVal}|Defs], Acc) -> -% case lists:keysearch(Key, 1, Confs0) of -% false -> -% ensure_manager_config(Confs0, Defs, [{Key, DefVal}|Acc]); -% {value, Conf} -> -% Confs = lists:keydelete(Key, 1, Confs0), -% ensure_manager_config(Confs, Defs, [Conf|Acc]) -% end. - + error({unknown_config, Conf}). -read_file(Dir, FileName, Check, Default) -> - File = filename:join(Dir, FileName), - case file:read_file_info(File) of - {ok, _} -> - case (catch do_read(File, Check)) of - {ok, Conf} -> - {ok, Conf}; - Error -> - ?vtrace("read_file -> read failed:" - "~n Error: ~p", [Error]), - Error - end; - {error, Reason} -> +read_file(Dir, FileName, Order, Check, Default) -> + try snmp_conf:read(filename:join(Dir, FileName), Order, Check) + catch + {error, Reason} when element(1, Reason) =:= failed_open -> ?vlog("failed reading config from ~s: ~p", [FileName, Reason]), - {ok, Default} + Default end. -read_file(Dir, FileName, Check) -> - File = filename:join(Dir, FileName), - case file:read_file_info(File) of - {ok, _} -> - case (catch do_read(File, Check)) of - {ok, Conf} -> - ?vtrace("read_file -> read ok" - "~n Conf: ~p", [Conf]), - {ok, Conf}; - Error -> - ?vtrace("read_file -> read failed:" - "~n Error: ~p", [Error]), - Error - end; - {error, Reason} -> +read_file(Dir, FileName, Order, Check) -> + try snmp_conf:read(filename:join(Dir, FileName), Order, Check) + catch + throw:{error, Reason} = Error + when element(1, Reason) =:= failed_open -> error_msg("failed reading config from ~s: ~p", [FileName, Reason]), - {error, {failed_reading, FileName, Reason}} + erlang:raise(throw, Error, erlang:get_stacktrace()) end. -do_read(File, Check) -> - {ok, snmp_conf:read(File, Check)}. - - %%-------------------------------------------------------------------- %% Func: handle_call/3 %% Returns: {reply, Reply, State} | @@ -2684,30 +2782,50 @@ handle_register_agent(UserId, TargetName, Config) -> "~n Config: ~p", [UserId, TargetName, Config]), case (catch agent_info(TargetName, user_id)) of {error, _} -> - ?vtrace("handle_register_agent -> user_id not found in config", []), + ?vtrace( + "handle_register_agent -> user_id not found in config", []), case ets:lookup(snmpm_user_table, UserId) of [#user{default_agent_config = DefConfig}] -> - ?vtrace("handle_register_agent -> " - "~n DefConfig: ~p", [DefConfig]), - %% First, insert this users default config - ?vtrace("handle_register_agent -> store default config", []), - do_handle_register_agent(TargetName, DefConfig), - %% Second, insert the config for this agent - ?vtrace("handle_register_agent -> store config", []), - do_handle_register_agent(TargetName, - [{user_id, UserId}|Config]), + ?vtrace("handle_register_agent ->~n" + " DefConfig: ~p", [DefConfig]), + FixedConfig = + fix_agent_config(ensure_config(DefConfig, Config)), + ?vtrace("handle_register_agent ->~n" + " FixedConfig: ~p", [FixedConfig]), + do_handle_register_agent( + TargetName, [{user_id, UserId}|FixedConfig]), %% <DIRTY-BACKWARD-COMPATIBILLITY> - %% And now for some (backward compatibillity) + %% And now for some (backward compatibillity) %% dirty crossref stuff - ?vtrace("handle_register_agent -> lookup address", []), - {ok, Addr} = agent_info(TargetName, address), - ?vtrace("handle_register_agent -> Addr: ~p, lookup Port", - [Addr]), - {ok, Port} = agent_info(TargetName, port), - ?vtrace("handle_register_agent -> register cross-ref fix", []), - ets:insert(snmpm_agent_table, - {{Addr, Port, target_name}, TargetName}), + {value, {_, Domain}} = + lists:keysearch(tdomain, 1, FixedConfig), + {value, {_, Address}} = + lists:keysearch(taddress, 1, FixedConfig), + ?vtrace( + "handle_register_agent -> register cross-ref fix", []), + ets:insert(snmpm_agent_table, + {{Domain, Address, target_name}, TargetName}), %% </DIRTY-BACKWARD-COMPATIBILLITY> + +%% %% First, insert this users default config +%% ?vtrace("handle_register_agent -> store default config", []), +%% do_handle_register_agent(TargetName, DefConfig), +%% %% Second, insert the config for this agent +%% ?vtrace("handle_register_agent -> store config", []), +%% do_handle_register_agent(TargetName, +%% [{user_id, UserId}|Config]), +%% %% <DIRTY-BACKWARD-COMPATIBILLITY> +%% %% And now for some (backward compatibillity) +%% %% dirty crossref stuff +%% ?vtrace("handle_register_agent -> lookup taddress", []), +%% {ok, {Addr, Port} = TAddress} = +%% agent_info(TargetName, taddress), +%% ?vtrace("handle_register_agent -> taddress: ~p", +%% [TAddress]), +%% ?vtrace("handle_register_agent -> register cross-ref fix", []), +%% ets:insert(snmpm_agent_table, +%% {{Addr, Port, target_name}, TargetName}), +%% %% </DIRTY-BACKWARD-COMPATIBILLITY> ok; _ -> {error, {not_found, UserId}} @@ -2729,7 +2847,7 @@ handle_register_agent(UserId, TargetName, Config) -> do_handle_register_agent(_TargetName, []) -> ok; do_handle_register_agent(TargetName, [{Item, Val}|Rest]) -> - ?vtrace("handle_register_agent -> entry with" + ?vtrace("do_handle_register_agent -> entry with" "~n TargetName: ~p" "~n Item: ~p" "~n Val: ~p" @@ -2738,7 +2856,7 @@ do_handle_register_agent(TargetName, [{Item, Val}|Rest]) -> ok -> do_handle_register_agent(TargetName, Rest); {error, Reason} -> - ?vtrace("handle_register_agent -> failed updating ~p" + ?vtrace("do_handle_register_agent -> failed updating ~p" "~n Item: ~p" "~n Reason: ~p", [Item, Reason]), ets:match_delete(snmpm_agent_table, {TargetName, '_'}), @@ -2762,9 +2880,9 @@ handle_unregister_agent(UserId, TargetName) -> %% <DIRTY-BACKWARD-COMPATIBILLITY> %% And now for some (backward compatibillity) %% dirty crossref stuff - {ok, Addr} = agent_info(TargetName, address), - {ok, Port} = agent_info(TargetName, port), - ets:delete(snmpm_agent_table, {Addr, Port, target_name}), + {ok, Domain} = agent_info(TargetName, tdomain), + {ok, Address} = agent_info(TargetName, taddress), + ets:delete(snmpm_agent_table, {Domain, Address, target_name}), %% </DIRTY-BACKWARD-COMPATIBILLITY> ets:match_delete(snmpm_agent_table, {{TargetName, '_'}, '_'}), ok; @@ -2790,21 +2908,26 @@ handle_update_agent_info(UserId, TargetName, Info) -> Error end. -handle_update_agent_info(TargetName, Info0) -> +handle_update_agent_info(TargetName, Info) -> ?vtrace("handle_update_agent_info -> entry with" "~n TargetName: ~p" - "~n Info0: ~p", [TargetName, Info0]), + "~n Info: ~p", [TargetName, Info]), %% Verify info - try verify_agent_info(TargetName, Info0) of - {ok, Info} -> - do_update_agent_info(TargetName, Info); + try + verify_illegal(Info, [user_id]), + %% If port or domain is part of the info, then use it. + %% If not, lookup what is already stored for + %% this agent and use that. + do_update_agent_info( + TargetName, + fix_agent_config( + verify_agent_config( + ensure_agent_info(TargetName, [port,tdomain], Info)))) + catch Error -> - Error - catch - throw:Error -> Error; T:E -> - {error, {failed_info_verification, Info0, T, E}} + {error, {failed_info_verification, Info, T, E}} end. handle_update_agent_info(UserId, TargetName, Item, Val) -> @@ -2816,6 +2939,9 @@ handle_update_agent_info(UserId, TargetName, Item, Val) -> handle_update_agent_info(TargetName, [{Item, Val}]). do_update_agent_info(TargetName, Info) -> + ?vtrace("do_update_agent_info -> entry with~n" + " TargetName: ~p~n" + " Info: ~p", [TargetName,Info]), InsertItem = fun({Item, Val}) -> ets:insert(snmpm_agent_table, {{TargetName, Item}, Val}) @@ -2997,109 +3123,42 @@ verify_mandatory(Conf, [Mand|Mands]) -> true -> verify_mandatory(Conf, Mands); false -> - throw({error, {missing_mandatory_config, Mand}}) + error({missing_mandatory_config, Mand}) end. -verify_invalid(_, []) -> +verify_illegal(_, []) -> ok; -verify_invalid(Conf, [Inv|Invs]) -> +verify_illegal(Conf, [Inv|Invs]) -> case lists:member(Inv, Conf) of false -> - verify_invalid(Conf, Invs); + verify_illegal(Conf, Invs); true -> - throw({error, {illegal_config, Inv}}) + error({illegal_config, Inv}) end. - -verify_val(user_id, UserId) -> - {ok, UserId}; -verify_val(reg_type, RegType) - when (RegType =:= addr_port) orelse (RegType =:= target_name) -> - {ok, RegType}; -verify_val(tdomain = Item, snmpUDPDomain = _Domain) -> - verify_val(Item, transportDomainUdpIpv4); -verify_val(tdomain, Domain) -> - case lists:member(Domain, ?SUPPORTED_DOMAINS) of +verify_someof(Conf, [Mand|Mands]) -> + case lists:keymember(Mand, 1, Conf) of true -> - {ok, Domain}; + ok; false -> - case lists:member(Domain, snmp_conf:all_domains()) of - true -> - error({unsupported_domain, Domain}); - false -> - error({unknown_domain, Domain}) + case Mands of + [] -> + error({missing_mandatory_config, Mand}); + _ -> + verify_someof(Conf, Mands) end - end; -verify_val(address, {Domain, Addr0}) -> - case normalize_address(Domain, Addr0) of - {_A1, _A2, _A3, _A4} = Addr -> - {ok, Addr}; - {_A1, _A2, _A3, _A4, _A5, _A6, _A7, _A8} = Addr -> - {ok, Addr}; - _ when is_list(Addr0) -> - case (catch snmp_conf:check_ip(Addr0)) of - ok -> - {ok, list_to_tuple(Addr0)}; - Err -> - Err - end; - _ -> - error({bad_address, Addr0}) - end; -verify_val(address, BadAddress) -> - error({bad_address, BadAddress}); -verify_val(port, Port) -> - case (catch snmp_conf:check_integer(Port, {gt, 0})) of - ok -> - {ok, Port}; - Err -> - Err - end; -verify_val(community, Comm) -> - case (catch snmp_conf:check_string(Comm)) of - ok -> - {ok, Comm}; - Err -> - Err - end; -verify_val(engine_id, discovery = EngineId) -> - {ok, EngineId}; -verify_val(engine_id, EngineId) -> - case (catch snmp_conf:check_string(EngineId)) of - ok -> - {ok, EngineId}; - Err -> - Err - end; -verify_val(timeout, Timeout) -> - (catch snmp_conf:check_timer(Timeout)); -verify_val(max_message_size, MMS) -> - case (catch snmp_conf:check_packet_size(MMS)) of - ok -> - {ok, MMS}; - Err -> - Err - end; -verify_val(version, V) - when (V =:= v1) orelse (V =:= v2) orelse (V =:= v3) -> - {ok, V}; -verify_val(version, BadVersion) -> - error({bad_version, BadVersion}); -verify_val(sec_model, Model) -> - (catch snmp_conf:check_sec_model(Model)); -verify_val(sec_name, Name) when is_list(Name) -> - case (catch snmp_conf:check_string(Name)) of - ok -> - {ok, Name}; - Err -> - Err - end; -verify_val(sec_name, BadName) -> - error({bad_sec_name, BadName}); -verify_val(sec_level, Level) -> - (catch snmp_conf:check_sec_level(Level)); -verify_val(Item, _) -> - {error, {unknown_item, Item}}. + end. + +ensure_config([], Config) -> + Config; +ensure_config([Default|Defaults], Config) -> + case lists:keymember(element(1, Default), 1, Config) of + true -> + ensure_config(Defaults, Config); + false -> + ensure_config(Defaults, [Default|Config]) + end. + %%%------------------------------------------------------------------- @@ -3257,31 +3316,14 @@ init_mini_mib_elems(MibName, [_|T], Res) -> %%---------------------------------------------------------------------- -normalize_address(Addr) -> - normalize_address(snmpUDPDomain, Addr). - -normalize_address(snmpUDPDomain, Addr) -> - normalize_address(transportDomainUdpIpv4, Addr); - -normalize_address(Domain, Addr) -> - case inet:getaddr(Addr, td2fam(Domain)) of - {ok, Addr2} -> - Addr2; - _ when is_list(Addr) -> - case (catch snmp_conf:check_ip(Domain, Addr)) of - ok -> - list_to_tuple(Addr); - _ -> - Addr - end; - _ -> - Addr +fix_address(Domain, Address) -> + case snmp_conf:check_address(Domain, Address) of + ok -> + Address; + {ok, NAddress} -> + NAddress end. -td2fam(transportDomainUdpIpv4) -> inet; -td2fam(transportDomainUdpIpv6) -> inet6. - - %%---------------------------------------------------------------------- call(Req) -> @@ -3391,4 +3433,3 @@ error_msg(F, A) -> %% p(F, A) -> %% io:format("~w:" ++ F ++ "~n", [?MODULE | A]). - diff --git a/lib/snmp/src/manager/snmpm_mpd.erl b/lib/snmp/src/manager/snmpm_mpd.erl index 953c94ab54..f8a7441c0a 100644 --- a/lib/snmp/src/manager/snmpm_mpd.erl +++ b/lib/snmp/src/manager/snmpm_mpd.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2012. All Rights Reserved. +%% Copyright Ericsson AB 2004-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -21,7 +21,7 @@ -export([init/1, - process_msg/7, + process_msg/7, process_msg/6, generate_msg/5, generate_response_msg/4, next_msg_id/0, @@ -92,8 +92,10 @@ reset(#state{v3 = V3}) -> %% Purpose: This is the main Message Dispatching function. (see %% section 4.2.1 in rfc2272) %%----------------------------------------------------------------- -process_msg(Msg, Domain, Addr, Port, State, NoteStore, Logger) -> +process_msg(Msg, Domain, Ip, Port, State, NoteStore, Logger) -> + process_msg(Msg, Domain, {Ip, Port}, State, NoteStore, Logger). +process_msg(Msg, Domain, Addr, State, NoteStore, Logger) -> inc(snmpInPkts), case (catch snmp_pdus:dec_message_only(binary_to_list(Msg))) of @@ -102,17 +104,17 @@ process_msg(Msg, Domain, Addr, Port, State, NoteStore, Logger) -> #message{version = 'version-1', vsn_hdr = Community, data = Data} when State#state.v1 =:= true -> HS = ?empty_msg_size + length(Community), - process_v1_v2c_msg('version-1', NoteStore, Msg, - Domain, Addr, Port, - Community, Data, HS, Logger); + process_v1_v2c_msg( + 'version-1', NoteStore, Msg, Domain, Addr, + Community, Data, HS, Logger); %% Version 2 #message{version = 'version-2', vsn_hdr = Community, data = Data} when State#state.v2c =:= true -> HS = ?empty_msg_size + length(Community), - process_v1_v2c_msg('version-2', NoteStore, Msg, - Domain, Addr, Port, - Community, Data, HS, Logger); + process_v1_v2c_msg( + 'version-2', NoteStore, Msg, Domain, Addr, + Community, Data, HS, Logger); %% Version 3 #message{version = 'version-3', vsn_hdr = H, data = Data} @@ -122,7 +124,7 @@ process_msg(Msg, Domain, Addr, Port, State, NoteStore, Logger) -> "~n msgFlags: ~p" "~n msgSecModel: ~p", [H#v3_hdr.msgID,H#v3_hdr.msgFlags,H#v3_hdr.msgSecurityModel]), - process_v3_msg(NoteStore, Msg, H, Data, Addr, Port, Logger); + process_v3_msg(NoteStore, Msg, H, Data, Addr, Logger); %% Crap {'EXIT', {bad_version, Vsn}} -> @@ -148,32 +150,27 @@ process_msg(Msg, Domain, Addr, Port, State, NoteStore, Logger) -> %%----------------------------------------------------------------- %% Handles a Community based message (v1 or v2c). %%----------------------------------------------------------------- -process_v1_v2c_msg(Vsn, _NoteStore, Msg, Domain, - Addr, Port, - Community, Data, HS, Log) -> +process_v1_v2c_msg( + Vsn, _NoteStore, Msg, Domain, Addr, Community, Data, HS, Log) -> ?vdebug("process_v1_v2c_msg -> entry with" "~n Vsn: ~p" "~n Domain: ~p" "~n Addr: ~p" - "~n Port: ~p" "~n Community: ~p" - "~n HS: ~p", [Vsn, Domain, Addr, Port, Community, HS]), - + "~n HS: ~p", [Vsn, Domain, Addr, Community, HS]), + {TDomain, TAddress} = try - begin - TD = snmp_conf:mk_tdomain(Domain), - TA = snmp_conf:mk_taddress(Domain, Addr, Port), - {TD, TA} - end + {snmp_conf:mk_tdomain(Domain), + snmp_conf:mk_taddress(Domain, Addr)} catch throw:{error, TReason} -> throw({discarded, {badarg, Domain, TReason}}) end, Max = get_max_message_size(), - AgentMax = get_agent_max_message_size(Addr, Port), + AgentMax = get_agent_max_message_size(Domain, Addr), PduMS = pdu_ms(Max, AgentMax, HS), ?vtrace("process_v1_v2c_msg -> PduMS: ~p", [PduMS]), @@ -213,13 +210,12 @@ sec_model('version-2') -> ?SEC_V2C. %% Handles a SNMPv3 Message, following the procedures in rfc2272, %% section 4.2 and 7.2 %%----------------------------------------------------------------- -process_v3_msg(NoteStore, Msg, Hdr, Data, Addr, Port, Log) -> +process_v3_msg(NoteStore, Msg, Hdr, Data, Address, Log) -> + ?vdebug( + "process_v3_msg -> entry with~n" + " Hdr: ~p~n" + " Address: ~p", [Hdr, Address]), - ?vdebug("process_v3_msg -> entry with" - "~n Hdr: ~p" - "~n Addr: ~p" - "~n Port: ~p", [Hdr, Addr, Port]), - %% 7.2.3 #v3_hdr{msgID = MsgID, msgMaxSize = MMS, @@ -352,8 +348,8 @@ process_v3_msg(NoteStore, Msg, Hdr, Data, Addr, Port, Log) -> %% 4.2.2.1.1 - we don't handle proxys yet => we only %% handle CtxEngineID to ourselves %% Check that we actually know of an agent with this - %% CtxEngineID and Addr/Port - case is_known_engine_id(CtxEngineID, Addr, Port) of + %% CtxEngineID and Address + case is_known_engine_id(CtxEngineID, Address) of true -> ?vtrace("and the agent EngineID (~p) " "is know to us", [CtxEngineID]), @@ -866,14 +862,23 @@ get_max_message_size() -> end. %% The the MMS of the agent -get_agent_max_message_size(Addr, Port) -> - case snmpm_config:get_agent_engine_max_message_size(Addr, Port) of +get_agent_max_message_size(Domain, Addr) -> + case snmpm_config:get_agent_engine_max_message_size(Domain, Addr) of {ok, MMS} -> MMS; _Error -> - ?vlog("unknown agent: ~w:~w", [Addr, Port]), + ?vlog("unknown agent: ~s", + [snmp_conf:mk_addr_string({Domain, Addr})]), get_max_message_size() end. +%% get_agent_max_message_size(Addr, Port) -> +%% case snmpm_config:get_agent_engine_max_message_size(Addr, Port) of +%% {ok, MMS} -> +%% MMS; +%% _Error -> +%% ?vlog("unknown agent: ~w:~w", [Addr, Port]), +%% get_max_message_size() +%% end. %% Get "our" (manager) engine id get_engine_id() -> @@ -888,9 +893,12 @@ get_engine_id() -> get_agent_engine_id(Name) -> snmpm_config:get_agent_engine_id(Name). -is_known_engine_id(EngineID, Addr, Port) -> +is_known_engine_id(EngineID, {Addr, Port}) -> snmpm_config:is_known_engine_id(EngineID, Addr, Port). +%% is_known_engine_id(EngineID, Addr, Port) -> +%% snmpm_config:is_known_engine_id(EngineID, Addr, Port). + % get_agent_engine_id(Addr, Port) -> % case snmpm_config:get_agent_engine_id(Addr, Port) of % {ok, Id} -> diff --git a/lib/snmp/src/manager/snmpm_net_if.erl b/lib/snmp/src/manager/snmpm_net_if.erl index 4d6bd9aa33..cb72871177 100644 --- a/lib/snmp/src/manager/snmpm_net_if.erl +++ b/lib/snmp/src/manager/snmpm_net_if.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2011. All Rights Reserved. +%% Copyright Ericsson AB 2004-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -17,7 +17,9 @@ %% %CopyrightEnd% %% +-ifndef(snmpm_net_if_mt). -module(snmpm_net_if). +-endif. -behaviour(gen_server). -behaviour(snmpm_network_interface). @@ -27,9 +29,9 @@ -export([ start_link/2, stop/1, - send_pdu/6, % Backward compatibillity - send_pdu/7, % Backward compatibillity - send_pdu/8, + send_pdu/6, % Backward compatibility + send_pdu/7, % Partly backward compatibility + send_pdu/8, % Backward compatibility inform_response/4, @@ -55,11 +57,11 @@ %% -define(VMODULE,"NET_IF"). -include("snmp_verbosity.hrl"). --record(state, +-record(state, { server, note_store, - sock, + transports = [], mpd_state, log, irb = auto, % auto | {user, integer()} @@ -67,6 +69,9 @@ filter }). +-record(transport, + {socket, + domain = snmpUDPDomain}). -define(DEFAULT_FILTER_MODULE, snmpm_net_if_filter). -define(DEFAULT_FILTER_OPTS, [{module, ?DEFAULT_FILTER_MODULE}]). @@ -99,30 +104,32 @@ start_link(Server, NoteStore) -> stop(Pid) -> call(Pid, stop). -send_pdu(Pid, Pdu, Vsn, MsgData, Addr, Port) -> - send_pdu(Pid, Pdu, Vsn, MsgData, Addr, Port, ?DEFAULT_EXTRA_INFO). - -send_pdu(Pid, Pdu, Vsn, MsgData, Addr, Port, ExtraInfo) -> - Domain = snmpm_config:default_transport_domain(), - send_pdu(Pid, Pdu, Vsn, MsgData, Domain, Addr, Port, ExtraInfo). +send_pdu(Pid, Pdu, Vsn, MsgData, Domain_or_Ip, Addr_or_Port) -> + send_pdu( + Pid, Pdu, Vsn, MsgData, Domain_or_Ip, Addr_or_Port, ?DEFAULT_EXTRA_INFO). -send_pdu(Pid, Pdu, Vsn, MsgData, Domain, Addr, Port, ExtraInfo) +send_pdu(Pid, Pdu, Vsn, MsgData, Domain_or_Ip, Addr_or_Port, ExtraInfo) when is_record(Pdu, pdu) -> - ?d("send_pdu -> entry with" - "~n Pid: ~p" - "~n Pdu: ~p" - "~n Vsn: ~p" - "~n MsgData: ~p" - "~n Domain: ~p" - "~n Addr: ~p" - "~n Port: ~p", [Pid, Pdu, Vsn, MsgData, Domain, Addr, Port]), - cast(Pid, {send_pdu, Pdu, Vsn, MsgData, Domain, Addr, Port, ExtraInfo}). + ?d("send_pdu -> entry with~n" + " Pid: ~p~n" + " Pdu: ~p~n" + " Vsn: ~p~n" + " MsgData: ~p~n" + " Domain/IP: ~p~n" + " Addr/Port: ~p", + [Pid, Pdu, Vsn, MsgData, Domain_or_Ip, Addr_or_Port]), + {Domain, Addr} = address(Domain_or_Ip, Addr_or_Port), + cast(Pid, {send_pdu, Pdu, Vsn, MsgData, Domain, Addr, ExtraInfo}). + +send_pdu(Pid, Pdu, Vsn, MsgData, Domain, Ip, Port, ExtraInfo) -> + send_pdu(Pid, Pdu, Vsn, MsgData, Domain, {Ip, Port}, ExtraInfo). note_store(Pid, NoteStore) -> call(Pid, {note_store, NoteStore}). -inform_response(Pid, Ref, Addr, Port) -> - cast(Pid, {inform_response, Ref, Addr, Port}). +inform_response(Pid, Ref, Domain_or_Ip, Addr_or_Port) -> + {Domain, Addr} = address(Domain_or_Ip, Addr_or_Port), + cast(Pid, {inform_response, Ref, Domain, Addr}). info(Pid) -> call(Pid, info). @@ -144,6 +151,64 @@ filter_reset(Pid) -> %%%------------------------------------------------------------------- +%%% Multi-thread manager +%%%------------------------------------------------------------------- + +-ifdef(snmpm_net_if_mt). + +%% This function is called through the macro below to +%% (in the not multithreaded case) avoid creating the +%% Failer/4 fun, and to avoid calling the Worker through a fun +%% (now it shall not be a fun, just a code snippet). + +worker(Worker, Failer, #state{log = Log} = State) -> + Verbosity = get(verbosity), + spawn_opt( + fun () -> + try + put(sname, mnifw), + put(verbosity, Verbosity), + NewState = + case do_reopen_log(Log) of + Log -> + State; + NewLog -> + State#state{log = NewLog} + end, + Worker(NewState) + of + Result -> + %% Winds up in handle_info {'DOWN', ...} + erlang:exit({net_if_worker, Result}) + catch + Class:Reason -> + %% Winds up in handle_info {'DOWN', ...} + erlang:exit( + {net_if_worker, Failer, + Class, Reason, erlang:get_stacktrace()}) + end + end, + [monitor]). +-define( + worker(S, Worker, Failer, State), + begin + worker( + fun (S) -> begin Worker end end, + begin Failer end, + (State)) + end). + +-else. + +-define( + worker(S, Worker, _Failer, State), + begin (S) = (State), begin Worker end end). + +-endif. + + + +%%%------------------------------------------------------------------- %%% Callback functions from gen_server %%%------------------------------------------------------------------- @@ -158,12 +223,19 @@ init([Server, NoteStore]) -> ?d("init -> entry with" "~n Server: ~p" "~n NoteStore: ~p", [Server, NoteStore]), - case (catch do_init(Server, NoteStore)) of + try do_init(Server, NoteStore) + catch {error, Reason} -> - {stop, Reason}; - {ok, State} -> - {ok, State} + {stop, Reason} end. + +-ifdef(snmpm_net_if_mt). +%% This should really be protected, but it also needs to +%% be writable for the worker processes, so... +-define(inform_table_opts, [set, public, named_table, {keypos, 1}]). +-else. +-define(inform_table_opts, [set, protected, named_table, {keypos, 1}]). +-endif. do_init(Server, NoteStore) -> process_flag(trap_exit, true), @@ -173,18 +245,18 @@ do_init(Server, NoteStore) -> process_flag(priority, Prio), %% -- Create inform request table -- - ets:new(snmpm_inform_request_table, - [set, protected, named_table, {keypos, 1}]), + ets:new(snmpm_inform_request_table, ?inform_table_opts), %% -- Verbosity -- {ok, Verbosity} = snmpm_config:system_info(net_if_verbosity), - put(sname,mnif), - put(verbosity,Verbosity), + put(sname, mnif), + put(verbosity, Verbosity), ?vlog("starting", []), %% -- MPD -- {ok, Vsns} = snmpm_config:system_info(versions), MpdState = snmpm_mpd:init(Vsns), + ?vdebug("MpdState: ~w", [MpdState]), %% -- Module dependent options -- {ok, Opts} = snmpm_config:system_info(net_if_options), @@ -193,14 +265,6 @@ do_init(Server, NoteStore) -> {ok, IRB} = snmpm_config:system_info(net_if_irb), IrGcRef = irgc_start(IRB), - %% -- Socket -- - SndBuf = get_opt(Opts, sndbuf, default), - RecBuf = get_opt(Opts, recbuf, default), - BindTo = get_opt(Opts, bind_to, false), - NoReuse = get_opt(Opts, no_reuse, false), - {ok, Port} = snmpm_config:system_info(port), - {ok, Sock} = do_open_port(Port, SndBuf, RecBuf, BindTo, NoReuse), - %% Flow control -- FilterOpts = get_opt(Opts, filter, []), FilterMod = create_filter(FilterOpts), @@ -209,77 +273,104 @@ do_init(Server, NoteStore) -> %% -- Audit trail log --- {ok, ATL} = snmpm_config:system_info(audit_trail_log), Log = do_init_log(ATL), + ?vdebug("Log: ~w", [Log]), + + {ok, DomainAddresses} = snmpm_config:system_info(transports), + ?vdebug("DomainAddresses: ~w",[DomainAddresses]), + CommonSocketOpts = common_socket_opts(Opts), + BindTo = get_opt(Opts, bind_to, false), + case + [begin + {IpPort, SocketOpts} = + socket_params(Domain, Address, BindTo, CommonSocketOpts), + Socket = socket_open(IpPort, SocketOpts), + #transport{socket = Socket, domain = Domain} + end || {Domain, Address} <- DomainAddresses] + of + [] -> + ?vinfo("No transports configured: ~p", [DomainAddresses]), + throw({error, {no_transports,DomainAddresses}}); + Transports -> + %% -- Initiate counters --- + init_counters(), + + %% -- We are done --- + State = #state{ + server = Server, + note_store = NoteStore, + mpd_state = MpdState, + transports = Transports, + log = Log, + irb = IRB, + irgc = IrGcRef, + filter = FilterMod}, + ?vdebug("started", []), + {ok, State} + end. - %% -- Initiate counters --- - init_counters(), - - %% -- We are done --- - State = #state{server = Server, - note_store = NoteStore, - mpd_state = MpdState, - sock = Sock, - log = Log, - irb = IRB, - irgc = IrGcRef, - filter = FilterMod}, - ?vdebug("started", []), - {ok, State}. - - -%% Open port -do_open_port(Port, SendSz, RecvSz, BindTo, NoReuse) -> - ?vtrace("do_open_port -> entry with" - "~n Port: ~p" - "~n SendSz: ~p" - "~n RecvSz: ~p" - "~n BindTo: ~p" - "~n NoReuse: ~p", [Port, SendSz, RecvSz, BindTo, NoReuse]), - IpOpts1 = bind_to(BindTo), - IpOpts2 = no_reuse(NoReuse), - IpOpts3 = recbuf(RecvSz), - IpOpts4 = sndbuf(SendSz), - IpOpts = [binary | IpOpts1 ++ IpOpts2 ++ IpOpts3 ++ IpOpts4], - OpenRes = - case init:get_argument(snmpm_fd) of - {ok, [[FdStr]]} -> - Fd = list_to_integer(FdStr), - gen_udp:open(0, [{fd, Fd}|IpOpts]); - error -> - gen_udp:open(Port, IpOpts) - end, - case OpenRes of +socket_open(IpPort, SocketOpts) -> + ?vtrace("socket_open -> entry with~n" + " IpPort: ~p~n" + " SocketOpts: ~p", [IpPort, SocketOpts]), + case gen_udp:open(IpPort, SocketOpts) of {error, _} = Error -> throw(Error); - OK -> - OK + {ok, Socket} -> + Socket end. -bind_to(true) -> - case snmpm_config:system_info(address) of - {ok, Addr} when is_list(Addr) -> - [{ip, list_to_tuple(Addr)}]; - {ok, Addr} -> - [{ip, Addr}]; +socket_params(Domain, {IpAddr, IpPort}, BindTo, CommonSocketOpts) -> + Family = snmp_conf:tdomain_to_family(Domain), + SocketOpts = + case Family of + inet6 -> + [Family, {ipv6_v6only, true} | CommonSocketOpts]; + Family -> + [Family | CommonSocketOpts] + end, + case Family of + inet -> + case init:get_argument(snmp_fd) of + {ok, [[FdStr]]} -> + Fd = list_to_integer(FdStr), + case BindTo of + true -> + {IpPort, [{ip, IpAddr}, {fd, Fd} | SocketOpts]}; + _ -> + {0, [{fd, Fd} | SocketOpts]} + end; + error -> + {IpPort, [{ip, IpAddr} | SocketOpts]} + end; _ -> - [] - end; -bind_to(_) -> - []. - -no_reuse(false) -> - [{reuseaddr, true}]; -no_reuse(_) -> - []. - -recbuf(default) -> - []; -recbuf(Sz) -> - [{recbuf, Sz}]. + case BindTo of + true -> + {IpPort, [{ip, IpAddr} | SocketOpts]}; + _ -> + {IpPort, SocketOpts} + end + end. -sndbuf(default) -> - []; -sndbuf(Sz) -> - [{sndbuf, Sz}]. +common_socket_opts(Opts) -> + [binary + | case get_opt(Opts, sndbuf, default) of + default -> + []; + Sz -> + [{sndbuf, Sz}] + end ++ + case get_opt(Opts, recbuf, default) of + default -> + []; + Sz -> + [{sndbuf, Sz}] + end ++ + case get_opt(Opts, no_reuse, false) of + false -> + [{reuseaddr, true}]; + _ -> + [] + end]. create_filter(Opts) when is_list(Opts) -> @@ -294,6 +385,10 @@ create_filter(BadOpts) -> throw({error, {bad_filter_opts, BadOpts}}). +%% ---------------------------------------------------------------------- +%% Audit Trail Logger +%% ---------------------------------------------------------------------- + %% Open log do_init_log(false) -> ?vtrace("do_init_log(false) -> entry", []), @@ -314,24 +409,69 @@ do_init_log(true) -> Function = increment_counter, Args = [atl_seqno, Initial, Max], SeqNoGen = {Module, Function, Args}, - case snmp_log:create(Name, File, - SeqNoGen, Size, Repair, true) of + case snmp_log:create( + Name, File, SeqNoGen, Size, Repair, true) of {ok, Log} -> ?vdebug("log created: ~w", [Log]), - {Log, Type}; + {Name, Log, Type}; {error, Reason} -> throw({error, {failed_create_audit_log, Reason}}) end; _ -> case snmp_log:create(Name, File, Size, Repair, true) of {ok, Log} -> - {Log, Type}; + ?vdebug("log created: ~w", [Log]), + {Name, Log, Type}; {error, Reason} -> throw({error, {failed_create_audit_log, Reason}}) end end. - +-ifdef(snmpm_net_if_mt). +do_reopen_log(undefined) -> + undefined; +do_reopen_log({Name, Log, Type}) -> + case snmp_log:open(Name, Log) of + {ok, NewLog} -> + {Name, NewLog, Type}; + {error, Reason} -> + warning_msg( + "NetIf worker ~p failed to open ATL:~n" + " ~p", [self(), Reason]), + undefined + end. +-endif. + +%% Close log +do_close_log(undefined) -> + ok; +do_close_log({_Name, Log, _Type}) -> + (catch snmp_log:sync(Log)), + (catch snmp_log:close(Log)), + ok; +do_close_log(_) -> + ok. + +%% Log +logger(undefined, _Type, _Domain, _Addr) -> + fun(_) -> + ok + end; +logger({_Name, Log, Types}, Type, Domain, Addr) -> + case lists:member(Type, Types) of + true -> + AddrString = + iolist_to_binary(snmp_conf:mk_addr_string({Domain, Addr})), + fun(Msg) -> + snmp_log:log(Log, Msg, AddrString) + end; + false -> + fun(_) -> + ok + end + end. + + %%-------------------------------------------------------------------- %% Func: handle_call/3 %% Returns: {reply, Reply, State} | @@ -386,25 +526,24 @@ handle_call(Req, From, State) -> %% {noreply, State, Timeout} | %% {stop, Reason, State} (terminate/2 is called) %%-------------------------------------------------------------------- -handle_cast({send_pdu, Pdu, Vsn, MsgData, Domain, Addr, Port, ExtraInfo}, +handle_cast({send_pdu, Pdu, Vsn, MsgData, Domain, Addr, ExtraInfo}, State) -> - ?vlog("received send_pdu message with" - "~n Pdu: ~p" - "~n Vsn: ~p" - "~n MsgData: ~p" - "~n Domain: ~p" - "~n Addr: ~p" - "~n Port: ~p", [Pdu, Vsn, MsgData, Domain, Addr, Port]), - maybe_process_extra_info(ExtraInfo), - maybe_handle_send_pdu(Pdu, Vsn, MsgData, Domain, Addr, Port, State), + ?vlog("received send_pdu message with~n" + " Pdu: ~p~n" + " Vsn: ~p~n" + " MsgData: ~p~n" + " Domain: ~p~n" + " Addr: ~p", [Pdu, Vsn, MsgData, Domain, Addr]), + maybe_process_extra_info(ExtraInfo), + maybe_handle_send_pdu(Pdu, Vsn, MsgData, Domain, Addr, State), {noreply, State}; -handle_cast({inform_response, Ref, Addr, Port}, State) -> - ?vlog("received inform_response message with" - "~n Ref: ~p" - "~n Addr: ~p" - "~n Port: ~p", [Ref, Addr, Port]), - handle_inform_response(Ref, Addr, Port, State), +handle_cast({inform_response, Ref, Domain, Addr}, State) -> + ?vlog("received inform_response message with~n" + " Ref: ~p~n" + " Domain: ~p~n" + " Addr: ~p", [Ref, Domain, Addr]), + handle_inform_response(Ref, Domain, Addr, State), {noreply, State}; handle_cast(filter_reset, State) -> @@ -423,10 +562,21 @@ handle_cast(Msg, State) -> %% {noreply, State, Timeout} | %% {stop, Reason, State} (terminate/2 is called) %%-------------------------------------------------------------------- -handle_info({udp, Sock, Ip, Port, Bytes}, #state{sock = Sock} = State) -> - ?vlog("received ~w bytes from ~p:~p [~w]", [size(Bytes), Ip, Port, Sock]), - maybe_handle_recv_msg(Ip, Port, Bytes, State), - {noreply, State}; +handle_info( + {udp, Socket, IpAddr, IpPort, Bytes}, + #state{transports = Transports} = State) -> + Size = byte_size(Bytes), + case lists:keyfind(Socket, #transport.socket, Transports) of + #transport{socket = Socket, domain = Domain} -> + ?vlog("received ~w bytes from ~p:~p [~w]", + [Size, IpAddr, IpPort, Socket]), + maybe_handle_recv_msg(Domain, {IpAddr, IpPort}, Bytes, State), + {noreply, State}; + false -> + warning_msg("Received ~w bytes on unknown port: ~p from ~s", + [Size, Socket, format_address({IpAddr, IpPort})]), + {noreply, State} + end; handle_info(inform_response_gc, State) -> ?vlog("received inform_response_gc message", []), @@ -439,11 +589,42 @@ handle_info({disk_log, _Node, Log, Info}, State) -> State2 = handle_disk_log(Log, Info, State), {noreply, State2}; +handle_info({'DOWN', _, _, _, _} = Info, State) -> + handle_info_down(Info, State); + handle_info(Info, State) -> + handle_info_unknown(Info, State). + + +handle_info_unknown(Info, State) -> warning_msg("received unknown info: ~n~p", [Info]), {noreply, State}. +-ifdef(snmpm_net_if_mt). +handle_info_down( + {'DOWN', _MRef, process, _Pid, + {net_if_worker, _Result}}, + State) -> + ?vdebug("received DOWN message from net_if worker [~w]: " + "~n Result: ~p", [_Pid, _Result]), + {noreply, State}; +handle_info_down( + {'DOWN', _MRef, process, Pid, + {net_if_worker, Failer, Class, Reason, Stacktrace} = _ExitStatus}, + State) -> + ?vdebug("received DOWN message from net_if worker [~w]: " + "~n ExitStatus: ~p", [Pid, _ExitStatus]), + Failer(Pid, Class, Reason, Stacktrace), + {noreply, State}; +handle_info_down(Info, State) -> + handle_info_unknown(Info, State). +-else. +handle_info_down(Info, State) -> + handle_info_unknown(Info, State). +-endif. + + %%-------------------------------------------------------------------- %% Func: terminate/2 %% Purpose: Shutdown the server @@ -457,68 +638,12 @@ terminate(Reason, #state{log = Log, irgc = IrGcRef}) -> ok. -do_close_log({Log, _Type}) -> - (catch snmp_log:sync(Log)), - (catch snmp_log:close(Log)), - ok; -do_close_log(_) -> - ok. - - %%---------------------------------------------------------------------- %% Func: code_change/3 %% Purpose: Convert process state when code is changed %% Returns: {ok, NewState} %%---------------------------------------------------------------------- -code_change({down, _Vsn}, OldState, downgrade_to_pre_4_14) -> - ?d("code_change(down, downgrade_to_pre_4_14) -> entry with" - "~n OldState: ~p", [OldState]), - #state{server = Server, - note_store = NoteStore, - sock = Sock, - mpd_state = MpdState, - log = {OldLog, Type}, - irb = IRB, - irgc = IRGC} = OldState, - NewLog = snmp_log:downgrade(OldLog), - State = - {state, Server, NoteStore, Sock, MpdState, {NewLog, Type}, IRB, IRGC}, - {ok, State}; - -code_change({down, _Vsn}, OldState, downgrade_to_pre_4_16) -> - ?d("code_change(down, downgrade_to_pre_4_16) -> entry with" - "~n OldState: ~p", [OldState]), - {OldLog, Type} = OldState#state.log, - NewLog = snmp_log:downgrade(OldLog), - State = OldState#state{log = {NewLog, Type}}, - {ok, State}; - -% upgrade -code_change(_Vsn, OldState, upgrade_from_pre_4_14) -> - ?d("code_change(up, upgrade_from_pre_4_14) -> entry with" - "~n OldState: ~p", [OldState]), - {state, Server, NoteStore, Sock, MpdState, {OldLog, Type}, IRB, IRGC} = - OldState, - NewLog = snmp_log:upgrade(OldLog), - State = #state{server = Server, - note_store = NoteStore, - sock = Sock, - mpd_state = MpdState, - log = {NewLog, Type}, - irb = IRB, - irgc = IRGC, - filter = ?DEFAULT_FILTER_MODULE}, - {ok, State}; - -code_change(_Vsn, OldState, upgrade_from_pre_4_16) -> - ?d("code_change(up, upgrade_from_pre_4_16) -> entry with" - "~n OldState: ~p", [OldState]), - {OldLog, Type} = OldState#state.log, - NewLog = snmp_log:upgrade(OldLog), - State = OldState#state{log = {NewLog, Type}}, - {ok, State}; - code_change(_Vsn, State, _Extra) -> ?d("code_change -> entry with" "~n Vsn: ~p" @@ -531,139 +656,155 @@ code_change(_Vsn, State, _Extra) -> %%% Internal functions %%%------------------------------------------------------------------- -maybe_handle_recv_msg(Addr, Port, Bytes, #state{filter = FilterMod} = State) -> - case (catch FilterMod:accept_recv(Addr, Port)) of +maybe_handle_recv_msg(Domain, Addr, Bytes, State) -> + ?worker( + S, maybe_handle_recv_msg_mt(Domain, Addr, Bytes, S), + fun (Pid, Class, Reason, Stacktrace) -> + warning_msg( + "Worker process (~p) terminated " + "while processing (incomming) message from %s:~n" + "~w:~w at ~p", + [Pid, snmp_conf:mk_addr_string({Domain, Addr}), + Class, Reason, Stacktrace]) + end, + State). + +maybe_handle_recv_msg_mt( + Domain, Addr, Bytes, + #state{filter = FilterMod, transports = Transports} = State) -> + {Arg1, Arg2} = fix_filter_address(Transports, {Domain, Addr}), + case (catch FilterMod:accept_recv(Arg1, Arg2)) of false -> %% Drop the received packet - inc(netIfMsgInDrops), - ok; + inc(netIfMsgInDrops); _ -> - handle_recv_msg(Addr, Port, Bytes, State) - end. + handle_recv_msg(Domain, Addr, Bytes, State) + end, + ok. -handle_recv_msg(Addr, Port, Bytes, #state{server = Pid}) +handle_recv_msg(Domain, Addr, Bytes, #state{server = Pid}) when is_binary(Bytes) andalso (size(Bytes) =:= 0) -> - Pid ! {snmp_error, {empty_message, Addr, Port}, Addr, Port}, - ok; - -handle_recv_msg(Addr, Port, Bytes, - #state{server = Pid, - note_store = NoteStore, - mpd_state = MpdState, - sock = Sock, - log = Log} = State) -> - Domain = snmp_conf:which_domain(Addr), % What the ****... - Logger = logger(Log, read, Addr, Port), - case (catch snmpm_mpd:process_msg(Bytes, Domain, Addr, Port, - MpdState, NoteStore, Logger)) of + Pid ! {snmp_error, {empty_message, Domain, Addr}, Domain, Addr}; +%% +handle_recv_msg( + Domain, Addr, Bytes, + #state{ + server = Pid, + note_store = NoteStore, + mpd_state = MpdState, + log = Log} = State) -> + Logger = logger(Log, read, Domain, Addr), + case (catch snmpm_mpd:process_msg( + Bytes, Domain, Addr, MpdState, NoteStore, Logger)) of {ok, Vsn, Pdu, MS, ACM} -> - maybe_handle_recv_pdu(Addr, Port, Vsn, Pdu, MS, ACM, - Logger, State); + maybe_handle_recv_pdu( + Domain, Addr, Vsn, Pdu, MS, ACM, Logger, State); {discarded, Reason, Report} -> ?vdebug("discarded: ~p", [Reason]), ErrorInfo = {failed_processing_message, Reason}, - Pid ! {snmp_error, ErrorInfo, Addr, Port}, - maybe_udp_send(State#state.filter, Sock, Addr, Port, Report), - ok; + Pid ! {snmp_error, ErrorInfo, Domain, Addr}, + maybe_udp_send(Domain, Addr, Report, State); {discarded, Reason} -> ?vdebug("discarded: ~p", [Reason]), ErrorInfo = {failed_processing_message, Reason}, - Pid ! {snmp_error, ErrorInfo, Addr, Port}, - ok; + Pid ! {snmp_error, ErrorInfo, Domain, Addr}; Error -> error_msg("processing of received message failed: " - "~n ~p", [Error]), - ok + "~n ~p", [Error]) end. -maybe_handle_recv_pdu(Addr, Port, - Vsn, #pdu{type = Type} = Pdu, PduMS, ACM, - Logger, - #state{filter = FilterMod} = State) -> - case (catch FilterMod:accept_recv_pdu(Addr, Port, Type)) of +maybe_handle_recv_pdu( + Domain, Addr, Vsn, #pdu{type = Type} = Pdu, PduMS, ACM, Logger, + #state{filter = FilterMod, transports = Transports} = State) -> + {Arg1, Arg2} = fix_filter_address(Transports, {Domain, Addr}), + case (catch FilterMod:accept_recv_pdu(Arg1, Arg2, Type)) of false -> - inc(netIfPduInDrops), - ok; + inc(netIfPduInDrops); _ -> - handle_recv_pdu(Addr, Port, Vsn, Pdu, PduMS, ACM, Logger, State) + handle_recv_pdu( + Domain, Addr, Vsn, Pdu, PduMS, ACM, Logger, State) end; -maybe_handle_recv_pdu(Addr, Port, Vsn, Trap, PduMS, ACM, Logger, - #state{filter = FilterMod} = State) +maybe_handle_recv_pdu( + Domain, Addr, Vsn, Trap, PduMS, ACM, Logger, + #state{filter = FilterMod, transports = Transports} = State) when is_record(Trap, trappdu) -> - case (catch FilterMod:accept_recv_pdu(Addr, Port, trappdu)) of + {Arg1, Arg2} = fix_filter_address(Transports, {Domain, Addr}), + case (catch FilterMod:accept_recv_pdu(Arg1, Arg2, trappdu)) of false -> - inc(netIfPduInDrops), - ok; + inc(netIfPduInDrops); _ -> - handle_recv_pdu(Addr, Port, Vsn, Trap, PduMS, ACM, Logger, State) + handle_recv_pdu( + Domain, Addr, Vsn, Trap, PduMS, ACM, Logger, State) end; -maybe_handle_recv_pdu(Addr, Port, Vsn, Pdu, PduMS, ACM, Logger, State) -> - handle_recv_pdu(Addr, Port, Vsn, Pdu, PduMS, ACM, Logger, State). - - -handle_recv_pdu(Addr, Port, - Vsn, #pdu{type = 'inform-request'} = Pdu, _PduMS, ACM, - Logger, #state{server = Pid, irb = IRB} = State) -> - handle_inform_request(IRB, Pid, Vsn, Pdu, ACM, - Addr, Port, Logger, State); -handle_recv_pdu(Addr, Port, - _Vsn, #pdu{type = report} = Pdu, _PduMS, ok, - _Logger, - #state{server = Pid} = _State) -> +maybe_handle_recv_pdu( + Domain, Addr, Vsn, Pdu, PduMS, ACM, Logger, State) -> + handle_recv_pdu(Domain, Addr, Vsn, Pdu, PduMS, ACM, Logger, State). + + +handle_recv_pdu( + Domain, Addr, Vsn, + #pdu{type = 'inform-request'} = Pdu, _PduMS, ACM, Logger, + #state{server = Pid, irb = IRB} = State) -> + handle_inform_request( + IRB, Pid, Vsn, Pdu, ACM, Domain, Addr, Logger, State); +handle_recv_pdu( + Domain, Addr, _Vsn, + #pdu{type = report} = Pdu, _PduMS, ok, _Logger, + #state{server = Pid} = _State) -> ?vtrace("received report - ok", []), - Pid ! {snmp_report, {ok, Pdu}, Addr, Port}; -handle_recv_pdu(Addr, Port, - _Vsn, #pdu{type = report} = Pdu, _PduMS, - {error, ReqId, Reason}, - _Logger, - #state{server = Pid} = _State) -> + Pid ! {snmp_report, {ok, Pdu}, Domain, Addr}; +handle_recv_pdu( + Domain, Addr, _Vsn, + #pdu{type = report} = Pdu, _PduMS, {error, ReqId, Reason}, _Logger, + #state{server = Pid} = _State) -> ?vtrace("received report - error", []), - Pid ! {snmp_report, {error, ReqId, Reason, Pdu}, Addr, Port}; -handle_recv_pdu(Addr, Port, - _Vsn, #pdu{type = 'snmpv2-trap'} = Pdu, _PduMS, _ACM, - _Logger, - #state{server = Pid} = _State) -> + Pid ! {snmp_report, {error, ReqId, Reason, Pdu}, Domain, Addr}; +handle_recv_pdu( + Domain, Addr, _Vsn, + #pdu{type = 'snmpv2-trap'} = Pdu, _PduMS, _ACM, _Logger, + #state{server = Pid} = _State) -> ?vtrace("received snmpv2-trap", []), - Pid ! {snmp_trap, Pdu, Addr, Port}; -handle_recv_pdu(Addr, Port, - _Vsn, Trap, _PduMS, _ACM, - _Logger, - #state{server = Pid} = _State) when is_record(Trap, trappdu) -> + Pid ! {snmp_trap, Pdu, Domain, Addr}; +handle_recv_pdu( + Domain, Addr, _Vsn, Trap, _PduMS, _ACM, _Logger, + #state{server = Pid} = _State) when is_record(Trap, trappdu) -> ?vtrace("received trappdu", []), - Pid ! {snmp_trap, Trap, Addr, Port}; -handle_recv_pdu(Addr, Port, - _Vsn, Pdu, _PduMS, _ACM, - _Logger, - #state{server = Pid} = _State) when is_record(Pdu, pdu) -> + Pid ! {snmp_trap, Trap, Domain, Addr}; +handle_recv_pdu( + Domain, Addr, _Vsn, Pdu, _PduMS, _ACM, _Logger, + #state{server = Pid} = _State) when is_record(Pdu, pdu) -> ?vtrace("received pdu", []), - Pid ! {snmp_pdu, Pdu, Addr, Port}; -handle_recv_pdu(_Addr, _Port, _Vsn, Pdu, _PduMS, ACM, _Logger, _State) -> + Pid ! {snmp_pdu, Pdu, Domain, Addr}; +handle_recv_pdu( + _Domain, _Addr, _Vsn, Pdu, _PduMS, ACM, _Logger, _State) -> ?vlog("received unexpected pdu: " - "~n Pdu: ~p" - "~n ACM: ~p", [Pdu, ACM]). + "~n Pdu: ~p" + "~n ACM: ~p", [Pdu, ACM]). -handle_inform_request(auto, Pid, Vsn, Pdu, ACM, Addr, Port, Logger, State) -> +handle_inform_request( + auto, Pid, Vsn, Pdu, ACM, Domain, Addr, Logger, State) -> ?vtrace("received inform-request (true)", []), - Pid ! {snmp_inform, ignore, Pdu, Addr, Port}, + Pid ! {snmp_inform, ignore, Pdu, Domain, Addr}, RePdu = make_response_pdu(Pdu), - maybe_send_inform_response(RePdu, Vsn, ACM, Addr, Port, Logger, State); -handle_inform_request({user, To}, Pid, Vsn, #pdu{request_id = ReqId} = Pdu, - ACM, Addr, Port, _Logger, _State) -> + maybe_send_inform_response(RePdu, Vsn, ACM, Domain, Addr, Logger, State); +handle_inform_request( + {user, To}, Pid, Vsn, #pdu{request_id = ReqId} = Pdu, + ACM, Domain, Addr, _Logger, _State) -> ?vtrace("received inform-request (false)", []), - Pid ! {snmp_inform, ReqId, Pdu, Addr, Port}, + Pid ! {snmp_inform, ReqId, Pdu, Domain, Addr}, %% Before we go any further, we need to check that we have not %% already received this message (possible resend). - Key = {ReqId, Addr, Port}, + Key = {ReqId, Domain, Addr}, case ets:lookup(snmpm_inform_request_table, Key) of [_] -> %% OK, we already know about this. We assume this @@ -676,40 +817,57 @@ handle_inform_request({user, To}, Pid, Vsn, #pdu{request_id = ReqId} = Pdu, Rec = {Key, Expire, {Vsn, ACM, RePdu}}, ets:insert(snmpm_inform_request_table, Rec) end. - -handle_inform_response(Ref, Addr, Port, State) -> - Key = {Ref, Addr, Port}, + +handle_inform_response(Ref, Domain, Addr, State) -> + ?worker( + S, handle_inform_response_mt(Ref, Domain, Addr, S), + fun (Pid, Class, Reason, Stacktrace) -> + warning_msg( + "Worker process (~p) terminated " + "while processing (outgoing) inform response for %s:~n" + "~w:~w at ~p", + [Pid, snmp_conf:mk_addr_string({Domain, Addr}), + Class, Reason, Stacktrace]) + end, + State). + +handle_inform_response_mt(Ref, Domain, Addr, State) -> + Key = {Ref, Domain, Addr}, case ets:lookup(snmpm_inform_request_table, Key) of [{Key, _, {Vsn, ACM, RePdu}}] -> - Logger = logger(State#state.log, read, Addr, Port), + Logger = logger(State#state.log, read, Domain, Addr), ets:delete(snmpm_inform_request_table, Key), - maybe_send_inform_response(RePdu, Vsn, ACM, Addr, Port, - Logger, State); + maybe_send_inform_response( + RePdu, Vsn, ACM, Domain, Addr, Logger, State); [] -> %% Already acknowledged, or the user was to slow to reply... ok end, ok. -maybe_send_inform_response(RePdu, Vsn, ACM, Addr, Port, Logger, - #state{server = Pid, - sock = Sock, - filter = FilterMod}) -> - case (catch FilterMod:accept_send_pdu(Addr, Port, pdu_type_of(RePdu))) of +maybe_send_inform_response( + RePdu, Vsn, ACM, Domain, Addr, Logger, + #state{ + server = Pid, + filter = FilterMod, + transports = Transports} = State) -> + {Arg1, Arg2} = fix_filter_address(Transports, {Domain, Addr}), + case (catch FilterMod:accept_send_pdu( + Arg1, Arg2, pdu_type_of(RePdu))) + of false -> inc(netIfPduOutDrops), ok; _ -> case snmpm_mpd:generate_response_msg(Vsn, RePdu, ACM, Logger) of {ok, Msg} -> - maybe_udp_send(FilterMod, Sock, Addr, Port, Msg); + maybe_udp_send(Domain, Addr, Msg, State); {discarded, Reason} -> ?vlog("failed generating response message:" "~n Reason: ~p", [Reason]), ReqId = RePdu#pdu.request_id, ErrorInfo = {failed_generating_response, {RePdu, Reason}}, - Pid ! {snmp_error, ReqId, ErrorInfo, Addr, Port}, - ok + Pid ! {snmp_error, ReqId, ErrorInfo, Domain, Addr} end end. @@ -742,64 +900,99 @@ irgc_stop(undefined) -> irgc_stop(Ref) -> (catch erlang:cancel_timer(Ref)). - -maybe_handle_send_pdu(Pdu, Vsn, MsgData, Domain, Addr, Port, - #state{filter = FilterMod} = State) -> - case (catch FilterMod:accept_send_pdu(Addr, Port, pdu_type_of(Pdu))) of +maybe_handle_send_pdu(Pdu, Vsn, MsgData, Domain, Addr, State) -> + ?worker( + S, maybe_handle_send_pdu_mt(Pdu, Vsn, MsgData, Domain, Addr, S), + fun (Pid, Class, Reason, Stacktrace) -> + warning_msg( + "Worker process (~p) terminated " + "while processing (outgoing) pdu for %s:~n" + "~w:~w at ~p", + [Pid, snmp_conf:mk_addr_string({Domain, Addr}), + Class, Reason, Stacktrace]) + end, + State). + +maybe_handle_send_pdu_mt( + Pdu, Vsn, MsgData, Domain, Addr, + #state{filter = FilterMod, transports = Transports} = State) -> + {Arg1, Arg2} = fix_filter_address(Transports, {Domain, Addr}), + case (catch FilterMod:accept_send_pdu(Arg1, Arg2, pdu_type_of(Pdu))) of false -> - inc(netIfPduOutDrops), - ok; + inc(netIfPduOutDrops); _ -> - handle_send_pdu(Pdu, Vsn, MsgData, Domain, Addr, Port, State) - end. + handle_send_pdu(Pdu, Vsn, MsgData, Domain, Addr, State) + end, + ok. -handle_send_pdu(Pdu, Vsn, MsgData, _Domain, Addr, Port, - #state{server = Pid, - note_store = NoteStore, - sock = Sock, - log = Log, - filter = FilterMod}) -> - Logger = logger(Log, write, Addr, Port), - case (catch snmpm_mpd:generate_msg(Vsn, NoteStore, - Pdu, MsgData, Logger)) of +handle_send_pdu( + Pdu, Vsn, MsgData, Domain, Addr, + #state{ + server = Pid, + note_store = NoteStore, + log = Log} = State) -> + Logger = logger(Log, write, Domain, Addr), + case (catch snmpm_mpd:generate_msg( + Vsn, NoteStore, Pdu, MsgData, Logger)) of {ok, Msg} -> ?vtrace("handle_send_pdu -> message generated", []), - maybe_udp_send(FilterMod, Sock, Addr, Port, Msg); + maybe_udp_send(Domain, Addr, Msg, State); {discarded, Reason} -> ?vlog("PDU not sent: " "~n PDU: ~p" "~n Reason: ~p", [Pdu, Reason]), - Pid ! {snmp_error, Pdu, Reason}, - ok + Pid ! {snmp_error, Pdu, Reason} end. -maybe_udp_send(FilterMod, Sock, Addr, Port, Msg) -> - case (catch FilterMod:accept_send(Addr, Port)) of +maybe_udp_send( + Domain, Addr, Msg, + #state{filter = FilterMod, transports = Transports}) -> + To = {Domain, Addr}, + {Arg1, Arg2} = fix_filter_address(Transports, To), + case (catch FilterMod:accept_send(Arg1, Arg2)) of false -> inc(netIfMsgOutDrops), ok; _ -> - udp_send(Sock, Addr, Port, Msg) + case select_transport_from_domain(Domain, Transports) of + false -> + error_msg( + "Can not find transport~n" + " size: ~p~n" + " to: ~s", + [sz(Msg), format_address(To)]); + #transport{socket = Socket} -> + udp_send(Socket, Addr, Msg) + end end. - - -udp_send(Sock, Addr, Port, Msg) -> - case (catch gen_udp:send(Sock, Addr, Port, Msg)) of + +udp_send(Sock, To, Msg) -> + {IpAddr, IpPort} = + case To of + {Domain, Addr} when is_atom(Domain) -> + Addr; + {_, P} = Addr when is_integer(P) -> + Addr + end, + try gen_udp:send(Sock, IpAddr, IpPort, Msg) of ok -> ?vdebug("sent ~w bytes to ~w:~w [~w]", - [sz(Msg), Addr, Port, Sock]), + [sz(Msg), IpAddr, IpPort, Sock]), ok; {error, Reason} -> - error_msg("failed sending message to ~p:~p: " - "~n ~p",[Addr, Port, Reason]); - Error -> - error_msg("failed sending message to ~p:~p: " - "~n ~p",[Addr, Port, Error]) + error_msg("failed sending message to ~p:~p:~n" + " ~p",[IpAddr, IpPort, Reason]) + catch + error:Error -> + error_msg("failed sending message to ~p:~p:~n" + " error:~p~n" + " ~p", + [IpAddr, IpPort, Error, erlang:get_stacktrace()]) end. sz(B) when is_binary(B) -> - size(B); + byte_size(B); sz(L) when is_list(L) -> length(L); sz(_) -> @@ -989,6 +1182,45 @@ handle_set_log_type(State, _NewType) -> {State, {error, not_enabled}}. +select_transport_from_domain(Domain, Transports) when is_atom(Domain) -> + Pos = #transport.domain, + case lists:keyfind(Domain, Pos, Transports) of + #transport{domain = Domain} = Transport -> + Transport; + false when Domain == snmpUDPDomain -> + lists:keyfind(transportDomainUdpIpv4, Pos, Transports); + false when Domain == transportDomainUdpIpv4 -> + lists:keyfind(snmpUDPDomain, Pos, Transports); + false -> + false + end. + +%% If the manager uses legacy snmpUDPDomain e.g has not set +%% {domain, _}, then make sure snmpm_network_interface_filter +%% gets legacy arguments to not break backwards compatibility. +%% +fix_filter_address(Transports, Address) -> + DefaultDomain = snmpm_config:default_transport_domain(), + case Transports of + [#transport{domain = DefaultDomain}, DefaultDomain] -> + case Address of + {Domain, Addr} when is_atom(Domain) -> + Addr; + {_, IpPort} = Addr when is_integer(IpPort) -> + Addr + end; + _ -> + Address + end. + +address(Domain, Addr) when is_atom(Domain) -> + {Domain, Addr}; +address(Ip, Port) when is_integer(Port) -> + {snmpm_config:default_transport_domain(), {Ip, Port}}. + +format_address(Address) -> + iolist_to_binary(snmp_conf:mk_addr_string(Address)). + %% ------------------------------------------------------------------- make_response_pdu(#pdu{request_id = ReqId, varbinds = Vbs}) -> @@ -1029,25 +1261,6 @@ t() -> %% ------------------------------------------------------------------- -logger(undefined, _Type, _Addr, _Port) -> - fun(_) -> - ok - end; -logger({Log, Types}, Type, Addr, Port) -> - case lists:member(Type, Types) of - true -> - fun(Msg) -> - snmp_log:log(Log, Msg, Addr, Port) - end; - false -> - fun(_) -> - ok - end - end. - - -%% ------------------------------------------------------------------- - %% info_msg(F, A) -> %% ?snmpm_info("NET-IF server: " ++ F, A). @@ -1072,10 +1285,11 @@ get_opt(Opts, Key, Def) -> %% ------------------------------------------------------------------- -get_info(#state{sock = Id}) -> +get_info(#state{transports = Transports}) -> ProcSize = proc_mem(self()), - PortInfo = get_port_info(Id), - [{process_memory, ProcSize}, {port_info, PortInfo}]. + [{process_memory, ProcSize} + | [{port_info, get_port_info(Socket)} + || #transport{socket = Socket} <- Transports]]. proc_mem(P) when is_pid(P) -> case (catch erlang:process_info(P, memory)) of @@ -1190,4 +1404,3 @@ call(Pid, Req, Timeout) -> cast(Pid, Msg) -> gen_server:cast(Pid, Msg). - diff --git a/lib/snmp/src/manager/snmpm_net_if_filter.erl b/lib/snmp/src/manager/snmpm_net_if_filter.erl index eb0c6efb11..d96ae5c145 100644 --- a/lib/snmp/src/manager/snmpm_net_if_filter.erl +++ b/lib/snmp/src/manager/snmpm_net_if_filter.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009. All Rights Reserved. +%% Copyright Ericsson AB 2009-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -25,29 +25,51 @@ -include("snmp_debug.hrl"). -accept_recv(_Addr, _Port) -> - ?d("accept_recv -> entry with" - "~n Addr: ~p" - "~n Port: ~p", [_Addr, _Port]), +accept_recv(Domain, _Address) when is_atom(Domain) -> + ?d("accept_recv -> entry with~n" + " Domain: ~p~n" + " Address: ~p", [Domain, _Address]), + true; +accept_recv(_Addr, Port) when is_integer(Port) -> + ?d("accept_recv -> entry with~n" + " Addr: ~p~n" + " Port: ~p", [_Addr, Port]), true. -accept_send(_Addr, _Port) -> - ?d("accept_send -> entry with" - "~n Addr: ~p" - "~n Port: ~p", [_Addr, _Port]), +accept_send(Domain, _Address) when is_atom(Domain) -> + ?d("accept_send -> entry with~n" + " Domain: ~p~n" + " Address: ~p", [Domain, _Address]), + true; +accept_send(_Addr, Port) when is_integer(Port) -> + ?d("accept_send -> entry with~n" + " Addr: ~p~n" + " Port: ~p", [_Addr, Port]), true. -accept_recv_pdu(_Addr, _Port, _PduType) -> - ?d("accept_recv_pdu -> entry with" - "~n Addr: ~p" - "~n Port: ~p" - "~n PduType: ~p", [_Addr, _Port, _PduType]), +accept_recv_pdu(Domain, _Address, _PduType) when is_atom(Domain) -> + ?d("accept_recv_pdu -> entry with~n" + " Domain: ~p~n" + " Address: ~p~n" + " PduType: ~p", [Domain, _Address, _PduType]), + true; +accept_recv_pdu(_Addr, Port, _PduType) when is_integer(Port) -> + ?d("accept_recv_pdu -> entry with~n" + " Addr: ~p~n" + " Port: ~p~n" + " PduType: ~p", [_Addr, Port, _PduType]), true. -accept_send_pdu(_Addr, _Port, _PduType) -> - ?d("accept_send_pdu -> entry with" - "~n Addr: ~p" - "~n Port: ~p" - "~n PduType: ~p", [_Addr, _Port, _PduType]), +accept_send_pdu(Domain, _Address, _PduType) when is_atom(Domain) -> + ?d("accept_send_pdu -> entry with~n" + " Domain: ~p~n" + " Address: ~p~n" + " PduType: ~p", [Domain, _Address, _PduType]), + true; +accept_send_pdu(_Addr, Port, _PduType) when is_integer(Port) -> + ?d("accept_send_pdu -> entry with~n" + " Addr: ~p~n" + " Port: ~p~n" + " PduType: ~p", [_Addr, Port, _PduType]), true. diff --git a/lib/snmp/src/manager/snmpm_net_if_mt.erl b/lib/snmp/src/manager/snmpm_net_if_mt.erl index 3e87f6a7fb..62f6023657 100644 --- a/lib/snmp/src/manager/snmpm_net_if_mt.erl +++ b/lib/snmp/src/manager/snmpm_net_if_mt.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2012. All Rights Reserved. +%% Copyright Ericsson AB 2004-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -17,1243 +17,7 @@ %% %CopyrightEnd% %% --module(snmpm_net_if_mt). - --behaviour(gen_server). --behaviour(snmpm_network_interface). - - -%% Network Interface callback functions --export([ - start_link/2, - stop/1, - send_pdu/6, % Backward compatibillity - send_pdu/7, % Backward compatibillity - send_pdu/8, - - inform_response/4, - - note_store/2, - - info/1, - verbosity/2, - %% system_info_updated/2, - get_log_type/1, set_log_type/2, - filter_reset/1 - ]). - -%% gen_server callbacks --export([init/1, handle_call/3, handle_cast/2, handle_info/2, - code_change/3, terminate/2]). - --define(SNMP_USE_V3, true). --include("snmp_types.hrl"). --include("snmpm_internal.hrl"). --include("snmpm_atl.hrl"). --include("snmp_debug.hrl"). - -%% -define(VMODULE,"NET_IF"). --include("snmp_verbosity.hrl"). - --record(state, - { - server, - note_store, - sock, - mpd_state, - log, - irb = auto, % auto | {user, integer()} - irgc, - filter - }). - - --define(DEFAULT_FILTER_MODULE, snmpm_net_if_filter). --define(DEFAULT_FILTER_OPTS, [{module, ?DEFAULT_FILTER_MODULE}]). - --ifdef(snmp_debug). --define(GS_START_LINK(Args), - gen_server:start_link(?MODULE, Args, [{debug,[trace]}])). --else. --define(GS_START_LINK(Args), - gen_server:start_link(?MODULE, Args, [])). --endif. - - --define(IRGC_TIMEOUT, timer:minutes(5)). - --define(ATL_SEQNO_INITIAL, 1). --define(ATL_SEQNO_MAX, 2147483647). - - -%%%------------------------------------------------------------------- -%%% API -%%%------------------------------------------------------------------- -start_link(Server, NoteStore) -> - ?d("start_link -> entry with" - "~n Server: ~p" - "~n NoteStore: ~p", [Server, NoteStore]), - Args = [Server, NoteStore], - ?GS_START_LINK(Args). - -stop(Pid) -> - call(Pid, stop). - -send_pdu(Pid, Pdu, Vsn, MsgData, Addr, Port) -> - send_pdu(Pid, Pdu, Vsn, MsgData, Addr, Port, ?DEFAULT_EXTRA_INFO). - -send_pdu(Pid, Pdu, Vsn, MsgData, Addr, Port, ExtraInfo) -> - Domain = snmpm_config:default_transport_domain(), - send_pdu(Pid, Pdu, Vsn, MsgData, Domain, Addr, Port, ExtraInfo). - -send_pdu(Pid, Pdu, Vsn, MsgData, Domain, Addr, Port, ExtraInfo) - when is_record(Pdu, pdu) -> - ?d("send_pdu -> entry with" - "~n Pid: ~p" - "~n Pdu: ~p" - "~n Vsn: ~p" - "~n MsgData: ~p" - "~n Domain: ~p" - "~n Addr: ~p" - "~n Port: ~p", [Pid, Pdu, Vsn, MsgData, Domain, Addr, Port]), - cast(Pid, {send_pdu, Pdu, Vsn, MsgData, Domain, Addr, Port, ExtraInfo}). - -note_store(Pid, NoteStore) -> - call(Pid, {note_store, NoteStore}). - -inform_response(Pid, Ref, Addr, Port) -> - cast(Pid, {inform_response, Ref, Addr, Port}). - -info(Pid) -> - call(Pid, info). - -verbosity(Pid, V) -> - call(Pid, {verbosity, V}). - -%% system_info_updated(Pid, What) -> -%% call(Pid, {system_info_updated, What}). - -get_log_type(Pid) -> - call(Pid, get_log_type). - -set_log_type(Pid, NewType) -> - call(Pid, {set_log_type, NewType}). - -filter_reset(Pid) -> - cast(Pid, filter_reset). - - -%%%------------------------------------------------------------------- -%%% Callback functions from gen_server -%%%------------------------------------------------------------------- - -%%-------------------------------------------------------------------- -%% Func: init/1 -%% Returns: {ok, State} | -%% {ok, State, Timeout} | -%% ignore | -%% {stop, Reason} -%%-------------------------------------------------------------------- -init([Server, NoteStore]) -> - ?d("init -> entry with" - "~n Server: ~p" - "~n NoteStore: ~p", [Server, NoteStore]), - case (catch do_init(Server, NoteStore)) of - {error, Reason} -> - {stop, Reason}; - {ok, State} -> - {ok, State} - end. - -do_init(Server, NoteStore) -> - process_flag(trap_exit, true), - - %% -- Prio -- - {ok, Prio} = snmpm_config:system_info(prio), - process_flag(priority, Prio), - - %% -- Create inform request table -- - %% This should really be protected, but it also needs to - %% be writable for the worker processes, so... - ets:new(snmpm_inform_request_table, - [set, public, named_table, {keypos, 1}]), - - %% -- Verbosity -- - {ok, Verbosity} = snmpm_config:system_info(net_if_verbosity), - put(sname, mnif), - put(verbosity, Verbosity), - ?vlog("starting", []), - - %% -- MPD -- - {ok, Vsns} = snmpm_config:system_info(versions), - MpdState = snmpm_mpd:init(Vsns), - ?vdebug("MpdState: ~w", [MpdState]), - - %% -- Module dependent options -- - {ok, Opts} = snmpm_config:system_info(net_if_options), - - %% -- Inform response behaviour -- - {ok, IRB} = snmpm_config:system_info(net_if_irb), - IrGcRef = irgc_start(IRB), - - %% -- Socket -- - SndBuf = get_opt(Opts, sndbuf, default), - RecBuf = get_opt(Opts, recbuf, default), - BindTo = get_opt(Opts, bind_to, false), - NoReuse = get_opt(Opts, no_reuse, false), - {ok, Port} = snmpm_config:system_info(port), - {ok, Sock} = do_open_port(Port, SndBuf, RecBuf, BindTo, NoReuse), - - %% Flow control -- - FilterOpts = get_opt(Opts, filter, []), - FilterMod = create_filter(FilterOpts), - ?vdebug("FilterMod: ~w", [FilterMod]), - - %% -- Audit trail log --- - {ok, ATL} = snmpm_config:system_info(audit_trail_log), - Log = do_init_log(ATL), - ?vdebug("Log: ~w", [Log]), - - %% -- Initiate counters --- - init_counters(), - - %% -- We are done --- - State = #state{server = Server, - note_store = NoteStore, - mpd_state = MpdState, - sock = Sock, - log = Log, - irb = IRB, - irgc = IrGcRef, - filter = FilterMod}, - ?vdebug("started", []), - {ok, State}. - - -%% Open port -do_open_port(Port, SendSz, RecvSz, BindTo, NoReuse) -> - ?vtrace("do_open_port -> entry with" - "~n Port: ~p" - "~n SendSz: ~p" - "~n RecvSz: ~p" - "~n BindTo: ~p" - "~n NoReuse: ~p", [Port, SendSz, RecvSz, BindTo, NoReuse]), - IpOpts1 = bind_to(BindTo), - IpOpts2 = no_reuse(NoReuse), - IpOpts3 = recbuf(RecvSz), - IpOpts4 = sndbuf(SendSz), - IpOpts = [binary | IpOpts1 ++ IpOpts2 ++ IpOpts3 ++ IpOpts4], - OpenRes = - case init:get_argument(snmpm_fd) of - {ok, [[FdStr]]} -> - Fd = list_to_integer(FdStr), - gen_udp:open(0, [{fd, Fd}|IpOpts]); - error -> - gen_udp:open(Port, IpOpts) - end, - case OpenRes of - {error, _} = Error -> - throw(Error); - OK -> - OK - end. - -bind_to(true) -> - case snmpm_config:system_info(address) of - {ok, Addr} when is_list(Addr) -> - [{ip, list_to_tuple(Addr)}]; - {ok, Addr} -> - [{ip, Addr}]; - _ -> - [] - end; -bind_to(_) -> - []. - -no_reuse(false) -> - [{reuseaddr, true}]; -no_reuse(_) -> - []. - -recbuf(default) -> - []; -recbuf(Sz) -> - [{recbuf, Sz}]. - -sndbuf(default) -> - []; -sndbuf(Sz) -> - [{sndbuf, Sz}]. - - -create_filter(Opts) when is_list(Opts) -> - case get_opt(Opts, module, ?DEFAULT_FILTER_MODULE) of - ?DEFAULT_FILTER_MODULE = Mod -> - Mod; - Module -> - snmpm_network_interface_filter:verify(Module), - Module - end; -create_filter(BadOpts) -> - throw({error, {bad_filter_opts, BadOpts}}). - - -%% ---------------------------------------------------------------------- -%% Audit Trail Logger -%% ---------------------------------------------------------------------- - -%% Open log -do_init_log(false) -> - ?vtrace("do_init_log(false) -> entry", []), - undefined; -do_init_log(true) -> - ?vtrace("do_init_log(true) -> entry", []), - {ok, Type} = snmpm_config:system_info(audit_trail_log_type), - {ok, Dir} = snmpm_config:system_info(audit_trail_log_dir), - {ok, Size} = snmpm_config:system_info(audit_trail_log_size), - {ok, Repair} = snmpm_config:system_info(audit_trail_log_repair), - Name = ?audit_trail_log_name, - File = filename:absname(?audit_trail_log_file, Dir), - case snmpm_config:system_info(audit_trail_log_seqno) of - {ok, true} -> - Initial = ?ATL_SEQNO_INITIAL, - Max = ?ATL_SEQNO_MAX, - Module = snmpm_config, - Function = increment_counter, - Args = [atl_seqno, Initial, Max], - SeqNoGen = {Module, Function, Args}, - case snmp_log:create(Name, File, SeqNoGen, Size, Repair, true) of - {ok, Log} -> - ?vdebug("log created: ~w", [Log]), - {Name, Log, Type}; - {error, Reason} -> - throw({error, {failed_create_audit_log, Reason}}) - end; - _ -> - case snmp_log:create(Name, File, Size, Repair, true) of - {ok, Log} -> - ?vdebug("log created: ~w", [Log]), - {Name, Log, Type}; - {error, Reason} -> - throw({error, {failed_create_audit_log, Reason}}) - end - end. - - -%% ---------------------------------------------------------------------- - -%%-------------------------------------------------------------------- -%% Func: handle_call/3 -%% Returns: {reply, Reply, State} | -%% {reply, Reply, State, Timeout} | -%% {noreply, State} | -%% {noreply, State, Timeout} | -%% {stop, Reason, Reply, State} | (terminate/2 is called) -%% {stop, Reason, State} (terminate/2 is called) -%%-------------------------------------------------------------------- -handle_call({verbosity, Verbosity}, _From, State) -> - ?vlog("received verbosity request", []), - put(verbosity, Verbosity), - {reply, ok, State}; - -%% handle_call({system_info_updated, What}, _From, State) -> -%% ?vlog("received system_info_updated request with What = ~p", [What]), -%% {NewState, Reply} = handle_system_info_updated(State, What), -%% {reply, Reply, NewState}; - -handle_call(get_log_type, _From, State) -> - ?vlog("received get-log-type request", []), - Reply = (catch handle_get_log_type(State)), - {reply, Reply, State}; - -handle_call({set_log_type, NewType}, _From, State) -> - ?vlog("received set-log-type request with NewType = ~p", [NewType]), - {NewState, Reply} = (catch handle_set_log_type(State, NewType)), - {reply, Reply, NewState}; - -handle_call({note_store, Pid}, _From, State) -> - ?vlog("received new note_store: ~w", [Pid]), - {reply, ok, State#state{note_store = Pid}}; - -handle_call(stop, _From, State) -> - ?vlog("received stop request", []), - Reply = ok, - {stop, normal, Reply, State}; - -handle_call(info, _From, State) -> - ?vlog("received info request", []), - Reply = get_info(State), - {reply, Reply, State}; - -handle_call(Req, From, State) -> - warning_msg("received unknown request (from ~p): ~n~p", [Req, From]), - {reply, {error, {invalid_request, Req}}, State}. - - -%%-------------------------------------------------------------------- -%% Func: handle_cast/2 -%% Returns: {noreply, State} | -%% {noreply, State, Timeout} | -%% {stop, Reason, State} (terminate/2 is called) -%%-------------------------------------------------------------------- -handle_cast({send_pdu, Pdu, Vsn, MsgData, Domain, Addr, Port, ExtraInfo}, - State) -> - ?vlog("received send_pdu message with" - "~n Pdu: ~p" - "~n Vsn: ~p" - "~n MsgData: ~p" - "~n Domain: ~p" - "~n Addr: ~p" - "~n Port: ~p", [Pdu, Vsn, MsgData, Domain, Addr, Port]), - maybe_process_extra_info(ExtraInfo), - handle_send_pdu(Pdu, Vsn, MsgData, Domain, Addr, Port, State), - {noreply, State}; - -handle_cast({inform_response, Ref, Addr, Port}, State) -> - ?vlog("received inform_response message with" - "~n Ref: ~p" - "~n Addr: ~p" - "~n Port: ~p", [Ref, Addr, Port]), - handle_inform_response(Ref, Addr, Port, State), - {noreply, State}; - -handle_cast(filter_reset, State) -> - ?vlog("received filter_reset message", []), - reset_counters(), - {noreply, State}; - -handle_cast(Msg, State) -> - warning_msg("received unknown message: ~n~p", [Msg]), - {noreply, State}. - - -%%-------------------------------------------------------------------- -%% Func: handle_info/2 -%% Returns: {noreply, State} | -%% {noreply, State, Timeout} | -%% {stop, Reason, State} (terminate/2 is called) -%%-------------------------------------------------------------------- -handle_info({udp, Sock, Ip, Port, Bytes}, #state{sock = Sock} = State) -> - ?vlog("received ~w bytes from ~p:~p", [size(Bytes), Ip, Port]), - handle_udp(Ip, Port, Bytes, State), - {noreply, State}; - -handle_info(inform_response_gc, State) -> - ?vlog("received inform_response_gc message", []), - State2 = handle_inform_response_gc(State), - {noreply, State2}; - -handle_info({disk_log, _Node, Log, Info}, State) -> - ?vlog("received disk_log message: " - "~n Info: ~p", [Info]), - State2 = handle_disk_log(Log, Info, State), - {noreply, State2}; - -handle_info({'DOWN', _MRef, process, Pid, {net_if_worker, ExitStatus}}, - State) -> - ?vdebug("received DOWN message from net_if-worker: " - "~n ExitStatus: ~p", [ExitStatus]), - handle_worker_exit(Pid, ExitStatus), - {noreply, State}; - -handle_info(Info, State) -> - warning_msg("received unknown info: ~n~p", [Info]), - {noreply, State}. - - -%%-------------------------------------------------------------------- -%% Func: terminate/2 -%% Purpose: Shutdown the server -%% Returns: any (ignored by gen_server) -%%-------------------------------------------------------------------- -terminate(Reason, #state{log = Log, irgc = IrGcRef}) -> - ?vdebug("terminate: ~p", [Reason]), - irgc_stop(IrGcRef), - %% Close logs - do_close_log(Log), - ok. - - -do_close_log({Log, _Type}) -> - (catch snmp_log:sync(Log)), - (catch snmp_log:close(Log)), - ok; -do_close_log(_) -> - ok. - - -%%---------------------------------------------------------------------- -%% Func: code_change/3 -%% Purpose: Convert process state when code is changed -%% Returns: {ok, NewState} -%%---------------------------------------------------------------------- - -code_change(_Vsn, State, _Extra) -> - ?d("code_change -> entry with" - "~n Vsn: ~p" - "~n State: ~p" - "~n Extra: ~p", [_Vsn, State, _Extra]), - {ok, State}. - - -%%%------------------------------------------------------------------- -%%% Internal functions -%%%------------------------------------------------------------------- - -handle_udp(Addr, Port, Bytes, State) -> - Verbosity = get(verbosity), - spawn_opt(fun() -> - Log = worker_init(State, Verbosity), - Res = (catch maybe_handle_recv_msg( - Addr, Port, Bytes, - State#state{log = Log})), - worker_exit(udp, {Addr, Port}, Res) - end, - [monitor]). - - -maybe_handle_recv_msg(Addr, Port, Bytes, #state{filter = FilterMod} = State) -> - case (catch FilterMod:accept_recv(Addr, Port)) of - false -> - %% Drop the received packet - inc(netIfMsgInDrops), - ok; - _ -> - handle_recv_msg(Addr, Port, Bytes, State) - end. - - -handle_recv_msg(Addr, Port, Bytes, #state{server = Pid}) - when is_binary(Bytes) andalso (size(Bytes) =:= 0) -> - Pid ! {snmp_error, {empty_message, Addr, Port}, Addr, Port}, - ok; - -handle_recv_msg(Addr, Port, Bytes, - #state{server = Pid, - note_store = NoteStore, - mpd_state = MpdState, - sock = Sock, - log = Log} = State) -> - Domain = snmp_conf:which_domain(Addr), % What the ****... - Logger = logger(Log, read, Addr, Port), - case (catch snmpm_mpd:process_msg(Bytes, Domain, Addr, Port, - MpdState, NoteStore, Logger)) of - - {ok, Vsn, Pdu, MS, ACM} -> - maybe_handle_recv_pdu(Addr, Port, Vsn, Pdu, MS, ACM, - Logger, State); - - {discarded, Reason, Report} -> - ?vdebug("discarded: ~p", [Reason]), - ErrorInfo = {failed_processing_message, Reason}, - Pid ! {snmp_error, ErrorInfo, Addr, Port}, - maybe_udp_send(State#state.filter, Sock, Addr, Port, Report), - ok; - - {discarded, Reason} -> - ?vdebug("discarded: ~p", [Reason]), - ErrorInfo = {failed_processing_message, Reason}, - Pid ! {snmp_error, ErrorInfo, Addr, Port}, - ok; - - Error -> - error_msg("processing of received message failed: " - "~n ~p", [Error]), - ok - end. - - -maybe_handle_recv_pdu(Addr, Port, - Vsn, #pdu{type = Type} = Pdu, PduMS, ACM, - Logger, - #state{filter = FilterMod} = State) -> - case (catch FilterMod:accept_recv_pdu(Addr, Port, Type)) of - false -> - inc(netIfPduInDrops), - ok; - _ -> - handle_recv_pdu(Addr, Port, Vsn, Pdu, PduMS, ACM, Logger, State) - end; -maybe_handle_recv_pdu(Addr, Port, Vsn, Trap, PduMS, ACM, Logger, - #state{filter = FilterMod} = State) - when is_record(Trap, trappdu) -> - case (catch FilterMod:accept_recv_pdu(Addr, Port, trappdu)) of - false -> - inc(netIfPduInDrops), - ok; - _ -> - handle_recv_pdu(Addr, Port, Vsn, Trap, PduMS, ACM, Logger, State) - end; -maybe_handle_recv_pdu(Addr, Port, Vsn, Pdu, PduMS, ACM, Logger, State) -> - handle_recv_pdu(Addr, Port, Vsn, Pdu, PduMS, ACM, Logger, State). - - -handle_recv_pdu(Addr, Port, - Vsn, #pdu{type = 'inform-request'} = Pdu, _PduMS, ACM, - Logger, #state{server = Pid, irb = IRB} = State) -> - handle_inform_request(IRB, Pid, Vsn, Pdu, ACM, - Addr, Port, Logger, State); -handle_recv_pdu(Addr, Port, - _Vsn, #pdu{type = report} = Pdu, _PduMS, ok, - _Logger, - #state{server = Pid} = _State) -> - ?vtrace("received report - ok", []), - Pid ! {snmp_report, {ok, Pdu}, Addr, Port}, - ok; -handle_recv_pdu(Addr, Port, - _Vsn, #pdu{type = report} = Pdu, _PduMS, - {error, ReqId, Reason}, - _Logger, - #state{server = Pid} = _State) -> - ?vtrace("received report - error", []), - Pid ! {snmp_report, {error, ReqId, Reason, Pdu}, Addr, Port}, - ok; -handle_recv_pdu(Addr, Port, - _Vsn, #pdu{type = 'snmpv2-trap'} = Pdu, _PduMS, _ACM, - _Logger, - #state{server = Pid} = _State) -> - ?vtrace("received snmpv2-trap", []), - Pid ! {snmp_trap, Pdu, Addr, Port}, - ok; -handle_recv_pdu(Addr, Port, - _Vsn, Trap, _PduMS, _ACM, - _Logger, - #state{server = Pid} = _State) when is_record(Trap, trappdu) -> - ?vtrace("received trappdu", []), - Pid ! {snmp_trap, Trap, Addr, Port}, - ok; -handle_recv_pdu(Addr, Port, - _Vsn, Pdu, _PduMS, _ACM, - _Logger, - #state{server = Pid} = _State) when is_record(Pdu, pdu) -> - ?vtrace("received pdu", []), - Pid ! {snmp_pdu, Pdu, Addr, Port}, - ok; -handle_recv_pdu(_Addr, _Port, _Vsn, Pdu, _PduMS, ACM, _Logger, _State) -> - ?vlog("received unexpected pdu: " - "~n Pdu: ~p" - "~n ACM: ~p", [Pdu, ACM]), - ok. - - -handle_inform_request(auto, Pid, Vsn, Pdu, ACM, Addr, Port, Logger, State) -> - ?vtrace("received inform-request (true)", []), - Pid ! {snmp_inform, ignore, Pdu, Addr, Port}, - RePdu = make_response_pdu(Pdu), - maybe_send_inform_response(RePdu, Vsn, ACM, Addr, Port, Logger, State); -handle_inform_request({user, To}, Pid, Vsn, #pdu{request_id = ReqId} = Pdu, - ACM, Addr, Port, _Logger, _State) -> - ?vtrace("received inform-request (false)", []), - - Pid ! {snmp_inform, ReqId, Pdu, Addr, Port}, - - %% Before we go any further, we need to check that we have not - %% already received this message (possible resend). - - Key = {ReqId, Addr, Port}, - case ets:lookup(snmpm_inform_request_table, Key) of - [_] -> - %% OK, we already know about this. We assume this - %% is a resend. Either the agent is really eager or - %% the user has not answered yet. Bad user! - ok; - [] -> - RePdu = make_response_pdu(Pdu), - Expire = t() + To, - Rec = {Key, Expire, {Vsn, ACM, RePdu}}, - ets:insert(snmpm_inform_request_table, Rec) - end, - ok. - -handle_inform_response(Ref, Addr, Port, State) -> - Verbosity = get(verbosity), - spawn_opt(fun() -> - Log = worker_init(State, Verbosity), - Res = (catch do_handle_inform_response( - Ref, - Addr, Port, - State#state{log = Log})), - worker_exit(inform_reponse, {Addr, Port}, Res) - end, - [monitor]). - - - -do_handle_inform_response(Ref, Addr, Port, State) -> - Key = {Ref, Addr, Port}, - case ets:lookup(snmpm_inform_request_table, Key) of - [{Key, _, {Vsn, ACM, RePdu}}] -> - Logger = logger(State#state.log, read, Addr, Port), - ets:delete(snmpm_inform_request_table, Key), - maybe_send_inform_response(RePdu, Vsn, ACM, Addr, Port, - Logger, State); - [] -> - %% Already acknowledged, or the user was to slow to reply... - ok - end, - ok. - -maybe_send_inform_response(RePdu, Vsn, ACM, Addr, Port, Logger, - #state{server = Pid, - sock = Sock, - filter = FilterMod}) -> - case (catch FilterMod:accept_send_pdu(Addr, Port, pdu_type_of(RePdu))) of - false -> - inc(netIfPduOutDrops), - ok; - _ -> - case snmpm_mpd:generate_response_msg(Vsn, RePdu, ACM, Logger) of - {ok, Msg} -> - maybe_udp_send(FilterMod, Sock, Addr, Port, Msg); - {discarded, Reason} -> - ?vlog("failed generating response message:" - "~n Reason: ~p", [Reason]), - ReqId = RePdu#pdu.request_id, - ErrorInfo = {failed_generating_response, {RePdu, Reason}}, - Pid ! {snmp_error, ReqId, ErrorInfo, Addr, Port}, - ok - end - end. - -handle_inform_response_gc(#state{irb = IRB} = State) -> - ets:safe_fixtable(snmpm_inform_request_table, true), - do_irgc(ets:first(snmpm_inform_request_table), t()), - ets:safe_fixtable(snmpm_inform_request_table, false), - State#state{irgc = irgc_start(IRB)}. - -%% We are deleting at the same time as we are traversing the table!!! -do_irgc('$end_of_table', _) -> - ok; -do_irgc(Key, Now) -> - Next = ets:next(snmpm_inform_request_table, Key), - case ets:lookup(snmpm_inform_request_table, Key) of - [{Key, BestBefore, _}] when BestBefore < Now -> - ets:delete(snmpm_inform_request_table, Key); - _ -> - ok - end, - do_irgc(Next, Now). - -irgc_start(auto) -> - undefined; -irgc_start(_) -> - erlang:send_after(?IRGC_TIMEOUT, self(), inform_response_gc). - -irgc_stop(undefined) -> - ok; -irgc_stop(Ref) -> - (catch erlang:cancel_timer(Ref)). - - -handle_send_pdu(Pdu, Vsn, MsgData, Domain, Addr, Port, State) -> - Verbosity = get(verbosity), - spawn_opt(fun() -> - Log = worker_init(State, Verbosity), - Res = (catch maybe_handle_send_pdu( - Pdu, Vsn, MsgData, - Domain, Addr, Port, - State#state{log = Log})), - worker_exit(send_pdu, {Domain, Addr, Port}, Res) - end, - [monitor]). - -maybe_handle_send_pdu(Pdu, Vsn, MsgData, Domain, Addr, Port, - #state{filter = FilterMod} = State) -> - case (catch FilterMod:accept_send_pdu(Addr, Port, pdu_type_of(Pdu))) of - false -> - inc(netIfPduOutDrops), - ok; - _ -> - do_handle_send_pdu(Pdu, Vsn, MsgData, Domain, Addr, Port, State) - end. - -do_handle_send_pdu(Pdu, Vsn, MsgData, _Domain, Addr, Port, - #state{server = Pid, - note_store = NoteStore, - sock = Sock, - log = Log, - filter = FilterMod}) -> - Logger = logger(Log, write, Addr, Port), - case (catch snmpm_mpd:generate_msg(Vsn, NoteStore, - Pdu, MsgData, Logger)) of - {ok, Msg} -> - ?vtrace("do_handle_send_pdu -> message generated", []), - maybe_udp_send(FilterMod, Sock, Addr, Port, Msg); - {discarded, Reason} -> - ?vlog("PDU not sent: " - "~n PDU: ~p" - "~n Reason: ~p", [Pdu, Reason]), - Pid ! {snmp_error, Pdu, Reason}, - ok - end. - - -maybe_udp_send(FilterMod, Sock, Addr, Port, Msg) -> - case (catch FilterMod:accept_send(Addr, Port)) of - false -> - inc(netIfMsgOutDrops), - ok; - _ -> - udp_send(Sock, Addr, Port, Msg) - end. - - -udp_send(Sock, Addr, Port, Msg) -> - case (catch gen_udp:send(Sock, Addr, Port, Msg)) of - ok -> - ?vdebug("sent ~w bytes to ~w:~w [~w]", - [sz(Msg), Addr, Port, Sock]), - ok; - {error, Reason} -> - error_msg("failed sending message to ~p:~p: " - "~n ~p",[Addr, Port, Reason]), - ok; - Error -> - error_msg("failed sending message to ~p:~p: " - "~n ~p",[Addr, Port, Error]), - ok - end. - -sz(B) when is_binary(B) -> - size(B); -sz(L) when is_list(L) -> - length(L); -sz(_) -> - undefined. - - -handle_disk_log(_Log, {wrap, NoLostItems}, State) -> - ?vlog("Audit Trail Log - wrapped: ~w previously logged items where lost", - [NoLostItems]), - State; -handle_disk_log(_Log, {truncated, NoLostItems}, State) -> - ?vlog("Audit Trail Log - truncated: ~w items where lost when truncating", - [NoLostItems]), - State; -handle_disk_log(_Log, full, State) -> - error_msg("Failed to write to Audit Trail Log (full)", []), - State; -handle_disk_log(_Log, {error_status, ok}, State) -> - State; -handle_disk_log(_Log, {error_status, {error, Reason}}, State) -> - error_msg("Error status received from Audit Trail Log: " - "~n~p", [Reason]), - State; -handle_disk_log(_Log, _Info, State) -> - State. - - -%% mk_discovery_msg('version-3', Pdu, _VsnHdr, UserName) -> -%% ScopedPDU = #scopedPdu{contextEngineID = "", -%% contextName = "", -%% data = Pdu}, -%% Bytes = snmp_pdus:enc_scoped_pdu(ScopedPDU), -%% MsgID = get(msg_id), -%% put(msg_id,MsgID+1), -%% UsmSecParams = -%% #usmSecurityParameters{msgAuthoritativeEngineID = "", -%% msgAuthoritativeEngineBoots = 0, -%% msgAuthoritativeEngineTime = 0, -%% msgUserName = UserName, -%% msgPrivacyParameters = "", -%% msgAuthenticationParameters = ""}, -%% SecBytes = snmp_pdus:enc_usm_security_parameters(UsmSecParams), -%% PduType = Pdu#pdu.type, -%% Hdr = #v3_hdr{msgID = MsgID, -%% msgMaxSize = 1000, -%% msgFlags = snmp_misc:mk_msg_flags(PduType, 0), -%% msgSecurityModel = ?SEC_USM, -%% msgSecurityParameters = SecBytes}, -%% Msg = #message{version = 'version-3', vsn_hdr = Hdr, data = Bytes}, -%% case (catch snmp_pdus:enc_message_only(Msg)) of -%% {'EXIT', Reason} -> -%% error("Encoding error. Pdu: ~w. Reason: ~w",[Pdu, Reason]), -%% error; -%% L when list(L) -> -%% {Msg, L} -%% end; -%% mk_discovery_msg(Version, Pdu, {Com, _, _, _, _}, UserName) -> -%% Msg = #message{version = Version, vsn_hdr = Com, data = Pdu}, -%% case catch snmp_pdus:enc_message(Msg) of -%% {'EXIT', Reason} -> -%% error("Encoding error. Pdu: ~w. Reason: ~w",[Pdu, Reason]), -%% error; -%% L when list(L) -> -%% {Msg, L} -%% end. - - -%% mk_msg('version-3', Pdu, {Context, User, EngineID, CtxEngineId, SecLevel}, -%% MsgData) -> -%% %% Code copied from snmp_mpd.erl -%% {MsgId, SecName, SecData} = -%% if -%% tuple(MsgData), Pdu#pdu.type == 'get-response' -> -%% MsgData; -%% true -> -%% Md = get(msg_id), -%% put(msg_id, Md + 1), -%% {Md, User, []} -%% end, -%% ScopedPDU = #scopedPdu{contextEngineID = CtxEngineId, -%% contextName = Context, -%% data = Pdu}, -%% ScopedPDUBytes = snmp_pdus:enc_scoped_pdu(ScopedPDU), - -%% PduType = Pdu#pdu.type, -%% V3Hdr = #v3_hdr{msgID = MsgId, -%% msgMaxSize = 1000, -%% msgFlags = snmp_misc:mk_msg_flags(PduType, SecLevel), -%% msgSecurityModel = ?SEC_USM}, -%% Message = #message{version = 'version-3', vsn_hdr = V3Hdr, -%% data = ScopedPDUBytes}, -%% SecEngineID = case PduType of -%% 'get-response' -> snmp_framework_mib:get_engine_id(); -%% _ -> EngineID -%% end, -%% case catch snmp_usm:generate_outgoing_msg(Message, SecEngineID, -%% SecName, SecData, SecLevel) of -%% {'EXIT', Reason} -> -%% error("Encoding error. Pdu: ~w. Reason: ~w",[Pdu, Reason]), -%% error; -%% {error, Reason} -> -%% error("Encoding error. Pdu: ~w. Reason: ~w",[Pdu, Reason]), -%% error; -%% Packet -> -%% Packet -%% end; -%% mk_msg(Version, Pdu, {Com, _User, _EngineID, _Ctx, _SecLevel}, _SecData) -> -%% Msg = #message{version = Version, vsn_hdr = Com, data = Pdu}, -%% case catch snmp_pdus:enc_message(Msg) of -%% {'EXIT', Reason} -> -%% error("Encoding error. Pdu: ~w. Reason: ~w",[Pdu, Reason]), -%% error; -%% B when list(B) -> -%% B -%% end. - - -%% handle_system_info_updated(#state{log = {Log, _OldType}} = State, -%% audit_trail_log_type = _What) -> -%% %% Just to make sure, check that ATL is actually enabled -%% case snmpm_config:system_info(audit_trail_log) of -%% {ok, true} -> -%% {ok, Type} = snmpm_config:system_info(audit_trail_log_type), -%% NewState = State#state{log = {Log, Type}}, -%% {NewState, ok}; -%% _ -> -%% {State, {error, {adt_not_enabled}}} -%% end; -%% handle_system_info_updated(_State, _What) -> -%% ok. - -handle_get_log_type(#state{log = {_Log, Value}} = State) -> - %% Just to make sure, check that ATL is actually enabled - case snmpm_config:system_info(audit_trail_log) of - {ok, true} -> - Type = - case {lists:member(read, Value), lists:member(write, Value)} of - {true, true} -> - read_write; - {true, false} -> - read; - {false, true} -> - write; - {false, false} -> - throw({State, {error, {bad_atl_type, Value}}}) - end, - {ok, Type}; - _ -> - {error, not_enabled} - end; -handle_get_log_type(_State) -> - {error, not_enabled}. - -handle_set_log_type(#state{log = {Log, OldValue}} = State, NewType) -> - %% Just to make sure, check that ATL is actually enabled - case snmpm_config:system_info(audit_trail_log) of - {ok, true} -> - NewValue = - case NewType of - read -> - [read]; - write -> - [write]; - read_write -> - [read,write]; - _ -> - throw({State, {error, {bad_atl_type, NewType}}}) - end, - NewState = State#state{log = {Log, NewValue}}, - OldType = - case {lists:member(read, OldValue), - lists:member(write, OldValue)} of - {true, true} -> - read_write; - {true, false} -> - read; - {false, true} -> - write; - {false, false} -> - throw({State, {error, {bad_atl_type, OldValue}}}) - end, - {NewState, {ok, OldType}}; - _ -> - {State, {error, not_enabled}} - end; -handle_set_log_type(State, _NewType) -> - {State, {error, not_enabled}}. - - -%% ------------------------------------------------------------------- - -worker_init(#state{log = undefined = Log}, Verbosity) -> - worker_init2(Log, Verbosity); -worker_init(#state{log = {Name, Log, Type}}, Verbosity) -> - case snmp_log:open(Name, Log) of - {ok, NewLog} -> - worker_init2({Name, NewLog, Type}, Verbosity); - {error, Reason} -> - warning_msg("NetIf worker ~p failed opening ATL: " - "~n ~p", [self(), Reason]), - NewLog = undefined, - worker_init2({Name, NewLog, Type}, Verbosity) - end; -worker_init(State, Verbosity) -> - ?vinfo("worker_init -> entry with invalid data: " - "~n State: ~p" - "~n Verbosity: ~p", [State, Verbosity]), - exit({worker_init, State, Verbosity}). - -worker_init2(Log, Verbosity) -> - put(sname, mnifw), - put(verbosity, Verbosity), - Log. - - -worker_exit(Tag, Info, Result) -> - exit({net_if_worker, {Tag, Info, Result}}). - -handle_worker_exit(_, {_, _, ok}) -> - ok; -handle_worker_exit(Pid, {udp, {Addr, Port}, ExitStatus}) -> - warning_msg("Worker process (~p) terminated " - "while processing (incomming) message from ~w:~w: " - "~n~p", [Pid, Addr, Port, ExitStatus]), - ok; -handle_worker_exit(Pid, {send_pdu, {Domain, Addr, Port}, ExitStatus}) -> - warning_msg("Worker process (~p) terminated " - "while processing (outgoing) pdu for [~w] ~w:~w: " - "~n~p", [Pid, Domain, Addr, Port, ExitStatus]), - ok; -handle_worker_exit(Pid, {inform_response, {Addr, Port}, ExitStatus}) -> - warning_msg("Worker process (~p) terminated " - "while processing (outgoing) inform response for ~w:~w: " - "~n~p", [Pid, Addr, Port, ExitStatus]), - ok; -handle_worker_exit(_, _) -> - ok. - - -%% ------------------------------------------------------------------- - -make_response_pdu(#pdu{request_id = ReqId, varbinds = Vbs}) -> - #pdu{type = 'get-response', - request_id = ReqId, - error_status = noError, - error_index = 0, - varbinds = Vbs}. - - -%% ---------------------------------------------------------------- - -pdu_type_of(#pdu{type = Type}) -> - Type; -pdu_type_of(TrapPdu) when is_record(TrapPdu, trappdu) -> - trap. - - -%% ------------------------------------------------------------------- - -%% At this point this function is used during testing -maybe_process_extra_info(?DEFAULT_EXTRA_INFO) -> - ok; -maybe_process_extra_info({?SNMPM_EXTRA_INFO_TAG, Fun}) - when is_function(Fun, 0) -> - (catch Fun()), - ok; -maybe_process_extra_info(_ExtraInfo) -> - ok. - - -%% ------------------------------------------------------------------- - -t() -> - {A,B,C} = erlang:now(), - A*1000000000+B*1000+(C div 1000). - - -%% ------------------------------------------------------------------- - -logger(undefined, _Type, _Addr, _Port) -> - fun(_) -> - ok - end; -logger({_Name, Log, Types}, Type, Addr, Port) -> - case lists:member(Type, Types) of - true -> - fun(Msg) -> - snmp_log:log(Log, Msg, Addr, Port) - end; - false -> - fun(_) -> - ok - end - end. - - -%% ------------------------------------------------------------------- - -%% info_msg(F, A) -> -%% ?snmpm_info("NET-IF server: " ++ F, A). - -warning_msg(F, A) -> - ?snmpm_warning("NET-IF server: " ++ F, A). - -error_msg(F, A) -> - ?snmpm_error("NET-IF server: " ++ F, A). - - - -%%%------------------------------------------------------------------- - -% get_opt(Key, Opts) -> -% ?vtrace("get option ~w", [Key]), -% snmp_misc:get_option(Key, Opts). - -get_opt(Opts, Key, Def) -> - ?vtrace("get option ~w with default ~p", [Key, Def]), - snmp_misc:get_option(Key, Opts, Def). - - -%% ------------------------------------------------------------------- - -get_info(#state{sock = Id}) -> - ProcSize = proc_mem(self()), - PortInfo = get_port_info(Id), - [{process_memory, ProcSize}, {port_info, PortInfo}]. - -proc_mem(P) when is_pid(P) -> - case (catch erlang:process_info(P, memory)) of - {memory, Sz} when is_integer(Sz) -> - Sz; - _ -> - undefined - end. -%% proc_mem(_) -> -%% undefined. - - -get_port_info(Id) -> - PortInfo = - case (catch erlang:port_info(Id)) of - PI when is_list(PI) -> - [{port_info, PI}]; - _ -> - [] - end, - PortStatus = - case (catch prim_inet:getstatus(Id)) of - {ok, PS} -> - [{port_status, PS}]; - _ -> - [] - end, - PortAct = - case (catch inet:getopts(Id, [active])) of - {ok, PA} -> - [{port_act, PA}]; - _ -> - [] - end, - PortStats = - case (catch inet:getstat(Id)) of - {ok, Stat} -> - [{port_stats, Stat}]; - _ -> - [] - end, - IfList = - case (catch inet:getif(Id)) of - {ok, IFs} -> - [{interfaces, IFs}]; - _ -> - [] - end, - BufSz = - case (catch inet:getopts(Id, [recbuf, sndbuf, buffer])) of - {ok, Sz} -> - [{buffer_size, Sz}]; - _ -> - [] - end, - [{socket, Id}] ++ - IfList ++ - PortStats ++ - PortInfo ++ - PortStatus ++ - PortAct ++ - BufSz. - - -%%----------------------------------------------------------------- -%% Counter functions -%%----------------------------------------------------------------- -init_counters() -> - F = fun(Counter) -> maybe_create_counter(Counter) end, - lists:map(F, counters()). - -reset_counters() -> - F = fun(Counter) -> snmpm_config:reset_stats_counter(Counter) end, - lists:map(F, counters()). - -maybe_create_counter(Counter) -> - snmpm_config:maybe_cre_stats_counter(Counter, 0). - -counters() -> - [ - netIfMsgOutDrops, - netIfMsgInDrops, - netIfPduOutDrops, - netIfPduInDrops - ]. - -inc(Name) -> inc(Name, 1). -inc(Name, N) -> snmpm_config:incr_stats_counter(Name, N). - -%% get_counters() -> -%% Counters = counters(), -%% get_counters(Counters, []). - -%% get_counters([], Acc) -> -%% lists:reverse(Acc); -%% get_counters([Counter|Counters], Acc) -> -%% case snmpm_config:get_stats_counter(Counter) of -%% {ok, CounterVal} -> -%% get_counters(Counters, [{Counter, CounterVal}|Acc]); -%% _ -> -%% get_counters(Counters, Acc) -%% end. - - -%% ---------------------------------------------------------------- - -call(Pid, Req) -> - call(Pid, Req, infinity). - -call(Pid, Req, Timeout) -> - gen_server:call(Pid, Req, Timeout). - -cast(Pid, Msg) -> - gen_server:cast(Pid, Msg). +-define(snmpm_net_if_mt, true). +-module(snmpm_net_if_mt). +-include("snmpm_net_if.erl"). diff --git a/lib/snmp/src/manager/snmpm_server.erl b/lib/snmp/src/manager/snmpm_server.erl index 9c79df2748..a75122d0bb 100644 --- a/lib/snmp/src/manager/snmpm_server.erl +++ b/lib/snmp/src/manager/snmpm_server.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2013. All Rights Reserved. +%% Copyright Ericsson AB 2004-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -163,8 +163,7 @@ reg_type, target, domain, - addr, - port, + address, type, data, ref, @@ -1033,14 +1032,14 @@ handle_info({snmp_error, Pdu, Reason}, State) -> handle_snmp_error(Pdu, Reason, State), {noreply, State}; -handle_info({snmp_error, Reason, Addr, Port}, State) -> +handle_info({snmp_error, Reason, Domain, Addr}, State) -> ?vlog("received snmp_error message", []), - handle_snmp_error(Addr, Port, -1, Reason, State), + handle_snmp_error(Domain, Addr, -1, Reason, State), {noreply, State}; -handle_info({snmp_error, ReqId, Reason, Addr, Port}, State) -> +handle_info({snmp_error, ReqId, Reason, Domain, Addr}, State) -> ?vlog("received snmp_error message", []), - handle_snmp_error(Addr, Port, ReqId, Reason, State), + handle_snmp_error(Domain, Addr, ReqId, Reason, State), {noreply, State}; %% handle_info({snmp_error, ReqId, Pdu, Reason, Addr, Port}, State) -> @@ -1049,30 +1048,30 @@ handle_info({snmp_error, ReqId, Reason, Addr, Port}, State) -> %% {noreply, State}; -handle_info({snmp_pdu, Pdu, Addr, Port}, State) -> +handle_info({snmp_pdu, Pdu, Domain, Addr}, State) -> ?vlog("received snmp_pdu message", []), - handle_snmp_pdu(Pdu, Addr, Port, State), + handle_snmp_pdu(Pdu, Domain, Addr, State), {noreply, State}; -handle_info({snmp_trap, Trap, Addr, Port}, State) -> +handle_info({snmp_trap, Trap, Domain, Addr}, State) -> ?vlog("received snmp_trap message", []), - handle_snmp_trap(Trap, Addr, Port, State), + handle_snmp_trap(Trap, Domain, Addr, State), {noreply, State}; -handle_info({snmp_inform, Ref, Pdu, Addr, Port}, State) -> +handle_info({snmp_inform, Ref, Pdu, Domain, Addr}, State) -> ?vlog("received snmp_inform message", []), - handle_snmp_inform(Ref, Pdu, Addr, Port, State), + handle_snmp_inform(Ref, Pdu, Domain, Addr, State), {noreply, State}; -handle_info({snmp_report, {ok, Pdu}, Addr, Port}, State) -> - handle_snmp_report(Pdu, Addr, Port, State), +handle_info({snmp_report, {ok, Pdu}, Domain, Addr}, State) -> + handle_snmp_report(Pdu, Domain, Addr, State), {noreply, State}; -handle_info({snmp_report, {error, ReqId, Info, Pdu}, Addr, Port}, State) -> - handle_snmp_report(ReqId, Pdu, Info, Addr, Port, State), +handle_info({snmp_report, {error, ReqId, Info, Pdu}, Domain, Addr}, State) -> + handle_snmp_report(ReqId, Pdu, Info, Domain, Addr, State), {noreply, State}; @@ -1176,11 +1175,11 @@ handle_sync_get(Pid, UserId, TargetName, Oids, SendOpts, From, State) -> "~n From: ~p", [Pid, UserId, TargetName, Oids, SendOpts, From]), case agent_data(TargetName, SendOpts) of - {ok, RegType, Domain, Addr, Port, Vsn, MsgData} -> + {ok, RegType, Domain, Addr, Vsn, MsgData} -> ?vtrace("handle_sync_get -> send a ~p message", [Vsn]), Extra = ?GET_EXTRA(SendOpts), ReqId = send_get_request(Oids, Vsn, MsgData, - Domain, Addr, Port, + Domain, Addr, Extra, State), ?vdebug("handle_sync_get -> ReqId: ~p", [ReqId]), Msg = {sync_timeout, ReqId, From}, @@ -1193,8 +1192,7 @@ handle_sync_get(Pid, UserId, TargetName, Oids, SendOpts, From, State) -> reg_type = RegType, target = TargetName, domain = Domain, - addr = Addr, - port = Port, + address = Addr, type = get, data = MsgData, ref = Ref, @@ -1230,11 +1228,11 @@ handle_sync_get_next(Pid, UserId, TargetName, Oids, SendOpts, "~n From: ~p", [Pid, UserId, TargetName, Oids, SendOpts, From]), case agent_data(TargetName, SendOpts) of - {ok, RegType, Domain, Addr, Port, Vsn, MsgData} -> + {ok, RegType, Domain, Addr, Vsn, MsgData} -> ?vtrace("handle_sync_get_next -> send a ~p message", [Vsn]), Extra = ?GET_EXTRA(SendOpts), ReqId = send_get_next_request(Oids, Vsn, MsgData, - Domain, Addr, Port, + Domain, Addr, Extra, State), ?vdebug("handle_sync_get_next -> ReqId: ~p", [ReqId]), Msg = {sync_timeout, ReqId, From}, @@ -1247,12 +1245,11 @@ handle_sync_get_next(Pid, UserId, TargetName, Oids, SendOpts, reg_type = RegType, target = TargetName, domain = Domain, - addr = Addr, - port = Port, + address = Addr, type = get_next, data = MsgData, ref = Ref, - mon = MonRef, + mon = MonRef, from = From}, ets:insert(snmpm_request_table, Req), ok; @@ -1290,11 +1287,11 @@ handle_sync_get_bulk(Pid, UserId, TargetName, NonRep, MaxRep, Oids, SendOpts, "~n From: ~p", [Pid, UserId, TargetName, NonRep, MaxRep, Oids, SendOpts, From]), case agent_data(TargetName, SendOpts) of - {ok, RegType, Domain, Addr, Port, Vsn, MsgData} -> + {ok, RegType, Domain, Addr, Vsn, MsgData} -> ?vtrace("handle_sync_get_bulk -> send a ~p message", [Vsn]), Extra = ?GET_EXTRA(SendOpts), ReqId = send_get_bulk_request(Oids, Vsn, MsgData, - Domain, Addr, Port, + Domain, Addr, NonRep, MaxRep, Extra, State), ?vdebug("handle_sync_get_bulk -> ReqId: ~p", [ReqId]), Msg = {sync_timeout, ReqId, From}, @@ -1307,8 +1304,7 @@ handle_sync_get_bulk(Pid, UserId, TargetName, NonRep, MaxRep, Oids, SendOpts, reg_type = RegType, target = TargetName, domain = Domain, - addr = Addr, - port = Port, + address = Addr, type = get_bulk, data = MsgData, ref = Ref, @@ -1346,11 +1342,11 @@ handle_sync_set(Pid, UserId, TargetName, VarsAndVals, SendOpts, From, State) -> "~n From: ~p", [Pid, UserId, TargetName, VarsAndVals, From]), case agent_data(TargetName, SendOpts) of - {ok, RegType, Domain, Addr, Port, Vsn, MsgData} -> + {ok, RegType, Domain, Addr, Vsn, MsgData} -> ?vtrace("handle_sync_set -> send a ~p message", [Vsn]), Extra = ?GET_EXTRA(SendOpts), ReqId = send_set_request(VarsAndVals, Vsn, MsgData, - Domain, Addr, Port, + Domain, Addr, Extra, State), ?vdebug("handle_sync_set -> ReqId: ~p", [ReqId]), Msg = {sync_timeout, ReqId, From}, @@ -1363,8 +1359,7 @@ handle_sync_set(Pid, UserId, TargetName, VarsAndVals, SendOpts, From, State) -> reg_type = RegType, target = TargetName, domain = Domain, - addr = Addr, - port = Port, + address = Addr, type = set, data = MsgData, ref = Ref, @@ -1400,11 +1395,11 @@ handle_async_get(Pid, UserId, TargetName, Oids, SendOpts, State) -> "~n SendOpts: ~p", [Pid, UserId, TargetName, Oids, SendOpts]), case agent_data(TargetName, SendOpts) of - {ok, RegType, Domain, Addr, Port, Vsn, MsgData} -> + {ok, RegType, Domain, Addr, Vsn, MsgData} -> ?vtrace("handle_async_get -> send a ~p message", [Vsn]), Extra = ?GET_EXTRA(SendOpts), ReqId = send_get_request(Oids, Vsn, MsgData, - Domain, Addr, Port, + Domain, Addr, Extra, State), ?vdebug("handle_async_get -> ReqId: ~p", [ReqId]), Expire = ?ASYNC_GET_TIMEOUT(SendOpts), @@ -1413,8 +1408,7 @@ handle_async_get(Pid, UserId, TargetName, Oids, SendOpts, State) -> reg_type = RegType, target = TargetName, domain = Domain, - addr = Addr, - port = Port, + address = Addr, type = get, data = MsgData, expire = t() + Expire}, @@ -1450,11 +1444,11 @@ handle_async_get_next(Pid, UserId, TargetName, Oids, SendOpts, State) -> "~n SendOpts: ~p", [Pid, UserId, TargetName, Oids, SendOpts]), case agent_data(TargetName, SendOpts) of - {ok, RegType, Domain, Addr, Port, Vsn, MsgData} -> + {ok, RegType, Domain, Addr, Vsn, MsgData} -> ?vtrace("handle_async_get_next -> send a ~p message", [Vsn]), Extra = ?GET_EXTRA(SendOpts), ReqId = send_get_next_request(Oids, Vsn, MsgData, - Domain, Addr, Port, + Domain, Addr, Extra, State), ?vdebug("handle_async_get_next -> ReqId: ~p", [ReqId]), Expire = ?ASYNC_GET_NEXT_TIMEOUT(SendOpts), @@ -1463,8 +1457,7 @@ handle_async_get_next(Pid, UserId, TargetName, Oids, SendOpts, State) -> reg_type = RegType, target = TargetName, domain = Domain, - addr = Addr, - port = Port, + address = Addr, type = get_next, data = MsgData, expire = t() + Expire}, @@ -1507,11 +1500,11 @@ handle_async_get_bulk(Pid, "~n SendOpts: ~p", [Pid, UserId, TargetName, NonRep, MaxRep, Oids, SendOpts]), case agent_data(TargetName, SendOpts) of - {ok, RegType, Domain, Addr, Port, Vsn, MsgData} -> + {ok, RegType, Domain, Addr, Vsn, MsgData} -> ?vtrace("handle_async_get_bulk -> send a ~p message", [Vsn]), Extra = ?GET_EXTRA(SendOpts), ReqId = send_get_bulk_request(Oids, Vsn, MsgData, - Domain, Addr, Port, + Domain, Addr, NonRep, MaxRep, Extra, State), ?vdebug("handle_async_get_bulk -> ReqId: ~p", [ReqId]), Expire = ?ASYNC_GET_BULK_TIMEOUT(SendOpts), @@ -1520,8 +1513,7 @@ handle_async_get_bulk(Pid, reg_type = RegType, target = TargetName, domain = Domain, - addr = Addr, - port = Port, + address = Addr, type = get_bulk, data = MsgData, expire = t() + Expire}, @@ -1556,11 +1548,11 @@ handle_async_set(Pid, UserId, TargetName, VarsAndVals, SendOpts, State) -> "~n SendOpts: ~p", [Pid, UserId, TargetName, VarsAndVals, SendOpts]), case agent_data(TargetName, SendOpts) of - {ok, RegType, Domain, Addr, Port, Vsn, MsgData} -> + {ok, RegType, Domain, Addr, Vsn, MsgData} -> ?vtrace("handle_async_set -> send a ~p message", [Vsn]), Extra = ?GET_EXTRA(SendOpts), ReqId = send_set_request(VarsAndVals, Vsn, MsgData, - Domain, Addr, Port, + Domain, Addr, Extra, State), ?vdebug("handle_async_set -> ReqId: ~p", [ReqId]), Expire = ?ASYNC_SET_TIMEOUT(SendOpts), @@ -1569,8 +1561,7 @@ handle_async_set(Pid, UserId, TargetName, VarsAndVals, SendOpts, State) -> reg_type = RegType, target = TargetName, domain = Domain, - addr = Addr, - port = Port, + address = Addr, type = set, data = MsgData, expire = t() + Expire}, @@ -1808,15 +1799,15 @@ handle_snmp_error(CrapError, Reason, _State) -> "~n~p~n~p", [CrapError, Reason]), ok. -handle_snmp_error(Addr, Port, ReqId, Reason, State) -> +handle_snmp_error(Domain, Addr, ReqId, Reason, State) -> - ?vtrace("handle_snmp_error -> entry with" - "~n Addr: ~p" - "~n Port: ~p" - "~n ReqId: ~p" - "~n Reason: ~p", [Addr, Port, ReqId, Reason]), + ?vtrace("handle_snmp_error -> entry with~n" + " Domain: ~p~n" + " Addr: ~p~n" + " ReqId: ~p~n" + " Reason: ~p", [Domain, Addr, ReqId, Reason]), - case snmpm_config:get_agent_user_id(Addr, Port) of + case snmpm_config:get_agent_user_id(Domain, Addr) of {ok, UserId} -> case snmpm_config:user_info(UserId) of {ok, UserMod, UserData} -> @@ -1831,7 +1822,7 @@ handle_snmp_error(Addr, Port, ReqId, Reason, State) -> error_msg("failed retreiving the default user " "info handling snmp error " "<~p,~p>: ~n~w~n~w", - [Addr, Port, ReqId, Reason]) + [Domain, Addr, ReqId, Reason]) end end; _Error -> @@ -1843,7 +1834,7 @@ handle_snmp_error(Addr, Port, ReqId, Reason, State) -> error_msg("failed retreiving the default user " "info handling snmp error " "<~p,~p>: ~n~w~n~w", - [Addr, Port, ReqId, Reason]) + [Domain, Addr, ReqId, Reason]) end end. @@ -1867,12 +1858,12 @@ handle_error(_UserId, Mod, Reason, ReqId, Data, _State) -> handle_snmp_pdu(#pdu{type = 'get-response', request_id = ReqId} = Pdu, - Addr, Port, State) -> + Domain, Addr, State) -> - ?vtrace("handle_snmp_pdu(get-response) -> entry with" - "~n Addr: ~p" - "~n Port: ~p" - "~n Pdu: ~p", [Addr, Port, Pdu]), + ?vtrace("handle_snmp_pdu(get-response) -> entry with~n" + " Domain: ~p~n" + " Addr: ~p~n" + " Pdu: ~p", [Domain, Addr, Pdu]), case ets:lookup(snmpm_request_table, ReqId) of @@ -1902,9 +1893,10 @@ handle_snmp_pdu(#pdu{type = 'get-response', request_id = ReqId} = Pdu, SnmpResponse = {EStatus, EIndex, Varbinds2}, case snmpm_config:user_info(UserId) of {ok, UserMod, UserData} -> - handle_pdu(UserId, UserMod, - RegType, Target, Addr, Port, - ReqId, SnmpResponse, UserData, State), + handle_pdu( + UserId, UserMod, + RegType, Target, Domain, Addr, + ReqId, SnmpResponse, UserData, State), maybe_delete(Disco, ReqId); _Error -> %% reply to outstanding request, for which there is no @@ -1912,15 +1904,16 @@ handle_snmp_pdu(#pdu{type = 'get-response', request_id = ReqId} = Pdu, %% Therefor send it to the default user case snmpm_config:user_info() of {ok, DefUserId, DefMod, DefData} -> - handle_pdu(DefUserId, DefMod, - RegType, Target, Addr, Port, - ReqId, SnmpResponse, DefData, State), + handle_pdu( + DefUserId, DefMod, + RegType, Target, Domain, Addr, + ReqId, SnmpResponse, DefData, State), maybe_delete(Disco, ReqId); Error -> error_msg("failed retreiving the default user " "info handling pdu from " "~p <~p,~p>: ~n~w~n~w", - [Target, Addr, Port, Error, Pdu]) + [Target, Domain, Addr, Error, Pdu]) end end; @@ -1974,7 +1967,7 @@ handle_snmp_pdu(#pdu{type = 'get-response', request_id = ReqId} = Pdu, varbinds = Varbinds} = Pdu, Varbinds2 = fix_vbs_BITS(Varbinds), SnmpInfo = {EStatus, EIndex, Varbinds2}, - case snmpm_config:get_agent_user_id(Addr, Port) of + case snmpm_config:get_agent_user_id(Domain, Addr) of {ok, UserId} -> %% A very late reply or a reply to a request %% that has been cancelled. @@ -1999,7 +1992,7 @@ handle_snmp_pdu(#pdu{type = 'get-response', request_id = ReqId} = Pdu, "user info handling (old) " "pdu from " "<~p,~p>: ~n~w~n~w", - [Addr, Port, Error, Pdu]) + [Domain, Addr, Error, Pdu]) end end; @@ -2016,28 +2009,30 @@ handle_snmp_pdu(#pdu{type = 'get-response', request_id = ReqId} = Pdu, "no agent info found", []), case snmpm_config:user_info() of {ok, DefUserId, DefMod, DefData} -> - handle_agent(DefUserId, DefMod, - Addr, Port, - pdu, ignore, - SnmpInfo, DefData, State); + handle_agent( + DefUserId, DefMod, + Domain, Addr, + pdu, ignore, + SnmpInfo, DefData, State); Error -> error_msg("failed retreiving the default user " "info handling (old) pdu when no user " "found from " "<~p,~p>: ~n~w~n~w", - [Addr, Port, Error, Pdu]) + [Domain, Addr, Error, Pdu]) end end end; -handle_snmp_pdu(CrapPdu, Addr, Port, _State) -> +handle_snmp_pdu(CrapPdu, Domain, Addr, _State) -> error_msg("received crap (snmp) Pdu from ~w:~w =>" - "~p", [Addr, Port, CrapPdu]), + "~p", [Domain, Addr, CrapPdu]), ok. -handle_pdu(_UserId, Mod, target_name = _RegType, TargetName, _Addr, _Port, - ReqId, SnmpResponse, Data, _State) -> +handle_pdu( + _UserId, Mod, target_name = _RegType, TargetName, _Domain, _Addr, + ReqId, SnmpResponse, Data, _State) -> ?vtrace("handle_pdu(target_name) -> entry when" "~n Mod: ~p", [Mod]), F = fun() -> @@ -2053,43 +2048,56 @@ handle_pdu(_UserId, Mod, target_name = _RegType, TargetName, _Addr, _Port, end, handle_callback(F), ok; -handle_pdu(_UserId, Mod, addr_port = _RegType, _TargetName, Addr, Port, - ReqId, SnmpResponse, Data, _State) -> +handle_pdu( + _UserId, Mod, addr_port = _RegType, _TargetName, _Domain, Addr, + ReqId, SnmpResponse, Data, _State) -> ?vtrace("handle_pdu(addr_port) -> entry when" "~n Mod: ~p", [Mod]), F = fun() -> - (catch Mod:handle_pdu(Addr, Port, ReqId, SnmpResponse, Data)) + {Ip, Port} = Addr, + (catch Mod:handle_pdu(Ip, Port, ReqId, SnmpResponse, Data)) end, handle_callback(F), ok. -handle_agent(UserId, Mod, Addr, Port, Type, Ref, SnmpInfo, Data, State) -> +handle_agent(UserId, Mod, Domain, Addr, Type, Ref, SnmpInfo, Data, State) -> ?vtrace("handle_agent -> entry when" "~n UserId: ~p" "~n Type: ~p" "~n Mod: ~p", [UserId, Type, Mod]), F = fun() -> - do_handle_agent(UserId, Mod, Addr, Port, + do_handle_agent(UserId, Mod, Domain, Addr, Type, Ref, SnmpInfo, Data, State) end, handle_callback(F), ok. do_handle_agent(DefUserId, DefMod, - Addr, Port, + Domain, Addr, Type, Ref, SnmpInfo, DefData, State) -> ?vdebug("do_handle_agent -> entry when" "~n DefUserId: ~p", [DefUserId]), - try DefMod:handle_agent(Addr, Port, Type, SnmpInfo, DefData) of + {Domain_or_Ip, Addr_or_Port} = + case Domain of + snmpUDPDomain -> + Addr; + _ -> + {Domain, Addr} + end, + try DefMod:handle_agent( + Domain_or_Ip, Addr_or_Port, Type, SnmpInfo, DefData) + of {register, UserId2, TargetName, Config} -> ?vtrace("do_handle_agent -> register: " "~n UserId2: ~p" "~n TargetName: ~p" "~n Config: ~p", [UserId2, TargetName, Config]), - Config2 = ensure_present([{address, Addr}, {port, Port}], Config), + Config2 = + ensure_present( + [{tdomain, Domain}, {taddress, Addr}], Config), Config3 = [{reg_type, target_name} | Config2], case snmpm_config:register_agent(UserId2, TargetName, Config3) of @@ -2099,7 +2107,7 @@ do_handle_agent(DefUserId, DefMod, error_msg("failed registering agent - " "handling agent " "~p <~p,~p>: ~n~w", - [TargetName, Addr, Port, Reason]), + [TargetName, Domain, Addr, Reason]), ok end; @@ -2108,31 +2116,33 @@ do_handle_agent(DefUserId, DefMod, ok; InvalidResult -> - CallbackArgs = [Addr, Port, Type, SnmpInfo, DefData], + CallbackArgs = [Domain, Addr, Type, SnmpInfo, DefData], handle_invalid_result(handle_agent, CallbackArgs, InvalidResult) catch error:{undef, _} when Type =:= pdu -> %% Maybe, still on the old API ?vdebug("do_handle_agent -> maybe still on the old api", []), - case (catch DefMod:handle_agent(Addr, Port, SnmpInfo, DefData)) of + {Ip, Port} = Addr, + case (catch DefMod:handle_agent(Ip, Port, SnmpInfo, DefData)) of {register, UserId2, Config} -> ?vtrace("do_handle_agent -> register: " "~n UserId2: ~p" "~n Config: ~p", [UserId2, Config]), - TargetName = mk_target_name(Addr, Port, Config), - Config2 = [{reg_type, addr_port}, - {address, Addr}, - {port, Port} | Config], - case snmpm_config:register_agent(UserId2, - TargetName, Config2) of + TargetName = mk_target_name(Domain, Addr, Config), + Config2 = + ensure_present( + [{tdomain, Domain}, {taddress, Addr}], Config), + Config3 = [{reg_type, addr_port} | Config2], + case snmpm_config:register_agent( + UserId2, TargetName, Config3) of ok -> ok; {error, Reason} -> error_msg("failed registering agent - " "handling agent " "~p <~p,~p>: ~n~w", - [TargetName, Addr, Port, Reason]), + [TargetName, Domain, Addr, Reason]), ok end; {register, UserId2, TargetName, Config} -> @@ -2141,18 +2151,19 @@ do_handle_agent(DefUserId, DefMod, "~n TargetName: ~p" "~n Config: ~p", [UserId2, TargetName, Config]), - Config2 = ensure_present([{address, Addr}, {port, Port}], - Config), + Config2 = + ensure_present( + [{tdomain, Domain}, {taddress, Addr}], Config), Config3 = [{reg_type, target_name} | Config2], - case snmpm_config:register_agent(UserId2, - TargetName, Config3) of + case snmpm_config:register_agent( + UserId2, TargetName, Config3) of ok -> ok; {error, Reason} -> error_msg("failed registering agent - " "handling agent " "~p <~p,~p>: ~n~w", - [TargetName, Addr, Port, Reason]), + [TargetName, Domain, Addr, Reason]), ok end; _Ignore -> @@ -2170,34 +2181,38 @@ do_handle_agent(DefUserId, DefMod, %% Backward compatibillity crap RegType = target_name, - Target = mk_target_name(Addr, Port, default_agent_config()), + Target = mk_target_name(Domain, Addr, default_agent_config()), case Type of report -> SnmpInform = SnmpInfo, - handle_report(DefUserId, DefMod, - RegType, Target, Addr, Port, - SnmpInform, DefData, State); + handle_report( + DefUserId, DefMod, + RegType, Target, Domain, Addr, + SnmpInform, DefData, State); inform -> SnmpInform = SnmpInfo, - handle_inform(DefUserId, DefMod, Ref, - RegType, Target, Addr, Port, - SnmpInform, DefData, State); + handle_inform( + DefUserId, DefMod, Ref, + RegType, Target, Domain, Addr, + SnmpInform, DefData, State); trap -> SnmpTrapInfo = SnmpInfo, - handle_trap(DefUserId, DefMod, - RegType, Target, Addr, Port, - SnmpTrapInfo, DefData, State); + handle_trap( + DefUserId, DefMod, + RegType, Target, Domain, Addr, + SnmpTrapInfo, DefData, State); _ -> - error_msg("failed delivering ~w info to default user - " - "regarding agent " - "<~p,~p>: ~n~w", [Type, Addr, Port, SnmpInfo]) + error_msg( + "failed delivering ~w info to default user - " + "regarding agent " + "<~p,~p>: ~n~w", [Type, Domain, Addr, SnmpInfo]) end; T:E -> - CallbackArgs = [Addr, Port, Type, SnmpInfo, DefData], + CallbackArgs = [Domain, Addr, Type, SnmpInfo, DefData], handle_invalid_result(handle_agent, CallbackArgs, T, E) end. @@ -2215,50 +2230,51 @@ ensure_present([{Key, _Val} = Elem|Ensure], Config) -> %% Retrieve user info for this agent. %% If this is an unknown agent, then use the default user -handle_snmp_trap(#trappdu{enterprise = Enteprise, - generic_trap = Generic, - specific_trap = Spec, - time_stamp = Timestamp, - varbinds = Varbinds} = Trap, - Addr, Port, State) -> - - ?vtrace("handle_snmp_trap [trappdu] -> entry with" - "~n Addr: ~p" - "~n Port: ~p" - "~n Trap: ~p", [Addr, Port, Trap]), +handle_snmp_trap( + #trappdu{enterprise = Enteprise, + generic_trap = Generic, + specific_trap = Spec, + time_stamp = Timestamp, + varbinds = Varbinds} = Trap, Domain, Addr, State) -> + + ?vtrace("handle_snmp_trap [trappdu] -> entry with~n" + " Domain: ~p~n" + " Addr: ~p~n" + " Trap: ~p", [Domain, Addr, Trap]), Varbinds2 = fix_vbs_BITS(Varbinds), SnmpTrapInfo = {Enteprise, Generic, Spec, Timestamp, Varbinds2}, - do_handle_snmp_trap(SnmpTrapInfo, Addr, Port, State); + do_handle_snmp_trap(SnmpTrapInfo, Domain, Addr, State); handle_snmp_trap(#pdu{error_status = EStatus, error_index = EIndex, varbinds = Varbinds} = Trap, - Addr, Port, State) -> + Domain, Addr, State) -> - ?vtrace("handle_snmp_trap [pdu] -> entry with" - "~n Addr: ~p" - "~n Port: ~p" - "~n Trap: ~p", [Addr, Port, Trap]), + ?vtrace("handle_snmp_trap [pdu] -> entry with~n" + " Domain: ~p~n" + " Addr: ~p~n" + " Trap: ~p", [Domain, Addr, Trap]), Varbinds2 = fix_vbs_BITS(Varbinds), SnmpTrapInfo = {EStatus, EIndex, Varbinds2}, - do_handle_snmp_trap(SnmpTrapInfo, Addr, Port, State); + do_handle_snmp_trap(SnmpTrapInfo, Domain, Addr, State); -handle_snmp_trap(CrapTrap, Addr, Port, _State) -> +handle_snmp_trap(CrapTrap, Domain, Addr, _State) -> error_msg("received crap (snmp) trap from ~w:~w =>" - "~p", [Addr, Port, CrapTrap]), + "~p", [Domain, Addr, CrapTrap]), ok. -do_handle_snmp_trap(SnmpTrapInfo, Addr, Port, State) -> - case snmpm_config:get_agent_user_info(Addr, Port) of +do_handle_snmp_trap(SnmpTrapInfo, Domain, Addr, State) -> + case snmpm_config:get_agent_user_info(Domain, Addr) of {ok, UserId, Target, RegType} -> ?vtrace("handle_snmp_trap -> found user: ~p", [UserId]), case snmpm_config:user_info(UserId) of {ok, Mod, Data} -> - handle_trap(UserId, Mod, - RegType, Target, Addr, Port, - SnmpTrapInfo, Data, State); + handle_trap( + UserId, Mod, + RegType, Target, Domain, Addr, + SnmpTrapInfo, Data, State); Error1 -> %% User no longer exists, unregister agent @@ -2270,66 +2286,72 @@ do_handle_snmp_trap(SnmpTrapInfo, Addr, Port, State) -> %% Try use the default user case snmpm_config:user_info() of {ok, DefUserId, DefMod, DefData} -> - handle_agent(DefUserId, DefMod, - Addr, Port, - trap, ignore, - SnmpTrapInfo, DefData, State); + handle_agent( + DefUserId, DefMod, + Domain, Addr, + trap, ignore, + SnmpTrapInfo, DefData, State); Error2 -> - error_msg("failed retreiving the default " - "user info handling report from " - "~p <~p,~p>: ~n~w~n~w", - [Target, Addr, Port, - Error2, SnmpTrapInfo]) + error_msg( + "failed retreiving the default " + "user info handling report from " + "~p <~p,~p>: ~n~w~n~w", + [Target, Domain, Addr, + Error2, SnmpTrapInfo]) end; Error3 -> %% Failed unregister agent, %% now its getting messy... - warning_msg("failed unregister agent ~p <~p,~p> " - "belonging to non-existing " - "user ~p, handling trap: " - "~n Error: ~w" - "~n Trap info: ~w", - [Target, Addr, Port, UserId, - Error3, SnmpTrapInfo]) + warning_msg( + "failed unregister agent ~p <~p,~p> " + "belonging to non-existing " + "user ~p, handling trap: " + "~n Error: ~w" + "~n Trap info: ~w", + [Target, Domain, Addr, UserId, + Error3, SnmpTrapInfo]) end end; Error4 -> %% Unknown agent, pass it on to the default user ?vlog("[trap] failed retreiving user id for agent <~p,~p>: " - "~n ~p", [Addr, Port, Error4]), + "~n ~p", [Domain, Addr, Error4]), case snmpm_config:user_info() of {ok, DefUserId, DefMod, DefData} -> - handle_agent(DefUserId, DefMod, - Addr, Port, - trap, ignore, - SnmpTrapInfo, DefData, State); + handle_agent( + DefUserId, DefMod, + Domain, Addr, + trap, ignore, + SnmpTrapInfo, DefData, State); Error5 -> - error_msg("failed retreiving " - "the default user info handling trap from " - "<~p,~p>: ~n~w~n~w", - [Addr, Port, Error5, SnmpTrapInfo]) + error_msg( + "failed retreiving " + "the default user info handling trap from " + "<~p,~p>: ~n~w~n~w", + [Domain, Addr, Error5, SnmpTrapInfo]) end end, ok. -handle_trap(UserId, Mod, - RegType, Target, Addr, Port, SnmpTrapInfo, Data, State) -> +handle_trap( + UserId, Mod, RegType, Target, Domain, Addr, SnmpTrapInfo, Data, State) -> ?vtrace("handle_trap -> entry with" "~n UserId: ~p" "~n Mod: ~p", [UserId, Mod]), F = fun() -> - do_handle_trap(UserId, Mod, - RegType, Target, Addr, Port, - SnmpTrapInfo, Data, State) + do_handle_trap( + UserId, Mod, + RegType, Target, Domain, Addr, + SnmpTrapInfo, Data, State) end, handle_callback(F), ok. -do_handle_trap(UserId, Mod, - RegType, Target, Addr, Port, SnmpTrapInfo, Data, _State) -> +do_handle_trap( + UserId, Mod, RegType, Target, Domain, Addr, SnmpTrapInfo, Data, _State) -> ?vdebug("do_handle_trap -> entry with" "~n UserId: ~p", [UserId]), {HandleTrap, CallbackArgs} = @@ -2338,8 +2360,9 @@ do_handle_trap(UserId, Mod, {fun() -> Mod:handle_trap(Target, SnmpTrapInfo, Data) end, [Target, SnmpTrapInfo, Data]}; addr_port -> - {fun() -> Mod:handle_trap(Addr, Port, SnmpTrapInfo, Data) end, - [Addr, Port, SnmpTrapInfo, Data]} + {Ip, Port} = Addr, + {fun() -> Mod:handle_trap(Ip, Port, SnmpTrapInfo, Data) end, + [Ip, Port, SnmpTrapInfo, Data]} end, try HandleTrap() of @@ -2347,9 +2370,10 @@ do_handle_trap(UserId, Mod, ?vtrace("do_handle_trap -> register: " "~n UserId2: ~p" "~n Config: ~p", [UserId2, Config]), - Target2 = mk_target_name(Addr, Port, Config), - Config2 = [{reg_type, target_name}, - {address, Addr}, {port, Port} | Config], + Target2 = mk_target_name(Domain, Addr, Config), + Config2 = + [{reg_type, target_name}, + {tdomain, Domain}, {taddress, Addr} | Config], case snmpm_config:register_agent(UserId2, Target2, Config2) of ok -> ok; @@ -2357,7 +2381,7 @@ do_handle_trap(UserId, Mod, error_msg("failed registering agent " "handling trap " "<~p,~p>: ~n~w", - [Addr, Port, Reason]), + [Domain, Addr, Reason]), ok end; {register, UserId2, Target2, Config} -> @@ -2375,20 +2399,19 @@ do_handle_trap(UserId, Mod, error_msg("failed registering agent " "handling trap " "~p <~p,~p>: ~n~w", - [Target2, Addr, Port, Reason]), + [Target2, Domain, Addr, Reason]), reply end; unregister -> ?vtrace("do_handle_trap -> unregister", []), - case snmpm_config:unregister_agent(UserId, - Addr, Port) of + case snmpm_config:unregister_agent(UserId, Domain, Addr) of ok -> ok; {error, Reason} -> error_msg("failed unregistering agent " "handling trap " "<~p,~p>: ~n~w", - [Addr, Port, Reason]), + [Domain, Addr, Reason]), ok end; ignore -> @@ -2405,28 +2428,30 @@ do_handle_trap(UserId, Mod, end. -handle_snmp_inform(Ref, - #pdu{error_status = EStatus, - error_index = EIndex, - varbinds = Varbinds} = Pdu, Addr, Port, State) -> +handle_snmp_inform( + Ref, + #pdu{error_status = EStatus, + error_index = EIndex, + varbinds = Varbinds} = Pdu, Domain, Addr, State) -> - ?vtrace("handle_snmp_inform -> entry with" - "~n Addr: ~p" - "~n Port: ~p" - "~n Pdu: ~p", [Addr, Port, Pdu]), + ?vtrace("handle_snmp_inform -> entry with~n" + " Domain: ~p~n" + " Addr: ~p~n" + " Pdu: ~p", [Domain, Addr, Pdu]), Varbinds2 = fix_vbs_BITS(Varbinds), SnmpInform = {EStatus, EIndex, Varbinds2}, - case snmpm_config:get_agent_user_info(Addr, Port) of + case snmpm_config:get_agent_user_info(Domain, Addr) of {ok, UserId, Target, RegType} -> case snmpm_config:user_info(UserId) of {ok, Mod, Data} -> ?vdebug("[inform] callback handle_inform with: " "~n UserId: ~p" "~n Mod: ~p", [UserId, Mod]), - handle_inform(UserId, Mod, Ref, - RegType, Target, Addr, Port, - SnmpInform, Data, State); + handle_inform( + UserId, Mod, Ref, + RegType, Target, Domain, Addr, + SnmpInform, Data, State); Error1 -> %% User no longer exists, unregister agent case snmpm_config:unregister_agent(UserId, Target) of @@ -2437,15 +2462,16 @@ handle_snmp_inform(Ref, "~n ~p", [UserId, Error1]), case snmpm_config:user_info() of {ok, DefUserId, DefMod, DefData} -> - handle_agent(DefUserId, DefMod, - Addr, Port, - inform, Ref, - SnmpInform, DefData, State); + handle_agent( + DefUserId, DefMod, + Domain, Addr, + inform, Ref, + SnmpInform, DefData, State); Error2 -> error_msg("failed retreiving the default " "user info handling inform from " "~p <~p,~p>: ~n~w~n~w", - [Target, Addr, Port, + [Target, Domain, Addr, Error2, Pdu]) end; Error3 -> @@ -2456,7 +2482,7 @@ handle_snmp_inform(Ref, "user ~p, handling inform: " "~n Error: ~w" "~n Pdu: ~w", - [Target, Addr, Port, UserId, + [Target, Domain, Addr, UserId, Error3, Pdu]) end end; @@ -2464,42 +2490,46 @@ handle_snmp_inform(Ref, Error4 -> %% Unknown agent, pass it on to the default user ?vlog("[inform] failed retreiving user id for agent <~p,~p>: " - "~n ~p", [Addr, Port, Error4]), + "~n ~p", [Domain, Addr, Error4]), case snmpm_config:user_info() of {ok, DefUserId, DefMod, DefData} -> - handle_agent(DefUserId, DefMod, - Addr, Port, - inform, Ref, - SnmpInform, DefData, State); + handle_agent( + DefUserId, DefMod, + Domain, Addr, + inform, Ref, + SnmpInform, DefData, State); Error5 -> error_msg("failed retreiving " "the default user info handling inform from " "<~p,~p>: ~n~w~n~w", - [Addr, Port, Error5, Pdu]) + [Domain, Addr, Error5, Pdu]) end end, ok; -handle_snmp_inform(_Ref, CrapInform, Addr, Port, _State) -> +handle_snmp_inform(_Ref, CrapInform, Domain, Addr, _State) -> error_msg("received crap (snmp) inform from ~w:~w =>" - "~p", [Addr, Port, CrapInform]), + "~p", [Domain, Addr, CrapInform]), ok. -handle_inform(UserId, Mod, Ref, - RegType, Target, Addr, Port, SnmpInform, Data, State) -> +handle_inform( + UserId, Mod, Ref, + RegType, Target, Domain, Addr, SnmpInform, Data, State) -> ?vtrace("handle_inform -> entry with" "~n UserId: ~p" "~n Mod: ~p", [UserId, Mod]), F = fun() -> - do_handle_inform(UserId, Mod, Ref, - RegType, Target, Addr, Port, SnmpInform, - Data, State) + do_handle_inform( + UserId, Mod, Ref, + RegType, Target, Domain, Addr, SnmpInform, + Data, State) end, handle_callback(F), ok. -do_handle_inform(UserId, Mod, Ref, - RegType, Target, Addr, Port, SnmpInform, Data, State) -> +do_handle_inform( + UserId, Mod, Ref, + RegType, Target, Domain, Addr, SnmpInform, Data, State) -> ?vdebug("do_handle_inform -> entry with" "~n UserId: ~p", [UserId]), {HandleInform, CallbackArgs} = @@ -2508,8 +2538,9 @@ do_handle_inform(UserId, Mod, Ref, {fun() -> Mod:handle_inform(Target, SnmpInform, Data) end, [Target, SnmpInform, Data]}; addr_port -> - {fun() -> Mod:handle_inform(Addr, Port, SnmpInform, Data) end, - [Addr, Port, SnmpInform, Data]} + {Ip, Port} = Addr, + {fun() -> Mod:handle_inform(Ip, Port, SnmpInform, Data) end, + [Ip, Port, SnmpInform, Data]} end, Rep = @@ -2520,9 +2551,11 @@ do_handle_inform(UserId, Mod, Ref, "~n Config: ~p", [UserId2, Config]), %% The only user which would do this is the %% default user - Target2 = mk_target_name(Addr, Port, Config), - Config2 = [{reg_type, target_name}, - {address, Addr}, {port, Port} | Config], + Target2 = mk_target_name(Domain, Addr, Config), + Config2 = + [{reg_type, target_name} | + ensure_present( + [{tdomain, Domain}, {taddress, Addr}], Config)], case snmpm_config:register_agent(UserId2, Target2, Config2) of ok -> reply; @@ -2530,7 +2563,7 @@ do_handle_inform(UserId, Mod, Ref, error_msg("failed registering agent " "handling inform " "~p <~p,~p>: ~n~w", - [Target2, Addr, Port, Reason]), + [Target2, Domain, Addr, Reason]), reply end; @@ -2549,21 +2582,21 @@ do_handle_inform(UserId, Mod, Ref, error_msg("failed registering agent " "handling inform " "~p <~p,~p>: ~n~w", - [Target2, Addr, Port, Reason]), + [Target2, Domain, Addr, Reason]), reply end; unregister -> ?vtrace("do_handle_inform -> unregister", []), - case snmpm_config:unregister_agent(UserId, - Addr, Port) of + case snmpm_config:unregister_agent( + UserId, Domain, Addr) of ok -> reply; {error, Reason} -> error_msg("failed unregistering agent " "handling inform " "<~p,~p>: ~n~w", - [Addr, Port, Reason]), + [Domain, Addr, Reason]), reply end; @@ -2576,8 +2609,8 @@ do_handle_inform(UserId, Mod, Ref, reply; InvalidResult -> - handle_invalid_result(handle_inform, CallbackArgs, - InvalidResult), + handle_invalid_result( + handle_inform, CallbackArgs, InvalidResult), reply catch @@ -2586,31 +2619,34 @@ do_handle_inform(UserId, Mod, Ref, reply end, - handle_inform_response(Rep, Ref, Addr, Port, State), + handle_inform_response(Rep, Ref, Domain, Addr, State), ok. -handle_inform_response(_, ignore, _Addr, _Port, _State) -> +handle_inform_response(_, ignore, _Domain, _Addr, _State) -> ignore; -handle_inform_response(no_reply, _Ref, _Addr, _Port, _State) -> +handle_inform_response(no_reply, _Ref, _Domain, _Addr, _State) -> no_reply; -handle_inform_response(_, Ref, Addr, Port, - #state{net_if = Pid, net_if_mod = Mod}) -> +handle_inform_response( + _, Ref, Domain, Addr, + #state{net_if = Pid, net_if_mod = Mod}) -> ?vdebug("handle_inform -> response", []), - (catch Mod:inform_response(Pid, Ref, Addr, Port)). + (catch Mod:inform_response(Pid, Ref, Domain, Addr)). -handle_snmp_report(#pdu{error_status = EStatus, - error_index = EIndex, - varbinds = Varbinds} = Pdu, Addr, Port, State) -> +handle_snmp_report( + #pdu{error_status = EStatus, + error_index = EIndex, + varbinds = Varbinds} = Pdu, + Domain, Addr, State) -> - ?vtrace("handle_snmp_report -> entry with" - "~n Addr: ~p" - "~n Port: ~p" - "~n Pdu: ~p", [Addr, Port, Pdu]), + ?vtrace("handle_snmp_report -> entry with~n" + " Domain: ~p~n" + " Addr: ~p~n" + " Pdu: ~p", [Domain, Addr, Pdu]), Varbinds2 = fix_vbs_BITS(Varbinds), SnmpReport = {EStatus, EIndex, Varbinds2}, - case snmpm_config:get_agent_user_info(Addr, Port) of + case snmpm_config:get_agent_user_info(Domain, Addr) of {ok, UserId, Target, RegType} -> case snmpm_config:user_info(UserId) of {ok, Mod, Data} -> @@ -2620,7 +2656,7 @@ handle_snmp_report(#pdu{error_status = EStatus, "~n ~p" "~n ~p", [UserId, Mod, Target, SnmpReport]), handle_report(UserId, Mod, - RegType, Target, Addr, Port, + RegType, Target, Domain, Addr, SnmpReport, Data, State); Error1 -> %% User no longer exists, unregister agent @@ -2633,7 +2669,7 @@ handle_snmp_report(#pdu{error_status = EStatus, case snmpm_config:user_info() of {ok, DefUserId, DefMod, DefData} -> handle_agent(DefUserId, DefMod, - Addr, Port, + Domain, Addr, report, ignore, SnmpReport, DefData, State); @@ -2641,7 +2677,7 @@ handle_snmp_report(#pdu{error_status = EStatus, error_msg("failed retreiving the default " "user info handling report from " "~p <~p,~p>: ~n~w~n~w", - [Target, Addr, Port, + [Target, Domain, Addr, Error2, Pdu]) end; Error3 -> @@ -2652,7 +2688,7 @@ handle_snmp_report(#pdu{error_status = EStatus, "user ~p, handling report: " "~n Error: ~w" "~n Report: ~w", - [Target, Addr, Port, UserId, + [Target, Domain, Addr, UserId, Error3, Pdu]) end end; @@ -2660,25 +2696,25 @@ handle_snmp_report(#pdu{error_status = EStatus, Error4 -> %% Unknown agent, pass it on to the default user ?vlog("[report] failed retreiving user id for agent <~p,~p>: " - "~n ~p", [Addr, Port, Error4]), + "~n ~p", [Domain, Addr, Error4]), case snmpm_config:user_info() of {ok, DefUserId, DefMod, DefData} -> handle_agent(DefUserId, DefMod, - Addr, Port, + Domain, Addr, report, ignore, SnmpReport, DefData, State); Error5 -> error_msg("failed retreiving " "the default user info handling report from " "<~p,~p>: ~n~w~n~w", - [Addr, Port, Error5, Pdu]) + [Domain, Addr, Error5, Pdu]) end end, ok; -handle_snmp_report(CrapReport, Addr, Port, _State) -> +handle_snmp_report(CrapReport, Domain, Addr, _State) -> error_msg("received crap (snmp) report from ~w:~w =>" - "~p", [Addr, Port, CrapReport]), + "~p", [Domain, Addr, CrapReport]), ok. %% This could be from a failed get-request, so we might have a user @@ -2686,20 +2722,20 @@ handle_snmp_report(CrapReport, Addr, Port, _State) -> %% get-response (except for tha data which is different). Otherwise, %% we handle it as an error (reported via the handle_error callback %% function). -handle_snmp_report(ReqId, - #pdu{error_status = EStatus, - error_index = EIndex, - varbinds = Varbinds} = Pdu, - {ReportReason, Info} = Rep, - Addr, Port, State) - when is_integer(ReqId) -> - - ?vtrace("handle_snmp_report -> entry with" - "~n Addr: ~p" - "~n Port: ~p" - "~n ReqId: ~p" - "~n Rep: ~p" - "~n Pdu: ~p", [Addr, Port, ReqId, Rep, Pdu]), +handle_snmp_report( + ReqId, + #pdu{error_status = EStatus, + error_index = EIndex, + varbinds = Varbinds} = Pdu, + {ReportReason, Info} = Rep, + Domain, Addr, State) when is_integer(ReqId) -> + + ?vtrace("handle_snmp_report -> entry with~n" + " Domain: ~p~n" + " Addr: ~p~n" + " ReqId: ~p~n" + " Rep: ~p~n" + " Pdu: ~p", [Domain, Addr, ReqId, Rep, Pdu]), Varbinds2 = fix_vbs_BITS(Varbinds), SnmpReport = {EStatus, EIndex, Varbinds2}, @@ -2744,7 +2780,7 @@ handle_snmp_report(ReqId, %% Either not a sync request or no such request. Either %% way, this is error info, so handle it as such. - case snmpm_config:get_agent_user_id(Addr, Port) of + case snmpm_config:get_agent_user_id(Domain, Addr) of {ok, UserId} -> case snmpm_config:user_info(UserId) of {ok, Mod, Data} -> @@ -2768,7 +2804,7 @@ handle_snmp_report(ReqId, "default user " "info handling report from " "<~p,~p>: ~n~w~n~w~n~w", - [Addr, Port, Error, + [Domain, Addr, Error, ReqId, Reason]) end end; @@ -2776,7 +2812,7 @@ handle_snmp_report(ReqId, %% Unknown agent, pass it on to the default user ?vlog("[report] failed retreiving user id for " "agent <~p,~p>: " - "~n ~p", [Addr, Port, Error]), + "~n ~p", [Domain, Addr, Error]), case snmpm_config:user_info() of {ok, DefUserId, DefMod, DefData} -> handle_error(DefUserId, DefMod, Reason, ReqId, @@ -2786,32 +2822,36 @@ handle_snmp_report(ReqId, "the default user info handling " "report from " "<~p,~p>: ~n~w~n~w~n~w", - [Addr, Port, Error, ReqId, Reason]) + [Domain, Addr, Error, ReqId, Reason]) end end end, ok; -handle_snmp_report(CrapReqId, CrapReport, CrapInfo, Addr, Port, _State) -> - error_msg("received crap (snmp) report from ~w:~w =>" - "~n~p~n~p~n~p", [Addr, Port, CrapReqId, CrapReport, CrapInfo]), +handle_snmp_report(CrapReqId, CrapReport, CrapInfo, Domain, Addr, _State) -> + error_msg( + "received crap (snmp) report from ~w:~w =>" + "~n~p~n~p~n~p", + [Domain, Addr, CrapReqId, CrapReport, CrapInfo]), ok. -handle_report(UserId, Mod, RegType, Target, Addr, Port, +handle_report(UserId, Mod, RegType, Target, Domain, Addr, SnmpReport, Data, State) -> ?vtrace("handle_report -> entry with" "~n UserId: ~p" "~n Mod: ~p", [UserId, Mod]), F = fun() -> - do_handle_report(UserId, Mod, RegType, Target, Addr, Port, - SnmpReport, Data, State) + do_handle_report( + UserId, Mod, RegType, Target, Domain, Addr, + SnmpReport, Data, State) end, handle_callback(F), ok. -do_handle_report(UserId, Mod, - RegType, Target, Addr, Port, SnmpReport, Data, _State) -> +do_handle_report( + UserId, Mod, RegType, Target, Domain, Addr, + SnmpReport, Data, _State) -> ?vdebug("do_handle_report -> entry with" "~n UserId: ~p", [UserId]), {HandleReport, CallbackArgs} = @@ -2820,8 +2860,9 @@ do_handle_report(UserId, Mod, {fun() -> Mod:handle_report(Target, SnmpReport, Data) end, [Target, SnmpReport, Data]}; addr_port -> - {fun() -> Mod:handle_report(Addr, Port, SnmpReport, Data) end, - [Addr, Port, SnmpReport, Data]} + {Ip, Port} = Addr, + {fun() -> Mod:handle_report(Ip, Port, SnmpReport, Data) end, + [Ip, Port, SnmpReport, Data]} end, try HandleReport() of @@ -2831,17 +2872,18 @@ do_handle_report(UserId, Mod, "~n Config: ~p", [UserId2, Config]), %% The only user which would do this is the %% default user - Target2 = mk_target_name(Addr, Port, Config), - Config2 = [{reg_type, target_name}, - {address, Addr}, {port, Port} | Config], + Target2 = mk_target_name(Domain, Addr, Config), + Config2 = + [{reg_type, target_name}, + {tdomain, Domain}, {taddress, Addr} | Config], case snmpm_config:register_agent(UserId2, Target2, Config2) of ok -> ok; {error, Reason} -> error_msg("failed registering agent " "handling report " - "<~p,~p>: ~n~w", - [Addr, Port, Reason]), + "<~p,~p>: ~n~w", + [Domain, Addr, Reason]), ok end; @@ -2860,21 +2902,20 @@ do_handle_report(UserId, Mod, error_msg("failed registering agent " "handling report " "~p <~p,~p>: ~n~w", - [Target2, Addr, Port, Reason]), + [Target2, Domain, Addr, Reason]), reply end; unregister -> ?vtrace("do_handle_trap -> unregister", []), - case snmpm_config:unregister_agent(UserId, - Addr, Port) of + case snmpm_config:unregister_agent(UserId, Domain, Addr) of ok -> ok; {error, Reason} -> error_msg("failed unregistering agent " "handling report " "<~p,~p>: ~n~w", - [Addr, Port, Reason]), + [Domain, Addr, Reason]), ok end; @@ -3012,50 +3053,49 @@ do_gc(Key, Now) -> %% %%---------------------------------------------------------------------- -send_get_request(Oids, Vsn, MsgData, Domain, Addr, Port, ExtraInfo, +send_get_request(Oids, Vsn, MsgData, Domain, Addr, ExtraInfo, #state{net_if = NetIf, net_if_mod = Mod, mini_mib = MiniMIB}) -> Pdu = make_pdu(get, Oids, MiniMIB), - ?vtrace("send_get_request -> send get-request:" - "~n Mod: ~p" - "~n NetIf: ~p" - "~n Pdu: ~p" - "~n Vsn: ~p" - "~n MsgData: ~p" - "~n Domain: ~p" - "~n Addr: ~p" - "~n Port: ~p", - [Mod, NetIf, Pdu, Vsn, MsgData, Domain, Addr, Port]), + ?vtrace("send_get_request -> send get-request:~n" + " Mod: ~p~n" + " NetIf: ~p~n" + " Pdu: ~p~n" + " Vsn: ~p~n" + " MsgData: ~p~n" + " Domain: ~p~n" + " Addr: ~p", + [Mod, NetIf, Pdu, Vsn, MsgData, Domain, Addr]), Res = (catch Mod:send_pdu(NetIf, Pdu, Vsn, MsgData, - Domain, Addr, Port, ExtraInfo)), + Domain, Addr, ExtraInfo)), ?vtrace("send_get_request -> send result:" "~n ~p", [Res]), Pdu#pdu.request_id. -send_get_next_request(Oids, Vsn, MsgData, Domain, Addr, Port, ExtraInfo, +send_get_next_request(Oids, Vsn, MsgData, Domain, Addr, ExtraInfo, #state{mini_mib = MiniMIB, net_if = NetIf, net_if_mod = Mod}) -> Pdu = make_pdu(get_next, Oids, MiniMIB), - Mod:send_pdu(NetIf, Pdu, Vsn, MsgData, Domain, Addr, Port, ExtraInfo), + Mod:send_pdu(NetIf, Pdu, Vsn, MsgData, Domain, Addr, ExtraInfo), Pdu#pdu.request_id. -send_get_bulk_request(Oids, Vsn, MsgData, Domain, Addr, Port, +send_get_bulk_request(Oids, Vsn, MsgData, Domain, Addr, NonRep, MaxRep, ExtraInfo, #state{mini_mib = MiniMIB, net_if = NetIf, net_if_mod = Mod}) -> Pdu = make_pdu(bulk, {NonRep, MaxRep, Oids}, MiniMIB), - Mod:send_pdu(NetIf, Pdu, Vsn, MsgData, Domain, Addr, Port, ExtraInfo), + Mod:send_pdu(NetIf, Pdu, Vsn, MsgData, Domain, Addr, ExtraInfo), Pdu#pdu.request_id. -send_set_request(VarsAndVals, Vsn, MsgData, Domain, Addr, Port, ExtraInfo, +send_set_request(VarsAndVals, Vsn, MsgData, Domain, Addr, ExtraInfo, #state{mini_mib = MiniMIB, net_if = NetIf, net_if_mod = Mod}) -> Pdu = make_pdu(set, VarsAndVals, MiniMIB), - Mod:send_pdu(NetIf, Pdu, Vsn, MsgData, Domain, Addr, Port, ExtraInfo), + Mod:send_pdu(NetIf, Pdu, Vsn, MsgData, Domain, Addr, ExtraInfo), Pdu#pdu.request_id. %% send_discovery(Vsn, MsgData, Addr, Port, ExtraInfo, @@ -3291,10 +3331,9 @@ agent_data(TargetName, SendOpts) -> {Comm, SecModel} end, Domain = agent_data_item(tdomain, TargetName), - Addr = agent_data_item(address, TargetName), - Port = agent_data_item(port, TargetName), + Addr = agent_data_item(taddress, TargetName), RegType = agent_data_item(reg_type, TargetName), - {ok, RegType, Domain, Addr, Port, version(Version), MsgData}; + {ok, RegType, Domain, Addr, version(Version), MsgData}; Error -> Error end. @@ -3440,9 +3479,9 @@ maybe_demonitor(MonRef) -> t() -> {A,B,C} = erlang:now(), A*1000000000+B*1000+(C div 1000). - -mk_target_name(Addr, Port, Config) -> - snmpm_config:mk_target_name(Addr, Port, Config). + +mk_target_name(Domain, Addr, Config) -> + snmpm_config:mk_target_name(Domain, Addr, Config). default_agent_config() -> case snmpm_config:agent_info() of diff --git a/lib/snmp/src/manager/snmpm_user.erl b/lib/snmp/src/manager/snmpm_user.erl index e6b0b6943e..8e5a874f77 100644 --- a/lib/snmp/src/manager/snmpm_user.erl +++ b/lib/snmp/src/manager/snmpm_user.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2013. All Rights Reserved. +%% Copyright Ericsson AB 2004-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -39,20 +39,25 @@ %% *** handle_error *** %% An "asynchronous" error has been detected --callback handle_error(ReqId :: integer(), - Reason :: {unexpected_pdu, SnmpInfo :: snmp_gen_info()} | - {invalid_sec_info, SecInfo :: term(), SnmpInfo :: snmp_gen_info()} | - {empty_message, Addr :: ip_address(), Port :: port_number()} | - term(), - UserData :: term()) -> +-callback handle_error( + ReqId :: integer(), + Reason :: {unexpected_pdu, SnmpInfo :: snmp_gen_info()} | + {invalid_sec_info, + SecInfo :: term(), + SnmpInfo :: snmp_gen_info()} | + {empty_message, + TransportDomain :: atom(), + {Addr :: ip_address(), Port :: port_number()}} | + term(), + UserData :: term()) -> snmp:void(). %% *** handle_agent *** %% A message was received from an unknown agent --callback handle_agent(Addr :: term(), - Port :: pos_integer(), +-callback handle_agent(Domain :: atom(), + Address :: term(), Type :: pdu | trap | inform | report, SnmpInfo :: snmp_gen_info() | snmp_v1_trap_info(), UserData :: term()) -> diff --git a/lib/snmp/src/manager/snmpm_user_default.erl b/lib/snmp/src/manager/snmpm_user_default.erl index 015198cb76..b827713f27 100644 --- a/lib/snmp/src/manager/snmpm_user_default.erl +++ b/lib/snmp/src/manager/snmpm_user_default.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2013. All Rights Reserved. +%% Copyright Ericsson AB 2004-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -36,13 +36,13 @@ handle_error(ReqId, Reason, UserData) -> ignore. -handle_agent(Addr, Port, Type, SnmpInfo, UserData) -> - info("received handle_agent:" - "~n Addr: ~p" - "~n Port: ~p" - "~n Type: ~p" - "~n SnmpInfo: ~p" - "~n UserData: ~p", [Addr, Port, Type, SnmpInfo, UserData]), +handle_agent(Domain, Address, Type, SnmpInfo, UserData) -> + info("received handle_agent:~n" + " Domain: ~p~n" + " Address: ~p~n" + " Type: ~p~n" + " SnmpInfo: ~p~n" + " UserData: ~p", [Domain, Address, Type, SnmpInfo, UserData]), ignore. diff --git a/lib/snmp/src/misc/snmp_conf.erl b/lib/snmp/src/misc/snmp_conf.erl index 46625989d5..153c8070c2 100644 --- a/lib/snmp/src/misc/snmp_conf.erl +++ b/lib/snmp/src/misc/snmp_conf.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2013. All Rights Reserved. +%% Copyright Ericsson AB 1996-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -25,35 +25,42 @@ %% External exports %% Avoid warning for local function error/1 clashing with autoimported BIF. -compile({no_auto_import,[error/1]}). --export([read_files/2, read/2]). +-export([read_files/2, no_gen/2, no_order/2, no_filter/1, keyorder/4]). +-export([read/2, read/3]). %% Basic (type) check functions -export([check_mandatory/2, check_integer/1, check_integer/2, - + check_string/1, check_string/2, - check_atom/2, + check_atom/2, check_timer/1, - all_domains/0, - check_domain/1, - all_tdomains/0, - check_tdomain/1, - mk_tdomain/1, - which_domain/1, - check_ip/1, check_ip/2, - check_taddress/1, check_taddress/2, - mk_taddress/3, - - check_packet_size/1, + all_domains/0, + check_domain/1, + domain_to_name/1, + all_tdomains/0, + check_tdomain/1, + mk_tdomain/0, mk_tdomain/1, + tdomain_to_family/1, tdomain_to_domain/1, + which_domain/1, + mk_addr_string/1, + check_ip/1, check_ip/2, + check_port/1, +%% ip_port_to_domaddr/2, + check_address/2, check_address/3, + check_taddress/2, + mk_taddress/1, mk_taddress/2, + + check_packet_size/1, check_oid/1, - check_imask/1, check_emask/1, - - check_mp_model/1, - check_sec_model/1, check_sec_model/2, check_sec_model/3, + check_imask/1, check_emask/1, + + check_mp_model/1, + check_sec_model/1, check_sec_model/2, check_sec_model/3, check_sec_level/1, all_integer/1 @@ -70,17 +77,53 @@ -include("snmp_verbosity.hrl"). +-define(is_word(P), (((P) band (bnot 65535)) =:= 0)). +-define(is_word(P0, P1), ((((P0) bor (P1)) band (bnot 255)) =:= 0)). + +mk_word(B0, B1) -> ((B0) bsl 8) bor (B1). +mk_bytes(W) -> [(W) bsr 8,(W) band 255]. + +-define( + is_ipv4_addr(A0, A1, A2, A3), + ((((A0) bor (A1) bor (A2) bor (A3)) band (bnot 255)) =:= 0)). + +-define( + is_ipv6_addr(A0, A1, A2, A3, A4, A5, A6, A7), + ((((A0) bor (A1) bor (A2) bor (A3) bor (A4) bor (A5) bor (A6) bor (A7)) + band (bnot 65535)) =:= 0)). +-define( + is_ipv6_addr( + A0, A1, A2, A3, A4, A5, A6, A7, A8, A9, A10, A11, A12, A13, A14, A15), + ((((A0) bor (A1) bor (A2) bor (A3) bor + (A4) bor (A5) bor (A6) bor (A7) bor + (A8) bor (A9) bor (A10) bor (A11) bor + (A12) bor (A13) bor (A14) bor (A15)) + band (bnot 65535)) =:= 0)). + + %%----------------------------------------------------------------- %% read_files(Dir, Files) -> Configs %% Dir - string() - Full path to the config dir. -%% Files - [{Gen, Filter, Check, FileName}] +%% Files - [{FileName, Gen, Order, Check, Filter}] +%% FileName - string() - Name of the config file. %% Gen - function/2 - In case of failure when reading the config file, %% this function is called to either generate a %% default file or issue the error. -%% Filter - function/1 - Filters all the config entries read from the file -%% Check - function/1 - Check each entry as they are read from the file. -%% FileName - string() - Name of the config file. +%% Returns a generated config list corresponding +%% to the written file. +%% (Dir, Error) -> Configs. +%% Order - function/2 - An ordering function that is used to process +%% the read config entries using lists:sort/2. +%% Returns true if arg 1 compares less than or +%% equal to arg 2, false otherwise. +%% Check - function/2 - Check each entry as they are read from the file. +%% (Entry, State) -> +%% {ok,NewState} | {{ok,NewEntry},NewState} | +%% throw(Error) +%% State =:= 'undefined' the first time. +%% Filter - function/1 - Process all the config entries read from the file +%% (Configs) -> [config_entry()]. %% Configs - [config_entry()] %% config_entry() - term() @@ -89,99 +132,143 @@ read_files(Dir, Files) when is_list(Dir) andalso is_list(Files) -> read_files(_Dir, [], Res) -> lists:reverse(Res); -read_files(Dir, [{Gen, Filter, Check, FileName}|Files], Res) - when is_function(Filter) andalso - is_function(Check) andalso - is_list(FileName) -> - ?vdebug("read_files -> entry with" - "~n FileName: ~p", [FileName]), +read_files(Dir, [{FileName, Gen, Order, Check, Filter}|Files], Res) + when is_list(FileName), + is_function(Gen), + is_function(Order), + is_function(Check), + is_function(Filter) -> + ?vdebug("read_files -> entry with~n" + " FileName: ~p", [FileName]), File = filename:join(Dir, FileName), - case file:read_file_info(File) of - {ok, _} -> - Confs = read(File, Check), - read_files(Dir, Files, [Filter(Confs)|Res]); - {error, R} -> - ?vlog("failed reading file info for ~s: " - "~n ~p", [FileName, R]), - Gen(Dir, R), - read_files(Dir, Files, [Filter([])|Res]) - end. - + Confs = + case file:read_file_info(File) of + {ok,_} -> + read(File, Order, Check); + {error, R} -> + ?vlog("failed reading file info for ~s: ~n" + " ~p", [FileName, R]), + Gen(Dir, R) + end, + read_files(Dir, Files, [Filter(Confs)|Res]). + + + +no_gen(_Dir, _R) -> []. +no_order(_, _) -> true. +no_filter(X) -> X. + +%% Order tuples on element N with Keys first in appearence order. +%% +%% An ordering function (A, B) shall return true iff +%% A is less than or equal to B i.e shall return +%% false iff A is to be ordered after B. +keyorder(N, A, B, _) when element(N, A) == element(N, B) -> + true; +keyorder(N, A, B, [Key | _]) + when tuple_size(A) >= 1, element(N, B) == Key -> + false; +keyorder(N, A, B, [Key | _]) + when element(N, A) == Key, tuple_size(B) >= 1 -> + true; +keyorder(N, A, B, [_ | Keys]) -> + keyorder(N, A, B, Keys); +keyorder(_, A, B, []) when tuple_size(A) >= 1, tuple_size(B) >= 1 -> + %% Do not order other keys + true; +keyorder(N, A, B, sort) -> + %% Order other keys according to standard sort order + element(N, A) =< element(N, B). + + +read(File, Verify) -> + Check = fun (Row, State) -> {Verify(Row), State} end, + read(File, fun no_order/2, Check). %% Ret. Res | exit(Reason) -read(File, Check) when is_function(Check) -> - ?vdebug("read -> entry with" - "~n File: ~p", [File]), - +read(File, Order, Check) when is_function(Order), is_function(Check) -> + ?vdebug("read -> entry with~n" + " File: ~p", [File]), Fd = open_file(File), + read_fd(File, Order, Check, Fd, 1, []). - case loop(Fd, [], Check, 1, File) of - {error, Reason} -> - file:close(Fd), - error(Reason); - {ok, Res} -> - file:close(Fd), - Res +read_fd(File, Order, Check, Fd, StartLine, Res) -> + case do_read(Fd, "", StartLine) of + {ok, Row, EndLine} -> + ?vtrace("read_fd ->~n" + " Row: ~p~n" + " EndLine: ~p", [Row,EndLine]), + read_fd( + File, Order, Check, Fd, EndLine, + [{StartLine, Row, EndLine}|Res]); + {error, Error, EndLine} -> + ?vtrace("read_fd -> read failure:~n" + " Error: ~p~n" + " EndLine: ~p", [Error,EndLine]), + file:close(Fd), + error({failed_reading, File, StartLine, EndLine, Error}); + {eof, _EndLine} -> + Lines = + lists:sort( + fun ({_, RowA, _}, {_, RowB, _}) -> + Order(RowA, RowB) + end, + lists:reverse(Res)), + ?vtrace("read_fd to read_check ->~n" + " Lines: ~p", [Lines]), + read_check(File, Check, Lines, undefined, []) end. -open_file(File) -> - case file:open(File, [read]) of - {ok, Fd} -> - Fd; - {error, Reason} -> - error({failed_open, File, Reason}) +read_check(_, _, [], _, Res) -> + lists:reverse(Res); +read_check(File, Check, [{StartLine, Row, EndLine}|Lines], State, Res) -> + try Check(Row, State) of + {Rows, NewState} when is_list(Rows) -> + ?vtrace("read_check -> ok:~n" + " Rows: ~p~n", [Rows]), + read_check(File, Check, Lines, NewState, Rows ++ Res); + {ok, NewState} -> + ?vtrace("read_check -> ok", []), + read_check(File, Check, Lines, NewState, [Row | Res]); + {{ok, NewRow}, NewState} -> + ?vtrace("read_check -> ok:~n" + " NewRow: ~p~n", [NewRow]), + read_check(File, Check, Lines, NewState, [NewRow | Res]) + catch + {error, Reason} -> + ?vtrace("read_check -> error:~n" + " Reason: ~p", [Reason]), + error({failed_check, File, StartLine, EndLine, Reason}); + Class:Reason -> + Error = {Class,Reason,erlang:get_stacktrace()}, + ?vtrace("read_check -> failure:~n" + " Error: ~p", [Error]), + error({failed_check, File, StartLine, EndLine, Error}) end. -loop(Fd, Res, Check, StartLine, File) -> - case do_read(Fd, "", StartLine) of - {ok, Row, EndLine} -> - ?vtrace("loop -> " - "~n Row: ~p" - "~n EndLine: ~p", [Row, EndLine]), - case (catch Check(Row)) of - ok -> - ?vtrace("loop -> ok", []), - loop(Fd, [Row | Res], Check, EndLine, File); - {ok, NewRow} -> - ?vtrace("loop -> ok: " - "~n NewRow: ~p", [NewRow]), - loop(Fd, [NewRow | Res], Check, EndLine, File); - {error, Reason} -> - ?vtrace("loop -> check error: " - "~n Reason: ~p", [Reason]), - {error, {failed_check, File, StartLine, EndLine, Reason}}; - Error -> - ?vtrace("loop -> check failure: " - "~n Error: ~p", [Error]), - {error, {failed_check, File, StartLine, EndLine, Error}} - end; - {error, EndLine, Error} -> - ?vtrace("loop -> read failure: " - "~n Error: ~p", [Error]), - {error, {failed_reading, File, StartLine, EndLine, Error}}; - eof -> - {ok, Res} +open_file(File) -> + case file:open(File, [read]) of + {ok, Fd} -> + Fd; + {error, Reason} -> + error({failed_open, File, Reason}) end. - do_read(Io, Prompt, StartLine) -> case io:request(Io, {get_until,Prompt,erl_scan,tokens,[StartLine]}) of - {ok, Toks, EndLine} -> - case erl_parse:parse_term(Toks) of - {ok, Term} -> - {ok, Term, EndLine}; - {error, {Line, erl_parse, Error}} -> - {error, Line, {parse_error, Error}} - end; - {error,E,EndLine} -> - {error, EndLine, E}; - {eof, _EndLine} -> - eof; - Other -> - Other + {ok, Toks, EndLine} -> + case erl_parse:parse_term(Toks) of + {ok, Term} -> + {ok, Term, EndLine}; + {error, {Line, erl_parse, Error}} -> + {error, {parse_error, Error}, Line} + end; + Other -> + Other end. + %%----------------------------------------------------------------- @@ -384,7 +471,7 @@ all_tdomains() -> check_tdomain(TDomain) -> SupportedTDomains = [ - ?snmpUDPDomain, + ?snmpUDPDomain, % Legacy ?transportDomainUdpIpv4, ?transportDomainUdpIpv6 ], @@ -404,6 +491,9 @@ check_tdomain(TDomain) -> %% --------- +mk_tdomain() -> + mk_tdomain(snmpUDPDomain). + mk_tdomain(snmpUDPDomain) -> mk_tdomain(transportDomainUdpIpv4); mk_tdomain(transportDomainUdpIpv4) -> @@ -416,40 +506,64 @@ mk_tdomain(BadDomain) -> %% --------- -check_taddress(X) -> - check_taddress(snmpUDPDomain, X). +tdomain_to_family(snmpUDPDomain) -> + inet; +tdomain_to_family(transportDomainUdpIpv4) -> + inet; +tdomain_to_family(transportDomainUdpIpv6) -> + inet6; +tdomain_to_family(?snmpUDPDomain) -> + inet; +tdomain_to_family(?transportDomainUdpIpv4) -> + inet; +tdomain_to_family(?transportDomainUdpIpv6) -> + inet6; +tdomain_to_family(BadDomain) -> + error({bad_domain, BadDomain}). + + +%% --------- + +tdomain_to_domain(?snmpUDPDomain) -> + snmpUDPDomain; +tdomain_to_domain(?transportDomainUdpIpv4) -> + transportDomainUdpIpv4; +tdomain_to_domain(?transportDomainUdpIpv6) -> + transportDomainUdpIpv6; +tdomain_to_domain(BadTDomain) -> + error({bad_tdomain, BadTDomain}). + + +%% --------- check_taddress(?snmpUDPDomain, X) -> check_taddress(transportDomainUdpIpv4, X); check_taddress(snmpUDPDomain, X) -> check_taddress(transportDomainUdpIpv4, X); - +%% check_taddress(?transportDomainUdpIpv4, X) -> check_taddress(transportDomainUdpIpv4, X); -check_taddress(transportDomainUdpIpv4, X) - when is_list(X) andalso (length(X) =:= 6) -> - case (catch all_integer(X)) of - true -> +check_taddress(transportDomainUdpIpv4, X) -> + case X of + [A0,A1,A2,A3,P0,P1] + when ?is_ipv4_addr(A0, A1, A2, A3), ?is_word(P0, P1) -> ok; - false -> + _ -> error({invalid_taddress, X}) end; -check_taddress(transportDomainUdpIpv4, X) -> - error({invalid_taddress, X}); - +%% check_taddress(?transportDomainUdpIpv6, X) -> check_taddress(transportDomainUdpIpv6, X); -check_taddress(transportDomainUdpIpv6, X) - when is_list(X) andalso (length(X) =:= 10) -> - case (catch all_integer(X)) of - true -> +check_taddress(transportDomainUdpIpv6, X) -> + case X of + [A0,A1,A2,A3,A4,A5,A6,A7,P0,P1] + when ?is_ipv6_addr(A0, A1, A2, A3, A4, A5, A6, A7), + ?is_word(P0, P1) -> ok; - false -> + _ -> error({invalid_taddress, X}) end; -check_taddress(transportDomainUdpIpv6, X) -> - error({invalid_taddress, X}); - +%% check_taddress(BadDomain, _X) -> error({invalid_tdomain, BadDomain}). @@ -512,96 +626,359 @@ all_domains() -> transportDomainSctpDns ]. +check_domain(snmpUDPDomain) -> ok; +check_domain(transportDomainUdpIpv4) -> ok; +check_domain(transportDomainUdpIpv6) -> ok; check_domain(Domain) -> - SupportedDomains = - [ - snmpUDPDomain, - transportDomainUdpIpv4, - transportDomainUdpIpv6 - ], - AllDomains = all_domains(), - case lists:member(Domain, SupportedDomains) of + case lists:member(Domain, all_domains()) of true -> - ok; + error({unsupported_domain, Domain}); false -> - case lists:member(Domain, AllDomains) of - true -> - error({unsupported_domain, Domain}); - false -> - error({unknown_domain, Domain}) - end + error({unknown_domain, Domain}) end. - + +domain_to_name(snmpUDPDomain) -> + undefined; +domain_to_name(transportDomainUdpIpv4) -> + udpIpv4; +domain_to_name(transportDomainUdpIpv6) -> + udpIpv6; +domain_to_name(transportDomainUdpIpv4z) -> + udpIpv4z; +domain_to_name(transportDomainUdpIpv6z) -> + udpIpv6z; +domain_to_name(transportDomainTcpIpv4) -> + tcpIpv4; +domain_to_name(transportDomainTcpIpv6) -> + tcpIpv6; +domain_to_name(transportDomainTcpIpv4z) -> + tcpIpv4z; +domain_to_name(transportDomainTcpIpv6z) -> + tcpIpv6z; +domain_to_name(transportDomainSctpIpv4) -> + sctpIpv4; +domain_to_name(transportDomainSctpIpv6) -> + sctpIpv6; +domain_to_name(transportDomainSctpIpv4z) -> + sctpIpv4z; +domain_to_name(transportDomainSctpIpv6z) -> + sctpIpv6z; +domain_to_name(transportDomainLocal) -> + local; +domain_to_name(transportDomainUdpDns) -> + udpDns; +domain_to_name(transportDomainTcpDns) -> + tcpDns; +domain_to_name(transportDomainSctpDns) -> + sctpDns; +domain_to_name(BadDomain) -> + error({bad_domain, BadDomain}). %% --------- -%% The values of Ip and Port has both been checked at this -%% point, so we dont need to do that again. -mk_taddress(snmpUDPDomain, Ip, Port) -> - mk_taddress(transportDomainUdpIpv4, Ip, Port); -mk_taddress(transportDomainUdpIpv4, Ip, Port) when is_list(Ip) -> - Ip ++ [Port div 256, Port rem 256]; -mk_taddress(transportDomainUdpIpv4 = Domain, Ip, Port) when is_tuple(Ip) -> - mk_taddress(Domain, tuple_to_list(Ip), Port); -mk_taddress(transportDomainUdpIpv6, Ip, Port) when is_list(Ip) -> - Ip ++ [Port div 256, Port rem 256]; -mk_taddress(transportDomainUdpIpv6 = Domain, Ip, Port) when is_tuple(Ip) -> - mk_taddress(Domain, tuple_to_list(Ip), Port); +mk_taddress(Address) -> + mk_taddress(snmpUDPDomain, Address). +%% The values of Domain, Ip and Port has both been checked at this +%% point, so we dont need to do that again, but this function is +%% also used on incoming packets from net_if so a little +%% check that net_if does not supply bad arguments is in order. +%% %% These are just for convenience -mk_taddress(?snmpUDPDomain, Ip, Port) -> - mk_taddress(snmpUDPDomain, Ip, Port); -mk_taddress(?transportDomainUdpIpv4, Ip, Port) -> - mk_taddress(transportDomainUdpIpv4, Ip, Port); -mk_taddress(?transportDomainUdpIpv6, Ip, Port) -> - mk_taddress(transportDomainUdpIpv6, Ip, Port); - +mk_taddress(?snmpUDPDomain, Address) -> + mk_taddress(snmpUDPDomain, Address); +mk_taddress(?transportDomainUdpIpv4, Address) -> + mk_taddress(transportDomainUdpIpv4, Address); +mk_taddress(?transportDomainUdpIpv6, Address) -> + mk_taddress(transportDomainUdpIpv6, Address); +%% +mk_taddress(snmpUDPDomain, Address) -> % Legacy + mk_taddress(transportDomainUdpIpv4, Address); +mk_taddress(transportDomainUdpIpv4 = Domain, Address) -> + case Address of + [] -> % Empty mask + []; + {Ip, Port} when tuple_size(Ip) =:= 4, is_integer(Port) -> + tuple_to_list(Ip) ++ mk_bytes(Port); + _ -> + erlang:error(badarg, [Domain,Address]) + end; +mk_taddress(transportDomainUdpIpv6 = Domain, Address) -> + case Address of + [] -> % Empty mask + []; + {{A, B, C, D, E, F, G, H}, Port} -> + [A bsr 8, A band 255, + B bsr 8, B band 255, + C bsr 8, C band 255, + D bsr 8, D band 255, + E bsr 8, E band 255, + F bsr 8, F band 255, + G bsr 8, G band 255, + H bsr 8, H band 255, + Port bsr 8, Port band 255]; + _ -> + erlang:error(badarg, [Domain,Address]) + end; %% Bad domain -mk_taddress(BadDomain, _Ip, _Port) -> +mk_taddress(BadDomain, _) -> error({bad_domain, BadDomain}). - + %% --------- -which_domain(Ip) when is_list(Ip) andalso (length(Ip) =:= 4) -> +%% XXX remove, when net_if handles one socket per transport domain + +which_domain([A0,A1,A2,A3]) + when ?is_ipv4_addr(A0, A1, A2, A3) -> transportDomainUdpIpv4; -which_domain(Ip) when is_tuple(Ip) andalso (size(Ip) =:= 4) -> +which_domain({A0, A1, A2, A3}) + when ?is_ipv4_addr(A0, A1, A2, A3) -> transportDomainUdpIpv4; -which_domain(Ip) when is_list(Ip) andalso (length(Ip) =:= 8) -> +which_domain([A0,A1,A2,A3,A4,A5,A6,A7]) + when ?is_ipv6_addr(A0, A1, A2, A3, A4, A5, A6, A7) -> + transportDomainUdpIpv6; +which_domain([A0,A1,A2,A3,A4,A5,A6,A7,A8,A9,A10,A11,A12,A13,A14,A15]) + when ?is_ipv6_addr( + A0, A1, A2, A3, A4, A5, A6, A7, + A8, A9, A10, A11, A12, A13, A14, A15) -> transportDomainUdpIpv6; -which_domain(Ip) when is_tuple(Ip) andalso (size(Ip) =:= 8) -> +which_domain({A0, A1, A2, A3, A4, A5, A6, A7}) + when ?is_ipv6_addr(A0, A1, A2, A3, A4, A5, A6, A7) -> transportDomainUdpIpv6. - %% --------- +mk_addr_string({Domain, Addr}) when is_atom(Domain) -> + %% XXX There is only code for IP domains here + case check_address_ip(Domain, Addr) of + false -> + case check_address_ip_port(Domain, Addr) of + false -> + error({bad_address, {Domain, Addr}}); + true -> + {IP, Port} = Addr, + mk_addr_string_ntoa(Domain, IP, Port); + {IP, Port} -> + mk_addr_string_ntoa(Domain, IP, Port) + end; + true -> + mk_addr_string_ntoa(Domain, Addr); + IP -> + mk_addr_string_ntoa(Domain, IP) + end; +mk_addr_string({_IP, Port} = Addr) when is_integer(Port) -> + mk_addr_string({snmpUDPDomain, Addr}); +mk_addr_string(Strange) -> + lists:flatten(io_lib:format("~w", [Strange])). + + +mk_addr_string_ntoa({_, _, _, _} = IP) -> + inet:ntoa(IP); +mk_addr_string_ntoa(IP) -> + lists:flatten(io_lib:format("[~s]", [inet:ntoa(IP)])). + +mk_addr_string_ntoa(Domain, IP) -> + case domain_to_name(Domain) of + undefined -> + mk_addr_string_ntoa(IP); + Name -> + lists:flatten( + io_lib:format("~w://~s", [Name, mk_addr_string_ntoa(IP)])) + end. + +mk_addr_string_ntoa(Domain, IP, Port) -> + lists:flatten( + io_lib:format( + "~s:~w", [mk_addr_string_ntoa(Domain, IP), Port])). + +%% --------- + check_ip(X) -> check_ip(snmpUDPDomain, X). -check_ip(snmpUDPDomain, X) -> - check_ip(transportDomainUdpIpv4, X); -check_ip(transportDomainUdpIpv4, X) when is_list(X) andalso (length(X) =:= 4) -> - case (catch all_integer(X)) of - true -> +check_ip(Domain, IP) -> + %% XXX There is only code for IP domains here + case check_address_ip(Domain, IP) of + false -> + error({bad_address, {Domain, IP}}); + true -> ok; - false -> - error({invalid_ip_address, X}) - end; -check_ip(transportDomainUdpIpv4, X) -> - error({invalid_ip_address, X}); + FixedIP -> + {ok, FixedIP} + end. -check_ip(transportDomainUdpIpv6, X) when is_list(X) andalso (length(X) =:= 8) -> - case (catch all_integer(X)) of - true -> + +%% --------- + +check_port(Port) when ?is_word(Port) -> + ok; +check_port(Port) -> + error({bad_port, Port}). + +%% ip_port_to_domaddr(IP, Port) when ?is_word(Port) -> +%% %% XXX There is only code for IP domains here +%% case check_address_ip(transportDomainUdpIpv4, IP) of +%% false -> +%% case check_address_ip(transportDomainUdpIpv6, IP) of +%% false -> +%% error({bad_address, {transportDomainUdpIpv4, {IP, Port}}}); +%% true -> +%% {transportDomainUdpIpv6, {IP, Port}}; +%% FixedIP -> +%% {transportDomainUdpIpv6, {FixedIP, Port}} +%% end; +%% true -> +%% {transportDomainUdpIpv4, {IP, Port}}; +%% FixedIP -> +%% {transportDomainUdpIpv4, {FixedIP, Port}} +%% end; +%% ip_port_to_domaddr(IP, Port) -> +%% error({bad_address, {transportDomainUdpIpv4, {IP, Port}}}). + +%% Check a configuration term field from a file to see if it +%% can be fixed to be fed to mk_taddress/2. + +check_address(Domain, Address, DefaultPort) -> + %% If Address does not contain Port or contains Port =:= 0 + %% create an address containing DefaultPort + %% + %% XXX There is only code for IP domains here + case check_address_ip(Domain, Address) of + false -> + case check_address_ip_port(Domain, Address) of + false -> + error({bad_address, {Domain, Address}}); + true -> + case Address of + {IP, 0} -> + {ok, {IP, DefaultPort}}; + _ -> + ok + end; + {FixedIP, 0} -> + {ok, {FixedIP, DefaultPort}}; + FixedAddress -> + {ok, FixedAddress} + end; + true -> + {ok, {Address, DefaultPort}}; + FixedIP -> + {ok, {FixedIP, DefaultPort}} + end. + +check_address(Domain, Address) -> + %% Address has to contain Port + %% + %% XXX There is only code for IP domains here + case check_address_ip_port(Domain, Address) of + false -> + error({bad_address, {Domain, Address}}); + true -> ok; - false -> - error({invalid_ip_address, X}) + FixedAddress -> + {ok, FixedAddress} + end. + +%% -> IP +check_address_ip(Domain, Address) + when Domain =:= snmpUDPDomain; + Domain =:= transportDomainUdpIpv4 -> + case Address of + %% Erlang native format + {A0, A1, A2, A3} + when ?is_ipv4_addr(A0, A1, A2, A3) -> + true; + %% Erlangish format + [A0,A1,A2,A3] + when ?is_ipv4_addr(A0, A1, A2, A3) -> + {A0, A1, A2, A3}; + _ -> + false + end; +check_address_ip(transportDomainUdpIpv6, Address) -> + case Address of + %% Erlang native format + {A0, A1, A2, A3, A4, A5, A6, A7} + when ?is_ipv6_addr(A0, A1, A2, A3, A4, A5, A6, A7) -> + true; + %% Erlangish format + [A0,A1,A2,A3,A4,A5,A6,A7] + when ?is_ipv6_addr(A0, A1, A2, A3, A4, A5, A6, A7) -> + {A0, A1, A2, A3, A4, A5, A6, A7}; + %% SNMP standards format + [A0,A1,A2,A3,A4,A5,A6,A7,A8,A9,A10,A11,A12,A13,A14,A15] + when ?is_ipv6_addr( + A0, A1, A2, A3, A4, A5, A6, A7, + A8, A9, A10, A11, A12, A13, A14, A15) -> + {mk_word(A0, A1), mk_word(A2, A3), + mk_word(A4, A5), mk_word(A6, A7), + mk_word(A8, A9), mk_word(A10, A11), + mk_word(A12, A13), mk_word(A14, A15)}; + _ -> + false end; -check_ip(transportDomainUdpIpv6, X) -> - error({invalid_ip_address, X}); +check_address_ip(BadDomain, _) -> + error({bad_domain, BadDomain}). -check_ip(BadDomain, _X) -> - error({invalid_domain, BadDomain}). +%% -> {IP, Port} +check_address_ip_port(Domain, Address) + when Domain =:= snmpUDPDomain; + Domain =:= transportDomainUdpIpv4 -> + case Address of + {IP, Port} when ?is_word(Port) -> + case check_address_ip(Domain, IP) of + false -> + false; + true -> + true; + FixedIP -> + {FixedIP, Port} + end; + %% SNMP standards format + [A0,A1,A2,A3,P0,P1] + when ?is_ipv4_addr(A0, A1, A2, A3), ?is_word(P0, P1) -> + {{A0, A1, A2, A3}, mk_word(P0, P1)}; + _ -> + false + end; +check_address_ip_port(transportDomainUdpIpv6 = Domain, Address) -> + case Address of + {IP, Port} when ?is_word(Port) -> + case check_address_ip(Domain, IP) of + false -> + false; + true -> + true; + FixedIP -> + {FixedIP, Port} + end; + %% Erlang friendly list format + [A0,A1,A2,A3,A4,A5,A6,A7,P] + when ?is_ipv6_addr(A0, A1, A2, A3, A4, A5, A6, A7), + ?is_word(P) -> + {{A0, A1, A2, A3, A4, A5, A6, A7}, P}; + %% Strange hybrid format with port as bytes + [A0,A1,A2,A3,A4,A5,A6,A7,P0,P1] + when ?is_ipv6_addr(A0, A1, A2, A3, A4, A5, A6, A7), + ?is_word(P0, P1) -> + {{A0, A1, A2, A3, A4, A5, A6, A7}, mk_word(P0, P1)}; + %% SNMP standards format + [A0,A1,A2,A3,A4,A5,A6,A7,A8,A9,A10,A11,A12,A13,A14,A15,P0,P1] + when ?is_ipv6_addr( + A0, A1, A2, A3, A4, A5, A6, A7, + A8, A9, A10, A11, A12, A13, A14, A15), + ?is_word(P0, P1) -> + {{mk_word(A0, A1), mk_word(A2, A3), + mk_word(A4, A5), mk_word(A6, A7), + mk_word(A8, A9), mk_word(A10, A11), + mk_word(A12, A13), mk_word(A14, A15)}, + mk_word(P0, P1)}; + _ -> + false + end; +check_address_ip_port(BadDomain, _) -> + error({bad_domain, BadDomain}). diff --git a/lib/snmp/src/misc/snmp_config.erl b/lib/snmp/src/misc/snmp_config.erl index a222f842e5..17dfcd70b4 100644 --- a/lib/snmp/src/misc/snmp_config.erl +++ b/lib/snmp/src/misc/snmp_config.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2013. All Rights Reserved. +%% Copyright Ericsson AB 1996-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -26,24 +26,29 @@ -compile({no_auto_import,[error/1]}). -export([config/0]). --export([write_config_file/4, append_config_file/4, read_config_file/3]). +%%-export([write_config_file/4, append_config_file/4, read_config_file/4]). + +-export([write_config_file/6, append_config_file/6, read_config_file/4]). -export([write_agent_snmp_files/7, write_agent_snmp_files/12, + write_agent_snmp_files/6, write_agent_snmp_files/11, - write_agent_snmp_conf/5, + write_agent_snmp_conf/4, write_agent_snmp_conf/5, write_agent_snmp_context_conf/1, write_agent_snmp_community_conf/1, write_agent_snmp_standard_conf/2, - write_agent_snmp_target_addr_conf/4, - write_agent_snmp_target_addr_conf/6, + write_agent_snmp_target_addr_conf/3, + write_agent_snmp_target_addr_conf/4, + write_agent_snmp_target_addr_conf/5, + write_agent_snmp_target_addr_conf/6, write_agent_snmp_target_params_conf/2, write_agent_snmp_notify_conf/2, write_agent_snmp_usm_conf/5, write_agent_snmp_vacm_conf/3, write_manager_snmp_files/8, - write_manager_snmp_conf/5, - write_manager_snmp_users_conf/2, + write_manager_snmp_conf/4, write_manager_snmp_conf/5, + write_manager_snmp_users_conf/2, write_manager_snmp_agents_conf/2, write_manager_snmp_usm_conf/2 @@ -90,9 +95,9 @@ ]). --export_type([void/0, - verify_config_entry_function/0, - verify_config_function/0, +-export_type([void/0, + order_config_entry_function/0, + check_config_entry_function/0, write_config_function/0]). @@ -567,11 +572,9 @@ config_agent_snmp(Dir, Vsns) -> end, NT end, - case (catch write_agent_snmp_files(Dir, - Vsns, ManagerIP, TrapUdp, - AgentIP, AgentUDP, - SysName, NotifType, SecType, - Passwd, EngineID, MMS)) of + case (catch write_agent_snmp_files( + Dir, Vsns, ManagerIP, TrapUdp, AgentIP, AgentUDP, SysName, + NotifType, SecType, Passwd, EngineID, MMS)) of ok -> i("~n- - - - - - - - - - - - -"), i("Info: 1. SecurityName \"initial\" has noAuthNoPriv read access~n" @@ -1580,35 +1583,63 @@ remove_newline(Str) -> %% File generation %%====================================================================== +write_agent_snmp_files( + Dir, Vsns, Domain, ManagerAddr, AgentAddr, SysName) + when is_list(Dir), + is_list(Vsns), + is_atom(Domain), + is_list(SysName) -> + write_agent_snmp_files( + Dir, Vsns, Domain, ManagerAddr, AgentAddr, SysName, + trap, none, "", "agentEngine", 484). + %%---------------------------------------------------------------------- %% Dir: string() (ex: "../conf/") %% ManagerIP, AgentIP: [int(),int(),int(),int()] %% TrapUdp, AgentUDP: integer() %% SysName: string() %%---------------------------------------------------------------------- -write_agent_snmp_files(Dir, Vsns, ManagerIP, TrapUdp, - AgentIP, AgentUDP, SysName) - when is_list(Dir) andalso - is_list(Vsns) andalso - is_list(ManagerIP) andalso - is_integer(TrapUdp) andalso - is_list(AgentIP) andalso - is_integer(AgentUDP) andalso +write_agent_snmp_files( + Dir, Vsns, ManagerIP, TrapUDP, AgentIP, AgentUDP, SysName) + when is_list(Dir) andalso + is_list(Vsns) andalso + is_list(ManagerIP) andalso + is_integer(TrapUDP) andalso + is_list(AgentIP) andalso + is_integer(AgentUDP) andalso is_list(SysName) -> - write_agent_snmp_files(Dir, Vsns, ManagerIP, TrapUdp, AgentIP, AgentUDP, - SysName, "trap", none, "", "agentEngine", 484). + write_agent_snmp_files( + Dir, Vsns, ManagerIP, TrapUDP, AgentIP, AgentUDP, SysName, + trap, none, "", "agentEngine", 484). %% %% ----- Agent config files generator functions ----- %% -write_agent_snmp_files(Dir, Vsns, ManagerIP, TrapUdp, AgentIP, AgentUDP, - SysName, NotifType, SecType, Passwd, EngineID, MMS) -> +write_agent_snmp_files( + Dir, Vsns, Domain, ManagerAddr, AgentAddr, SysName, + NotifType, SecType, Passwd, EngineID, MMS) -> + write_agent_snmp_conf(Dir, Domain, AgentAddr, EngineID, MMS), + write_agent_snmp_context_conf(Dir), + write_agent_snmp_community_conf(Dir), + write_agent_snmp_standard_conf(Dir, SysName), + write_agent_snmp_target_addr_conf(Dir, Domain, ManagerAddr, Vsns), + write_agent_snmp_target_params_conf(Dir, Vsns), + write_agent_snmp_notify_conf(Dir, NotifType), + write_agent_snmp_usm_conf(Dir, Vsns, EngineID, SecType, Passwd), + write_agent_snmp_vacm_conf(Dir, Vsns, SecType), + ok. + +write_agent_snmp_files( + Dir, Vsns, ManagerIP, TrapUDP, AgentIP, AgentUDP, SysName, + NotifType, SecType, Passwd, EngineID, MMS) -> + Domain = snmp_target_mib:default_domain(), + ManagerAddr = {ManagerIP, TrapUDP}, write_agent_snmp_conf(Dir, AgentIP, AgentUDP, EngineID, MMS), write_agent_snmp_context_conf(Dir), write_agent_snmp_community_conf(Dir), write_agent_snmp_standard_conf(Dir, SysName), - write_agent_snmp_target_addr_conf(Dir, ManagerIP, TrapUdp, Vsns), + write_agent_snmp_target_addr_conf(Dir, Domain, ManagerAddr, Vsns), write_agent_snmp_target_params_conf(Dir, Vsns), write_agent_snmp_notify_conf(Dir, NotifType), write_agent_snmp_usm_conf(Dir, Vsns, EngineID, SecType, Passwd), @@ -1616,11 +1647,40 @@ write_agent_snmp_files(Dir, Vsns, ManagerIP, TrapUdp, AgentIP, AgentUDP, ok. + %% %% ------ [agent] agent.conf ------ %% -write_agent_snmp_conf(Dir, AgentIP, AgentUDP, EngineID, MMS) -> +write_agent_snmp_conf(Dir, Transports, EngineID, MMS) -> + Conf = + [{intAgentTransports, Transports}, + {snmpEngineID, EngineID}, + {snmpEngineMaxMessageSize, MMS}], + do_write_agent_snmp_conf(Dir, Conf). + +write_agent_snmp_conf(Dir, Domain, AgentAddr, EngineID, MMS) + when is_atom(Domain) -> + {AgentIP, AgentUDP} = AgentAddr, + Conf = + [{intAgentTransportDomain, Domain}, + {intAgentUDPPort, AgentUDP}, + {intAgentIpAddress, AgentIP}, + {snmpEngineID, EngineID}, + {snmpEngineMaxMessageSize, MMS}], + do_write_agent_snmp_conf(Dir, Conf); +write_agent_snmp_conf(Dir, AgentIP, AgentUDP, EngineID, MMS) + when is_integer(AgentUDP) -> + Conf = + [{intAgentUDPPort, AgentUDP}, + {intAgentIpAddress, AgentIP}, + {snmpEngineID, EngineID}, + {snmpEngineMaxMessageSize, MMS}], + do_write_agent_snmp_conf(Dir, Conf); +write_agent_snmp_conf(_Dir, Domain, AgentAddr, _EngineID, _MMS) -> + error({bad_address, {Domain, AgentAddr}}). + +do_write_agent_snmp_conf(Dir, Conf) -> Comment = "%% This file defines the Agent local configuration info\n" "%% The data is inserted into the snmpEngine* variables defined\n" @@ -1635,11 +1695,7 @@ write_agent_snmp_conf(Dir, AgentIP, AgentUDP, EngineID, MMS) -> "%% {snmpEngineID, \"agentEngine\"}.\n" "%% {snmpEngineMaxMessageSize, 484}.\n" "%%\n\n", - Hdr = header() ++ Comment, - Conf = [{intAgentUDPPort, AgentUDP}, - {intAgentIpAddress, AgentIP}, - {snmpEngineID, EngineID}, - {snmpEngineMaxMessageSize, MMS}], + Hdr = header() ++ Comment, write_agent_config(Dir, Hdr, Conf). write_agent_config(Dir, Hdr, Conf) -> @@ -1745,17 +1801,19 @@ update_agent_standard_config(Dir, Conf) -> %% ------ target_addr.conf ------ %% -write_agent_snmp_target_addr_conf(Dir, ManagerIp, UDP, Vsns) -> - Timeout = 1500, - RetryCount = 3, - write_agent_snmp_target_addr_conf(Dir, ManagerIp, UDP, - Timeout, RetryCount, - Vsns). +write_agent_snmp_target_addr_conf(Dir, Addresses, Vsns) -> + Timeout = 1500, + RetryCount = 3, + write_agent_snmp_target_addr_conf( + Dir, Addresses, Timeout, RetryCount, Vsns). -write_agent_snmp_target_addr_conf(Dir, ManagerIp, UDP, - Timeout, RetryCount, - Vsns) -> - Comment = +write_agent_snmp_target_addr_conf(Dir, Domain_or_Ip, Addr_or_Port, Vsns) -> + Addresses = [{Domain_or_Ip, Addr_or_Port}], + write_agent_snmp_target_addr_conf(Dir, Addresses, Vsns). + +write_agent_snmp_target_addr_conf( + Dir, Addresses, Timeout, RetryCount, Vsns) -> + Comment = "%% This file defines the target address parameters.\n" "%% The data is inserted into the snmpTargetAddrTable defined\n" "%% in SNMP-TARGET-MIB, and in the snmpTargetAddrExtTable defined\n" @@ -1774,36 +1832,59 @@ write_agent_snmp_target_addr_conf(Dir, ManagerIp, UDP, "%% [127,0,0,0], 2048}.\n" "%%\n\n", Hdr = header() ++ Comment, - F = fun(v1 = Vsn, Acc) -> - [{mk_ip(ManagerIp, Vsn), - snmp_target_mib:default_domain(), - ManagerIp, UDP, Timeout, RetryCount, - "std_trap", mk_param(Vsn), "", [], 2048}| Acc]; - (v2 = Vsn, Acc) -> - [{mk_ip(ManagerIp, Vsn), - snmp_target_mib:default_domain(), - ManagerIp, UDP, Timeout, RetryCount, - "std_trap", mk_param(Vsn), "", [], 2048}, - {lists:flatten(io_lib:format("~s.2",[mk_ip(ManagerIp, Vsn)])), - ManagerIp, UDP, Timeout, RetryCount, - "std_inform", mk_param(Vsn), "", [], 2048}| Acc]; - (v3 = Vsn, Acc) -> - [{mk_ip(ManagerIp, Vsn), - snmp_target_mib:default_domain(), - ManagerIp, UDP, Timeout, RetryCount, - "std_trap", mk_param(Vsn), "", [], 2048}, - {lists:flatten(io_lib:format("~s.3",[mk_ip(ManagerIp, Vsn)])), - ManagerIp, UDP, Timeout, RetryCount, - "std_inform", mk_param(Vsn), "mgrEngine", [], 2048}| Acc] - end, - Conf = lists:foldl(F, [], Vsns), + Conf = + lists:foldl( + fun ({Domain_or_Ip, Addr_or_Port} = Address, OuterAcc) -> + lists:foldl( + fun(v1 = Vsn, Acc) -> + [{mk_name(Address, Vsn), + Domain_or_Ip, Addr_or_Port, + Timeout, RetryCount, + "std_trap", mk_param(Vsn), "", + [], 2048}| Acc]; + (v2 = Vsn, Acc) -> + [{mk_name(Address, Vsn), + Domain_or_Ip, Addr_or_Port, + Timeout, RetryCount, + "std_trap", mk_param(Vsn), "", + [], 2048}, + {lists:flatten( + io_lib:format( + "~s.2", [mk_name(Address, Vsn)])), + Domain_or_Ip, Addr_or_Port, + Timeout, RetryCount, + "std_inform", mk_param(Vsn), "", + [], 2048}| Acc]; + (v3 = Vsn, Acc) -> + [{mk_name(Address, Vsn), + Domain_or_Ip, Addr_or_Port, + Timeout, RetryCount, + "std_trap", mk_param(Vsn), "", + [], 2048}, + {lists:flatten( + io_lib:format( + "~s.3", [mk_name(Address, Vsn)])), + Domain_or_Ip, Addr_or_Port, + Timeout, RetryCount, + "std_inform", mk_param(Vsn), "mgrEngine", + [], 2048} | Acc] + end, OuterAcc, Vsns) + end, [], Addresses), write_agent_target_addr_config(Dir, Hdr, Conf). +write_agent_snmp_target_addr_conf( + Dir, Domain_or_Ip, Addr_or_Port, Timeout, RetryCount, Vsns) -> + Addresses = [{Domain_or_Ip, Addr_or_Port}], + write_agent_snmp_target_addr_conf( + Dir, Addresses, Timeout, RetryCount, Vsns). + mk_param(Vsn) -> lists:flatten(io_lib:format("target_~w", [Vsn])). -mk_ip([A,B,C,D], Vsn) -> - lists:flatten(io_lib:format("~w.~w.~w.~w ~w", [A,B,C,D,Vsn])). +mk_name(Address, Vsn) -> + lists:flatten( + io_lib:format( + "~s ~w", [snmp_conf:mk_addr_string(Address), Vsn])). write_agent_target_addr_config(Dir, Hdr, Conf) -> snmpa_conf:write_target_addr_config(Dir, Hdr, Conf). @@ -2037,7 +2118,24 @@ write_manager_snmp_files(Dir, IP, Port, MMS, EngineID, %% ------ manager.conf ------ %% -write_manager_snmp_conf(Dir, IP, Port, MMS, EngineID) -> +write_manager_snmp_conf(Dir, Transports, MMS, EngineID) -> + Comment = +"%% This file defines the Manager local configuration info\n" +"%% Each row is a 2-tuple:\n" +"%% {Variable, Value}.\n" +"%% For example\n" +"%% {transports, [{transportDomainUdpIpv4, {{127,42,17,5}, 5000}}]}.\n" +"%% {engine_id, \"managerEngine\"}.\n" +"%% {max_message_size, 484}.\n" +"%%\n\n", + Hdr = header() ++ Comment, + Conf = + [{transports, Transports}, + {engine_id, EngineID}, + {max_message_size, MMS}], + write_manager_config(Dir, Hdr, Conf). + +write_manager_snmp_conf(Dir, Domain_or_IP, Addr_or_Port, MMS, EngineID) -> Comment = "%% This file defines the Manager local configuration info\n" "%% Each row is a 2-tuple:\n" @@ -2049,10 +2147,20 @@ write_manager_snmp_conf(Dir, IP, Port, MMS, EngineID) -> "%% {max_message_size, 484}.\n" "%%\n\n", Hdr = header() ++ Comment, - Conf = [{port, Port}, - {address, IP}, - {engine_id, EngineID}, - {max_message_size, MMS}], + Conf = + case Addr_or_Port of + {IP, Port} when is_integer(Port), is_atom(Domain_or_IP) -> + [{domain, Domain_or_IP}, + {port, Port}, + {address, IP}]; + _ when is_integer(Addr_or_Port) -> + [{port, Addr_or_Port}, + {address, Domain_or_IP}]; + _ -> + error({bad_address, {Domain_or_IP, Addr_or_Port}}) + end ++ + [{engine_id, EngineID}, + {max_message_size, MMS}], write_manager_config(Dir, Hdr, Conf). write_manager_config(Dir, Hdr, Conf) -> @@ -2410,83 +2518,92 @@ header() -> [?MODULE, ?version, Y, Mo, D, H, Mi, S]). -%% *If* these functions are successfull, they successfully return anything -%% (which is ignored), but they fail with either a throw or an exit or -%% something similar. - -%% Verification of one config entry read from a file --type(verify_config_entry_function() :: - fun((Entry :: term()) -> ok | {error, Reason :: term()})). - -%% Verification of config to be written --type(verify_config_function() :: - fun(() -> void())). - -%% Write config to file (as defined by Fd) --type(write_config_function() :: - fun((Fd :: file:io_device()) -> void())). - --spec write_config_file(Dir :: string(), - FileName :: string(), - Verify :: verify_config_function(), - Write :: write_config_function()) -> - ok | {error, Reason :: term()}. - -write_config_file(Dir, FileName, Verify, Write) - when (is_list(Dir) andalso - is_list(FileName) andalso - is_function(Verify) andalso - is_function(Write)) -> +%% *If* these functions are successfull, they successfully return +%% (value is ignored), but they fail preferably with +%% throw({error, Reason}). Other exceptions are also handled. + +%% Sorting order for config entries (see lists:sort/2) +-type(order_config_entry_function() :: + fun((term(), term()) -> boolean())). + +%% Check of config entries. Initial State is 'undefined' +-type(check_config_entry_function() :: + fun((Entry :: term(), State :: undefined | term()) -> + {ok | {ok, NewEntry :: term()}, NewState :: term()})). + +%% Write configuration entries to file descriptor Fd +-type(write_config_function() :: + fun((Fd :: file:io_device(), [Entry :: term()]) -> ok)). + +-spec write_config_file( + Dir :: string(), + FileName :: string(), + Order :: order_config_entry_function(), + Check :: check_config_entry_function(), + Write :: write_config_function(), + Entries :: [term()]) -> + ok | {error, term()}. + +write_config_file(Dir, FileName, Order, Check, Write, Entries) + when is_list(Dir), is_list(FileName), + is_function(Order), is_function(Check), is_function(Write), + is_list(Entries) -> try - begin - do_write_config_file(Dir, FileName, Verify, Write) - end + SortedEntries = lists:sort(Order, Entries), + _ = + lists:foldl( + fun (Entry, State) -> + case Check(Entry, State) of + {Ok, NewState} when is_list(Ok) -> + NewState; + {ok, NewState} -> + NewState; + {{ok, _}, NewState} -> + NewState + end + end, undefined, SortedEntries), + ok + of + _ -> + case file:open(filename:join(Dir, FileName), [write]) of + {ok, Fd} -> + write_config_file(Dir, FileName, Write, Entries, Fd); + Error -> + Error + end catch - throw:Error -> - Error; - T:E -> - {error, {failed_write, Dir, FileName, T, E}} - end. - - -do_write_config_file(Dir, FileName, Verify, Write) -> - Verify(), - case file:open(filename:join(Dir, FileName), [write]) of - {ok, Fd} -> - file_write_and_close(Write, Fd, Dir, FileName); Error -> - Error - end. - -append_config_file(Dir, FileName, Verify, Write) - when (is_list(Dir) andalso - is_list(FileName) andalso - is_function(Verify) andalso - is_function(Write)) -> - try - begin - do_append_config_file(Dir, FileName, Verify, Write) - end - catch - throw:Error -> + S = erlang:get_stacktrace(), + d("File write of ~s throwed: ~p~n ~p~n", + [FileName, Error, S]), Error; - T:E -> - {error, {failed_append, Dir, FileName, T, E}} + C:E -> + S = erlang:get_stacktrace(), + d("File write of ~s exception: ~p:~p~n ~p~n", + [FileName,C,E,S]), + {error, {failed_write, Dir, FileName, {C, E, S}}} end. -do_append_config_file(Dir, FileName, Verify, Write) -> - Verify(), - case file:open(filename:join(Dir, FileName), [read, write]) of - {ok, Fd} -> - file:position(Fd, eof), - file_write_and_close(Write, Fd, Dir, FileName); +write_config_file(Dir, FileName, Write, Entries, Fd) -> + try Write(Fd, Entries) of + ok -> + close_config_file(Dir, FileName, Fd) + catch Error -> - Error + S = erlang:get_stacktrace(), + d("File write of ~s throwed: ~p~n ~p~n", + [FileName, Error, S]), + close_config_file(Dir, FileName, Fd), + Error; + C:E -> + S = erlang:get_stacktrace(), + d("File write of ~s exception: ~p:~p~n ~p~n", + [FileName,C,E,S]), + close_config_file(Dir, FileName, Fd), + {error, {failed_write, Dir, FileName, {C, E, S}}} end. - -file_write_and_close(Write, Fd, Dir, FileName) -> - ok = Write(Fd), +close_config_file(Dir, FileName, Fd) -> case file:sync(Fd) of ok -> case file:close(Fd) of @@ -2496,63 +2613,159 @@ file_write_and_close(Write, Fd, Dir, FileName) -> {error, {failed_closing, Dir, FileName, Reason}} end; {error, Reason} -> + _ = file:close(Fd), {error, {failed_syncing, Dir, FileName, Reason}} end. --spec read_config_file(Dir :: string(), - FileName :: string(), - Verify :: verify_config_entry_function()) -> - {ok, Config :: list()} | {error, Reason :: term()}. -read_config_file(Dir, FileName, Verify) - when is_list(Dir) andalso is_list(FileName) andalso is_function(Verify) -> - (catch do_read_config_file(Dir, FileName, Verify)). +-spec append_config_file( + Dir :: string(), + FileName :: string(), + Order :: order_config_entry_function(), + Check :: check_config_entry_function(), + Write :: write_config_function(), + Entries :: [term()]) -> + ok | {error, term()}. + +append_config_file(Dir, FileName, Order, Check, Write, Entries) + when is_list(Dir), is_list(FileName), + is_function(Order), is_function(Check), is_function(Write), + is_list(Entries) -> + case file:open(filename:join(Dir, FileName), [read, write]) of + {ok, Fd} -> + append_config_file( + Dir, FileName, Order, Check, Write, Entries, Fd); + Error -> + Error + end. + +append_config_file(Dir, FileName, Order, Check, Write, Entries, Fd) -> + try + %% Verify the entries together with the file content + LinesInFileR = read_lines(Fd, [], 1), + StartLine = + case LinesInFileR of + [] -> + 1; + [{_, _, EndLine} | _] -> + EndLine + end, + LinesR = prepend_lines(LinesInFileR, Entries, StartLine), + SortedLines = sort_lines(lists:reverse(LinesR), Order), + _ = verify_lines(SortedLines, Check, undefined, []), + %% Append to the file + Write(Fd, Entries) + of + ok -> + close_config_file(Dir, FileName, Fd) + catch + Error -> + S = erlang:get_stacktrace(), + d("File append of ~s throwed: ~p~n ~p~n", + [FileName, Error, S]), + close_config_file(Dir, FileName, Fd), + Error; + C:E -> + S = erlang:get_stacktrace(), + d("File append of ~s exception: ~p:~p~n ~p~n", + [FileName,C,E,S]), + close_config_file(Dir, FileName, Fd), + {error, {failed_append, Dir, FileName, {C, E, S}}} + end. + +%% Fake line numbers, one per entry +prepend_lines(Lines, [], _) -> + Lines; +prepend_lines(Lines, [Entry | Entries], StartLine) -> + EndLine = StartLine + 1, + prepend_lines([{StartLine, Entry, EndLine} | Lines], Entries, EndLine). + + -do_read_config_file(Dir, FileName, Verify) -> +-spec read_config_file( + Dir :: string(), + FileName :: string(), + Order :: order_config_entry_function(), + Check :: check_config_entry_function()) -> + {ok, Config :: [Entry :: term()]} | + {error, Reason :: term()}. + +read_config_file(Dir, FileName, Order, Check) + when is_list(Dir), is_list(FileName), + is_function(Order), is_function(Check) -> case file:open(filename:join(Dir, FileName), [read]) of {ok, Fd} -> - Result = read_loop(Fd, [], Verify, 1), - file:close(Fd), - Result; + try + Lines = lists:reverse(read_lines(Fd, [], 1)), + SortedLines = sort_lines(Lines, Order), + {ok, verify_lines(SortedLines, Check, undefined, [])} + catch + Error -> + S = erlang:get_stacktrace(), + d("File read of ~s throwed: ~p~n ~p~n", + [FileName, Error, S]), + {error, Error}; + T:E -> + S = erlang:get_stacktrace(), + d("File read of ~s exception: ~p:~p~n ~p~n", + [FileName,T,E,S]), + {error, {failed_read, Dir, FileName, {T, E, S}}} + after + file:close(Fd) + end; {error, Reason} -> {error, {Reason, FileName}} end. -read_loop(Fd, Acc, Check, StartLine) -> - case read_term(Fd, StartLine) of +read_lines(Fd, Acc, StartLine) -> + case read_and_parse_term(Fd, StartLine) of {ok, Term, EndLine} -> - case (catch Check(Term)) of - ok -> - read_loop(Fd, [Term | Acc], Check, EndLine); - {error, Reason} -> - {error, {failed_check, StartLine, EndLine, Reason}}; - Error -> - {error, {failed_check, StartLine, EndLine, Error}} - end; - {error, EndLine, Error} -> - {error, {failed_reading, StartLine, EndLine, Error}}; - eof -> - {ok, lists:reverse(Acc)} + read_lines(Fd, [{StartLine, Term, EndLine}|Acc], EndLine); + {error, Error, EndLine} -> + throw({failed_reading, StartLine, EndLine, Error}); + {eof, _EndLine} -> + Acc end. - -read_term(Fd, StartLine) -> + +read_and_parse_term(Fd, StartLine) -> case io:request(Fd, {get_until, "", erl_scan, tokens, [StartLine]}) of {ok, Tokens, EndLine} -> case erl_parse:parse_term(Tokens) of {ok, Term} -> {ok, Term, EndLine}; {error, {Line, erl_parse, Error}} -> - {error, Line, {parse_error, Error}} + {error, {parse_error, Error}, Line} end; - {error, E, EndLine} -> - {error, EndLine, E}; - {eof, _EndLine} -> - eof; Other -> Other end. +sort_lines(Lines, Order) -> + lists:sort( + fun ({_, T1, _}, {_, T2, _}) -> + Order(T1, T2) + end, Lines). + +verify_lines([], _, _, Acc) -> + lists:reverse(Acc); +verify_lines( + [{StartLine, Term, EndLine}|Lines], Check, State, Acc) -> + try Check(Term, State) of + {Terms, NewState} when is_list(Terms) -> + verify_lines(Lines, Check, NewState, Terms ++ Acc); + {ok, NewState} -> + verify_lines(Lines, Check, NewState, [Term|Acc]); + {{ok, NewTerm}, NewState} -> + verify_lines(Lines, Check, NewState, [NewTerm|Acc]) + catch + {error, Reason} -> + throw({failed_check, StartLine, EndLine, Reason}); + C:R -> + S = erlang:get_stacktrace(), + throw({failed_check, StartLine, EndLine, {C, R, S}}) + end. + agent_snmp_mk_secret(Alg, Passwd, EngineID) -> snmp_usm:passwd2localized_key(Alg, Passwd, EngineID). @@ -2575,8 +2788,8 @@ ensure_started(App) -> %% ------------------------------------------------------------------------- -% d(F, A) -> -% i("DBG: " ++ F, A). +d(F, A) -> + i("DBG: " ++ F, A). i(F) -> i(F, []). diff --git a/lib/snmp/src/misc/snmp_log.erl b/lib/snmp/src/misc/snmp_log.erl index ae28df37fa..4b22281b89 100644 --- a/lib/snmp/src/misc/snmp_log.erl +++ b/lib/snmp/src/misc/snmp_log.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2012. All Rights Reserved. +%% Copyright Ericsson AB 1997-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -23,7 +23,7 @@ -export([ create/4, create/5, create/6, open/1, open/2, change_size/2, close/1, sync/1, info/1, - log/4, + log/3, log/4, log_to_txt/6, log_to_txt/7, log_to_txt/8, log_to_io/5, log_to_io/6, log_to_io/7 ]). @@ -231,7 +231,20 @@ validate(Log, SeqNoReq) validate_seqno(PrevSN, SeqNo), {Timestamp, SeqNo}; - ({Timestamp, _Packet, _Addr, _Port}, {PrevTS, _PrevSN}) when SeqNoReq =:= true -> + ({Timestamp, SeqNo, _Packet, _AddrStr}, {PrevTS, PrevSN}) + when is_integer(SeqNo) -> + ?vtrace("validating log entry when" + "~n Timestamp: ~p" + "~n SeqNo: ~p" + "~n PrevTS: ~p" + "~n PrevSN: ~p", + [Timestamp, SeqNo, PrevTS, PrevSN]), + validate_timestamp(PrevTS, Timestamp), + validate_seqno(PrevSN, SeqNo), + {Timestamp, SeqNo}; + + ({Timestamp, _Packet, _Addr, _Port}, {PrevTS, _PrevSN}) + when SeqNoReq =:= true -> ?vtrace("validating log entry when" "~n Timestamp: ~p" "~n PrevTS: ~p", @@ -344,12 +357,20 @@ validate_loop(Error, _Log, _Write, _PrevTS, _PrevSN) -> %% log(Log, Packet, Addr, Port) %%----------------------------------------------------------------- -log(#snmp_log{id = Log, seqno = SeqNo}, Packet, Addr, Port) -> +log(#snmp_log{id = Log, seqno = SeqNo}, Packet, AddrStr) -> + ?vtrace( + "log -> entry with~n" + " Log: ~p~n" + " AddrStr: ~s", [Log, AddrStr]), + Entry = make_entry(SeqNo, Packet, AddrStr), + disk_log:alog(Log, Entry). + +log(#snmp_log{id = Log, seqno = SeqNo}, Packet, Ip, Port) -> ?vtrace("log -> entry with" "~n Log: ~p" - "~n Addr: ~p" - "~n Port: ~p", [Log, Addr, Port]), - Entry = make_entry(SeqNo, Packet, Addr, Port), + "~n Ip: ~p" + "~n Port: ~p", [Log, Ip, Port]), + Entry = make_entry(SeqNo, Packet, Ip, Port), %% io:format("log -> " %% "~n Entry: ~p" %% "~n Info: ~p" @@ -361,18 +382,35 @@ log(#snmp_log{id = Log, seqno = SeqNo}, Packet, Addr, Port) -> %% "~n", [Res, disk_log:info(Log)]), %% disk_log:sync(Log), Res. - -make_entry(SeqNoGen, Packet, Addr, Port) -> + +make_entry(SeqNoGen, Packet, AddrStr) + when is_integer(Packet); + is_tuple(AddrStr) -> + erlang:error(badarg, [SeqNoGen, Packet, AddrStr]); +make_entry(SeqNoGen, Packet, AddrStr) -> try next_seqno(SeqNoGen) of disabled -> - {timestamp(), Packet, Addr, Port}; - {ok, NextSeqNo} -> - {timestamp(), NextSeqNo, Packet, Addr, Port} + {timestamp(), Packet, AddrStr}; + {ok, NextSeqNo} when is_integer(NextSeqNo) -> + {timestamp(), NextSeqNo, Packet, AddrStr} catch _:_ -> - {timestamp(), Packet, Addr, Port} + {timestamp(), Packet, AddrStr} + end. + +make_entry(SeqNoGen, Packet, Ip, Port) when is_integer(Packet) -> + erlang:error(badarg, [SeqNoGen, Packet, Ip, Port]); +make_entry(SeqNoGen, Packet, Ip, Port) -> + try next_seqno(SeqNoGen) of + disabled -> + {timestamp(), Packet, Ip, Port}; + {ok, NextSeqNo} when is_integer(NextSeqNo) -> + {timestamp(), NextSeqNo, Packet, Ip, Port} + catch + _:_ -> + {timestamp(), Packet, Ip, Port} end. next_seqno({M, F, A}) -> @@ -674,60 +712,68 @@ format_msg(Entry, Mib, Start, Stop) -> end. %% This is an old-style entry, that never had the sequence-number -do_format_msg({Timestamp, Packet, {Addr, Port}}, Mib) -> - do_format_msg(Timestamp, Packet, Addr, Port, Mib); +do_format_msg({Timestamp, Packet, {Ip, Port}}, Mib) -> + do_format_msg(Timestamp, Packet, ipPort2Str(Ip, Port), Mib); +%% This is the format without sequence-number +do_format_msg({Timestamp, Packet, AddrStr}, Mib) -> + do_format_msg(Timestamp, Packet, AddrStr, Mib); +%% This is the format with sequence-number +do_format_msg({Timestamp, SeqNo, Packet, AddrStr}, Mib) + when is_integer(SeqNo) -> + do_format_msg(Timestamp, Packet, AddrStr, Mib); %% This is the format without sequence-number -do_format_msg({Timestamp, Packet, Addr, Port}, Mib) -> - do_format_msg(Timestamp, Packet, Addr, Port, Mib); +do_format_msg({Timestamp, Packet, Ip, Port}, Mib) -> + do_format_msg(Timestamp, Packet, ipPort2Str(Ip, Port), Mib); %% This is the format with sequence-number -do_format_msg({Timestamp, SeqNo, Packet, Addr, Port}, Mib) -> - do_format_msg(Timestamp, SeqNo, Packet, Addr, Port, Mib); +do_format_msg({Timestamp, SeqNo, Packet, Ip, Port}, Mib) -> + do_format_msg(Timestamp, SeqNo, Packet, ipPort2Str(Ip, Port), Mib); %% This is crap... do_format_msg(_, _) -> format_tab("** unknown entry in log file\n\n", []). -do_format_msg(TimeStamp, {V3Hdr, ScopedPdu}, Addr, Port, Mib) -> +do_format_msg(TimeStamp, {V3Hdr, ScopedPdu}, AddrStr, Mib) -> case (catch snmp_pdus:dec_scoped_pdu(ScopedPdu)) of ScopedPDU when is_record(ScopedPDU, scopedPdu) -> Msg = #message{version = 'version-3', vsn_hdr = V3Hdr, data = ScopedPDU}, - f(ts2str(TimeStamp), "", Msg, Addr, Port, Mib); + f(ts2str(TimeStamp), "", Msg, AddrStr, Mib); {'EXIT', Reason} -> - format_tab("** error in log file at ~s from ~p:~w ~p\n\n", - [ts2str(TimeStamp), ip(Addr), Port, Reason]) + format_tab( + "** error in log file at ~s from ~s ~p\n\n", + [ts2str(TimeStamp), AddrStr, Reason]) end; -do_format_msg(TimeStamp, Packet, Addr, Port, Mib) -> +do_format_msg(TimeStamp, Packet, AddrStr, Mib) -> case (catch snmp_pdus:dec_message(binary_to_list(Packet))) of Msg when is_record(Msg, message) -> - f(ts2str(TimeStamp), "", Msg, Addr, Port, Mib); + f(ts2str(TimeStamp), "", Msg, AddrStr, Mib); {'EXIT', Reason} -> format_tab("** error in log file ~p\n\n", [Reason]) end. -do_format_msg(TimeStamp, SeqNo, {V3Hdr, ScopedPdu}, Addr, Port, Mib) -> +do_format_msg(TimeStamp, SeqNo, {V3Hdr, ScopedPdu}, AddrStr, Mib) -> case (catch snmp_pdus:dec_scoped_pdu(ScopedPdu)) of ScopedPDU when is_record(ScopedPDU, scopedPdu) -> Msg = #message{version = 'version-3', vsn_hdr = V3Hdr, data = ScopedPDU}, - f(ts2str(TimeStamp), sn2str(SeqNo), Msg, Addr, Port, Mib); + f(ts2str(TimeStamp), sn2str(SeqNo), Msg, AddrStr, Mib); {'EXIT', Reason} -> - format_tab("** error in log file at ~s from ~p:~w ~p\n\n", - [ts2str(TimeStamp), sn2str(SeqNo), - ip(Addr), Port, Reason]) + format_tab( + "** error in log file at ~s from ~s ~p\n\n", + [ts2str(TimeStamp), sn2str(SeqNo), AddrStr, Reason]) end; -do_format_msg(TimeStamp, SeqNo, Packet, Addr, Port, Mib) -> +do_format_msg(TimeStamp, SeqNo, Packet, AddrStr, Mib) -> case (catch snmp_pdus:dec_message(binary_to_list(Packet))) of Msg when is_record(Msg, message) -> - f(ts2str(TimeStamp), sn2str(SeqNo), Msg, Addr, Port, Mib); + f(ts2str(TimeStamp), sn2str(SeqNo), Msg, AddrStr, Mib); {'EXIT', Reason} -> - format_tab("** error in log file ~s from ~p:~w ~p\n\n", - [ts2str(TimeStamp), sn2str(SeqNo), - ip(Addr), Port, Reason]) + format_tab( + "** error in log file ~s from ~s ~p\n\n", + [ts2str(TimeStamp), sn2str(SeqNo), AddrStr, Reason]) end. @@ -771,44 +817,71 @@ do_format_msg(TimeStamp, SeqNo, Packet, Addr, Port, Mib) -> f(TimeStamp, SeqNo, #message{version = Vsn, vsn_hdr = VsnHdr, data = Data}, - Addr, Port, Mib) -> + AddrStr, Mib) -> Str = format_pdu(Data, Mib), HdrStr = format_header(Vsn, VsnHdr), - case get_type(Data) of - trappdu -> - f_trap(TimeStamp, SeqNo, Vsn, HdrStr, Str, Addr, Port); - 'snmpv2-trap' -> - f_trap(TimeStamp, SeqNo, Vsn, HdrStr, Str, Addr, Port); - 'inform-request' -> - f_inform(TimeStamp, SeqNo, Vsn, HdrStr, Str, Addr, Port); - 'get-response' -> - f_response(TimeStamp, SeqNo, Vsn, HdrStr, Str, Addr, Port); - report -> - f_report(TimeStamp, SeqNo, Vsn, HdrStr, Str, Addr, Port); - _ -> - f_request(TimeStamp, SeqNo, Vsn, HdrStr, Str, Addr, Port) - end. + Class = + case get_type(Data) of + trappdu -> + trap; + 'snmpv2-trap' -> + trap; + 'inform-request' -> + inform; + 'get-response' -> + response; + report -> + report; + _ -> + request + end, + format_tab( + "~w ~s - ~s [~s]~s ~w\n~s", + [Class, AddrStr, HdrStr, TimeStamp, SeqNo, Vsn, Str]). + +%% f(TimeStamp, SeqNo, +%% #message{version = Vsn, vsn_hdr = VsnHdr, data = Data}, +%% Addr, Port, Mib) -> +%% Str = format_pdu(Data, Mib), +%% HdrStr = format_header(Vsn, VsnHdr), +%% case get_type(Data) of +%% trappdu -> +%% f_trap(TimeStamp, SeqNo, Vsn, HdrStr, Str, Addr, Port); +%% 'snmpv2-trap' -> +%% f_trap(TimeStamp, SeqNo, Vsn, HdrStr, Str, Addr, Port); +%% 'inform-request' -> +%% f_inform(TimeStamp, SeqNo, Vsn, HdrStr, Str, Addr, Port); +%% 'get-response' -> +%% f_response(TimeStamp, SeqNo, Vsn, HdrStr, Str, Addr, Port); +%% report -> +%% f_report(TimeStamp, SeqNo, Vsn, HdrStr, Str, Addr, Port); +%% _ -> +%% f_request(TimeStamp, SeqNo, Vsn, HdrStr, Str, Addr, Port) +%% end. -f_request(TimeStamp, SeqNo, Vsn, HdrStr, Str, Addr, Port) -> - format_tab("request ~s:~w - ~s [~s]~s ~w\n~s", - [ip(Addr), Port, HdrStr, TimeStamp, SeqNo, Vsn, Str]). +%% f_request(TimeStamp, SeqNo, Vsn, HdrStr, Str, Addr, Port) -> +%% format_tab("request ~s:~w - ~s [~s]~s ~w\n~s", +%% [ip(Addr), Port, HdrStr, TimeStamp, SeqNo, Vsn, Str]). -f_response(TimeStamp, SeqNo, Vsn, HdrStr, Str, Addr, Port) -> - format_tab("response ~s:~w - ~s [~s]~s ~w\n~s", - [ip(Addr), Port, HdrStr, TimeStamp, SeqNo, Vsn, Str]). +%% f_response(TimeStamp, SeqNo, Vsn, HdrStr, Str, Addr, Port) -> +%% format_tab("response ~s:~w - ~s [~s]~s ~w\n~s", +%% [ip(Addr), Port, HdrStr, TimeStamp, SeqNo, Vsn, Str]). -f_report(TimeStamp, SeqNo, Vsn, HdrStr, Str, Addr, Port) -> - format_tab("report ~s:~w - ~s [~s]~s ~w\n~s", - [ip(Addr), Port, HdrStr, TimeStamp, SeqNo, Vsn, Str]). +%% f_report(TimeStamp, SeqNo, Vsn, HdrStr, Str, Addr, Port) -> +%% format_tab("report ~s:~w - ~s [~s]~s ~w\n~s", +%% [ip(Addr), Port, HdrStr, TimeStamp, SeqNo, Vsn, Str]). -f_trap(TimeStamp, SeqNo, Vsn, HdrStr, Str, Addr, Port) -> - format_tab("trap ~s:~w - ~s [~s]~s ~w\n~s", - [ip(Addr), Port, HdrStr, TimeStamp, SeqNo, Vsn, Str]). +%% f_trap(TimeStamp, SeqNo, Vsn, HdrStr, Str, Addr, Port) -> +%% format_tab("trap ~s:~w - ~s [~s]~s ~w\n~s", +%% [ip(Addr), Port, HdrStr, TimeStamp, SeqNo, Vsn, Str]). -f_inform(TimeStamp, SeqNo, Vsn, HdrStr, Str, Addr, Port) -> - format_tab("inform ~s:~w - ~s [~s]~s ~w\n~s", - [ip(Addr), Port, HdrStr, TimeStamp, SeqNo, Vsn, Str]). +%% f_inform(TimeStamp, SeqNo, Vsn, HdrStr, Str, Addr, Port) -> +%% format_tab("inform ~s:~w - ~s [~s]~s ~w\n~s", +%% [ip(Addr), Port, HdrStr, TimeStamp, SeqNo, Vsn, Str]). +ipPort2Str(Ip, Port) -> + snmp_conf:mk_addr_string({Ip, Port}). + %% io_lib:format("~s:~w", [ip(Ip), Port]). %% Convert a timestamp 2-tupple to a printable string %% @@ -909,8 +982,10 @@ get_type(#pdu{type = Type}) -> Type. -ip({A,B,C,D}) -> - io_lib:format("~w.~w.~w.~w", [A,B,C,D]). +%% ip(Domain, Addr) -> +%% snmp_conf:mk_addr_string(Domain, Addr). +%% ip({A,B,C,D}) -> +%% io_lib:format("~w.~w.~w.~w", [A,B,C,D]). diff --git a/lib/snmp/src/misc/snmp_pdus.erl b/lib/snmp/src/misc/snmp_pdus.erl index 15156f7467..90fa4c0dea 100644 --- a/lib/snmp/src/misc/snmp_pdus.erl +++ b/lib/snmp/src/misc/snmp_pdus.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2012. All Rights Reserved. +%% Copyright Ericsson AB 1996-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -174,7 +174,7 @@ dec_pdu_tag(168) -> dec_pdu([164 | Bytes]) -> % It's a trap Bytes2 = get_data_bytes(Bytes), {Enterprise, Rest1} = dec_oid_tag(Bytes2), - {{'IpAddress', AgentAddr}, Rest2} = dec_value(Rest1), + {{'IpAddress', [_, _, _, _] = AgentAddr}, Rest2} = dec_value(Rest1), {GenericTrap, Rest3} = dec_int_tag(Rest2), {SpecificTrap, Rest4} = dec_int_tag(Rest3), {{'TimeTicks', TimeStamp}, VBBytes} = dec_value(Rest4), @@ -664,7 +664,9 @@ enc_value('BITS', Val) -> enc_oct_str_tag(bits_to_str(Val)); enc_value('OBJECT IDENTIFIER', Val) -> enc_oid_tag(Val); -enc_value('IpAddress', Val) -> +enc_value('IpAddress', {A, B, C, D}) -> + enc_value('IpAddress', [A,B,C,D]); +enc_value('IpAddress', Val) when is_list(Val) -> Bytes2 = enc_oct_str_notag(Val), Len2 = elength(length(Bytes2)), lists:append([64 | Len2],Bytes2); diff --git a/lib/snmp/test/Makefile b/lib/snmp/test/Makefile index f22b7ea8ee..a9bbe7fe62 100644 --- a/lib/snmp/test/Makefile +++ b/lib/snmp/test/Makefile @@ -2,7 +2,7 @@ # %CopyrightBegin% # -# Copyright Ericsson AB 1997-2012. All Rights Reserved. +# Copyright Ericsson AB 1997-2014. All Rights Reserved. # # The contents of this file are subject to the Erlang Public License, # Version 1.1, (the "License"); you may not use this file except in @@ -36,7 +36,7 @@ VSN = $(SNMP_VSN) include modules.mk SNMP_ROOT = .. -SNMP_SUITE = snmp_SUITE +SNMP_SUITE = snmp_SUITE ERL_FILES = $(MODULES:%=%.erl) @@ -93,10 +93,10 @@ ifeq ($(SNMP_DEBUG),e) SNMP_FLAGS += -Dsnmp_error endif ifeq ($(SNMP_DEBUG),l) - SNMP_FLAGS += -Dsnmp_log + SNMP_FLAGS += -Dsnmp_error -Dsnmp_log endif ifeq ($(SNMP_DEBUG),d) - SNMP_FLAGS += -Dsnmp_debug + SNMP_FLAGS += -Dsnmp_error -Dsnmp_log -Dsnmp_debug endif ifeq ($(DONT_USE_TS),true) @@ -239,6 +239,7 @@ release_tests_spec: opt $(INSTALL_DATA) $(RELTEST_FILES) $(COVER_SPEC_FILE) "$(RELSYSDIR)" chmod -R u+w "$(RELSYSDIR)" tar cf - snmp_test_data | (cd "$(RELSYSDIR)"; tar xf -) + tar cf - *_SUITE_data | (cd "$(RELSYSDIR)"; tar xf -) release_docs_spec: diff --git a/lib/snmp/test/klas3.erl b/lib/snmp/test/klas3.erl index ec78d19dbb..4cbd852b2d 100644 --- a/lib/snmp/test/klas3.erl +++ b/lib/snmp/test/klas3.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2010. All Rights Reserved. +%% Copyright Ericsson AB 1997-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -67,11 +67,15 @@ fname(get) -> end, case snmpa:current_address() of {value, {[_A,_B,_C,_D], E}} when is_integer(E) -> ok; - {value, _} -> throw("bad_ip"); - _ -> throw("bad_adr") + {value, {D, _}} when is_atom(D) -> ok; + {value, Ip} -> + throw(format_string("bad_ip: ~p", [Ip])); + Other -> + throw(format_string("bad_adr: ~p", [Other])) end, case snmpa:current_net_if_data() of {value, []} -> ok; + {value, [{request_ref, R}]} when is_reference(R) -> ok; {value, _} -> throw("bad_nil"); _ -> throw("bad_nid") end, @@ -160,3 +164,6 @@ ftab2(get_next, [9], _Cols) -> % bad return value io:format("** Here comes Error Report get_next 3 bad return~n"), [{[1,5],1},{[2,5],3},{[2,6],3}]. + +format_string(Format, Args) -> + lists:flatten(io_lib:format(Format, Args)). diff --git a/lib/snmp/test/modules.mk b/lib/snmp/test/modules.mk index 3d658bf8e8..1bf08a9729 100644 --- a/lib/snmp/test/modules.mk +++ b/lib/snmp/test/modules.mk @@ -2,7 +2,7 @@ # %CopyrightBegin% # -# Copyright Ericsson AB 2004-2012. All Rights Reserved. +# Copyright Ericsson AB 2004-2014. All Rights Reserved. # # The contents of this file are subject to the Erlang Public License, # Version 1.1, (the "License"); you may not use this file except in @@ -19,6 +19,7 @@ SUITE_MODULES = \ snmp_SUITE \ + snmp_to_snmpnet_SUITE \ snmp_app_test \ snmp_appup_test \ snmp_compiler_test \ @@ -42,6 +43,7 @@ TEST_UTIL_MODULES = \ snmp_test_manager \ snmp_test_mgr \ snmp_test_mgr_misc \ + snmp_test_mgr_counter_server \ sa \ klas3 \ test1 \ diff --git a/lib/snmp/test/snmp_SUITE.erl b/lib/snmp/test/snmp_SUITE.erl index 22b9c64588..6fabf6410f 100644 --- a/lib/snmp/test/snmp_SUITE.erl +++ b/lib/snmp/test/snmp_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2012. All Rights Reserved. +%% Copyright Ericsson AB 1997-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -80,7 +80,8 @@ groups() -> {group, note_store_test}]}, {agent, [], [{group, mibs_test}, {group, nfilter_test}, - {group, agent_test}]}, + {group, agent_test}, + {group, snmpnet_test}]}, {manager, [], [{group, manager_config_test}, {group, manager_user_test}, {group, manager_test}]}, @@ -95,6 +96,7 @@ groups() -> {mibs_test, [], [{snmp_agent_mibs_test, all}]}, {nfilter_test, [], [{snmp_agent_nfilter_test, all}]}, {agent_test, [], [{snmp_agent_test, all}]}, + {snmpnet_test, [], [{snmp_to_snmpnet_SUITE, all}]}, {manager_config_test, [], [{snmp_manager_config_test, all}]}, {manager_user_test, [], [{snmp_manager_user_test, all}]}, {manager_test, [], [{snmp_manager_test, all}]} @@ -107,15 +109,18 @@ init_per_group(GroupName, Config0) -> "~n GroupName: ~p" "~n Config0: ~p", [GroupName, Config0]), - %% Group name is not really the suite name - %% (but it is a good enough approximation), - %% but it does not matter since we only need - %% it to be unique. - snmp_test_lib:init_suite_top_dir(GroupName, Config0). - - + case GroupName of + snmpnet_test -> + Config0; + _ -> + %% Group name is not really the suite name + %% (but it is a good enough approximation), + %% but it does not matter since we only need + %% it to be unique. + snmp_test_lib:init_suite_top_dir(GroupName, Config0) + end. + +end_per_group(snmpnet_test, Config) -> + Config; end_per_group(_GroupName, Config) -> lists:keydelete(snmp_suite_top_dir, 1, Config). - - - diff --git a/lib/snmp/test/snmp_agent_test.erl b/lib/snmp/test/snmp_agent_test.erl index 89a6ce1253..b4770ad0a9 100644 --- a/lib/snmp/test/snmp_agent_test.erl +++ b/lib/snmp/test/snmp_agent_test.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2003-2013. All Rights Reserved. +%% Copyright Ericsson AB 2003-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -19,10 +19,6 @@ -module(snmp_agent_test). -%% TODO -%% * Test fault-tolerance (kill master etc) -%% - -export([ all/0, groups/0, @@ -41,7 +37,7 @@ v1_processing/1, big/1, big2/1, - loop_mib/1, + loop_mib_1/1, api/1, subagent/1, mnesia/1, @@ -302,7 +298,6 @@ %% tickets2 otp8395/1, otp9884/1 - ]). %% Internal exports @@ -394,8 +389,9 @@ usm_read/0, usm_del_user/0, usm_bad/0, - loop_mib_1/0, - loop_mib_2/0, + loop_mib_1_test/0, + loop_mib_2_test/0, + loop_mib_3_test/0, otp_1129_i/1, otp_1162_test/0, otp_1131_test/0, @@ -521,9 +517,13 @@ groups() -> {mib_storage_varm_mnesia, [], varm_mib_storage_mnesia_cases()}, {misc, [], misc_cases()}, {test_v1, [], v1_cases()}, + {test_v1_ipv6, [], v1_cases_ipv6()}, {test_v2, [], v2_cases()}, + {test_v2_ipv6, [], v2_cases_ipv6()}, {test_v1_v2, [], v1_v2_cases()}, + {test_v1_v2_ipv6, [], v1_v2_cases()}, {test_v3, [], v3_cases()}, + {test_v3_ipv6, [], v3_cases_ipv6()}, {test_multi_threaded, [], mt_cases()}, {multiple_reqs, [], mul_cases()}, {multiple_reqs_2, [], mul_cases_2()}, @@ -532,6 +532,7 @@ groups() -> {v3_inform, [], v3_inform_cases()}, {v3_security, [], v3_security_cases()}, {standard_mibs, [], standard_mibs_cases()}, + {standard_mibs_ipv6, [], standard_mibs_cases_ipv6()}, {standard_mibs_2, [], standard_mibs2_cases()}, {standard_mibs_3, [], standard_mibs3_cases()}, {reported_bugs, [], reported_bugs_cases()}, @@ -546,8 +547,9 @@ groups() -> init_per_suite(Config0) when is_list(Config0) -> - ?DBG("init_per_suite -> entry with" - "~n Config0: ~p", [Config0]), + p("init_per_suite -> entry with" + "~n Config: ~p" + "~n Nodes: ~p", [Config0, erlang:nodes()]), Config1 = snmp_test_lib:init_suite_top_dir(?MODULE, Config0), Config2 = snmp_test_lib:fix_data_dir(Config1), @@ -558,16 +560,32 @@ init_per_suite(Config0) when is_list(Config0) -> Config3 = [{mib_dir, MibDir}, {std_mib_dir, StdMibDir} | Config2], - ?DBG("init_per_suite -> end with" - "~n Config3: ~p", [Config3]), + snmp_test_mgr_counter_server:start(), + + p("init_per_suite -> end when" + "~n Config: ~p" + "~n Nodes: ~p", [Config3, erlang:nodes()]), Config3. end_per_suite(Config) when is_list(Config) -> - ?DBG("end_per_suite -> entry with" - "~n Config: ~p", [Config]), + p("end_per_suite -> entry with" + "~n Config: ~p" + "~n Nodes: ~p", [Config, erlang:nodes()]), + case snmp_test_mgr_counter_server:stop() of + {ok, _Counters} -> + p("end_per_suite -> sucessfully stopped counter server" + "~n Counters: ~p", [_Counters]); + + {error, Reason} -> + p("end_per_suite -> failed stopping counter server" + "~n Reason: ~p", [Reason]) + end, + + p("end_per_suite -> end when" + "~n Nodes: ~p", [erlang:nodes()]), Config. @@ -597,6 +615,14 @@ init_per_group(test_v2 = GroupName, Config) -> init_v2(snmp_test_lib:init_group_top_dir(GroupName, Config)); init_per_group(test_v1 = GroupName, Config) -> init_v1(snmp_test_lib:init_group_top_dir(GroupName, Config)); +init_per_group(test_v1_ipv6 = GroupName, Config) -> + init_per_group_ipv6(GroupName, Config, fun init_v1/1); +init_per_group(test_v2_ipv6 = GroupName, Config) -> + init_per_group_ipv6(GroupName, Config, fun init_v2/1); +init_per_group(test_v1_v2_ipv6 = GroupName, Config) -> + init_per_group_ipv6(GroupName, Config, fun init_v1_v2/1); +init_per_group(test_v3_ipv6 = GroupName, Config) -> + init_per_group_ipv6(GroupName, Config, fun init_v3/1); init_per_group(misc = GroupName, Config) -> init_misc(snmp_test_lib:init_group_top_dir(GroupName, Config)); init_per_group(mib_storage_varm_mnesia = GroupName, Config) -> @@ -623,6 +649,25 @@ init_per_group(mib_storage_ets = GroupName, Config) -> init_per_group(GroupName, Config) -> snmp_test_lib:init_group_top_dir(GroupName, Config). +init_per_group_ipv6(GroupName, Config, Init) -> + case ct:require(ipv6_hosts) of + ok -> + case gen_udp:open(0, [inet6]) of + {ok, S} -> + ok = gen_udp:close(S), + Init( + snmp_test_lib:init_group_top_dir( + GroupName, + [{ipfamily, inet6}, + {ip, ?LOCALHOST(inet6)} + | lists:keydelete(ip, 1, Config)])); + {error, _} -> + {skip, "Host seems to not support IPv6"} + end; + _ -> + {skip, "Host does not support IPV6"} + end. + end_per_group(all_tcs, Config) -> finish_all(Config); end_per_group(otp7157, Config) -> @@ -641,31 +686,39 @@ end_per_group(multiple_reqs_3, Config) -> finish_mul(Config); end_per_group(test_multi_threaded, Config) -> finish_mt(Config); -end_per_group(test_v3, Config) -> +end_per_group(test_v3_ipv6, Config) -> finish_v3(Config); -end_per_group(test_v1_v2, Config) -> +end_per_group(test_v1_v2_ipv6, Config) -> finish_v1_v2(Config); -end_per_group(test_v2, Config) -> +end_per_group(test_v2_ipv6, Config) -> finish_v2(Config); -end_per_group(test_v1, Config) -> +end_per_group(test_v1_ipv6, Config) -> finish_v1(Config); -end_per_group(misc, Config) -> +end_per_group(test_v3, Config) -> + finish_v3(Config); +end_per_group(test_v1_v2, Config) -> + finish_v1_v2(Config); +end_per_group(test_v2, Config) -> + finish_v2(Config); +end_per_group(test_v1, Config) -> + finish_v1(Config); +end_per_group(misc, Config) -> finish_misc(Config); -end_per_group(mib_storage_varm_mnesia, Config) -> +end_per_group(mib_storage_varm_mnesia, Config) -> finish_varm_mib_storage_mnesia(Config); -end_per_group(mib_storage_varm_dets, Config) -> +end_per_group(mib_storage_varm_dets, Config) -> finish_varm_mib_storage_dets(Config); -end_per_group(mib_storage_size_check_mnesia, Config) -> +end_per_group(mib_storage_size_check_mnesia, Config) -> finish_size_check_msm(Config); -end_per_group(mib_storage_size_check_dets, Config) -> +end_per_group(mib_storage_size_check_dets, Config) -> finish_size_check_msd(Config); -end_per_group(mib_storage_size_check_ets, Config) -> +end_per_group(mib_storage_size_check_ets, Config) -> finish_size_check_mse(Config); -end_per_group(mib_storage_mnesia, Config) -> +end_per_group(mib_storage_mnesia, Config) -> finish_mib_storage_mnesia(Config); -end_per_group(mib_storage_dets, Config) -> +end_per_group(mib_storage_dets, Config) -> finish_mib_storage_dets(Config); -end_per_group(mib_storage_ets, Config) -> +end_per_group(mib_storage_ets, Config) -> finish_mib_storage_ets(Config); end_per_group(_GroupName, Config) -> Config. @@ -675,10 +728,16 @@ end_per_group(_GroupName, Config) -> %% ---- Init Per TestCase ---- init_per_testcase(Case, Config) when is_list(Config) -> - ?DBG("init_per_testcase -> entry with" - "~n Config: ~p", [Config]), + p("init_per_testcase -> entry with" + "~n Config: ~p" + "~n Nodes: ~p", [Config, erlang:nodes()]), + + Result = init_per_testcase1(Case, Config), - init_per_testcase1(Case, Config). + p("init_per_testcase -> done when" + "~n Result: ~p" + "~n Nodes: ~p", [Result, erlang:nodes()]), + Result. init_per_testcase1(otp8395 = Case, Config) when is_list(Config) -> ?DBG("init_per_testcase1 -> entry with" @@ -719,12 +778,18 @@ init_per_testcase1(_Case, Config) when is_list(Config) -> %% ---- End Per TestCase ---- end_per_testcase(Case, Config) when is_list(Config) -> - ?DBG("end_per_testcase -> entry with" - "~n Config: ~p", [Config]), + p("end_per_testcase -> entry with" + "~n Config: ~p" + "~n Nodes: ~p", [Config, erlang:nodes()]), display_log(Config), - end_per_testcase1(Case, Config). + Result = end_per_testcase1(Case, Config), + + p("end_per_testcase -> done with" + "~n Result: ~p" + "~n Nodes: ~p", [Result, erlang:nodes()]), + Result. end_per_testcase1(otp8395, Config) when is_list(Config) -> otp8395({fin, Config}); @@ -784,6 +849,10 @@ cases() -> {group, test_v2}, {group, test_v1_v2}, {group, test_v3}, + {group, test_v1_ipv6}, + {group, test_v2_ipv6}, + {group, test_v1_v2_ipv6}, + {group, test_v3_ipv6}, {group, test_multi_threaded}, {group, mib_storage}, {group, tickets1} @@ -839,8 +908,8 @@ start_v2_agent(Config) -> start_v2_agent(Config, Opts) -> snmp_agent_test_lib:start_v2_agent(Config, Opts). -start_v3_agent(Config) -> - snmp_agent_test_lib:start_v3_agent(Config). +%% start_v3_agent(Config) -> +%% snmp_agent_test_lib:start_v3_agent(Config). start_v3_agent(Config, Opts) -> snmp_agent_test_lib:start_v3_agent(Config, Opts). @@ -1173,7 +1242,7 @@ mse_simple(X) -> ?P(mse_simple), simple(X). mse_v1_processing(X) -> ?P(mse_v1_processing), v1_processing(X). mse_big(X) -> ?P(mse_big), big(X). mse_big2(X) -> ?P(mse_big2), big2(X). -mse_loop_mib(X) -> ?P(mse_loop_mib), loop_mib(X). +mse_loop_mib(X) -> ?P(mse_loop_mib), loop_mib_1(X). mse_api(X) -> ?P(mse_api), api(X). mse_sa_register(X) -> ?P(mse_sa_register), sa_register(X). mse_v1_trap(X) -> ?P(mse_v1_trap), v1_trap(X). @@ -1194,7 +1263,7 @@ msd_simple(X) -> ?P(msd_simple), simple(X). msd_v1_processing(X) -> ?P(msd_v1_processing), v1_processing(X). msd_big(X) -> ?P(msd_big), big(X). msd_big2(X) -> ?P(msd_big2), big2(X). -msd_loop_mib(X) -> ?P(msd_loop_mib), loop_mib(X). +msd_loop_mib(X) -> ?P(msd_loop_mib), loop_mib_1(X). msd_api(X) -> ?P(msd_api), api(X). msd_sa_register(X) -> ?P(msd_sa_register), sa_register(X). msd_v1_trap(X) -> ?P(msd_v1_trap), v1_trap(X). @@ -1215,7 +1284,7 @@ msm_simple(X) -> ?P(msm_simple), simple(X). msm_v1_processing(X) -> ?P(msm_v1_processing), v1_processing(X). msm_big(X) -> ?P(msm_big2), big(X). msm_big2(X) -> ?P(msm_loop_mib), big2(X). -msm_loop_mib(X) -> ?P(msm_loop_mib), loop_mib(X). +msm_loop_mib(X) -> ?P(msm_loop_mib), loop_mib_1(X). msm_api(X) -> ?P(msm_api), api(X). msm_sa_register(X) -> ?P(msm_sa_register), sa_register(X). msm_v1_trap(X) -> ?P(msm_v1_trap), v1_trap(X). @@ -1610,7 +1679,7 @@ del_dir(Dir, Depth) -> ok end. -%v1_cases() -> [loop_mib]; +%v1_cases() -> [loop_mib_1]; v1_cases() -> [ simple, @@ -1618,7 +1687,7 @@ v1_cases() -> v1_processing, big, big2, - loop_mib, + loop_mib_1, api, subagent, mnesia, @@ -1636,14 +1705,40 @@ v1_cases() -> change_target_addr_config ]. +v1_cases_ipv6() -> + [ + simple, + v1_processing, + loop_mib_1, +%% big, +%% big2, + api, + subagent, +%% mnesia, +%% {group, multiple_reqs}, + sa_register, +%% v1_trap, % sends v1 trap +%% sa_error, + next_across_sa, + undo, +%% {group, reported_bugs}, + {group, standard_mibs_ipv6}, + sparse_table, +%% cnt_64, % sends v1 trap + opaque +%% change_target_addr_config % sends v1 trap + ]. + init_v1(Config) when is_list(Config) -> ?line SaNode = ?config(snmp_sa, Config), ?line create_tables(SaNode), ?line AgentConfDir = ?config(agent_conf_dir, Config), ?line MgrDir = ?config(mgr_dir, Config), ?line Ip = ?config(ip, Config), - ?line config([v1], MgrDir, AgentConfDir, - tuple_to_list(Ip), tuple_to_list(Ip)), + ?line IpFamily = config_ipfamily(Config), + ?line config( + [v1], MgrDir, AgentConfDir, + tuple_to_list(Ip), tuple_to_list(Ip), IpFamily), [{vsn, v1} | start_v1_agent(Config)]. finish_v1(Config) when is_list(Config) -> @@ -1680,14 +1775,43 @@ v2_cases() -> v2_caps ]. +v2_cases_ipv6() -> + [ + simple_2, + v2_processing, +%% big_2, +%% big2_2, + loop_mib_2, + api_2, + subagent_2, +%% mnesia_2, +%% {group, multiple_reqs_2}, + sa_register_2, + v2_trap, + {group, v2_inform}, +%% sa_error_2, + next_across_sa_2, + undo_2, +%% {group, reported_bugs_2}, + {group, standard_mibs_2}, + v2_types, + implied, + sparse_table_2, + cnt_64_2, + opaque_2, + v2_caps + ]. + init_v2(Config) when is_list(Config) -> SaNode = ?config(snmp_sa, Config), create_tables(SaNode), AgentConfDir = ?config(agent_conf_dir, Config), MgrDir = ?config(mgr_dir, Config), Ip = ?config(ip, Config), - config([v2], MgrDir, AgentConfDir, - tuple_to_list(Ip), tuple_to_list(Ip)), + IpFamily = config_ipfamily(Config), + config( + [v2], MgrDir, AgentConfDir, + tuple_to_list(Ip), tuple_to_list(Ip), IpFamily), [{vsn, v2} | start_v2_agent(Config)]. finish_v2(Config) when is_list(Config) -> @@ -1706,8 +1830,9 @@ init_v1_v2(Config) when is_list(Config) -> AgentConfDir = ?config(agent_conf_dir, Config), MgrDir = ?config(mgr_dir, Config), Ip = ?config(ip, Config), + IpFamily = config_ipfamily(Config), config([v1,v2], MgrDir, AgentConfDir, - tuple_to_list(Ip), tuple_to_list(Ip)), + tuple_to_list(Ip), tuple_to_list(Ip), IpFamily), [{vsn, bilingual} | start_bilingual_agent(Config)]. finish_v1_v2(Config) when is_list(Config) -> @@ -1745,6 +1870,34 @@ v3_cases() -> v2_caps_3 ]. +v3_cases_ipv6() -> + [ + simple_3, + v3_processing, +%% big_3, +%% big2_3, + api_3, + subagent_3, +%% mnesia_3, + loop_mib_3, +%% {group, multiple_reqs_3}, + sa_register_3, + v3_trap, + {group, v3_inform}, +%% sa_error_3, + next_across_sa_3, + undo_3, +%% {group, reported_bugs_3}, + {group, standard_mibs_3}, + {group, v3_security}, + v2_types_3, + implied_3, + sparse_table_3, + cnt_64_3, + opaque_3, + v2_caps_3 + ]. + init_v3(Config) when is_list(Config) -> %% Make sure crypto works, otherwise start_agent will fail %% and we will be stuck with a bunch of mnesia tables for @@ -1766,9 +1919,16 @@ init_v3(Config) when is_list(Config) -> AgentConfDir = ?config(agent_conf_dir, Config), MgrDir = ?config(mgr_dir, Config), Ip = ?config(ip, Config), - ?line ok = config([v3], MgrDir, AgentConfDir, - tuple_to_list(Ip), tuple_to_list(Ip)), - [{vsn, v3} | start_v3_agent(Config)]. + IpFamily = config_ipfamily(Config), + ?line ok = + config( + [v3], MgrDir, AgentConfDir, + tuple_to_list(Ip), tuple_to_list(Ip), IpFamily), + Opts = + [{master_agent_verbosity, trace}, + {agent_verbosity, trace}, + {net_if_verbosity, trace}], + [{vsn, v3} | start_v3_agent(Config, Opts)]. finish_v3(Config) when is_list(Config) -> delete_tables(), @@ -2095,9 +2255,9 @@ await_dummy_manager_started(Pid) -> {ok,Pid,Port}; {'EXIT', Pid, Reason} -> {error, Pid, Reason}; - O -> + _O -> ?LOG("dummy_manager_start -> received unknown message:" - "~n ~p",[O]), + "~n ~p",[_O]), await_dummy_manager_started(Pid) end. @@ -2120,16 +2280,16 @@ dummy_manager_send_trap2(Pid) -> dummy_manager_await_trap2_ack() -> ?DBG("dummy_manager_await_trap2 -> entry",[]), receive - {received_trap,Trap} -> - ?LOG("dummy_manager_await_trap2 -> received trap: ~p",[Trap]), + {received_trap, _Trap} -> + ?LOG("dummy_manager_await_trap2 -> received trap: ~p", [_Trap]), %% Note: %% Without this sleep the v2_inform_i testcase failes! There %% is no relation between these two test cases as far as I %% able to figure out... ?SLEEP(60000), ok; - O -> - ?ERR("dummy_manager_await_trap2 -> unexpected message: ~p",[O]), + _O -> + ?ERR("dummy_manager_await_trap2 -> unexpected message: ~p",[_O]), ok after 10000 -> ?ERR("dummy_manager_await_trap2 -> timeout",[]), @@ -2155,32 +2315,34 @@ dummy_manager_loop(P,S,MA) -> "~n Trap: ~p",[Trap]), snmpa:send_trap(MA, Trap, "standard trap"), dummy_manager_loop(P,S,MA); - {udp, _UdpId, Ip, UdpPort, Bytes} -> + {udp, _UdpId, _Ip, _UdpPort, Bytes} -> ?LOG("dummy_manager_loop -> received upd message" "~n from: ~p:~p" "~n size: ~p", - [Ip, UdpPort, dummy_manager_message_sz(Bytes)]), + [_Ip, _UdpPort, dummy_manager_message_sz(Bytes)]), R = dummy_manager_handle_message(Bytes), - ?DBG("dummy_manager_loop -> R: ~p",[R]), + ?DBG("dummy_manager_loop -> R: ~p", [R]), P ! R, - dummy_manager_loop(P,S,MA); + dummy_manager_loop(P, S, MA); stop -> ?DBG("dummy_manager_loop -> received stop request",[]), P ! {dummy_manager_stopping, self()}, gen_udp:close(S), exit(normal); - O -> + _O -> ?LOG("dummy_manager_loop -> received unknown message:" - "~n ~p",[O]), - dummy_manager_loop(P,S,MA) + "~n ~p", [_O]), + dummy_manager_loop(P, S, MA) end. +-ifdef(snmp_log). dummy_manager_message_sz(B) when is_binary(B) -> size(B); dummy_manager_message_sz(L) when is_list(L) -> length(L); dummy_manager_message_sz(_) -> undefined. +-endif. dummy_manager_handle_message(Bytes) -> case (catch snmp_pdus:dec_message(Bytes)) of @@ -3398,11 +3560,11 @@ simple_standard_test() -> db_notify_client(suite) -> []; db_notify_client(Config) when is_list(Config) -> ?P(db_notify_client), - {SaNode, MgrNode, MibDir} = init_case(Config), + {_SaNode, _MgrNode, _MibDir} = init_case(Config), ?DBG("db_notify_client -> case initiated: " "~n SaNode: ~p" "~n MgrNode: ~p" - "~n MibDir: ~p", [SaNode, MgrNode, MibDir]), + "~n MibDir: ~p", [_SaNode, _MgrNode, _MibDir]), ?DBG("db_notify_client -> maximize verbosity", []), snmpa_local_db:verbosity(trace), Self = self(), @@ -4153,8 +4315,8 @@ ma_v2_inform1(MA) -> CmdExp = fun(ok) -> ok; - ({ok, Val}) -> - ?DBG("ma_v2_inform -> [cmd2] Val: ~p", [Val]), + ({ok, _Val}) -> + ?DBG("ma_v2_inform -> [cmd2] Val: ~p", [_Val]), ok; ({error, Id, Extra}) -> {error, {unexpected, Id, Extra}}; @@ -4189,10 +4351,10 @@ ma_v2_inform1(MA) -> CmdSnmpTargets = fun(T) -> receive - {snmp_targets, T, [Addr]} -> + {snmp_targets, T, [_Addr]} -> ?DBG("ma_v2_inform1 -> " "received expected snmp_targets " - "~n with receiver: ~p",[Addr]), + "~n with receiver: ~p", [_Addr]), ok; {snmp_targets, T, Addrs} -> ?ERR("ma_v2_inform1 -> " @@ -4210,16 +4372,16 @@ ma_v2_inform1(MA) -> Cmd06 = fun() -> receive - {snmp_notification, Tag03, {got_response, Addr}} -> + {snmp_notification, Tag03, {got_response, _Addr}} -> ?DBG("ma_v2_inform1 -> " "received expected snmp_notification " - "[with manager response] from: ~n ~p",[Addr]), + "[with manager response] from: ~n ~p", [_Addr]), ok; - {snmp_notification, Tag03, {no_response, Addr}} -> + {snmp_notification, Tag03, {no_response, _Addr}} -> ?ERR("ma_v2_inform1 -> " "received unexpected snmp_notification " "[without manager response] from: ~n ~p", - [Addr]), + [_Addr]), {error, no_response} after 20000 -> @@ -4249,16 +4411,16 @@ ma_v2_inform1(MA) -> Cmd10 = fun() -> receive - {snmp_notification, Tag07, {got_response, Addr}} -> + {snmp_notification, Tag07, {got_response, _Addr}} -> ?ERR("ma_v2_inform1 -> " "received unexpected snmp_notification " - "[with manager response] from: ~n ~p", [Addr]), + "[with manager response] from: ~n ~p", [_Addr]), {error, got_response}; - {snmp_notification, Tag07, {no_response, Addr}} -> + {snmp_notification, Tag07, {no_response, _Addr}} -> ?DBG("ma_v2_inform1 -> " "received expected snmp_notification " "[without manager response] from: ~n ~p", - [Addr]), + [_Addr]), ok after 240000 -> @@ -4302,8 +4464,8 @@ ma_v2_inform2(MA) -> CmdExp = fun(ok) -> ok; - ({ok, Val}) -> - ?DBG("ma_v2_inform -> [cmd2] Val: ~p", [Val]), + ({ok, _Val}) -> + ?DBG("ma_v2_inform -> [cmd2] Val: ~p", [_Val]), ok; ({error, Id, Extra}) -> {error, {unexpected, Id, Extra}}; @@ -4383,8 +4545,8 @@ ma_v2_inform3(MA) -> "~n send notification: testTrapv22", [MA]), CmdExpectInform = - fun(No, Response) -> - ?DBG("CmdExpectInform -> ~p: ~n~p", [No, Response]), + fun(_No, Response) -> + ?DBG("CmdExpectInform -> ~p: ~n~p", [_No, Response]), ?expect2({inform, Response}, [{[sysUpTime, 0], any}, {[snmpTrapOID, 0], ?system ++ [0,1]}]) @@ -4393,8 +4555,8 @@ ma_v2_inform3(MA) -> CmdExp = fun(ok) -> ok; - ({ok, Val}) -> - ?DBG("CmdExp -> Val: ~p", [Val]), + ({ok, _Val}) -> + ?DBG("CmdExp -> Val: ~p", [_Val]), ok; ({error, Id, Extra}) -> {error, {unexpected, Id, Extra}}; @@ -4505,17 +4667,17 @@ delivery_info(Tag, Address, DeliveryResult, Extra) -> command_handler([]) -> ok; -command_handler([{No, Desc, Cmd}|Rest]) -> - ?LOG("command_handler -> command ~w: ~n ~s", [No, Desc]), +command_handler([{_No, _Desc, Cmd}|Rest]) -> + ?LOG("command_handler -> command ~w: ~n ~s", [_No, _Desc]), case (catch Cmd()) of ok -> - ?LOG("command_handler -> ~w: ok",[No]), + ?LOG("command_handler -> ~w: ok", [_No]), command_handler(Rest); {error, Reason} -> - ?ERR("command_handler -> ~w error: ~n~p",[No, Reason]), + ?ERR("command_handler -> ~w error: ~n~p", [_No, Reason]), ?line ?FAIL(Reason); Error -> - ?ERR("command_handler -> ~w unexpected: ~n~p",[No, Error]), + ?ERR("command_handler -> ~w unexpected: ~n~p", [_No, Error]), ?line ?FAIL({unexpected_command_result, Error}) end. @@ -4682,6 +4844,15 @@ standard_mibs_cases() -> snmp_view_based_acm_mib ]. +standard_mibs_cases_ipv6() -> + [ + %% snmp_standard_mib, % Sending v1 traps does not work over IPv6 + snmp_community_mib, + snmp_framework_mib, + snmp_target_mib, + snmp_notification_mib, + snmp_view_based_acm_mib + ]. %%----------------------------------------------------------------- %% For this test, the agent is configured for v1. @@ -5516,57 +5687,59 @@ usm_bad() -> %% works. %% Load all std mibs that are not loaded by default. %%----------------------------------------------------------------- -loop_mib(suite) -> []; -loop_mib(Config) when is_list(Config) -> - ?P(loop_mib), - ?LOG("loop_mib -> initiate case",[]), +loop_mib_1(suite) -> []; +loop_mib_1(Config) when is_list(Config) -> + ?P(loop_mib_1), + ?LOG("loop_mib_1 -> initiate case",[]), %% snmpa:verbosity(master_agent,debug), %% snmpa:verbosity(mib_server,info), - {SaNode, MgrNode, MibDir} = init_case(Config), - ?DBG("loop_mib -> ~n" + {_SaNode, _MgrNode, _MibDir} = init_case(Config), + ?DBG("loop_mib_1 -> ~n" "\tSaNode: ~p~n" "\tMgrNode: ~p~n" - "\tMibDir: ~p",[SaNode, MgrNode, MibDir]), - ?DBG("loop_mib -> load mib SNMP-COMMUNITY-MIB",[]), + "\tMibDir: ~p",[_SaNode, _MgrNode, _MibDir]), + ?DBG("loop_mib_1 -> load mib SNMP-COMMUNITY-MIB",[]), ?line load_master_std("SNMP-COMMUNITY-MIB"), - ?DBG("loop_mib -> load mib SNMP-MPD-MIB",[]), + ?DBG("loop_mib_1 -> load mib SNMP-MPD-MIB",[]), ?line load_master_std("SNMP-MPD-MIB"), - ?DBG("loop_mib -> load mib SNMP-TARGET-MIB",[]), + ?DBG("loop_mib_1 -> load mib SNMP-TARGET-MIB",[]), ?line load_master_std("SNMP-TARGET-MIB"), - ?DBG("loop_mib -> load mib SNMP-NOTIFICATION-MIB",[]), + ?DBG("loop_mib_1 -> load mib SNMP-NOTIFICATION-MIB",[]), ?line load_master_std("SNMP-NOTIFICATION-MIB"), - ?DBG("loop_mib -> load mib SNMP-FRAMEWORK-MIB",[]), + ?DBG("loop_mib_1 -> load mib SNMP-FRAMEWORK-MIB",[]), ?line load_master_std("SNMP-FRAMEWORK-MIB"), - ?DBG("loop_mib -> load mib SNMP-VIEW-BASED-ACM-MIB",[]), + ?DBG("loop_mib_1 -> load mib SNMP-VIEW-BASED-ACM-MIB",[]), ?line load_master_std("SNMP-VIEW-BASED-ACM-MIB"), - ?DBG("loop_mib -> try",[]), - try_test(loop_mib_1), - ?DBG("loop_mib -> unload mib SNMP-COMMUNITY-MIB",[]), + ?DBG("loop_mib_1 -> try",[]), + + try_test(loop_mib_1_test), + + ?DBG("loop_mib_1 -> unload mib SNMP-COMMUNITY-MIB",[]), ?line unload_master("SNMP-COMMUNITY-MIB"), - ?DBG("loop_mib -> unload mib SNMP-MPD-MIB",[]), + ?DBG("loop_mib_1 -> unload mib SNMP-MPD-MIB",[]), ?line unload_master("SNMP-MPD-MIB"), - ?DBG("loop_mib -> unload mib SNMP-TARGET-MIB",[]), + ?DBG("loop_mib_1 -> unload mib SNMP-TARGET-MIB",[]), ?line unload_master("SNMP-TARGET-MIB"), - ?DBG("loop_mib -> unload mib SNMP-NOTIFICATION-MIB",[]), + ?DBG("loop_mib_1 -> unload mib SNMP-NOTIFICATION-MIB",[]), ?line unload_master("SNMP-NOTIFICATION-MIB"), - ?DBG("loop_mib -> unload mib SNMP-FRAMEWORK-MIB",[]), + ?DBG("loop_mib_1 -> unload mib SNMP-FRAMEWORK-MIB",[]), ?line unload_master("SNMP-FRAMEWORK-MIB"), - ?DBG("loop_mib -> unload mib SNMP-VIEW-BASED-ACM-MIB",[]), + ?DBG("loop_mib_1 -> unload mib SNMP-VIEW-BASED-ACM-MIB",[]), ?line unload_master("SNMP-VIEW-BASED-ACM-MIB"), %% snmpa:verbosity(master_agent,log), %% snmpa:verbosity(mib_server,silence), - ?LOG("loop_mib -> done",[]). + ?LOG("loop_mib_1 -> done",[]). loop_mib_2(suite) -> []; loop_mib_2(Config) when is_list(Config) -> ?P(loop_mib_2), ?LOG("loop_mib_2 -> initiate case",[]), - {SaNode, MgrNode, MibDir} = init_case(Config), - ?DBG("loop_mib_2 -> ~n" + {_SaNode, _MgrNode, _MibDir} = init_case(Config), + ?DBG("do_loop_mib_2 -> ~n" "\tSaNode: ~p~n" "\tMgrNode: ~p~n" - "\tMibDir: ~p",[SaNode, MgrNode, MibDir]), + "\tMibDir: ~p", [_SaNode, _MgrNode, _MibDir]), ?DBG("loop_mib_2 -> load mibs",[]), ?line load_master_std("SNMP-COMMUNITY-MIB"), ?line load_master_std("SNMP-MPD-MIB"), @@ -5574,7 +5747,9 @@ loop_mib_2(Config) when is_list(Config) -> ?line load_master_std("SNMP-NOTIFICATION-MIB"), ?line load_master_std("SNMP-FRAMEWORK-MIB"), ?line load_master_std("SNMP-VIEW-BASED-ACM-MIB"), - try_test(loop_mib_2), + + try_test(loop_mib_2_test), + ?DBG("loop_mib_2 -> unload mibs",[]), ?line unload_master("SNMP-COMMUNITY-MIB"), ?line unload_master("SNMP-MPD-MIB"), @@ -5589,18 +5764,18 @@ loop_mib_3(suite) -> []; loop_mib_3(Config) when is_list(Config) -> ?P(loop_mib_3), ?LOG("loop_mib_3 -> initiate case",[]), - {SaNode, MgrNode, MibDir} = init_case(Config), + {_SaNode, _MgrNode, _MibDir} = init_case(Config), ?DBG("loop_mib_3 -> ~n" "\tSaNode: ~p~n" "\tMgrNode: ~p~n" - "\tMibDir: ~p",[SaNode, MgrNode, MibDir]), + "\tMibDir: ~p", [_SaNode, _MgrNode, _MibDir]), ?DBG("loop_mib_3 -> load mibs",[]), ?line load_master_std("SNMP-TARGET-MIB"), ?line load_master_std("SNMP-NOTIFICATION-MIB"), ?line load_master_std("SNMP-VIEW-BASED-ACM-MIB"), ?line load_master_std("SNMP-USER-BASED-SM-MIB"), - try_test(loop_mib_2), + try_test(loop_mib_3_test), ?DBG("loop_mib_3 -> unload mibs",[]), ?line unload_master("SNMP-TARGET-MIB"), @@ -5611,17 +5786,16 @@ loop_mib_3(Config) when is_list(Config) -> %% Req. As many mibs all possible -loop_mib_1() -> - ?DBG("loop_mib_1 -> entry",[]), +loop_mib_1_test() -> + ?DBG("loop_mib_1_test -> entry",[]), N = loop_it_1([1,1], 0), io:format(user, "found ~w varibles\n", [N]), ?line N = if N < 100 -> 100; true -> N end. - loop_it_1(Oid, N) -> - ?DBG("loop_it_1 -> entry with~n" + ?DBG("loop_it_1_test -> entry with~n" "\tOid: ~p~n" "\tN: ~p",[Oid,N]), case get_next_req([Oid]) of @@ -5629,13 +5803,13 @@ loop_it_1(Oid, N) -> error_status = noError, error_index = 0, varbinds = [#varbind{oid = NOid, - value = Value}]} when NOid > Oid -> - ?DBG("loop_it_1 -> " + value = _Value}]} when NOid > Oid -> + ?DBG("loop_it_1_test -> " "~n NOid: ~p" - "~n Value: ~p",[NOid, Value]), - ?line [Value2] = get_req(1, [NOid]), % must not be same - ?DBG("loop_it_1 -> " - "~n Value2: ~p",[Value2]), + "~n Value: ~p", [NOid, _Value]), + ?line [_Value2] = get_req(1, [NOid]), % must not be same + ?DBG("loop_it_1_test -> " + "~n Value2: ~p", [_Value2]), loop_it_1(NOid, N+1); #pdu{type = 'get-response', @@ -5648,7 +5822,7 @@ loop_it_1(Oid, N) -> error_status = noSuchName, error_index = 1, varbinds = [_]} -> - ?DBG("loop_it_1 -> done: ~p",[N]), + ?DBG("loop_it_1_test -> done: ~p",[N]), N; #pdu{type = 'get-response', @@ -5669,14 +5843,13 @@ loop_it_1(Oid, N) -> %% Req. As many mibs all possible -loop_mib_2() -> - ?DBG("loop_mib_1 -> entry",[]), +loop_mib_2_test() -> + ?DBG("loop_mib_2_test -> entry",[]), N = loop_it_2([1,1], 0), io:format(user, "found ~w varibles\n", [N]), ?line N = if N < 100 -> 100; true -> N end. - loop_it_2(Oid, N) -> ?DBG("loop_it_2 -> entry with" @@ -5686,22 +5859,22 @@ loop_it_2(Oid, N) -> #pdu{type = 'get-response', error_status = noError, error_index = 0, - varbinds = [#varbind{oid = NOid, value = endOfMibView}]} -> + varbinds = [#varbind{oid = _NOid, value = endOfMibView}]} -> ?DBG("loop_it_2 -> " - "~n NOid: ~p",[NOid]), + "~n NOid: ~p", [_NOid]), N; #pdu{type = 'get-response', error_status = noError, error_index = 0, varbinds = [#varbind{oid = NOid, - value = Value}]} when NOid > Oid -> + value = _Value}]} when NOid > Oid -> ?DBG("loop_it_2 -> " "~n NOid: ~p" - "~n Value: ~p",[NOid, Value]), - ?line [Value2] = get_req(1, [NOid]), % must not be same + "~n Value: ~p", [NOid, _Value]), + ?line [_Value2] = get_req(1, [NOid]), % must not be same ?DBG("loop_it_2 -> " - "~n Value2: ~p",[Value2]), + "~n Value2: ~p", [_Value2]), loop_it_2(NOid, N+1); #pdu{type = 'get-response', @@ -5744,6 +5917,10 @@ loop_it_2(Oid, N) -> end. +loop_mib_3_test() -> + ?DBG("loop_mib_3_test -> entry",[]), + loop_mib_2_test(). + %%%----------------------------------------------------------------- %%% Testing of reported bugs and other tickets. @@ -6344,8 +6521,6 @@ otp_4394_config(AgentConfDir, MgrDir, Ip0) -> ?line write_notify_conf(AgentConfDir), ok. - - otp_4394_finish(Config) when is_list(Config) -> ?DBG("finish_otp_4394 -> entry", []), C1 = stop_agent(Config), @@ -6498,6 +6673,7 @@ otp8395({init, Config}) when is_list(Config) -> %% SubAgentHost = ?HPSTNAME(SubAgentNode), ManagerHost = ?HOSTNAME(ManagerNode), + IpFamily = inet, Host = snmp_test_lib:hostname(), Ip = ?LOCALHOST(), {ok, AgentIP0} = snmp_misc:ip(AgentHost), @@ -6515,9 +6691,7 @@ otp8395({init, Config}) when is_list(Config) -> Vsns = [v1], AgentConfDir = ?config(agent_conf_dir, Config), ManagerConfDir = ?config(manager_top_dir, Config), - snmp_agent_test_lib:config(Vsns, - ManagerConfDir, AgentConfDir, - ManagerIP, AgentIP), + config(Vsns, ManagerConfDir, AgentConfDir, ManagerIP, AgentIP, IpFamily), %% -- @@ -6526,6 +6700,7 @@ otp8395({init, Config}) when is_list(Config) -> Config2 = start_agent([{host, Host}, {ip, Ip}, + {ipfamily, IpFamily}, {agent_node, AgentNode}, {agent_host, AgentHost}, {agent_ip, AgentIP}, @@ -6605,22 +6780,23 @@ otp8395(Config) when is_list(Config) -> put(mib_dir, ?config(mib_dir, Config)), put(vsn, v1), put(master_host, ?config(agent_host, Config)), + put(ipfamily, ?config(ipfamily, Config)), try_test(simple_standard_test), ?SLEEP(1000), AgentNode = ?config(agent_node, Config), AgentLogDir = ?config(agent_log_dir, Config), OutFile = join([AgentLogDir, "otp8395.txt"]), - {ok, LogInfo} = rpc:call(AgentNode, snmpa, log_info, []), - ?DBG("otp8395 -> LogInfo: ~p", [LogInfo]), + {ok, _LogInfo} = rpc:call(AgentNode, snmpa, log_info, []), + ?DBG("otp8395 -> LogInfo: ~p", [_LogInfo]), %% SyncRes = rpc:call(AgentNode, snmp, log_sync, [?audit_trail_log_name]), %% ?DBG("otp8395 -> SyncRes: ~p", [SyncRes]), ok = agent_log_validation(AgentNode), - LTTRes = + _LTTRes = rpc:call(AgentNode, snmpa, log_to_txt, [AgentLogDir, [], OutFile]), - ?DBG("otp8395 -> LTTRes: ~p", [LTTRes]), + ?DBG("otp8395 -> LTTRes: ~p", [_LTTRes]), ?SLEEP(1000), ?DBG("otp8395 -> done", []), @@ -6632,126 +6808,12 @@ otp8395(Config) when is_list(Config) -> otp9884({init, Config}) when is_list(Config) -> ?DBG("otp9884(init) -> entry with" "~n Config: ~p", [Config]), - - %% -- - %% Start nodes - %% - - {ok, AgentNode} = start_node(agent), - - %% We don't use a manager in this test but the (common) config - %% function takes an argument that is derived from this - {ok, ManagerNode} = start_node(manager), - - %% -- - %% Mnesia init - %% - - AgentDbDir = ?config(agent_db_dir, Config), - AgentMnesiaDir = join([AgentDbDir, "mnesia"]), - mnesia_init(AgentNode, AgentMnesiaDir), - - mnesia_create_schema(AgentNode, [AgentNode]), - - mnesia_start(AgentNode), - - %% -- - %% Host & IP - %% - - AgentHost = ?HOSTNAME(AgentNode), - ManagerHost = ?HOSTNAME(ManagerNode), - - Host = snmp_test_lib:hostname(), - Ip = ?LOCALHOST(), - {ok, AgentIP0} = snmp_misc:ip(AgentHost), - AgentIP = tuple_to_list(AgentIP0), - {ok, ManagerIP0} = snmp_misc:ip(ManagerHost), - ManagerIP = tuple_to_list(ManagerIP0), - - - %% -- - %% Write agent config - %% - - Vsns = [v1], - ManagerConfDir = ?config(manager_top_dir, Config), - AgentConfDir = ?config(agent_conf_dir, Config), - AgentTopDir = ?config(agent_top_dir, Config), - AgentBkpDir1 = join([AgentTopDir, backup1]), - AgentBkpDir2 = join([AgentTopDir, backup2]), - ok = file:make_dir(AgentBkpDir1), - ok = file:make_dir(AgentBkpDir2), - AgentBkpDirs = [AgentBkpDir1, AgentBkpDir2], - snmp_agent_test_lib:config(Vsns, - ManagerConfDir, AgentConfDir, - ManagerIP, AgentIP), - - - %% -- - %% Start the agent - %% - - Config2 = start_agent([{host, Host}, - {ip, Ip}, - {agent_node, AgentNode}, - {agent_host, AgentHost}, - {agent_ip, AgentIP}, - {agent_backup_dirs, AgentBkpDirs}|Config]), - - %% -- - %% Create watchdog - %% - - Dog = ?WD_START(?MINS(1)), - - [{watchdog, Dog} | Config2]; + init_v1_agent([{ipfamily, inet} | Config]); otp9884({fin, Config}) when is_list(Config) -> ?DBG("otp9884(fin) -> entry with" "~n Config: ~p", [Config]), - - AgentNode = ?config(agent_node, Config), - ManagerNode = ?config(manager_node, Config), - - %% - - %% Stop agent (this is the nice way to do it, - %% so logs and files can be closed in the proper way). - %% - - AgentSup = ?config(agent_sup, Config), - ?DBG("otp9884(fin) -> stop (stand-alone) agent: ~p", [AgentSup]), - stop_stdalone_agent(AgentSup), - - %% - - %% Stop mnesia - %% - ?DBG("otp9884(fin) -> stop mnesia", []), - mnesia_stop(AgentNode), - - - %% - - %% Stop the agent node - %% - - ?DBG("otp9884(fin) -> stop agent node", []), - stop_node(AgentNode), - - - %% SubAgentNode = ?config(sub_agent_node, Config), - %% stop_node(SubAgentNode), - - - %% - - %% Stop the manager node - %% - - ?DBG("otp9884(fin) -> stop manager node", []), - stop_node(ManagerNode), - - Dog = ?config(watchdog, Config), - ?WD_STOP(Dog), - lists:keydelete(watchdog, 1, Config); + fin_v1_agent(Config); otp9884(doc) -> "OTP-9884 - Simlutaneous backup call should not work. "; @@ -6811,8 +6873,6 @@ otp9884_await_backup_completion(First, Second) end; otp9884_await_backup_completion(First, Second) -> throw({error, {bad_completion, First, Second}}). - - %%----------------------------------------------------------------- agent_log_validation(Node) -> @@ -6941,10 +7001,10 @@ stop_stdalone_agent(Pid) when (node(Pid) =/= node()) -> MRef = erlang:monitor(process, Pid), rpc:call(node(Pid), ?MODULE, stop_stdalone_agent, [Pid]), receive - {'DOWN', MRef, process, Pid, Info} -> + {'DOWN', MRef, process, Pid, _Info} -> ?DBG("received expected DOWN message " "regarding snmp agent supervisor: " - "~n Info: ~p", [Info]), + "~n Info: ~p", [_Info]), ok after 5000 -> ?DBG("no DOWN message " @@ -7003,9 +7063,9 @@ do_info(MaNode) -> tree_size_bytes, db_memory]}], verify_info(Info, Keys), - OldInfo = snmpa:old_info_format(Info), - ?DBG("info_test1 -> OldInfo: ~n~p", [OldInfo]), - verify_old_info(OldInfo), + %% OldInfo = snmpa:old_info_format(Info), + %% ?DBG("info_test1 -> OldInfo: ~n~p", [OldInfo]), + %% verify_old_info(OldInfo), ok. verify_info([], []) -> @@ -7048,20 +7108,20 @@ verify_subinfo(Info0, [Key|Keys]) -> verify_subinfo(Info, Keys) end. -verify_old_info(Info) -> - Keys = [vsns, subagents, loaded_mibs, - tree_size_bytes, process_memory, db_memory], - verify_old_info(Keys, Info). - -verify_old_info([], _) -> - ok; -verify_old_info([Key|Keys], Info) -> - case lists:keymember(Key, 1, Info) of - true -> - verify_old_info(Keys, Info); - false -> - ?FAIL({missing_old_info, Key}) - end. +%% verify_old_info(Info) -> +%% Keys = [vsns, subagents, loaded_mibs, +%% tree_size_bytes, process_memory, db_memory], +%% verify_old_info(Keys, Info). + +%% verify_old_info([], _) -> +%% ok; +%% verify_old_info([Key|Keys], Info) -> +%% case lists:keymember(Key, 1, Info) of +%% true -> +%% verify_old_info(Keys, Info); +%% false -> +%% ?FAIL({missing_old_info, Key}) +%% end. %% Index String - string used in index is(S) -> [length(S) | S]. @@ -7109,6 +7169,9 @@ delete_files(Config) -> config(Vsns, MgrDir, AgentDir, MIp, AIp) -> snmp_agent_test_lib:config(Vsns, MgrDir, AgentDir, MIp, AIp). +config(Vsns, MgrDir, AgentDir, MIp, AIp, IpFamily) -> + snmp_agent_test_lib:config(Vsns, MgrDir, AgentDir, MIp, AIp, IpFamily). + update_usm(Vsns, Dir) -> snmp_agent_test_lib:update_usm(Vsns, Dir). @@ -7353,3 +7416,124 @@ p(F, A) -> formated_timestamp() -> snmp_test_lib:formated_timestamp(). + +init_v1_agent(Config) -> + %% -- + %% Start nodes + %% + + {ok, AgentNode} = start_node(agent), + + %% We don't use a manager in this test but the (common) config + %% function takes an argument that is derived from this + {ok, ManagerNode} = start_node(manager), + + %% -- + %% Mnesia init + %% + + AgentDbDir = ?config(agent_db_dir, Config), + AgentMnesiaDir = join([AgentDbDir, "mnesia"]), + mnesia_init(AgentNode, AgentMnesiaDir), + + mnesia_create_schema(AgentNode, [AgentNode]), + + mnesia_start(AgentNode), + + %% -- + %% Host & IP + %% + + AgentHost = ?HOSTNAME(AgentNode), + ManagerHost = ?HOSTNAME(ManagerNode), + + Host = snmp_test_lib:hostname(), + IpFamily = config_ipfamily(Config), + Ip = ?LOCALHOST(IpFamily), + {ok, AgentIP0} = snmp_misc:ip(AgentHost, IpFamily), + AgentIP = tuple_to_list(AgentIP0), + {ok, ManagerIP0} = snmp_misc:ip(ManagerHost, IpFamily), + ManagerIP = tuple_to_list(ManagerIP0), + + + %% -- + %% Write agent config + %% + + Vsns = [v1], + ManagerConfDir = ?config(manager_top_dir, Config), + AgentConfDir = ?config(agent_conf_dir, Config), + AgentTopDir = ?config(agent_top_dir, Config), + AgentBkpDir1 = join([AgentTopDir, backup1]), + AgentBkpDir2 = join([AgentTopDir, backup2]), + ok = file:make_dir(AgentBkpDir1), + ok = file:make_dir(AgentBkpDir2), + AgentBkpDirs = [AgentBkpDir1, AgentBkpDir2], + config(Vsns, ManagerConfDir, AgentConfDir, ManagerIP, AgentIP, IpFamily), + + + %% -- + %% Start the agent + %% + + Config2 = start_agent([{host, Host}, + {ip, Ip}, + {agent_node, AgentNode}, + {agent_host, AgentHost}, + {agent_ip, AgentIP}, + {agent_backup_dirs, AgentBkpDirs}|Config]), + + %% -- + %% Create watchdog + %% + + Dog = ?WD_START(?MINS(1)), + + [{watchdog, Dog} | Config2]. + +fin_v1_agent(Config) -> + AgentNode = ?config(agent_node, Config), + ManagerNode = ?config(manager_node, Config), + + %% - + %% Stop agent (this is the nice way to do it, + %% so logs and files can be closed in the proper way). + %% + + AgentSup = ?config(agent_sup, Config), + stop_stdalone_agent(AgentSup), + + %% - + %% Stop mnesia + %% + mnesia_stop(AgentNode), + + + %% - + %% Stop the agent node + %% + stop_node(AgentNode), + + + %% SubAgentNode = ?config(sub_agent_node, Config), + %% stop_node(SubAgentNode), + + + %% - + %% Stop the manager node + %% + stop_node(ManagerNode), + + Dog = ?config(watchdog, Config), + ?WD_STOP(Dog), + lists:keydelete(watchdog, 1, Config). + + + +config_ipfamily(Config) -> + case ?config(ipfamily, Config) of + undefined -> + inet; + Value -> + Value + end. diff --git a/lib/snmp/test/snmp_agent_test_lib.erl b/lib/snmp/test/snmp_agent_test_lib.erl index 122289c28e..333fe6eb66 100644 --- a/lib/snmp/test/snmp_agent_test_lib.erl +++ b/lib/snmp/test/snmp_agent_test_lib.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2013. All Rights Reserved. +%% Copyright Ericsson AB 2005-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -39,7 +39,7 @@ get_req/2, get_next_req/1, - config/5, + config/5, config/6, delete_files/1, copy_file/2, update_usm/2, @@ -232,13 +232,14 @@ init_case(Config) when is_list(Config) -> MgrNode = ?config(snmp_mgr, Config), MasterNode = ?config(snmp_master, Config), %% MasterNode = node(), - + IpFamily = proplists:get_value(ipfamily, Config, inet), + SaHost = ?HOSTNAME(SaNode), MgrHost = ?HOSTNAME(MgrNode), MasterHost = ?HOSTNAME(MasterNode), - {ok, MasterIP} = snmp_misc:ip(MasterHost), - {ok, MIP} = snmp_misc:ip(MgrHost), - {ok, SIP} = snmp_misc:ip(SaHost), + {ok, MasterIP} = snmp_misc:ip(MasterHost, IpFamily), + {ok, MIP} = snmp_misc:ip(MgrHost, IpFamily), + {ok, SIP} = snmp_misc:ip(SaHost, IpFamily), put(mgr_node, MgrNode), @@ -250,6 +251,7 @@ init_case(Config) when is_list(Config) -> put(mip, tuple_to_list(MIP)), put(masterip, tuple_to_list(MasterIP)), put(sip, tuple_to_list(SIP)), + put(ipfamily, IpFamily), MibDir = ?config(mib_dir, Config), put(mib_dir, MibDir), @@ -299,10 +301,10 @@ call(N,M,F,A) -> "~n Loc: ~p", [Rn, Loc]), put(test_server_loc, Loc), exit(Rn); - {done, Ret, Zed} -> + {done, Ret, _Zed} -> ?DBG("call -> done:" "~n Ret: ~p" - "~n Zed: ~p", [Ret, Zed]), + "~n Zed: ~p", [Ret, _Zed]), case Ret of {error, Reason} -> exit(Reason); @@ -338,8 +340,8 @@ run(Mod, Func, Args, Opts) -> CtxEngineID = snmp_misc:get_option(context_engine_id, Opts, EngineID), Community = snmp_misc:get_option(community, Opts, "all-rights"), ?DBG("run -> start crypto app",[]), - Crypto = ?CRYPTO_START(), - ?DBG("run -> Crypto: ~p", [Crypto]), + _CryptoRes = ?CRYPTO_START(), + ?DBG("run -> Crypto: ~p", [_CryptoRes]), catch snmp_test_mgr:stop(), % If we had a running mgr from a failed case StdM = join(code:priv_dir(snmp), "mibs") ++ "/", Vsn = get(vsn), @@ -358,6 +360,7 @@ run(Mod, Func, Args, Opts) -> {packet_server_debug,true}, {debug,true}, {agent, get(master_host)}, + {ipfamily, get(ipfamily)}, {agent_udp, 4000}, {trap_udp, 5000}, {recbuf,65535}, @@ -676,9 +679,9 @@ stop_agent(Config) when is_list(Config) -> (catch process_info(Sup)), (catch process_info(Par))]), - Info = agent_info(Sup), + _Info = agent_info(Sup), ?DBG("stop_agent -> Agent info: " - "~n ~p", [Info]), + "~n ~p", [_Info]), stop_sup(Sup, Par), @@ -1303,10 +1306,10 @@ get_req(Id, Vars) -> {ok, Val} -> ?DBG("get_req -> response: ~p",[Val]), Val; - {error, _, {ExpFmt, ExpArg}, {ActFmt, ActArg}} -> + {error, _, {_ExpFmt, ExpArg}, {_ActFmt, ActArg}} -> ?DBG("get_req -> error for ~p: " - "~n " ++ ExpFmt ++ - "~n " ++ ActFmt, + "~n " ++ _ExpFmt ++ + "~n " ++ _ActFmt, [Id] ++ ExpArg ++ ActArg), exit({unexpected_response, ExpArg, ActArg}); Error -> @@ -1368,16 +1371,35 @@ stop_node(Node) -> %%%----------------------------------------------------------------- config(Vsns, MgrDir, AgentConfDir, MIp, AIp) -> + config(Vsns, MgrDir, AgentConfDir, MIp, AIp, inet). + +config(Vsns, MgrDir, AgentConfDir, MIp, AIp, IpFamily) -> ?LOG("config -> entry with" - "~n Vsns: ~p" - "~n MgrDir: ~p" - "~n AgentConfDir: ~p" - "~n MIp: ~p" - "~n AIp: ~p", - [Vsns, MgrDir, AgentConfDir, MIp, AIp]), - ?line snmp_config:write_agent_snmp_files(AgentConfDir, Vsns, - MIp, ?TRAP_UDP, AIp, 4000, - "test"), + "~n Vsns: ~p" + "~n MgrDir: ~p" + "~n AgentConfDir: ~p" + "~n MIp: ~p" + "~n AIp: ~p" + "~n IpFamily: ~p", + [Vsns, MgrDir, AgentConfDir, MIp, AIp, IpFamily]), + ?line {Domain, ManagerAddr} = + case IpFamily of + inet6 -> + Ipv6Domain = transportDomainUdpIpv6, + AgentIpv6Addr = {AIp, 4000}, + ManagerIpv6Addr = {MIp, ?TRAP_UDP}, + ?line ok = + snmp_config:write_agent_snmp_files( + AgentConfDir, Vsns, + Ipv6Domain, ManagerIpv6Addr, AgentIpv6Addr, "test"), + {Ipv6Domain, ManagerIpv6Addr}; + _ -> + ?line ok = + snmp_config:write_agent_snmp_files( + AgentConfDir, Vsns, MIp, ?TRAP_UDP, AIp, 4000, "test"), + {snmpUDPDomain, {MIp, ?TRAP_UDP}} + end, + ?line case update_usm(Vsns, AgentConfDir) of true -> ?line copy_file(join(AgentConfDir, "usm.conf"), @@ -1388,7 +1410,7 @@ config(Vsns, MgrDir, AgentConfDir, MIp, AIp) -> end, ?line update_community(Vsns, AgentConfDir), ?line update_vacm(Vsns, AgentConfDir), - ?line write_target_addr_conf(AgentConfDir, MIp, ?TRAP_UDP, Vsns), + ?line write_target_addr_conf(AgentConfDir, Domain, ManagerAddr, Vsns), ?line write_target_params_conf(AgentConfDir, Vsns), ?line write_notify_conf(AgentConfDir), ok. @@ -1486,7 +1508,7 @@ rewrite_usm_mgr(Dir, ShaKey, DesKey) -> {"mgrEngine", "newUser", "newUser", zeroDotZero, usmHMACSHAAuthProtocol, "", "", usmDESPrivProtocol, "", "", "", ShaKey, DesKey}], - ok = snmp_config:write_agent_usm_config(Dir, "", Conf). + ?line ok = snmp_config:write_agent_usm_config(Dir, "", Conf). reset_usm_mgr(Dir) -> ?line ok = file:rename(join(Dir,"usm.old"), @@ -1512,13 +1534,15 @@ update_vacm(_Vsn, Dir) -> write_community_conf(Dir, Conf) -> - snmp_config:write_agent_community_config(Dir, "", Conf). + ?line ok = snmp_config:write_agent_community_config(Dir, "", Conf). write_target_addr_conf(Dir, Conf) -> - snmp_config:write_agent_target_addr_config(Dir, "", Conf). + ?line ok = snmp_config:write_agent_target_addr_config(Dir, "", Conf). -write_target_addr_conf(Dir, ManagerIp, UDP, Vsns) -> - snmp_config:write_agent_snmp_target_addr_conf(Dir, ManagerIp, UDP, Vsns). +write_target_addr_conf(Dir, Ip_or_Domain, Port_or_Addr, Vsns) -> + ?line ok = + snmp_config:write_agent_snmp_target_addr_conf( + Dir, Ip_or_Domain, Port_or_Addr, Vsns). rewrite_target_addr_conf(Dir, NewPort) -> ?DBG("rewrite_target_addr_conf -> entry with" @@ -1527,15 +1551,14 @@ rewrite_target_addr_conf(Dir, NewPort) -> case file:read_file_info(TAFile) of {ok, _} -> ok; - {error, R} -> + {error, _R} -> ?ERR("failure reading file info of " - "target address config file: ~p",[R]), + "target address config file: ~p", [_R]), ok end, ?line [TrapAddr|Addrs] = - snmp_conf:read(TAFile, - fun(R) -> rewrite_target_addr_conf_check(R) end), + snmp_conf:read(TAFile, fun rewrite_target_addr_conf_check/1), ?DBG("rewrite_target_addr_conf -> TrapAddr: ~p",[TrapAddr]), @@ -1571,14 +1594,14 @@ write_target_params_conf(Dir, Vsns) -> (v3) -> {"target_v3", v3, usm, "all-rights", noAuthNoPriv} end, Conf = [F(Vsn) || Vsn <- Vsns], - snmp_config:write_agent_target_params_config(Dir, "", Conf). + ?line ok = snmp_config:write_agent_target_params_config(Dir, "", Conf). rewrite_target_params_conf(Dir, SecName, SecLevel) when is_list(SecName) andalso is_atom(SecLevel) -> ?line ok = file:rename(join(Dir,"target_params.conf"), join(Dir,"target_params.old")), Conf = [{"target_v3", v3, usm, SecName, SecLevel}], - snmp_config:write_agent_target_params_config(Dir, "", Conf). + ?line ok = snmp_config:write_agent_target_params_config(Dir, "", Conf). reset_target_params_conf(Dir) -> ?line ok = file:rename(join(Dir,"target_params.old"), @@ -1587,12 +1610,12 @@ reset_target_params_conf(Dir) -> write_notify_conf(Dir) -> Conf = [{"standard trap", "std_trap", trap}, {"standard inform", "std_inform", inform}], - snmp_config:write_agent_notify_config(Dir, "", Conf). + ?line ok = snmp_config:write_agent_notify_config(Dir, "", Conf). write_view_conf(Dir) -> Conf = [{2, [1,3,6], included, null}, {2, ?tDescr_instance, excluded, null}], - snmp_config:write_agent_view_config(Dir, "", Conf). + ?line ok = snmp_config:write_agent_view_config(Dir, "", Conf). %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% diff --git a/lib/snmp/test/snmp_conf_test.erl b/lib/snmp/test/snmp_conf_test.erl index c4341d8d7e..7f5d11c0e7 100644 --- a/lib/snmp/test/snmp_conf_test.erl +++ b/lib/snmp/test/snmp_conf_test.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2003-2010. All Rights Reserved. +%% Copyright Ericsson AB 2003-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -373,6 +373,8 @@ verify_ip(Val) -> case (catch snmp_conf:check_ip(Val)) of {error, Reason} -> ?FAIL({verify_ip, Val, Reason}); + {ok, _} -> + ok; ok -> ok end. @@ -401,7 +403,7 @@ check_taddress(Config) when is_list(Config) -> ok. verify_taddress(Val) -> - case (catch snmp_conf:check_taddress(Val)) of + case (catch snmp_conf:check_taddress(snmpUDPDomain, Val)) of {error, Reason} -> ?FAIL({verify_taddress, Val, Reason}); ok -> @@ -409,7 +411,7 @@ verify_taddress(Val) -> end. verify_not_taddress(Val) -> - case (catch snmp_conf:check_taddress(Val)) of + case (catch snmp_conf:check_taddress(snmpUDPDomain, Val)) of ok -> ?FAIL({verify_taddress, Val}); {error, _Reason} -> diff --git a/lib/snmp/test/snmp_manager_config_test.erl b/lib/snmp/test/snmp_manager_config_test.erl index 7b9924b83c..f37e957dae 100644 --- a/lib/snmp/test/snmp_manager_config_test.erl +++ b/lib/snmp/test/snmp_manager_config_test.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2013. All Rights Reserved. +%% Copyright Ericsson AB 2004-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -693,7 +693,7 @@ start_with_invalid_manager_conf_file1(Conf) when is_list(Conf) -> "arne_anka", "4001", "500", "\"bmkEngine\""), ?line {error, Reason12} = config_start(Opts), p("start failed (as expected): ~p", [Reason12]), - ?line {failed_check, _, _, 2, {invalid_ip_address, _}} = Reason12, + ?line {failed_check, _, _, 2, {bad_address, _}} = Reason12, await_config_not_running(), %% -- @@ -702,7 +702,7 @@ start_with_invalid_manager_conf_file1(Conf) when is_list(Conf) -> "9999", "4001", "500", "\"bmkEngine\""), ?line {error, Reason13} = config_start(Opts), p("start failed (as expected): ~p", [Reason13]), - ?line {failed_check, _, _, 2, {invalid_ip_address, _}} = Reason13, + ?line {failed_check, _, _, 2, {bad_address, _}} = Reason13, await_config_not_running(), %% -- @@ -720,7 +720,8 @@ start_with_invalid_manager_conf_file1(Conf) when is_list(Conf) -> "[134,138,177,189]", "-1", "500", "\"bmkEngine\""), ?line {error, Reason22} = config_start(Opts), p("start failed (as expected): ~p", [Reason22]), - ?line {failed_check, _, _, 3, {invalid_integer, _}} = Reason22, + io:format("Reason22: ~p~n", [Reason22]), + ?line {failed_check, _, _, 3, {bad_port, _}} = Reason22, await_config_not_running(), %% -- @@ -729,7 +730,7 @@ start_with_invalid_manager_conf_file1(Conf) when is_list(Conf) -> "[134,138,177,189]", "\"kalle-anka\"", "500", "\"bmkEngine\""), ?line {error, Reason23} = config_start(Opts), p("start failed (as expected): ~p", [Reason23]), - ?line {failed_check, _, _, 3, {invalid_integer, _}} = Reason23, + ?line {failed_check, _, _, 3, {bad_port, _}} = Reason23, await_config_not_running(), %% -- @@ -1047,7 +1048,7 @@ start_with_invalid_agents_conf_file1(Conf) when is_list(Conf) -> case config_start(Opts) of {error, Reason51} -> p("start failed (as expected): ~p", [Reason51]), - ?line {failed_check, _, _, _, {bad_address, _}} = Reason51, + ?line {failed_check, _, _, _, {bad_domain, _}} = Reason51, await_config_not_running(); OK_51 -> exit({error, {unexpected_success, "51", OK_51}}) @@ -1073,7 +1074,7 @@ start_with_invalid_agents_conf_file1(Conf) when is_list(Conf) -> case config_start(Opts) of {error, Reason53} -> p("start failed (as expected): ~p", [Reason53]), - ?line {failed_check, _, _, _, {invalid_ip_address, _}} = Reason53, + ?line {failed_check, _, _, _, {bad_address, _}} = Reason53, await_config_not_running(); OK_53 -> exit({error, {unexpected_success, "53", OK_53}}) @@ -1086,7 +1087,7 @@ start_with_invalid_agents_conf_file1(Conf) when is_list(Conf) -> case config_start(Opts) of {error, Reason54} -> p("start failed (as expected): ~p", [Reason54]), - ?line {failed_check, _, _, _, {invalid_ip_address, _}} = Reason54, + ?line {failed_check, _, _, _, {bad_address, _}} = Reason54, await_config_not_running(); OK_54 -> exit({error, {unexpected_success, "54", OK_54}}) @@ -1098,7 +1099,7 @@ start_with_invalid_agents_conf_file1(Conf) when is_list(Conf) -> write_agents_conf(ConfDir, [Agent55]), ?line {error, Reason55} = config_start(Opts), p("start failed (as expected): ~p", [Reason55]), - ?line {failed_check, _, _, _, {invalid_ip_address, _}} = Reason55, + ?line {failed_check, _, _, _, {bad_address, _}} = Reason55, await_config_not_running(), %% -- @@ -1107,7 +1108,7 @@ start_with_invalid_agents_conf_file1(Conf) when is_list(Conf) -> write_agents_conf(ConfDir, [Agent61]), ?line {error, Reason61} = config_start(Opts), p("start failed (as expected): ~p", [Reason61]), - ?line {failed_check, _, _, _, {invalid_integer, _}} = Reason61, + ?line {failed_check, _, _, _, {bad_address, _}} = Reason61, await_config_not_running(), %% -- @@ -1116,7 +1117,7 @@ start_with_invalid_agents_conf_file1(Conf) when is_list(Conf) -> write_agents_conf(ConfDir, [Agent62]), ?line {error, Reason62} = config_start(Opts), p("start failed (as expected): ~p", [Reason62]), - ?line {failed_check, _, _, _, {invalid_integer, _}} = Reason62, + ?line {failed_check, _, _, _, {bad_address, _}} = Reason62, await_config_not_running(), %% -- @@ -1125,7 +1126,7 @@ start_with_invalid_agents_conf_file1(Conf) when is_list(Conf) -> write_agents_conf(ConfDir, [Agent63]), ?line {error, Reason63} = config_start(Opts), p("start failed (as expected): ~p", [Reason63]), - ?line {failed_check, _, _, _, {invalid_integer, _}} = Reason63, + ?line {failed_check, _, _, _, {bad_address, _}} = Reason63, await_config_not_running(), %% -- diff --git a/lib/snmp/test/snmp_manager_test.erl b/lib/snmp/test/snmp_manager_test.erl index 5fe18980bc..fa90872172 100644 --- a/lib/snmp/test/snmp_manager_test.erl +++ b/lib/snmp/test/snmp_manager_test.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2003-2013. All Rights Reserved. +%% Copyright Ericsson AB 2003-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -139,6 +139,8 @@ -define(NS_TIMEOUT, 10000). +-define(DEFAULT_MNESIA_DEBUG, none). + %%---------------------------------------------------------------------- %% Records @@ -173,7 +175,9 @@ end_per_suite(Config) when is_list(Config) -> init_per_testcase(Case, Config) when is_list(Config) -> - io:format(user, "~n~n*** INIT ~w:~w ***~n~n", [?MODULE,Case]), + io:format(user, "~n~n*** INIT ~w:~w ***~n~n", [?MODULE, Case]), + p(Case, "init_per_testcase begin when" + "~n Nodes: ~p~n~n", [erlang:nodes()]), %% This version of the API, based on Addr and Port, has been deprecated DeprecatedApiCases = [ @@ -187,16 +191,25 @@ init_per_testcase(Case, Config) when is_list(Config) -> simple_async_get_bulk1, misc_async1 ], - case lists:member(Case, DeprecatedApiCases) of - true -> - %% ?SKIP(api_no_longer_supported); - {skip, api_no_longer_supported}; - false -> - init_per_testcase2(Case, Config) - end. + Result = + case lists:member(Case, DeprecatedApiCases) of + true -> + %% ?SKIP(api_no_longer_supported); + {skip, api_no_longer_supported}; + false -> + init_per_testcase2(Case, Config) + end, + p(Case, "init_per_testcase end when" + "~n Nodes: ~p" + "~n Result: ~p" + "~n~n", [Result, erlang:nodes()]), + Result. init_per_testcase2(Case, Config) -> - ?DBG("init_per_testcase2 -> ~p", [erlang:nodes()]), + ?DBG("init_per_testcase2 -> " + "~n Case: ~p" + "~n Config: ~p" + "~n Nodes: ~p", [Case, Config, erlang:nodes()]), CaseTopDir = snmp_test_lib:init_testcase_top_dir(Case, Config), @@ -228,8 +241,11 @@ init_per_testcase2(Case, Config) -> AgLogDir = filename:join(AgTopDir, "log/"), ?line ok = file:make_dir(AgLogDir), + Family = proplists:get_value(ipfamily, Config, inet), + Conf = [{watchdog, ?WD_START(?MINS(5))}, - {ip, ?LOCALHOST()}, + {ipfamily, Family}, + {ip, ?LOCALHOST(Family)}, {case_top_dir, CaseTopDir}, {agent_dir, AgTopDir}, {agent_conf_dir, AgConfDir}, @@ -314,6 +330,8 @@ init_per_testcase3(Case, Config) -> end. end_per_testcase(Case, Config) when is_list(Config) -> + p(Case, "end_per_testcase begin when" + "~n Nodes: ~p~n~n", [erlang:nodes()]), ?DBG("fin [~w] Nodes [1]: ~p", [Case, erlang:nodes()]), Dog = ?config(watchdog, Config), ?WD_STOP(Dog), @@ -322,6 +340,8 @@ end_per_testcase(Case, Config) when is_list(Config) -> ?DBG("fin [~w] Nodes [2]: ~p", [Case, erlang:nodes()]), %% TopDir = ?config(top_dir, Conf2), %% ?DEL_DIR(TopDir), + p(Case, "end_per_testcase end when" + "~n Nodes: ~p~n~n", [erlang:nodes()]), Conf2. end_per_testcase2(Case, Config) -> @@ -393,7 +413,9 @@ all() -> {group, event_tests}, {group, event_tests_mt}, discovery, - {group, tickets} + {group, tickets}, + {group, ipv6}, + {group, ipv6_mt} ]. groups() -> @@ -428,10 +450,10 @@ groups() -> {request_tests, [], [ {group, get_tests}, - {group, get_next_tests}, + {group, get_next_tests}, {group, set_tests}, - {group, bulk_tests}, - {group, misc_request_tests} + {group, bulk_tests}, + {group, misc_request_tests} ] }, {request_tests_mt, [], @@ -528,9 +550,29 @@ groups() -> [ otp8395_1 ] - } + }, + {ipv6, [], ipv6_tests()}, + {ipv6_mt, [], ipv6_tests()} + ]. +ipv6_tests() -> + [ + register_agent1, + simple_sync_get_next3, + simple_async_get2, + simple_sync_get3, + simple_async_get_next2, + simple_sync_set3, + simple_async_set2, + simple_sync_get_bulk2, + simple_async_get_bulk3, + misc_async2, + inform1, + inform_swarm + ]. + + init_per_group(request_tests_mt = GroupName, Config) -> snmp_test_lib:init_group_top_dir( GroupName, @@ -539,10 +581,39 @@ init_per_group(event_tests_mt = GroupName, Config) -> snmp_test_lib:init_group_top_dir( GroupName, [{manager_net_if_module, snmpm_net_if_mt} | Config]); +init_per_group(ipv6_mt = GroupName, Config) -> + case ct:require(ipv6_hosts) of + ok -> + case gen_udp:open(0, [inet6]) of + {ok, S} -> + ok = gen_udp:close(S), + ipv6_init( + snmp_test_lib:init_group_top_dir( + GroupName, + [{manager_net_if_module, snmpm_net_if_mt} + | Config])); + {error, _} -> + {skip, "Host seems to not support IPv6"} + end; + _ -> + {skip, "Host does not support IPV6"} + end; +init_per_group(ipv6 = GroupName, Config) -> + case ct:require(ipv6_hosts) of + ok -> + case gen_udp:open(0, [inet6]) of + {ok, S} -> + ok = gen_udp:close(S), + ipv6_init(snmp_test_lib:init_group_top_dir(GroupName, Config)); + {error, _} -> + {skip, "Host seems to not support IPv6"} + end; + _ -> + {skip, "Host does not support IPV6"} + end; init_per_group(GroupName, Config) -> snmp_test_lib:init_group_top_dir(GroupName, Config). - - + end_per_group(_GroupName, Config) -> %% Do we really need to do this? lists:keydelete(snmp_group_top_dir, 1, Config). @@ -1491,7 +1562,7 @@ register_agent3(Config) when is_list(Config) -> TargetName2 = "agent3", ?line ok = mgr_register_agent(ManagerNode, user_alfa, TargetName2, [{tdomain, transportDomainUdpIpv6}, - {address, LocalHost}, + {address, {0,0,0,0,0,0,0,1}}, {port, 5002}, {engine_id, "agentEngineId-2"}]), TargetName3 = "agent4", @@ -5057,7 +5128,7 @@ inform_swarm_collector(N) -> inform_swarm_collector(N, SentAckCnt, RecvCnt, RespCnt, _) when ((N == SentAckCnt) and (N == RespCnt) and - (N >= RecvCnt)) -> + (N =< RecvCnt)) -> p("inform_swarm_collector -> done when" "~n N: ~w" "~n SentAckCnt: ~w" @@ -5303,34 +5374,59 @@ init_manager(AutoInform, Config) -> ?line Node = start_manager_node(), + %% The point with this (try catch block) is to be + %% able to do some cleanup in case we fail to + %% start some of the apps. That is, if we fail to + %% start the apps (mnesia, crypto and snmp agent) + %% we stop the (agent) node! - %% -- - %% Start and initiate crypto on manager node - %% + try + begin - ?line ok = init_crypto(Node), - - %% - %% Write manager config - %% + %% -- + %% Start and initiate crypto on manager node + %% + + ?line ok = init_crypto(Node), + + %% + %% Write manager config + %% + + ?line ok = write_manager_config(Config), + + IRB = case AutoInform of + true -> + auto; + _ -> + user + end, + Conf = [{manager_node, Node}, {irb, IRB} | Config], + Vsns = [v1,v2,v3], + start_manager(Node, Vsns, Conf) + end + catch + T:E -> + StackTrace = ?STACK(), + p("Failure during manager start: " + "~n Error Type: ~p" + "~n Error: ~p" + "~n StackTrace: ~p", [T, E, StackTrace]), + %% And now, *try* to cleanup + (catch stop_node(Node)), + ?FAIL({failed_starting_manager, T, E, StackTrace}) + end. - ?line ok = write_manager_config(Config), - - IRB = case AutoInform of - true -> - auto; - _ -> - user - end, - Conf = [{manager_node, Node}, {irb, IRB} | Config], - Vsns = [v1,v2,v3], - start_manager(Node, Vsns, Conf). - fin_manager(Config) -> Node = ?config(manager_node, Config), - stop_manager(Node, Config), - fin_crypto(Node), - stop_node(Node), + StopMgrRes = stop_manager(Node), + StopCryptoRes = fin_crypto(Node), + StopNode = stop_node(Node), + p("fin_agent -> stop apps and (mgr node ~p) node results: " + "~n SNMP Mgr: ~p" + "~n Crypto: ~p" + "~n Node: ~p", + [Node, StopMgrRes, StopCryptoRes, StopNode]), Config. @@ -5352,52 +5448,93 @@ init_agent(Config) -> ?line Node = start_agent_node(), + %% The point with this (try catch block) is to be + %% able to do some cleanup in case we fail to + %% start some of the apps. That is, if we fail to + %% start the apps (mnesia, crypto and snmp agent) + %% we stop the (agent) node! - %% -- - %% Start and initiate mnesia on agent node - %% - - ?line ok = init_mnesia(Node, Dir), - - - %% -- - %% Start and initiate crypto on agent node - %% - - ?line ok = init_crypto(Node), - - - %% - %% Write agent config - %% - - Vsns = [v1,v2], - ?line ok = write_agent_config(Vsns, Config), - - Conf = [{agent_node, Node}, - {mib_dir, MibDir} | Config], + try + begin + + %% -- + %% Start and initiate mnesia on agent node + %% + + ?line ok = init_mnesia(Node, Dir, ?config(mnesia_debug, Config)), + + + %% -- + %% Start and initiate crypto on agent node + %% + + ?line ok = init_crypto(Node), + + + %% + %% Write agent config + %% + + Vsns = [v1,v2], + ?line ok = write_agent_config(Vsns, Config), + + Conf = [{agent_node, Node}, + {mib_dir, MibDir} | Config], - %% - %% Start the agent - %% - - start_agent(Node, Vsns, Conf). + %% + %% Start the agent + %% + + start_agent(Node, Vsns, Conf) + end + catch + T:E -> + StackTrace = ?STACK(), + p("Failure during agent start: " + "~n Error Type: ~p" + "~n Error: ~p" + "~n StackTrace: ~p", [T, E, StackTrace]), + %% And now, *try* to cleanup + (catch stop_node(Node)), + ?FAIL({failed_starting_agent, T, E, StackTrace}) + end. + fin_agent(Config) -> Node = ?config(agent_node, Config), - stop_agent(Node, Config), - fin_crypto(Node), - fin_mnesia(Node), - stop_node(Node), + StopAgentRes = stop_agent(Node), + StopCryptoRes = fin_crypto(Node), + StopMnesiaRes = fin_mnesia(Node), + StopNode = stop_node(Node), + p("fin_agent -> stop apps and (agent node ~p) node results: " + "~n SNMP Agent: ~p" + "~n Crypto: ~p" + "~n Mnesia: ~p" + "~n Node: ~p", + [Node, StopAgentRes, StopCryptoRes, StopMnesiaRes, StopNode]), Config. -init_mnesia(Node, Dir) -> +init_mnesia(Node, Dir, MnesiaDebug) + when ((MnesiaDebug =/= none) andalso + (MnesiaDebug =/= debug) andalso (MnesiaDebug =/= trace)) -> + init_mnesia(Node, Dir, ?DEFAULT_MNESIA_DEBUG); +init_mnesia(Node, Dir, MnesiaDebug) -> ?DBG("init_mnesia -> load application mnesia", []), ?line ok = load_mnesia(Node), ?DBG("init_mnesia -> application mnesia: set_env dir: ~n~p",[Dir]), ?line ok = set_mnesia_env(Node, dir, filename:join(Dir, "mnesia")), + %% Just in case, only set (known to be) valid values for debug + if + ((MnesiaDebug =:= debug) orelse (MnesiaDebug =:= trace)) -> + ?DBG("init_mnesia -> application mnesia: set_env debug: ~w", + [MnesiaDebug]), + ?line ok = set_mnesia_env(Node, debug, MnesiaDebug); + true -> + ok + end, + ?DBG("init_mnesia -> create mnesia schema",[]), ?line case create_schema(Node) of ok -> @@ -5434,25 +5571,89 @@ fin_crypto(Node) -> %% -- Misc application wrapper functions -- -load_app(Node, App) when (Node =:= node()) andalso is_atom(App) -> - application:load(App); -load_app(Node, App) when is_atom(App) -> - rcall(Node, application, load, [App]). - -start_app(Node, App) when (Node =:= node()) andalso is_atom(App) -> - application:start(App); +load_app(Node, App) -> + VerifySuccess = fun(ok) -> + ok; + ({error, {already_loaded, LoadedApp}}) when (LoadedApp =:= App) -> + ok; + ({error, Reason}) -> + p("failed loading app ~w on ~p: " + "~n ~p", [App, Node, Reason]), + ?FAIL({failed_load, Node, App, Reason}) + end, + do_load_app(Node, App, VerifySuccess). + +do_load_app(Node, App, VerifySuccess) + when (Node =:= node()) andalso is_atom(App) -> + %% Local app + exec(fun() -> application:load(App) end, VerifySuccess); +do_load_app(Node, App, VerifySuccess) -> + %% Remote app + exec(fun() -> rcall(Node, application, load, [App]) end, VerifySuccess). + + start_app(Node, App) -> - rcall(Node, application, start, [App]). + VerifySuccess = fun(ok) -> + ok; + ({error, {already_started, LoadedApp}}) when (LoadedApp =:= App) -> + ok; + ({error, Reason}) -> + p("failed starting app ~w on ~p: " + "~n ~p", [App, Node, Reason]), + ?FAIL({failed_start, Node, App, Reason}) + end, + start_app(Node, App, VerifySuccess). + +start_app(Node, App, VerifySuccess) + when (Node =:= node()) andalso is_atom(App) -> + exec(fun() -> application:start(App) end, VerifySuccess); +start_app(Node, App, VerifySuccess) -> + exec(fun() -> rcall(Node, application, start, [App]) end, VerifySuccess). + + +stop_app(Node, App) -> + VerifySuccess = fun(ok) -> + ok; + ({error, {not_started, LoadedApp}}) when (LoadedApp =:= App) -> + ok; + ({error, Reason}) -> + p("failed stopping app ~w on ~p: " + "~n ~p", [App, Node, Reason]), + ?FAIL({failed_stop, Node, App, Reason}) + end, + stop_app(Node, App, VerifySuccess). + +stop_app(Node, App, VerifySuccess) + when (Node =:= node()) andalso is_atom(App) -> + exec(fun() -> application:stop(App) end, VerifySuccess); +stop_app(Node, App, VerifySuccess) when is_atom(App) -> + exec(fun() -> rcall(Node, application, stop, [App]) end, VerifySuccess). + + +set_app_env(Node, App, Key, Val) -> + VerifySuccess = fun(ok) -> + ok; + ({error, Reason}) -> + p("failed setting app ~w env on ~p" + "~n Key: ~p" + "~n Val: ~p" + "~n Reason: ~p" + "~n ~p", [App, Node, Key, Val, Reason]), + ?FAIL({failed_set_app_env, + Node, App, Key, Val, Reason}) + end, + set_app_env(Node, App, Key, Val, VerifySuccess). -stop_app(Node, App) when (Node =:= node()) andalso is_atom(App) -> - application:stop(App); -stop_app(Node, App) when is_atom(App) -> - rcall(Node, application, stop, [App]). +set_app_env(Node, App, Key, Val, VerifySuccess) + when (Node =:= node()) andalso is_atom(App) -> + exec(fun() -> application:set_env(App, Key, Val) end, VerifySuccess); +set_app_env(Node, App, Key, Val, VerifySuccess) when is_atom(App) -> + exec(fun() -> rcall(Node, application, set_env, [App, Key, Val]) end, + VerifySuccess). -set_app_env(Node, App, Key, Val) when (Node =:= node()) andalso is_atom(App) -> - application:set_env(App, Key, Val); -set_app_env(Node, App, Key, Val) when is_atom(App) -> - rcall(Node, application, set_env, [App, Key, Val]). + +exec(Cmd, VerifySuccess) -> + VerifySuccess(Cmd()). %% -- Misc snmp wrapper functions -- @@ -5603,12 +5804,24 @@ fin_mgr_user(Conf) -> init_mgr_user_data1(Conf) -> Node = ?config(manager_node, Conf), TargetName = ?config(manager_agent_target_name, Conf), - Addr = ?config(ip, Conf), + IpFamily = ?config(ipfamily, Conf), + Ip = ?config(ip, Conf), Port = ?AGENT_PORT, - ?line ok = mgr_user_register_agent(Node, TargetName, - [{address, Addr}, - {port, Port}, - {engine_id, "agentEngine"}]), + ?line ok = + case IpFamily of + inet -> + mgr_user_register_agent( + Node, TargetName, + [{address, Ip}, + {port, Port}, + {engine_id, "agentEngine"}]); + inet6 -> + mgr_user_register_agent( + Node, TargetName, + [{tdomain, transportDomainUdpIpv6}, + {taddress, {Ip, Port}}, + {engine_id, "agentEngine"}]) + end, _Agents = mgr_user_which_own_agents(Node), ?DBG("Own agents: ~p", [_Agents]), @@ -5633,12 +5846,24 @@ init_mgr_user_data2(Conf) -> "~n Conf: ~p", [Conf]), Node = ?config(manager_node, Conf), TargetName = ?config(manager_agent_target_name, Conf), - Addr = ?config(ip, Conf), + IpFamily = ?config(ipfamily, Conf), + Ip = ?config(ip, Conf), Port = ?AGENT_PORT, - ?line ok = mgr_user_register_agent(Node, TargetName, - [{address, Addr}, - {port, Port}, - {engine_id, "agentEngine"}]), + ?line ok = + case IpFamily of + inet -> + mgr_user_register_agent( + Node, TargetName, + [{address, Ip}, + {port, Port}, + {engine_id, "agentEngine"}]); + inet6 -> + mgr_user_register_agent( + Node, TargetName, + [{tdomain, transportDomainUdpIpv6}, + {taddress, {Ip, Port}}, + {engine_id, "agentEngine"}]) + end, _Agents = mgr_user_which_own_agents(Node), ?DBG("Own agents: ~p", [_Agents]), @@ -5900,9 +6125,9 @@ start_manager(Node, Vsns, Conf0, _Opts) -> Conf0. -stop_manager(Node, Conf) -> - stop_snmp(Node), - Conf. +stop_manager(Node) -> + stop_snmp(Node). + %% -- Misc agent wrapper functions -- @@ -5951,9 +6176,8 @@ start_agent(Node, Vsns, Conf0, _Opts) -> ?line ok = start_snmp(Node), Conf0. -stop_agent(Node, Conf) -> - stop_snmp(Node), - Conf. +stop_agent(Node) -> + stop_snmp(Node). agent_load_mib(Node, Mib) -> rcall(Node, snmpa, load_mibs, [[Mib]]). @@ -6015,17 +6239,18 @@ stop_node(Node) -> rpc:cast(Node, erlang, halt, []), await_stopped(Node, 5). -await_stopped(_, 0) -> +await_stopped(Node, 0) -> + p("await_stopped -> ~p still exist: giving up", [Node]), ok; await_stopped(Node, N) -> Nodes = erlang:nodes(), case lists:member(Node, Nodes) of true -> - ?DBG("[~w] ~p still exist", [N, Node]), + p("await_stopped -> ~p still exist: ~w", [Node, N]), ?SLEEP(1000), await_stopped(Node, N-1); false -> - ?DBG("[~w] ~p gone", [N, Node]), + p("await_stopped -> ~p gone: ~w", [Node, N]), ok end. @@ -6035,10 +6260,16 @@ await_stopped(Node, N) -> write_manager_config(Config) -> Dir = ?config(manager_conf_dir, Config), - Ip = ?config(ip, Config), - Addr = tuple_to_list(Ip), - snmp_config:write_manager_snmp_files(Dir, Addr, ?MGR_PORT, - ?MGR_MMS, ?MGR_ENGINE_ID, [], [], []). + Ip = tuple_to_list(?config(ip, Config)), + {Addr, Port} = + case ?config(ipfamily, Config) of + inet -> + {Ip, ?MGR_PORT}; + inet6 -> + {transportDomainUdpIpv6, {Ip, ?MGR_PORT}} + end, + snmp_config:write_manager_snmp_files( + Dir, Addr, Port, ?MGR_MMS, ?MGR_ENGINE_ID, [], [], []). write_manager_conf(Dir) -> Port = "5000", @@ -6067,25 +6298,27 @@ write_manager_conf(Dir, Str) -> write_agent_config(Vsns, Conf) -> Dir = ?config(agent_conf_dir, Conf), - Ip = ?config(ip, Conf), - ?line Addr = tuple_to_list(Ip), - ?line ok = write_agent_config_files(Dir, Vsns, Addr), + ?line Ip = tuple_to_list(?config(ip, Conf)), + ?line Domain = + case ?config(ipfamily, Conf) of + inet -> + snmpUDPDomain; + inet6 -> + transportDomainUdpIpv6 + end, + ?line ok = write_agent_config_files(Dir, Vsns, Domain, Ip), ?line ok = update_agent_usm(Vsns, Dir), ?line ok = update_agent_community(Vsns, Dir), ?line ok = update_agent_vacm(Vsns, Dir), - ?line ok = write_agent_target_addr_conf(Dir, Addr, Vsns), + ?line ok = write_agent_target_addr_conf(Dir, Domain, Ip, Vsns), ?line ok = write_agent_target_params_conf(Dir, Vsns), ?line ok = write_agent_notify_conf(Dir), ok. -write_agent_config_files(Dir, Vsns, Addr) -> - snmp_config:write_agent_snmp_files(Dir, Vsns, - Addr, ?MGR_PORT, - Addr, ?AGENT_PORT, - "mgr-test", "trap", - none, "", - ?AGENT_ENGINE_ID, - ?AGENT_MMS). +write_agent_config_files(Dir, Vsns, Domain, Ip) -> + snmp_config:write_agent_snmp_files( + Dir, Vsns, Domain, {Ip, ?MGR_PORT}, {Ip, ?AGENT_PORT}, "mgr-test", + trap, none, "", ?AGENT_ENGINE_ID, ?AGENT_MMS). update_agent_usm(Vsns, Dir) -> case lists:member(v3, Vsns) of @@ -6153,9 +6386,9 @@ update_agent_vacm(_Vsns, Dir) -> excluded, null}], snmp_config:update_agent_vacm_config(Dir, Conf). -write_agent_target_addr_conf(Dir, Addr, Vsns) -> - snmp_config:write_agent_snmp_target_addr_conf(Dir, Addr, ?MGR_PORT, - 300, 3, Vsns). +write_agent_target_addr_conf(Dir, Domain, Ip, Vsns) -> + snmp_config:write_agent_snmp_target_addr_conf( + Dir, Domain, {Ip, ?MGR_PORT}, 300, 3, Vsns). write_agent_target_params_conf(Dir, Vsns) -> F = fun(v1) -> {"target_v1", v1, v1, "all-rights", noAuthNoPriv}; @@ -6271,7 +6504,7 @@ p(F, A) -> p(TName, F, A) -> io:format("*** [~w][~s] ***" - "~n" ++ F ++ "~n", [TName, formated_timestamp()|A]). + "~n " ++ F ++ "~n", [TName, formated_timestamp()|A]). formated_timestamp() -> snmp_test_lib:formated_timestamp(). @@ -6279,3 +6512,5 @@ formated_timestamp() -> %% p(TName, F, A) -> %% io:format("~w -> " ++ F ++ "~n", [TName|A]). +ipv6_init(Config) when is_list(Config) -> + [{ipfamily, inet6} | Config]. diff --git a/lib/snmp/test/snmp_manager_user.erl b/lib/snmp/test/snmp_manager_user.erl index ddbe156130..46c2b316be 100644 --- a/lib/snmp/test/snmp_manager_user.erl +++ b/lib/snmp/test/snmp_manager_user.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2013. All Rights Reserved. +%% Copyright Ericsson AB 2005-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -109,7 +109,18 @@ start_link(Parent, Id) -> proc_lib:start_link(?MODULE, main, [true, Parent, self(), Id]). stop() -> - cast(stop). + MRef = erlang:monitor(process, ?SERVER), + cast(stop), + receive {'DOWN', MRef, _, _, Info} -> + case Info of + noproc -> + ok; + noconnection -> + ok; + normal -> + ok + end + end. info() -> call(info). diff --git a/lib/snmp/test/snmp_test_lib.erl b/lib/snmp/test/snmp_test_lib.erl index fbb891e40d..5e611340a3 100644 --- a/lib/snmp/test/snmp_test_lib.erl +++ b/lib/snmp/test/snmp_test_lib.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2002-2013. All Rights Reserved. +%% Copyright Ericsson AB 2002-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -22,7 +22,7 @@ -include_lib("kernel/include/file.hrl"). --export([hostname/0, hostname/1, localhost/0, os_type/0, sz/1, +-export([hostname/0, hostname/1, localhost/0, localhost/1, os_type/0, sz/1, display_suite_info/1]). -export([non_pc_tc_maybe_skip/4, os_based_skip/1]). -export([fix_data_dir/1, @@ -60,6 +60,9 @@ from(_H, []) -> []. localhost() -> {ok, Ip} = snmp_misc:ip(net_adm:localhost()), Ip. +localhost(Family) -> + {ok, Ip} = snmp_misc:ip(net_adm:localhost(), Family), + Ip. sz(L) when is_list(L) -> length(L); diff --git a/lib/snmp/test/snmp_test_lib.hrl b/lib/snmp/test/snmp_test_lib.hrl index 8cc3f75dc5..9b7609b831 100644 --- a/lib/snmp/test/snmp_test_lib.hrl +++ b/lib/snmp/test/snmp_test_lib.hrl @@ -1,5 +1,5 @@ %%<copyright> -%% <year>2002-2008</year> +%% <year>2002-2014</year> %% <holder>Ericsson AB, All Rights Reserved</holder> %%</copyright> %%<legalnotice> @@ -32,14 +32,15 @@ -define(APPLICATION, snmp). -endif. --define(SCONF(K,D,C), snmp_test_lib:set_config(K,D,C)). --define(GCONF(K,C), snmp_test_lib:get_config(K,C)). --define(RCONF(K,C,V), snmp_test_lib:replace_config(K,C,V)). --define(HOSTNAME(N), snmp_test_lib:hostname(N)). --define(LOCALHOST(), snmp_test_lib:localhost()). --define(SZ(X), snmp_test_lib:sz(X)). --define(OSTYPE(), snmp_test_lib:os_type()). --define(DISPLAY_SUITE_INFO(), snmp_test_lib:display_suite_info(?MODULE)). +-define(SCONF(K,D,C), snmp_test_lib:set_config(K,D,C)). +-define(GCONF(K,C), snmp_test_lib:get_config(K,C)). +-define(RCONF(K,C,V), snmp_test_lib:replace_config(K,C,V)). +-define(HOSTNAME(N), snmp_test_lib:hostname(N)). +-define(LOCALHOST(), snmp_test_lib:localhost()). +-define(LOCALHOST(Family), snmp_test_lib:localhost(Family)). +-define(SZ(X), snmp_test_lib:sz(X)). +-define(OSTYPE(), snmp_test_lib:os_type()). +-define(DISPLAY_SUITE_INFO(), snmp_test_lib:display_suite_info(?MODULE)). %% - Test case macros - diff --git a/lib/snmp/test/snmp_test_manager.erl b/lib/snmp/test/snmp_test_manager.erl index 925ae77ab5..6d8673eecd 100644 --- a/lib/snmp/test/snmp_test_manager.erl +++ b/lib/snmp/test/snmp_test_manager.erl @@ -56,7 +56,7 @@ -export([init/1, handle_call/3, handle_cast/2, handle_info/2, code_change/3, terminate/2]). --record(state, {mgr, parent, req, agent_target_name}). +-record(state, {parent, req, agent_target_name}). -define(SERVER, ?MODULE). -define(USER, ?MODULE). @@ -130,10 +130,10 @@ init([Parent, Opts]) -> do_init(Opts) -> {MgrDir, MgrConf, MgrOpts, AgentTargetName, AgentConf} = parse_opts(Opts), ok = snmp_config:write_manager_config(MgrDir, "", MgrConf), - {ok, Pid} = snmpm:start_link(MgrOpts), + ok = snmpm:start_link(MgrOpts), ok = snmpm:register_user(?USER, ?MODULE, self()), ok = snmpm:register_agent(?USER, AgentTargetName, AgentConf), - {ok, #state{mgr = Pid, agent_target_name = AgentTargetName}}. + {ok, #state{agent_target_name = AgentTargetName}}. parse_opts(Opts) -> diff --git a/lib/snmp/test/snmp_test_mgr.erl b/lib/snmp/test/snmp_test_mgr.erl index 40fcbce8f1..cf62edba1c 100644 --- a/lib/snmp/test/snmp_test_mgr.erl +++ b/lib/snmp/test/snmp_test_mgr.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2012. All Rights Reserved. +%% Copyright Ericsson AB 1996-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -235,21 +235,23 @@ init({Options, CallerPid}) -> VsnHdrD = {Com, User, EngineId, CtxEngineId, mk_seclevel(SecLevel)}, io:format("[~w] ~p -> VsnHdrD: ~p~n", [?MODULE, self(), VsnHdrD]), + IpFamily = get_value(ipfamily, Options, inet), + io:format("[~w] ~p -> IpFamily: ~p~n", [?MODULE, self(), IpFamily]), AgIp = case snmp_misc:assq(agent, Options) of {value, Tuple4} when is_tuple(Tuple4) andalso (size(Tuple4) =:= 4) -> Tuple4; {value, Host} when is_list(Host) -> - {ok, Ip} = snmp_misc:ip(Host), + {ok, Ip} = snmp_misc:ip(Host, IpFamily), Ip end, io:format("[~w] ~p -> AgIp: ~p~n", [?MODULE, self(), AgIp]), Quiet = lists:member(quiet, Options), io:format("[~w] ~p -> Quiet: ~p~n", [?MODULE, self(), Quiet]), - PackServ = start_packet_server(Quiet, Options, CallerPid, - AgIp, Udp, TrapUdp, - VsnHdrD, Version, Dir, RecBufSz, - PacksDbg), + PackServ = + start_packet_server( + Quiet, Options, CallerPid, AgIp, Udp, TrapUdp, + VsnHdrD, Version, Dir, RecBufSz, PacksDbg, IpFamily), d("init -> packet server: ~p",[PackServ]), State = #state{parent = CallerPid, quiet = Quiet, @@ -263,23 +265,21 @@ init({Options, CallerPid}) -> end. start_packet_server(false, _Options, _CallerPid, AgIp, Udp, TrapUdp, - VsnHdrD, Version, Dir, RecBufSz, PacksDbg) -> + VsnHdrD, Version, Dir, RecBufSz, PacksDbg, IpFamily) -> d("start_packet_server -> entry", []), - ?PACK_SERV:start_link_packet({msg, self()}, - AgIp, Udp, TrapUdp, - VsnHdrD, Version, Dir, RecBufSz, - PacksDbg); + ?PACK_SERV:start_link_packet( + {msg, self()}, AgIp, Udp, TrapUdp, + VsnHdrD, Version, Dir, RecBufSz, PacksDbg, IpFamily); start_packet_server(true, Options, CallerPid, AgIp, Udp, TrapUdp, - VsnHdrD, Version, Dir, RecBufSz, PacksDbg) -> + VsnHdrD, Version, Dir, RecBufSz, PacksDbg, IpFamily) -> Type = get_value(receive_type, Options, pdu), d("start_packet_server -> entry with" "~n CallerPid: ~p" "~n when" "~n Type: ~p",[CallerPid, Type]), - ?PACK_SERV:start_link_packet({Type, CallerPid}, - AgIp, Udp, TrapUdp, - VsnHdrD, Version, Dir, RecBufSz, - PacksDbg). + ?PACK_SERV:start_link_packet( + {Type, CallerPid}, AgIp, Udp, TrapUdp, + VsnHdrD, Version, Dir, RecBufSz, PacksDbg, IpFamily). is_options_ok([{mibs,List}|Opts]) when is_list(List) -> is_options_ok(Opts); @@ -287,6 +287,10 @@ is_options_ok([quiet|Opts]) -> is_options_ok(Opts); is_options_ok([{agent,_}|Opts]) -> is_options_ok(Opts); +is_options_ok([{ipfamily,IpFamily}|Opts]) + when IpFamily =:= inet; + IpFamily =:= inet6 -> + is_options_ok(Opts); is_options_ok([{agent_udp,Int}|Opts]) when is_integer(Int) -> is_options_ok(Opts); is_options_ok([{trap_udp,Int}|Opts]) when is_integer(Int) -> @@ -657,7 +661,8 @@ make_vb(Oid) -> #varbind{oid = Oid, variabletype = 'NULL', value = 'NULL'}. make_request_id() -> - random:uniform(16#FFFFFFF-1). + %% random:uniform(16#FFFFFFF-1). + snmp_test_mgr_counter_server:increment(mgr_request_id, 1, 1, 2147483647). echo_pdu(PDU, MiniMIB) -> io:format("~s", [snmp_misc:format_pdu(PDU, MiniMIB)]). diff --git a/lib/snmp/test/snmp_test_mgr_counter_server.erl b/lib/snmp/test/snmp_test_mgr_counter_server.erl new file mode 100644 index 0000000000..db31e0380b --- /dev/null +++ b/lib/snmp/test/snmp_test_mgr_counter_server.erl @@ -0,0 +1,152 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2014-2014. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% + +%% +%% The reason for this (test) counter server is that the +%% agent test suite is implemented in such a way that the +%% agent is started once and then used for several test cases. +%% Each request is given a request id which *was* generated using +%% random! It is therefor possible, although unlikely, that a +%% request may get a request id that has recently been used, +%% which will cause the agent to silently reject the request. +%% For this reason, we start this server at the start of the +%% agent suite and stop it at the end and all request ids are +%% generated by this server. +%% + +-module(snmp_test_mgr_counter_server). + +-export([start/0, stop/0, increment/4]). + +-define(SERVER, ?MODULE). +-define(TAB, snmp_test_mgr_counter_tab). + + +%%%------------------------------------------------------------------- +%%% API +%%%------------------------------------------------------------------- + +-spec start() -> ok. + +start() -> + Parent = self(), + ReqIdServer = spawn(fun() -> init(Parent) end), + receive + {ReqIdServer, ok} -> + ok; + {ReqIdServer, {error, Reason}} -> + exit({failed_starting_counter_server, Reason}) + after 5000 -> + exit(ReqIdServer, kill), % Cleanup, just in case + exit({failed_starting_counter_server, timeout}) + end. + +-spec stop() -> {ok, Counters :: list()} | {error, Reason :: term()}. + +stop() -> + request(stop). + + +-spec increment(Counter :: atom(), + Initial :: non_neg_integer(), + Increment :: pos_integer(), + Max :: pos_integer()) -> + Next :: pos_integer(). + +increment(Counter, Initial, Increment, Max) -> + Request = {increment, Counter, Initial, Increment, Max}, + case request(Request) of + {ok, ReqId} -> + ReqId; + {error, Reason} -> + exit(Reason) + end. + + +request(Request) -> + Id = make_ref(), + Msg = {self(), Id, Request}, + try + begin + global:send(?SERVER, Msg), + receive + {reply, Id, Reply} -> + {ok, Reply} + end + end + catch + T:E -> + {error, {T, E}} + end. + + +%%%------------------------------------------------------------------- +%%% Internal functions +%%%------------------------------------------------------------------- + +init(Parent) -> + p("starting"), + case global:register_name(?SERVER, self()) of + yes -> + p("name registration ok"), + Parent ! {self(), ok}; + no -> + p("name registration failed"), + Parent ! {self(), registration_failed}, + exit(registration_failed) + end, + ets:new(?TAB, [set, named_table, {keypos, 1}]), + loop(). + +loop() -> + receive + {From, Id, {increment, Counter, Initial, Increment, Max}} -> + Position = 2, + Threshold = Max, + SetValue = Initial, + UpdateOp = {Position, Increment, Threshold, SetValue}, + NextVal = + try ets:update_counter(?TAB, Counter, UpdateOp) of + Next when is_integer(Next) -> + p("increment ~w: (next) ~w", [Counter, Next]), + Next + catch + error:badarg -> + %% Oups, first time + p("increment ~w: (initial) ~w", [Counter, Initial]), + ets:insert(?TAB, {Counter, Initial}), + Initial + end, + From ! {reply, Id, NextVal}, + loop(); + + {From, Id, stop} -> + p("stop"), + Counters = ets:tab2list(?TAB), + From ! {reply, Id, Counters}, + exit(normal) + end. + + +p(F) -> + p(F, []). + +p(F, A) -> + io:format("*** [~s] COUNTER-SERVER [~w] " ++ F ++ "~n", + [snmp_test_lib:formated_timestamp(), self() | A]). diff --git a/lib/snmp/test/snmp_test_mgr_misc.erl b/lib/snmp/test/snmp_test_mgr_misc.erl index 5525c5c3ec..5274dcacd9 100644 --- a/lib/snmp/test/snmp_test_mgr_misc.erl +++ b/lib/snmp/test/snmp_test_mgr_misc.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2012. All Rights Reserved. +%% Copyright Ericsson AB 1996-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -23,7 +23,7 @@ -module(snmp_test_mgr_misc). %% API --export([start_link_packet/8, start_link_packet/9, +-export([start_link_packet/8, start_link_packet/9, start_link_packet/10, stop/1, send_discovery_pdu/2, send_pdu/2, send_msg/4, send_bytes/2, @@ -31,7 +31,7 @@ get_pdu/1, set_pdu/2, format_hdr/1]). %% internal exports --export([init_packet/10]). +-export([init_packet/11]). -compile({no_auto_import, [error/2]}). @@ -42,22 +42,26 @@ %%---------------------------------------------------------------------- %% The InHandler process will receive messages on the form {snmp_pdu, Pdu}. %%---------------------------------------------------------------------- -start_link_packet(InHandler, - AgentIp, UdpPort, TrapUdp, - VsnHdr, Version, Dir, BufSz) -> - start_link_packet(InHandler, - AgentIp, UdpPort, TrapUdp, - VsnHdr, Version, Dir, BufSz, - false). - -start_link_packet(InHandler, - AgentIp, UdpPort, TrapUdp, - VsnHdr, Version, Dir, BufSz, - Dbg) when is_integer(UdpPort) -> - Args = [self(), InHandler, - AgentIp, UdpPort, TrapUdp, - VsnHdr, Version, Dir, BufSz, - Dbg], +start_link_packet( + InHandler, AgentIp, UdpPort, TrapUdp, VsnHdr, Version, Dir, BufSz) -> + start_link_packet( + InHandler, AgentIp, UdpPort, TrapUdp, VsnHdr, Version, Dir, BufSz, + false). + +start_link_packet( + InHandler, AgentIp, UdpPort, TrapUdp, VsnHdr, Version, Dir, BufSz, + Dbg) -> + start_link_packet( + InHandler, AgentIp, UdpPort, TrapUdp, VsnHdr, Version, Dir, BufSz, + Dbg, inet). + +start_link_packet( + InHandler, AgentIp, UdpPort, TrapUdp, VsnHdr, Version, Dir, BufSz, + Dbg, IpFamily) when is_integer(UdpPort) -> + Args = + [self(), + InHandler, AgentIp, UdpPort, TrapUdp, VsnHdr, Version, Dir, BufSz, + Dbg, IpFamily], proc_lib:start_link(?MODULE, init_packet, Args). stop(Pid) -> @@ -90,12 +94,14 @@ send_bytes(Bytes, PacketPid) -> %%-------------------------------------------------- %% The SNMP encode/decode process %%-------------------------------------------------- -init_packet(Parent, SnmpMgr, - AgentIp, UdpPort, TrapUdp, - VsnHdr, Version, Dir, BufSz, DbgOptions) -> +init_packet( + Parent, + SnmpMgr, AgentIp, UdpPort, TrapUdp, VsnHdr, Version, Dir, BufSz, + DbgOptions, IpFamily) -> put(sname, mgr_misc), init_debug(DbgOptions), - {ok, UdpId} = gen_udp:open(TrapUdp, [{recbuf,BufSz},{reuseaddr, true}]), + {ok, UdpId} = + gen_udp:open(TrapUdp, [{recbuf,BufSz}, {reuseaddr, true}, IpFamily]), put(msg_id, 1), proc_lib:init_ack(Parent, self()), init_usm(Version, Dir), diff --git a/lib/snmp/test/snmp_to_snmpnet_SUITE.erl b/lib/snmp/test/snmp_to_snmpnet_SUITE.erl new file mode 100644 index 0000000000..2f96493ac5 --- /dev/null +++ b/lib/snmp/test/snmp_to_snmpnet_SUITE.erl @@ -0,0 +1,559 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2014-2014. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%% This test suite uses the following external programs: +%% snmpget From packet 'snmp' (in Ubuntu 12.04) +%% snmpd From packet 'snmpd' (in Ubuntu 12.04) +%% snmptrapd From packet 'snmpd' (in Ubuntu 12.04) +%% They originate from the Net-SNMP applications, see: +%% http://net-snmp.sourceforge.net/ + + +-module(snmp_to_snmpnet_SUITE). + +%% Note: This directive should only be used in test suites. +-compile(export_all). + +-include_lib("common_test/include/ct.hrl"). +-include_lib("snmp/include/STANDARD-MIB.hrl"). + +-define(AGENT_ENGINE_ID, "ErlangSnmpAgent"). +-define(MANAGER_ENGINE_ID, "ErlangSnmpManager"). +-define(AGENT_PORT, 4000). +-define(MANAGER_PORT, 8989). +-define(DEFAULT_MAX_MESSAGE_SIZE, 484). + +expected(?sysDescr_instance = Oid, get) -> + OidStr = oid_str(Oid), + iolist_to_binary([OidStr | " = STRING: \"Erlang SNMP agent\""]). + +%%-------------------------------------------------------------------- +%% Common Test interface functions ----------------------------------- +%%-------------------------------------------------------------------- + +suite() -> [{ct_hooks,[ts_install_cth]}]. + +all() -> + [ + {group, ipv4}, + {group, ipv6}, + {group, ipv4_ipv6} + ]. + +groups() -> + [{ipv4, [], + [{group, snmpget}, + {group, snmptrapd}, + {group, snmpd_mt}, + {group, snmpd} + ]}, + {ipv6, [], + [{group, snmpget}, + {group, snmptrapd}, + {group, snmpd_mt}, + {group, snmpd} + ]}, + {ipv4_ipv6, [], + [{group, snmpget}, + {group, snmptrapd}, + {group, snmpd_mt}, + {group, snmpd} + ]}, + %% + {snmpget, [], + [erlang_agent_netsnmp_get]}, + {snmptrapd, [], + [erlang_agent_netsnmp_inform]}, + {snmpd_mt, [], + [erlang_manager_netsnmp_get]}, + {snmpd, [], + [erlang_manager_netsnmp_get]} + ]. + +init_per_suite(Config) -> + [{agent_port, ?AGENT_PORT}, {manager_port, ?MANAGER_PORT} | Config]. + +end_per_suite(_Config) -> + ok. + +init_per_group(ipv4, Config) -> + init_per_group_ip([inet], Config); +init_per_group(ipv6, Config) -> + init_per_group_ipv6([inet6], Config); +init_per_group(ipv4_ipv6, Config) -> + init_per_group_ipv6([inet, inet6], Config); +%% +init_per_group(snmpget = Exec, Config) -> + %% From Ubuntu package snmp + init_per_group_agent(Exec, Config); +init_per_group(snmptrapd = Exec, Config) -> + %% From Ubuntu package snmpd + init_per_group_agent(Exec, Config); +init_per_group(snmpd_mt, Config) -> + %% From Ubuntu package snmp + init_per_group_manager( + snmpd, + [{manager_net_if_module, snmpm_net_if_mt} | Config]); +init_per_group(snmpd = Exec, Config) -> + %% From Ubuntu package snmp + init_per_group_manager( + Exec, + [{manager_net_if_module, snmpm_net_if} | Config]); +%% +init_per_group(_, Config) -> + Config. + +init_per_group_ipv6(Families, Config) -> + case ct:require(ipv6_hosts) of + ok -> + case gen_udp:open(0, [inet6]) of + {ok, S} -> + ok = gen_udp:close(S), + init_per_group_ip(Families, Config); + {error, _} -> + {skip, "Host seems to not support IPv6"} + end; + _ -> + {skip, "Test config ipv6_hosts is missing"} + end. + +init_per_group_ip(Families, Config) -> + AgentPort = ?config(agent_port, Config), + ManagerPort = ?config(manager_port, Config), + {ok, Host} = inet:gethostname(), + Transports = + [begin + {ok, Addr} = inet:getaddr(Host, Family), + {domain(Family), {Addr, AgentPort}} + end || Family <- Families], + Targets = + [begin + {ok, Addr} = inet:getaddr(Host, Family), + {domain(Family), {Addr, ManagerPort}} + end || Family <- Families], + [{transports, Transports}, {targets, Targets} | Config]. + +init_per_group_agent(Exec, Config) -> + Versions = [v2], + Dir = ?config(priv_dir, Config), + Transports = ?config(transports, Config), + Targets = ?config(targets, Config), + agent_config(Dir, Transports, Targets, Versions), + find_executable(Exec, [{snmp_versions, Versions} | Config]). + +init_per_group_manager(Exec, Config) -> + Versions = [v2], + Dir = ?config(priv_dir, Config), + Targets = ?config(targets, Config), + manager_config(Dir, Targets), + find_executable(Exec, [{snmp_versions, Versions} | Config]). + + + +end_per_group(_GroupName, Config) -> + Config. + +init_per_testcase(_Case, Config) -> + Dog = ct:timetrap(20000), + application:stop(snmp), + application:unload(snmp), + [{watchdog, Dog} | Config]. + +end_per_testcase(_, Config) -> + case application:stop(snmp) of + ok -> + ok; + E1 -> + ct:pal("application:stop(snmp) -> ~p", [E1]) + end, + case application:unload(snmp) of + ok -> + ok; + E2 -> + ct:pal("application:unload(snmp) -> ~p", [E2]) + end, + Config. + +find_executable(Exec, Config) -> + ExecStr = atom_to_list(Exec), + case os:find_executable(ExecStr) of + false -> + %% The sbin dirs are not in the PATH on all platforms... + find_sys_executable( + Exec, ExecStr, + [["usr", "local", "sbin"], + ["usr", "sbin"], + ["sbin"]], + Config); + Path -> + [{Exec, Path} | Config] + end. + +find_sys_executable(_Exec, ExecStr, [], _Config) -> + {skip, ExecStr ++ " not found"}; +find_sys_executable(Exec, ExecStr, [Dir | Dirs], Config) -> + case os:find_executable(filename:join(["/" | Dir] ++ [ExecStr])) of + false -> + find_sys_executable(Exec, ExecStr, Dirs, Config); + Path -> + [{Exec, Path} | Config] + end. + +start_agent(Config) -> + ok = application:load(snmp), + ok = application:set_env(snmp, agent, agent_app_env(Config)), + ok = application:start(snmp). + +start_manager(Config) -> + ok = application:load(snmp), + ok = application:set_env(snmp, manager, manager_app_env(Config)), + ok = application:start(snmp). + +%%-------------------------------------------------------------------- +%% Test Cases -------------------------------------------------------- +%%-------------------------------------------------------------------- +erlang_agent_netsnmp_get() -> + [{doc,"Test that we can access erlang snmp agent " + "from snmpnet manager"}]. + +erlang_agent_netsnmp_get(Config) when is_list(Config) -> + Transports = ?config(transports, Config), + start_agent(Config), + Oid = ?sysDescr_instance, + Expected = expected(Oid, get), + [Expected = snmpget(Oid, Transport, Config) + || Transport <- Transports], + ok. + +%%-------------------------------------------------------------------- +erlang_manager_netsnmp_get() -> + [{doc,"Test that the erlang snmp manager can access snmpnet agent"}]. + +erlang_manager_netsnmp_get(Config) when is_list(Config) -> + Community = "happy-testing", + SysDescr = "Net-SNMP agent", + TargetName = "Target Net-SNMP agent", + Transports = ?config(transports, Config), + ProgHandle = start_snmpd(Community, SysDescr, Config), + start_manager(Config), + snmp_manager_user:start_link(self(), test_user), + [snmp_manager_user:register_agent( + TargetName++domain_suffix(Domain), + [{reg_type, target_name}, + {tdomain, Domain}, {taddress, Addr}, + {community, Community}, {engine_id, "EngineId"}, + {version, v2}, {sec_model, v2c}, {sec_level, noAuthNoPriv}]) + || {Domain, Addr} <- Transports], + Results = + [snmp_manager_user:sync_get( + TargetName++domain_suffix(Domain), + [?sysDescr_instance]) + || {Domain, _} <- Transports], + ct:pal("sync_get -> ~p", [Results]), + snmp_manager_user:stop(), + stop_program(ProgHandle), + [{ok, + {noError, 0, + [{varbind, ?sysDescr_instance, 'OCTET STRING', SysDescr,1}] }, + _} = R || R <- Results], + ok. + +%%-------------------------------------------------------------------- +erlang_agent_netsnmp_inform(Config) when is_list(Config) -> + DataDir = ?config(data_dir, Config), + Mib = "TestTrapv2", + + start_agent(Config), + ok = snmpa:load_mib(snmp_master_agent, filename:join(DataDir, Mib)), + + ProgHandle = start_snmptrapd(Mib, Config), + + snmpa:send_notification( + snmp_master_agent, testTrapv22, {erlang_agent_test, self()}), + + receive + {snmp_targets, erlang_agent_test, Addresses} -> + ct:pal("Notification sent to: ~p~n", [Addresses]), + erlang_agent_netsnmp_inform_responses(Addresses) + end, + stop_program(ProgHandle). + +erlang_agent_netsnmp_inform_responses([]) -> + receive + {snmp_notification, erlang_agent_test, _} = Unexpected -> + ct:pal("Unexpected response: ~p", [Unexpected]), + erlang_agent_netsnmp_inform_responses([]) + after 0 -> + ok + end; +erlang_agent_netsnmp_inform_responses([Address | Addresses]) -> + receive + {snmp_notification, erlang_agent_test, + {got_response, Address}} -> + ct:pal("Got response from: ~p~n", [Address]), + erlang_agent_netsnmp_inform_responses(Addresses); + {snmp_notification, erlang_agent_test, + {no_response, _} = NoResponse} -> + ct:fail(NoResponse) + end. + +%%-------------------------------------------------------------------- +%% Internal functions ------------------------------------------------ +%%-------------------------------------------------------------------- +snmpget(Oid, Transport, Config) -> + Versions = ?config(snmp_versions, Config), + + Args = + ["-c", "public", net_snmp_version(Versions), + "-m", "", + "-Cf", + net_snmp_addr_str(Transport), + oid_str(Oid)], + ProgHandle = start_program(snmpget, Args, none, Config), + {_, line, Line} = get_program_output(ProgHandle), + stop_program(ProgHandle), + Line. + +start_snmptrapd(Mibs, Config) -> + DataDir = ?config(data_dir, Config), + MibDir = filename:join(code:lib_dir(snmp), "mibs"), + Targets = ?config(targets, Config), + SnmptrapdArgs = + ["-f", "-Lo", "-C", + "-m", Mibs, + "-M", MibDir++":"++DataDir, + "--disableAuthorization=yes", + "--snmpTrapdAddr=" ++ net_snmp_addr_str(Targets)], + {ok, StartCheckMP} = re:compile("NET-SNMP version ", [anchored]), + start_program(snmptrapd, SnmptrapdArgs, StartCheckMP, Config). + +start_snmpd(Community, SysDescr, Config) -> + DataDir = ?config(data_dir, Config), + Targets = ?config(targets, Config), + Transports = ?config(transports, Config), + Port = mk_port_number(), + CommunityArgs = + ["--rocommunity"++domain_suffix(Domain)++"=" + ++Community++" "++inet_parse:ntoa(Ip) + || {Domain, {Ip, _}} <- Targets], + SnmpdArgs = + ["-f", "-r", %"-Dverbose", + "-c", filename:join(DataDir, "snmpd.conf"), + "-C", "-Lo", + "-m", "", + "--sysDescr="++SysDescr, + "--agentXSocket=tcp:localhost:"++integer_to_list(Port)] + ++ CommunityArgs + ++ [net_snmp_addr_str(Transports)], + {ok, StartCheckMP} = re:compile("NET-SNMP version ", [anchored]), + start_program(snmpd, SnmpdArgs, StartCheckMP, Config). + +start_program(Prog, Args, StartCheckMP, Config) -> + ct:pal("Starting program: ~w ~p", [Prog, Args]), + Path = ?config(Prog, Config), + DataDir = ?config(data_dir, Config), + StartWrapper = filename:join(DataDir, "start_stop_wrapper"), + Parent = self(), + Pid = + spawn_link( + fun () -> + run_program(Parent, StartWrapper, [Path | Args]) + end), + start_check(Pid, erlang:monitor(process, Pid), StartCheckMP). + +start_check(Pid, Mon, none) -> + {Pid, Mon}; +start_check(Pid, Mon, StartCheckMP) -> + receive + {Pid, line, Line} -> + case re:run(Line, StartCheckMP, [{capture, none}]) of + match -> + {Pid, Mon}; + nomatch -> + start_check(Pid, Mon, StartCheckMP) + end; + {'DOWN', Mon, _, _, Reason} -> + ct:fail("Prog ~p start failed: ~p", [Pid, Reason]) + end. + +get_program_output({Pid, Mon}) -> + receive + {Pid, _, _} = Msg -> + Msg; + {'DOWN', Mon, _, _, Reason} -> + ct:fail("Prog ~p crashed: ~p", [Pid, Reason]) + end. + +stop_program({Pid, _} = Handle) -> + Pid ! {self(), stop}, + wait_program_stop(Handle). + +wait_program_stop({Pid, Mon}) -> + receive + {Pid, exit, ExitStatus} -> + receive + {'DOWN', Mon, _, _, _} -> + ExitStatus + end; + {'DOWN', Mon, _, _, Reason} -> + ct:fail("Prog stop: ~p", [Reason]) + end. + +run_program(Parent, StartWrapper, ProgAndArgs) -> + [Prog | _] = ProgAndArgs, + Port = + open_port( + {spawn_executable, StartWrapper}, + [{args, ProgAndArgs}, binary, stderr_to_stdout, {line, 80}, + exit_status]), + ct:pal("Prog ~p started: ~p", [Port, Prog]), + run_program_loop(Parent, Port, []). + +run_program_loop(Parent, Port, Buf) -> + receive + {Parent, stop} -> + true = port_command(Port, <<"stop\n">>), + ct:pal("Prog ~p stop", [Port]), + run_program_loop(Parent, Port, Buf); + {Port, {data, {Flag, Data}}} -> + case Flag of + eol -> + Line = iolist_to_binary(lists:reverse(Buf, Data)), + ct:pal("Prog ~p output: ~s", [Port, Line]), + Parent ! {self(), line, Line}, + run_program_loop(Parent, Port, []); + noeol -> + run_program_loop(Parent, Port, [Data | Buf]) + end; + {Port, {exit_status,ExitStatus}} -> + ct:pal("Prog ~p exit: ~p", [Port, ExitStatus]), + catch port_close(Port), + Parent ! {self(), exit, ExitStatus}; + Unexpected -> + ct:pal("run_program_loop Unexpected: ~p", [Unexpected]), + run_program_loop(Parent, Port, Buf) + end. + + +agent_app_env(Config) -> + Dir = ?config(priv_dir, Config), + Vsns = ?config(snmp_versions, Config), + [{versions, Vsns}, + {agent_type, master}, + {agent_verbosity, trace}, + {db_dir, Dir}, + {audit_trail_log, [{type, read_write}, + {dir, Dir}, + {size, {10240, 10}}]}, + {config, [{dir, Dir}, + {force_load, true}, + {verbosity, trace}]}, + {local_db, [{repair, true}, + {verbosity, silence}]}, + {mib_server, [{verbosity, silence}]}, + {symbolic_store, [{verbosity, silence}]}, + {note_store, [{verbosity, silence}]}, + {net_if, [{verbosity, trace}]}]. + +manager_app_env(Config) -> + Dir = ?config(priv_dir, Config), + Vsns = ?config(snmp_versions, Config), + NetIfModule = ?config(manager_net_if_module, Config), + [{versions, Vsns}, + {audit_trail_log, [{type, read_write}, + {dir, Dir}, + {size, {10240, 10}}]}, + {net_if, [{module, NetIfModule}]}, + {config, [{dir, Dir}, + {db_dir, Dir}, + {verbosity, trace}]} + ]. + +oid_str([1 | Ints]) -> + "iso." ++ oid_str_tl(Ints); +oid_str(Ints) -> + oid_str_tl(Ints). + +oid_str_tl([]) -> + ""; +oid_str_tl([Int]) -> + integer_to_list(Int); +oid_str_tl([Int | Ints]) -> + integer_to_list(Int) ++ "." ++ oid_str_tl(Ints). + +agent_config(Dir, Transports, Targets, Versions) -> + EngineID = ?AGENT_ENGINE_ID, + MMS = ?DEFAULT_MAX_MESSAGE_SIZE, + ok = snmp_config:write_agent_snmp_conf(Dir, Transports, EngineID, MMS), + ok = snmp_config:write_agent_snmp_context_conf(Dir), + ok = snmp_config:write_agent_snmp_community_conf(Dir), + ok = + snmp_config:write_agent_snmp_standard_conf( + Dir, "snmp_to_snmpnet_SUITE"), + ok = + snmp_config:write_agent_snmp_target_addr_conf( + Dir, Targets, Versions), + ok = snmp_config:write_agent_snmp_target_params_conf(Dir, Versions), + ok = snmp_config:write_agent_snmp_notify_conf(Dir, inform), + ok = snmp_config:write_agent_snmp_vacm_conf(Dir, Versions, none). + +manager_config(Dir, Targets) -> + EngineID = ?MANAGER_ENGINE_ID, + MMS = ?DEFAULT_MAX_MESSAGE_SIZE, + ok = snmp_config:write_manager_snmp_conf(Dir, Targets, MMS, EngineID). + +net_snmp_version([v3 | _]) -> + "-v3"; +net_snmp_version([v2 | _]) -> + "-v2c"; +net_snmp_version([v1 | _]) -> + "-v1". + +domain(inet) -> + transportDomainUdpIpv4; +domain(inet6) -> + transportDomainUdpIpv6. + +net_snmp_addr_str([Target | Targets]) -> + net_snmp_addr_str(Target) ++ + case Targets of + [] -> + []; + [_ | _] -> + "," ++ net_snmp_addr_str(Targets) + end; +net_snmp_addr_str({transportDomainUdpIpv4, {Addr, Port}}) -> + "udp:" ++ + inet_parse:ntoa(Addr) ++ ":" ++ + integer_to_list(Port); +net_snmp_addr_str({transportDomainUdpIpv6, {Addr, Port}}) -> + "udp6:[" ++ + inet_parse:ntoa(Addr) ++ "]:" ++ + integer_to_list(Port). + +domain_suffix(transportDomainUdpIpv4) -> + ""; +domain_suffix(transportDomainUdpIpv6) -> + "6". + +mk_port_number() -> + {ok, Socket} = gen_udp:open(0, [{reuseaddr, true}]), + {ok, PortNum} = inet:port(Socket), + ok = gen_udp:close(Socket), + PortNum. diff --git a/lib/snmp/test/snmp_to_snmpnet_SUITE_data/TestTrapv2.bin b/lib/snmp/test/snmp_to_snmpnet_SUITE_data/TestTrapv2.bin Binary files differnew file mode 100644 index 0000000000..9d0790498d --- /dev/null +++ b/lib/snmp/test/snmp_to_snmpnet_SUITE_data/TestTrapv2.bin diff --git a/lib/snmp/test/snmp_to_snmpnet_SUITE_data/TestTrapv2.mib b/lib/snmp/test/snmp_to_snmpnet_SUITE_data/TestTrapv2.mib new file mode 100644 index 0000000000..679ddc14b0 --- /dev/null +++ b/lib/snmp/test/snmp_to_snmpnet_SUITE_data/TestTrapv2.mib @@ -0,0 +1,71 @@ +TestTrapv2 DEFINITIONS ::= BEGIN + +IMPORTS + MODULE-IDENTITY, OBJECT-TYPE, NOTIFICATION-TYPE, + TimeTicks, Counter32, snmpModules, mib-2, enterprises, IpAddress, + Integer32 + FROM SNMPv2-SMI + DisplayString, TestAndIncr, TimeStamp, RowStatus, TruthValue, + TEXTUAL-CONVENTION + FROM SNMPv2-TC + MODULE-COMPLIANCE, OBJECT-GROUP, NOTIFICATION-GROUP + FROM SNMPv2-CONF + + system, snmp, ifIndex, ifAdminStatus, ifOperStatus + FROM RFC1213-MIB + snmpTraps + FROM SNMPv2-MIB; + +testTrapv2 MODULE-IDENTITY + LAST-UPDATED "9511090000Z" + ORGANIZATION "IETF SNMPv2 Working Group" + CONTACT-INFO + " Marshall T. Rose + + Postal: Dover Beach Consulting, Inc. + 420 Whisman Court + Mountain View, CA 94043-2186 + US + + Tel: +1 415 968 1052 + + E-mail: [email protected]" + DESCRIPTION + "The MIB module for SNMPv2 entities." + REVISION "9304010000Z" + DESCRIPTION + "The initial revision of this MIB module was published as + RFC 1450." + ::= { system 100 } + + +tst OBJECT IDENTIFIER ::= { system 0 } + +testTrapv21 NOTIFICATION-TYPE + STATUS current + DESCRIPTION + "This trap is exactly the v2 correspondance of testTrap1 in + TestTrap mib." + ::= { snmp 1 } + +testTrapv22 NOTIFICATION-TYPE + STATUS current + DESCRIPTION + "This trap is exactly the v2 correspondance of testTrap2 in + TestTrap mib." + ::= { system 0 1 } + +linkUp NOTIFICATION-TYPE + OBJECTS { ifIndex, ifAdminStatus, ifOperStatus } + STATUS current + DESCRIPTION + "A linkUp trap signifies that the SNMPv2 entity, + acting in an agent role, has detected that the + ifOperStatus object for one of its communication links + has transitioned out of the down state." + ::= { snmpTraps 4 } + + + + +END diff --git a/lib/snmp/test/snmp_to_snmpnet_SUITE_data/snmpd.conf b/lib/snmp/test/snmp_to_snmpnet_SUITE_data/snmpd.conf new file mode 100644 index 0000000000..2a5f31680f --- /dev/null +++ b/lib/snmp/test/snmp_to_snmpnet_SUITE_data/snmpd.conf @@ -0,0 +1,12 @@ +sysLocation On the lab network +sysContact otptest <[email protected]> + +createUser myinternaluser SHA dropdead + +agentSecName myinternaluser + +master agentx + +[snmp] +noPersistentLoad yes +noPersistentSave yes diff --git a/lib/snmp/test/snmp_to_snmpnet_SUITE_data/start_stop_wrapper b/lib/snmp/test/snmp_to_snmpnet_SUITE_data/start_stop_wrapper new file mode 100755 index 0000000000..f806ab5c12 --- /dev/null +++ b/lib/snmp/test/snmp_to_snmpnet_SUITE_data/start_stop_wrapper @@ -0,0 +1,47 @@ +#! /bin/sh +## +## %CopyrightBegin% +## +## Copyright Ericsson AB 2014-2014. All Rights Reserved. +## +## The contents of this file are subject to the Erlang Public License, +## Version 1.1, (the "License"); you may not use this file except in +## compliance with the License. You should have received a copy of the +## Erlang Public License along with this software. If not, it can be +## retrieved online at http://www.erlang.org/. +## +## Software distributed under the License is distributed on an "AS IS" +## basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +## the License for the specific language governing rights and limitations +## under the License. +## +## %CopyrightEnd% +## +# + +## Start the given executable, wait for stop command, +## stop the running executable and wait for exit. + +die () { + r=$? + echo "$0:" "$*" 1>&2 + exit $r +} + +test -x "$1" || die "Not Executable: $1" + +# Redirect stdin to make sure the stop command is read by us below +# and does not go to the executable +"$@" 0< /dev/null & +PID=$! + +# Wait for stop command +while read LINE; do + case :"$LINE" in + :"stop") + break;; + esac +done + +kill $PID +wait $PID diff --git a/lib/snmp/vsn.mk b/lib/snmp/vsn.mk index 04c3cc9392..b436a79076 100644 --- a/lib/snmp/vsn.mk +++ b/lib/snmp/vsn.mk @@ -18,6 +18,6 @@ # %CopyrightEnd% APPLICATION = snmp -SNMP_VSN = 4.25.1 +SNMP_VSN = 5.1 PRE_VSN = APP_VSN = "$(APPLICATION)-$(SNMP_VSN)$(PRE_VSN)" diff --git a/lib/ssh/doc/src/notes.xml b/lib/ssh/doc/src/notes.xml index 0d88cbda7a..0b587db810 100644 --- a/lib/ssh/doc/src/notes.xml +++ b/lib/ssh/doc/src/notes.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2004</year><year>2013</year> + <year>2004</year><year>2014</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -29,6 +29,289 @@ <file>notes.xml</file> </header> +<section><title>Ssh 3.0.5</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + When starting an ssh-daemon giving the option + {parallel_login, true}, the timeout for authentication + negotiation ({negotiation_timeout, integer()}) was never + removed.</p> + <p> + This caused the session to always be terminated after the + timeout if parallel_login was set.</p> + <p> + Own Id: OTP-12057 Aux Id: seq12663 </p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Warning: this is experimental and may disappear or change + without previous warning.</p> + <p> + Experimental support for running Quickcheck and PropEr + tests from common_test suites is added to common_test. + See the reference manual for the new module + <c>ct_property_testing</c>.</p> + <p> + Experimental property tests are added under + <c>lib/{inet,ssh}/test/property_test</c>. They can be run + directly or from the commont_test suites + <c>inet/ftp_property_test_SUITE.erl</c> and + <c>ssh/test/ssh_property_test_SUITE.erl</c>.</p> + <p> + See the code in the <c>test</c> directories and the man + page for details.</p> + <p> + (Thanks to Tuncer Ayaz for a patch adding Triq)</p> + <p> + Own Id: OTP-12119</p> + </item> + </list> + </section> + +</section> + +<section><title>Ssh 3.0.4</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + When starting an ssh-daemon giving the option + {parallel_login, true}, the timeout for authentication + negotiation ({negotiation_timeout, integer()}) was never + removed.</p> + <p> + This caused the session to always be terminated after the + timeout if parallel_login was set.</p> + <p> + Own Id: OTP-12057 Aux Id: seq12663 </p> + </item> + </list> + </section> + +</section> + +<section><title>Ssh 3.0.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Removed mail address from error reports and corrected + spelling error (Stacktace -> stacktrace)</p> + <p> + Own Id: OTP-11883 Aux Id: seq12586 </p> + </item> + <item> + <p> + Decode/encode fixes in SSH_MSG_IGNORE and + SSH_MSG_UNIMPLEMENTED.</p> + <p> + Own Id: OTP-11983</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Accepts that some older OpenSSH clients sends incorrect + disconnect messages.</p> + <p> + Own Id: OTP-11972</p> + </item> + <item> + <p> + Handle inet and inet6 option correctly</p> + <p> + Own Id: OTP-11976</p> + </item> + </list> + </section> + +</section> + +<section><title>Ssh 3.0.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fixed timeout bug in ssh:connect.</p> + <p> + Own Id: OTP-11908</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Option <c>max_sessions</c> added to + <c>ssh:daemon/{2,3}</c>. This option, if set, limits the + number of simultaneous connections accepted by the + daemon.</p> + <p> + Own Id: OTP-11885</p> + </item> + </list> + </section> + +</section> + +<section><title>Ssh 3.0.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fixes the problem that ssh_cli in some cases could delay + the prompt if a tty was not requested by the client.</p> + <p> + Own Id: OTP-10732</p> + </item> + <item> + <p> + The variable NewCol is now correctly calculated allowing + for tab-completion of function calls even when preceded + with blank space (Thanks to Alexander Demidenko)</p> + <p> + Own Id: OTP-11566</p> + </item> + <item> + <p> + Fix incorrect dialyzer spec and types, also enhance + documentation. </p> + <p> + Thanks to Ayaz Tuncer.</p> + <p> + Own Id: OTP-11627</p> + </item> + <item> + <p> + Fixed a bug when ssh:exec executes a linux command on a + linux ssh daemon. If the result is sent back from + standard error, the length information was not stripped + off correctly.</p> + <p> + Own Id: OTP-11667</p> + </item> + <item> + <p> + Fixed a bug with the ssh file 'known_hosts' which made + the file grow with many equal entries.</p> + <p> + Own Id: OTP-11671</p> + </item> + <item> + <p> + Some local implementations of removing the last element + from a list are replaced by <c>lists:droplast/1</c>. Note + that this requires at least <c>stdlib-2.0</c>, which is + the stdlib version delivered in OTP 17.0. (Thanks to Hans + Svensson)</p> + <p> + Own Id: OTP-11678</p> + </item> + <item> + <p> + Bug fix for <c>ssh:daemon/2,3</c> so that the failfun is + called when it should.</p> + <p> + Own Id: OTP-11680</p> + </item> + <item> + <p> + Fixed bug which crashed ssh when SSH_MSG_KEX_DH_GEX_GROUP + is received. This could cause a vm-crash for eheap_alloc + during garbage collect.</p> + <p> + Own Id: OTP-11696 Aux Id: 12547, 12532 </p> + </item> + <item> + <p> + Fixes a bug that breaks keyboard-interactive + authentication. Thanks to Simon Cornish for reporting and + suggesting a fix.</p> + <p> + Own Id: OTP-11698</p> + </item> + <item> + <p> + dialyzer specs are now correct for <c>ssh:start/0</c>, + <c>ssh:start/1</c>, <c>ssh:stop/0</c> and + <c>ssh_connection_handler:open_channel/5</c>. (Thanks to + Johannes Weißl )</p> + <p> + Own Id: OTP-11705</p> + </item> + <item> + <p> + Application upgrade (appup) files are corrected for the + following applications: </p> + <p> + <c>asn1, common_test, compiler, crypto, debugger, + dialyzer, edoc, eldap, erl_docgen, et, eunit, gs, hipe, + inets, observer, odbc, os_mon, otp_mibs, parsetools, + percept, public_key, reltool, runtime_tools, ssh, + syntax_tools, test_server, tools, typer, webtool, wx, + xmerl</c></p> + <p> + A new test utility for testing appup files is added to + test_server. This is now used by most applications in + OTP.</p> + <p> + (Thanks to Tobias Schlager)</p> + <p> + Own Id: OTP-11744</p> + </item> + <item> + <p> + Fixed dialyzer warning for <c>ssh_connection:send</c>.</p> + <p> + Own Id: OTP-11821</p> + </item> + <item> + <p> + <c>ssh:daemon/2,3</c> : Added options + <c>negotiation_timeout</c> and <c>parallel_login</c> to + tune the authentication behaviour.</p> + <p> + Own Id: OTP-11823</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Ssh now fully supports unicode filenames, filecontents, + shell and cli. Please note that the underlying os and + emulator must also give support for unicode. You may want + to start the emulator with "<c>erl +fnu</c>" on Linux.</p> + <p> + Own Id: OTP-10953</p> + </item> + </list> + </section> + +</section> + <section><title>Ssh 3.0</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/ssh/doc/src/ssh.xml b/lib/ssh/doc/src/ssh.xml index eaf96d0230..9f5d1c003d 100644 --- a/lib/ssh/doc/src/ssh.xml +++ b/lib/ssh/doc/src/ssh.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>2004</year><year>2013</year> + <year>2004</year><year>2014</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -36,8 +36,8 @@ <list type="bulleted"> <item>SSH requires the crypto and public_key applications.</item> <item>Supported SSH version is 2.0 </item> - <item>Supported MAC algorithms: hmac-sha1</item> - <item>Supported encryption algorithms: aes128-cb and 3des-cbc</item> + <item>Supported MAC algorithms: hmac-sha2-256 and hmac-sha1</item> + <item>Supported encryption algorithms: aes128-ctr, aes128-cb and 3des-cbc</item> <item>Supports unicode filenames if the emulator and the underlaying OS supports it. See the DESCRIPTION section in <seealso marker="kernel:file">file</seealso> for information about this subject</item> <item>Supports unicode in shell and cli</item> </list> @@ -97,6 +97,8 @@ <seealso marker="ssh_connection#session_channel/2">ssh_connection:session_channel/[2, 4]</seealso>.</p> <p>Options are:</p> <taglist> + <tag><c><![CDATA[{inet, inet | inet6}]]></c></tag> + <item> IP version to use.</item> <tag><c><![CDATA[{user_dir, string()}]]></c></tag> <item> <p>Sets the user directory i.e. the directory containing @@ -230,6 +232,8 @@ port.</p> <p>Options are:</p> <taglist> + <tag><c><![CDATA[{inet, inet | inet6}]]></c></tag> + <item> IP version to use when the host address is specified as <c>any</c>. </item> <tag><c><![CDATA[{subsystems, [subsystem_spec()]]]></c></tag> <item> Provides specifications for handling of subsystems. The @@ -304,6 +308,36 @@ <c><![CDATA[true]]></c> if the password is valid and <c><![CDATA[false]]></c> otherwise.</p> </item> + + <tag><c><![CDATA[{negotiation_timeout, integer()}]]></c></tag> + <item> + <p>Max time in milliseconds for the authentication negotiation. The default value is 2 minutes. If the client fails to login within this time, the connection is closed. + </p> + </item> + + <tag><c><![CDATA[{max_sessions, pos_integer()}]]></c></tag> + <item> + <p>The maximum number of simultaneous sessions that are accepted at any time for this daemon. This includes sessions that are being authorized. So if set to <c>N</c>, and <c>N</c> clients have connected but not started the login process, the <c>N+1</c> connection attempt will be aborted. If <c>N</c> connections are authenticated and still logged in, no more loggins will be accepted until one of the existing ones log out. + </p> + <p>The counter is per listening port, so if two daemons are started, one with <c>{max_sessions,N}</c> and the other with <c>{max_sessions,M}</c> there will be in total <c>N+M</c> connections accepted for the whole ssh application. + </p> + <p>Note that if <c>parallel_login</c> is <c>false</c>, only one client at a time may be in the authentication phase. + </p> + <p>As default, the option is not set. This means that the number is not limited. + </p> + </item> + + <tag><c><![CDATA[{parallel_login, boolean()}]]></c></tag> + <item> + <p>If set to false (the default value), only one login is handled a time. If set to true, an unlimited number of login attempts will be allowed simultanously. + </p> + <p>If the <c>max_sessions</c> option is set to <c>N</c> and <c>parallel_login</c> is set to <c>true</c>, the max number of simultaneous login attempts at any time is limited to <c>N-K</c> where <c>K</c> is the number of authenticated connections present at this daemon. + </p> + <warning> + <p>Do not enable <c>parallel_logins</c> without protecting the server by other means, for example the <c>max_sessions</c> option or a firewall configuration. If set to <c>true</c>, there is no protection against DOS attacks.</p> + </warning> + </item> + <tag><c><![CDATA[{key_cb, atom()}]]></c></tag> <item> <p>Module implementing the behaviour <seealso marker="ssh_server_key_api">ssh_server_key_api</seealso>. diff --git a/lib/ssh/doc/src/using_ssh.xml b/lib/ssh/doc/src/using_ssh.xml index 4d73366f5e..9ab71260d3 100644 --- a/lib/ssh/doc/src/using_ssh.xml +++ b/lib/ssh/doc/src/using_ssh.xml @@ -33,7 +33,7 @@ all needed applications (crypto, public_key and ssh). All examples are run in an Erlang shell, or in a bash shell using openssh to illustrate how the erlang ssh application can be used. The - exampels are run as the user otptest on a local network where the + examples are run as the user otptest on a local network where the user is authorized to login in over ssh to the host "tarlop". If nothing else is stated it is persumed that the otptest user has an entry in tarlop's authorized_keys file (may log in via ssh without @@ -88,7 +88,7 @@ [...] </code> - <p>Create the file /tmp/otptest_user/.ssh/authrized_keys and add the content + <p>Create the file /tmp/otptest_user/.ssh/authorized_keys and add the content of /tmp/otptest_user/.ssh/id_rsa.pub Now we can do</p> <code type="erl"> diff --git a/lib/ssh/src/ssh.appup.src b/lib/ssh/src/ssh.appup.src index 1917c95f5a..8269f89e40 100644 --- a/lib/ssh/src/ssh.appup.src +++ b/lib/ssh/src/ssh.appup.src @@ -19,9 +19,25 @@ {"%VSN%", [ + {"3.0.2", [{load_module, ssh_message, soft_purge, soft_purge, []}, + {load_module, ssh_connection_handler, soft_purge, soft_purge, []}, + {load_module, ssh_io, soft_purge, soft_purge, []}]}, + {"3.0.1", [{load_module, ssh, soft_purge, soft_purge, []}, + {load_module, ssh_acceptor, soft_purge, soft_purge, []}, + {load_module, ssh_message, soft_purge, soft_purge, []}, + {load_module, ssh_connection_handler, soft_purge, soft_purge, []}, + {load_module, ssh_io, soft_purge, soft_purge, []}]}, {<<".*">>, [{restart_application, ssh}]} ], [ + {"3.0.2", [{load_module, ssh_message, soft_purge, soft_purge, []}, + {load_module, ssh_connection_handler, soft_purge, soft_purge, []}, + {load_module, ssh_io, soft_purge, soft_purge, []}]}, + {"3.0.1", [{load_module, ssh, soft_purge, soft_purge, []}, + {load_module, ssh_acceptor, soft_purge, soft_purge, []}, + {load_module, ssh_message, soft_purge, soft_purge, []}, + {load_module, ssh_connection_handler, soft_purge, soft_purge, []}, + {load_module, ssh_io, soft_purge, soft_purge, []}]}, {<<".*">>, [{restart_application, ssh}]} ] }. diff --git a/lib/ssh/src/ssh.erl b/lib/ssh/src/ssh.erl index d50d5a0cb3..8a8d4bb89e 100644 --- a/lib/ssh/src/ssh.erl +++ b/lib/ssh/src/ssh.erl @@ -1,7 +1,7 @@ -%% +% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2013. All Rights Reserved. +%% Copyright Ericsson AB 2004-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -73,8 +73,8 @@ connect(Host, Port, Options, Timeout) -> {SocketOptions, SshOptions} -> {_, Transport, _} = TransportOpts = proplists:get_value(transport, Options, {tcp, gen_tcp, tcp_closed}), - Inet = proplists:get_value(inet, SshOptions, inet), - try Transport:connect(Host, Port, [ {active, false}, Inet | SocketOptions], Timeout) of + ConnectionTimeout = proplists:get_value(connect_timeout, Options, infinity), + try Transport:connect(Host, Port, [ {active, false} | SocketOptions], ConnectionTimeout) of {ok, Socket} -> Opts = [{user_pid, self()}, {host, Host} | fix_idle_time(SshOptions)], ssh_connection_handler:start_connection(client, Socket, Opts, Timeout); @@ -255,8 +255,8 @@ do_start_daemon(Host, Port, Options, SocketOptions) -> handle_options(Opts) -> try handle_option(proplists:unfold(Opts), [], []) of - {_,_} = Options -> - Options + {Inet, Ssh} -> + {handle_ip(Inet), Ssh} catch throw:Error -> Error @@ -332,6 +332,14 @@ handle_option([{idle_time, _} = Opt | Rest], SocketOptions, SshOptions) -> handle_option(Rest, SocketOptions, [handle_ssh_option(Opt) | SshOptions]); handle_option([{rekey_limit, _} = Opt|Rest], SocketOptions, SshOptions) -> handle_option(Rest, SocketOptions, [handle_ssh_option(Opt) | SshOptions]); +handle_option([{max_sessions, _} = Opt|Rest], SocketOptions, SshOptions) -> + handle_option(Rest, SocketOptions, [handle_ssh_option(Opt) | SshOptions]); +handle_option([{negotiation_timeout, _} = Opt|Rest], SocketOptions, SshOptions) -> + handle_option(Rest, SocketOptions, [handle_ssh_option(Opt) | SshOptions]); +handle_option([{parallel_login, _} = Opt|Rest], SocketOptions, SshOptions) -> + handle_option(Rest, SocketOptions, [handle_ssh_option(Opt) | SshOptions]); +handle_option([parallel_login|Rest], SocketOptions, SshOptions) -> + handle_option(Rest, SocketOptions, [handle_ssh_option({parallel_login,true}) | SshOptions]); handle_option([Opt | Rest], SocketOptions, SshOptions) -> handle_option(Rest, [handle_inet_option(Opt) | SocketOptions], SshOptions). @@ -360,6 +368,12 @@ handle_ssh_option({pref_public_key_algs, Value} = Opt) when is_list(Value), leng end; handle_ssh_option({connect_timeout, Value} = Opt) when is_integer(Value); Value == infinity -> Opt; +handle_ssh_option({max_sessions, Value} = Opt) when is_integer(Value), Value>0 -> + Opt; +handle_ssh_option({negotiation_timeout, Value} = Opt) when is_integer(Value); Value == infinity -> + Opt; +handle_ssh_option({parallel_login, Value} = Opt) when Value==true ; Value==false -> + Opt; handle_ssh_option({user, Value} = Opt) when is_list(Value) -> Opt; handle_ssh_option({dsa_pass_phrase, Value} = Opt) when is_list(Value) -> @@ -378,7 +392,8 @@ handle_ssh_option({compression, Value} = Opt) when is_atom(Value) -> Opt; handle_ssh_option({exec, {Module, Function, _}} = Opt) when is_atom(Module), is_atom(Function) -> - + Opt; +handle_ssh_option({exec, Function} = Opt) when is_function(Function) -> Opt; handle_ssh_option({auth_methods, Value} = Opt) when is_list(Value) -> Opt; @@ -418,13 +433,14 @@ handle_ssh_option(Opt) -> throw({error, {eoptions, Opt}}). handle_inet_option({active, _} = Opt) -> - throw({error, {{eoptions, Opt}, "Ssh has built in flow control, " - "and activ is handled internaly user is not allowd" + throw({error, {{eoptions, Opt}, "SSH has built in flow control, " + "and active is handled internally, user is not allowed" "to specify this option"}}); -handle_inet_option({inet, Value} = Opt) when (Value == inet) or (Value == inet6) -> - Opt; + +handle_inet_option({inet, Value}) when (Value == inet) or (Value == inet6) -> + Value; handle_inet_option({reuseaddr, _} = Opt) -> - throw({error, {{eoptions, Opt},"Is set internaly user is not allowd" + throw({error, {{eoptions, Opt},"Is set internally, user is not allowed" "to specify this option"}}); %% Option verified by inet handle_inet_option(Opt) -> @@ -445,3 +461,17 @@ handle_pref_algs([H|T], Acc) -> _ -> false end. + +handle_ip(Inet) -> %% Default to ipv4 + case lists:member(inet, Inet) of + true -> + Inet; + false -> + case lists:member(inet6, Inet) of + true -> + Inet; + false -> + [inet | Inet] + end + end. + diff --git a/lib/ssh/src/ssh_acceptor.erl b/lib/ssh/src/ssh_acceptor.erl index 91905b2eaf..7302196674 100644 --- a/lib/ssh/src/ssh_acceptor.erl +++ b/lib/ssh/src/ssh_acceptor.erl @@ -80,15 +80,36 @@ acceptor_loop(Callback, Port, Address, Opts, ListenSocket, AcceptTimeout) -> ListenSocket, AcceptTimeout) end. -handle_connection(_Callback, Address, Port, Options, Socket) -> +handle_connection(Callback, Address, Port, Options, Socket) -> SystemSup = ssh_system_sup:system_supervisor(Address, Port), - {ok, SubSysSup} = ssh_system_sup:start_subsystem(SystemSup, Options), - ConnectionSup = ssh_subsystem_sup:connection_supervisor(SubSysSup), - ssh_connection_handler:start_connection(server, Socket, - [{supervisors, [{system_sup, SystemSup}, - {subsystem_sup, SubSysSup}, - {connection_sup, ConnectionSup}]} - | Options], infinity). + SSHopts = proplists:get_value(ssh_opts, Options, []), + MaxSessions = proplists:get_value(max_sessions,SSHopts,infinity), + case number_of_connections(SystemSup) < MaxSessions of + true -> + {ok, SubSysSup} = ssh_system_sup:start_subsystem(SystemSup, Options), + ConnectionSup = ssh_subsystem_sup:connection_supervisor(SubSysSup), + Timeout = proplists:get_value(negotiation_timeout, SSHopts, 2*60*1000), + ssh_connection_handler:start_connection(server, Socket, + [{supervisors, [{system_sup, SystemSup}, + {subsystem_sup, SubSysSup}, + {connection_sup, ConnectionSup}]} + | Options], Timeout); + false -> + Callback:close(Socket), + IPstr = if is_tuple(Address) -> inet:ntoa(Address); + true -> Address + end, + Str = try io_lib:format('~s:~p',[IPstr,Port]) + catch _:_ -> "port "++integer_to_list(Port) + end, + error_logger:info_report("Ssh login attempt to "++Str++" denied due to option " + "max_sessions limits to "++ io_lib:write(MaxSessions) ++ + " sessions." + ), + {error,max_sessions} + end. + + handle_error(timeout) -> ok; @@ -114,3 +135,10 @@ handle_error(Reason) -> String = lists:flatten(io_lib:format("Accept error: ~p", [Reason])), error_logger:error_report(String), exit({accept_failed, String}). + + +number_of_connections(SystemSup) -> + length([X || + {R,X,supervisor,[ssh_subsystem_sup]} <- supervisor:which_children(SystemSup), + is_reference(R) + ]). diff --git a/lib/ssh/src/ssh_auth.erl b/lib/ssh/src/ssh_auth.erl index 409a1db6d5..45fd907383 100644 --- a/lib/ssh/src/ssh_auth.erl +++ b/lib/ssh/src/ssh_auth.erl @@ -196,7 +196,7 @@ handle_userauth_request(#ssh_msg_userauth_request{user = User, {authorized, User, ssh_transport:ssh_packet(#ssh_msg_userauth_success{}, Ssh)}; false -> - {not_authorized, {User, {passwd, Password}}, + {not_authorized, {User, {error,"Bad user or password"}}, ssh_transport:ssh_packet(#ssh_msg_userauth_failure{ authentications = "", partial_success = false}, Ssh)} @@ -228,7 +228,7 @@ handle_userauth_request(#ssh_msg_userauth_request{user = User, ssh_transport:ssh_packet( #ssh_msg_userauth_success{}, Ssh)}; false -> - {not_authorized, {User, {error, "Invalid signature"}}, + {not_authorized, {User, undefined}, ssh_transport:ssh_packet(#ssh_msg_userauth_failure{ authentications="publickey,password", partial_success = false}, Ssh)} diff --git a/lib/ssh/src/ssh_cli.erl b/lib/ssh/src/ssh_cli.erl index 77453e8fd7..18841e3d2d 100644 --- a/lib/ssh/src/ssh_cli.erl +++ b/lib/ssh/src/ssh_cli.erl @@ -457,17 +457,17 @@ bin_to_list(I) when is_integer(I) -> start_shell(ConnectionHandler, State) -> Shell = State#state.shell, - ConnectionInfo = ssh_connection_handler:info(ConnectionHandler, + ConnectionInfo = ssh_connection_handler:connection_info(ConnectionHandler, [peer, user]), ShellFun = case is_function(Shell) of true -> - {ok, User} = + User = proplists:get_value(user, ConnectionInfo), case erlang:fun_info(Shell, arity) of {arity, 1} -> fun() -> Shell(User) end; {arity, 2} -> - [{_, PeerAddr}] = + {_, PeerAddr} = proplists:get_value(peer, ConnectionInfo), fun() -> Shell(User, PeerAddr) end; _ -> @@ -485,9 +485,9 @@ start_shell(_ConnectionHandler, Cmd, #state{exec={M, F, A}} = State) -> State#state{group = Group, buf = empty_buf()}; start_shell(ConnectionHandler, Cmd, #state{exec=Shell} = State) when is_function(Shell) -> - ConnectionInfo = ssh_connection_handler:info(ConnectionHandler, + ConnectionInfo = ssh_connection_handler:connection_info(ConnectionHandler, [peer, user]), - {ok, User} = + User = proplists:get_value(user, ConnectionInfo), ShellFun = case erlang:fun_info(Shell, arity) of @@ -496,7 +496,7 @@ start_shell(ConnectionHandler, Cmd, #state{exec=Shell} = State) when is_function {arity, 2} -> fun() -> Shell(Cmd, User) end; {arity, 3} -> - [{_, PeerAddr}] = + {_, PeerAddr} = proplists:get_value(peer, ConnectionInfo), fun() -> Shell(Cmd, User, PeerAddr) end; _ -> diff --git a/lib/ssh/src/ssh_connection.erl b/lib/ssh/src/ssh_connection.erl index b377614949..33849f4527 100644 --- a/lib/ssh/src/ssh_connection.erl +++ b/lib/ssh/src/ssh_connection.erl @@ -782,9 +782,8 @@ handle_cli_msg(#connection{channel_cache = Cache} = Connection, erlang:monitor(process, Pid), Channel = Channel0#channel{user = Pid}, ssh_channel:cache_update(Cache, Channel), - Reply = {connection_reply, - channel_success_msg(RemoteId)}, - {{replies, [{channel_data, Pid, Reply0}, Reply]}, Connection}; + {Reply, Connection1} = reply_msg(Channel, Connection, Reply0), + {{replies, [Reply]}, Connection1}; _Other -> Reply = {connection_reply, channel_failure_msg(RemoteId)}, diff --git a/lib/ssh/src/ssh_connection_handler.erl b/lib/ssh/src/ssh_connection_handler.erl index d7fff14f92..86804c4436 100644 --- a/lib/ssh/src/ssh_connection_handler.erl +++ b/lib/ssh/src/ssh_connection_handler.erl @@ -104,14 +104,12 @@ start_connection(client = Role, Socket, Options, Timeout) -> start_connection(server = Role, Socket, Options, Timeout) -> try - Sups = proplists:get_value(supervisors, Options), - ConnectionSup = proplists:get_value(connection_sup, Sups), - Opts = [{supervisors, Sups}, {user_pid, self()} | proplists:get_value(ssh_opts, Options, [])], - {ok, Pid} = ssh_connection_sup:start_child(ConnectionSup, [Role, Socket, Opts]), - {_, Callback, _} = proplists:get_value(transport, Options, {tcp, gen_tcp, tcp_closed}), - socket_control(Socket, Pid, Callback), - Ref = erlang:monitor(process, Pid), - handshake(Pid, Ref, Timeout) + case proplists:get_value(parallel_login, Options, false) of + true -> + spawn(fun() -> start_server_connection(Role, Socket, Options, Timeout) end); + false -> + start_server_connection(Role, Socket, Options, Timeout) + end catch exit:{noproc, _} -> {error, ssh_not_started}; @@ -119,6 +117,18 @@ start_connection(server = Role, Socket, Options, Timeout) -> {error, Error} end. + +start_server_connection(server = Role, Socket, Options, Timeout) -> + Sups = proplists:get_value(supervisors, Options), + ConnectionSup = proplists:get_value(connection_sup, Sups), + Opts = [{supervisors, Sups}, {user_pid, self()} | proplists:get_value(ssh_opts, Options, [])], + {ok, Pid} = ssh_connection_sup:start_child(ConnectionSup, [Role, Socket, Opts]), + {_, Callback, _} = proplists:get_value(transport, Options, {tcp, gen_tcp, tcp_closed}), + socket_control(Socket, Pid, Callback), + Ref = erlang:monitor(process, Pid), + handshake(Pid, Ref, Timeout). + + start_link(Role, Socket, Options) -> {ok, proc_lib:spawn_link(?MODULE, init, [[Role, Socket, Options]])}. @@ -1474,8 +1484,7 @@ ssh_channel_info([ _ | Rest], Channel, Acc) -> log_error(Reason) -> Report = io_lib:format("Erlang ssh connection handler failed with reason: " - "~p ~n, Stacktace: ~p ~n" - "please report this to [email protected] \n", + "~p ~n, Stacktrace: ~p ~n", [Reason, erlang:get_stacktrace()]), error_logger:error_report(Report), "Internal error". diff --git a/lib/ssh/src/ssh_io.erl b/lib/ssh/src/ssh_io.erl index 832b144db9..35336bce8b 100644 --- a/lib/ssh/src/ssh_io.erl +++ b/lib/ssh/src/ssh_io.erl @@ -81,6 +81,8 @@ format(Fmt, Args) -> trim(Line) when is_list(Line) -> lists:reverse(trim1(lists:reverse(trim1(Line)))); +trim(Line) when is_binary(Line) -> + trim(unicode:characters_to_list(Line)); trim(Other) -> Other. trim1([$\s|Cs]) -> trim(Cs); diff --git a/lib/ssh/src/ssh_message.erl b/lib/ssh/src/ssh_message.erl index 8d6c77c0ed..76b57cb995 100644 --- a/lib/ssh/src/ssh_message.erl +++ b/lib/ssh/src/ssh_message.erl @@ -255,7 +255,7 @@ encode(#ssh_msg_ignore{data = Data}) -> ssh_bits:encode([?SSH_MSG_IGNORE, Data], [byte, string]); encode(#ssh_msg_unimplemented{sequence = Seq}) -> - ssh_bits:encode([?SSH_MSG_IGNORE, Seq], [byte, uint32]); + ssh_bits:encode([?SSH_MSG_UNIMPLEMENTED, Seq], [byte, uint32]); encode(#ssh_msg_debug{always_display = Bool, message = Msg, @@ -391,13 +391,6 @@ decode(<<?BYTE(?SSH_MSG_USERAUTH_INFO_REQUEST), ?UINT32(Len0), Name:Len0/binary, data = Data}; %%% Unhandled message, also masked by same 1:st byte value as ?SSH_MSG_USERAUTH_INFO_REQUEST: -decode(<<?BYTE(?SSH_MSG_USERAUTH_PK_OK), ?UINT32(Len), Alg:Len/binary, KeyBlob/binary>>) -> - #ssh_msg_userauth_pk_ok{ - algorithm_name = Alg, - key_blob = KeyBlob - }; - -%%% Unhandled message, also masked by same 1:st byte value as ?SSH_MSG_USERAUTH_INFO_REQUEST: decode(<<?BYTE(?SSH_MSG_USERAUTH_PASSWD_CHANGEREQ), ?UINT32(Len0), Prompt:Len0/binary, ?UINT32(Len1), Lang:Len1/binary>>) -> #ssh_msg_userauth_passwd_changereq{ @@ -405,6 +398,13 @@ decode(<<?BYTE(?SSH_MSG_USERAUTH_PASSWD_CHANGEREQ), ?UINT32(Len0), Prompt:Len0/b languge = Lang }; +%%% Unhandled message, also masked by same 1:st byte value as ?SSH_MSG_USERAUTH_INFO_REQUEST: +decode(<<?BYTE(?SSH_MSG_USERAUTH_PK_OK), ?UINT32(Len), Alg:Len/binary, KeyBlob/binary>>) -> + #ssh_msg_userauth_pk_ok{ + algorithm_name = Alg, + key_blob = KeyBlob + }; + decode(<<?BYTE(?SSH_MSG_USERAUTH_INFO_RESPONSE), ?UINT32(Num), Data/binary>>) -> #ssh_msg_userauth_info_response{ num_responses = Num, @@ -461,10 +461,19 @@ decode(<<?BYTE(?SSH_MSG_DISCONNECT), ?UINT32(Code), language = Lang }; +%% Accept bad disconnects from ancient openssh clients that doesn't send language tag. Use english as a work-around. +decode(<<?BYTE(?SSH_MSG_DISCONNECT), ?UINT32(Code), + ?UINT32(Len0), Desc:Len0/binary>>) -> + #ssh_msg_disconnect{ + code = Code, + description = unicode:characters_to_list(Desc), + language = <<"en">> + }; + decode(<<?SSH_MSG_NEWKEYS>>) -> #ssh_msg_newkeys{}; -decode(<<?BYTE(?SSH_MSG_IGNORE), Data/binary>>) -> +decode(<<?BYTE(?SSH_MSG_IGNORE), ?UINT32(Len), Data:Len/binary>>) -> #ssh_msg_ignore{data = Data}; decode(<<?BYTE(?SSH_MSG_UNIMPLEMENTED), ?UINT32(Seq)>>) -> diff --git a/lib/ssh/src/ssh_transport.erl b/lib/ssh/src/ssh_transport.erl index 27723dc870..ea05c849b7 100644 --- a/lib/ssh/src/ssh_transport.erl +++ b/lib/ssh/src/ssh_transport.erl @@ -113,15 +113,28 @@ key_init(client, Ssh, Value) -> key_init(server, Ssh, Value) -> Ssh#ssh{s_keyinit = Value}. +available_ssh_algos() -> + Supports = crypto:supports(), + CipherAlgos = [{aes_ctr, "aes128-ctr"}, {aes_cbc128, "aes128-cbc"}, {des3_cbc, "3des-cbc"}], + Ciphers = [SshAlgo || + {CryptoAlgo, SshAlgo} <- CipherAlgos, + lists:member(CryptoAlgo, proplists:get_value(ciphers, Supports, []))], + HashAlgos = [{sha256, "hmac-sha2-256"}, {sha, "hmac-sha1"}], + Hashs = [SshAlgo || + {CryptoAlgo, SshAlgo} <- HashAlgos, + lists:member(CryptoAlgo, proplists:get_value(hashs, Supports, []))], + {Ciphers, Hashs}. + kexinit_messsage(client, Random, Compression, HostKeyAlgs) -> + {CipherAlgs, HashAlgs} = available_ssh_algos(), #ssh_msg_kexinit{ cookie = Random, kex_algorithms = ["diffie-hellman-group1-sha1"], server_host_key_algorithms = HostKeyAlgs, - encryption_algorithms_client_to_server = ["aes128-cbc","3des-cbc"], - encryption_algorithms_server_to_client = ["aes128-cbc","3des-cbc"], - mac_algorithms_client_to_server = ["hmac-sha1"], - mac_algorithms_server_to_client = ["hmac-sha1"], + encryption_algorithms_client_to_server = CipherAlgs, + encryption_algorithms_server_to_client = CipherAlgs, + mac_algorithms_client_to_server = HashAlgs, + mac_algorithms_server_to_client = HashAlgs, compression_algorithms_client_to_server = Compression, compression_algorithms_server_to_client = Compression, languages_client_to_server = [], @@ -129,14 +142,15 @@ kexinit_messsage(client, Random, Compression, HostKeyAlgs) -> }; kexinit_messsage(server, Random, Compression, HostKeyAlgs) -> + {CipherAlgs, HashAlgs} = available_ssh_algos(), #ssh_msg_kexinit{ cookie = Random, kex_algorithms = ["diffie-hellman-group1-sha1"], server_host_key_algorithms = HostKeyAlgs, - encryption_algorithms_client_to_server = ["aes128-cbc","3des-cbc"], - encryption_algorithms_server_to_client = ["aes128-cbc","3des-cbc"], - mac_algorithms_client_to_server = ["hmac-sha1"], - mac_algorithms_server_to_client = ["hmac-sha1"], + encryption_algorithms_client_to_server = CipherAlgs, + encryption_algorithms_server_to_client = CipherAlgs, + mac_algorithms_client_to_server = HashAlgs, + mac_algorithms_server_to_client = HashAlgs, compression_algorithms_client_to_server = Compression, compression_algorithms_server_to_client = Compression, languages_client_to_server = [], @@ -636,7 +650,21 @@ encrypt_init(#ssh{encrypt = 'aes128-cbc', role = server} = Ssh) -> <<K:16/binary>> = hash(Ssh, "D", 128), {ok, Ssh#ssh{encrypt_keys = K, encrypt_block_size = 16, - encrypt_ctx = IV}}. + encrypt_ctx = IV}}; +encrypt_init(#ssh{encrypt = 'aes128-ctr', role = client} = Ssh) -> + IV = hash(Ssh, "A", 128), + <<K:16/binary>> = hash(Ssh, "C", 128), + State = crypto:stream_init(aes_ctr, K, IV), + {ok, Ssh#ssh{encrypt_keys = K, + encrypt_block_size = 16, + encrypt_ctx = State}}; +encrypt_init(#ssh{encrypt = 'aes128-ctr', role = server} = Ssh) -> + IV = hash(Ssh, "B", 128), + <<K:16/binary>> = hash(Ssh, "D", 128), + State = crypto:stream_init(aes_ctr, K, IV), + {ok, Ssh#ssh{encrypt_keys = K, + encrypt_block_size = 16, + encrypt_ctx = State}}. encrypt_final(Ssh) -> {ok, Ssh#ssh{encrypt = none, @@ -658,7 +686,11 @@ encrypt(#ssh{encrypt = 'aes128-cbc', encrypt_ctx = IV0} = Ssh, Data) -> Enc = crypto:block_encrypt(aes_cbc128, K,IV0,Data), IV = crypto:next_iv(aes_cbc, Enc), - {Ssh#ssh{encrypt_ctx = IV}, Enc}. + {Ssh#ssh{encrypt_ctx = IV}, Enc}; +encrypt(#ssh{encrypt = 'aes128-ctr', + encrypt_ctx = State0} = Ssh, Data) -> + {State, Enc} = crypto:stream_encrypt(State0,Data), + {Ssh#ssh{encrypt_ctx = State}, Enc}. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -690,7 +722,21 @@ decrypt_init(#ssh{decrypt = 'aes128-cbc', role = server} = Ssh) -> hash(Ssh, "C", 128)}, <<K:16/binary>> = KD, {ok, Ssh#ssh{decrypt_keys = K, decrypt_ctx = IV, - decrypt_block_size = 16}}. + decrypt_block_size = 16}}; +decrypt_init(#ssh{decrypt = 'aes128-ctr', role = client} = Ssh) -> + IV = hash(Ssh, "B", 128), + <<K:16/binary>> = hash(Ssh, "D", 128), + State = crypto:stream_init(aes_ctr, K, IV), + {ok, Ssh#ssh{decrypt_keys = K, + decrypt_block_size = 16, + decrypt_ctx = State}}; +decrypt_init(#ssh{decrypt = 'aes128-ctr', role = server} = Ssh) -> + IV = hash(Ssh, "A", 128), + <<K:16/binary>> = hash(Ssh, "C", 128), + State = crypto:stream_init(aes_ctr, K, IV), + {ok, Ssh#ssh{decrypt_keys = K, + decrypt_block_size = 16, + decrypt_ctx = State}}. decrypt_final(Ssh) -> @@ -711,7 +757,11 @@ decrypt(#ssh{decrypt = 'aes128-cbc', decrypt_keys = Key, decrypt_ctx = IV0} = Ssh, Data) -> Dec = crypto:block_decrypt(aes_cbc128, Key,IV0,Data), IV = crypto:next_iv(aes_cbc, Data), - {Ssh#ssh{decrypt_ctx = IV}, Dec}. + {Ssh#ssh{decrypt_ctx = IV}, Dec}; +decrypt(#ssh{decrypt = 'aes128-ctr', + decrypt_ctx = State0} = Ssh, Data) -> + {State, Enc} = crypto:stream_decrypt(State0,Data), + {Ssh#ssh{decrypt_ctx = State}, Enc}. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% Compression @@ -846,7 +896,9 @@ mac('hmac-sha1-96', Key, SeqNum, Data) -> mac('hmac-md5', Key, SeqNum, Data) -> crypto:hmac(md5, Key, [<<?UINT32(SeqNum)>>, Data]); mac('hmac-md5-96', Key, SeqNum, Data) -> - crypto:hmac(md5, Key, [<<?UINT32(SeqNum)>>, Data], mac_digest_size('hmac-md5-96')). + crypto:hmac(md5, Key, [<<?UINT32(SeqNum)>>, Data], mac_digest_size('hmac-md5-96')); +mac('hmac-sha2-256', Key, SeqNum, Data) -> + crypto:hmac(sha256, Key, [<<?UINT32(SeqNum)>>, Data]). %% return N hash bytes (HASH) hash(SSH, Char, Bits) -> @@ -911,12 +963,14 @@ mac_key_size('hmac-sha1') -> 20*8; mac_key_size('hmac-sha1-96') -> 20*8; mac_key_size('hmac-md5') -> 16*8; mac_key_size('hmac-md5-96') -> 16*8; +mac_key_size('hmac-sha2-256')-> 32*8; mac_key_size(none) -> 0. mac_digest_size('hmac-sha1') -> 20; mac_digest_size('hmac-sha1-96') -> 12; mac_digest_size('hmac-md5') -> 20; mac_digest_size('hmac-md5-96') -> 12; +mac_digest_size('hmac-sha2-256') -> 32; mac_digest_size(none) -> 0. peer_name({Host, _}) -> diff --git a/lib/ssh/test/property_test/README b/lib/ssh/test/property_test/README new file mode 100644 index 0000000000..57602bf719 --- /dev/null +++ b/lib/ssh/test/property_test/README @@ -0,0 +1,12 @@ + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%%% %%% +%%% WARNING %%% +%%% %%% +%%% This is experimental code which may be changed or removed %%% +%%% anytime without any warning. %%% +%%% %%% +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +The test in this directory are written assuming that the user has a QuickCheck license. They are to be run manually. Some may be possible to be run with other tools, e.g. PropEr. + diff --git a/lib/ssh/test/property_test/ssh_eqc_client_server.erl b/lib/ssh/test/property_test/ssh_eqc_client_server.erl new file mode 100644 index 0000000000..cf895ae85e --- /dev/null +++ b/lib/ssh/test/property_test/ssh_eqc_client_server.erl @@ -0,0 +1,607 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2004-2014. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% + +-module(ssh_eqc_client_server). + +-compile(export_all). + +-include_lib("common_test/include/ct.hrl"). + +-ifdef(PROPER). +%% Proper is not supported. +-else. +-ifdef(TRIQ). +%% Proper is not supported. +-else. + + +-include_lib("eqc/include/eqc.hrl"). +-include_lib("eqc/include/eqc_statem.hrl"). +-eqc_group_commands(true). + +-define(SSH_DIR,"ssh_eqc_client_server_dirs"). + +-define(sec, *1000). +-define(min, *60?sec). + +-record(srvr,{ref, + address, + port + }). + +-record(conn,{ref, + srvr_ref + }). + +-record(chan, {ref, + conn_ref, + subsystem, + client_pid + }). + +-record(state,{ + initialized = false, + servers = [], % [#srvr{}] + clients = [], + connections = [], % [#conn{}] + channels = [], % [#chan{}] + data_dir + }). + +%%%=============================================================== +%%% +%%% Specification of addresses, subsystems and such. +%%% + +-define(MAX_NUM_SERVERS, 3). +-define(MAX_NUM_CLIENTS, 3). + +-define(SUBSYSTEMS, ["echo1", "echo2", "echo3", "echo4"]). + +-define(SERVER_ADDRESS, { {127,1,1,1}, inet_port({127,1,1,1}) }). + +-define(SERVER_EXTRA_OPTIONS, [{parallel_login,bool()}] ). + + +%%%================================================================ +%%% +%%% The properties - one sequantial and one parallel with the same model +%%% +%%% Run as +%%% +%%% $ (cd ..; make) +%%% $ erl -pz .. +%%% +%%% eqc:quickcheck( ssh_eqc_client_server:prop_seq() ). +%%% eqc:quickcheck( ssh_eqc_client_server:prop_parallel() ). +%%% eqc:quickcheck( ssh_eqc_client_server:prop_parallel_multi() ). +%%% + + +%% To be called as eqc:quickcheck( ssh_eqc_client_server:prop_seq() ). +prop_seq() -> + do_prop_seq(?SSH_DIR). + +%% To be called from a common_test test suite +prop_seq(CT_Config) -> + do_prop_seq(full_path(?SSH_DIR, CT_Config)). + + +do_prop_seq(DataDir) -> + ?FORALL(Cmds,commands(?MODULE, #state{data_dir=DataDir}), + begin + {H,Sf,Result} = run_commands(?MODULE,Cmds), + present_result(?MODULE, Cmds, {H,Sf,Result}, Result==ok) + end). + +full_path(SSHdir, CT_Config) -> + filename:join(proplists:get_value(property_dir, CT_Config), + SSHdir). +%%%---- +prop_parallel() -> + do_prop_parallel(?SSH_DIR). + +%% To be called from a common_test test suite +prop_parallel(CT_Config) -> + do_prop_parallel(full_path(?SSH_DIR, CT_Config)). + +do_prop_parallel(DataDir) -> + ?FORALL(Cmds,parallel_commands(?MODULE, #state{data_dir=DataDir}), + begin + {H,Sf,Result} = run_parallel_commands(?MODULE,Cmds), + present_result(?MODULE, Cmds, {H,Sf,Result}, Result==ok) + end). + +%%%---- +prop_parallel_multi() -> + do_prop_parallel_multi(?SSH_DIR). + +%% To be called from a common_test test suite +prop_parallel_multi(CT_Config) -> + do_prop_parallel_multi(full_path(?SSH_DIR, CT_Config)). + +do_prop_parallel_multi(DataDir) -> + ?FORALL(Repetitions,?SHRINK(1,[10]), + ?FORALL(Cmds,parallel_commands(?MODULE, #state{data_dir=DataDir}), + ?ALWAYS(Repetitions, + begin + {H,Sf,Result} = run_parallel_commands(?MODULE,Cmds), + present_result(?MODULE, Cmds, {H,Sf,Result}, Result==ok) + end))). + +%%%================================================================ +%%% State machine spec + +%%% called when using commands/1 +initial_state() -> + S = initial_state(#state{}), + S#state{initialized=true}. + +%%% called when using commands/2 +initial_state(S) -> + application:stop(ssh), + ssh:start(), + setup_rsa(S#state.data_dir). + +%%%---------------- +weight(S, ssh_send) -> 5*length([C || C<-S#state.channels, has_subsyst(C)]); +weight(S, ssh_start_subsyst) -> 3*length([C || C<-S#state.channels, no_subsyst(C)]); +weight(S, ssh_close_channel) -> 2*length([C || C<-S#state.channels, has_subsyst(C)]); +weight(S, ssh_open_channel) -> length(S#state.connections); +weight(_S, _) -> 1. + +%%%---------------- +%%% Initialize + +initial_state_pre(S) -> not S#state.initialized. + +initial_state_args(S) -> [S]. + +initial_state_next(S, _, _) -> S#state{initialized=true}. + +%%%---------------- +%%% Start a new daemon +%%% Precondition: not more than ?MAX_NUM_SERVERS started + +ssh_server_pre(S) -> S#state.initialized andalso + length(S#state.servers) < ?MAX_NUM_SERVERS. + +ssh_server_args(S) -> [?SERVER_ADDRESS, S#state.data_dir, ?SERVER_EXTRA_OPTIONS]. + +ssh_server({IP,Port}, DataDir, ExtraOptions) -> + ok(ssh:daemon(IP, Port, + [ + {system_dir, system_dir(DataDir)}, + {user_dir, user_dir(DataDir)}, + {subsystems, [{SS, {ssh_eqc_subsys, [SS]}} || SS <- ?SUBSYSTEMS]} + | ExtraOptions + ])). + +ssh_server_post(_S, _Args, Result) -> is_ok(Result). + +ssh_server_next(S, Result, [{IP,Port},_,_]) -> + S#state{servers=[#srvr{ref = Result, + address = IP, + port = Port} + | S#state.servers]}. + +%%%---------------- +%%% Start a new client +%%% Precondition: not more than ?MAX_NUM_CLIENTS started + +ssh_client_pre(S) -> S#state.initialized andalso + length(S#state.clients) < ?MAX_NUM_CLIENTS. + +ssh_client_args(_S) -> []. + +ssh_client() -> spawn(fun client_init/0). + +ssh_client_next(S, Pid, _) -> S#state{clients=[Pid|S#state.clients]}. + + +client_init() -> client_loop(). + +client_loop() -> + receive + {please_do,Fun,Ref,Pid} -> + Pid ! {my_pleasure, catch Fun(), Ref}, + client_loop() + end. + +do(Pid, Fun) -> do(Pid, Fun, 30?sec). + +do(Pid, Fun, Timeout) when is_function(Fun,0) -> + Pid ! {please_do,Fun,Ref=make_ref(),self()}, + receive + {my_pleasure, Result, Ref} -> Result + after + Timeout -> {error,do_timeout} + end. + +%%%---------------- +%%% Start a new connection +%%% Precondition: deamon exists + +ssh_open_connection_pre(S) -> S#state.servers /= []. + +ssh_open_connection_args(S) -> [oneof(S#state.servers), S#state.data_dir]. + +ssh_open_connection(#srvr{address=Ip, port=Port}, DataDir) -> + ok(ssh:connect(ensure_string(Ip), Port, + [ + {silently_accept_hosts, true}, + {user_dir, user_dir(DataDir)}, + {user_interaction, false} + ])). + +ssh_open_connection_post(_S, _Args, Result) -> is_ok(Result). + +ssh_open_connection_next(S, ConnRef, [#srvr{ref=SrvrRef},_]) -> + S#state{connections=[#conn{ref=ConnRef, srvr_ref=SrvrRef}|S#state.connections]}. + +%%%---------------- +%%% Stop a new connection +%%% Precondition: connection exists + +ssh_close_connection_pre(S) -> S#state.connections /= []. + +ssh_close_connection_args(S) -> [oneof(S#state.connections)]. + +ssh_close_connection(#conn{ref=ConnectionRef}) -> ssh:close(ConnectionRef). + +ssh_close_connection_next(S, _, [Conn=#conn{ref=ConnRef}]) -> + S#state{connections = S#state.connections--[Conn], + channels = [C || C <- S#state.channels, + C#chan.conn_ref /= ConnRef] + }. + +%%%---------------- +%%% Start a new channel without a sub system +%%% Precondition: connection exists + +ssh_open_channel_pre(S) -> S#state.connections /= []. + +ssh_open_channel_args(S) -> [oneof(S#state.connections)]. + +%%% For re-arrangement in parallel tests. +ssh_open_channel_pre(S,[C]) -> lists:member(C,S#state.connections). + +ssh_open_channel(#conn{ref=ConnectionRef}) -> + ok(ssh_connection:session_channel(ConnectionRef, 20?sec)). + +ssh_open_channel_post(_S, _Args, Result) -> is_ok(Result). + +ssh_open_channel_next(S, ChannelRef, [#conn{ref=ConnRef}]) -> + S#state{channels=[#chan{ref=ChannelRef, + conn_ref=ConnRef} + | S#state.channels]}. + +%%%---------------- +%%% Stop a channel +%%% Precondition: a channel exists, with or without a subsystem + +ssh_close_channel_pre(S) -> S#state.channels /= []. + +ssh_close_channel_args(S) -> [oneof(S#state.channels)]. + +ssh_close_channel(#chan{ref=ChannelRef, conn_ref=ConnectionRef}) -> + ssh_connection:close(ConnectionRef, ChannelRef). + +ssh_close_channel_next(S, _, [C]) -> + S#state{channels = [Ci || Ci <- S#state.channels, + sig(C) /= sig(Ci)]}. + + +sig(C) -> {C#chan.ref, C#chan.conn_ref}. + + +%%%---------------- +%%% Start a sub system on a channel +%%% Precondition: A channel without subsystem exists + +ssh_start_subsyst_pre(S) -> lists:any(fun no_subsyst/1, S#state.channels) andalso + S#state.clients /= []. + +ssh_start_subsyst_args(S) -> [oneof(lists:filter(fun no_subsyst/1, S#state.channels)), + oneof(?SUBSYSTEMS), + oneof(S#state.clients) + ]. + +%% For re-arrangement in parallel tests. +ssh_start_subsyst_pre(S, [C|_]) -> lists:member(C,S#state.channels) + andalso no_subsyst(C). + +ssh_start_subsyst(#chan{ref=ChannelRef, conn_ref=ConnectionRef}, SubSystem, Pid) -> + do(Pid, fun()->ssh_connection:subsystem(ConnectionRef, ChannelRef, SubSystem, 120?sec) end). + +ssh_start_subsyst_post(_S, _Args, Result) -> Result==success. + +ssh_start_subsyst_next(S, _Result, [C,SS,Pid|_]) -> + S#state{channels = [C#chan{subsystem=SS, + client_pid=Pid}|(S#state.channels--[C])] }. + +%%%---------------- +%%% Send a message on a channel +%%% Precondition: a channel exists with a subsystem connected + +ssh_send_pre(S) -> lists:any(fun has_subsyst/1, S#state.channels). + +ssh_send_args(S) -> [oneof(lists:filter(fun has_subsyst/1, S#state.channels)), + choose(0,1), + message()]. + +%% For re-arrangement in parallel tests. +ssh_send_pre(S, [C|_]) -> lists:member(C, S#state.channels). + +ssh_send(C=#chan{conn_ref=ConnectionRef, ref=ChannelRef, client_pid=Pid}, Type, Msg) -> + do(Pid, + fun() -> + case ssh_connection:send(ConnectionRef, ChannelRef, Type, modify_msg(C,Msg), 10?sec) of + ok -> + receive + {ssh_cm,ConnectionRef,{data,ChannelRef,Type,Answer}} -> Answer + after 15?sec -> + %% receive + %% Other -> {error,{unexpected,Other}} + %% after 0 -> + {error,receive_timeout} + %% end + end; + Other -> + Other + end + end). + +ssh_send_blocking(_S, _Args) -> + true. + +ssh_send_post(_S, [C,_,Msg], Response) when is_binary(Response) -> + Expected = ssh_eqc_subsys:response(modify_msg(C,Msg), C#chan.subsystem), + case Response of + Expected -> true; + _ -> {send_failed, size(Response), size(Expected)} + end; + +ssh_send_post(_S, _Args, Response) -> + {error,Response}. + + +modify_msg(_, <<>>) -> <<>>; +modify_msg(#chan{subsystem=SS}, Msg) -> <<(list_to_binary(SS))/binary,Msg/binary>>. + +%%%================================================================ +%%% Misc functions + +message() -> + resize(500, binary()). + + %% binary(). + + %% oneof([binary(), + %% ?LET(Size, choose(0,10000), binary(Size)) + %% ]). + +has_subsyst(C) -> C#chan.subsystem /= undefined. + +no_subsyst(C) -> not has_subsyst(C). + + +ok({ok,X}) -> X; +ok({error,Err}) -> {error,Err}. + +is_ok({error,_}) -> false; +is_ok(_) -> true. + +ensure_string({A,B,C,D}) -> lists:flatten(io_lib:format("~w.~w.~w.~w",[A,B,C,D])); +ensure_string(X) -> X. + +%%%---------------------------------------------------------------- +present_result(_Module, Cmds, _Triple, true) -> + aggregate(with_title("Distribution sequential/parallel"), sequential_parallel(Cmds), + aggregate(with_title("Function calls"), cmnd_names(Cmds), + aggregate(with_title("Message sizes"), empty_msgs(Cmds), + aggregate(print_frequencies(), message_sizes(Cmds), + aggregate(title("Length of command sequences",print_frequencies()), num_calls(Cmds), + true))))); + +present_result(Module, Cmds, Triple, false) -> + pretty_commands(Module, Cmds, Triple, [{show_states,true}], false). + + + +cmnd_names(Cs) -> traverse_commands(fun cmnd_name/1, Cs). +cmnd_name(L) -> [F || {set,_Var,{call,_Mod,F,_As}} <- L]. + +empty_msgs(Cs) -> traverse_commands(fun empty_msg/1, Cs). +empty_msg(L) -> [empty || {set,_,{call,_,ssh_send,[_,_,Msg]}} <- L, + size(Msg)==0]. + +message_sizes(Cs) -> traverse_commands(fun message_size/1, Cs). +message_size(L) -> [size(Msg) || {set,_,{call,_,ssh_send,[_,_,Msg]}} <- L]. + +num_calls(Cs) -> traverse_commands(fun num_call/1, Cs). +num_call(L) -> [length(L)]. + +sequential_parallel(Cs) -> + traverse_commands(fun(L) -> dup_module(L, sequential) end, + fun(L) -> [dup_module(L1, mkmod("parallel",num(L1,L))) || L1<-L] end, + Cs). +dup_module(L, ModName) -> lists:duplicate(length(L), ModName). +mkmod(PfxStr,N) -> list_to_atom(PfxStr++"_"++integer_to_list(N)). + +%% Meta functions for the aggregate functions +traverse_commands(Fun, L) when is_list(L) -> Fun(L); +traverse_commands(Fun, {Seq, ParLs}) -> Fun(lists:append([Seq|ParLs])). + +traverse_commands(Fseq, _Fpar, L) when is_list(L) -> Fseq(L); +traverse_commands(Fseq, Fpar, {Seq, ParLs}) -> lists:append([Fseq(Seq)|Fpar(ParLs)]). + +%%%---------------- +%% PrintMethod([{term(), int()}]) -> any(). +print_frequencies() -> print_frequencies(10). + +print_frequencies(Ngroups) -> fun([]) -> io:format('Empty list!~n',[]); + (L ) -> print_frequencies(L,Ngroups,0,element(1,lists:last(L))) + end. + +print_frequencies(Ngroups, MaxValue) -> fun(L) -> print_frequencies(L,Ngroups,0,MaxValue) end. + +print_frequencies(L, N, Min, Max) when N>Max -> print_frequencies(L++[{N,0}], N, Min, N); +print_frequencies(L, N, Min, Max) -> +%%io:format('L=~p~n',[L]), + try + IntervalUpperLimits = + lists:reverse( + [Max | tl(lists:reverse(lists:seq(Min,Max,round((Max-Min)/N))))] + ), + {Acc0,_} = lists:mapfoldl(fun(Upper,Lower) -> + {{{Lower,Upper},0}, Upper+1} + end, hd(IntervalUpperLimits), tl(IntervalUpperLimits)), + Fs0 = get_frequencies(L, Acc0), + SumVal = lists:sum([V||{_,V}<-Fs0]), + Fs = with_percentage(Fs0, SumVal), + Mean = mean(L), + Median = median(L), + Npos_value = num_digits(SumVal), + Npos_range = num_digits(Max), + io:format("Range~*s: ~s~n",[2*Npos_range-2,"", "Number in range"]), + io:format("~*c:~*c~n",[2*Npos_range+3,$-, max(16,Npos_value+10),$- ]), + [begin + io:format("~*w - ~*w: ~*w ~5.1f%",[Npos_range,Rlow, + Npos_range,Rhigh, + Npos_value,Val, + Percent]), + [io:format(" <-- mean=~.1f",[Mean]) || in_interval(Mean, Interval)], + [io:format(" <-- median=" ++ + if + is_float(Median) -> "~.1f"; + true -> "~p" + end, [Median]) || in_interval(Median, Interval)], + io:nl() + end + || {Interval={Rlow,Rhigh},Val,Percent} <- Fs], + io:format('~*c ~*c~n',[2*Npos_range,32,Npos_value+2,$-]), + io:format('~*c ~*w~n',[2*Npos_range,32,Npos_value,SumVal]) + %%,io:format('L=~p~n',[L]) + catch + C:E -> + io:format('*** Faild printing (~p:~p) for~n~p~n',[C,E,L]) + end. + +get_frequencies([{I,Num}|T], [{{Lower,Upper},Cnt}|Acc]) when Lower=<I,I=<Upper -> + get_frequencies(T, [{{Lower,Upper},Cnt+Num}|Acc]); +get_frequencies(L=[{I,_Num}|_], [Ah={{_Lower,Upper},_Cnt}|Acc]) when I>Upper -> + [Ah | get_frequencies(L,Acc)]; +get_frequencies([], Acc) -> + Acc. + +with_percentage(Fs, Sum) -> + [{Rng,Val,100*Val/Sum} || {Rng,Val} <- Fs]. + + +title(Str, Fun) -> + fun(L) -> + io:format('~s~n',[Str]), + Fun(L) + end. + +num_digits(I) -> 1+trunc(math:log(I)/math:log(10)). + +num(Elem, List) -> length(lists:takewhile(fun(E) -> E /= Elem end, List)) + 1. + +%%%---- Just for naming an operation for readability +is_odd(I) -> (I rem 2) == 1. + +in_interval(Value, {Rlow,Rhigh}) -> + try + Rlow=<round(Value) andalso round(Value)=<Rhigh + catch + _:_ -> false + end. + +%%%================================================================ +%%% Statistical functions + +%%%---- Mean value +mean(L = [X|_]) when is_number(X) -> + lists:sum(L) / length(L); +mean(L = [{_Value,_Weight}|_]) -> + SumOfWeights = lists:sum([W||{_,W}<-L]), + WeightedSum = lists:sum([W*V||{V,W}<-L]), + WeightedSum / SumOfWeights; +mean(_) -> + undefined. + +%%%---- Median +median(L = [X|_]) when is_number(X) -> + case is_odd(length(L)) of + true -> + hd(lists:nthtail(length(L) div 2, L)); + false -> + %% 1) L has at least on element (the when test). + %% 2) Length is even. + %% => Length >= 2 + [M1,M2|_] = lists:nthtail((length(L) div 2)-1, L), + (M1+M2) / 2 + end; +%% integer Weights... +median(L = [{_Value,_Weight}|_]) -> + median( lists:append([lists:duplicate(W,V) || {V,W} <- L]) ); +median(_) -> + undefined. + +%%%================================================================ +%%% The rest is taken and modified from ssh_test_lib.erl +inet_port(IpAddress)-> + {ok, Socket} = gen_tcp:listen(0, [{ip,IpAddress},{reuseaddr,true}]), + {ok, Port} = inet:port(Socket), + gen_tcp:close(Socket), + Port. + +setup_rsa(Dir) -> + erase_dir(system_dir(Dir)), + erase_dir(user_dir(Dir)), + file:make_dir(system_dir(Dir)), + file:make_dir(user_dir(Dir)), + + file:copy(data_dir(Dir,"id_rsa"), user_dir(Dir,"id_rsa")), + file:copy(data_dir(Dir,"ssh_host_rsa_key"), system_dir(Dir,"ssh_host_rsa_key")), + file:copy(data_dir(Dir,"ssh_host_rsa_key"), system_dir(Dir,"ssh_host_rsa_key.pub")), + ssh_test_lib:setup_rsa_known_host(data_dir(Dir), user_dir(Dir)), + ssh_test_lib:setup_rsa_auth_keys(data_dir(Dir), user_dir(Dir)). + +data_dir(Dir, File) -> filename:join(Dir, File). +system_dir(Dir, File) -> filename:join([Dir, "system", File]). +user_dir(Dir, File) -> filename:join([Dir, "user", File]). + +data_dir(Dir) -> Dir. +system_dir(Dir) -> system_dir(Dir,""). +user_dir(Dir) -> user_dir(Dir,""). + +erase_dir(Dir) -> + case file:list_dir(Dir) of + {ok,Files} -> lists:foreach(fun(F) -> file:delete(filename:join(Dir,F)) end, + Files); + _ -> ok + end, + file:del_dir(Dir). + +-endif. +-endif. diff --git a/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_dsa b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_dsa new file mode 100644 index 0000000000..d306f8b26e --- /dev/null +++ b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_dsa @@ -0,0 +1,13 @@ +-----BEGIN DSA PRIVATE KEY----- +MIIBvAIBAAKBgQDfi2flSTZZofwT4yQT0NikX/LGNT7UPeB/XEWe/xovEYCElfaQ +APFixXvEgXwoojmZ5kiQRKzLM39wBP0jPERLbnZXfOOD0PDnw0haMh7dD7XKVMod +/EigVgHf/qBdM2M8yz1s/rRF7n1UpLSypziKjkzCm7JoSQ2zbWIPdmBIXwIVAMgP +kpr7Sq3O7sHdb8D601DRjoExAoGAMOQxDfB2Fd8ouz6G96f/UOzRMI/Kdv8kYYKW +JIGY+pRYrLPyYzUeJznwZreOJgrczAX+luHnKFWJ2Dnk5CyeXk67Wsr7pJ/4MBMD +OKeIS0S8qoSBN8+Krp79fgA+yS3IfqbkJLtLu4EBaCX4mKQIX4++k44d4U5lc8pt ++9hlEI8CgYEAznKxx9kyC6bVo7LUYKaGhofRFt0SYFc5PVmT2VUGRs1R6+6DPD+e +uEO6IhFct7JFSRbP9p0JD4Uk+3zlZF+XX6b2PsZkeV8f/02xlNGUSmEzCSiNg1AX +Cy/WusYhul0MncWCHMcOZB5rIvU/aP5EJJtn3xrRaz6u0SThF6AnT34CFQC63czE +ZU8w8Q+H7z0j+a+70x2iAw== +-----END DSA PRIVATE KEY----- + diff --git a/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_rsa b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_rsa new file mode 100644 index 0000000000..9d7e0dd5fb --- /dev/null +++ b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_rsa @@ -0,0 +1,15 @@ +-----BEGIN RSA PRIVATE KEY----- +MIICXAIBAAKBgQD1OET+3O/Bvj/dtjxDTXmj1oiJt4sIph5kGy0RfjoPrZfaS+CU +DhakCmS6t2ivxWFgtpKWaoGMZMJqWj6F6ZsumyFl3FPBtujwY/35cgifrI9Ns4Tl +zR1uuengNBmV+WRQ5cd9F2qS6Z8aDQihzt0r8JUqLcK+VQbrmNzboCCQQwIDAQAB +AoGAPQEyqPTt8JUT7mRXuaacjFXiweAXhp9NEDpyi9eLOjtFe9lElZCrsUOkq47V +TGUeRKEm9qSodfTbKPoqc8YaBJGJPhUaTAcha+7QcDdfHBvIsgxvU7ePVnlpXRp3 +CCUEMPhlnx6xBoTYP+fRU0e3+xJIPVyVCqX1jAdUMkzfRoECQQD6ux7B1QJAIWyK +SGkbDUbBilNmzCFNgIpOP6PA+bwfi5d16diTpra5AX09keQABAo/KaP1PdV8Vg0p +z4P3A7G3AkEA+l+AKG6m0kQTTBMJDqOdVPYwe+5GxunMaqmhokpEbuGsrZBl5Dvd +WpcBjR7jmenrhKZRIuA+Fz5HPo/UQJPl1QJBAKxstDkeED8j/S2XoFhPKAJ+6t39 +sUVICVTIZQeXdmzHJXCcUSkw8+WEhakqw/3SyW0oaK2FSWQJFWJUZ+8eJj8CQEh3 +xeduB5kKnS9CvzdeghZqX6QvVosSdtlUmfUYW/BgH5PpHKTP8wTaeld3XldZTpMJ +dKiMkUw2+XYROVUrubUCQD+Na1LhULlpn4ISEtIEfqpdlUhxDgO15Wg8USmsng+x +ICliVOSQtwaZjm8kwaFt0W7XnpnDxbRs37vIEbIMWak= +-----END RSA PRIVATE KEY----- diff --git a/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/ssh_host_dsa_key b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/ssh_host_dsa_key new file mode 100644 index 0000000000..51ab6fbd88 --- /dev/null +++ b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/ssh_host_dsa_key @@ -0,0 +1,13 @@ +-----BEGIN DSA PRIVATE KEY----- +MIIBuwIBAAKBgQCClaHzE2ul0gKSUxah5W0W8UiJLy4hXngKEqpaUq9SSdVdY2LK +wVfKH1gt5iuaf1FfzOhsIC9G/GLnjYttXZc92cv/Gfe3gR+s0ni2++MX+T++mE/Q +diltXv/Hp27PybS67SmiFW7I+RWnT2OKlMPtw2oUuKeztCe5UWjaj/y5FQIVAPLA +l9RpiU30Z87NRAHY3NTRaqtrAoGANMRxw8UfdtNVR0CrQj3AgPaXOGE4d+G4Gp4X +skvnCHycSVAjtYxebUkzUzt5Q6f/IabuLUdge3gXrc8BetvrcKbp+XZgM0/Vj2CF +Ymmy3in6kzGZq7Fw1sZaku6AOU8vLa5woBT2vAcHLLT1bLAzj7viL048T6MfjrOP +ef8nHvACgYBhDWFQJ1mf99sg92LalVq1dHLmVXb3PTJDfCO/Gz5NFmj9EZbAtdah +/XcF3DeRF+eEoz48wQF/ExVxSMIhLdL+o+ElpVhlM7Yii+T7dPhkQfEul6zZXu+U +ykSTXYUbtsfTNRFQGBW2/GfnEc0mnIxfn9v10NEWMzlq5z9wT9P0CgIVAN4wtL5W +Lv62jKcdskxNyz2NQoBx +-----END DSA PRIVATE KEY----- + diff --git a/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/ssh_host_dsa_key.pub b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/ssh_host_dsa_key.pub new file mode 100644 index 0000000000..4dbb1305b0 --- /dev/null +++ b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/ssh_host_dsa_key.pub @@ -0,0 +1,11 @@ +---- BEGIN SSH2 PUBLIC KEY ---- +AAAAB3NzaC1kc3MAAACBAIKVofMTa6XSApJTFqHlbRbxSIkvLiFeeAoSqlpSr1JJ1V1j +YsrBV8ofWC3mK5p/UV/M6GwgL0b8YueNi21dlz3Zy/8Z97eBH6zSeLb74xf5P76YT9B2 +KW1e/8enbs/JtLrtKaIVbsj5FadPY4qUw+3DahS4p7O0J7lRaNqP/LkVAAAAFQDywJfU +aYlN9GfOzUQB2NzU0WqrawAAAIA0xHHDxR9201VHQKtCPcCA9pc4YTh34bganheyS+cI +fJxJUCO1jF5tSTNTO3lDp/8hpu4tR2B7eBetzwF62+twpun5dmAzT9WPYIViabLeKfqT +MZmrsXDWxlqS7oA5Ty8trnCgFPa8BwcstPVssDOPu+IvTjxPox+Os495/yce8AAAAIBh +DWFQJ1mf99sg92LalVq1dHLmVXb3PTJDfCO/Gz5NFmj9EZbAtdah/XcF3DeRF+eEoz48 +wQF/ExVxSMIhLdL+o+ElpVhlM7Yii+T7dPhkQfEul6zZXu+UykSTXYUbtsfTNRFQGBW2 +/GfnEc0mnIxfn9v10NEWMzlq5z9wT9P0Cg== +---- END SSH2 PUBLIC KEY ---- diff --git a/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/ssh_host_rsa_key b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/ssh_host_rsa_key new file mode 100644 index 0000000000..79968bdd7d --- /dev/null +++ b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/ssh_host_rsa_key @@ -0,0 +1,16 @@ +-----BEGIN RSA PRIVATE KEY----- +MIICXQIBAAKBgQDCZX+4FBDwZIh9y/Uxee1VJnEXlowpz2yDKwj8semM4q843337 +zbNfxHmladB1lpz2NqyxI175xMIJuDxogyZdsOxGnFAzAnthR4dqL/RWRWzjaxSB +6IAO9SPYVVlrpZ+1hsjLW79fwXK/yc8VdhRuWTeQiRgYY2ek8+OKbOqz4QIDAQAB +AoGANmvJzJO5hkLuvyDZHKfAnGTtpifcR1wtSa9DjdKUyn8vhKF0mIimnbnYQEmW +NUUb3gXCZLi9PvkpRSVRrASDOZwcjoU/Kvww163vBUVb2cOZfFhyn6o2Sk88Tt++ +udH3hdjpf9i7jTtUkUe+QYPsia+wgvvrmn4QrahLAH86+kECQQDx5gFeXTME3cnW +WMpFz3PPumduzjqgqMMWEccX4FtQkMX/gyGa5UC7OHFyh0N/gSWvPbRHa8A6YgIt +n8DO+fh5AkEAzbqX4DOn8NY6xJIi42q7l/2jIA0RkB6P7YugW5NblhqBZ0XDnpA5 +sMt+rz+K07u9XZtxgh1xi7mNfwY6lEAMqQJBAJBEauCKmRj35Z6OyeQku59SPsnY ++SJEREVvSNw2lH9SOKQQ4wPsYlTGbvKtNVZgAcen91L5MmYfeckYE/fdIZECQQCt +64zxsTnM1I8iFxj/gP/OYlJBikrKt8udWmjaghzvLMEw+T2DExJyb9ZNeT53+UMB +m6O+B/4xzU/djvp+0hbhAkAemIt+rA5kTmYlFndhpvzkSSM8a2EXsO4XIPgGWCTT +tQKS/tTly0ADMjN/TVy11+9d6zcqadNVuHXHGtR4W0GR +-----END RSA PRIVATE KEY----- + diff --git a/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/ssh_host_rsa_key.pub b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/ssh_host_rsa_key.pub new file mode 100644 index 0000000000..75d2025c71 --- /dev/null +++ b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/ssh_host_rsa_key.pub @@ -0,0 +1,5 @@ +---- BEGIN SSH2 PUBLIC KEY ---- +AAAAB3NzaC1yc2EAAAADAQABAAAAgQDCZX+4FBDwZIh9y/Uxee1VJnEXlowpz2yDKwj8 +semM4q843337zbNfxHmladB1lpz2NqyxI175xMIJuDxogyZdsOxGnFAzAnthR4dqL/RW +RWzjaxSB6IAO9SPYVVlrpZ+1hsjLW79fwXK/yc8VdhRuWTeQiRgYY2ek8+OKbOqz4Q== +---- END SSH2 PUBLIC KEY ---- diff --git a/lib/ssh/test/property_test/ssh_eqc_encode_decode.erl b/lib/ssh/test/property_test/ssh_eqc_encode_decode.erl new file mode 100644 index 0000000000..34630bdc91 --- /dev/null +++ b/lib/ssh/test/property_test/ssh_eqc_encode_decode.erl @@ -0,0 +1,397 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2004-2014. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% + +-module(ssh_eqc_encode_decode). + +-compile(export_all). + +-proptest(eqc). +-proptest([triq,proper]). + +-include_lib("ct_property_test.hrl"). + +-ifndef(EQC). +-ifndef(PROPER). +-ifndef(TRIQ). +-define(EQC,true). +%%-define(PROPER,true). +%%-define(TRIQ,true). +-endif. +-endif. +-endif. + +-ifdef(EQC). +-include_lib("eqc/include/eqc.hrl"). +-define(MOD_eqc,eqc). + +-else. +-ifdef(PROPER). +-include_lib("proper/include/proper.hrl"). +-define(MOD_eqc,proper). + +-else. +-ifdef(TRIQ). +-define(MOD_eqc,triq). +-include_lib("triq/include/triq.hrl"). + +-endif. +-endif. +-endif. + + +%%% Properties: + +prop_ssh_decode() -> + ?FORALL(Msg, ssh_msg(), + try ssh_message:decode(Msg) + of + _ -> true + catch + C:E -> io:format('~p:~p~n',[C,E]), + false + end + ). + + +%%% This fails because ssh_message is not symmetric in encode and decode regarding data types +prop_ssh_decode_encode() -> + ?FORALL(Msg, ssh_msg(), + Msg == ssh_message:encode(ssh_message:decode(Msg)) + ). + + +%%%================================================================ +%%% +%%% Scripts to generate message generators +%%% + +%% awk '/^( |\t)+byte( |\t)+SSH/,/^( |\t)*$/{print}' rfc425?.txt | sed 's/^\( \|\\t\)*//' > msgs.txt + +%% awk '/^byte( |\t)+SSH/{print $2","}' < msgs.txt + +%% awk 'BEGIN{print "%%%---- BEGIN GENERATED";prev=0} END{print " >>.\n%%%---- END GENERATED"} /^byte( |\t)+SSH/{if (prev==1) print " >>.\n"; prev=1; printf "%c%s%c",39,$2,39; print "()->\n <<?"$2;next} /^string( |\t)+\"/{print " ,"$2;next} /^string( |\t)+.*address/{print " ,(ssh_string_address())/binary %%",$2,$3,$4,$5,$6;next}/^string( |\t)+.*US-ASCII/{print " ,(ssh_string_US_ASCII())/binary %%",$2,$3,$4,$5,$6;next} /^string( |\t)+.*UTF-8/{print " ,(ssh_string_UTF_8())/binary %% ",$2,$3,$4,$5,$6;next} /^[a-z0-9]+( |\t)/{print " ,(ssh_"$1"())/binary %%",$2,$3,$4,$5,$6;next} /^byte\[16\]( |\t)+/{print" ,(ssh_byte_16())/binary %%",$2,$3,$4,$5,$6;next} /^name-list( |\t)+/{print" ,(ssh_name_list())/binary %%",$2,$3,$4,$5,$6;next} /./{print "?? %%",$0}' < msgs.txt > gen.txt + +%%%================================================================ +%%% +%%% Generators +%%% + +ssh_msg() -> ?LET(M,oneof( +[[msg_code('SSH_MSG_CHANNEL_CLOSE'),gen_uint32()], + [msg_code('SSH_MSG_CHANNEL_DATA'),gen_uint32(),gen_string( )], + [msg_code('SSH_MSG_CHANNEL_EOF'),gen_uint32()], + [msg_code('SSH_MSG_CHANNEL_EXTENDED_DATA'),gen_uint32(),gen_uint32(),gen_string( )], + [msg_code('SSH_MSG_CHANNEL_FAILURE'),gen_uint32()], + [msg_code('SSH_MSG_CHANNEL_OPEN'),gen_string("direct-tcpip"),gen_uint32(),gen_uint32(),gen_uint32(),gen_string( ),gen_uint32(),gen_string( ),gen_uint32()], + [msg_code('SSH_MSG_CHANNEL_OPEN'),gen_string("forwarded-tcpip"),gen_uint32(),gen_uint32(),gen_uint32(),gen_string( ),gen_uint32(),gen_string( ),gen_uint32()], + [msg_code('SSH_MSG_CHANNEL_OPEN'),gen_string("session"),gen_uint32(),gen_uint32(),gen_uint32()], + [msg_code('SSH_MSG_CHANNEL_OPEN'),gen_string("x11"),gen_uint32(),gen_uint32(),gen_uint32(),gen_string( ),gen_uint32()], + [msg_code('SSH_MSG_CHANNEL_OPEN'),gen_string( ),gen_uint32(),gen_uint32(),gen_uint32()], + [msg_code('SSH_MSG_CHANNEL_OPEN_CONFIRMATION'),gen_uint32(),gen_uint32(),gen_uint32(),gen_uint32()], + [msg_code('SSH_MSG_CHANNEL_OPEN_FAILURE'),gen_uint32(),gen_uint32(),gen_string( ),gen_string( )], + [msg_code('SSH_MSG_CHANNEL_REQUEST'),gen_uint32(),gen_string("env"),gen_boolean(),gen_string( ),gen_string( )], + [msg_code('SSH_MSG_CHANNEL_REQUEST'),gen_uint32(),gen_string("exec"),gen_boolean(),gen_string( )], + [msg_code('SSH_MSG_CHANNEL_REQUEST'),gen_uint32(),gen_string("exit-signal"),0,gen_string( ),gen_boolean(),gen_string( ),gen_string( )], + [msg_code('SSH_MSG_CHANNEL_REQUEST'),gen_uint32(),gen_string("exit-status"),0,gen_uint32()], + [msg_code('SSH_MSG_CHANNEL_REQUEST'),gen_uint32(),gen_string("pty-req"),gen_boolean(),gen_string( ),gen_uint32(),gen_uint32(),gen_uint32(),gen_uint32(),gen_string( )], + [msg_code('SSH_MSG_CHANNEL_REQUEST'),gen_uint32(),gen_string("shell"),gen_boolean()], + [msg_code('SSH_MSG_CHANNEL_REQUEST'),gen_uint32(),gen_string("signal"),0,gen_string( )], + [msg_code('SSH_MSG_CHANNEL_REQUEST'),gen_uint32(),gen_string("subsystem"),gen_boolean(),gen_string( )], + [msg_code('SSH_MSG_CHANNEL_REQUEST'),gen_uint32(),gen_string("window-change"),0,gen_uint32(),gen_uint32(),gen_uint32(),gen_uint32()], + [msg_code('SSH_MSG_CHANNEL_REQUEST'),gen_uint32(),gen_string("x11-req"),gen_boolean(),gen_boolean(),gen_string( ),gen_string( ),gen_uint32()], + [msg_code('SSH_MSG_CHANNEL_REQUEST'),gen_uint32(),gen_string("xon-xoff"),0,gen_boolean()], + [msg_code('SSH_MSG_CHANNEL_REQUEST'),gen_uint32(),gen_string( ),gen_boolean()], + [msg_code('SSH_MSG_CHANNEL_SUCCESS'),gen_uint32()], + [msg_code('SSH_MSG_CHANNEL_WINDOW_ADJUST'),gen_uint32(),gen_uint32()], +%%Assym [msg_code('SSH_MSG_DEBUG'),gen_boolean(),gen_string( ),gen_string( )], + [msg_code('SSH_MSG_DISCONNECT'),gen_uint32(),gen_string( ),gen_string( )], +%%Assym [msg_code('SSH_MSG_GLOBAL_REQUEST'),gen_string("cancel-tcpip-forward"),gen_boolean(),gen_string( ),gen_uint32()], +%%Assym [msg_code('SSH_MSG_GLOBAL_REQUEST'),gen_string("tcpip-forward"),gen_boolean(),gen_string( ),gen_uint32()], +%%Assym [msg_code('SSH_MSG_GLOBAL_REQUEST'),gen_string( ),gen_boolean()], + [msg_code('SSH_MSG_IGNORE'),gen_string( )], + %% [msg_code('SSH_MSG_KEXDH_INIT'),gen_mpint()], + %% [msg_code('SSH_MSG_KEXDH_REPLY'),gen_string( ),gen_mpint(),gen_string( )], + %% [msg_code('SSH_MSG_KEXINIT'),gen_byte(16),gen_name_list(),gen_name_list(),gen_name_list(),gen_name_list(),gen_name_list(),gen_name_list(),gen_name_list(),gen_name_list(),gen_name_list(),gen_name_list(),gen_boolean(),gen_uint32()], + [msg_code('SSH_MSG_KEX_DH_GEX_GROUP'),gen_mpint(),gen_mpint()], + [msg_code('SSH_MSG_NEWKEYS')], + [msg_code('SSH_MSG_REQUEST_FAILURE')], + [msg_code('SSH_MSG_REQUEST_SUCCESS')], + [msg_code('SSH_MSG_REQUEST_SUCCESS'),gen_uint32()], + [msg_code('SSH_MSG_SERVICE_ACCEPT'),gen_string( )], + [msg_code('SSH_MSG_SERVICE_REQUEST'),gen_string( )], + [msg_code('SSH_MSG_UNIMPLEMENTED'),gen_uint32()], + [msg_code('SSH_MSG_USERAUTH_BANNER'),gen_string( ),gen_string( )], + [msg_code('SSH_MSG_USERAUTH_FAILURE'),gen_name_list(),gen_boolean()], + [msg_code('SSH_MSG_USERAUTH_PASSWD_CHANGEREQ'),gen_string( ),gen_string( )], + [msg_code('SSH_MSG_USERAUTH_PK_OK'),gen_string( ),gen_string( )], + [msg_code('SSH_MSG_USERAUTH_SUCCESS')] +] + +), list_to_binary(M)). + + +%%%================================================================ +%%% +%%% Generator +%%% + +do() -> + io_lib:format('[~s~n]', + [write_gen( + files(["rfc4254.txt", + "rfc4253.txt", + "rfc4419.txt", + "rfc4252.txt", + "rfc4256.txt"]))]). + + +write_gen(L) when is_list(L) -> + string:join(lists:map(fun write_gen/1, L), ",\n "); +write_gen({MsgName,Args}) -> + lists:flatten(["[",generate_args([MsgName|Args]),"]"]). + +generate_args(As) -> string:join([generate_arg(A) || A <- As], ","). + +generate_arg({<<"string">>, <<"\"",B/binary>>}) -> + S = get_string($",B), + ["gen_string(\"",S,"\")"]; +generate_arg({<<"string">>, _}) -> "gen_string( )"; +generate_arg({<<"byte[",B/binary>>, _}) -> + io_lib:format("gen_byte(~p)",[list_to_integer(get_string($],B))]); +generate_arg({<<"byte">> ,_}) -> "gen_byte()"; +generate_arg({<<"uint16">>,_}) -> "gen_uint16()"; +generate_arg({<<"uint32">>,_}) -> "gen_uint32()"; +generate_arg({<<"uint64">>,_}) -> "gen_uint64()"; +generate_arg({<<"mpint">>,_}) -> "gen_mpint()"; +generate_arg({<<"name-list">>,_}) -> "gen_name_list()"; +generate_arg({<<"boolean">>,<<"FALSE">>}) -> "0"; +generate_arg({<<"boolean">>,<<"TRUE">>}) -> "1"; +generate_arg({<<"boolean">>,_}) -> "gen_boolean()"; +generate_arg({<<"....">>,_}) -> ""; %% FIXME +generate_arg(Name) when is_binary(Name) -> + lists:flatten(["msg_code('",binary_to_list(Name),"')"]). + + +gen_boolean() -> choose(0,1). + +gen_byte() -> choose(0,255). + +gen_uint16() -> gen_byte(2). + +gen_uint32() -> gen_byte(4). + +gen_uint64() -> gen_byte(8). + +gen_byte(N) when N>0 -> [gen_byte() || _ <- lists:seq(1,N)]. + +gen_char() -> choose($a,$z). + +gen_mpint() -> ?LET(Size, choose(1,20), + ?LET(Str, vector(Size, gen_byte()), + gen_string( strip_0s(Str) ) + )). + +strip_0s([0|T]) -> strip_0s(T); +strip_0s(X) -> X. + + +gen_string() -> + ?LET(Size, choose(0,10), + ?LET(Vector,vector(Size, gen_char()), + gen_string(Vector) + )). + +gen_string(S) when is_binary(S) -> gen_string(binary_to_list(S)); +gen_string(S) when is_list(S) -> uint32_to_list(length(S)) ++ S. + +gen_name_list() -> + ?LET(NumNames, choose(0,10), + ?LET(L, [gen_name() || _ <- lists:seq(1,NumNames)], + gen_string( string:join(L,"," ) ) + )). + +gen_name() -> gen_string(). + +uint32_to_list(I) -> binary_to_list(<<I:32/unsigned-big-integer>>). + +%%%---- +get_string(Delim, B) -> + binary_to_list( element(1, split_binary(B, count_string_chars(Delim,B,0))) ). + +count_string_chars(Delim, <<Delim,_/binary>>, Acc) -> Acc; +count_string_chars(Delim, <<_,B/binary>>, Acc) -> count_string_chars(Delim, B, Acc+1). + + +-define(MSG_CODE(Name,Num), +msg_code(Name) -> Num; +msg_code(Num) -> Name +). + +?MSG_CODE('SSH_MSG_USERAUTH_REQUEST', 50); +?MSG_CODE('SSH_MSG_USERAUTH_FAILURE', 51); +?MSG_CODE('SSH_MSG_USERAUTH_SUCCESS', 52); +?MSG_CODE('SSH_MSG_USERAUTH_BANNER', 53); +?MSG_CODE('SSH_MSG_USERAUTH_PK_OK', 60); +?MSG_CODE('SSH_MSG_USERAUTH_PASSWD_CHANGEREQ', 60); +?MSG_CODE('SSH_MSG_DISCONNECT', 1); +?MSG_CODE('SSH_MSG_IGNORE', 2); +?MSG_CODE('SSH_MSG_UNIMPLEMENTED', 3); +?MSG_CODE('SSH_MSG_DEBUG', 4); +?MSG_CODE('SSH_MSG_SERVICE_REQUEST', 5); +?MSG_CODE('SSH_MSG_SERVICE_ACCEPT', 6); +?MSG_CODE('SSH_MSG_KEXINIT', 20); +?MSG_CODE('SSH_MSG_NEWKEYS', 21); +?MSG_CODE('SSH_MSG_GLOBAL_REQUEST', 80); +?MSG_CODE('SSH_MSG_REQUEST_SUCCESS', 81); +?MSG_CODE('SSH_MSG_REQUEST_FAILURE', 82); +?MSG_CODE('SSH_MSG_CHANNEL_OPEN', 90); +?MSG_CODE('SSH_MSG_CHANNEL_OPEN_CONFIRMATION', 91); +?MSG_CODE('SSH_MSG_CHANNEL_OPEN_FAILURE', 92); +?MSG_CODE('SSH_MSG_CHANNEL_WINDOW_ADJUST', 93); +?MSG_CODE('SSH_MSG_CHANNEL_DATA', 94); +?MSG_CODE('SSH_MSG_CHANNEL_EXTENDED_DATA', 95); +?MSG_CODE('SSH_MSG_CHANNEL_EOF', 96); +?MSG_CODE('SSH_MSG_CHANNEL_CLOSE', 97); +?MSG_CODE('SSH_MSG_CHANNEL_REQUEST', 98); +?MSG_CODE('SSH_MSG_CHANNEL_SUCCESS', 99); +?MSG_CODE('SSH_MSG_CHANNEL_FAILURE', 100); +?MSG_CODE('SSH_MSG_USERAUTH_INFO_REQUEST', 60); +?MSG_CODE('SSH_MSG_USERAUTH_INFO_RESPONSE', 61); +?MSG_CODE('SSH_MSG_KEX_DH_GEX_REQUEST_OLD', 30); +?MSG_CODE('SSH_MSG_KEX_DH_GEX_REQUEST', 34); +?MSG_CODE('SSH_MSG_KEX_DH_GEX_GROUP', 31); +?MSG_CODE('SSH_MSG_KEX_DH_GEX_INIT', 32); +?MSG_CODE('SSH_MSG_KEX_DH_GEX_REPLY', 33). + +%%%============================================================================= +%%%============================================================================= +%%%============================================================================= + +files(Fs) -> + Defs = lists:usort(lists:flatten(lists:map(fun file/1, Fs))), + DefinedIDs = lists:usort([binary_to_list(element(1,D)) || D <- Defs]), + WantedIDs = lists:usort(wanted_messages()), + Missing = WantedIDs -- DefinedIDs, + case Missing of + [] -> ok; + _ -> io:format('%% Warning: missing ~p~n', [Missing]) + end, + Defs. + + +file(F) -> + {ok,B} = file:read_file(F), + hunt_msg_def(B). + + +hunt_msg_def(<<"\n",B/binary>>) -> some_hope(skip_blanks(B)); +hunt_msg_def(<<_, B/binary>>) -> hunt_msg_def(B); +hunt_msg_def(<<>>) -> []. + +some_hope(<<"byte ", B/binary>>) -> try_message(skip_blanks(B)); +some_hope(B) -> hunt_msg_def(B). + +try_message(B = <<"SSH_MSG_",_/binary>>) -> + {ID,Rest} = get_id(B), + case lists:member(binary_to_list(ID), wanted_messages()) of + true -> + {Lines,More} = get_def_lines(skip_blanks(Rest), []), + [{ID,lists:reverse(Lines)} | hunt_msg_def(More)]; + false -> + hunt_msg_def(Rest) + end; +try_message(B) -> hunt_msg_def(B). + + +skip_blanks(<<32, B/binary>>) -> skip_blanks(B); +skip_blanks(<< 9, B/binary>>) -> skip_blanks(B); +skip_blanks(B) -> B. + +get_def_lines(B0 = <<"\n",B/binary>>, Acc) -> + {ID,Rest} = get_id(skip_blanks(B)), + case {size(ID), skip_blanks(Rest)} of + {0,<<"....",More/binary>>} -> + {Text,LineEnd} = get_to_eol(skip_blanks(More)), + get_def_lines(LineEnd, [{<<"....">>,Text}|Acc]); + {0,_} -> + {Acc,B0}; + {_,Rest1} -> + {Text,LineEnd} = get_to_eol(Rest1), + get_def_lines(LineEnd, [{ID,Text}|Acc]) + end; +get_def_lines(B, Acc) -> + {Acc,B}. + + +get_to_eol(B) -> split_binary(B, count_to_eol(B,0)). + +count_to_eol(<<"\n",_/binary>>, Acc) -> Acc; +count_to_eol(<<>>, Acc) -> Acc; +count_to_eol(<<_,B/binary>>, Acc) -> count_to_eol(B,Acc+1). + + +get_id(B) -> split_binary(B, count_id_chars(B,0)). + +count_id_chars(<<C,B/binary>>, Acc) when $A=<C,C=<$Z -> count_id_chars(B,Acc+1); +count_id_chars(<<C,B/binary>>, Acc) when $a=<C,C=<$z -> count_id_chars(B,Acc+1); +count_id_chars(<<C,B/binary>>, Acc) when $0=<C,C=<$9 -> count_id_chars(B,Acc+1); +count_id_chars(<<"_",B/binary>>, Acc) -> count_id_chars(B,Acc+1); +count_id_chars(<<"-",B/binary>>, Acc) -> count_id_chars(B,Acc+1); %% e.g name-list +count_id_chars(<<"[",B/binary>>, Acc) -> count_id_chars(B,Acc+1); %% e.g byte[16] +count_id_chars(<<"]",B/binary>>, Acc) -> count_id_chars(B,Acc+1); %% e.g byte[16] +count_id_chars(_, Acc) -> Acc. + +wanted_messages() -> + ["SSH_MSG_CHANNEL_CLOSE", + "SSH_MSG_CHANNEL_DATA", + "SSH_MSG_CHANNEL_EOF", + "SSH_MSG_CHANNEL_EXTENDED_DATA", + "SSH_MSG_CHANNEL_FAILURE", + "SSH_MSG_CHANNEL_OPEN", + "SSH_MSG_CHANNEL_OPEN_CONFIRMATION", + "SSH_MSG_CHANNEL_OPEN_FAILURE", + "SSH_MSG_CHANNEL_REQUEST", + "SSH_MSG_CHANNEL_SUCCESS", + "SSH_MSG_CHANNEL_WINDOW_ADJUST", + "SSH_MSG_DEBUG", + "SSH_MSG_DISCONNECT", + "SSH_MSG_GLOBAL_REQUEST", + "SSH_MSG_IGNORE", + "SSH_MSG_KEXDH_INIT", + "SSH_MSG_KEXDH_REPLY", + "SSH_MSG_KEXINIT", + "SSH_MSG_KEX_DH_GEX_GROUP", + "SSH_MSG_KEX_DH_GEX_REQUEST", + "SSH_MSG_KEX_DH_GEX_REQUEST_OLD", + "SSH_MSG_NEWKEYS", + "SSH_MSG_REQUEST_FAILURE", + "SSH_MSG_REQUEST_SUCCESS", + "SSH_MSG_SERVICE_ACCEPT", + "SSH_MSG_SERVICE_REQUEST", + "SSH_MSG_UNIMPLEMENTED", + "SSH_MSG_USERAUTH_BANNER", + "SSH_MSG_USERAUTH_FAILURE", +%% hard args "SSH_MSG_USERAUTH_INFO_REQUEST", +%% "SSH_MSG_USERAUTH_INFO_RESPONSE", + "SSH_MSG_USERAUTH_PASSWD_CHANGEREQ", + "SSH_MSG_USERAUTH_PK_OK", +%%rfc4252 p12 error "SSH_MSG_USERAUTH_REQUEST", + "SSH_MSG_USERAUTH_SUCCESS"]. + diff --git a/lib/ssh/test/property_test/ssh_eqc_subsys.erl b/lib/ssh/test/property_test/ssh_eqc_subsys.erl new file mode 100644 index 0000000000..e4b6af166f --- /dev/null +++ b/lib/ssh/test/property_test/ssh_eqc_subsys.erl @@ -0,0 +1,63 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2004-2014. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% + +-module(ssh_eqc_subsys). + +-behaviour(ssh_daemon_channel). + +-export([init/1, handle_msg/2, handle_ssh_msg/2, terminate/2]). + +-export([response/2]). + +-record(state, {id, + cm, + subsyst + }). + +init([SS]) -> + {ok, #state{subsyst=SS}}. + +handle_msg({ssh_channel_up, ChannelId, ConnectionManager}, State) -> + {ok, State#state{id = ChannelId, + cm = ConnectionManager}}. + +handle_ssh_msg({ssh_cm, CM, {data, ChannelId, Type, Data}}, S) -> + ssh_connection:send(CM, ChannelId, Type, response(Data,S)), + {ok, S}; + +handle_ssh_msg({ssh_cm, _ConnectionManager, {eof, _ChannelId}}, State) -> + {ok, State}; + +handle_ssh_msg({ssh_cm, _, {signal, _, _}}, State) -> + %% Ignore signals according to RFC 4254 section 6.9. + {ok, State}; + +handle_ssh_msg({ssh_cm, _, {exit_signal, ChannelId, _, _Error, _}}, State) -> + {stop, ChannelId, State}; + +handle_ssh_msg({ssh_cm, _, {exit_status, ChannelId, _Status}}, State) -> + {stop, ChannelId, State}. + +terminate(_Reason, _State) -> + ok. + + +response(Msg, #state{subsyst=SS}) -> response(Msg, SS); +response(Msg, SS) -> <<"Resp: ",Msg/binary,(list_to_binary(SS))/binary>>. diff --git a/lib/ssh/test/ssh_basic_SUITE.erl b/lib/ssh/test/ssh_basic_SUITE.erl index d2e52379fa..9242731924 100644 --- a/lib/ssh/test/ssh_basic_SUITE.erl +++ b/lib/ssh/test/ssh_basic_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2013. All Rights Reserved. +%% Copyright Ericsson AB 2008-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -47,20 +47,36 @@ all() -> daemon_already_started, server_password_option, server_userpassword_option, - double_close]. + double_close, + ssh_connect_timeout, + ssh_connect_arg4_timeout, + {group, hardening_tests} + ]. -groups() -> +groups() -> [{dsa_key, [], basic_tests()}, {rsa_key, [], basic_tests()}, {dsa_pass_key, [], [pass_phrase]}, {rsa_pass_key, [], [pass_phrase]}, - {internal_error, [], [internal_error]} + {internal_error, [], [internal_error]}, + {hardening_tests, [], [ssh_connect_nonegtimeout_connected_parallel, + ssh_connect_nonegtimeout_connected_sequential, + ssh_connect_negtimeout_parallel, + ssh_connect_negtimeout_sequential, + max_sessions_ssh_connect_parallel, + max_sessions_ssh_connect_sequential, + max_sessions_sftp_start_channel_parallel, + max_sessions_sftp_start_channel_sequential + ]} ]. + basic_tests() -> [send, close, peername_sockname, exec, exec_compressed, shell, cli, known_hosts, - idle_time, rekey, openssh_zlib_basic_test]. + idle_time, rekey, openssh_zlib_basic_test, + misc_ssh_options, inet_option]. + %%-------------------------------------------------------------------- init_per_suite(Config) -> @@ -74,6 +90,8 @@ end_per_suite(_Config) -> ssh:stop(), crypto:stop(). %%-------------------------------------------------------------------- +init_per_group(hardening_tests, Config) -> + init_per_group(dsa_key, Config); init_per_group(dsa_key, Config) -> DataDir = ?config(data_dir, Config), PrivDir = ?config(priv_dir, Config), @@ -103,6 +121,8 @@ init_per_group(internal_error, Config) -> init_per_group(_, Config) -> Config. +end_per_group(hardening_tests, Config) -> + end_per_group(dsa_key, Config); end_per_group(dsa_key, Config) -> PrivDir = ?config(priv_dir, Config), ssh_test_lib:clean_dsa(PrivDir), @@ -164,16 +184,47 @@ misc_ssh_options(Config) when is_list(Config) -> SystemDir = filename:join(?config(priv_dir, Config), system), UserDir = ?config(priv_dir, Config), - CMiscOpt0 = [{connecect_timeout, 1000}, {ip_v6_disabled, false}, {user_dir, UserDir}], - CMiscOpt1 = [{connecect_timeout, infinity}, {ip_v6_disabled, true}, {user_dir, UserDir}], - SMiscOpt0 = [{ip_v6_disabled, false}, {user_dir, UserDir}, {system_dir, SystemDir}], - SMiscOpt1 = [{ip_v6_disabled, true}, {user_dir, UserDir}, {system_dir, SystemDir}], + CMiscOpt0 = [{connect_timeout, 1000}, {user_dir, UserDir}], + CMiscOpt1 = [{connect_timeout, infinity}, {user_dir, UserDir}], + SMiscOpt0 = [{user_dir, UserDir}, {system_dir, SystemDir}], + SMiscOpt1 = [{user_dir, UserDir}, {system_dir, SystemDir}], + + basic_test([{client_opts, CMiscOpt0}, {server_opts, SMiscOpt0}]), + basic_test([{client_opts, CMiscOpt1}, {server_opts, SMiscOpt1}]). + +%%-------------------------------------------------------------------- +inet_option() -> + [{doc, "Test configuring IPv4"}]. +inet_option(Config) when is_list(Config) -> + SystemDir = filename:join(?config(priv_dir, Config), system), + UserDir = ?config(priv_dir, Config), - ClientOpts = ?config(client_opts, Config), - ServerOpts = ?config(server_opts, Config), + ClientOpts = [{silently_accept_hosts, true}, + {user_dir, UserDir}, + {user_interaction, false}], + ServerOpts = [{system_dir, SystemDir}, + {user_dir, UserDir}, + {failfun, fun ssh_test_lib:failfun/2}], - basic_test([{client_opts, CMiscOpt0 ++ ClientOpts}, {server_opts, SMiscOpt0 ++ ServerOpts}]), - basic_test([{client_opts, CMiscOpt1 ++ ClientOpts}, {server_opts, SMiscOpt1 ++ ServerOpts}]). + basic_test([{client_opts, [{inet, inet} | ClientOpts]}, + {server_opts, [{inet, inet} | ServerOpts]}]). + +%%-------------------------------------------------------------------- +inet6_option() -> + [{doc, "Test configuring IPv6"}]. +inet6_option(Config) when is_list(Config) -> + SystemDir = filename:join(?config(priv_dir, Config), system), + UserDir = ?config(priv_dir, Config), + + ClientOpts = [{silently_accept_hosts, true}, + {user_dir, UserDir}, + {user_interaction, false}], + ServerOpts = [{system_dir, SystemDir}, + {user_dir, UserDir}, + {failfun, fun ssh_test_lib:failfun/2}], + + basic_test([{client_opts, [{inet, inet6} | ClientOpts]}, + {server_opts, [{inet, inet6} | ServerOpts]}]). %%-------------------------------------------------------------------- exec() -> @@ -620,6 +671,178 @@ double_close(Config) when is_list(Config) -> ok = ssh:close(CM). %%-------------------------------------------------------------------- +ssh_connect_timeout() -> + [{doc, "Test connect_timeout option in ssh:connect/4"}]. +ssh_connect_timeout(_Config) -> + ConnTimeout = 2000, + {error,{faked_transport,connect,TimeoutToTransport}} = + ssh:connect("localhost", 12345, + [{transport,{tcp,?MODULE,tcp_closed}}, + {connect_timeout,ConnTimeout}], + 1000), + case TimeoutToTransport of + ConnTimeout -> ok; + Other -> + ct:log("connect_timeout is ~p but transport received ~p",[ConnTimeout,Other]), + {fail,"ssh:connect/4 wrong connect_timeout received in transport"} + end. + +%% Help for the test above +connect(_Host, _Port, _Opts, Timeout) -> + {error, {faked_transport,connect,Timeout}}. + + +%%-------------------------------------------------------------------- +ssh_connect_arg4_timeout() -> + [{doc, "Test fourth argument in ssh:connect/4"}]. +ssh_connect_arg4_timeout(_Config) -> + Timeout = 1000, + Parent = self(), + %% start the server + Server = spawn(fun() -> + {ok,Sl} = gen_tcp:listen(0,[]), + {ok,{_,Port}} = inet:sockname(Sl), + Parent ! {port,self(),Port}, + Rsa = gen_tcp:accept(Sl), + ct:log("Server gen_tcp:accept got ~p",[Rsa]), + receive after 2*Timeout -> ok end %% let client timeout first + end), + + %% Get listening port + Port = receive + {port,Server,ServerPort} -> ServerPort + end, + + %% try to connect with a timeout, but "supervise" it + Client = spawn(fun() -> + T0 = now(), + Rc = ssh:connect("localhost",Port,[],Timeout), + ct:log("Client ssh:connect got ~p",[Rc]), + Parent ! {done,self(),Rc,T0} + end), + + %% Wait for client reaction on the connection try: + receive + {done, Client, {error,_E}, T0} -> + Msp = ms_passed(T0, now()), + exit(Server,hasta_la_vista___baby), + Low = 0.9*Timeout, + High = 1.1*Timeout, + ct:log("Timeout limits: ~p--~p, timeout was ~p, expected ~p",[Low,High,Msp,Timeout]), + if + Low<Msp, Msp<High -> ok; + true -> {fail, "timeout not within limits"} + end; + {done, Client, {ok,_Ref}, _T0} -> + {fail,"ssh-connected ???"} + after + 5000 -> + exit(Server,hasta_la_vista___baby), + exit(Client,hasta_la_vista___baby), + {fail, "Didn't timeout"} + end. + + +%% Help function +%% N2-N1 +ms_passed(N1={_,_,M1}, N2={_,_,M2}) -> + {0,{0,Min,Sec}} = calendar:time_difference(calendar:now_to_local_time(N1), + calendar:now_to_local_time(N2)), + 1000 * (Min*60 + Sec + (M2-M1)/1000000). + +%%-------------------------------------------------------------------- +ssh_connect_negtimeout_parallel(Config) -> ssh_connect_negtimeout(Config,true). +ssh_connect_negtimeout_sequential(Config) -> ssh_connect_negtimeout(Config,false). + +ssh_connect_negtimeout(Config, Parallel) -> + process_flag(trap_exit, true), + SystemDir = filename:join(?config(priv_dir, Config), system), + UserDir = ?config(priv_dir, Config), + NegTimeOut = 2000, % ms + ct:log("Parallel: ~p",[Parallel]), + + {_Pid, Host, Port} = ssh_test_lib:daemon([{system_dir, SystemDir},{user_dir, UserDir}, + {parallel_login, Parallel}, + {negotiation_timeout, NegTimeOut}, + {failfun, fun ssh_test_lib:failfun/2}]), + + {ok,Socket} = gen_tcp:connect(Host, Port, []), + ct:pal("And now sleeping 1.2*NegTimeOut (~p ms)...", [round(1.2 * NegTimeOut)]), + receive after round(1.2 * NegTimeOut) -> ok end, + + case inet:sockname(Socket) of + {ok,_} -> ct:fail("Socket not closed"); + {error,_} -> ok + end. + +%%-------------------------------------------------------------------- +ssh_connect_nonegtimeout_connected_parallel() -> + [{doc, "Test that ssh connection does not timeout if the connection is established (parallel)"}]. +ssh_connect_nonegtimeout_connected_parallel(Config) -> + ssh_connect_nonegtimeout_connected(Config, true). + +ssh_connect_nonegtimeout_connected_sequential() -> + [{doc, "Test that ssh connection does not timeout if the connection is established (non-parallel)"}]. +ssh_connect_nonegtimeout_connected_sequential(Config) -> + ssh_connect_nonegtimeout_connected(Config, false). + + +ssh_connect_nonegtimeout_connected(Config, Parallel) -> + process_flag(trap_exit, true), + SystemDir = filename:join(?config(priv_dir, Config), system), + UserDir = ?config(priv_dir, Config), + NegTimeOut = 20000, % ms + ct:log("Parallel: ~p",[Parallel]), + + {_Pid, _Host, Port} = ssh_test_lib:daemon([{system_dir, SystemDir},{user_dir, UserDir}, + {parallel_login, Parallel}, + {negotiation_timeout, NegTimeOut}, + {failfun, fun ssh_test_lib:failfun/2}]), + ct:sleep(500), + + IO = ssh_test_lib:start_io_server(), + Shell = ssh_test_lib:start_shell(Port, IO, UserDir), + receive + {'EXIT', _, _} -> + ct:fail(no_ssh_connection); + ErlShellStart -> + ct:pal("---Erlang shell start: ~p~n", [ErlShellStart]), + one_shell_op(IO, NegTimeOut), + one_shell_op(IO, NegTimeOut), + ct:pal("And now sleeping 1.2*NegTimeOut (~p ms)...", [round(1.2 * NegTimeOut)]), + receive after round(1.2 * NegTimeOut) -> ok end, + one_shell_op(IO, NegTimeOut) + end, + exit(Shell, kill). + + +one_shell_op(IO, TimeOut) -> + ct:pal("One shell op: Waiting for prompter"), + receive + ErlPrompt0 -> ct:log("Erlang prompt: ~p~n", [ErlPrompt0]) + after TimeOut -> ct:fail("Timeout waiting for promter") + end, + + IO ! {input, self(), "2*3*7.\r\n"}, + receive + Echo0 -> ct:log("Echo: ~p ~n", [Echo0]) + after TimeOut -> ct:fail("Timeout waiting for echo") + end, + + receive + ?NEWLINE -> ct:log("NEWLINE received", []) + after TimeOut -> + receive Any1 -> ct:log("Bad NEWLINE: ~p",[Any1]) + after 0 -> ct:fail("Timeout waiting for NEWLINE") + end + end, + + receive + Result0 -> ct:log("Result: ~p~n", [Result0]) + after TimeOut -> ct:fail("Timeout waiting for result") + end. + +%%-------------------------------------------------------------------- openssh_zlib_basic_test() -> [{doc, "Test basic connection with openssh_zlib"}]. @@ -639,6 +862,96 @@ openssh_zlib_basic_test(Config) -> ssh:stop_daemon(Pid). %%-------------------------------------------------------------------- + +max_sessions_ssh_connect_parallel(Config) -> + max_sessions(Config, true, connect_fun(ssh__connect,Config)). +max_sessions_ssh_connect_sequential(Config) -> + max_sessions(Config, false, connect_fun(ssh__connect,Config)). + +max_sessions_sftp_start_channel_parallel(Config) -> + max_sessions(Config, true, connect_fun(ssh_sftp__start_channel, Config)). +max_sessions_sftp_start_channel_sequential(Config) -> + max_sessions(Config, false, connect_fun(ssh_sftp__start_channel, Config)). + + +%%%---- helpers: +connect_fun(ssh__connect, Config) -> + fun(Host,Port) -> + ssh_test_lib:connect(Host, Port, + [{silently_accept_hosts, true}, + {user_dir, ?config(priv_dir,Config)}, + {user_interaction, false}, + {user, "carni"}, + {password, "meat"} + ]) + %% ssh_test_lib returns R when ssh:connect returns {ok,R} + end; +connect_fun(ssh_sftp__start_channel, _Config) -> + fun(Host,Port) -> + {ok,_Pid,ConnRef} = + ssh_sftp:start_channel(Host, Port, + [{silently_accept_hosts, true}, + {user, "carni"}, + {password, "meat"} + ]), + ConnRef + end. + + +max_sessions(Config, ParallelLogin, Connect) when is_function(Connect,2) -> + SystemDir = filename:join(?config(priv_dir, Config), system), + UserDir = ?config(priv_dir, Config), + MaxSessions = 5, + {Pid, Host, Port} = ssh_test_lib:daemon([ + {system_dir, SystemDir}, + {user_dir, UserDir}, + {user_passwords, [{"carni", "meat"}]}, + {parallel_login, ParallelLogin}, + {max_sessions, MaxSessions} + ]), + + try [Connect(Host,Port) || _ <- lists:seq(1,MaxSessions)] + of + Connections -> + %% Step 1 ok: could set up max_sessions connections + ct:log("Connections up: ~p",[Connections]), + [_|_] = Connections, + + %% Now try one more than alowed: + ct:pal("Info Report might come here...",[]), + try Connect(Host,Port) + of + _ConnectionRef1 -> + ssh:stop_daemon(Pid), + {fail,"Too many connections accepted"} + catch + error:{badmatch,{error,"Connection closed"}} -> + %% Step 2 ok: could not set up max_sessions+1 connections + %% This is expected + %% Now stop one connection and try to open one more + ok = ssh:close(hd(Connections)), + try Connect(Host,Port) + of + _ConnectionRef1 -> + %% Step 3 ok: could set up one more connection after killing one + %% Thats good. + ssh:stop_daemon(Pid), + ok + catch + error:{badmatch,{error,"Connection closed"}} -> + %% Bad indeed. Could not set up one more connection even after killing + %% one existing. Very bad. + ssh:stop_daemon(Pid), + {fail,"Does not decrease # active sessions"} + end + end + catch + error:{badmatch,{error,"Connection closed"}} -> + ssh:stop_daemon(Pid), + {fail,"Too few connections accepted"} + end. + +%%-------------------------------------------------------------------- %% Internal functions ------------------------------------------------ %%-------------------------------------------------------------------- diff --git a/lib/ssh/test/ssh_connection_SUITE.erl b/lib/ssh/test/ssh_connection_SUITE.erl index f4f0682b40..c52b91986b 100644 --- a/lib/ssh/test/ssh_connection_SUITE.erl +++ b/lib/ssh/test/ssh_connection_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2013. All Rights Reserved. +%% Copyright Ericsson AB 2008-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -37,7 +37,10 @@ suite() -> all() -> [ {group, openssh_payload}, - interrupted_send + interrupted_send, + start_shell, + start_shell_exec, + start_shell_exec_fun ]. groups() -> [{openssh_payload, [], [simple_exec, @@ -276,6 +279,106 @@ interrupted_send(Config) when is_list(Config) -> ssh:stop_daemon(Pid). %%-------------------------------------------------------------------- +start_shell() -> + [{doc, "Start a shell"}]. + +start_shell(Config) when is_list(Config) -> + PrivDir = ?config(priv_dir, Config), + UserDir = filename:join(PrivDir, nopubkey), % to make sure we don't use public-key-auth + file:make_dir(UserDir), + SysDir = ?config(data_dir, Config), + {Pid, Host, Port} = ssh_test_lib:daemon([{system_dir, SysDir}, + {user_dir, UserDir}, + {password, "morot"}, + {shell, fun(U, H) -> start_our_shell(U, H) end} ]), + + ConnectionRef = ssh_test_lib:connect(Host, Port, [{silently_accept_hosts, true}, + {user, "foo"}, + {password, "morot"}, + {user_interaction, true}, + {user_dir, UserDir}]), + + {ok, ChannelId0} = ssh_connection:session_channel(ConnectionRef, infinity), + ok = ssh_connection:shell(ConnectionRef,ChannelId0), + + receive + {ssh_cm, ConnectionRef, {data, _ChannelId, 0, <<"Enter command\r\n">>}} -> + ok + after 5000 -> + ct:fail("CLI Timeout") + end, + + ssh:close(ConnectionRef), + ssh:stop_daemon(Pid). +%%-------------------------------------------------------------------- +start_shell_exec() -> + [{doc, "start shell to exec command"}]. + +start_shell_exec(Config) when is_list(Config) -> + PrivDir = ?config(priv_dir, Config), + UserDir = filename:join(PrivDir, nopubkey), % to make sure we don't use public-key-auth + file:make_dir(UserDir), + SysDir = ?config(data_dir, Config), + {Pid, Host, Port} = ssh_test_lib:daemon([{system_dir, SysDir}, + {user_dir, UserDir}, + {password, "morot"}, + {exec, {?MODULE,ssh_exec,[]}} ]), + + ConnectionRef = ssh_test_lib:connect(Host, Port, [{silently_accept_hosts, true}, + {user, "foo"}, + {password, "morot"}, + {user_interaction, true}, + {user_dir, UserDir}]), + + {ok, ChannelId0} = ssh_connection:session_channel(ConnectionRef, infinity), + + success = ssh_connection:exec(ConnectionRef, ChannelId0, + "testing", infinity), + receive + {ssh_cm, ConnectionRef, {data, _ChannelId, 0, <<"testing\r\n">>}} -> + ok + after 5000 -> + ct:fail("Exec Timeout") + end, + + ssh:close(ConnectionRef), + ssh:stop_daemon(Pid). + +%%-------------------------------------------------------------------- +start_shell_exec_fun() -> + [{doc, "start shell to exec command"}]. + +start_shell_exec_fun(Config) when is_list(Config) -> + PrivDir = ?config(priv_dir, Config), + UserDir = filename:join(PrivDir, nopubkey), % to make sure we don't use public-key-auth + file:make_dir(UserDir), + SysDir = ?config(data_dir, Config), + {Pid, Host, Port} = ssh_test_lib:daemon([{system_dir, SysDir}, + {user_dir, UserDir}, + {password, "morot"}, + {exec, fun ssh_exec/1}]), + + ConnectionRef = ssh_test_lib:connect(Host, Port, [{silently_accept_hosts, true}, + {user, "foo"}, + {password, "morot"}, + {user_interaction, true}, + {user_dir, UserDir}]), + + {ok, ChannelId0} = ssh_connection:session_channel(ConnectionRef, infinity), + + success = ssh_connection:exec(ConnectionRef, ChannelId0, + "testing", infinity), + + receive + {ssh_cm, ConnectionRef, {data, _ChannelId, 0, <<"testing\r\n">>}} -> + ok + after 5000 -> + ct:fail("Exec Timeout") + end, + + ssh:close(ConnectionRef), + ssh:stop_daemon(Pid). +%%-------------------------------------------------------------------- %% Internal functions ------------------------------------------------ %%-------------------------------------------------------------------- big_cat_rx(ConnectionRef, ChannelId) -> @@ -308,3 +411,16 @@ collect_data(ConnectionRef, ChannelId, Acc) -> after 5000 -> timeout end. + +%%%------------------------------------------------------------------- +% This is taken from the ssh example code. +start_our_shell(_User, _Peer) -> + spawn(fun() -> + io:format("Enter command\n") + %% Don't actually loop, just exit + end). + +ssh_exec(Cmd) -> + spawn(fun() -> + io:format(Cmd ++ "\n") + end). diff --git a/lib/ssh/test/ssh_property_test_SUITE.erl b/lib/ssh/test/ssh_property_test_SUITE.erl new file mode 100644 index 0000000000..ffad8ebbb7 --- /dev/null +++ b/lib/ssh/test/ssh_property_test_SUITE.erl @@ -0,0 +1,107 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2004-2014. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% + +%%% Run like this: +%%% ct:run_test([{suite,"ssh_property_test_SUITE"}, {logdir,"/ldisk/OTP/LOG"}]). + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%%% %%% +%%% WARNING %%% +%%% %%% +%%% This is experimental code which may be changed or removed %%% +%%% anytime without any warning. %%% +%%% %%% +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +-module(ssh_property_test_SUITE). + +-compile(export_all). + +-include_lib("common_test/include/ct.hrl"). + +all() -> [{group, messages}, + {group, client_server} + ]. + +groups() -> + [{messages, [], [decode, + decode_encode]}, + {client_server, [], [client_server_sequential, + client_server_parallel, + client_server_parallel_multi]} + ]. + + +%%% First prepare Config and compile the property tests for the found tool: +init_per_suite(Config) -> + ct_property_test:init_per_suite(Config). + +%%% One group in this suite happens to support only QuickCheck, so skip it +%%% if we run proper. +init_per_group(client_server, Config) -> + case ?config(property_test_tool,Config) of + eqc -> Config; + X -> {skip, lists:concat([X," is not supported"])} + end; +init_per_group(_, Config) -> + Config. + +end_per_group(_, Config) -> + Config. + +%%% Always skip the testcase that is not quite in phase with the +%%% ssh_message.erl code +init_per_testcase(decode_encode, _) -> {skip, "Fails - testcase is not ok"}; +init_per_testcase(_TestCase, Config) -> Config. + +end_per_testcase(_TestCase, Config) -> Config. + +%%%================================================================ +%%% Test suites +%%% +decode(Config) -> + ct_property_test:quickcheck( + ssh_eqc_encode_decode:prop_ssh_decode(), + Config + ). + +decode_encode(Config) -> + ct_property_test:quickcheck( + ssh_eqc_encode_decode:prop_ssh_decode_encode(), + Config + ). + +client_server_sequential(Config) -> + ct_property_test:quickcheck( + ssh_eqc_client_server:prop_seq(Config), + Config + ). + +client_server_parallel(Config) -> + ct_property_test:quickcheck( + ssh_eqc_client_server:prop_parallel(Config), + Config + ). + +client_server_parallel_multi(Config) -> + ct_property_test:quickcheck( + ssh_eqc_client_server:prop_parallel_multi(Config), + Config + ). diff --git a/lib/ssh/test/ssh_to_openssh_SUITE.erl b/lib/ssh/test/ssh_to_openssh_SUITE.erl index 8b5343cecc..41fbd324c4 100644 --- a/lib/ssh/test/ssh_to_openssh_SUITE.erl +++ b/lib/ssh/test/ssh_to_openssh_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2013. All Rights Reserved. +%% Copyright Ericsson AB 2008-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -54,7 +54,9 @@ groups() -> ]}, {erlang_server, [], [erlang_server_openssh_client_exec, erlang_server_openssh_client_exec_compressed, - erlang_server_openssh_client_pulic_key_dsa]} + erlang_server_openssh_client_pulic_key_dsa, + erlang_server_openssh_client_cipher_suites, + erlang_server_openssh_client_macs]} ]. init_per_suite(Config) -> @@ -89,6 +91,12 @@ end_per_group(erlang_server, Config) -> end_per_group(_, Config) -> Config. +init_per_testcase(erlang_server_openssh_client_cipher_suites, Config) -> + check_ssh_client_support(Config); + +init_per_testcase(erlang_server_openssh_client_macs, Config) -> + check_ssh_client_support(Config); + init_per_testcase(_TestCase, Config) -> ssh:start(), Config. @@ -111,15 +119,7 @@ erlang_shell_client_openssh_server(Config) when is_list(Config) -> IO ! {input, self(), "echo Hej\n"}, receive_hej(), IO ! {input, self(), "exit\n"}, - receive - <<"logout">> -> - receive - <<"Connection closed">> -> - ok - end; - Other0 -> - ct:fail({unexpected_msg, Other0}) - end, + receive_logout(), receive {'EXIT', Shell, normal} -> ok; @@ -221,6 +221,108 @@ erlang_server_openssh_client_exec(Config) when is_list(Config) -> ssh:stop_daemon(Pid). %%-------------------------------------------------------------------- +erlang_server_openssh_client_cipher_suites() -> + [{doc, "Test that we can connect with different cipher suites."}]. + +erlang_server_openssh_client_cipher_suites(Config) when is_list(Config) -> + SystemDir = ?config(data_dir, Config), + PrivDir = ?config(priv_dir, Config), + KnownHosts = filename:join(PrivDir, "known_hosts"), + + {Pid, Host, Port} = ssh_test_lib:daemon([{system_dir, SystemDir}, + {failfun, fun ssh_test_lib:failfun/2}]), + + + ct:sleep(500), + + Supports = crypto:supports(), + Ciphers = proplists:get_value(ciphers, Supports), + Tests = [ + {"3des-cbc", lists:member(des3_cbc, Ciphers)}, + {"aes128-cbc", lists:member(aes_cbc128, Ciphers)}, + {"aes128-ctr", lists:member(aes_ctr, Ciphers)}, + {"aes256-cbc", false} + ], + lists:foreach(fun({Cipher, Expect}) -> + Cmd = "ssh -p " ++ integer_to_list(Port) ++ + " -o UserKnownHostsFile=" ++ KnownHosts ++ " " ++ Host ++ " " ++ + " -c " ++ Cipher ++ " 1+1.", + + ct:pal("Cmd: ~p~n", [Cmd]), + + SshPort = open_port({spawn, Cmd}, [binary, stderr_to_stdout]), + + case Expect of + true -> + receive + {SshPort,{data, <<"2\n">>}} -> + ok + after ?TIMEOUT -> + ct:fail("Did not receive answer") + end; + false -> + receive + {SshPort,{data, <<"no matching cipher found", _/binary>>}} -> + ok + after ?TIMEOUT -> + ct:fail("Did not receive no matching cipher message") + end + end + end, Tests), + + ssh:stop_daemon(Pid). + +%%-------------------------------------------------------------------- +erlang_server_openssh_client_macs() -> + [{doc, "Test that we can connect with different MACs."}]. + +erlang_server_openssh_client_macs(Config) when is_list(Config) -> + SystemDir = ?config(data_dir, Config), + PrivDir = ?config(priv_dir, Config), + KnownHosts = filename:join(PrivDir, "known_hosts"), + + {Pid, Host, Port} = ssh_test_lib:daemon([{system_dir, SystemDir}, + {failfun, fun ssh_test_lib:failfun/2}]), + + + ct:sleep(500), + + Supports = crypto:supports(), + Hashs = proplists:get_value(hashs, Supports), + MACs = [{"hmac-sha1", lists:member(sha, Hashs)}, + {"hmac-sha2-256", lists:member(sha256, Hashs)}, + {"hmac-md5-96", false}, + {"hmac-ripemd160", false}], + lists:foreach(fun({MAC, Expect}) -> + Cmd = "ssh -p " ++ integer_to_list(Port) ++ + " -o UserKnownHostsFile=" ++ KnownHosts ++ " " ++ Host ++ " " ++ + " -o MACs=" ++ MAC ++ " 1+1.", + + ct:pal("Cmd: ~p~n", [Cmd]), + + SshPort = open_port({spawn, Cmd}, [binary, stderr_to_stdout]), + + case Expect of + true -> + receive + {SshPort,{data, <<"2\n">>}} -> + ok + after ?TIMEOUT -> + ct:fail("Did not receive answer") + end; + false -> + receive + {SshPort,{data, <<"no matching mac found", _/binary>>}} -> + ok + after ?TIMEOUT -> + ct:fail("Did not receive no matching mac message") + end + end + end, MACs), + + ssh:stop_daemon(Pid). + +%%-------------------------------------------------------------------- erlang_server_openssh_client_exec_compressed() -> [{doc, "Test that exec command works."}]. @@ -433,3 +535,43 @@ receive_hej() -> ct:pal("Extra info: ~p~n", [Info]), receive_hej() end. + +receive_logout() -> + receive + <<"logout">> -> + receive + <<"Connection closed">> -> + ok + end; + <<"TERM environment variable not set.\n">> -> %% Windows work around + receive_logout(); + Other0 -> + ct:fail({unexpected_msg, Other0}) + end. + + + +%%-------------------------------------------------------------------- +%%-------------------------------------------------------------------- +%% Check if we have a "newer" ssh client that supports these test cases +%%-------------------------------------------------------------------- +check_ssh_client_support(Config) -> + Port = open_port({spawn, "ssh -Q cipher"}, [exit_status, stderr_to_stdout]), + case check_ssh_client_support2(Port) of + 0 -> % exit status from command (0 == ok) + ssh:start(), + Config; + _ -> + {skip, "test case not supported by ssh client"} + end. + +check_ssh_client_support2(P) -> + receive + {P, {data, _A}} -> + check_ssh_client_support2(P); + {P, {exit_status, E}} -> + E + after 5000 -> + ct:pal("Openssh command timed out ~n"), + -1 + end. diff --git a/lib/ssh/vsn.mk b/lib/ssh/vsn.mk index 9ffc59dbaf..73bf73971f 100644 --- a/lib/ssh/vsn.mk +++ b/lib/ssh/vsn.mk @@ -1,5 +1,5 @@ #-*-makefile-*- ; force emacs to enter makefile-mode -SSH_VSN = 3.0.1 +SSH_VSN = 3.0.5 APP_VSN = "ssh-$(SSH_VSN)" diff --git a/lib/ssl/doc/src/notes.xml b/lib/ssl/doc/src/notes.xml index 0b28b1ebd4..8643cd3745 100644 --- a/lib/ssl/doc/src/notes.xml +++ b/lib/ssl/doc/src/notes.xml @@ -25,7 +25,316 @@ <file>notes.xml</file> </header> <p>This document describes the changes made to the SSL application.</p> - <section><title>SSL 5.3.3</title> + <section><title>SSL 5.3.6</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Corrected handling of ECC certificates, there where + several small issues with the handling of such + certificates in the ssl and public_key application. Now + ECC signed ECC certificates shall work and not only RSA + signed ECC certificates.</p> + <p> + Own Id: OTP-12026</p> + </item> + <item> + <p> + Check that the certificate chain ends with a trusted ROOT + CA e.i. a self-signed certificate, but provide an option + partial_chain to enable the application to define an + intermediat CA as trusted.</p> + <p> + Own Id: OTP-12149</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Add decode functions for SNI (Server Name Indication)</p> + <p> + Own Id: OTP-12048</p> + </item> + </list> + </section> + +</section> + +<section><title>SSL 5.3.5</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + ssl:recv now returns {error, einval} if applied to a non + passive socket, the same as gen_tcp:recv. </p> + <p> + Thanks to Danil Zagoskin for reporting this issue</p> + <p> + Own Id: OTP-11878</p> + </item> + <item> + <p> + Corrected handling of default values for + signature_algorithms extension in TLS-1.2 and + corresponding values used in previous versions that does + not support this extension. </p> + <p> + Thanks to Danil Zagoskin</p> + <p> + Own Id: OTP-11886</p> + </item> + <item> + <p> + Handle socket option inheritance when pooling of accept + sockets is used</p> + <p> + Own Id: OTP-11897</p> + </item> + <item> + <p> + Make sure that the list of versions, possibly supplied in + the versions option, is not order dependent.</p> + <p> + Thanks to Ransom Richardson for reporting this issue</p> + <p> + Own Id: OTP-11912</p> + </item> + <item> + <p> + Reject connection if the next_protocol message is sent + twice.</p> + <p> + Own Id: OTP-11926</p> + </item> + <item> + <p> + Correct options handling when ssl:ssl_accept/3 is called + with new ssl options after calling ssl:listen/2</p> + <p> + Own Id: OTP-11950</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Gracefully handle unknown alerts</p> + <p> + Thanks to Atul Atri for reporting this issue</p> + <p> + Own Id: OTP-11874</p> + </item> + <item> + <p> + Gracefully ignore cipher suites sent by client not + supported by the SSL/TLS version that the client has + negotiated.</p> + <p> + Thanks to Danil Zagoskin for reporting this issue</p> + <p> + Own Id: OTP-11875</p> + </item> + <item> + <p> + Gracefully handle structured garbage, i.e a client sends + some garbage in a ssl record instead of a valid fragment.</p> + <p> + Thanks to Danil Zagoskin</p> + <p> + Own Id: OTP-11880</p> + </item> + <item> + <p> + Gracefully handle invalid alerts</p> + <p> + Own Id: OTP-11890</p> + </item> + <item> + <p> + Generalize handling of default ciphers</p> + <p> + Thanks to Andreas Schultz</p> + <p> + Own Id: OTP-11966</p> + </item> + <item> + <p> + Make sure change cipher spec is correctly handled</p> + <p> + Own Id: OTP-11975</p> + </item> + </list> + </section> + +</section> + +<section><title>SSL 5.3.4</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fix incorrect dialyzer spec and types, also enhance + documentation. </p> + <p> + Thanks to Ayaz Tuncer.</p> + <p> + Own Id: OTP-11627</p> + </item> + <item> + <p> + Fix possible mismatch between SSL/TLS version and default + ciphers. Could happen when you specified SSL/TLS-version + in optionlist to listen or accept.</p> + <p> + Own Id: OTP-11712</p> + </item> + <item> + <p> + Application upgrade (appup) files are corrected for the + following applications: </p> + <p> + <c>asn1, common_test, compiler, crypto, debugger, + dialyzer, edoc, eldap, erl_docgen, et, eunit, gs, hipe, + inets, observer, odbc, os_mon, otp_mibs, parsetools, + percept, public_key, reltool, runtime_tools, ssh, + syntax_tools, test_server, tools, typer, webtool, wx, + xmerl</c></p> + <p> + A new test utility for testing appup files is added to + test_server. This is now used by most applications in + OTP.</p> + <p> + (Thanks to Tobias Schlager)</p> + <p> + Own Id: OTP-11744</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Moved elliptic curve definition from the crypto + NIF/OpenSSL into Erlang code, adds the RFC-5639 brainpool + curves and makes TLS use them (RFC-7027).</p> + <p> + Thanks to Andreas Schultz</p> + <p> + Own Id: OTP-11578</p> + </item> + <item> + <p> + Unicode adaptations</p> + <p> + Own Id: OTP-11620</p> + </item> + <item> + <p> + Added option honor_cipher_order. This instructs the + server to prefer its own cipher ordering rather than the + client's and can help protect against things like BEAST + while maintaining compatability with clients which only + support older ciphers. </p> + <p> + Thanks to Andrew Thompson for the implementation, and + Andreas Schultz for the test cases.</p> + <p> + Own Id: OTP-11621</p> + </item> + <item> + <p> + Replace boolean checking in validate_option with + is_boolean guard. </p> + <p> + Thanks to Andreas Schultz.</p> + <p> + Own Id: OTP-11634</p> + </item> + <item> + <p> + Some function specs are corrected or moved and some edoc + comments are corrected in order to allow use of edoc. + (Thanks to Pierre Fenoll)</p> + <p> + Own Id: OTP-11702</p> + </item> + <item> + <p> + Correct clean up of certificate database when certs are + inputed in pure DER format.The incorrect code could cause + a memory leek when certs where inputed in DER. Thanks to + Bernard Duggan for reporting this.</p> + <p> + Own Id: OTP-11733</p> + </item> + <item> + <p> + Improved documentation of the cacertfile option</p> + <p> + Own Id: OTP-11759 Aux Id: seq12535 </p> + </item> + <item> + <p> + Avoid next protocol negotiation failure due to incorrect + option format.</p> + <p> + Own Id: OTP-11760</p> + </item> + <item> + <p> + Handle v1 CRLs, with no extensions and fixes issues with + IDP (Issuing Distribution Point) comparison during CRL + validation. </p> + <p> + Thanks to Andrew Thompson</p> + <p> + Own Id: OTP-11761</p> + </item> + <item> + <p> + Server now ignores client ECC curves that it does not + support instead of crashing. </p> + <p> + Thanks to Danil Zagoskin for reporting the issue and + suggesting a solution.</p> + <p> + Own Id: OTP-11780</p> + </item> + <item> + <p> + Handle SNI (Server Name Indication) alert + unrecognized_name and gracefully deal with unexpected + alerts. </p> + <p> + Thanks to Masatake Daimon for reporting this.</p> + <p> + Own Id: OTP-11815</p> + </item> + <item> + <p> + Add possibility to specify ssl options when calling + ssl:ssl_accept</p> + <p> + Own Id: OTP-11837</p> + </item> + </list> + </section> + +</section> + +<section><title>SSL 5.3.3</title> <section><title>Fixed Bugs and Malfunctions</title> <list> diff --git a/lib/ssl/doc/src/ssl.xml b/lib/ssl/doc/src/ssl.xml index 4bc1a9a644..f14d0b8bb7 100644 --- a/lib/ssl/doc/src/ssl.xml +++ b/lib/ssl/doc/src/ssl.xml @@ -226,7 +226,7 @@ <p>The verification fun should be defined as:</p> <code> -fun(OtpCert :: #'OTPCertificate'{}, Event :: {bad_cert, Reason :: atom()} | +fun(OtpCert :: #'OTPCertificate'{}, Event :: {bad_cert, Reason :: atom() | {revoked, atom()}} | {extension, #'Extension'{}}, InitialUserState :: term()) -> {valid, UserState :: term()} | {valid_peer, UserState :: term()} | {fail, Reason :: term()} | {unknown, UserState :: term()}. @@ -252,7 +252,7 @@ fun(OtpCert :: #'OTPCertificate'{}, Event :: {bad_cert, Reason :: atom()} | always returns {valid, UserState}, the TLS/SSL handshake will not be terminated with respect to verification failures and the connection will be established. If called with an - extension unknown to the user application the return value + extension unknown to the user application, the return value {unknown, UserState} should be used.</p> <p>The default verify_fun option in verify_peer mode:</p> @@ -283,9 +283,29 @@ fun(OtpCert :: #'OTPCertificate'{}, Event :: {bad_cert, Reason :: atom()} | end, []} </code> -<p>Possible path validation errors: </p> + <p>Possible path validation errors are given on the form {bad_cert, Reason} where Reason is:</p> -<p> {bad_cert, cert_expired}, {bad_cert, invalid_issuer}, {bad_cert, invalid_signature}, {bad_cert, unknown_ca},{bad_cert, selfsigned_peer}, {bad_cert, name_not_permitted}, {bad_cert, missing_basic_constraint}, {bad_cert, invalid_key_usage}</p> + <taglist> + <tag>unknown_ca</tag> + <item>No trusted CA was found in the trusted store. The trusted CA is + normally a so called ROOT CA that is a self-signed cert. Trust may + be claimed for an intermediat CA (trusted anchor does not have to be self signed + according to X-509) by using the option <c>partial_chain</c></item> + + <tag>selfsigned_peer</tag> + <item>The chain consisted only of one self-signed certificate.</item> + + <tag>PKIX X-509-path validation error</tag> + <item> Possible such reasons see <seealso + marker="public_key#pkix_path_validation-3"> public_key:pkix_path_validation/3 </seealso></item> + </taglist> + + </item> + + <tag>{partial_chain, fun(Chain::[DerCert]) -> {trusted_ca, DerCert} | unknown_ca </tag> + <item> + Claim an intermediat CA in the chain as trusted. TLS will then perform the public_key:pkix_path_validation/3 + with the selected CA as trusted anchor and the rest of the chain. </item> <tag>{versions, [protocol()]}</tag> @@ -768,39 +788,45 @@ fun(srp, Username :: string(), UserState :: term()) -> </func> <func> - <name>ssl_accept(ListenSocket) -> </name> - <name>ssl_accept(ListenSocket, Timeout) -> ok | {error, Reason}</name> - <fsummary>Perform server-side SSL handshake</fsummary> + <name>ssl_accept(Socket) -> </name> + <name>ssl_accept(Socket, Timeout) -> ok | {error, Reason}</name> + <fsummary>Perform server-side SSL/TLS handshake</fsummary> <type> - <v>ListenSocket = sslsocket()</v> + <v>Socket = sslsocket()</v> <v>Timeout = integer()</v> <v>Reason = term()</v> </type> <desc> - <p>The <c>ssl_accept</c> function establish the SSL connection - on the server side. It should be called directly after - <c>transport_accept</c>, in the spawned server-loop.</p> + <p> Performs the SSL/TLS server-side handshake <c>Socket</c> is a socket as returned + by <seealso + marker="#transport_accept-2">ssl:transport_accept/[1,2]</seealso> + </p> </desc> </func> <func> - <name>ssl_accept(ListenSocket, SslOptions) -> </name> - <name>ssl_accept(ListenSocket, SslOptions, Timeout) -> {ok, Socket} | {error, Reason}</name> - <fsummary>Perform server-side SSL handshake</fsummary> + <name>ssl_accept(Socket, SslOptions) -> </name> + <name>ssl_accept(Socket, SslOptions, Timeout) -> {ok, Socket} | ok | {error, Reason}</name> + <fsummary>Perform server-side SSL/TLS handshake</fsummary> <type> - <v>ListenSocket = socket()</v> + <v>Socket = socket() | sslsocket() </v> <v>SslOptions = ssloptions()</v> <v>Timeout = integer()</v> <v>Reason = term()</v> </type> <desc> - <p> Upgrades a gen_tcp, or - equivalent, socket to an ssl socket i.e. performs the - ssl server-side handshake.</p> + <p> If <c>Socket</c> is a socket() - upgrades a gen_tcp, or equivalent, socket to an ssl socket + i.e. performs the SSL/TLS server-side handshake and returns the ssl socket. + </p> + <warning><p>Note that the listen socket should be in {active, false} mode before telling the client that the server is ready to upgrade - and calling this function, otherwise the upgrade may + by calling this function, otherwise the upgrade may or may not succeed depending on timing.</p></warning> + + <p> If <c>Socket</c> is an sslsocket() - provides additional SSL/TLS options to those specified in <seealso + marker="#listen-2">ssl:listen/2 </seealso> and then performs the SSL/TLS handshake. + </p> </desc> </func> @@ -842,33 +868,38 @@ fun(srp, Username :: string(), UserState :: term()) -> </func> <func> - <name>transport_accept(Socket) -></name> - <name>transport_accept(Socket, Timeout) -> + <name>transport_accept(ListenSocket) -></name> + <name>transport_accept(ListenSocket, Timeout) -> {ok, NewSocket} | {error, Reason}</name> <fsummary>Accept an incoming connection and prepare for <c>ssl_accept</c></fsummary> <type> - <v>Socket = NewSocket = sslsocket()</v> + <v>ListenSocket = NewSocket = sslsocket()</v> <v>Timeout = integer()</v> <v>Reason = reason()</v> </type> <desc> <p>Accepts an incoming connection request on a listen socket. - <c>ListenSocket</c> must be a socket returned from - <c>listen/2</c>. The socket returned should be passed to - <c>ssl_accept</c> to complete ssl handshaking and - establishing the connection.</p> + <c>ListenSocket</c> must be a socket returned from + <seealso + marker="#listen-2"> ssl:listen/2</seealso>. + The socket returned should be passed to + <seealso marker="#ssl_accept-2"> ssl:ssl_accept[2,3]</seealso> + to complete handshaking i.e + establishing the SSL/TLS connection.</p> <warning> - <p>The socket returned can only be used with <c>ssl_accept</c>, - no traffic can be sent or received before that call.</p> + <p>The socket returned can only be used with + <seealso marker="#ssl_accept-2"> ssl:ssl_accept[2,3]</seealso> + no traffic can be sent or received before that call.</p> </warning> <p>The accepted socket inherits the options set for - <c>ListenSocket</c> in <c>listen/2</c>.</p> + <c>ListenSocket</c> in <seealso + marker="#listen-2"> ssl:listen/2</seealso>.</p> <p>The default - value for <c>Timeout</c> is <c>infinity</c>. If - <c>Timeout</c> is specified, and no connection is accepted - within the given time, <c>{error, timeout}</c> is - returned.</p> + value for <c>Timeout</c> is <c>infinity</c>. If + <c>Timeout</c> is specified, and no connection is accepted + within the given time, <c>{error, timeout}</c> is + returned.</p> </desc> </func> diff --git a/lib/ssl/doc/src/ssl_protocol.xml b/lib/ssl/doc/src/ssl_protocol.xml index cdfafe224b..80d9cc4ee8 100644 --- a/lib/ssl/doc/src/ssl_protocol.xml +++ b/lib/ssl/doc/src/ssl_protocol.xml @@ -83,7 +83,7 @@ <em>subject</em>. The certificate is signed with the private key of the issuer of the certificate. A chain of trust is build by having the issuer in its turn being - certified by an other certificate and so on until you reach the + certified by another certificate and so on until you reach the so called root certificate that is self signed i.e. issued by itself.</p> diff --git a/lib/ssl/internal_doc/ssl-implementation.txt b/lib/ssl/internal_doc/ssl-implementation.txt deleted file mode 100644 index e5d6ac8cd0..0000000000 --- a/lib/ssl/internal_doc/ssl-implementation.txt +++ /dev/null @@ -1,52 +0,0 @@ - -Important modules: - - module behaviour children - ------ --------- - ssl_app application ssl_sup - ssl_sup supervisor ssl_server, ssl_broker_sup - ssl_server gen_server - - ssl_broker_sup supervisor ssl_broker - ssl_broker gen_server - - -The ssl_server controls a port program that implements the SSL functionality. -That port program uses the OpenSSL package. - -Each socket has a corresponding broker (listen, accept or connect). A broker -is created and supervised by the ssl_broker_sup. - -All communication is between a user and a broker. The broker communicates -with the ssl_server, that sends its commands to the port program and handles -the port program responses, that are distributed to users through the -brokers. - -There is a distinction between commands and data flow between the ssl_server -and the port program. Each established connection between the user and the -outside world consists of a local erlang socket (owned by the broker) that -is read from and written to by the broker. At the other end of the local -connection is a local socket in the port program. - -The "real" socket that connects to the outside world is in the port program -(including listen sockets). The main purpose of the port program is to -shuffle data between local sockets and outside world sockets, and detect and -propagate read and write errors (including detection of closed sockets) to -the ssl_server. - -There is documentation in the ssl_broker.erl module. - -There is also documentation in the esock.c and esock_openssl.c files. - -The ssl_pem.erl, ssl_pkix.erl and ssl_base64.erl modules are support -modules for reading SSL certificates. Modules for parsing certificates -are generated from ASN.1 modules in the `pkix' directory. - -The `examples' directory contains functions for generating certificates. -Those certificates are used in the test suites. - - - - - - - - diff --git a/lib/ssl/src/Makefile b/lib/ssl/src/Makefile index 131b615277..7c4c8ec2cc 100644 --- a/lib/ssl/src/Makefile +++ b/lib/ssl/src/Makefile @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 1999-2013. All Rights Reserved. +# Copyright Ericsson AB 1999-2014. All Rights Reserved. # # The contents of this file are subject to the Erlang Public License, # Version 1.1, (the "License"); you may not use this file except in @@ -66,6 +66,7 @@ MODULES= \ ssl_session \ ssl_session_cache \ ssl_socket \ + ssl_listen_tracker_sup \ tls_record \ dtls_record \ ssl_record \ diff --git a/lib/ssl/src/dtls_connection.erl b/lib/ssl/src/dtls_connection.erl index da2e076856..508983ddac 100644 --- a/lib/ssl/src/dtls_connection.erl +++ b/lib/ssl/src/dtls_connection.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2013-2013. All Rights Reserved. +%% Copyright Ericsson AB 2013-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -20,222 +20,515 @@ %% Internal application API -%%==================================================================== +-behaviour(gen_fsm). + +-include("dtls_connection.hrl"). +-include("dtls_handshake.hrl"). +-include("ssl_alert.hrl"). +-include("dtls_record.hrl"). +-include("ssl_cipher.hrl"). +-include("ssl_api.hrl"). +-include("ssl_internal.hrl"). +-include("ssl_srp.hrl"). +-include_lib("public_key/include/public_key.hrl"). + %% Internal application API -%%==================================================================== +%% Setup +-export([start_fsm/8]). + +%% State transition handling +-export([next_record/1, next_state/4%, + %%next_state_connection/2 + ]). + +%% Handshake handling +-export([%%renegotiate/1, + send_handshake/2, send_change_cipher/2]). + +%% Alert and close handling +-export([send_alert/2, handle_own_alert/4, %%handle_close_alert/3, + handle_normal_shutdown/3 + %%handle_unexpected_message/3, + %%alert_user/5, alert_user/8 + ]). + +%% Data handling +-export([%%write_application_data/3, + read_application_data/2%%, +%% passive_receive/2, next_record_if_active/1 + ]). + +%% Called by tls_connection_sup +-export([start_link/7]). +%% gen_fsm callbacks +-export([init/1, hello/2, certify/2, cipher/2, + abbreviated/2, connection/2, handle_event/3, + handle_sync_event/4, handle_info/3, terminate/3, code_change/4]). -%% %%==================================================================== -%% %% State functions -%% %%==================================================================== - -%% -spec hello(start | #hello_request{} | #client_hello{} | #server_hello{} | term(), -%% #state{}) -> gen_fsm_state_return(). -%% %%-------------------------------------------------------------------- -%% hello(start, #state{host = Host, port = Port, role = client, -%% ssl_options = SslOpts, -%% session = #session{own_certificate = Cert} = Session0, -%% session_cache = Cache, session_cache_cb = CacheCb, -%% connection_states = ConnectionStates0, -%% renegotiation = {Renegotiation, _}, -%% client_cookie = Cookie} = State0) -> -%% Hello = dtls_handshake:client_hello(Host, Port, Cookie, ConnectionStates0, SslOpts, -%% Cache, CacheCb, Renegotiation, Cert), - -%% Version = Hello#client_hello.client_version, -%% State1 = State0#state{negotiated_version = Version, %% Requested version -%% session = -%% Session0#session{session_id = Hello#client_hello.session_id}, -%% dtls_handshake_history = ssl_handshake:init_handshake_history()}, - -%% State2 = send_flight(Hello, waiting, State1), - -%% {Record, State} = next_record(State2), -%% next_state(hello, hello, Record, State); - -%% hello(start, #state{role = server} = State0) -> -%% {Record, State} = next_record(State0), -%% next_state(hello, hello, Record, State); - -%% hello(#hello_request{}, #state{role = client} = State0) -> -%% {Record, State} = next_record(State0), -%% next_state(hello, hello, Record, State); - -%% hello(#server_hello{cipher_suite = CipherSuite, -%% compression_method = Compression} = Hello, -%% #state{session = #session{session_id = OldId}, -%% connection_states = ConnectionStates0, -%% role = client, -%% negotiated_version = ReqVersion, -%% renegotiation = {Renegotiation, _}, -%% ssl_options = SslOptions} = State1) -> -%% State0 = flight_done(State1), -%% case ssl_handshake:hello(Hello, SslOptions, ConnectionStates0, Renegotiation) of -%% #alert{} = Alert -> -%% handle_own_alert(Alert, ReqVersion, hello, State0); -%% {Version, NewId, ConnectionStates, NextProtocol} -> -%% {KeyAlgorithm, _, _, _} = -%% ssl_cipher:suite_definition(CipherSuite), - -%% PremasterSecret = make_premaster_secret(ReqVersion, KeyAlgorithm), - -%% NewNextProtocol = case NextProtocol of -%% undefined -> -%% State0#state.next_protocol; -%% _ -> -%% NextProtocol -%% end, - -%% State = State0#state{key_algorithm = KeyAlgorithm, -%% hashsign_algorithm = default_hashsign(Version, KeyAlgorithm), -%% negotiated_version = Version, -%% connection_states = ConnectionStates, -%% premaster_secret = PremasterSecret, -%% expecting_next_protocol_negotiation = NextProtocol =/= undefined, -%% next_protocol = NewNextProtocol}, - -%% case ssl_session:is_new(OldId, NewId) of -%% true -> -%% handle_new_session(NewId, CipherSuite, Compression, -%% State#state{connection_states = ConnectionStates}); -%% false -> -%% handle_resumed_session(NewId, State#state{connection_states = ConnectionStates}) -%% end -%% end; - -%% hello(#hello_verify_request{cookie = Cookie}, -%% #state{host = Host, port = Port, -%% session = #session{own_certificate = Cert}, -%% session_cache = Cache, session_cache_cb = CacheCb, -%% ssl_options = SslOpts, -%% connection_states = ConnectionStates0, -%% renegotiation = {Renegotiation, _}} = State0) -> -%% Hello = ssl_handshake:client_hello(Host, Port, Cookie, ConnectionStates0, SslOpts, -%% Cache, CacheCb, Renegotiation, Cert), -%% State1 = State0#state{ -%% tls_handshake_history = ssl_handshake:init_handshake_history(), -%% client_cookie = Cookie}, -%% State2 = send_flight(Hello, waiting, State1), - -%% {Record, State} = next_record(State2), -%% next_state(hello, hello, Record, State); - - -%% %%-------------------------------------------------------------------- -%% -spec abbreviated(#hello_request{} | #finished{} | term(), -%% #state{}) -> gen_fsm_state_return(). -%% %%-------------------------------------------------------------------- - -%% abbreviated(timeout, State) -> -%% { next_state, abbreviated, State, hibernate }; - -%% abbreviated(Msg, State) -> -%% handle_unexpected_message(Msg, abbreviated, State). - -%% %%-------------------------------------------------------------------- -%% -spec certify(#hello_request{} | #certificate{} | #server_key_exchange{} | -%% #certificate_request{} | #server_hello_done{} | #client_key_exchange{} | term(), -%% #state{}) -> gen_fsm_state_return(). -%% %%-------------------------------------------------------------------- - - -%% certify(timeout, State) -> -%% { next_state, certify, State, hibernate }; - -%% certify(Msg, State) -> -%% handle_unexpected_message(Msg, certify, State). - - -%% %%-------------------------------------------------------------------- -%% -spec cipher(#hello_request{} | #certificate_verify{} | #finished{} | term(), -%% #state{}) -> gen_fsm_state_return(). -%% %%-------------------------------------------------------------------- - -%% cipher(timeout, State) -> -%% { next_state, cipher, State, hibernate }; - -%% cipher(Msg, State) -> -%% handle_unexpected_message(Msg, cipher, State). - -%% %%-------------------------------------------------------------------- -%% -spec connection(#hello_request{} | #client_hello{} | term(), -%% #state{}) -> gen_fsm_state_return(). -%% %%-------------------------------------------------------------------- - -%% connection(timeout, State) -> -%% {next_state, connection, State, hibernate}; - -%% connection(Msg, State) -> -%% handle_unexpected_message(Msg, connection, State). - -%% %%-------------------------------------------------------------------- -%% %%% Internal functions -%% %%-------------------------------------------------------------------- -%% handle_unexpected_message(Msg, Info, #state{negotiated_version = Version} = State) -> -%% Alert = ?ALERT_REC(?FATAL,?UNEXPECTED_MESSAGE), -%% handle_own_alert(Alert, Version, {Info, Msg}, State). - -%% send_flight(HandshakeRec, FlightState, State) -> -%% send_flight(FlightState, buffer_flight(HandshakeRec, State)). - -%% send_flight(FlightState, State = #state{negotiated_version = Version, -%% flight_buffer = Buffer}) -> - -%% State1 = do_send_flight(queue:to_list(Buffer), [], State), -%% finish_send_flight(Version, FlightState, State1). - -%% resend_flight(State = #state{negotiated_version = Version, -%% flight_state = FlightState, -%% flight_buffer = Buffer}) -%% when FlightState == finished; FlightState == waiting -> -%% State1 = do_send_flight(queue:to_list(Buffer), [], State), -%% finish_send_flight(Version, FlightState, State1); - -%% resend_flight(State) -> -%% State. - -%% flight_done(State) -> -%% cancel_dtls_retransmit_timer(State#state{flight_state = done, -%% flight_buffer = undefined}). - -%% do_send_flight([], BinMsgs, State = #state{transport_cb = Transport, socket = Socket}) -> -%% Transport:send(Socket, lists:reverse(BinMsgs)), -%% State; -%% do_send_flight([{Epoch, MsgSeq, HandshakeRec}|T], BinMsgs0, -%% State = #state{negotiated_version = Version, -%% connection_states = ConnectionStates0}) -> -%% CS0 = ssl_record:connection_state_by_epoch(ConnectionStates0, Epoch, write), -%% {BinMsgs, CS1} = encode_handshake_rec(HandshakeRec, Version, MsgSeq, BinMsgs0, CS0), -%% ConnectionStates1 = ssl_record:set_connection_state_by_epoch(ConnectionStates0, CS1, write), -%% do_send_flight(T, BinMsgs, State#state{connection_states = ConnectionStates1}). - -%% cancel_dtls_retransmit_timer(State = #state{dtls_retransmit_timer = TimerRef}) -> -%% cancel_timer(TimerRef), -%% State#state{dtls_retransmit_timer = undefined}. - -%% rearm_dtls_retransmit_timer(State = #state{dtls_retransmit_timer = undefined}) -> -%% TimerRef = erlang:start_timer(1000, self(), dtls_retransmit), -%% State#state{dtls_retransmit_timer = TimerRef}; -%% rearm_dtls_retransmit_timer(State) -> -%% State. - -%% finish_send_flight({254, _}, waiting, State) -> -%% TimerRef = erlang:start_timer(1000, self(), dtls_retransmit), -%% State#state{ -%% dtls_retransmit_timer = TimerRef, -%% last_retransmit = timestamp(), -%% flight_state = waiting}; - -%% finish_send_flight(_, FlightState, State) -> -%% State#state{flight_state = FlightState}. - -%% timestamp() -> -%% {Mega, Sec, Micro} = erlang:now(), -%% Mega * 1000000 * 1000 + Sec * 1000 + (Micro div 1000). - -%% encode_handshake_rec(HandshakeRec, Version, MsgSeq, BinMsgs0, CS0) -> -%% {_, Fragments} = ssl_handshake:encode_handshake(HandshakeRec, Version, MsgSeq, 1400), -%% lists:foldl(fun(F, {Bin, C0}) -> -%% {B, C1} = ssl_record:encode_handshake(F, Version, C0), -%% {[B|Bin], C1} end, {BinMsgs0, CS0}, Fragments). +%%==================================================================== +%% Internal application API +%%==================================================================== +start_fsm(Role, Host, Port, Socket, {#ssl_options{erl_dist = false},_} = Opts, + User, {CbModule, _,_, _} = CbInfo, + Timeout) -> + try + {ok, Pid} = dtls_connection_sup:start_child([Role, Host, Port, Socket, + Opts, User, CbInfo]), + {ok, SslSocket} = ssl_connection:socket_control(?MODULE, Socket, Pid, CbModule), + ok = ssl_connection:handshake(SslSocket, Timeout), + {ok, SslSocket} + catch + error:{badmatch, {error, _} = Error} -> + Error + end; + +start_fsm(Role, Host, Port, Socket, {#ssl_options{erl_dist = true},_} = Opts, + User, {CbModule, _,_, _} = CbInfo, + Timeout) -> + try + {ok, Pid} = dtls_connection_sup:start_child_dist([Role, Host, Port, Socket, + Opts, User, CbInfo]), + {ok, SslSocket} = ssl_connection:socket_control(?MODULE, Socket, Pid, CbModule), + ok = ssl_connection:handshake(SslSocket, Timeout), + {ok, SslSocket} + catch + error:{badmatch, {error, _} = Error} -> + Error + end. + +send_handshake(Handshake, #state{negotiated_version = Version, + tls_handshake_history = Hist0, + connection_states = ConnectionStates0} = State0) -> + {BinHandshake, ConnectionStates, Hist} = + encode_handshake(Handshake, Version, ConnectionStates0, Hist0), + send_flight(BinHandshake, State0#state{connection_states = ConnectionStates, + tls_handshake_history = Hist + }). + +send_alert(Alert, #state{negotiated_version = Version, + socket = Socket, + transport_cb = Transport, + connection_states = ConnectionStates0} = State0) -> + {BinMsg, ConnectionStates} = + ssl_alert:encode(Alert, Version, ConnectionStates0), + Transport:send(Socket, BinMsg), + State0#state{connection_states = ConnectionStates}. + +send_change_cipher(Msg, #state{connection_states = ConnectionStates0, + socket = Socket, + negotiated_version = Version, + transport_cb = Transport} = State0) -> + {BinChangeCipher, ConnectionStates} = + encode_change_cipher(Msg, Version, ConnectionStates0), + Transport:send(Socket, BinChangeCipher), + State0#state{connection_states = ConnectionStates}. + +%%==================================================================== +%% tls_connection_sup API +%%==================================================================== + +%%-------------------------------------------------------------------- +-spec start_link(atom(), host(), inet:port_number(), port(), list(), pid(), tuple()) -> + {ok, pid()} | ignore | {error, reason()}. +%% +%% Description: Creates a gen_fsm process which calls Module:init/1 to +%% initialize. To ensure a synchronized start-up procedure, this function +%% does not return until Module:init/1 has returned. +%%-------------------------------------------------------------------- +start_link(Role, Host, Port, Socket, Options, User, CbInfo) -> + {ok, proc_lib:spawn_link(?MODULE, init, [[Role, Host, Port, Socket, Options, User, CbInfo]])}. + +init([Role, Host, Port, Socket, {SSLOpts0, _} = Options, User, CbInfo]) -> + process_flag(trap_exit, true), + State0 = initial_state(Role, Host, Port, Socket, Options, User, CbInfo), + Handshake = ssl_handshake:init_handshake_history(), + TimeStamp = calendar:datetime_to_gregorian_seconds({date(), time()}), + try ssl_config:init(SSLOpts0, Role) of + {ok, Ref, CertDbHandle, FileRefHandle, CacheHandle, OwnCert, Key, DHParams} -> + Session = State0#state.session, + State = State0#state{ + tls_handshake_history = Handshake, + session = Session#session{own_certificate = OwnCert, + time_stamp = TimeStamp}, + file_ref_db = FileRefHandle, + cert_db_ref = Ref, + cert_db = CertDbHandle, + session_cache = CacheHandle, + private_key = Key, + diffie_hellman_params = DHParams}, + gen_fsm:enter_loop(?MODULE, [], hello, State, get_timeout(State)) + catch + throw:Error -> + gen_fsm:enter_loop(?MODULE, [], error, {Error,State0}, get_timeout(State0)) + end. + +%%-------------------------------------------------------------------- +%% Description:There should be one instance of this function for each +%% possible state name. Whenever a gen_fsm receives an event sent +%% using gen_fsm:send_event/2, the instance of this function with the +%% same name as the current state name StateName is called to handle +%% the event. It is also called if a timeout occurs. +%% +hello(start, #state{host = Host, port = Port, role = client, + ssl_options = SslOpts, + session = #session{own_certificate = Cert} = Session0, + session_cache = Cache, session_cache_cb = CacheCb, + transport_cb = Transport, socket = Socket, + connection_states = ConnectionStates0, + renegotiation = {Renegotiation, _}} = State0) -> + Hello = dtls_handshake:client_hello(Host, Port, ConnectionStates0, SslOpts, + Cache, CacheCb, Renegotiation, Cert), + + Version = Hello#client_hello.client_version, + Handshake0 = ssl_handshake:init_handshake_history(), + {BinMsg, ConnectionStates, Handshake} = + encode_handshake(Hello, Version, ConnectionStates0, Handshake0), + Transport:send(Socket, BinMsg), + State1 = State0#state{connection_states = ConnectionStates, + negotiated_version = Version, %% Requested version + session = + Session0#session{session_id = Hello#client_hello.session_id}, + tls_handshake_history = Handshake}, + {Record, State} = next_record(State1), + next_state(hello, hello, Record, State); + +hello(Hello = #client_hello{client_version = ClientVersion, + extensions = #hello_extensions{hash_signs = HashSigns}}, + State = #state{connection_states = ConnectionStates0, + port = Port, session = #session{own_certificate = Cert} = Session0, + renegotiation = {Renegotiation, _}, + session_cache = Cache, + session_cache_cb = CacheCb, + ssl_options = SslOpts}) -> + case dtls_handshake:hello(Hello, SslOpts, {Port, Session0, Cache, CacheCb, + ConnectionStates0, Cert}, Renegotiation) of + {Version, {Type, Session}, + ConnectionStates, + #hello_extensions{ec_point_formats = EcPointFormats, + elliptic_curves = EllipticCurves} = ServerHelloExt} -> + HashSign = ssl_handshake:select_hashsign(HashSigns, Cert, + dtls_v1:corresponding_tls_version(Version)), + ssl_connection:hello({common_client_hello, Type, ServerHelloExt, HashSign}, + State#state{connection_states = ConnectionStates, + negotiated_version = Version, + session = Session, + client_ecc = {EllipticCurves, EcPointFormats}}, ?MODULE); + #alert{} = Alert -> + handle_own_alert(Alert, ClientVersion, hello, State) + end; +hello(Hello, + #state{connection_states = ConnectionStates0, + negotiated_version = ReqVersion, + role = client, + renegotiation = {Renegotiation, _}, + ssl_options = SslOptions} = State) -> + case dtls_handshake:hello(Hello, SslOptions, ConnectionStates0, Renegotiation) of + #alert{} = Alert -> + handle_own_alert(Alert, ReqVersion, hello, State); + {Version, NewId, ConnectionStates, NextProtocol} -> + ssl_connection:handle_session(Hello, + Version, NewId, ConnectionStates, NextProtocol, State) + end; + +hello(Msg, State) -> + ssl_connection:hello(Msg, State, ?MODULE). + +abbreviated(Msg, State) -> + ssl_connection:abbreviated(Msg, State, ?MODULE). + +certify(Msg, State) -> + ssl_connection:certify(Msg, State, ?MODULE). + +cipher(Msg, State) -> + ssl_connection:cipher(Msg, State, ?MODULE). + +connection(#hello_request{}, #state{host = Host, port = Port, + session = #session{own_certificate = Cert} = Session0, + session_cache = Cache, session_cache_cb = CacheCb, + ssl_options = SslOpts, + connection_states = ConnectionStates0, + renegotiation = {Renegotiation, _}} = State0) -> + Hello = dtls_handshake:client_hello(Host, Port, ConnectionStates0, SslOpts, + Cache, CacheCb, Renegotiation, Cert), + %% TODO DTLS version State1 = send_handshake(Hello, State0), + State1 = State0, + {Record, State} = + next_record( + State1#state{session = Session0#session{session_id + = Hello#client_hello.session_id}}), + next_state(connection, hello, Record, State); + +connection(#client_hello{} = Hello, #state{role = server, allow_renegotiate = true} = State) -> + %% Mitigate Computational DoS attack + %% http://www.educatedguesswork.org/2011/10/ssltls_and_computational_dos.html + %% http://www.thc.org/thc-ssl-dos/ Rather than disabling client + %% initiated renegotiation we will disallow many client initiated + %% renegotiations immediately after each other. + erlang:send_after(?WAIT_TO_ALLOW_RENEGOTIATION, self(), allow_renegotiate), + hello(Hello, State#state{allow_renegotiate = false}); + +connection(#client_hello{}, #state{role = server, allow_renegotiate = false} = State0) -> + Alert = ?ALERT_REC(?WARNING, ?NO_RENEGOTIATION), + State = send_alert(Alert, State0), + next_state_connection(connection, State); + +connection(Msg, State) -> + ssl_connection:connection(Msg, State, tls_connection). + +%%-------------------------------------------------------------------- +%% Description: Whenever a gen_fsm receives an event sent using +%% gen_fsm:send_all_state_event/2, this function is called to handle +%% the event. Not currently used! +%%-------------------------------------------------------------------- +handle_event(_Event, StateName, State) -> + {next_state, StateName, State, get_timeout(State)}. + +%%-------------------------------------------------------------------- +%% Description: Whenever a gen_fsm receives an event sent using +%% gen_fsm:sync_send_all_state_event/2,3, this function is called to handle +%% the event. +%%-------------------------------------------------------------------- +handle_sync_event(Event, From, StateName, State) -> + ssl_connection:handle_sync_event(Event, From, StateName, State). + +%%-------------------------------------------------------------------- +%% Description: This function is called by a gen_fsm when it receives any +%% other message than a synchronous or asynchronous event +%% (or a system message). +%%-------------------------------------------------------------------- + +%% raw data from socket, unpack records +handle_info({Protocol, _, Data}, StateName, + #state{data_tag = Protocol} = State0) -> + %% Simplify for now to avoid dialzer warnings before implementation is compleate + %% case next_tls_record(Data, State0) of + %% {Record, State} -> + %% next_state(StateName, StateName, Record, State); + %% #alert{} = Alert -> + %% handle_normal_shutdown(Alert, StateName, State0), + %% {stop, {shutdown, own_alert}, State0} + %% end; + {Record, State} = next_tls_record(Data, State0), + next_state(StateName, StateName, Record, State); + +handle_info({CloseTag, Socket}, StateName, + #state{socket = Socket, close_tag = CloseTag, + negotiated_version = _Version} = State) -> + handle_normal_shutdown(?ALERT_REC(?FATAL, ?CLOSE_NOTIFY), StateName, State), + {stop, {shutdown, transport_closed}, State}; + +handle_info(Msg, StateName, State) -> + ssl_connection:handle_info(Msg, StateName, State). + +%%-------------------------------------------------------------------- +%% Description:This function is called by a gen_fsm when it is about +%% to terminate. It should be the opposite of Module:init/1 and do any +%% necessary cleaning up. When it returns, the gen_fsm terminates with +%% Reason. The return value is ignored. +%%-------------------------------------------------------------------- +terminate(Reason, StateName, State) -> + ssl_connection:terminate(Reason, StateName, State). + +%%-------------------------------------------------------------------- +%% code_change(OldVsn, StateName, State, Extra) -> {ok, StateName, NewState} +%% Description: Convert process state when code is changed +%%-------------------------------------------------------------------- +code_change(_OldVsn, StateName, State, _Extra) -> + {ok, StateName, State}. + +%%-------------------------------------------------------------------- +%%% Internal functions +%%-------------------------------------------------------------------- +encode_handshake(Handshake, Version, ConnectionStates0, Hist0) -> + Seq = sequence(ConnectionStates0), + {EncHandshake, FragmentedHandshake} = dtls_handshake:encode_handshake(Handshake, Version, + Seq), + Hist = ssl_handshake:update_handshake_history(Hist0, EncHandshake), + {Encoded, ConnectionStates} = + dtls_record:encode_handshake(FragmentedHandshake, + Version, ConnectionStates0), + {Encoded, ConnectionStates, Hist}. + +next_record(#state{%%flight = #flight{state = finished}, + protocol_buffers = + #protocol_buffers{dtls_packets = [], dtls_cipher_texts = [CT | Rest]} + = Buffers, + connection_states = ConnStates0} = State) -> + case dtls_record:decode_cipher_text(CT, ConnStates0) of + {Plain, ConnStates} -> + {Plain, State#state{protocol_buffers = + Buffers#protocol_buffers{dtls_cipher_texts = Rest}, + connection_states = ConnStates}}; + #alert{} = Alert -> + {Alert, State} + end; +next_record(#state{socket = Socket, + transport_cb = Transport} = State) -> %% when FlightState =/= finished + ssl_socket:setopts(Transport, Socket, [{active,once}]), + {no_record, State}; + + +next_record(State) -> + {no_record, State}. + +next_state(Current,_, #alert{} = Alert, #state{negotiated_version = Version} = State) -> + handle_own_alert(Alert, Version, Current, State); + +next_state(_,Next, no_record, State) -> + {next_state, Next, State, get_timeout(State)}; + +%% next_state(_,Next, #ssl_tls{type = ?ALERT, fragment = EncAlerts}, State) -> +%% Alerts = decode_alerts(EncAlerts), +%% handle_alerts(Alerts, {next_state, Next, State, get_timeout(State)}); + +next_state(Current, Next, #ssl_tls{type = ?HANDSHAKE, fragment = Data}, + State0 = #state{protocol_buffers = + #protocol_buffers{dtls_handshake_buffer = Buf0} = Buffers, + negotiated_version = Version}) -> + Handle = + fun({#hello_request{} = Packet, _}, {next_state, connection = SName, State}) -> + %% This message should not be included in handshake + %% message hashes. Starts new handshake (renegotiation) + Hs0 = ssl_handshake:init_handshake_history(), + ?MODULE:SName(Packet, State#state{tls_handshake_history=Hs0, + renegotiation = {true, peer}}); + ({#hello_request{} = Packet, _}, {next_state, SName, State}) -> + %% This message should not be included in handshake + %% message hashes. Already in negotiation so it will be ignored! + ?MODULE:SName(Packet, State); + ({#client_hello{} = Packet, Raw}, {next_state, connection = SName, State}) -> + Version = Packet#client_hello.client_version, + Hs0 = ssl_handshake:init_handshake_history(), + Hs1 = ssl_handshake:update_handshake_history(Hs0, Raw), + ?MODULE:SName(Packet, State#state{tls_handshake_history=Hs1, + renegotiation = {true, peer}}); + ({Packet, Raw}, {next_state, SName, State = #state{tls_handshake_history=Hs0}}) -> + Hs1 = ssl_handshake:update_handshake_history(Hs0, Raw), + ?MODULE:SName(Packet, State#state{tls_handshake_history=Hs1}); + (_, StopState) -> StopState + end, + try + {Packets, Buf} = tls_handshake:get_tls_handshake(Version,Data,Buf0), + State = State0#state{protocol_buffers = + Buffers#protocol_buffers{dtls_packets = Packets, + dtls_handshake_buffer = Buf}}, + handle_dtls_handshake(Handle, Next, State) + catch throw:#alert{} = Alert -> + handle_own_alert(Alert, Version, Current, State0) + end; + +next_state(_, StateName, #ssl_tls{type = ?APPLICATION_DATA, fragment = Data}, State0) -> + %% Simplify for now to avoid dialzer warnings before implementation is compleate + %% case read_application_data(Data, State0) of + %% Stop = {stop,_,_} -> + %% Stop; + %% {Record, State} -> + %% next_state(StateName, StateName, Record, State) + %% end; + {Record, State} = read_application_data(Data, State0), + next_state(StateName, StateName, Record, State); + +next_state(Current, Next, #ssl_tls{type = ?CHANGE_CIPHER_SPEC, fragment = <<1>>} = + _ChangeCipher, + #state{connection_states = ConnectionStates0} = State0) -> + ConnectionStates1 = + ssl_record:activate_pending_connection_state(ConnectionStates0, read), + {Record, State} = next_record(State0#state{connection_states = ConnectionStates1}), + next_state(Current, Next, Record, State); +next_state(Current, Next, #ssl_tls{type = _Unknown}, State0) -> + %% Ignore unknown type + {Record, State} = next_record(State0), + next_state(Current, Next, Record, State). + +handle_dtls_handshake(Handle, StateName, + #state{protocol_buffers = + #protocol_buffers{dtls_packets = [Packet]} = Buffers} = State) -> + FsmReturn = {next_state, StateName, State#state{protocol_buffers = + Buffers#protocol_buffers{dtls_packets = []}}}, + Handle(Packet, FsmReturn); + +handle_dtls_handshake(Handle, StateName, + #state{protocol_buffers = + #protocol_buffers{dtls_packets = [Packet | Packets]} = Buffers} = + State0) -> + FsmReturn = {next_state, StateName, State0#state{protocol_buffers = + Buffers#protocol_buffers{dtls_packets = + Packets}}}, + case Handle(Packet, FsmReturn) of + {next_state, NextStateName, State, _Timeout} -> + handle_dtls_handshake(Handle, NextStateName, State); + {stop, _,_} = Stop -> + Stop + end. + + +send_flight(Fragments, #state{transport_cb = Transport, socket = Socket, + protocol_buffers = _PBuffers} = State) -> + Transport:send(Socket, Fragments), + %% Start retransmission + %% State#state{protocol_buffers = + %% (PBuffers#protocol_buffers){ #flight{state = waiting}}}}. + State. + +handle_own_alert(_,_,_, State) -> %% Place holder + {stop, {shutdown, own_alert}, State}. + +handle_normal_shutdown(_, _, _State) -> %% Place holder + ok. + +encode_change_cipher(#change_cipher_spec{}, Version, ConnectionStates) -> + dtls_record:encode_change_cipher_spec(Version, ConnectionStates). + +initial_state(Role, Host, Port, Socket, {SSLOptions, SocketOptions}, User, + {CbModule, DataTag, CloseTag, ErrorTag}) -> + ConnectionStates = ssl_record:init_connection_states(Role), + + SessionCacheCb = case application:get_env(ssl, session_cb) of + {ok, Cb} when is_atom(Cb) -> + Cb; + _ -> + ssl_session_cache + end, + + Monitor = erlang:monitor(process, User), + + #state{socket_options = SocketOptions, + %% We do not want to save the password in the state so that + %% could be written in the clear into error logs. + ssl_options = SSLOptions#ssl_options{password = undefined}, + session = #session{is_resumable = new}, + transport_cb = CbModule, + data_tag = DataTag, + close_tag = CloseTag, + error_tag = ErrorTag, + role = Role, + host = Host, + port = Port, + socket = Socket, + connection_states = ConnectionStates, + protocol_buffers = #protocol_buffers{}, + user_application = {Monitor, User}, + user_data_buffer = <<>>, + session_cache_cb = SessionCacheCb, + renegotiation = {false, first}, + start_or_recv_from = undefined, + send_queue = queue:new(), + protocol_cb = ?MODULE + }. +read_application_data(_,State) -> + {#ssl_tls{fragment = <<"place holder">>}, State}. + +next_tls_record(_, State) -> + {#ssl_tls{fragment = <<"place holder">>}, State}. + +get_timeout(_) -> %% Place holder + infinity. + +next_state_connection(_, State) -> %% Place holder + {next_state, connection, State, get_timeout(State)}. + +sequence(_) -> + %%TODO real imp + 1. diff --git a/lib/ssl/src/dtls_connection.hrl b/lib/ssl/src/dtls_connection.hrl index b8dff479d5..08707dc8de 100644 --- a/lib/ssl/src/dtls_connection.hrl +++ b/lib/ssl/src/dtls_connection.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2013-2013. All Rights Reserved. +%% Copyright Ericsson AB 2013-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -28,24 +28,19 @@ -include("ssl_connection.hrl"). -record(protocol_buffers, { - dtls_packets = [] ::[binary()], % Not yet handled decode ssl/tls packets. - dtls_record_buffer :: binary(), % Buffer of incomplete records - dtls_handshake_buffer :: binary(), % Buffer of incomplete handshakes - dtls_cipher_texts :: [binary()], - dtls_cipher_texts_next :: [binary()] % Received for Epoch not yet active + dtls_packets = [], %%::[binary()], % Not yet handled decode ssl/tls packets. + dtls_record_buffer = <<>>, %%:: binary(), % Buffer of incomplete records + dtls_handshake_buffer = <<>>, %%:: binary(), % Buffer of incomplete handshakes + dtls_cipher_texts = [], %%:: [binary()], + dtls_cipher_texts_next %%:: [binary()] % Received for Epoch not yet active }). -record(flight, { last_retransmit, last_read_seq, msl_timer, - flight_state, - flight_buffer, % buffer of not yet ACKed TLS records - }). - --record(message_sequences, { - read = 0, - write = 0 + state, + buffer % buffer of not yet ACKed TLS records }). -endif. % -ifdef(dtls_connection). diff --git a/lib/ssl/src/dtls_connection_sup.erl b/lib/ssl/src/dtls_connection_sup.erl index 9fe545be18..0b4711cfb4 100644 --- a/lib/ssl/src/dtls_connection_sup.erl +++ b/lib/ssl/src/dtls_connection_sup.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2013. All Rights Reserved. +%% Copyright Ericsson AB 2007-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -26,8 +26,8 @@ -behaviour(supervisor). %% API --export([start_link/0]). --export([start_child/1]). +-export([start_link/0, start_link_dist/0]). +-export([start_child/1, start_child_dist/1]). %% Supervisor callback -export([init/1]). @@ -38,8 +38,14 @@ start_link() -> supervisor:start_link({local, ?MODULE}, ?MODULE, []). +start_link_dist() -> + supervisor:start_link({local, dtls_connection_sup_dist}, ?MODULE, []). + start_child(Args) -> supervisor:start_child(?MODULE, Args). + +start_child_dist(Args) -> + supervisor:start_child(dtls_connection_sup_dist, Args). %%%========================================================================= %%% Supervisor callback diff --git a/lib/ssl/src/dtls_handshake.erl b/lib/ssl/src/dtls_handshake.erl index 5db2434753..31d525b295 100644 --- a/lib/ssl/src/dtls_handshake.erl +++ b/lib/ssl/src/dtls_handshake.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2013-2013. All Rights Reserved. +%% Copyright Ericsson AB 2013-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -20,11 +20,15 @@ -include("dtls_handshake.hrl"). -include("dtls_record.hrl"). -include("ssl_internal.hrl"). +-include("ssl_alert.hrl"). --export([client_hello/8, client_hello/9, hello/3, +-export([client_hello/8, client_hello/9, hello/4, get_dtls_handshake/2, - dtls_handshake_new_flight/1, dtls_handshake_new_epoch/1, - encode_handshake/4]). + %%dtls_handshake_new_flight/1, dtls_handshake_new_epoch/1, + encode_handshake/3]). + +-type dtls_handshake() :: #client_hello{} | #hello_verify_request{} | + ssl_handshake:ssl_handshake(). %%==================================================================== %% Internal application API @@ -54,12 +58,12 @@ client_hello(Host, Port, Cookie, ConnectionStates, ciphers = UserSuites } = SslOpts, Cache, CacheCb, Renegotiation, OwnCert) -> - Version = dtls_record:highest_protocol_version(Versions), + Version = dtls_record:highest_protocol_version(Versions), Pending = ssl_record:pending_connection_state(ConnectionStates, read), SecParams = Pending#connection_state.security_parameters, CipherSuites = ssl_handshake:available_suites(UserSuites, Version), - Extensions = ssl_handshake:client_hello_extensions(Host, Version, CipherSuites, + Extensions = ssl_handshake:client_hello_extensions(Host, dtls_v1:corresponding_tls_version(Version), CipherSuites, SslOpts, ConnectionStates, Renegotiation), Id = ssl_session:client_id({Host, Port, SslOpts}, Cache, CacheCb, OwnCert), @@ -73,163 +77,197 @@ client_hello(Host, Port, Cookie, ConnectionStates, extensions = Extensions }. -hello(Address, Port, - #ssl_tls{epoch = _Epoch, record_seq = _Seq, - version = Version} = Record) -> - {[{Hello, _}], _, _} = - get_dtls_handshake(Record, - dtls_handshake_new_flight(undefined)), - #client_hello{client_version = {Major, Minor}, - random = Random, - session_id = SessionId, - cipher_suites = CipherSuites, - compression_methods = CompressionMethods} = Hello, - CookieData = [address_to_bin(Address, Port), - <<?BYTE(Major), ?BYTE(Minor)>>, - Random, SessionId, CipherSuites, CompressionMethods], - Cookie = crypto:hmac(sha, <<"secret">>, CookieData), - - case Hello of - #client_hello{cookie = Cookie} -> - accept; - _ -> - %% generate HelloVerifyRequest - HelloVerifyRequest = encode_handshake(#hello_verify_request{protocol_version = Version, - cookie = Cookie}, - Version, 0, 1400), - {reply, HelloVerifyRequest} - end. +hello(#server_hello{server_version = Version, random = Random, + cipher_suite = CipherSuite, + compression_method = Compression, + session_id = SessionId, extensions = HelloExt}, + #ssl_options{versions = SupportedVersions} = SslOpt, + ConnectionStates0, Renegotiation) -> + case dtls_record:is_acceptable_version(Version, SupportedVersions) of + true -> + handle_server_hello_extensions(Version, SessionId, Random, CipherSuite, + Compression, HelloExt, SslOpt, ConnectionStates0, Renegotiation); + false -> + ?ALERT_REC(?FATAL, ?PROTOCOL_VERSION) + end; -%%-------------------------------------------------------------------- -encode_handshake(Package, Version, MsgSeq, Mss) -> - {MsgType, Bin} = enc_hs(Package, Version), +hello(#client_hello{client_version = ClientVersion}, _Options, {_,_,_,_,ConnectionStates,_}, _Renegotiation) -> + %% Return correct typ to make dialyzer happy until we have time to make the real imp. + {ClientVersion, {new, #session{}}, ConnectionStates, #hello_extensions{}}. + +%% hello(Address, Port, +%% #ssl_tls{epoch = _Epoch, sequence_number = _Seq, +%% version = Version} = Record) -> +%% case get_dtls_handshake(Record, +%% dtls_handshake_new_flight(undefined)) of +%% {[Hello | _], _} -> +%% hello(Address, Port, Version, Hello); +%% {retransmit, HandshakeState} -> +%% {retransmit, HandshakeState} +%% end. + +%% hello(Address, Port, Version, Hello) -> +%% #client_hello{client_version = {Major, Minor}, +%% random = Random, +%% session_id = SessionId, +%% cipher_suites = CipherSuites, +%% compression_methods = CompressionMethods} = Hello, +%% CookieData = [address_to_bin(Address, Port), +%% <<?BYTE(Major), ?BYTE(Minor)>>, +%% Random, SessionId, CipherSuites, CompressionMethods], +%% Cookie = crypto:hmac(sha, <<"secret">>, CookieData), + +%% case Hello of +%% #client_hello{cookie = Cookie} -> +%% accept; +%% _ -> +%% %% generate HelloVerifyRequest +%% HelloVerifyRequest = enc_hs(#hello_verify_request{protocol_version = Version, +%% cookie = Cookie}, +%% Version, 0, 1400), +%% {reply, HelloVerifyRequest} +%% end. + +%% %%-------------------------------------------------------------------- +encode_handshake(Handshake, Version, MsgSeq) -> + {MsgType, Bin} = enc_handshake(Handshake, Version), Len = byte_size(Bin), - HsHistory = [MsgType, ?uint24(Len), ?uint16(MsgSeq), ?uint24(0), ?uint24(Len), Bin], - BinMsg = dtls_split_handshake(Mss, MsgType, Len, MsgSeq, Bin, 0, []), - {HsHistory, BinMsg}. + EncHandshake = [MsgType, ?uint24(Len), ?uint16(MsgSeq), ?uint24(0), ?uint24(Len), Bin], + FragmentedHandshake = dtls_fragment(erlang:iolist_size(EncHandshake), MsgType, Len, MsgSeq, Bin, 0, []), + {EncHandshake, FragmentedHandshake}. -%-------------------------------------------------------------------- +%%-------------------------------------------------------------------- -spec get_dtls_handshake(#ssl_tls{}, #dtls_hs_state{} | binary()) -> - {[dtls_handshake()], #dtls_hs_state{}} | {retransmit, #dtls_hs_state{}}. -% -% Description: Given a DTLS state and new data from ssl_record, collects -% and returns it as a list of handshake messages, also returns a new -% DTLS state -%-------------------------------------------------------------------- -% get_dtls_handshake(Record, <<>>) -> -% get_dtls_handshake_aux(Record, dtls_hs_state_init()); + {[dtls_handshake()], #dtls_hs_state{}} | {retransmit, #dtls_hs_state{}}. +%% +%% Description: Given a DTLS state and new data from ssl_record, collects +%% and returns it as a list of handshake messages, also returns a new +%% DTLS state +%%-------------------------------------------------------------------- +get_dtls_handshake(Record, <<>>) -> + get_dtls_handshake_aux(Record, #dtls_hs_state{}); %% Init handshake state!? get_dtls_handshake(Record, HsState) -> get_dtls_handshake_aux(Record, HsState). -%-------------------------------------------------------------------- --spec dtls_handshake_new_epoch(#dtls_hs_state{}) -> #dtls_hs_state{}. -% -% Description: Reset the DTLS decoder state for a new Epoch -%-------------------------------------------------------------------- -% dtls_handshake_new_epoch(<<>>) -> -% dtls_hs_state_init(); -dtls_handshake_new_epoch(HsState) -> - HsState#dtls_hs_state{highest_record_seq = 0, - starting_read_seq = HsState#dtls_hs_state.current_read_seq, - fragments = gb_trees:empty(), completed = []}. - -%-------------------------------------------------------------------- --spec dtls_handshake_new_flight(integer() | undefined) -> #dtls_hs_state{}. -% -% Description: Init the DTLS decoder state for a new Flight -dtls_handshake_new_flight(ExpectedReadReq) -> - #dtls_hs_state{current_read_seq = ExpectedReadReq, - highest_record_seq = 0, - starting_read_seq = 0, - fragments = gb_trees:empty(), completed = []}. +%% %%-------------------------------------------------------------------- +%% -spec dtls_handshake_new_epoch(#dtls_hs_state{}) -> #dtls_hs_state{}. +%% %% +%% %% Description: Reset the DTLS decoder state for a new Epoch +%% %%-------------------------------------------------------------------- +%% dtls_handshake_new_epoch(<<>>) -> +%% dtls_hs_state_init(); +%% dtls_handshake_new_epoch(HsState) -> +%% HsState#dtls_hs_state{highest_record_seq = 0, +%% starting_read_seq = HsState#dtls_hs_state.current_read_seq, +%% fragments = gb_trees:empty(), completed = []}. + +%% %-------------------------------------------------------------------- +%% -spec dtls_handshake_new_flight(integer() | undefined) -> #dtls_hs_state{}. +%% % +%% % Description: Init the DTLS decoder state for a new Flight +%% dtls_handshake_new_flight(ExpectedReadReq) -> +%% #dtls_hs_state{current_read_seq = ExpectedReadReq, +%% highest_record_seq = 0, +%% starting_read_seq = 0, +%% fragments = gb_trees:empty(), completed = []}. %%-------------------------------------------------------------------- %%% Internal functions %%-------------------------------------------------------------------- +handle_server_hello_extensions(Version, SessionId, Random, CipherSuite, + Compression, HelloExt, SslOpt, ConnectionStates0, Renegotiation) -> + case ssl_handshake:handle_server_hello_extensions(dtls_record, Random, CipherSuite, + Compression, HelloExt, Version, + SslOpt, ConnectionStates0, Renegotiation) of + #alert{} = Alert -> + Alert; + {ConnectionStates, Protocol} -> + {Version, SessionId, ConnectionStates, Protocol} + end. -dtls_split_handshake(Mss, MsgType, Len, MsgSeq, Bin, Offset, Acc) +dtls_fragment(Mss, MsgType, Len, MsgSeq, Bin, Offset, Acc) when byte_size(Bin) + 12 < Mss -> FragmentLen = byte_size(Bin), BinMsg = [MsgType, ?uint24(Len), ?uint16(MsgSeq), ?uint24(Offset), ?uint24(FragmentLen), Bin], lists:reverse([BinMsg|Acc]); -dtls_split_handshake(Mss, MsgType, Len, MsgSeq, Bin, Offset, Acc) -> +dtls_fragment(Mss, MsgType, Len, MsgSeq, Bin, Offset, Acc) -> FragmentLen = Mss - 12, <<Fragment:FragmentLen/bytes, Rest/binary>> = Bin, BinMsg = [MsgType, ?uint24(Len), ?uint16(MsgSeq), ?uint24(Offset), ?uint24(FragmentLen), Fragment], - dtls_split_handshake(Mss, MsgType, Len, MsgSeq, Rest, Offset + FragmentLen, [BinMsg|Acc]). + dtls_fragment(Mss, MsgType, Len, MsgSeq, Rest, Offset + FragmentLen, [BinMsg|Acc]). get_dtls_handshake_aux(#ssl_tls{version = Version, - record_seq = SeqNo, - fragment = Data}, HsState) -> + sequence_number = SeqNo, + fragment = Data}, HsState) -> get_dtls_handshake_aux(Version, SeqNo, Data, HsState). get_dtls_handshake_aux(Version, SeqNo, - <<?BYTE(Type), ?UINT24(Length), - ?UINT16(MessageSeq), - ?UINT24(FragmentOffset), ?UINT24(FragmentLength), - Body:FragmentLength/binary, Rest/binary>>, - HsState0) -> + <<?BYTE(Type), ?UINT24(Length), + ?UINT16(MessageSeq), + ?UINT24(FragmentOffset), ?UINT24(FragmentLength), + Body:FragmentLength/binary, Rest/binary>>, + HsState0) -> case reassemble_dtls_fragment(SeqNo, Type, Length, MessageSeq, - FragmentOffset, FragmentLength, - Body, HsState0) of - {retransmit, HsState1} -> - case Rest of - <<>> -> - {retransmit, HsState1}; - _ -> - get_dtls_handshake_aux(Version, SeqNo, Rest, HsState1) - end; - {HsState1, HighestSeqNo, MsgBody} -> - HsState2 = dec_dtls_fragment(Version, HighestSeqNo, Type, Length, MessageSeq, MsgBody, HsState1), - HsState3 = process_dtls_fragments(Version, HsState2), - get_dtls_handshake_aux(Version, SeqNo, Rest, HsState3); - HsState2 -> - HsState3 = process_dtls_fragments(Version, HsState2), - get_dtls_handshake_aux(Version, SeqNo, Rest, HsState3) - end; + FragmentOffset, FragmentLength, + Body, HsState0) of + {retransmit, HsState1} -> + case Rest of + <<>> -> + {retransmit, HsState1}; + _ -> + get_dtls_handshake_aux(Version, SeqNo, Rest, HsState1) + end; + {HsState1, HighestSeqNo, MsgBody} -> + HsState2 = dec_dtls_fragment(Version, HighestSeqNo, Type, Length, MessageSeq, MsgBody, HsState1), + HsState3 = process_dtls_fragments(Version, HsState2), + get_dtls_handshake_aux(Version, SeqNo, Rest, HsState3); + HsState2 -> + HsState3 = process_dtls_fragments(Version, HsState2), + get_dtls_handshake_aux(Version, SeqNo, Rest, HsState3) + end; get_dtls_handshake_aux(_Version, _SeqNo, <<>>, HsState) -> - {lists:reverse(HsState#dtls_hs_state.completed), - HsState#dtls_hs_state{completed = []}}. + {lists:reverse(HsState#dtls_hs_state.completed), + HsState#dtls_hs_state{completed = []}}. dec_dtls_fragment(Version, SeqNo, Type, Length, MessageSeq, MsgBody, - HsState = #dtls_hs_state{highest_record_seq = HighestSeqNo, completed = Acc}) -> + HsState = #dtls_hs_state{highest_record_seq = HighestSeqNo, completed = Acc}) -> Raw = <<?BYTE(Type), ?UINT24(Length), ?UINT16(MessageSeq), ?UINT24(0), ?UINT24(Length), MsgBody/binary>>, H = decode_handshake(Version, Type, MsgBody), HsState#dtls_hs_state{completed = [{H,Raw}|Acc], highest_record_seq = erlang:max(HighestSeqNo, SeqNo)}. process_dtls_fragments(Version, - HsState0 = #dtls_hs_state{current_read_seq = CurrentReadSeq, - fragments = Fragments0}) -> + HsState0 = #dtls_hs_state{current_read_seq = CurrentReadSeq, + fragments = Fragments0}) -> case gb_trees:is_empty(Fragments0) of - true -> - HsState0; - _ -> - case gb_trees:smallest(Fragments0) of - {CurrentReadSeq, {SeqNo, Type, Length, CurrentReadSeq, {Length, [{0, Length}], MsgBody}}} -> - HsState1 = dtls_hs_state_process_seq(HsState0), - HsState2 = dec_dtls_fragment(Version, SeqNo, Type, Length, CurrentReadSeq, MsgBody, HsState1), - process_dtls_fragments(Version, HsState2); - _ -> - HsState0 - end - end. + true -> + HsState0; + _ -> + case gb_trees:smallest(Fragments0) of + {CurrentReadSeq, {SeqNo, Type, Length, CurrentReadSeq, {Length, [{0, Length}], MsgBody}}} -> + HsState1 = dtls_hs_state_process_seq(HsState0), + HsState2 = dec_dtls_fragment(Version, SeqNo, Type, Length, CurrentReadSeq, MsgBody, HsState1), + process_dtls_fragments(Version, HsState2); + _ -> + HsState0 + end + end. dtls_hs_state_process_seq(HsState0 = #dtls_hs_state{current_read_seq = CurrentReadSeq, - fragments = Fragments0}) -> + fragments = Fragments0}) -> Fragments1 = gb_trees:delete_any(CurrentReadSeq, Fragments0), HsState0#dtls_hs_state{current_read_seq = CurrentReadSeq + 1, - fragments = Fragments1}. + fragments = Fragments1}. dtls_hs_state_add_fragment(MessageSeq, Fragment, HsState0 = #dtls_hs_state{fragments = Fragments0}) -> Fragments1 = gb_trees:enter(MessageSeq, Fragment, Fragments0), HsState0#dtls_hs_state{fragments = Fragments1}. reassemble_dtls_fragment(SeqNo, Type, Length, MessageSeq, 0, Length, - Body, HsState0 = #dtls_hs_state{current_read_seq = undefined}) + Body, HsState0 = #dtls_hs_state{current_read_seq = undefined}) when Type == ?CLIENT_HELLO; Type == ?SERVER_HELLO; - Type == ?HELLO_VERIFY_REQUEST -> + Type == ?HELLO_VERIFY_REQUEST -> %% First message, should be client hello %% return the current message and set the next expected Sequence %% @@ -245,8 +283,8 @@ reassemble_dtls_fragment(_SeqNo, _Type, Length, _MessageSeq, _, Length, HsState; reassemble_dtls_fragment(SeqNo, _Type, Length, MessageSeq, 0, Length, - Body, HsState0 = - #dtls_hs_state{starting_read_seq = StartingReadSeq}) + Body, HsState0 = + #dtls_hs_state{starting_read_seq = StartingReadSeq}) when MessageSeq < StartingReadSeq -> %% this has to be the start of a new flight, let it through %% @@ -257,69 +295,69 @@ reassemble_dtls_fragment(SeqNo, _Type, Length, MessageSeq, 0, Length, {HsState, SeqNo, Body}; reassemble_dtls_fragment(_SeqNo, _Type, Length, MessageSeq, 0, Length, - _Body, HsState = - #dtls_hs_state{current_read_seq = CurrentReadSeq}) + _Body, HsState = + #dtls_hs_state{current_read_seq = CurrentReadSeq}) when MessageSeq < CurrentReadSeq -> {retransmit, HsState}; reassemble_dtls_fragment(_SeqNo, _Type, Length, MessageSeq, 0, Length, - _Body, HsState = #dtls_hs_state{current_read_seq = CurrentReadSeq}) + _Body, HsState = #dtls_hs_state{current_read_seq = CurrentReadSeq}) when MessageSeq < CurrentReadSeq -> HsState; reassemble_dtls_fragment(SeqNo, _Type, Length, MessageSeq, 0, Length, - Body, HsState0 = #dtls_hs_state{current_read_seq = MessageSeq}) -> + Body, HsState0 = #dtls_hs_state{current_read_seq = MessageSeq}) -> %% Message fully contained and it's the current seq HsState1 = dtls_hs_state_process_seq(HsState0), {HsState1, SeqNo, Body}; reassemble_dtls_fragment(SeqNo, Type, Length, MessageSeq, 0, Length, - Body, HsState) -> + Body, HsState) -> %% Message fully contained and it's the NOT the current seq -> buffer Fragment = {SeqNo, Type, Length, MessageSeq, - dtls_fragment_init(Length, 0, Length, Body)}, + dtls_fragment_init(Length, 0, Length, Body)}, dtls_hs_state_add_fragment(MessageSeq, Fragment, HsState); reassemble_dtls_fragment(_SeqNo, _Type, Length, MessageSeq, FragmentOffset, FragmentLength, - _Body, - HsState = #dtls_hs_state{current_read_seq = CurrentReadSeq}) + _Body, + HsState = #dtls_hs_state{current_read_seq = CurrentReadSeq}) when FragmentOffset + FragmentLength == Length andalso MessageSeq == (CurrentReadSeq - 1) -> {retransmit, HsState}; reassemble_dtls_fragment(_SeqNo, _Type, _Length, MessageSeq, _FragmentOffset, _FragmentLength, - _Body, - HsState = #dtls_hs_state{current_read_seq = CurrentReadSeq}) + _Body, + HsState = #dtls_hs_state{current_read_seq = CurrentReadSeq}) when MessageSeq < CurrentReadSeq -> HsState; reassemble_dtls_fragment(SeqNo, Type, Length, MessageSeq, - FragmentOffset, FragmentLength, - Body, - HsState = #dtls_hs_state{fragments = Fragments0}) -> + FragmentOffset, FragmentLength, + Body, + HsState = #dtls_hs_state{fragments = Fragments0}) -> case gb_trees:lookup(MessageSeq, Fragments0) of - {value, Fragment} -> - dtls_fragment_reassemble(SeqNo, Type, Length, MessageSeq, - FragmentOffset, FragmentLength, - Body, Fragment, HsState); - none -> - dtls_fragment_start(SeqNo, Type, Length, MessageSeq, - FragmentOffset, FragmentLength, - Body, HsState) + {value, Fragment} -> + dtls_fragment_reassemble(SeqNo, Type, Length, MessageSeq, + FragmentOffset, FragmentLength, + Body, Fragment, HsState); + none -> + dtls_fragment_start(SeqNo, Type, Length, MessageSeq, + FragmentOffset, FragmentLength, + Body, HsState) end. dtls_fragment_start(SeqNo, Type, Length, MessageSeq, - FragmentOffset, FragmentLength, - Body, HsState = #dtls_hs_state{fragments = Fragments0}) -> + FragmentOffset, FragmentLength, + Body, HsState = #dtls_hs_state{fragments = Fragments0}) -> Fragment = {SeqNo, Type, Length, MessageSeq, - dtls_fragment_init(Length, FragmentOffset, FragmentLength, Body)}, - Fragments1 = gb_trees:insert(MessageSeq, Fragment, Fragments0), + dtls_fragment_init(Length, FragmentOffset, FragmentLength, Body)}, + Fragments1 = gb_trees:insert(MessageSeq, Fragment, Fragments0), HsState#dtls_hs_state{fragments = Fragments1}. dtls_fragment_reassemble(SeqNo, Type, Length, MessageSeq, FragmentOffset, FragmentLength, - Body, - {LastSeqNo, Type, Length, MessageSeq, FragBuffer0}, - HsState = #dtls_hs_state{fragments = Fragments0}) -> + Body, + {LastSeqNo, Type, Length, MessageSeq, FragBuffer0}, + HsState = #dtls_hs_state{fragments = Fragments0}) -> FragBuffer1 = dtls_fragment_add(FragBuffer0, FragmentOffset, FragmentLength, Body), Fragment = {erlang:max(SeqNo, LastSeqNo), Type, Length, MessageSeq, FragBuffer1}, Fragments1 = gb_trees:enter(MessageSeq, Fragment, Fragments0), @@ -328,8 +366,8 @@ dtls_fragment_reassemble(SeqNo, Type, Length, MessageSeq, %% Type, Length or Seq mismatch, drop everything... %% Note: the RFC is not clear on how to handle this... dtls_fragment_reassemble(_SeqNo, _Type, _Length, MessageSeq, - _FragmentOffset, _FragmentLength, _Body, _Fragment, - HsState = #dtls_hs_state{fragments = Fragments0}) -> + _FragmentOffset, _FragmentLength, _Body, _Fragment, + HsState = #dtls_hs_state{fragments = Fragments0}) -> Fragments1 = gb_trees:delete_any(MessageSeq, Fragments0), HsState#dtls_hs_state{fragments = Fragments1}. @@ -360,7 +398,7 @@ merge_fragment_list(Rest = [{HStart, _HEnd}|_], Frag = {_FStart, FEnd}, Acc) lists:reverse(Acc) ++ [Frag|Rest]; merge_fragment_list([{HStart, HEnd}|Rest], _Frag = {FStart, FEnd}, Acc) - when + when FStart =< HEnd orelse FEnd >= HStart -> Start = erlang:min(HStart, FStart), End = erlang:max(HEnd, FEnd), @@ -370,20 +408,20 @@ merge_fragment_list([{HStart, HEnd}|Rest], _Frag = {FStart, FEnd}, Acc) add_fragment(List, {FragmentOffset, FragmentLength}) -> merge_fragment_list(List, {FragmentOffset, FragmentOffset + FragmentLength}, []). -enc_hs(#hello_verify_request{protocol_version = {Major, Minor}, - cookie = Cookie}, _Version) -> - CookieLength = byte_size(Cookie), +enc_handshake(#hello_verify_request{protocol_version = {Major, Minor}, + cookie = Cookie}, _Version) -> + CookieLength = byte_size(Cookie), {?HELLO_VERIFY_REQUEST, <<?BYTE(Major), ?BYTE(Minor), - ?BYTE(CookieLength), - Cookie/binary>>}; - -enc_hs(#client_hello{client_version = {Major, Minor}, - random = Random, - session_id = SessionID, - cookie = Cookie, - cipher_suites = CipherSuites, - compression_methods = CompMethods, - extensions = HelloExtensions}, Version) -> + ?BYTE(CookieLength), + Cookie/binary>>}; + +enc_handshake(#client_hello{client_version = {Major, Minor}, + random = Random, + session_id = SessionID, + cookie = Cookie, + cipher_suites = CipherSuites, + compression_methods = CompMethods, + extensions = HelloExtensions}, Version) -> SIDLength = byte_size(SessionID), BinCookie = enc_client_hello_cookie(Version, Cookie), BinCompMethods = list_to_binary(CompMethods), @@ -391,13 +429,13 @@ enc_hs(#client_hello{client_version = {Major, Minor}, BinCipherSuites = list_to_binary(CipherSuites), CsLength = byte_size(BinCipherSuites), ExtensionsBin = ssl_handshake:encode_hello_extensions(HelloExtensions), - + {?CLIENT_HELLO, <<?BYTE(Major), ?BYTE(Minor), Random:32/binary, - ?BYTE(SIDLength), SessionID/binary, - BinCookie/binary, - ?UINT16(CsLength), BinCipherSuites/binary, - ?BYTE(CmLength), BinCompMethods/binary, ExtensionsBin/binary>>}; -enc_hs(HandshakeMsg, Version) -> + ?BYTE(SIDLength), SessionID/binary, + BinCookie/binary, + ?UINT16(CsLength), BinCipherSuites/binary, + ?BYTE(CmLength), BinCompMethods/binary, ExtensionsBin/binary>>}; +enc_handshake(HandshakeMsg, Version) -> ssl_handshake:encode_handshake(HandshakeMsg, Version). enc_client_hello_cookie(_, <<>>) -> @@ -407,26 +445,26 @@ enc_client_hello_cookie(_, Cookie) -> <<?BYTE(CookieLength), Cookie/binary>>. decode_handshake(_Version, ?CLIENT_HELLO, <<?BYTE(Major), ?BYTE(Minor), Random:32/binary, - ?BYTE(SID_length), Session_ID:SID_length/binary, - ?BYTE(Cookie_length), Cookie:Cookie_length/binary, - ?UINT16(Cs_length), CipherSuites:Cs_length/binary, - ?BYTE(Cm_length), Comp_methods:Cm_length/binary, - Extensions/binary>>) -> - + ?BYTE(SID_length), Session_ID:SID_length/binary, + ?BYTE(Cookie_length), Cookie:Cookie_length/binary, + ?UINT16(Cs_length), CipherSuites:Cs_length/binary, + ?BYTE(Cm_length), Comp_methods:Cm_length/binary, + Extensions/binary>>) -> + DecodedExtensions = ssl_handshake:decode_hello_extensions(Extensions), - + #client_hello{ client_version = {Major,Minor}, random = Random, - session_id = Session_ID, + session_id = Session_ID, cookie = Cookie, cipher_suites = ssl_handshake:decode_suites('2_bytes', CipherSuites), compression_methods = Comp_methods, extensions = DecodedExtensions - }; + }; decode_handshake(_Version, ?HELLO_VERIFY_REQUEST, <<?BYTE(Major), ?BYTE(Minor), - ?BYTE(CookieLength), Cookie:CookieLength/binary>>) -> + ?BYTE(CookieLength), Cookie:CookieLength/binary>>) -> #hello_verify_request{ protocol_version = {Major,Minor}, @@ -434,7 +472,7 @@ decode_handshake(_Version, ?HELLO_VERIFY_REQUEST, <<?BYTE(Major), ?BYTE(Minor), decode_handshake(Version, Tag, Msg) -> ssl_handshake:decode_handshake(Version, Tag, Msg). -address_to_bin({A,B,C,D}, Port) -> - <<0:80,16#ffff:16,A,B,C,D,Port:16>>; -address_to_bin({A,B,C,D,E,F,G,H}, Port) -> - <<A:16,B:16,C:16,D:16,E:16,F:16,G:16,H:16,Port:16>>. +%% address_to_bin({A,B,C,D}, Port) -> +%% <<0:80,16#ffff:16,A,B,C,D,Port:16>>; +%% address_to_bin({A,B,C,D,E,F,G,H}, Port) -> +%% <<A:16,B:16,C:16,D:16,E:16,F:16,G:16,H:16,Port:16>>. diff --git a/lib/ssl/src/dtls_handshake.hrl b/lib/ssl/src/dtls_handshake.hrl index 5bdf45f627..3b57575b6d 100644 --- a/lib/ssl/src/dtls_handshake.hrl +++ b/lib/ssl/src/dtls_handshake.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2013-2013. All Rights Reserved. +%% Copyright Ericsson AB 2013-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -53,6 +53,4 @@ completed }). --type dtls_handshake() :: #client_hello{} | #hello_verify_request{} | ssl_handshake(). - -endif. % -ifdef(dtls_handshake). diff --git a/lib/ssl/src/dtls_record.erl b/lib/ssl/src/dtls_record.erl index b0a7976864..a7bbb6bc40 100644 --- a/lib/ssl/src/dtls_record.erl +++ b/lib/ssl/src/dtls_record.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2013-2013. All Rights Reserved. +%% Copyright Ericsson AB 2013-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -35,7 +35,7 @@ -export([decode_cipher_text/2]). %% Encoding --export([encode_plain_text/4]). +-export([encode_plain_text/4, encode_handshake/3, encode_change_cipher_spec/2]). %% Protocol version handling -export([protocol_version/1, lowest_protocol_version/2, @@ -46,6 +46,11 @@ -export([init_connection_state_seq/2, current_connection_state_epoch/2, set_connection_state_by_epoch/3, connection_state_by_epoch/3]). +-export_type([dtls_version/0, dtls_atom_version/0]). + +-type dtls_version() :: ssl_record:ssl_version(). +-type dtls_atom_version() :: dtlsv1 | 'dtlsv1.2'. + -compile(inline). %%==================================================================== @@ -70,7 +75,7 @@ get_dtls_records_aux(<<?BYTE(?APPLICATION_DATA),?BYTE(MajVer),?BYTE(MinVer), Acc) -> get_dtls_records_aux(Rest, [#ssl_tls{type = ?APPLICATION_DATA, version = {MajVer, MinVer}, - epoch = Epoch, record_seq = SequenceNumber, + epoch = Epoch, sequence_number = SequenceNumber, fragment = Data} | Acc]); get_dtls_records_aux(<<?BYTE(?HANDSHAKE),?BYTE(MajVer),?BYTE(MinVer), ?UINT16(Epoch), ?UINT48(SequenceNumber), @@ -78,7 +83,7 @@ get_dtls_records_aux(<<?BYTE(?HANDSHAKE),?BYTE(MajVer),?BYTE(MinVer), Data:Length/binary, Rest/binary>>, Acc) when MajVer >= 128 -> get_dtls_records_aux(Rest, [#ssl_tls{type = ?HANDSHAKE, version = {MajVer, MinVer}, - epoch = Epoch, record_seq = SequenceNumber, + epoch = Epoch, sequence_number = SequenceNumber, fragment = Data} | Acc]); get_dtls_records_aux(<<?BYTE(?ALERT),?BYTE(MajVer),?BYTE(MinVer), ?UINT16(Epoch), ?UINT48(SequenceNumber), @@ -86,7 +91,7 @@ get_dtls_records_aux(<<?BYTE(?ALERT),?BYTE(MajVer),?BYTE(MinVer), Rest/binary>>, Acc) -> get_dtls_records_aux(Rest, [#ssl_tls{type = ?ALERT, version = {MajVer, MinVer}, - epoch = Epoch, record_seq = SequenceNumber, + epoch = Epoch, sequence_number = SequenceNumber, fragment = Data} | Acc]); get_dtls_records_aux(<<?BYTE(?CHANGE_CIPHER_SPEC),?BYTE(MajVer),?BYTE(MinVer), ?UINT16(Epoch), ?UINT48(SequenceNumber), @@ -94,7 +99,7 @@ get_dtls_records_aux(<<?BYTE(?CHANGE_CIPHER_SPEC),?BYTE(MajVer),?BYTE(MinVer), Acc) -> get_dtls_records_aux(Rest, [#ssl_tls{type = ?CHANGE_CIPHER_SPEC, version = {MajVer, MinVer}, - epoch = Epoch, record_seq = SequenceNumber, + epoch = Epoch, sequence_number = SequenceNumber, fragment = Data} | Acc]); get_dtls_records_aux(<<0:1, _CT:7, ?BYTE(_MajVer), ?BYTE(_MinVer), @@ -125,14 +130,15 @@ encode_plain_text(Type, Version, Data, {Comp, CompS1} = ssl_record:compress(CompAlg, Data, CompS0), WriteState1 = WriteState0#connection_state{compression_state = CompS1}, MacHash = calc_mac_hash(WriteState1, Type, Version, Epoch, Seq, Comp), - {CipherFragment, WriteState} = ssl_record:cipher(Version, Comp, WriteState1, MacHash), + {CipherFragment, WriteState} = ssl_record:cipher(dtls_v1:corresponding_tls_version(Version), + Comp, WriteState1, MacHash), CipherText = encode_tls_cipher_text(Type, Version, Epoch, Seq, CipherFragment), {CipherText, ConnectionStates#connection_states{current_write = WriteState#connection_state{sequence_number = Seq +1}}}. decode_cipher_text(#ssl_tls{type = Type, version = Version, epoch = Epoch, - record_seq = Seq, + sequence_number = Seq, fragment = CipherFragment} = CipherText, #connection_states{current_read = #connection_state{compression_state = CompressionS0, @@ -141,7 +147,7 @@ decode_cipher_text(#ssl_tls{type = Type, version = Version, CompressAlg = SecParams#security_parameters.compression_algorithm, {PlainFragment, Mac, ReadState1} = ssl_record:decipher(dtls_v1:corresponding_tls_version(Version), CipherFragment, ReadState0), - MacHash = calc_mac_hash(Type, Version, Epoch, Seq, PlainFragment, ReadState1), + MacHash = calc_mac_hash(ReadState1, Type, Version, Epoch, Seq, PlainFragment), case ssl_record:is_correct_mac(Mac, MacHash) of true -> {Plain, CompressionS1} = ssl_record:uncompress(CompressAlg, @@ -153,10 +159,27 @@ decode_cipher_text(#ssl_tls{type = Type, version = Version, false -> ?ALERT_REC(?FATAL, ?BAD_RECORD_MAC) end. +%%-------------------------------------------------------------------- +-spec encode_handshake(iolist(), dtls_version(), #connection_states{}) -> + {iolist(), #connection_states{}}. +%% +%% Description: Encodes a handshake message to send on the ssl-socket. +%%-------------------------------------------------------------------- +encode_handshake(Frag, Version, ConnectionStates) -> + encode_plain_text(?HANDSHAKE, Version, Frag, ConnectionStates). + +%%-------------------------------------------------------------------- +-spec encode_change_cipher_spec(dtls_version(), #connection_states{}) -> + {iolist(), #connection_states{}}. +%% +%% Description: Encodes a change_cipher_spec-message to send on the ssl socket. +%%-------------------------------------------------------------------- +encode_change_cipher_spec(Version, ConnectionStates) -> + encode_plain_text(?CHANGE_CIPHER_SPEC, Version, <<1:8>>, ConnectionStates). %%-------------------------------------------------------------------- --spec protocol_version(tls_atom_version() | tls_version()) -> - tls_version() | tls_atom_version(). +-spec protocol_version(dtls_atom_version() | dtls_version()) -> + dtls_version() | dtls_atom_version(). %% %% Description: Creates a protocol version record from a version atom %% or vice versa. @@ -170,7 +193,7 @@ protocol_version({254, 253}) -> protocol_version({254, 255}) -> dtlsv1. %%-------------------------------------------------------------------- --spec lowest_protocol_version(tls_version(), tls_version()) -> tls_version(). +-spec lowest_protocol_version(dtls_version(), dtls_version()) -> dtls_version(). %% %% Description: Lowes protocol version of two given versions %%-------------------------------------------------------------------- @@ -183,7 +206,7 @@ lowest_protocol_version(Version = {M,_}, {N, _}) when M > N -> lowest_protocol_version(_,Version) -> Version. %%-------------------------------------------------------------------- --spec highest_protocol_version([tls_version()]) -> tls_version(). +-spec highest_protocol_version([dtls_version()]) -> dtls_version(). %% %% Description: Highest protocol version present in a list %%-------------------------------------------------------------------- @@ -203,7 +226,7 @@ highest_protocol_version(_, [Version | Rest]) -> %%-------------------------------------------------------------------- --spec supported_protocol_versions() -> [tls_version()]. +-spec supported_protocol_versions() -> [dtls_version()]. %% %% Description: Protocol versions supported %%-------------------------------------------------------------------- @@ -234,7 +257,7 @@ supported_connection_protocol_versions([]) -> ?ALL_DATAGRAM_SUPPORTED_VERSIONS. %%-------------------------------------------------------------------- --spec is_acceptable_version(tls_version(), Supported :: [tls_version()]) -> boolean(). +-spec is_acceptable_version(dtls_version(), Supported :: [dtls_version()]) -> boolean(). %% %% Description: ssl version 2 is not acceptable security risks are too big. %% @@ -244,7 +267,7 @@ is_acceptable_version(Version, Versions) -> %%-------------------------------------------------------------------- --spec init_connection_state_seq(tls_version(), #connection_states{}) -> +-spec init_connection_state_seq(dtls_version(), #connection_states{}) -> #connection_state{}. %% %% Description: Copy the read sequence number to the write sequence number @@ -343,5 +366,5 @@ calc_mac_hash(#connection_state{mac_secret = MacSecret, Length, Fragment). mac_hash(Version, MacAlg, MacSecret, SeqNo, Type, Length, Fragment) -> - dtls_v1:mac_hash(MacAlg, MacSecret, SeqNo, Type, Version, + dtls_v1:mac_hash(Version, MacAlg, MacSecret, SeqNo, Type, Length, Fragment). diff --git a/lib/ssl/src/dtls_record.hrl b/lib/ssl/src/dtls_record.hrl index e935d84bdf..edb77fb2b1 100644 --- a/lib/ssl/src/dtls_record.hrl +++ b/lib/ssl/src/dtls_record.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2013-2013. All Rights Reserved. +%% Copyright Ericsson AB 2013-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -28,16 +28,15 @@ -include("ssl_record.hrl"). %% Common TLS and DTLS records and Constantes -%% Used to handle tls_plain_text, tls_compressed and tls_cipher_text +%% Used to handle dtls_plain_text, dtls_compressed and dtls_cipher_text -record(ssl_tls, { type, version, - record_seq, % used in plain_text - epoch, % used in plain_text - message_seq, - fragment_offset, - fragment_length, + epoch, + sequence_number, + offset, + length, fragment }). diff --git a/lib/ssl/src/dtls_v1.erl b/lib/ssl/src/dtls_v1.erl index 6e41641483..5a7ab32887 100644 --- a/lib/ssl/src/dtls_v1.erl +++ b/lib/ssl/src/dtls_v1.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2013-2013. All Rights Reserved. +%% Copyright Ericsson AB 2013-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -22,7 +22,7 @@ -export([suites/1, mac_hash/7, ecc_curves/1, corresponding_tls_version/1]). --spec suites(Minor:: 253|255) -> [cipher_suite()]. +-spec suites(Minor:: 253|255) -> [ssl_cipher:cipher_suite()]. suites(Minor) -> tls_v1:suites(corresponding_minor_tls_version(Minor)). diff --git a/lib/ssl/src/ssl.app.src b/lib/ssl/src/ssl.app.src index 99839f6149..36681e2897 100644 --- a/lib/ssl/src/ssl.app.src +++ b/lib/ssl/src/ssl.app.src @@ -28,6 +28,7 @@ ssl_srp_primes, ssl_alert, ssl_socket, + ssl_listen_tracker_sup, %% Erlang Distribution over SSL/TLS inet_tls_dist, ssl_tls_dist_proxy, diff --git a/lib/ssl/src/ssl.appup.src b/lib/ssl/src/ssl.appup.src index b0ef292c4e..650901ef54 100644 --- a/lib/ssl/src/ssl.appup.src +++ b/lib/ssl/src/ssl.appup.src @@ -1,33 +1,23 @@ %% -*- erlang -*- {"%VSN%", [ - {"5.3.3", [{load_module, ssl, soft_purge, soft_purge, []}, - {load_module, ssl_connection, soft_purge, soft_purge, []}, - {load_module, ssl_handshake, soft_purge, soft_purge, []}, - {load_module, tls_handshake, soft_purge, soft_purge, []}, - {load_module, tls_connection, soft_purge, soft_purge, []}]}, - {"5.3.2", [{load_module, ssl, soft_purge, soft_purge, []}, - {load_module, ssl_connection, soft_purge, soft_purge, []}, - {load_module, ssl_handshake, soft_purge, soft_purge, []}, - {load_module, tls_handshake, soft_purge, soft_purge, []}, - {load_module, tls_connection, soft_purge, soft_purge, []}]}, - {<<"5\\.3\\.1($|\\..*)">>, [{restart_application, ssl}]}, + {"5.3.5", [{load_module, ssl, soft_purge, soft_purge, [ssl_connection]}, + {load_module, ssl_handshake, soft_purge, soft_purge, [ssl_certificate]}, + {load_module, ssl_certificate, soft_purge, soft_purge, []}, + {load_module, ssl_connection, soft_purge, soft_purge, [tls_connection]}, + {update, tls_connection, {advanced, {up, "5.3.5", "5.3.6"}}, [ssl_handshake]}]}, + {<<"5\\.3\\.[1-4]($|\\..*)">>, [{restart_application, ssl}]}, {<<"5\\.[0-2]($|\\..*)">>, [{restart_application, ssl}]}, {<<"4\\..*">>, [{restart_application, ssl}]}, {<<"3\\..*">>, [{restart_application, ssl}]} ], [ - {"5.3.3", [{load_module, ssl, soft_purge, soft_purge, []}, - {load_module, ssl_connection, soft_purge, soft_purge, []}, - {load_module, ssl_handshake, soft_purge, soft_purge, []}, - {load_module, tls_handshake, soft_purge, soft_purge, []}, - {load_module, tls_connection, soft_purge, soft_purge, []}]}, - {"5.3.2", [{load_module, ssl, soft_purge, soft_purge, []}, - {load_module, ssl_connection, soft_purge, soft_purge, []}, - {load_module, ssl_handshake, soft_purge, soft_purge, []}, - {load_module, tls_handshake, soft_purge, soft_purge, []}, - {load_module, tls_connection, soft_purge, soft_purge, []}]}, - {<<"5\\.3\\.1($|\\..*)">>, [{restart_application, ssl}]}, + {"5.3.5", [{load_module, ssl, soft_purge, soft_purge,[ssl_certificate]}, + {load_module, ssl_handshake, soft_purge, soft_purge,[ssl_certificate]}, + {load_module, ssl_certificate, soft_purge, soft_purge,[]}, + {load_module, ssl_connection, soft_purge, soft_purge,[tls_connection]}, + {update, tls_connection, {advanced, {down, "5.3.6", "5.3.5"}}, [ssl_handshake]}]}, + {<<"5\\.3\\.[1-4]($|\\..*)">>, [{restart_application, ssl}]}, {<<"5\\.[0-2]($|\\..*)">>, [{restart_application, ssl}]}, {<<"4\\..*">>, [{restart_application, ssl}]}, {<<"3\\..*">>, [{restart_application, ssl}]} diff --git a/lib/ssl/src/ssl.erl b/lib/ssl/src/ssl.erl index 9e098e12c4..b4bea25942 100644 --- a/lib/ssl/src/ssl.erl +++ b/lib/ssl/src/ssl.erl @@ -97,17 +97,17 @@ connect(Socket, SslOptions) when is_port(Socket) -> connect(Socket, SslOptions0, Timeout) when is_port(Socket) -> {Transport,_,_,_} = proplists:get_value(cb_info, SslOptions0, {gen_tcp, tcp, tcp_closed, tcp_error}), - EmulatedOptions = emulated_options(), + EmulatedOptions = ssl_socket:emulated_options(), {ok, SocketValues} = ssl_socket:getopts(Transport, Socket, EmulatedOptions), - try handle_options(SslOptions0 ++ SocketValues, client) of + try handle_options(SslOptions0 ++ SocketValues) of {ok, #config{transport_info = CbInfo, ssl = SslOptions, emulated = EmOpts, connection_cb = ConnectionCb}} -> - ok = ssl_socket:setopts(Transport, Socket, internal_inet_values()), + ok = ssl_socket:setopts(Transport, Socket, ssl_socket:internal_inet_values()), case ssl_socket:peername(Transport, Socket) of {ok, {Address, Port}} -> ssl_connection:connect(ConnectionCb, Address, Port, Socket, - {SslOptions, EmOpts}, + {SslOptions, emulated_socket_options(EmOpts, #socket_options{}), undefined}, self(), CbInfo, Timeout); {error, Error} -> {error, Error} @@ -121,7 +121,7 @@ connect(Host, Port, Options) -> connect(Host, Port, Options, infinity). connect(Host, Port, Options, Timeout) -> - try handle_options(Options, client) of + try handle_options(Options) of {ok, Config} -> do_connect(Host,Port,Config,Timeout) catch @@ -139,12 +139,15 @@ listen(_Port, []) -> {error, nooptions}; listen(Port, Options0) -> try - {ok, Config} = handle_options(Options0, server), + {ok, Config} = handle_options(Options0), ConnectionCb = connection_cb(Options0), - #config{transport_info = {Transport, _, _, _}, inet_user = Options, connection_cb = ConnectionCb} = Config, + #config{transport_info = {Transport, _, _, _}, inet_user = Options, connection_cb = ConnectionCb, + ssl = SslOpts, emulated = EmOpts} = Config, case Transport:listen(Port, Options) of {ok, ListenSocket} -> - {ok, #sslsocket{pid = {ListenSocket, Config}}}; + ok = ssl_socket:setopts(Transport, ListenSocket, ssl_socket:internal_inet_values()), + {ok, Tracker} = ssl_socket:inherit_tracker(ListenSocket, EmOpts, SslOpts), + {ok, #sslsocket{pid = {ListenSocket, Config#config{emulated = Tracker}}}}; Err = {error, _} -> Err end @@ -164,25 +167,20 @@ transport_accept(ListenSocket) -> transport_accept(ListenSocket, infinity). transport_accept(#sslsocket{pid = {ListenSocket, - #config{transport_info = CbInfo, + #config{transport_info = {Transport,_,_, _} =CbInfo, connection_cb = ConnectionCb, - ssl = SslOpts}}}, Timeout) -> - %% The setopt could have been invoked on the listen socket - %% and options should be inherited. - EmOptions = emulated_options(), - {Transport,_,_, _} = CbInfo, - {ok, SocketValues} = ssl_socket:getopts(Transport, ListenSocket, EmOptions), - ok = ssl_socket:setopts(Transport, ListenSocket, internal_inet_values()), + ssl = SslOpts, + emulated = Tracker}}}, Timeout) -> case Transport:accept(ListenSocket, Timeout) of {ok, Socket} -> - ok = ssl_socket:setopts(Transport, ListenSocket, SocketValues), + {ok, EmOpts} = ssl_socket:get_emulated_opts(Tracker), {ok, Port} = ssl_socket:port(Transport, Socket), ConnArgs = [server, "localhost", Port, Socket, - {SslOpts, socket_options(SocketValues)}, self(), CbInfo], + {SslOpts, emulated_socket_options(EmOpts, #socket_options{}), Tracker}, self(), CbInfo], ConnectionSup = connection_sup(ConnectionCb), case ConnectionSup:start_child(ConnArgs) of {ok, Pid} -> - ssl_connection:socket_control(ConnectionCb, Socket, Pid, Transport); + ssl_connection:socket_control(ConnectionCb, Socket, Pid, Transport, Tracker); {error, Reason} -> {error, Reason} end; @@ -195,7 +193,8 @@ transport_accept(#sslsocket{pid = {ListenSocket, -spec ssl_accept(#sslsocket{} | port(), timeout()| [ssl_option() | transport_option()]) -> ok | {ok, #sslsocket{}} | {error, reason()}. --spec ssl_accept(port(), [ssl_option()| transport_option()], timeout()) -> + +-spec ssl_accept(#sslsocket{} | port(), [ssl_option()] | [ssl_option()| transport_option()], timeout()) -> {ok, #sslsocket{}} | {error, reason()}. %% %% Description: Performs accept on an ssl listen socket. e.i. performs @@ -210,19 +209,29 @@ ssl_accept(#sslsocket{} = Socket, Timeout) -> ssl_accept(ListenSocket, SslOptions) when is_port(ListenSocket) -> ssl_accept(ListenSocket, SslOptions, infinity). +ssl_accept(#sslsocket{} = Socket, [], Timeout) -> + ssl_accept(#sslsocket{} = Socket, Timeout); +ssl_accept(#sslsocket{fd = {_, _, _, Tracker}} = Socket, SslOpts0, Timeout) -> + try + {ok, EmOpts, InheritedSslOpts} = ssl_socket:get_all_opts(Tracker), + SslOpts = handle_options(SslOpts0, InheritedSslOpts), + ssl_connection:handshake(Socket, {SslOpts, emulated_socket_options(EmOpts, #socket_options{})}, Timeout) + catch + Error = {error, _Reason} -> Error + end; ssl_accept(Socket, SslOptions, Timeout) when is_port(Socket) -> {Transport,_,_,_} = proplists:get_value(cb_info, SslOptions, {gen_tcp, tcp, tcp_closed, tcp_error}), - EmulatedOptions = emulated_options(), + EmulatedOptions = ssl_socket:emulated_options(), {ok, SocketValues} = ssl_socket:getopts(Transport, Socket, EmulatedOptions), ConnetionCb = connection_cb(SslOptions), - try handle_options(SslOptions ++ SocketValues, server) of + try handle_options(SslOptions ++ SocketValues) of {ok, #config{transport_info = CbInfo, ssl = SslOpts, emulated = EmOpts}} -> - ok = ssl_socket:setopts(Transport, Socket, internal_inet_values()), + ok = ssl_socket:setopts(Transport, Socket, ssl_socket:internal_inet_values()), {ok, Port} = ssl_socket:port(Transport, Socket), ssl_connection:ssl_accept(ConnetionCb, Port, Socket, - {SslOpts, EmOpts}, - self(), CbInfo, Timeout) + {SslOpts, emulated_socket_options(EmOpts, #socket_options{}), undefined}, + self(), CbInfo, Timeout) catch Error = {error, _Reason} -> Error end. @@ -276,7 +285,7 @@ controlling_process(#sslsocket{pid = {Listen, Transport:controlling_process(Listen, NewOwner). %%-------------------------------------------------------------------- --spec connection_info(#sslsocket{}) -> {ok, {tls_atom_version(), erl_cipher_suite()}} | +-spec connection_info(#sslsocket{}) -> {ok, {tls_record:tls_atom_version(), ssl_cipher:erl_cipher_suite()}} | {error, reason()}. %% %% Description: Returns ssl protocol and cipher used for the connection @@ -291,7 +300,7 @@ connection_info(#sslsocket{pid = {Listen, _}}) when is_port(Listen) -> %% %% Description: same as inet:peername/1. %%-------------------------------------------------------------------- -peername(#sslsocket{pid = Pid, fd = {Transport, Socket, _}}) when is_pid(Pid)-> +peername(#sslsocket{pid = Pid, fd = {Transport, Socket, _, _}}) when is_pid(Pid)-> ssl_socket:peername(Transport, Socket); peername(#sslsocket{pid = {ListenSocket, #config{transport_info = {Transport,_,_,_}}}}) -> ssl_socket:peername(Transport, ListenSocket). %% Will return {error, enotconn} @@ -312,7 +321,7 @@ peercert(#sslsocket{pid = {Listen, _}}) when is_port(Listen) -> {error, enotconn}. %%-------------------------------------------------------------------- --spec suite_definition(cipher_suite()) -> erl_cipher_suite(). +-spec suite_definition(ssl_cipher:cipher_suite()) -> ssl_cipher:erl_cipher_suite(). %% %% Description: Return erlang cipher suite definition. %%-------------------------------------------------------------------- @@ -330,28 +339,28 @@ negotiated_next_protocol(#sslsocket{pid = Pid}) -> ssl_connection:negotiated_next_protocol(Pid). %%-------------------------------------------------------------------- --spec cipher_suites() -> [erl_cipher_suite()]. --spec cipher_suites(erlang | openssl | all) -> [erl_cipher_suite()] | [string()]. - +-spec cipher_suites(erlang | openssl | all) -> [ssl_cipher:erl_cipher_suite()] | + [string()]. %% Description: Returns all supported cipher suites. %%-------------------------------------------------------------------- -cipher_suites() -> - cipher_suites(erlang). - cipher_suites(erlang) -> Version = tls_record:highest_protocol_version([]), - [suite_definition(S) || S <- ssl_cipher:suites(Version)]; - + ssl_cipher:filter_suites([suite_definition(S) + || S <- ssl_cipher:suites(Version)]); cipher_suites(openssl) -> Version = tls_record:highest_protocol_version([]), - [ssl_cipher:openssl_suite_name(S) || S <- ssl_cipher:suites(Version)]; + [ssl_cipher:openssl_suite_name(S) + || S <- ssl_cipher:filter_suites(ssl_cipher:suites(Version))]; cipher_suites(all) -> Version = tls_record:highest_protocol_version([]), - Supported = ssl_cipher:suites(Version) + Supported = ssl_cipher:all_suites(Version) ++ ssl_cipher:anonymous_suites() ++ ssl_cipher:psk_suites(Version) ++ ssl_cipher:srp_suites(), - [suite_definition(S) || S <- Supported]. + ssl_cipher:filter_suites([suite_definition(S) || S <- Supported]). + +cipher_suites() -> + cipher_suites(erlang). %%-------------------------------------------------------------------- -spec getopts(#sslsocket{}, [gen_tcp:option_name()]) -> @@ -361,7 +370,7 @@ cipher_suites(all) -> %%-------------------------------------------------------------------- getopts(#sslsocket{pid = Pid}, OptionTags) when is_pid(Pid), is_list(OptionTags) -> ssl_connection:get_opts(Pid, OptionTags); -getopts(#sslsocket{pid = {ListenSocket, #config{transport_info = {Transport,_,_,_}}}}, +getopts(#sslsocket{pid = {_, #config{transport_info = {Transport,_,_,_}}}} = ListenSocket, OptionTags) when is_list(OptionTags) -> try ssl_socket:getopts(Transport, ListenSocket, OptionTags) of {ok, _} = Result -> @@ -369,8 +378,8 @@ getopts(#sslsocket{pid = {ListenSocket, #config{transport_info = {Transport,_,_ {error, InetError} -> {error, {options, {socket_options, OptionTags, InetError}}} catch - _:_ -> - {error, {options, {socket_options, OptionTags}}} + _:Error -> + {error, {options, {socket_options, OptionTags, Error}}} end; getopts(#sslsocket{}, OptionTags) -> {error, {options, {socket_options, OptionTags}}}. @@ -390,7 +399,7 @@ setopts(#sslsocket{pid = Pid}, Options0) when is_pid(Pid), is_list(Options0) -> {error, {options, {not_a_proplist, Options0}}} end; -setopts(#sslsocket{pid = {ListenSocket, #config{transport_info = {Transport,_,_,_}}}}, Options) when is_list(Options) -> +setopts(#sslsocket{pid = {_, #config{transport_info = {Transport,_,_,_}}}} = ListenSocket, Options) when is_list(Options) -> try ssl_socket:setopts(Transport, ListenSocket, Options) of ok -> ok; @@ -419,10 +428,10 @@ shutdown(#sslsocket{pid = Pid}, How) -> %% %% Description: Same as inet:sockname/1 %%-------------------------------------------------------------------- -sockname(#sslsocket{pid = {Listen, #config{transport_info = {Transport,_, _, _}}}}) when is_port(Listen) -> +sockname(#sslsocket{pid = {Listen, #config{transport_info = {Transport, _, _, _}}}}) when is_port(Listen) -> ssl_socket:sockname(Transport, Listen); -sockname(#sslsocket{pid = Pid, fd = {Transport, Socket, _}}) when is_pid(Pid) -> +sockname(#sslsocket{pid = Pid, fd = {Transport, Socket, _, _}}) when is_pid(Pid) -> ssl_socket:sockname(Transport, Socket). %%--------------------------------------------------------------- @@ -437,8 +446,8 @@ session_info(#sslsocket{pid = {Listen,_}}) when is_port(Listen) -> {error, enotconn}. %%--------------------------------------------------------------- --spec versions() -> [{ssl_app, string()} | {supported, [tls_atom_version()]} | - {available, [tls_atom_version()]}]. +-spec versions() -> [{ssl_app, string()} | {supported, [tls_record:tls_atom_version()]} | + {available, [tls_record:tls_atom_version()]}]. %% %% Description: Returns a list of relevant versions. %%-------------------------------------------------------------------- @@ -541,7 +550,8 @@ do_connect(Address, Port, {Transport, _, _, _} = CbInfo, try Transport:connect(Address, Port, SocketOpts, Timeout) of {ok, Socket} -> - ssl_connection:connect(ConnetionCb, Address, Port, Socket, {SslOpts,EmOpts}, + ssl_connection:connect(ConnetionCb, Address, Port, Socket, + {SslOpts, emulated_socket_options(EmOpts, #socket_options{}), undefined}, self(), CbInfo, Timeout); {error, Reason} -> {error, Reason} @@ -554,64 +564,66 @@ do_connect(Address, Port, {error, {options, {socket_options, UserOpts}}} end. -handle_options(Opts0, _Role) -> +%% Handle extra ssl options given to ssl_accept +handle_options(Opts0, #ssl_options{protocol = Protocol, cacerts = CaCerts0, + cacertfile = CaCertFile0} = InheritedSslOpts) -> + RecordCB = record_cb(Protocol), + CaCerts = handle_option(cacerts, Opts0, CaCerts0), + {Verify, FailIfNoPeerCert, CaCertDefault, VerifyFun, PartialChainHanlder} = handle_verify_options(Opts0, CaCerts), + CaCertFile = case proplists:get_value(cacertfile, Opts0, CaCertFile0) of + undefined -> + CaCertDefault; + CAFile -> + CAFile + end, + + NewVerifyOpts = InheritedSslOpts#ssl_options{cacerts = CaCerts, + cacertfile = CaCertFile, + verify = Verify, + verify_fun = VerifyFun, + partial_chain = PartialChainHanlder, + fail_if_no_peer_cert = FailIfNoPeerCert}, + SslOpts1 = lists:foldl(fun(Key, PropList) -> + proplists:delete(Key, PropList) + end, Opts0, [cacerts, cacertfile, verify, verify_fun, partial_chain, + fail_if_no_peer_cert]), + case handle_option(versions, SslOpts1, []) of + [] -> + new_ssl_options(SslOpts1, NewVerifyOpts, RecordCB); + Value -> + Versions = [RecordCB:protocol_version(Vsn) || Vsn <- Value], + new_ssl_options(proplists:delete(versions, SslOpts1), + NewVerifyOpts#ssl_options{versions = Versions}, record_cb(Protocol)) + end. + +%% Handle all options in listen and connect +handle_options(Opts0) -> Opts = proplists:expand([{binary, [{mode, binary}]}, {list, [{mode, list}]}], Opts0), assert_proplist(Opts), - ReuseSessionFun = fun(_, _, _, _) -> true end, - - DefaultVerifyNoneFun = - {fun(_,{bad_cert, _}, UserState) -> - {valid, UserState}; - (_,{extension, _}, UserState) -> - {unknown, UserState}; - (_, valid, UserState) -> - {valid, UserState}; - (_, valid_peer, UserState) -> - {valid, UserState} - end, []}, + RecordCb = record_cb(Opts), - VerifyNoneFun = handle_option(verify_fun, Opts, DefaultVerifyNoneFun), - - UserFailIfNoPeerCert = handle_option(fail_if_no_peer_cert, Opts, false), - UserVerifyFun = handle_option(verify_fun, Opts, undefined), + ReuseSessionFun = fun(_, _, _, _) -> true end, CaCerts = handle_option(cacerts, Opts, undefined), - {Verify, FailIfNoPeerCert, CaCertDefault, VerifyFun} = - %% Handle 0, 1, 2 for backwards compatibility - case proplists:get_value(verify, Opts, verify_none) of - 0 -> - {verify_none, false, - ca_cert_default(verify_none, VerifyNoneFun, CaCerts), VerifyNoneFun}; - 1 -> - {verify_peer, false, - ca_cert_default(verify_peer, UserVerifyFun, CaCerts), UserVerifyFun}; - 2 -> - {verify_peer, true, - ca_cert_default(verify_peer, UserVerifyFun, CaCerts), UserVerifyFun}; - verify_none -> - {verify_none, false, - ca_cert_default(verify_none, VerifyNoneFun, CaCerts), VerifyNoneFun}; - verify_peer -> - {verify_peer, UserFailIfNoPeerCert, - ca_cert_default(verify_peer, UserVerifyFun, CaCerts), UserVerifyFun}; - Value -> - throw({error, {options, {verify, Value}}}) - end, - + {Verify, FailIfNoPeerCert, CaCertDefault, VerifyFun, PartialChainHanlder} = + handle_verify_options(Opts, CaCerts), + CertFile = handle_option(certfile, Opts, <<>>), - + RecordCb = record_cb(Opts), + Versions = case handle_option(versions, Opts, []) of [] -> - tls_record:supported_protocol_versions(); + RecordCb:supported_protocol_versions(); Vsns -> - [tls_record:protocol_version(Vsn) || Vsn <- Vsns] + [RecordCb:protocol_version(Vsn) || Vsn <- Vsns] end, SSLOptions = #ssl_options{ versions = Versions, verify = validate_option(verify, Verify), verify_fun = VerifyFun, + partial_chain = PartialChainHanlder, fail_if_no_peer_cert = FailIfNoPeerCert, verify_client_once = handle_option(verify_client_once, Opts, false), depth = handle_option(depth, Opts, 1), @@ -627,7 +639,8 @@ handle_options(Opts0, _Role) -> user_lookup_fun = handle_option(user_lookup_fun, Opts, undefined), psk_identity = handle_option(psk_identity, Opts, undefined), srp_identity = handle_option(srp_identity, Opts, undefined), - ciphers = handle_cipher_option(proplists:get_value(ciphers, Opts, []), hd(Versions)), + ciphers = handle_cipher_option(proplists:get_value(ciphers, Opts, []), + RecordCb:highest_protocol_version(Versions)), %% Server side option reuse_session = handle_option(reuse_session, Opts, ReuseSessionFun), reuse_sessions = handle_option(reuse_sessions, Opts, true), @@ -642,11 +655,12 @@ handle_options(Opts0, _Role) -> handle_option(client_preferred_next_protocols, Opts, undefined)), log_alert = handle_option(log_alert, Opts, true), server_name_indication = handle_option(server_name_indication, Opts, undefined), - honor_cipher_order = handle_option(honor_cipher_order, Opts, false) + honor_cipher_order = handle_option(honor_cipher_order, Opts, false), + protocol = proplists:get_value(protocol, Opts, tls) }, CbInfo = proplists:get_value(cb_info, Opts, {gen_tcp, tcp, tcp_closed, tcp_error}), - SslOptions = [protocol, versions, verify, verify_fun, + SslOptions = [protocol, versions, verify, verify_fun, partial_chain, fail_if_no_peer_cert, verify_client_once, depth, cert, certfile, key, keyfile, password, cacerts, cacertfile, dh, dhfile, @@ -661,10 +675,10 @@ handle_options(Opts0, _Role) -> proplists:delete(Key, PropList) end, Opts, SslOptions), - {SSLsock, Emulated} = emulated_options(SockOpts), + {Sock, Emulated} = emulated_options(SockOpts), ConnetionCb = connection_cb(Opts), - {ok, #config{ssl = SSLOptions, emulated = Emulated, inet_ssl = SSLsock, + {ok, #config{ssl = SSLOptions, emulated = Emulated, inet_ssl = Sock, inet_user = SockOpts, transport_info = CbInfo, connection_cb = ConnetionCb }}. @@ -698,6 +712,8 @@ validate_option(verify_fun, Fun) when is_function(Fun) -> end, Fun}; validate_option(verify_fun, {Fun, _} = Value) when is_function(Fun) -> Value; +validate_option(partial_chain, Value) when is_function(Value) -> + Value; validate_option(fail_if_no_peer_cert, Value) when is_boolean(Value) -> Value; validate_option(verify_client_once, Value) when is_boolean(Value) -> @@ -890,40 +906,24 @@ ca_cert_default(verify_peer, {Fun,_}, _) when is_function(Fun) -> %% some trusted certs. ca_cert_default(verify_peer, undefined, _) -> "". - -emulated_options() -> - [mode, packet, active, header, packet_size]. - -internal_inet_values() -> - [{packet_size,0},{packet, 0},{header, 0},{active, false},{mode,binary}]. - -socket_options(InetValues) -> - #socket_options{ - mode = proplists:get_value(mode, InetValues, lists), - header = proplists:get_value(header, InetValues, 0), - active = proplists:get_value(active, InetValues, active), - packet = proplists:get_value(packet, InetValues, 0), - packet_size = proplists:get_value(packet_size, InetValues) - }. - emulated_options(Opts) -> - emulated_options(Opts, internal_inet_values(), #socket_options{}). - -emulated_options([{mode,Opt}|Opts], Inet, Emulated) -> - validate_inet_option(mode,Opt), - emulated_options(Opts, Inet, Emulated#socket_options{mode=Opt}); -emulated_options([{header,Opt}|Opts], Inet, Emulated) -> - validate_inet_option(header,Opt), - emulated_options(Opts, Inet, Emulated#socket_options{header=Opt}); -emulated_options([{active,Opt}|Opts], Inet, Emulated) -> - validate_inet_option(active,Opt), - emulated_options(Opts, Inet, Emulated#socket_options{active=Opt}); -emulated_options([{packet,Opt}|Opts], Inet, Emulated) -> - validate_inet_option(packet,Opt), - emulated_options(Opts, Inet, Emulated#socket_options{packet=Opt}); -emulated_options([{packet_size,Opt}|Opts], Inet, Emulated) -> - validate_inet_option(packet_size,Opt), - emulated_options(Opts, Inet, Emulated#socket_options{packet_size=Opt}); + emulated_options(Opts, ssl_socket:internal_inet_values(), ssl_socket:default_inet_values()). + +emulated_options([{mode, Value} = Opt |Opts], Inet, Emulated) -> + validate_inet_option(mode, Value), + emulated_options(Opts, Inet, [Opt | proplists:delete(mode, Emulated)]); +emulated_options([{header, Value} = Opt | Opts], Inet, Emulated) -> + validate_inet_option(header, Value), + emulated_options(Opts, Inet, [Opt | proplists:delete(header, Emulated)]); +emulated_options([{active, Value} = Opt |Opts], Inet, Emulated) -> + validate_inet_option(active, Value), + emulated_options(Opts, Inet, [Opt | proplists:delete(active, Emulated)]); +emulated_options([{packet, Value} = Opt |Opts], Inet, Emulated) -> + validate_inet_option(packet, Value), + emulated_options(Opts, Inet, [Opt | proplists:delete(packet, Emulated)]); +emulated_options([{packet_size, Value} = Opt | Opts], Inet, Emulated) -> + validate_inet_option(packet_size, Value), + emulated_options(Opts, Inet, [Opt | proplists:delete(packet_size, Emulated)]); emulated_options([Opt|Opts], Inet, Emulated) -> emulated_options(Opts, [Opt|Inet], Emulated); emulated_options([], Inet,Emulated) -> @@ -939,8 +939,11 @@ handle_cipher_option(Value, Version) when is_list(Value) -> error:_-> throw({error, {options, {ciphers, Value}}}) end. -binary_cipher_suites(Version, []) -> %% Defaults to all supported suits - ssl_cipher:suites(Version); + +binary_cipher_suites(Version, []) -> + %% Defaults to all supported suites that does + %% not require explicit configuration + ssl_cipher:filter_suites(ssl_cipher:suites(Version)); binary_cipher_suites(Version, [{_,_,_,_}| _] = Ciphers0) -> %% Backwards compatibility Ciphers = [{KeyExchange, Cipher, Hash} || {KeyExchange, Cipher, Hash, _} <- Ciphers0], binary_cipher_suites(Version, Ciphers); @@ -949,14 +952,15 @@ binary_cipher_suites(Version, [{_,_,_}| _] = Ciphers0) -> binary_cipher_suites(Version, Ciphers); binary_cipher_suites(Version, [Cipher0 | _] = Ciphers0) when is_binary(Cipher0) -> - Supported0 = ssl_cipher:suites(Version) + All = ssl_cipher:suites(Version) ++ ssl_cipher:anonymous_suites() ++ ssl_cipher:psk_suites(Version) ++ ssl_cipher:srp_suites(), - Supported = ssl_cipher:filter_suites(Supported0), - case [Cipher || Cipher <- Ciphers0, lists:member(Cipher, Supported)] of + case [Cipher || Cipher <- Ciphers0, lists:member(Cipher, All)] of [] -> - Supported; %% Defaults to all supported suits + %% Defaults to all supported suites that does + %% not require explicit configuration + ssl_cipher:filter_suites(ssl_cipher:suites(Version)); Ciphers -> Ciphers end; @@ -1035,6 +1039,13 @@ connection_cb(dtls) -> connection_cb(Opts) -> connection_cb(proplists:get_value(protocol, Opts, tls)). +record_cb(tls) -> + tls_record; +record_cb(dtls) -> + dtls_record; +record_cb(Opts) -> + record_cb(proplists:get_value(protocol, Opts, tls)). + connection_sup(tls_connection) -> tls_connection_sup; connection_sup(dtls_connection) -> @@ -1055,3 +1066,119 @@ assert_proplist([inet6 | Rest]) -> assert_proplist(Rest); assert_proplist([Value | _]) -> throw({option_not_a_key_value_tuple, Value}). + +emulated_socket_options(InetValues, #socket_options{ + mode = Mode, + header = Header, + active = Active, + packet = Packet, + packet_size = Size}) -> + #socket_options{ + mode = proplists:get_value(mode, InetValues, Mode), + header = proplists:get_value(header, InetValues, Header), + active = proplists:get_value(active, InetValues, Active), + packet = proplists:get_value(packet, InetValues, Packet), + packet_size = proplists:get_value(packet_size, InetValues, Size) + }. + +new_ssl_options([], #ssl_options{} = Opts, _) -> + Opts; +new_ssl_options([{verify_client_once, Value} | Rest], #ssl_options{} = Opts, RecordCB) -> + new_ssl_options(Rest, Opts#ssl_options{verify_client_once = validate_option(verify_client_once, Value)}, RecordCB); +new_ssl_options([{depth, Value} | Rest], #ssl_options{} = Opts, RecordCB) -> + new_ssl_options(Rest, Opts#ssl_options{depth = validate_option(depth, Value)}, RecordCB); +new_ssl_options([{cert, Value} | Rest], #ssl_options{} = Opts, RecordCB) -> + new_ssl_options(Rest, Opts#ssl_options{cert = validate_option(cert, Value)}, RecordCB); +new_ssl_options([{certfile, Value} | Rest], #ssl_options{} = Opts, RecordCB) -> + new_ssl_options(Rest, Opts#ssl_options{certfile = validate_option(certfile, Value)}, RecordCB); +new_ssl_options([{key, Value} | Rest], #ssl_options{} = Opts, RecordCB) -> + new_ssl_options(Rest, Opts#ssl_options{key = validate_option(key, Value)}, RecordCB); +new_ssl_options([{keyfile, Value} | Rest], #ssl_options{} = Opts, RecordCB) -> + new_ssl_options(Rest, Opts#ssl_options{keyfile = validate_option(keyfile, Value)}, RecordCB); +new_ssl_options([{password, Value} | Rest], #ssl_options{} = Opts, RecordCB) -> + new_ssl_options(Rest, Opts#ssl_options{password = validate_option(password, Value)}, RecordCB); +new_ssl_options([{dh, Value} | Rest], #ssl_options{} = Opts, RecordCB) -> + new_ssl_options(Rest, Opts#ssl_options{dh = validate_option(dh, Value)}, RecordCB); +new_ssl_options([{dhfile, Value} | Rest], #ssl_options{} = Opts, RecordCB) -> + new_ssl_options(Rest, Opts#ssl_options{dhfile = validate_option(dhfile, Value)}, RecordCB); +new_ssl_options([{user_lookup_fun, Value} | Rest], #ssl_options{} = Opts, RecordCB) -> + new_ssl_options(Rest, Opts#ssl_options{user_lookup_fun = validate_option(user_lookup_fun, Value)}, RecordCB); +new_ssl_options([{psk_identity, Value} | Rest], #ssl_options{} = Opts, RecordCB) -> + new_ssl_options(Rest, Opts#ssl_options{psk_identity = validate_option(psk_identity, Value)}, RecordCB); +new_ssl_options([{srp_identity, Value} | Rest], #ssl_options{} = Opts, RecordCB) -> + new_ssl_options(Rest, Opts#ssl_options{srp_identity = validate_option(srp_identity, Value)}, RecordCB); +new_ssl_options([{ciphers, Value} | Rest], #ssl_options{versions = Versions} = Opts, RecordCB) -> + Ciphers = handle_cipher_option(Value, RecordCB:highest_protocol_version(Versions)), + new_ssl_options(Rest, + Opts#ssl_options{ciphers = Ciphers}, RecordCB); +new_ssl_options([{reuse_session, Value} | Rest], #ssl_options{} = Opts, RecordCB) -> + new_ssl_options(Rest, Opts#ssl_options{reuse_session = validate_option(reuse_session, Value)}, RecordCB); +new_ssl_options([{reuse_sessions, Value} | Rest], #ssl_options{} = Opts, RecordCB) -> + new_ssl_options(Rest, Opts#ssl_options{reuse_sessions = validate_option(reuse_sessions, Value)}, RecordCB); +new_ssl_options([{ssl_imp, _Value} | Rest], #ssl_options{} = Opts, RecordCB) -> %% Not used backwards compatibility + new_ssl_options(Rest, Opts, RecordCB); +new_ssl_options([{renegotiate_at, Value} | Rest], #ssl_options{} = Opts, RecordCB) -> + new_ssl_options(Rest, Opts#ssl_options{ renegotiate_at = validate_option(renegotiate_at, Value)}, RecordCB); +new_ssl_options([{secure_renegotiate, Value} | Rest], #ssl_options{} = Opts, RecordCB) -> + new_ssl_options(Rest, Opts#ssl_options{secure_renegotiate = validate_option(secure_renegotiate, Value)}, RecordCB); +new_ssl_options([{hibernate_after, Value} | Rest], #ssl_options{} = Opts, RecordCB) -> + new_ssl_options(Rest, Opts#ssl_options{hibernate_after = validate_option(hibernate_after, Value)}, RecordCB); +new_ssl_options([{next_protocols_advertised, Value} | Rest], #ssl_options{} = Opts, RecordCB) -> + new_ssl_options(Rest, Opts#ssl_options{next_protocols_advertised = validate_option(next_protocols_advertised, Value)}, RecordCB); +new_ssl_options([{client_preferred_next_protocols, Value} | Rest], #ssl_options{} = Opts, RecordCB) -> + new_ssl_options(Rest, Opts#ssl_options{next_protocol_selector = + make_next_protocol_selector(validate_option(client_preferred_next_protocols, Value))}, RecordCB); +new_ssl_options([{log_alert, Value} | Rest], #ssl_options{} = Opts, RecordCB) -> + new_ssl_options(Rest, Opts#ssl_options{log_alert = validate_option(log_alert, Value)}, RecordCB); +new_ssl_options([{server_name_indication, Value} | Rest], #ssl_options{} = Opts, RecordCB) -> + new_ssl_options(Rest, Opts#ssl_options{server_name_indication = validate_option(server_name_indication, Value)}, RecordCB); +new_ssl_options([{honor_cipher_order, Value} | Rest], #ssl_options{} = Opts, RecordCB) -> + new_ssl_options(Rest, Opts#ssl_options{honor_cipher_order = validate_option(honor_cipher_order, Value)}, RecordCB); +new_ssl_options([{Key, Value} | _Rest], #ssl_options{}, _) -> + throw({error, {options, {Key, Value}}}). + + +handle_verify_options(Opts, CaCerts) -> + DefaultVerifyNoneFun = + {fun(_,{bad_cert, _}, UserState) -> + {valid, UserState}; + (_,{extension, _}, UserState) -> + {unknown, UserState}; + (_, valid, UserState) -> + {valid, UserState}; + (_, valid_peer, UserState) -> + {valid, UserState} + end, []}, + VerifyNoneFun = handle_option(verify_fun, Opts, DefaultVerifyNoneFun), + + UserFailIfNoPeerCert = handle_option(fail_if_no_peer_cert, Opts, false), + UserVerifyFun = handle_option(verify_fun, Opts, undefined), + + PartialChainHanlder = handle_option(partial_chain, Opts, + fun(_) -> unknown_ca end), + + %% Handle 0, 1, 2 for backwards compatibility + case proplists:get_value(verify, Opts, verify_none) of + 0 -> + {verify_none, false, + ca_cert_default(verify_none, VerifyNoneFun, CaCerts), + VerifyNoneFun, PartialChainHanlder}; + 1 -> + {verify_peer, false, + ca_cert_default(verify_peer, UserVerifyFun, CaCerts), + UserVerifyFun, PartialChainHanlder}; + 2 -> + {verify_peer, true, + ca_cert_default(verify_peer, UserVerifyFun, CaCerts), + UserVerifyFun, PartialChainHanlder}; + verify_none -> + {verify_none, false, + ca_cert_default(verify_none, VerifyNoneFun, CaCerts), + VerifyNoneFun, PartialChainHanlder}; + verify_peer -> + {verify_peer, UserFailIfNoPeerCert, + ca_cert_default(verify_peer, UserVerifyFun, CaCerts), + UserVerifyFun, PartialChainHanlder}; + Value -> + throw({error, {options, {verify, Value}}}) + end. diff --git a/lib/ssl/src/ssl_alert.erl b/lib/ssl/src/ssl_alert.erl index 5c842b4d19..78dc98bc25 100644 --- a/lib/ssl/src/ssl_alert.erl +++ b/lib/ssl/src/ssl_alert.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2013. All Rights Reserved. +%% Copyright Ericsson AB 2007-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -31,22 +31,31 @@ -include("ssl_record.hrl"). -include("ssl_internal.hrl"). --export([encode/3, alert_txt/1, reason_code/2]). +-export([encode/3, decode/1, alert_txt/1, reason_code/2]). %%==================================================================== %% Internal application API %%==================================================================== %%-------------------------------------------------------------------- --spec encode(#alert{}, tls_version(), #connection_states{}) -> +-spec encode(#alert{}, ssl_record:ssl_version(), #connection_states{}) -> {iolist(), #connection_states{}}. %% -%% Description: +%% Description: Encodes an alert %%-------------------------------------------------------------------- encode(#alert{} = Alert, Version, ConnectionStates) -> ssl_record:encode_alert_record(Alert, Version, ConnectionStates). %%-------------------------------------------------------------------- +-spec decode(binary()) -> [#alert{}] | #alert{}. +%% +%% Description: Decode alert(s), will return a singel own alert if peer +%% sends garbage or too many warning alerts. +%%-------------------------------------------------------------------- +decode(Bin) -> + decode(Bin, [], 0). + +%%-------------------------------------------------------------------- -spec reason_code(#alert{}, client | server) -> closed | {essl, string()}. %% %% Description: Returns the error reason that will be returned to the @@ -71,6 +80,22 @@ alert_txt(#alert{level = Level, description = Description, where = {Mod,Line}}) %%-------------------------------------------------------------------- %%% Internal functions %%-------------------------------------------------------------------- + +%% It is very unlikely that an correct implementation will send more than one alert at the time +%% So it there is more than 10 warning alerts we consider it an error +decode(<<?BYTE(Level), ?BYTE(_), _/binary>>, _, N) when Level == ?WARNING, N > ?MAX_ALERTS -> + ?ALERT_REC(?FATAL, ?DECODE_ERROR); +decode(<<?BYTE(Level), ?BYTE(Description), Rest/binary>>, Acc, N) when Level == ?WARNING -> + Alert = ?ALERT_REC(Level, Description), + decode(Rest, [Alert | Acc], N + 1); +decode(<<?BYTE(Level), ?BYTE(Description), _Rest/binary>>, Acc, _) when Level == ?FATAL-> + Alert = ?ALERT_REC(Level, Description), + lists:reverse([Alert | Acc]); %% No need to decode rest fatal alert will end the connection +decode(<<?BYTE(_Level), _/binary>>, _, _) -> + ?ALERT_REC(?FATAL, ?ILLEGAL_PARAMETER); +decode(<<>>, Acc, _) -> + lists:reverse(Acc, []). + level_txt(?WARNING) -> "Warning:"; level_txt(?FATAL) -> @@ -124,5 +149,17 @@ description_txt(?USER_CANCELED) -> "user canceled"; description_txt(?NO_RENEGOTIATION) -> "no renegotiation"; +description_txt(?UNSUPPORTED_EXTENSION) -> + "unsupported extension"; +description_txt(?CERTIFICATE_UNOBTAINABLE) -> + "certificate unobtainable"; +description_txt(?UNRECOGNISED_NAME) -> + "unrecognised name"; +description_txt(?BAD_CERTIFICATE_STATUS_RESPONSE) -> + "bad certificate status response"; +description_txt(?BAD_CERTIFICATE_HASH_VALUE) -> + "bad certificate hash value"; description_txt(?UNKNOWN_PSK_IDENTITY) -> - "unknown psk identity". + "unknown psk identity"; +description_txt(Enum) -> + lists:flatten(io_lib:format("unsupported/unknown alert: ~p", [Enum])). diff --git a/lib/ssl/src/ssl_alert.hrl b/lib/ssl/src/ssl_alert.hrl index 2a8a91aefa..f4f1d74264 100644 --- a/lib/ssl/src/ssl_alert.hrl +++ b/lib/ssl/src/ssl_alert.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2012. All Rights Reserved. +%% Copyright Ericsson AB 2007-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -48,7 +48,7 @@ %% unsupported_certificate(43), %% certificate_revoked(44), %% certificate_expired(45), - %% certificate_unknown(46), +%% certificate_unknown(46), %% illegal_parameter(47), %% unknown_ca(48), %% access_denied(49), @@ -60,6 +60,13 @@ %% internal_error(80), %% user_canceled(90), %% no_renegotiation(100), +%% RFC 4366 +%% unsupported_extension(110), +%% certificate_unobtainable(111), +%% unrecognized_name(112), +%% bad_certificate_status_response(113), +%% bad_certificate_hash_value(114), +%% RFC 4366 %% unknown_psk_identity(115), %% (255) %% } AlertDescription; @@ -88,10 +95,17 @@ -define(INTERNAL_ERROR, 80). -define(USER_CANCELED, 90). -define(NO_RENEGOTIATION, 100). +-define(UNSUPPORTED_EXTENSION, 110). +-define(CERTIFICATE_UNOBTAINABLE, 111). +-define(UNRECOGNISED_NAME, 112). +-define(BAD_CERTIFICATE_STATUS_RESPONSE, 113). +-define(BAD_CERTIFICATE_HASH_VALUE, 114). -define(UNKNOWN_PSK_IDENTITY, 115). -define(ALERT_REC(Level,Desc), #alert{level=Level,description=Desc,where={?FILE, ?LINE}}). +-define(MAX_ALERTS, 10). + %% Alert -record(alert, { level, diff --git a/lib/ssl/src/ssl_api.hrl b/lib/ssl/src/ssl_api.hrl index 607991750f..22185ff60a 100644 --- a/lib/ssl/src/ssl_api.hrl +++ b/lib/ssl/src/ssl_api.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2013-2013. All Rights Reserved. +%% Copyright Ericsson AB 2013-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -24,8 +24,6 @@ %% Visible in API -export_type([connect_option/0, listen_option/0, ssl_option/0, transport_option/0, - erl_cipher_suite/0, %% From ssl_cipher.hrl - tls_atom_version/0, %% From ssl_internal.hrl prf_random/0, sslsocket/0]). @@ -39,23 +37,24 @@ -type listen_option() :: socket_listen_option() | ssl_option() | transport_option(). -type socket_listen_option() :: gen_tcp:listen_option(). --type ssl_option() :: {verify, verify_type()} | - {verify_fun, {fun(), InitialUserState::term()}} | - {fail_if_no_peer_cert, boolean()} | {depth, integer()} | - {cert, Der::binary()} | {certfile, path()} | {key, Der::binary()} | - {keyfile, path()} | {password, string()} | {cacerts, [Der::binary()]} | - {cacertfile, path()} | {dh, Der::binary()} | {dhfile, path()} | - {user_lookup_fun, {fun(), InitialUserState::term()}} | - {psk_identity, string()} | - {srp_identity, {string(), string()}} | - {ciphers, ciphers()} | {ssl_imp, ssl_imp()} | {reuse_sessions, boolean()} | - {reuse_session, fun()} | {hibernate_after, integer()|undefined} | - {next_protocols_advertised, list(binary())} | - {client_preferred_next_protocols, binary(), client | server, list(binary())}. +-type ssl_option() :: {versions, ssl_record:ssl_atom_version()} | + {verify, verify_type()} | + {verify_fun, {fun(), InitialUserState::term()}} | + {fail_if_no_peer_cert, boolean()} | {depth, integer()} | + {cert, Der::binary()} | {certfile, path()} | {key, Der::binary()} | + {keyfile, path()} | {password, string()} | {cacerts, [Der::binary()]} | + {cacertfile, path()} | {dh, Der::binary()} | {dhfile, path()} | + {user_lookup_fun, {fun(), InitialUserState::term()}} | + {psk_identity, string()} | + {srp_identity, {string(), string()}} | + {ciphers, ciphers()} | {ssl_imp, ssl_imp()} | {reuse_sessions, boolean()} | + {reuse_session, fun()} | {hibernate_after, integer()|undefined} | + {next_protocols_advertised, list(binary())} | + {client_preferred_next_protocols, binary(), client | server, list(binary())}. -type verify_type() :: verify_none | verify_peer. -type path() :: string(). --type ciphers() :: [erl_cipher_suite()] | +-type ciphers() :: [ssl_cipher:erl_cipher_suite()] | string(). % (according to old API) -type ssl_imp() :: new | old. diff --git a/lib/ssl/src/ssl_certificate.erl b/lib/ssl/src/ssl_certificate.erl index b186a1015a..9c0ed181fe 100644 --- a/lib/ssl/src/ssl_certificate.erl +++ b/lib/ssl/src/ssl_certificate.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2013. All Rights Reserved. +%% Copyright Ericsson AB 2007-2014 All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -30,7 +30,7 @@ -include("ssl_internal.hrl"). -include_lib("public_key/include/public_key.hrl"). --export([trusted_cert_and_path/3, +-export([trusted_cert_and_path/4, certificate_chain/3, file_to_certificats/2, validate_extension/3, @@ -46,14 +46,14 @@ %%==================================================================== %%-------------------------------------------------------------------- --spec trusted_cert_and_path([der_cert()], db_handle(), certdb_ref()) -> +-spec trusted_cert_and_path([der_cert()], db_handle(), certdb_ref(), fun()) -> {der_cert() | unknown_ca, [der_cert()]}. %% %% Description: Extracts the root cert (if not presents tries to %% look it up, if not found {bad_cert, unknown_ca} will be added verification %% errors. Returns {RootCert, Path, VerifyErrors} %%-------------------------------------------------------------------- -trusted_cert_and_path(CertChain, CertDbHandle, CertDbRef) -> +trusted_cert_and_path(CertChain, CertDbHandle, CertDbRef, PartialChainHandler) -> Path = [Cert | _] = lists:reverse(CertChain), OtpCert = public_key:pkix_decode_cert(Cert, otp), SignedAndIssuerID = @@ -62,32 +62,23 @@ trusted_cert_and_path(CertChain, CertDbHandle, CertDbRef) -> {ok, IssuerId} = public_key:pkix_issuer_id(OtpCert, self), {self, IssuerId}; false -> - case public_key:pkix_issuer_id(OtpCert, other) of - {ok, IssuerId} -> - {other, IssuerId}; - {error, issuer_not_found} -> - case find_issuer(OtpCert, CertDbHandle) of - {ok, IssuerId} -> - {other, IssuerId}; - Other -> - Other - end - end + other_issuer(OtpCert, CertDbHandle) end, case SignedAndIssuerID of {error, issuer_not_found} -> %% The root CA was not sent and can not be found. - {unknown_ca, Path}; + handle_incomplete_chain(Path, PartialChainHandler); {self, _} when length(Path) == 1 -> {selfsigned_peer, Path}; {_ ,{SerialNr, Issuer}} -> case ssl_manager:lookup_trusted_cert(CertDbHandle, CertDbRef, SerialNr, Issuer) of - {ok, {BinCert,_}} -> - {BinCert, Path}; + {ok, Trusted} -> + %% Trusted must be selfsigned or it is an incomplete chain + handle_path(Trusted, Path, PartialChainHandler); _ -> %% Root CA could not be verified - {unknown_ca, Path} + handle_incomplete_chain(Path, PartialChainHandler) end end. @@ -222,23 +213,27 @@ certificate_chain(CertDbHandle, CertsDbRef, Chain, SerialNr, Issuer, _SelfSigned _ -> %% The trusted cert may be obmitted from the chain as the %% counter part needs to have it anyway to be able to - %% verify it. This will be the normal case for servers - %% that does not verify the clients and hence have not - %% specified the cacertfile. + %% verify it. {ok, lists:reverse(Chain)} end. find_issuer(OtpCert, CertDbHandle) -> - IsIssuerFun = fun({_Key, {_Der, #'OTPCertificate'{} = ErlCertCandidate}}, Acc) -> - case public_key:pkix_is_issuer(OtpCert, ErlCertCandidate) of - true -> - throw(public_key:pkix_issuer_id(ErlCertCandidate, self)); - false -> - Acc - end; - (_, Acc) -> - Acc - end, + IsIssuerFun = + fun({_Key, {_Der, #'OTPCertificate'{} = ErlCertCandidate}}, Acc) -> + case public_key:pkix_is_issuer(OtpCert, ErlCertCandidate) of + true -> + case verify_cert_signer(OtpCert, ErlCertCandidate#'OTPCertificate'.tbsCertificate) of + true -> + throw(public_key:pkix_issuer_id(ErlCertCandidate, self)); + false -> + Acc + end; + false -> + Acc + end; + (_, Acc) -> + Acc + end, try ssl_pkix_db:foldl(IsIssuerFun, issuer_not_found, CertDbHandle) of issuer_not_found -> @@ -254,3 +249,57 @@ is_valid_extkey_usage(KeyUse, client) -> is_valid_extkey_usage(KeyUse, server) -> %% Server wants to verify client is_valid_key_usage(KeyUse, ?'id-kp-clientAuth'). + +verify_cert_signer(OtpCert, SignerTBSCert) -> + PublicKey = public_key(SignerTBSCert#'OTPTBSCertificate'.subjectPublicKeyInfo), + public_key:pkix_verify(public_key:pkix_encode('OTPCertificate', OtpCert, otp), PublicKey). + +public_key(#'OTPSubjectPublicKeyInfo'{algorithm = #'PublicKeyAlgorithm'{algorithm = ?'id-ecPublicKey', + parameters = Params}, + subjectPublicKey = Point}) -> + {Point, Params}; +public_key(#'OTPSubjectPublicKeyInfo'{algorithm = #'PublicKeyAlgorithm'{algorithm = ?'rsaEncryption'}, + subjectPublicKey = Key}) -> + Key; +public_key(#'OTPSubjectPublicKeyInfo'{algorithm = #'PublicKeyAlgorithm'{algorithm = ?'id-dsa', + parameters = {params, Params}}, + subjectPublicKey = Key}) -> + {Key, Params}. + +other_issuer(OtpCert, CertDbHandle) -> + case public_key:pkix_issuer_id(OtpCert, other) of + {ok, IssuerId} -> + {other, IssuerId}; + {error, issuer_not_found} -> + case find_issuer(OtpCert, CertDbHandle) of + {ok, IssuerId} -> + {other, IssuerId}; + Other -> + Other + end + end. + +handle_path({BinCert, OTPCert}, Path, PartialChainHandler) -> + case public_key:pkix_is_self_signed(OTPCert) of + true -> + {BinCert, Path}; + false -> + handle_incomplete_chain(Path, PartialChainHandler) + end. + +handle_incomplete_chain(Chain, Fun) -> + case catch Fun(Chain) of + {trusted_ca, DerCert} -> + new_trusteded_chain(DerCert, Chain); + unknown_ca = Error -> + {Error, Chain}; + _ -> + {unknown_ca, Chain} + end. + +new_trusteded_chain(DerCert, [DerCert | Chain]) -> + {DerCert, Chain}; +new_trusteded_chain(DerCert, [_ | Rest]) -> + new_trusteded_chain(DerCert, Rest); +new_trusteded_chain(_, []) -> + unknown_ca. diff --git a/lib/ssl/src/ssl_cipher.erl b/lib/ssl/src/ssl_cipher.erl index b2077c662a..72467ea2a0 100644 --- a/lib/ssl/src/ssl_cipher.erl +++ b/lib/ssl/src/ssl_cipher.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2013. All Rights Reserved. +%% Copyright Ericsson AB 2007-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -34,10 +34,26 @@ -export([security_parameters/2, security_parameters/3, suite_definition/1, decipher/5, cipher/5, - suite/1, suites/1, ec_keyed_suites/0, anonymous_suites/0, psk_suites/1, srp_suites/0, + suite/1, suites/1, all_suites/1, + ec_keyed_suites/0, anonymous_suites/0, psk_suites/1, srp_suites/0, openssl_suite/1, openssl_suite_name/1, filter/2, filter_suites/1, hash_algorithm/1, sign_algorithm/1, is_acceptable_hash/2]). +-export_type([cipher_suite/0, + erl_cipher_suite/0, openssl_cipher_suite/0, + key_algo/0]). + +-type cipher() :: null |rc4_128 | idea_cbc | des40_cbc | des_cbc | '3des_ede_cbc' + | aes_128_cbc | aes_256_cbc. +-type hash() :: null | sha | md5 | sha224 | sha256 | sha384 | sha512. +-type key_algo() :: null | rsa | dhe_rsa | dhe_dss | ecdhe_ecdsa| ecdh_ecdsa | ecdh_rsa| srp_rsa| srp_dss | psk | dhe_psk | rsa_psk | dh_anon | ecdh_anon | srp_anon. +-type erl_cipher_suite() :: {key_algo(), cipher(), hash()}. +-type int_cipher_suite() :: {key_algo(), cipher(), hash(), hash() | default_prf}. +-type cipher_suite() :: binary(). +-type cipher_enum() :: integer(). +-type openssl_cipher_suite() :: string(). + + -compile(inline). %%-------------------------------------------------------------------- @@ -51,7 +67,7 @@ security_parameters(?TLS_NULL_WITH_NULL_NULL = CipherSuite, SecParams) -> security_parameters(undefined, CipherSuite, SecParams). %%-------------------------------------------------------------------- --spec security_parameters(tls_version() | undefined, cipher_suite(), #security_parameters{}) -> +-spec security_parameters(ssl_record:ssl_version() | undefined, cipher_suite(), #security_parameters{}) -> #security_parameters{}. %% %% Description: Returns a security parameters record where the @@ -72,7 +88,7 @@ security_parameters(Version, CipherSuite, SecParams) -> hash_size = hash_size(Hash)}. %%-------------------------------------------------------------------- --spec cipher(cipher_enum(), #cipher_state{}, binary(), iolist(), tls_version()) -> +-spec cipher(cipher_enum(), #cipher_state{}, binary(), iodata(), ssl_record:ssl_version()) -> {binary(), #cipher_state{}}. %% %% Description: Encrypts the data and the MAC using chipher described @@ -127,7 +143,7 @@ block_cipher(Fun, BlockSz, #cipher_state{key=Key, iv=IV} = CS0, {T, CS0#cipher_state{iv=NextIV}}. %%-------------------------------------------------------------------- --spec decipher(cipher_enum(), integer(), #cipher_state{}, binary(), tls_version()) -> +-spec decipher(cipher_enum(), integer(), #cipher_state{}, binary(), ssl_record:ssl_version()) -> {binary(), binary(), #cipher_state{}} | #alert{}. %% %% Description: Decrypts the data and the MAC using cipher described @@ -200,7 +216,7 @@ block_decipher(Fun, #cipher_state{key=Key, iv=IV} = CipherState0, ?ALERT_REC(?FATAL, ?BAD_RECORD_MAC) end. %%-------------------------------------------------------------------- --spec suites(tls_version()) -> [cipher_suite()]. +-spec suites(ssl_record:ssl_version()) -> [cipher_suite()]. %% %% Description: Returns a list of supported cipher suites. %%-------------------------------------------------------------------- @@ -209,6 +225,11 @@ suites({3, 0}) -> suites({3, N}) -> tls_v1:suites(N). +all_suites(Version) -> + suites(Version) + ++ ssl_cipher:anonymous_suites() + ++ ssl_cipher:psk_suites(Version) + ++ ssl_cipher:srp_suites(). %%-------------------------------------------------------------------- -spec anonymous_suites() -> [cipher_suite()]. %% @@ -229,7 +250,7 @@ anonymous_suites() -> ?TLS_ECDH_anon_WITH_AES_256_CBC_SHA]. %%-------------------------------------------------------------------- --spec psk_suites(tls_version() | integer()) -> [cipher_suite()]. +-spec psk_suites(ssl_record:ssl_version() | integer()) -> [cipher_suite()]. %% %% Description: Returns a list of the PSK cipher suites, only supported %% if explicitly set by user. @@ -998,7 +1019,8 @@ openssl_suite_name(Cipher) -> %%-------------------------------------------------------------------- -spec filter(undefined | binary(), [cipher_suite()]) -> [cipher_suite()]. %% -%% Description: . +%% Description: Select the cipher suites that can be used together with the +%% supplied certificate. (Server side functionality) %%------------------------------------------------------------------- filter(undefined, Ciphers) -> Ciphers; @@ -1032,7 +1054,7 @@ filter(DerCert, Ciphers) -> %%-------------------------------------------------------------------- -spec filter_suites([cipher_suite()]) -> [cipher_suite()]. %% -%% Description: filter suites for algorithms +%% Description: Filter suites for algorithms supported by crypto. %%------------------------------------------------------------------- filter_suites(Suites = [{_,_,_}|_]) -> Algos = crypto:supports(), diff --git a/lib/ssl/src/ssl_cipher.hrl b/lib/ssl/src/ssl_cipher.hrl index 62a5269def..3ce9c19aa9 100644 --- a/lib/ssl/src/ssl_cipher.hrl +++ b/lib/ssl/src/ssl_cipher.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2013. All Rights Reserved. +%% Copyright Ericsson AB 2007-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -26,16 +26,6 @@ -ifndef(ssl_cipher). -define(ssl_cipher, true). --type cipher() :: null |rc4_128 | idea_cbc | des40_cbc | des_cbc | '3des_ede_cbc' - | aes_128_cbc | aes_256_cbc. --type hash() :: null | sha | md5 | sha224 | sha256 | sha384 | sha512. --type key_algo() :: null | rsa | dhe_rsa | dhe_dss | ecdhe_ecdsa| ecdh_ecdsa | ecdh_rsa| srp_rsa| srp_dss | psk | dhe_psk | rsa_psk | dh_anon | ecdh_anon | srp_anon. --type erl_cipher_suite() :: {key_algo(), cipher(), hash()}. --type int_cipher_suite() :: {key_algo(), cipher(), hash(), hash() | default_prf}. --type cipher_suite() :: binary(). --type cipher_enum() :: integer(). --type openssl_cipher_suite() :: string(). - %%% SSL cipher protocol %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -define(CHANGE_CIPHER_SPEC_PROTO, 1). % _PROTO to not clash with % SSL record protocol diff --git a/lib/ssl/src/ssl_connection.erl b/lib/ssl/src/ssl_connection.erl index e283e6079e..8ff9913cee 100644 --- a/lib/ssl/src/ssl_connection.erl +++ b/lib/ssl/src/ssl_connection.erl @@ -36,8 +36,8 @@ -include_lib("public_key/include/public_key.hrl"). %% Setup --export([connect/8, ssl_accept/7, handshake/2, - socket_control/4]). +-export([connect/8, ssl_accept/7, handshake/2, handshake/3, + socket_control/4, socket_control/5]). %% User Events -export([send/2, recv/3, close/1, shutdown/2, @@ -50,14 +50,18 @@ %% SSL FSM state functions -export([hello/3, abbreviated/3, certify/3, cipher/3, connection/3]). %% SSL all state functions --export([handle_sync_event/4, handle_info/3, terminate/3]). +-export([handle_sync_event/4, handle_info/3, terminate/3, format_status/2]). + %%==================================================================== %% Internal application API %%==================================================================== %%-------------------------------------------------------------------- -spec connect(tls_connection | dtls_connection, - host(), inet:port_number(), port(), {#ssl_options{}, #socket_options{}}, + host(), inet:port_number(), port(), + {#ssl_options{}, #socket_options{}, + %% Tracker only needed on server side + undefined}, pid(), tuple(), timeout()) -> {ok, #sslsocket{}} | {error, reason()}. %% @@ -72,9 +76,10 @@ connect(Connection, Host, Port, Socket, Options, User, CbInfo, Timeout) -> end. %%-------------------------------------------------------------------- -spec ssl_accept(tls_connection | dtls_connection, - inet:port_number(), port(), {#ssl_options{}, #socket_options{}}, - pid(), tuple(), timeout()) -> - {ok, #sslsocket{}} | {error, reason()}. + inet:port_number(), port(), + {#ssl_options{}, #socket_options{}, undefined | pid()}, + pid(), tuple(), timeout()) -> + {ok, #sslsocket{}} | {error, reason()}. %% %% Description: Performs accept on an ssl listen socket. e.i. performs %% ssl handshake. @@ -99,6 +104,21 @@ handshake(#sslsocket{pid = Pid}, Timeout) -> Error -> Error end. + +%%-------------------------------------------------------------------- +-spec handshake(#sslsocket{}, {#ssl_options{},#socket_options{}}, + timeout()) -> ok | {error, reason()}. +%% +%% Description: Starts ssl handshake with some new options +%%-------------------------------------------------------------------- +handshake(#sslsocket{pid = Pid}, SslOptions, Timeout) -> + case sync_send_all_state_event(Pid, {start, SslOptions, Timeout}) of + connected -> + ok; + Error -> + Error + end. + %-------------------------------------------------------------------- -spec socket_control(tls_connection | dtls_connection, port(), pid(), atom()) -> {ok, #sslsocket{}} | {error, reason()}. @@ -106,9 +126,16 @@ handshake(#sslsocket{pid = Pid}, Timeout) -> %% Description: Set the ssl process to own the accept socket %%-------------------------------------------------------------------- socket_control(Connection, Socket, Pid, Transport) -> + socket_control(Connection, Socket, Pid, Transport, undefined). + +%-------------------------------------------------------------------- +-spec socket_control(tls_connection | dtls_connection, port(), pid(), atom(), pid()| undefined) -> + {ok, #sslsocket{}} | {error, reason()}. +%%-------------------------------------------------------------------- +socket_control(Connection, Socket, Pid, Transport, ListenTracker) -> case Transport:controlling_process(Socket, Pid) of ok -> - {ok, ssl_socket:socket(Pid, Transport, Socket, Connection)}; + {ok, ssl_socket:socket(Pid, Transport, Socket, Connection, ListenTracker)}; {error, Reason} -> {error, Reason} end. @@ -275,12 +302,11 @@ hello(#hello_request{}, #state{role = client} = State0, Connection) -> {Record, State} = Connection:next_record(State0), Connection:next_state(hello, hello, Record, State); -hello({common_client_hello, Type, ServerHelloExt, HashSign}, - #state{session = #session{cipher_suite = CipherSuite}, - negotiated_version = Version} = State, Connection) -> - {KeyAlg, _, _, _} = ssl_cipher:suite_definition(CipherSuite), - NegotiatedHashSign = negotiated_hashsign(HashSign, KeyAlg, Version), +hello({common_client_hello, Type, ServerHelloExt, NegotiatedHashSign}, + State, Connection) -> do_server_hello(Type, ServerHelloExt, + %% Note NegotiatedHashSign is only negotiated for real if + %% if TLS version is at least TLS-1.2 State#state{hashsign_algorithm = NegotiatedHashSign}, Connection); hello(timeout, State, _) -> @@ -301,6 +327,7 @@ abbreviated(#hello_request{}, State0, Connection) -> abbreviated(#finished{verify_data = Data} = Finished, #state{role = server, negotiated_version = Version, + expecting_finished = true, tls_handshake_history = Handshake, session = #session{master_secret = MasterSecret}, connection_states = ConnectionStates0} = @@ -313,7 +340,8 @@ abbreviated(#finished{verify_data = Data} = Finished, ssl_record:set_client_verify_data(current_both, Data, ConnectionStates0), Connection:next_state_connection(abbreviated, ack_connection( - State#state{connection_states = ConnectionStates})); + State#state{connection_states = ConnectionStates, + expecting_finished = false})); #alert{} = Alert -> Connection:handle_own_alert(Alert, Version, abbreviated, State) end; @@ -333,7 +361,7 @@ abbreviated(#finished{verify_data = Data} = Finished, finalize_handshake(State0#state{connection_states = ConnectionStates1}, abbreviated, Connection), Connection:next_state_connection(abbreviated, - ack_connection(State)); + ack_connection(State#state{expecting_finished = false})); #alert{} = Alert -> Connection:handle_own_alert(Alert, Version, abbreviated, State0) end; @@ -344,7 +372,7 @@ abbreviated(#next_protocol{selected_protocol = SelectedProtocol}, #state{role = server, expecting_next_protocol_negotiation = true} = State0, Connection) -> {Record, State} = Connection:next_record(State0#state{next_protocol = SelectedProtocol}), - Connection:next_state(abbreviated, abbreviated, Record, State); + Connection:next_state(abbreviated, abbreviated, Record, State#state{expecting_next_protocol_negotiation = false}); abbreviated(timeout, State, _) -> {next_state, abbreviated, State, hibernate }; @@ -386,7 +414,9 @@ certify(#certificate{} = Cert, ssl_options = Opts} = State, Connection) -> case ssl_handshake:certify(Cert, CertDbHandle, CertDbRef, Opts#ssl_options.depth, Opts#ssl_options.verify, - Opts#ssl_options.verify_fun, Role) of + Opts#ssl_options.verify_fun, + Opts#ssl_options.partial_chain, + Role) of {PeerCert, PublicKeyInfo} -> handle_peer_cert(Role, PeerCert, PublicKeyInfo, State#state{client_certificate_requested = false}, Connection); @@ -417,7 +447,8 @@ certify(#server_key_exchange{exchange_keys = Keys}, calculate_secret(Params#server_key_params.params, State#state{hashsign_algorithm = HashSign}, Connection); false -> - ?ALERT_REC(?FATAL, ?DECRYPT_ERROR) + Connection:handle_own_alert(?ALERT_REC(?FATAL, ?DECRYPT_ERROR), + Version, certify, State) end end; @@ -426,8 +457,9 @@ certify(#server_key_exchange{} = Msg, Connection:handle_unexpected_message(Msg, certify_server_keyexchange, State); certify(#certificate_request{hashsign_algorithms = HashSigns}, - #state{session = #session{own_certificate = Cert}} = State0, Connection) -> - HashSign = ssl_handshake:select_hashsign(HashSigns, Cert), + #state{session = #session{own_certificate = Cert}, + negotiated_version = Version} = State0, Connection) -> + HashSign = ssl_handshake:select_hashsign(HashSigns, Cert, Version), {Record, State} = Connection:next_record(State0#state{client_certificate_requested = true}), Connection:next_state(certify, certify, Record, State#state{cert_hashsign_algorithm = HashSign}); @@ -544,7 +576,7 @@ cipher(#certificate_verify{signature = Signature, hashsign_algorithm = CertHashS tls_handshake_history = Handshake } = State0, Connection) -> - HashSign = ssl_handshake:select_cert_hashsign(CertHashSign, Algo, Version), + HashSign = ssl_handshake:select_hashsign_algs(CertHashSign, Algo, Version), case ssl_handshake:certificate_verify(Signature, PublicKeyInfo, Version, HashSign, MasterSecret, Handshake) of valid -> @@ -566,6 +598,7 @@ cipher(#finished{verify_data = Data} = Finished, host = Host, port = Port, role = Role, + expecting_finished = true, session = #session{master_secret = MasterSecret} = Session0, connection_states = ConnectionStates0, @@ -576,7 +609,7 @@ cipher(#finished{verify_data = Data} = Finished, MasterSecret, Handshake0) of verified -> Session = register_session(Role, Host, Port, Session0), - cipher_role(Role, Data, Session, State, Connection); + cipher_role(Role, Data, Session, State#state{expecting_finished = false}, Connection); #alert{} = Alert -> Connection:handle_own_alert(Alert, Version, cipher, State) end; @@ -584,9 +617,10 @@ cipher(#finished{verify_data = Data} = Finished, %% only allowed to send next_protocol message after change cipher spec %% & before finished message and it is not allowed during renegotiation cipher(#next_protocol{selected_protocol = SelectedProtocol}, - #state{role = server, expecting_next_protocol_negotiation = true} = State0, Connection) -> + #state{role = server, expecting_next_protocol_negotiation = true, + expecting_finished = true} = State0, Connection) -> {Record, State} = Connection:next_record(State0#state{next_protocol = SelectedProtocol}), - Connection:next_state(cipher, cipher, Record, State); + Connection:next_state(cipher, cipher, Record, State#state{expecting_next_protocol_negotiation = false}); cipher(timeout, State, _) -> {next_state, cipher, State, hibernate}; @@ -626,12 +660,27 @@ handle_sync_event({application_data, Data}, From, StateName, State#state{send_queue = queue:in({From, Data}, Queue)}, get_timeout(State)}; -handle_sync_event({start, Timeout}, StartFrom, hello, #state{protocol_cb = Connection} = State) -> - Timer = start_or_recv_cancel_timer(Timeout, StartFrom), - Connection:hello(start, State#state{start_or_recv_from = StartFrom, - timer = Timer}); +handle_sync_event({start, Timeout}, StartFrom, hello, #state{role = Role, + protocol_cb = Connection, + ssl_options = SSLOpts} = State0) -> + try + State = ssl_config(SSLOpts, Role, State0), + Timer = start_or_recv_cancel_timer(Timeout, StartFrom), + Connection:hello(start, State#state{start_or_recv_from = StartFrom, + timer = Timer}) + catch throw:Error -> + {stop, normal, {error, Error}, State0} + end; -%% The two clauses below could happen if a server upgrades a socket in +handle_sync_event({start, {Opts, EmOpts}, Timeout}, From, StateName, State) -> + try + handle_sync_event({start, Timeout}, From, StateName, State#state{socket_options = EmOpts, + ssl_options = Opts}) + catch throw:Error -> + {stop, normal, {error, Error}, State} + end; + +%% These two clauses below could happen if a server upgrades a socket in %% active mode. Note that in this case we are lucky that %% controlling_process has been evalueated before receiving handshake %% messages from client. The server should put the socket in passive @@ -641,13 +690,16 @@ handle_sync_event({start, Timeout}, StartFrom, hello, #state{protocol_cb = Conne %% they upgrade an active socket. handle_sync_event({start,_}, _, connection, State) -> {reply, connected, connection, State, get_timeout(State)}; -handle_sync_event({start,_}, _From, error, {Error, State = #state{}}) -> - {stop, {shutdown, Error}, {error, Error}, State}; -handle_sync_event({start, Timeout}, StartFrom, StateName, State) -> - Timer = start_or_recv_cancel_timer(Timeout, StartFrom), - {next_state, StateName, State#state{start_or_recv_from = StartFrom, - timer = Timer}, get_timeout(State)}; +handle_sync_event({start, Timeout}, StartFrom, StateName, #state{role = Role, ssl_options = SslOpts} = State0) -> + try + State = ssl_config(SslOpts, Role, State0), + Timer = start_or_recv_cancel_timer(Timeout, StartFrom), + {next_state, StateName, State#state{start_or_recv_from = StartFrom, + timer = Timer}, get_timeout(State)} + catch throw:Error -> + {stop, normal, {error, Error}, State0} + end; handle_sync_event(close, _, StateName, #state{protocol_cb = Connection} = State) -> %% Run terminate before returning @@ -655,7 +707,6 @@ handle_sync_event(close, _, StateName, #state{protocol_cb = Connection} = State) %% as intended. (catch Connection:terminate(user_close, StateName, State)), {stop, normal, ok, State#state{terminated = true}}; - handle_sync_event({shutdown, How0}, _, StateName, #state{transport_cb = Transport, negotiated_version = Version, @@ -677,13 +728,14 @@ handle_sync_event({shutdown, How0}, _, StateName, Error -> {stop, normal, Error, State} end; - +handle_sync_event({recv, _N, _Timeout}, _RecvFrom, StateName, + #state{socket_options = #socket_options{active = Active}} = State) when Active =/= false -> + {reply, {error, einval}, StateName, State, get_timeout(State)}; handle_sync_event({recv, N, Timeout}, RecvFrom, connection = StateName, #state{protocol_cb = Connection} = State0) -> Timer = start_or_recv_cancel_timer(Timeout, RecvFrom), Connection:passive_receive(State0#state{bytes_to_read = N, start_or_recv_from = RecvFrom, timer = Timer}, StateName); - %% Doing renegotiate wait with handling request until renegotiate is %% finished. Will be handled by next_state_is_connection/2. handle_sync_event({recv, N, Timeout}, RecvFrom, StateName, State) -> @@ -691,26 +743,22 @@ handle_sync_event({recv, N, Timeout}, RecvFrom, StateName, State) -> {next_state, StateName, State#state{bytes_to_read = N, start_or_recv_from = RecvFrom, timer = Timer}, get_timeout(State)}; - handle_sync_event({new_user, User}, _From, StateName, State =#state{user_application = {OldMon, _}}) -> NewMon = erlang:monitor(process, User), erlang:demonitor(OldMon, [flush]), {reply, ok, StateName, State#state{user_application = {NewMon,User}}, get_timeout(State)}; - handle_sync_event({get_opts, OptTags}, _From, StateName, #state{socket = Socket, transport_cb = Transport, socket_options = SockOpts} = State) -> OptsReply = get_socket_opts(Transport, Socket, OptTags, SockOpts, []), {reply, OptsReply, StateName, State, get_timeout(State)}; - handle_sync_event(negotiated_next_protocol, _From, StateName, #state{next_protocol = undefined} = State) -> {reply, {error, next_protocol_not_negotiated}, StateName, State, get_timeout(State)}; handle_sync_event(negotiated_next_protocol, _From, StateName, #state{next_protocol = NextProtocol} = State) -> {reply, {ok, NextProtocol}, StateName, State, get_timeout(State)}; - handle_sync_event({set_opts, Opts0}, _From, StateName0, #state{socket_options = Opts1, protocol_cb = Connection, @@ -749,13 +797,10 @@ handle_sync_event({set_opts, Opts0}, _From, StateName0, end end end; - handle_sync_event(renegotiate, From, connection, #state{protocol_cb = Connection} = State) -> Connection:renegotiate(State#state{renegotiation = {true, From}}); - handle_sync_event(renegotiate, _, StateName, State) -> {reply, {error, already_renegotiating}, StateName, State, get_timeout(State)}; - handle_sync_event({prf, Secret, Label, Seed, WantedLength}, _, StateName, #state{connection_states = ConnectionStates, negotiated_version = Version} = State) -> @@ -781,7 +826,6 @@ handle_sync_event({prf, Secret, Label, Seed, WantedLength}, _, StateName, error:Reason -> {error, Reason} end, {reply, Reply, StateName, State, get_timeout(State)}; - handle_sync_event(info, _, StateName, #state{negotiated_version = Version, session = #session{cipher_suite = Suite}} = State) -> @@ -789,14 +833,12 @@ handle_sync_event(info, _, StateName, AtomVersion = tls_record:protocol_version(Version), {reply, {ok, {AtomVersion, ssl:suite_definition(Suite)}}, StateName, State, get_timeout(State)}; - handle_sync_event(session_info, _, StateName, #state{session = #session{session_id = Id, cipher_suite = Suite}} = State) -> {reply, [{session_id, Id}, {cipher_suite, ssl:suite_definition(Suite)}], StateName, State, get_timeout(State)}; - handle_sync_event(peer_certificate, _, StateName, #state{session = #session{peer_certificate = Cert}} = State) -> @@ -806,8 +848,9 @@ handle_info({ErrorTag, Socket, econnaborted}, StateName, #state{socket = Socket, transport_cb = Transport, start_or_recv_from = StartFrom, role = Role, protocol_cb = Connection, - error_tag = ErrorTag} = State) when StateName =/= connection -> - Connection:alert_user(Transport, Socket, StartFrom, ?ALERT_REC(?FATAL, ?CLOSE_NOTIFY), Role), + error_tag = ErrorTag, + tracker = Tracker} = State) when StateName =/= connection -> + Connection:alert_user(Transport, Tracker,Socket, StartFrom, ?ALERT_REC(?FATAL, ?CLOSE_NOTIFY), Role), {stop, normal, State}; handle_info({ErrorTag, Socket, Reason}, StateName, #state{socket = Socket, @@ -857,7 +900,6 @@ terminate(_, _, #state{terminated = true}) -> %% we want to guarantee that Transport:close has been called %% when ssl:close/1 returns. ok; - terminate({shutdown, transport_closed}, StateName, #state{send_queue = SendQueue, renegotiation = Renegotiate} = State) -> handle_unrecv_data(StateName, State), @@ -870,7 +912,6 @@ terminate({shutdown, own_alert}, _StateName, #state{send_queue = SendQueue, handle_trusted_certs_db(State), notify_senders(SendQueue), notify_renegotiater(Renegotiate); - terminate(Reason, connection, #state{negotiated_version = Version, protocol_cb = Connection, connection_states = ConnectionStates, @@ -887,7 +928,6 @@ terminate(Reason, connection, #state{negotiated_version = Version, _ -> ok end; - terminate(_Reason, _StateName, #state{transport_cb = Transport, socket = Socket, send_queue = SendQueue, renegotiation = Renegotiate} = State) -> @@ -896,9 +936,50 @@ terminate(_Reason, _StateName, #state{transport_cb = Transport, notify_renegotiater(Renegotiate), Transport:close(Socket). +format_status(normal, [_, State]) -> + [{data, [{"StateData", State}]}]; +format_status(terminate, [_, State]) -> + SslOptions = (State#state.ssl_options), + NewOptions = SslOptions#ssl_options{password = "***", + cert = "***", + cacerts = "***", + key = "***", + dh = "***", + psk_identity = "***", + srp_identity = "***"}, + [{data, [{"StateData", State#state{connection_states = "***", + protocol_buffers = "***", + user_data_buffer = "***", + tls_handshake_history = "***", + session = "***", + private_key = "***", + diffie_hellman_params = "***", + diffie_hellman_keys = "***", + srp_params = "***", + srp_keys = "***", + premaster_secret = "***", + ssl_options = NewOptions + }}]}]. %%-------------------------------------------------------------------- %%% Internal functions %%-------------------------------------------------------------------- +ssl_config(Opts, Role, State) -> + {ok, Ref, CertDbHandle, FileRefHandle, CacheHandle, OwnCert, Key, DHParams} = + ssl_config:init(Opts, Role), + Handshake = ssl_handshake:init_handshake_history(), + TimeStamp = calendar:datetime_to_gregorian_seconds({date(), time()}), + Session = State#state.session, + State#state{tls_handshake_history = Handshake, + session = Session#session{own_certificate = OwnCert, + time_stamp = TimeStamp}, + file_ref_db = FileRefHandle, + cert_db_ref = Ref, + cert_db = CertDbHandle, + session_cache = CacheHandle, + private_key = Key, + diffie_hellman_params = DHParams, + ssl_options = Opts}. + do_server_hello(Type, #hello_extensions{next_protocol_negotiation = NextProtocols} = ServerHelloExt, #state{negotiated_version = Version, @@ -964,9 +1045,6 @@ server_hello_done(State, Connection) -> HelloDone = ssl_handshake:server_hello_done(), Connection:send_handshake(HelloDone, State). - - - handle_peer_cert(Role, PeerCert, PublicKeyInfo, #state{session = #session{cipher_suite = CipherSuite} = Session} = State0, Connection) -> @@ -1540,60 +1618,6 @@ cipher_role(server, Data, Session, #state{connection_states = ConnectionStates0 session = Session}, cipher, Connection), Connection:next_state_connection(cipher, ack_connection(State#state{session = Session})). -negotiated_hashsign(undefined, Algo, Version) -> - default_hashsign(Version, Algo); -negotiated_hashsign(HashSign = {_, _}, _, _) -> - HashSign. - -%% RFC 5246, Sect. 7.4.1.4.1. Signature Algorithms -%% If the client does not send the signature_algorithms extension, the -%% server MUST do the following: -%% -%% - If the negotiated key exchange algorithm is one of (RSA, DHE_RSA, -%% DH_RSA, RSA_PSK, ECDH_RSA, ECDHE_RSA), behave as if client had -%% sent the value {sha1,rsa}. -%% -%% - If the negotiated key exchange algorithm is one of (DHE_DSS, -%% DH_DSS), behave as if the client had sent the value {sha1,dsa}. -%% -%% - If the negotiated key exchange algorithm is one of (ECDH_ECDSA, -%% ECDHE_ECDSA), behave as if the client had sent value {sha1,ecdsa}. - -default_hashsign(_Version = {Major, Minor}, KeyExchange) - when Major >= 3 andalso Minor >= 3 andalso - (KeyExchange == rsa orelse - KeyExchange == dhe_rsa orelse - KeyExchange == dh_rsa orelse - KeyExchange == ecdhe_rsa orelse - KeyExchange == ecdh_rsa orelse - KeyExchange == srp_rsa) -> - {sha, rsa}; -default_hashsign(_Version, KeyExchange) - when KeyExchange == rsa; - KeyExchange == dhe_rsa; - KeyExchange == dh_rsa; - KeyExchange == ecdhe_rsa; - KeyExchange == ecdh_rsa; - KeyExchange == srp_rsa -> - {md5sha, rsa}; -default_hashsign(_Version, KeyExchange) - when KeyExchange == ecdhe_ecdsa; - KeyExchange == ecdh_ecdsa -> - {sha, ecdsa}; -default_hashsign(_Version, KeyExchange) - when KeyExchange == dhe_dss; - KeyExchange == dh_dss; - KeyExchange == srp_dss -> - {sha, dsa}; -default_hashsign(_Version, KeyExchange) - when KeyExchange == dh_anon; - KeyExchange == ecdh_anon; - KeyExchange == psk; - KeyExchange == dhe_psk; - KeyExchange == rsa_psk; - KeyExchange == srp_anon -> - {null, anon}. - select_curve(#state{client_ecc = {[Curve|_], _}}) -> {namedCurve, Curve}; select_curve(_) -> @@ -1854,3 +1878,15 @@ make_premaster_secret({MajVer, MinVer}, rsa) -> <<?BYTE(MajVer), ?BYTE(MinVer), Rand/binary>>; make_premaster_secret(_, _) -> undefined. + +negotiated_hashsign(undefined, Alg, Version) -> + %% Not negotiated choose default + case is_anonymous(Alg) of + true -> + {null, anon}; + false -> + ssl_handshake:select_hashsign_algs(Alg, Version) + end; +negotiated_hashsign(HashSign = {_, _}, _, _) -> + HashSign. + diff --git a/lib/ssl/src/ssl_connection.hrl b/lib/ssl/src/ssl_connection.hrl index 341a4217e4..c544a0591f 100644 --- a/lib/ssl/src/ssl_connection.hrl +++ b/lib/ssl/src/ssl_connection.hrl @@ -48,17 +48,17 @@ socket_options :: #socket_options{}, connection_states :: #connection_states{}, protocol_buffers :: term(), %% #protocol_buffers{} from tls_record.hrl or dtls_recor.hrl - tls_handshake_history ::tls_handshake_history(), + tls_handshake_history :: ssl_handshake:ssl_handshake_history(), cert_db :: reference(), session :: #session{}, session_cache :: db_handle(), session_cache_cb :: atom(), - negotiated_version :: tls_version(), + negotiated_version :: ssl_record:ssl_version(), client_certificate_requested = false :: boolean(), - key_algorithm :: key_algo(), + key_algorithm :: ssl_cipher:key_algo(), hashsign_algorithm = {undefined, undefined}, cert_hashsign_algorithm, - public_key_info ::public_key_info(), + public_key_info ::ssl_handshake:public_key_info(), private_key ::public_key:private_key(), diffie_hellman_params, % PKIX: #'DHParameter'{} relevant for server side diffie_hellman_keys, % {PublicKey, PrivateKey} @@ -77,8 +77,10 @@ terminated = false ::boolean(), allow_renegotiate = true ::boolean(), expecting_next_protocol_negotiation = false ::boolean(), + expecting_finished = false ::boolean(), next_protocol = undefined :: undefined | binary(), - client_ecc % {Curves, PointFmt} + client_ecc, % {Curves, PointFmt} + tracker :: pid() %% Tracker process for listen socket }). -define(DEFAULT_DIFFIE_HELLMAN_PARAMS, diff --git a/lib/ssl/src/ssl_dist_sup.erl b/lib/ssl/src/ssl_dist_sup.erl index 22614a2d34..58efeaf892 100644 --- a/lib/ssl/src/ssl_dist_sup.erl +++ b/lib/ssl/src/ssl_dist_sup.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2011-2013. All Rights Reserved. +%% Copyright Ericsson AB 2011-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -45,9 +45,11 @@ start_link() -> init([]) -> SessionCertManager = session_and_cert_manager_child_spec(), ConnetionManager = connection_manager_child_spec(), + ListenOptionsTracker = listen_options_tracker_child_spec(), ProxyServer = proxy_server_child_spec(), - {ok, {{one_for_all, 10, 3600}, [SessionCertManager, ConnetionManager, + {ok, {{one_for_all, 10, 3600}, [SessionCertManager, ConnetionManager, + ListenOptionsTracker, ProxyServer]}}. %%-------------------------------------------------------------------- @@ -68,7 +70,7 @@ connection_manager_child_spec() -> StartFunc = {tls_connection_sup, start_link_dist, []}, Restart = permanent, Shutdown = 4000, - Modules = [ssl_connection], + Modules = [tls_connection_sup], Type = supervisor, {Name, StartFunc, Restart, Shutdown, Type, Modules}. @@ -81,3 +83,11 @@ proxy_server_child_spec() -> Type = worker, {Name, StartFunc, Restart, Shutdown, Type, Modules}. +listen_options_tracker_child_spec() -> + Name = ssl_socket_dist, + StartFunc = {ssl_listen_tracker_sup, start_link_dist, []}, + Restart = permanent, + Shutdown = 4000, + Modules = [ssl_socket], + Type = supervisor, + {Name, StartFunc, Restart, Shutdown, Type, Modules}. diff --git a/lib/ssl/src/ssl_handshake.erl b/lib/ssl/src/ssl_handshake.erl index 74ca7ca699..22673e46e2 100644 --- a/lib/ssl/src/ssl_handshake.erl +++ b/lib/ssl/src/ssl_handshake.erl @@ -31,13 +31,25 @@ -include("ssl_srp.hrl"). -include_lib("public_key/include/public_key.hrl"). +-export_type([ssl_handshake/0, ssl_handshake_history/0, + public_key_info/0, oid/0]). + +-type oid() :: tuple(). +-type public_key_params() :: #'Dss-Parms'{} | {namedCurve, oid()} | #'ECParameters'{} | term(). +-type public_key_info() :: {oid(), #'RSAPublicKey'{} | integer() | #'ECPoint'{}, public_key_params()}. +-type ssl_handshake_history() :: {[binary()], [binary()]}. + +-type ssl_handshake() :: #server_hello{} | #server_hello_done{} | #certificate{} | #certificate_request{} | + #client_key_exchange{} | #finished{} | #certificate_verify{} | + #hello_request{} | #next_protocol{}. + %% Handshake messages -export([hello_request/0, server_hello/4, server_hello_done/0, certificate/4, certificate_request/4, key_exchange/3, finished/5, next_protocol/1]). %% Handle handshake messages --export([certify/7, client_certificate_verify/6, certificate_verify/6, verify_signature/5, +-export([certify/8, client_certificate_verify/6, certificate_verify/6, verify_signature/5, master_secret/5, server_key_exchange_hash/2, verify_connection/6, init_handshake_history/0, update_handshake_history/2, verify_server_key/5 ]). @@ -61,7 +73,8 @@ ]). %% MISC --export([select_version/3, prf/5, select_hashsign/2, select_cert_hashsign/3, +-export([select_version/3, prf/5, select_hashsign/3, + select_hashsign_algs/2, select_hashsign_algs/3, premaster_secret/2, premaster_secret/3, premaster_secret/4]). %%==================================================================== @@ -80,7 +93,7 @@ hello_request() -> #hello_request{}. %%-------------------------------------------------------------------- --spec server_hello(#session{}, tls_version(), #connection_states{}, +-spec server_hello(#session{}, ssl_record:ssl_version(), #connection_states{}, #hello_extensions{}) -> #server_hello{}. %% %% Description: Creates a server hello message. @@ -164,8 +177,8 @@ next_protocol(SelectedProtocol) -> %%-------------------------------------------------------------------- -spec client_certificate_verify(undefined | der_cert(), binary(), - tls_version(), term(), public_key:private_key(), - tls_handshake_history()) -> + ssl_record:ssl_version(), term(), public_key:private_key(), + ssl_handshake_history()) -> #certificate_verify{} | ignore | #alert{}. %% %% Description: Creates a certificate_verify message, called by the client. @@ -188,13 +201,13 @@ client_certificate_verify(OwnCert, MasterSecret, Version, end. %%-------------------------------------------------------------------- --spec certificate_request(erl_cipher_suite(), db_handle(), certdb_ref(), tls_version()) -> +-spec certificate_request(ssl_cipher:cipher_suite(), db_handle(), certdb_ref(), ssl_record:ssl_version()) -> #certificate_request{}. %% %% Description: Creates a certificate_request message, called by the server. %%-------------------------------------------------------------------- certificate_request(CipherSuite, CertDbHandle, CertDbRef, Version) -> - Types = certificate_types(CipherSuite), + Types = certificate_types(ssl_cipher:suite_definition(CipherSuite), Version), HashSigns = advertised_hash_signs(Version), Authorities = certificate_authorities(CertDbHandle, CertDbRef), #certificate_request{ @@ -203,7 +216,7 @@ certificate_request(CipherSuite, CertDbHandle, CertDbRef, Version) -> certificate_authorities = Authorities }. %%-------------------------------------------------------------------- --spec key_exchange(client | server, tls_version(), +-spec key_exchange(client | server, ssl_record:ssl_version(), {premaster_secret, binary(), public_key_info()} | {dh, binary()} | {dh, {binary(), binary()}, #'DHParameter'{}, {HashAlgo::atom(), SignAlgo::atom()}, @@ -304,7 +317,7 @@ key_exchange(server, Version, {srp, {PublicKey, _}, ClientRandom, ServerRandom, PrivateKey). %%-------------------------------------------------------------------- --spec finished(tls_version(), client | server, integer(), binary(), tls_handshake_history()) -> +-spec finished(ssl_record:ssl_version(), client | server, integer(), binary(), ssl_handshake_history()) -> #finished{}. %% %% Description: Creates a handshake finished message @@ -331,8 +344,8 @@ verify_server_key(#server_key_params{params_bin = EncParams, verify_signature(Version, Hash, HashSign, Signature, PubKeyInfo). %%-------------------------------------------------------------------- --spec certificate_verify(binary(), public_key_info(), tls_version(), term(), - binary(), tls_handshake_history()) -> valid | #alert{}. +-spec certificate_verify(binary(), public_key_info(), ssl_record:ssl_version(), term(), + binary(), ssl_handshake_history()) -> valid | #alert{}. %% %% Description: Checks that the certificate_verify message is valid. %%-------------------------------------------------------------------- @@ -346,7 +359,7 @@ certificate_verify(Signature, PublicKeyInfo, Version, ?ALERT_REC(?FATAL, ?BAD_CERTIFICATE) end. %%-------------------------------------------------------------------- --spec verify_signature(tls_version(), binary(), {term(), term()}, binary(), +-spec verify_signature(ssl_record:ssl_version(), binary(), {term(), term()}, binary(), public_key_info()) -> true | false. %% %% Description: Checks that a public_key signature is valid. @@ -370,13 +383,13 @@ verify_signature(_Version, Hash, {HashAlgo, ecdsa}, Signature, %%-------------------------------------------------------------------- -spec certify(#certificate{}, db_handle(), certdb_ref(), integer() | nolimit, - verify_peer | verify_none, {fun(), term}, + verify_peer | verify_none, {fun(), term}, fun(), client | server) -> {der_cert(), public_key_info()} | #alert{}. %% %% Description: Handles a certificate handshake message %%-------------------------------------------------------------------- certify(#certificate{asn1_certificates = ASN1Certs}, CertDbHandle, CertDbRef, - MaxPathLen, _Verify, VerifyFunAndState, Role) -> + MaxPathLen, _Verify, VerifyFunAndState, PartialChain, Role) -> [PeerCert | _] = ASN1Certs, ValidationFunAndState = @@ -408,7 +421,7 @@ certify(#certificate{asn1_certificates = ASN1Certs}, CertDbHandle, CertDbRef, try {TrustedErlCert, CertPath} = - ssl_certificate:trusted_cert_and_path(ASN1Certs, CertDbHandle, CertDbRef), + ssl_certificate:trusted_cert_and_path(ASN1Certs, CertDbHandle, CertDbRef, PartialChain), case public_key:pkix_path_validation(TrustedErlCert, CertPath, [{max_path_length, @@ -426,8 +439,8 @@ certify(#certificate{asn1_certificates = ASN1Certs}, CertDbHandle, CertDbRef, end. %%-------------------------------------------------------------------- --spec verify_connection(tls_version(), #finished{}, client | server, integer(), binary(), - tls_handshake_history()) -> verified | #alert{}. +-spec verify_connection(ssl_record:ssl_version(), #finished{}, client | server, integer(), binary(), + ssl_handshake_history()) -> verified | #alert{}. %% %% Description: Checks the ssl handshake finished message to verify %% the connection. @@ -443,7 +456,7 @@ verify_connection(Version, #finished{verify_data = Data}, end. %%-------------------------------------------------------------------- --spec init_handshake_history() -> tls_handshake_history(). +-spec init_handshake_history() -> ssl_handshake_history(). %% %% Description: Initialize the empty handshake history buffer. @@ -452,8 +465,8 @@ init_handshake_history() -> {[], []}. %%-------------------------------------------------------------------- --spec update_handshake_history(tls_handshake_history(), Data ::term()) -> - tls_handshake_history(). +-spec update_handshake_history(ssl_handshake:ssl_handshake_history(), Data ::term()) -> + ssl_handshake:ssl_handshake_history(). %% %% Description: Update the handshake history buffer with Data. %%-------------------------------------------------------------------- @@ -567,7 +580,7 @@ server_key_exchange_hash(md5sha, Value) -> server_key_exchange_hash(Hash, Value) -> crypto:hash(Hash, Value). %%-------------------------------------------------------------------- --spec prf(tls_version(), binary(), binary(), [binary()], non_neg_integer()) -> +-spec prf(ssl_record:ssl_version(), binary(), binary(), [binary()], non_neg_integer()) -> {ok, binary()} | {error, undefined}. %% %% Description: use the TLS PRF to generate key material @@ -578,23 +591,25 @@ prf({3,1}, Secret, Label, Seed, WantedLength) -> {ok, tls_v1:prf(?MD5SHA, Secret, Label, Seed, WantedLength)}; prf({3,_N}, Secret, Label, Seed, WantedLength) -> {ok, tls_v1:prf(?SHA256, Secret, Label, Seed, WantedLength)}. + + %%-------------------------------------------------------------------- --spec select_hashsign(#hash_sign_algos{}| undefined, undefined | binary()) -> - [{atom(), atom()}] | undefined. +-spec select_hashsign(#hash_sign_algos{}| undefined, undefined | binary(), ssl_record:ssl_version()) -> + {atom(), atom()} | undefined. %% %% Description: %%-------------------------------------------------------------------- -select_hashsign(_, undefined) -> +select_hashsign(_, undefined, _Version) -> {null, anon}; -select_hashsign(undefined, Cert) -> +select_hashsign(undefined, Cert, Version) -> #'OTPCertificate'{tbsCertificate = TBSCert} = public_key:pkix_decode_cert(Cert, otp), #'OTPSubjectPublicKeyInfo'{algorithm = {_,Algo, _}} = TBSCert#'OTPTBSCertificate'.subjectPublicKeyInfo, - select_cert_hashsign(undefined, Algo, {undefined, undefined}); -select_hashsign(#hash_sign_algos{hash_sign_algos = HashSigns}, Cert) -> + select_hashsign_algs(undefined, Algo, Version); +select_hashsign(#hash_sign_algos{hash_sign_algos = HashSigns}, Cert, Version) -> #'OTPCertificate'{tbsCertificate = TBSCert} =public_key:pkix_decode_cert(Cert, otp), #'OTPSubjectPublicKeyInfo'{algorithm = {_,Algo, _}} = TBSCert#'OTPTBSCertificate'.subjectPublicKeyInfo, - DefaultHashSign = {_, Sign} = select_cert_hashsign(undefined, Algo, {undefined, undefined}), + DefaultHashSign = {_, Sign} = select_hashsign_algs(undefined, Algo, Version), case lists:filter(fun({sha, dsa}) -> true; ({_, dsa}) -> @@ -610,28 +625,61 @@ select_hashsign(#hash_sign_algos{hash_sign_algos = HashSigns}, Cert) -> [HashSign| _] -> HashSign end. + %%-------------------------------------------------------------------- --spec select_cert_hashsign(#hash_sign_algos{}| undefined, oid(), tls_version() | {undefined, undefined}) -> +-spec select_hashsign_algs(#hash_sign_algos{}| undefined, oid(), ssl_record:ssl_version()) -> {atom(), atom()}. +%% Description: For TLS 1.2 hash function and signature algorithm pairs can be +%% negotiated with the signature_algorithms extension, +%% for previous versions always use appropriate defaults. +%% RFC 5246, Sect. 7.4.1.4.1. Signature Algorithms +%% If the client does not send the signature_algorithms extension, the +%% server MUST do the following: (e.i defaults for TLS 1.2) +%% +%% - If the negotiated key exchange algorithm is one of (RSA, DHE_RSA, +%% DH_RSA, RSA_PSK, ECDH_RSA, ECDHE_RSA), behave as if client had +%% sent the value {sha1,rsa}. +%% +%% - If the negotiated key exchange algorithm is one of (DHE_DSS, +%% DH_DSS), behave as if the client had sent the value {sha1,dsa}. %% -%% Description: For TLS 1.2 selected cert_hash_sign will be recived -%% in the handshake message, for previous versions use appropriate defaults. -%% This function is also used by select_hashsign to extract -%% the alogrithm of the server cert key. +%% - If the negotiated key exchange algorithm is one of (ECDH_ECDSA, +%% ECDHE_ECDSA), behave as if the client had sent value {sha1,ecdsa}. + %%-------------------------------------------------------------------- -select_cert_hashsign(HashSign, _, {Major, Minor}) when HashSign =/= undefined andalso +select_hashsign_algs(HashSign, _, {Major, Minor}) when HashSign =/= undefined andalso Major >= 3 andalso Minor >= 3 -> HashSign; -select_cert_hashsign(undefined,?'id-ecPublicKey', _) -> +select_hashsign_algs(undefined, ?rsaEncryption, {Major, Minor}) when Major >= 3 andalso Minor >= 3 -> + {sha, rsa}; +select_hashsign_algs(undefined,?'id-ecPublicKey', _) -> {sha, ecdsa}; -select_cert_hashsign(undefined, ?rsaEncryption, _) -> +select_hashsign_algs(undefined, ?rsaEncryption, _) -> {md5sha, rsa}; -select_cert_hashsign(undefined, ?'id-dsa', _) -> +select_hashsign_algs(undefined, ?'id-dsa', _) -> {sha, dsa}. +-spec select_hashsign_algs(atom(), ssl_record:ssl_version()) -> {atom(), atom()}. +%% Wrap function to keep the knowledge of the default values in +%% one place only +select_hashsign_algs(Alg, Version) when (Alg == rsa orelse + Alg == dhe_rsa orelse + Alg == dh_rsa orelse + Alg == ecdhe_rsa orelse + Alg == ecdh_rsa orelse + Alg == srp_rsa) -> + select_hashsign_algs(undefined, ?rsaEncryption, Version); +select_hashsign_algs(Alg, Version) when (Alg == dhe_dss orelse + Alg == dh_dss orelse + Alg == srp_dss) -> + select_hashsign_algs(undefined, ?'id-dsa', Version); +select_hashsign_algs(Alg, Version) when (Alg == ecdhe_ecdsa orelse + Alg == ecdh_ecdsa) -> + select_hashsign_algs(undefined, ?'id-ecPublicKey', Version). + %%-------------------------------------------------------------------- --spec master_secret(atom(), tls_version(), #session{} | binary(), #connection_states{}, +-spec master_secret(atom(), ssl_record:ssl_version(), #session{} | binary(), #connection_states{}, client | server) -> {binary(), #connection_states{}} | #alert{}. %% %% Description: Sets or calculates the master secret and calculate keys, @@ -816,7 +864,7 @@ enc_server_key_exchange(Version, Params, {HashAlgo, SignAlgo}, end. %%-------------------------------------------------------------------- --spec decode_client_key(binary(), key_algo(), tls_version()) -> +-spec decode_client_key(binary(), ssl_cipher:key_algo(), ssl_record:ssl_version()) -> #encrypted_premaster_secret{} | #client_diffie_hellman_public{} | #client_ec_diffie_hellman_public{} @@ -831,7 +879,7 @@ decode_client_key(ClientKey, Type, Version) -> dec_client_key(ClientKey, key_exchange_alg(Type), Version). %%-------------------------------------------------------------------- --spec decode_server_key(binary(), key_algo(), tls_version()) -> +-spec decode_server_key(binary(), ssl_cipher:key_algo(), ssl_record:ssl_version()) -> #server_key_params{}. %% %% Description: Decode server_key data and return appropriate type @@ -1005,12 +1053,9 @@ decode_suites('3_bytes', Dec) -> %%-------------Cipeher suite handling -------------------------------- available_suites(UserSuites, Version) -> - case UserSuites of - [] -> - ssl_cipher:suites(Version); - _ -> - UserSuites - end. + lists:filtermap(fun(Suite) -> + lists:member(Suite, ssl_cipher:all_suites(Version)) + end, UserSuites). available_suites(ServerCert, UserSuites, Version, Curve) -> ssl_cipher:filter(ServerCert, available_suites(UserSuites, Version)) @@ -1053,19 +1098,31 @@ supported_ecc(_) -> %%-------------certificate handling -------------------------------- -certificate_types({KeyExchange, _, _, _}) - when KeyExchange == rsa; - KeyExchange == dhe_dss; - KeyExchange == dhe_rsa; - KeyExchange == ecdhe_rsa -> - <<?BYTE(?RSA_SIGN), ?BYTE(?DSS_SIGN)>>; +certificate_types(_, {N, M}) when N >= 3 andalso M >= 3 -> + case proplists:get_bool(ecdsa, + proplists:get_value(public_keys, crypto:supports())) of + true -> + <<?BYTE(?ECDSA_SIGN), ?BYTE(?RSA_SIGN), ?BYTE(?DSS_SIGN)>>; + false -> + <<?BYTE(?RSA_SIGN), ?BYTE(?DSS_SIGN)>> + end; + +certificate_types({KeyExchange, _, _, _}, _) when KeyExchange == rsa; + KeyExchange == dhe_rsa; + KeyExchange == ecdhe_rsa -> + <<?BYTE(?RSA_SIGN)>>; + +certificate_types({KeyExchange, _, _, _}, _) when KeyExchange == dhe_dss; + KeyExchange == srp_dss -> + <<?BYTE(?DSS_SIGN)>>; -certificate_types({KeyExchange, _, _, _}) - when KeyExchange == dh_ecdsa; - KeyExchange == dhe_ecdsa -> +certificate_types({KeyExchange, _, _, _}, _) when KeyExchange == dh_ecdsa; + KeyExchange == dhe_ecdsa; + KeyExchange == ecdh_ecdsa; + KeyExchange == ecdhe_ecdsa -> <<?BYTE(?ECDSA_SIGN)>>; -certificate_types(_) -> +certificate_types(_, _) -> <<?BYTE(?RSA_SIGN)>>. certificate_authorities(CertDbHandle, CertDbRef) -> @@ -1674,6 +1731,11 @@ dec_hello_extensions(<<?UINT16(?EC_POINT_FORMATS_EXT), ?UINT16(Len), dec_hello_extensions(Rest, Acc#hello_extensions{ec_point_formats = #ec_point_formats{ec_point_format_list = ECPointFormats}}); + +dec_hello_extensions(<<?UINT16(?SNI_EXT), ?UINT16(Len), + ExtData:Len/binary, Rest/binary>>, Acc) -> + <<?UINT16(_), NameList/binary>> = ExtData, + dec_hello_extensions(Rest, Acc#hello_extensions{sni = dec_sni(NameList)}); %% Ignore data following the ClientHello (i.e., %% extensions) if not understood. @@ -1686,6 +1748,13 @@ dec_hello_extensions(_, Acc) -> dec_hashsign(<<?BYTE(HashAlgo), ?BYTE(SignAlgo)>>) -> {ssl_cipher:hash_algorithm(HashAlgo), ssl_cipher:sign_algorithm(SignAlgo)}. +%% Ignore unknown names (only host_name is supported) +dec_sni(<<?BYTE(?SNI_NAMETYPE_HOST_NAME), ?UINT16(Len), + HostName:Len/binary, _/binary>>) -> + #sni{hostname = binary_to_list(HostName)}; +dec_sni(<<?BYTE(_), ?UINT16(Len), _:Len, Rest/binary>>) -> dec_sni(Rest); +dec_sni(_) -> undefined. + decode_next_protocols({next_protocol_negotiation, Protocols}) -> decode_next_protocols(Protocols, []). decode_next_protocols(<<>>, Acc) -> diff --git a/lib/ssl/src/ssl_handshake.hrl b/lib/ssl/src/ssl_handshake.hrl index 75160526b9..80284faef0 100644 --- a/lib/ssl/src/ssl_handshake.hrl +++ b/lib/ssl/src/ssl_handshake.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2013. All Rights Reserved. +%% Copyright Ericsson AB 2007-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -352,18 +352,4 @@ hostname = undefined }). -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% Dialyzer types -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - --type oid() :: tuple(). --type public_key_params() :: #'Dss-Parms'{} | {namedCurve, oid()} | #'ECParameters'{} | term(). --type public_key_info() :: {oid(), #'RSAPublicKey'{} | integer() | #'ECPoint'{}, public_key_params()}. --type tls_handshake_history() :: {[binary()], [binary()]}. - --type ssl_handshake() :: #server_hello{} | #server_hello_done{} | #certificate{} | #certificate_request{} | - #client_key_exchange{} | #finished{} | #certificate_verify{} | - #hello_request{} | #next_protocol{}. - - -endif. % -ifdef(ssl_handshake). diff --git a/lib/ssl/src/ssl_internal.hrl b/lib/ssl/src/ssl_internal.hrl index 64b89e9f95..85724de4bd 100644 --- a/lib/ssl/src/ssl_internal.hrl +++ b/lib/ssl/src/ssl_internal.hrl @@ -30,8 +30,6 @@ -type from() :: term(). -type host() :: inet:ip_address() | inet:hostname(). -type session_id() :: 0 | binary(). --type tls_version() :: {integer(), integer()}. --type tls_atom_version() :: sslv3 | tlsv1 | 'tlsv1.1' | 'tlsv1.2'. -type certdb_ref() :: reference(). -type db_handle() :: term(). -type der_cert() :: binary(). @@ -73,9 +71,10 @@ -record(ssl_options, { protocol :: tls | dtls, - versions :: ['tlsv1.2' | 'tlsv1.1' | tlsv1 | sslv3] | ['dtlsv1.2' | dtlsv1], + versions :: [ssl_record:ssl_version()], %% ssl_record:atom_version() in API verify :: verify_none | verify_peer, verify_fun, %%:: fun(CertVerifyErrors::term()) -> boolean(), + partial_chain :: fun(), fail_if_no_peer_cert :: boolean(), verify_client_once :: boolean(), %% fun(Extensions, State, Verify, AccError) -> {Extensions, State, AccError} @@ -103,7 +102,6 @@ reuse_sessions :: boolean(), renegotiate_at, secure_renegotiate, - debug, %% undefined if not hibernating, or number of ms of %% inactivity after which ssl_connection will go into %% hibernation @@ -119,14 +117,6 @@ honor_cipher_order = false }). --record(config, {ssl, %% SSL parameters - inet_user, %% User set inet options - emulated, %% #socket_option{} emulated - inet_ssl, %% inet options for internal ssl socket - transport_info, %% Callback info - connection_cb - }). - -record(socket_options, { mode = list, @@ -136,6 +126,15 @@ active = true }). +-record(config, {ssl, %% SSL parameters + inet_user, %% User set inet options + emulated, %% Emulated option list or "inherit_tracker" pid + inet_ssl, %% inet options for internal ssl socket + transport_info, %% Callback info + connection_cb + }). + + -type state_name() :: hello | abbreviated | certify | cipher | connection. -type gen_fsm_state_return() :: {next_state, state_name(), term()} | {next_state, state_name(), term(), timeout()} | diff --git a/lib/ssl/src/ssl_listen_tracker_sup.erl b/lib/ssl/src/ssl_listen_tracker_sup.erl new file mode 100644 index 0000000000..29f40e846d --- /dev/null +++ b/lib/ssl/src/ssl_listen_tracker_sup.erl @@ -0,0 +1,71 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2014-2014. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% + +%% +%%---------------------------------------------------------------------- +%% Purpose: Supervisor for a listen options tracker +%%---------------------------------------------------------------------- +-module(ssl_listen_tracker_sup). + +-behaviour(supervisor). + +%% API +-export([start_link/0, start_link_dist/0]). +-export([start_child/1, start_child_dist/1]). + +%% Supervisor callback +-export([init/1]). + +%%%========================================================================= +%%% API +%%%========================================================================= +start_link() -> + supervisor:start_link({local, tracker_name(normal)}, ?MODULE, []). + +start_link_dist() -> + supervisor:start_link({local, tracker_name(dist)}, ?MODULE, []). + +start_child(Args) -> + supervisor:start_child(tracker_name(normal), Args). + +start_child_dist(Args) -> + supervisor:start_child(tracker_name(dist), Args). + +%%%========================================================================= +%%% Supervisor callback +%%%========================================================================= +init(_O) -> + RestartStrategy = simple_one_for_one, + MaxR = 0, + MaxT = 3600, + + Name = undefined, % As simple_one_for_one is used. + StartFunc = {ssl_socket, start_link, []}, + Restart = temporary, % E.g. should not be restarted + Shutdown = 4000, + Modules = [ssl_socket], + Type = worker, + + ChildSpec = {Name, StartFunc, Restart, Shutdown, Type, Modules}, + {ok, {{RestartStrategy, MaxR, MaxT}, [ChildSpec]}}. + +tracker_name(normal) -> + ?MODULE; +tracker_name(dist) -> + list_to_atom(atom_to_list(?MODULE) ++ "dist"). diff --git a/lib/ssl/src/ssl_manager.erl b/lib/ssl/src/ssl_manager.erl index fbc73e0e42..d6e5064c39 100644 --- a/lib/ssl/src/ssl_manager.erl +++ b/lib/ssl/src/ssl_manager.erl @@ -52,8 +52,8 @@ last_delay_timer = {undefined, undefined}%% Keep for testing purposes }). --define('24H_in_msec', 8640000). --define('24H_in_sec', 8640). +-define('24H_in_msec', 86400000). +-define('24H_in_sec', 86400). -define(GEN_UNIQUE_ID_MAX_TRIES, 10). -define(SESSION_VALIDATION_INTERVAL, 60000). -define(CLEAR_PEM_CACHE, 120000). diff --git a/lib/ssl/src/ssl_record.erl b/lib/ssl/src/ssl_record.erl index 018c8befe0..7337225bc4 100644 --- a/lib/ssl/src/ssl_record.erl +++ b/lib/ssl/src/ssl_record.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2013-2013. All Rights Reserved. +%% Copyright Ericsson AB 2013-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -50,6 +50,11 @@ %% Payload encryption/decryption -export([cipher/4, decipher/3, is_correct_mac/2]). +-export_type([ssl_version/0, ssl_atom_version/0]). + +-type ssl_version() :: {integer(), integer()}. +-type ssl_atom_version() :: tls_record:tls_atom_version(). + %%==================================================================== %% Internal application API %%==================================================================== @@ -299,7 +304,7 @@ set_pending_cipher_state(#connection_states{pending_read = Read, %%-------------------------------------------------------------------- --spec encode_handshake(iolist(), tls_version(), #connection_states{}) -> +-spec encode_handshake(iolist(), ssl_version(), #connection_states{}) -> {iolist(), #connection_states{}}. %% %% Description: Encodes a handshake message to send on the ssl-socket. @@ -308,7 +313,7 @@ encode_handshake(Frag, Version, ConnectionStates) -> encode_plain_text(?HANDSHAKE, Version, Frag, ConnectionStates). %%-------------------------------------------------------------------- --spec encode_alert_record(#alert{}, tls_version(), #connection_states{}) -> +-spec encode_alert_record(#alert{}, ssl_version(), #connection_states{}) -> {iolist(), #connection_states{}}. %% %% Description: Encodes an alert message to send on the ssl-socket. @@ -319,7 +324,7 @@ encode_alert_record(#alert{level = Level, description = Description}, ConnectionStates). %%-------------------------------------------------------------------- --spec encode_change_cipher_spec(tls_version(), #connection_states{}) -> +-spec encode_change_cipher_spec(ssl_version(), #connection_states{}) -> {iolist(), #connection_states{}}. %% %% Description: Encodes a change_cipher_spec-message to send on the ssl socket. @@ -328,7 +333,7 @@ encode_change_cipher_spec(Version, ConnectionStates) -> encode_plain_text(?CHANGE_CIPHER_SPEC, Version, <<1:8>>, ConnectionStates). %%-------------------------------------------------------------------- --spec encode_data(binary(), tls_version(), #connection_states{}) -> +-spec encode_data(binary(), ssl_version(), #connection_states{}) -> {iolist(), #connection_states{}}. %% %% Description: Encodes data to send on the ssl-socket. @@ -356,7 +361,7 @@ compressions() -> [?byte(?NULL)]. %%-------------------------------------------------------------------- --spec cipher(tls_version(), iolist(), #connection_state{}, MacHash::binary()) -> +-spec cipher(ssl_version(), iodata(), #connection_state{}, MacHash::binary()) -> {CipherFragment::binary(), #connection_state{}}. %% %% Description: Payload encryption @@ -372,7 +377,7 @@ cipher(Version, Fragment, ssl_cipher:cipher(BulkCipherAlgo, CipherS0, MacHash, Fragment, Version), {CipherFragment, WriteState0#connection_state{cipher_state = CipherS1}}. %%-------------------------------------------------------------------- --spec decipher(tls_version(), binary(), #connection_state{}) -> {binary(), binary(), #connection_state{}}. +-spec decipher(ssl_version(), binary(), #connection_state{}) -> {binary(), binary(), #connection_state{}} | #alert{}. %% %% Description: Payload decryption %%-------------------------------------------------------------------- diff --git a/lib/ssl/src/ssl_record.hrl b/lib/ssl/src/ssl_record.hrl index c17fa53a62..6aab35d6da 100644 --- a/lib/ssl/src/ssl_record.hrl +++ b/lib/ssl/src/ssl_record.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2013. All Rights Reserved. +%% Copyright Ericsson AB 2007-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -20,7 +20,7 @@ %% %%---------------------------------------------------------------------- %% Purpose: Record and constant defenitions for the SSL-record protocol -%% see RFC 2246 +% see RFC 2246 %%---------------------------------------------------------------------- -ifndef(ssl_record). @@ -70,7 +70,7 @@ -define(INITIAL_BYTES, 5). --define(MAX_SEQENCE_NUMBER, 18446744073709552000). %% math:pow(2, 64) - 1 = 1.8446744073709552e19 +-define(MAX_SEQENCE_NUMBER, 18446744073709551615). %% (1 bsl 64) - 1 = 18446744073709551615 %% Sequence numbers can not wrap so when max is about to be reached we should renegotiate. %% We will renegotiate a little before so that there will be sequence numbers left %% for the rehandshake and a little data. Currently we decided to renegotiate a little more diff --git a/lib/ssl/src/ssl_socket.erl b/lib/ssl/src/ssl_socket.erl index 1b6e637cd3..55eb569b20 100644 --- a/lib/ssl/src/ssl_socket.erl +++ b/lib/ssl/src/ssl_socket.erl @@ -1,20 +1,73 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 1998-2014. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% -module(ssl_socket). +-behaviour(gen_server). + -include("ssl_internal.hrl"). -include("ssl_api.hrl"). --export([socket/4, setopts/3, getopts/3, peername/2, sockname/2, port/2]). +-export([socket/5, setopts/3, getopts/3, peername/2, sockname/2, port/2]). +-export([emulated_options/0, internal_inet_values/0, default_inet_values/0, + init/1, start_link/3, terminate/2, inherit_tracker/3, get_emulated_opts/1, + set_emulated_opts/2, get_all_opts/1, handle_call/3, handle_cast/2, + handle_info/2, code_change/3]). + +-record(state, { + emulated_opts, + port, + ssl_opts + }). -socket(Pid, Transport, Socket, ConnectionCb) -> +%%-------------------------------------------------------------------- +%%% Internal API +%%-------------------------------------------------------------------- +socket(Pid, Transport, Socket, ConnectionCb, Tracker) -> #sslsocket{pid = Pid, %% "The name "fd" is keept for backwards compatibility - fd = {Transport, Socket, ConnectionCb}}. - + fd = {Transport, Socket, ConnectionCb, Tracker}}. +setopts(gen_tcp, #sslsocket{pid = {ListenSocket, #config{emulated = Tracker}}}, Options) -> + {SockOpts, EmulatedOpts} = split_options(Options), + ok = set_emulated_opts(Tracker, EmulatedOpts), + inet:setopts(ListenSocket, SockOpts); +setopts(_, #sslsocket{pid = {ListenSocket, #config{transport_info = {Transport,_,_,_}, + emulated = Tracker}}}, Options) -> + {SockOpts, EmulatedOpts} = split_options(Options), + ok = set_emulated_opts(Tracker, EmulatedOpts), + Transport:setopts(ListenSocket, SockOpts); +%%% Following clauses will not be called for emulated options, they are handled in the connection process setopts(gen_tcp, Socket, Options) -> inet:setopts(Socket, Options); setopts(Transport, Socket, Options) -> Transport:setopts(Socket, Options). +getopts(gen_tcp, #sslsocket{pid = {ListenSocket, #config{emulated = Tracker}}}, Options) -> + {SockOptNames, EmulatedOptNames} = split_options(Options), + EmulatedOpts = get_emulated_opts(Tracker, EmulatedOptNames), + SocketOpts = get_socket_opts(ListenSocket, SockOptNames, inet), + {ok, EmulatedOpts ++ SocketOpts}; +getopts(Transport, #sslsocket{pid = {ListenSocket, #config{emulated = Tracker}}}, Options) -> + {SockOptNames, EmulatedOptNames} = split_options(Options), + EmulatedOpts = get_emulated_opts(Tracker, EmulatedOptNames), + SocketOpts = get_socket_opts(ListenSocket, SockOptNames, Transport), + {ok, EmulatedOpts ++ SocketOpts}; +%%% Following clauses will not be called for emulated options, they are handled in the connection process getopts(gen_tcp, Socket, Options) -> inet:getopts(Socket, Options); getopts(Transport, Socket, Options) -> @@ -34,3 +87,151 @@ port(gen_tcp, Socket) -> inet:port(Socket); port(Transport, Socket) -> Transport:port(Socket). + +emulated_options() -> + [mode, packet, active, header, packet_size]. + +internal_inet_values() -> + [{packet_size,0}, {packet, 0}, {header, 0}, {active, false}, {mode,binary}]. + +default_inet_values() -> + [{packet_size, 0}, {packet,0}, {header, 0}, {active, true}, {mode, list}]. + +inherit_tracker(ListenSocket, EmOpts, #ssl_options{erl_dist = false} = SslOpts) -> + ssl_listen_tracker_sup:start_child([ListenSocket, EmOpts, SslOpts]); +inherit_tracker(ListenSocket, EmOpts, #ssl_options{erl_dist = true} = SslOpts) -> + ssl_listen_tracker_sup:start_child_dist([ListenSocket, EmOpts, SslOpts]). + +get_emulated_opts(TrackerPid) -> + call(TrackerPid, get_emulated_opts). +set_emulated_opts(TrackerPid, InetValues) -> + call(TrackerPid, {set_emulated_opts, InetValues}). +get_all_opts(TrackerPid) -> + call(TrackerPid, get_all_opts). + +%%==================================================================== +%% ssl_listen_tracker_sup API +%%==================================================================== + +start_link(Port, SockOpts, SslOpts) -> + gen_server:start_link(?MODULE, [Port, SockOpts, SslOpts], []). + +%%-------------------------------------------------------------------- +-spec init(list()) -> {ok, #state{}}. +%% Possible return values not used now. +%% | {ok, #state{}, timeout()} | ignore | {stop, term()}. +%% +%% Description: Initiates the server +%%-------------------------------------------------------------------- +init([Port, Opts, SslOpts]) -> + process_flag(trap_exit, true), + true = link(Port), + {ok, #state{emulated_opts = Opts, port = Port, ssl_opts = SslOpts}}. + +%%-------------------------------------------------------------------- +-spec handle_call(msg(), from(), #state{}) -> {reply, reply(), #state{}}. +%% Possible return values not used now. +%% {reply, reply(), #state{}, timeout()} | +%% {noreply, #state{}} | +%% {noreply, #state{}, timeout()} | +%% {stop, reason(), reply(), #state{}} | +%% {stop, reason(), #state{}}. +%% +%% Description: Handling call messages +%%-------------------------------------------------------------------- +handle_call({set_emulated_opts, Opts0}, _From, + #state{emulated_opts = Opts1} = State) -> + Opts = do_set_emulated_opts(Opts0, Opts1), + {reply, ok, State#state{emulated_opts = Opts}}; +handle_call(get_emulated_opts, _From, + #state{emulated_opts = Opts} = State) -> + {reply, {ok, Opts}, State}; +handle_call(get_all_opts, _From, + #state{emulated_opts = EmOpts, + ssl_opts = SslOpts} = State) -> + {reply, {ok, EmOpts, SslOpts}, State}. + +%%-------------------------------------------------------------------- +-spec handle_cast(msg(), #state{}) -> {noreply, #state{}}. +%% Possible return values not used now. +%% | {noreply, #state{}, timeout()} | +%% {stop, reason(), #state{}}. +%% +%% Description: Handling cast messages +%%-------------------------------------------------------------------- +handle_cast(_, State)-> + {noreply, State}. + +%%-------------------------------------------------------------------- +-spec handle_info(msg(), #state{}) -> {stop, reason(), #state{}}. +%% Possible return values not used now. +%% {noreply, #state{}}. +%% |{noreply, #state{}, timeout()} | +%% +%% +%% Description: Handling all non call/cast messages +%%------------------------------------------------------------------- +handle_info({'EXIT', Port, _}, #state{port = Port} = State) -> + {stop, normal, State}. + + +%%-------------------------------------------------------------------- +-spec terminate(reason(), #state{}) -> ok. +%% +%% Description: This function is called by a gen_server when it is about to +%% terminate. It should be the opposite of Module:init/1 and do any necessary +%% cleaning up. When it returns, the gen_server terminates with Reason. +%% The return value is ignored. +%%-------------------------------------------------------------------- +terminate(_Reason, _State) -> + ok. + +%%-------------------------------------------------------------------- +-spec code_change(term(), #state{}, list()) -> {ok, #state{}}. +%% +%% Description: Convert process state when code is changed +%%-------------------------------------------------------------------- +code_change(_OldVsn, State, _Extra) -> + {ok, State}. + +%%-------------------------------------------------------------------- +%%% Internal functions +%%-------------------------------------------------------------------- +call(Pid, Msg) -> + gen_server:call(Pid, Msg, infinity). + +split_options(Opts) -> + split_options(Opts, emulated_options(), [], []). +split_options([], _, SocketOpts, EmuOpts) -> + {SocketOpts, EmuOpts}; +split_options([{Name, _} = Opt | Opts], Emu, SocketOpts, EmuOpts) -> + case lists:member(Name, Emu) of + true -> + split_options(Opts, Emu, SocketOpts, [Opt | EmuOpts]); + false -> + split_options(Opts, Emu, [Opt | SocketOpts], EmuOpts) + end; +split_options([Name | Opts], Emu, SocketOptNames, EmuOptNames) -> + case lists:member(Name, Emu) of + true -> + split_options(Opts, Emu, SocketOptNames, [Name | EmuOptNames]); + false -> + split_options(Opts, Emu, [Name | SocketOptNames], EmuOptNames) + end. + +do_set_emulated_opts([], Opts) -> + Opts; +do_set_emulated_opts([{Name,_} = Opt | Rest], Opts) -> + do_set_emulated_opts(Rest, [Opt | proplists:delete(Name, Opts)]). + +get_socket_opts(_, [], _) -> + []; +get_socket_opts(ListenSocket, SockOptNames, Cb) -> + {ok, Opts} = Cb:getopts(ListenSocket, SockOptNames), + Opts. + +get_emulated_opts(TrackerPid, EmOptNames) -> + {ok, EmOpts} = get_emulated_opts(TrackerPid), + lists:map(fun(Name) -> {value, Value} = lists:keysearch(Name, 1, EmOpts), + Value end, + EmOptNames). diff --git a/lib/ssl/src/ssl_sup.erl b/lib/ssl/src/ssl_sup.erl index e1aeb11ca4..7cccf8d5a5 100644 --- a/lib/ssl/src/ssl_sup.erl +++ b/lib/ssl/src/ssl_sup.erl @@ -47,8 +47,10 @@ init([]) -> TLSConnetionManager = tls_connection_manager_child_spec(), %% Not supported yet %%DTLSConnetionManager = tls_connection_manager_child_spec(), - - {ok, {{one_for_all, 10, 3600}, [SessionCertManager, TLSConnetionManager]}}. + %% Handles emulated options so that they inherited by the accept socket, even when setopts is performed on + %% the listen socket + ListenOptionsTracker = listen_options_tracker_child_spec(), + {ok, {{one_for_all, 10, 3600}, [SessionCertManager, TLSConnetionManager, ListenOptionsTracker]}}. manager_opts() -> @@ -85,7 +87,7 @@ tls_connection_manager_child_spec() -> StartFunc = {tls_connection_sup, start_link, []}, Restart = permanent, Shutdown = 4000, - Modules = [tls_connection, ssl_connection], + Modules = [tls_connection_sup], Type = supervisor, {Name, StartFunc, Restart, Shutdown, Type, Modules}. @@ -98,6 +100,17 @@ tls_connection_manager_child_spec() -> %% Type = supervisor, %% {Name, StartFunc, Restart, Shutdown, Type, Modules}. + +listen_options_tracker_child_spec() -> + Name = ssl_socket, + StartFunc = {ssl_listen_tracker_sup, start_link, []}, + Restart = permanent, + Shutdown = 4000, + Modules = [ssl_socket], + Type = supervisor, + {Name, StartFunc, Restart, Shutdown, Type, Modules}. + + session_cb_init_args() -> case application:get_env(ssl, session_cb_init_args) of {ok, Args} when is_list(Args) -> diff --git a/lib/ssl/src/ssl_v3.erl b/lib/ssl/src/ssl_v3.erl index d477b3df81..68f7f5dee2 100644 --- a/lib/ssl/src/ssl_v3.erl +++ b/lib/ssl/src/ssl_v3.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2013. All Rights Reserved. +%% Copyright Ericsson AB 2007-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -130,7 +130,7 @@ setup_keys(MasterSecret, ServerRandom, ClientRandom, HS, KML, _EKML, IVS) -> {ClientWriteMacSecret, ServerWriteMacSecret, ClientWriteKey, ServerWriteKey, ClientIV, ServerIV}. --spec suites() -> [cipher_suite()]. +-spec suites() -> [ssl_cipher:cipher_suite()]. suites() -> [ diff --git a/lib/ssl/src/tls_connection.erl b/lib/ssl/src/tls_connection.erl index ffa04ee8ba..7df73fb581 100644 --- a/lib/ssl/src/tls_connection.erl +++ b/lib/ssl/src/tls_connection.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2013. All Rights Reserved. +%% Copyright Ericsson AB 2007-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -53,7 +53,7 @@ %% Alert and close handling -export([send_alert/2, handle_own_alert/4, handle_close_alert/3, handle_normal_shutdown/3, handle_unexpected_message/3, - workaround_transport_delivery_problems/2, alert_user/5, alert_user/8 + workaround_transport_delivery_problems/2, alert_user/6, alert_user/9 ]). %% Data handling @@ -66,18 +66,18 @@ %% gen_fsm callbacks -export([init/1, hello/2, certify/2, cipher/2, abbreviated/2, connection/2, handle_event/3, - handle_sync_event/4, handle_info/3, terminate/3, code_change/4]). + handle_sync_event/4, handle_info/3, terminate/3, code_change/4, format_status/2]). %%==================================================================== %% Internal application API %%==================================================================== -start_fsm(Role, Host, Port, Socket, {#ssl_options{erl_dist = false},_} = Opts, +start_fsm(Role, Host, Port, Socket, {#ssl_options{erl_dist = false},_, Tracker} = Opts, User, {CbModule, _,_, _} = CbInfo, Timeout) -> try {ok, Pid} = tls_connection_sup:start_child([Role, Host, Port, Socket, Opts, User, CbInfo]), - {ok, SslSocket} = ssl_connection:socket_control(?MODULE, Socket, Pid, CbModule), + {ok, SslSocket} = ssl_connection:socket_control(?MODULE, Socket, Pid, CbModule, Tracker), ok = ssl_connection:handshake(SslSocket, Timeout), {ok, SslSocket} catch @@ -85,13 +85,13 @@ start_fsm(Role, Host, Port, Socket, {#ssl_options{erl_dist = false},_} = Opts, Error end; -start_fsm(Role, Host, Port, Socket, {#ssl_options{erl_dist = true},_} = Opts, +start_fsm(Role, Host, Port, Socket, {#ssl_options{erl_dist = true},_, Tracker} = Opts, User, {CbModule, _,_, _} = CbInfo, Timeout) -> try {ok, Pid} = tls_connection_sup:start_child_dist([Role, Host, Port, Socket, Opts, User, CbInfo]), - {ok, SslSocket} = ssl_connection:socket_control(?MODULE, Socket, Pid, CbModule), + {ok, SslSocket} = ssl_connection:socket_control(?MODULE, Socket, Pid, CbModule, Tracker), ok = ssl_connection:handshake(SslSocket, Timeout), {ok, SslSocket} catch @@ -144,29 +144,10 @@ send_change_cipher(Msg, #state{connection_states = ConnectionStates0, start_link(Role, Host, Port, Socket, Options, User, CbInfo) -> {ok, proc_lib:spawn_link(?MODULE, init, [[Role, Host, Port, Socket, Options, User, CbInfo]])}. -init([Role, Host, Port, Socket, {SSLOpts0, _} = Options, User, CbInfo]) -> +init([Role, Host, Port, Socket, Options, User, CbInfo]) -> process_flag(trap_exit, true), - State0 = initial_state(Role, Host, Port, Socket, Options, User, CbInfo), - Handshake = ssl_handshake:init_handshake_history(), - TimeStamp = calendar:datetime_to_gregorian_seconds({date(), time()}), - try ssl_config:init(SSLOpts0, Role) of - {ok, Ref, CertDbHandle, FileRefHandle, CacheHandle, OwnCert, Key, DHParams} -> - Session = State0#state.session, - State = State0#state{ - tls_handshake_history = Handshake, - session = Session#session{own_certificate = OwnCert, - time_stamp = TimeStamp}, - file_ref_db = FileRefHandle, - cert_db_ref = Ref, - cert_db = CertDbHandle, - session_cache = CacheHandle, - private_key = Key, - diffie_hellman_params = DHParams}, - gen_fsm:enter_loop(?MODULE, [], hello, State, get_timeout(State)) - catch - throw:Error -> - gen_fsm:enter_loop(?MODULE, [], error, {Error,State0}, get_timeout(State0)) - end. + State = initial_state(Role, Host, Port, Socket, Options, User, CbInfo), + gen_fsm:enter_loop(?MODULE, [], hello, State, get_timeout(State)). %%-------------------------------------------------------------------- %% Description:There should be one instance of this function for each @@ -208,11 +189,11 @@ hello(Hello = #client_hello{client_version = ClientVersion, session_cache = Cache, session_cache_cb = CacheCb, ssl_options = SslOpts}) -> - HashSign = ssl_handshake:select_hashsign(HashSigns, Cert), case tls_handshake:hello(Hello, SslOpts, {Port, Session0, Cache, CacheCb, ConnectionStates0, Cert}, Renegotiation) of {Version, {Type, Session}, ConnectionStates, ServerHelloExt} -> + HashSign = ssl_handshake:select_hashsign(HashSigns, Cert, Version), ssl_connection:hello({common_client_hello, Type, ServerHelloExt, HashSign}, State#state{connection_states = ConnectionStates, negotiated_version = Version, @@ -342,16 +323,21 @@ handle_info(Msg, StateName, State) -> %% Reason. The return value is ignored. %%-------------------------------------------------------------------- terminate(Reason, StateName, State) -> - ssl_connection:terminate(Reason, StateName, State). - + catch ssl_connection:terminate(Reason, StateName, State). %%-------------------------------------------------------------------- %% code_change(OldVsn, StateName, State, Extra) -> {ok, StateName, NewState} %% Description: Convert process state when code is changed %%-------------------------------------------------------------------- -code_change(_OldVsn, StateName, State, _Extra) -> +code_change(_OldVsn, StateName, State0, {Direction, From, To}) -> + State = convert_state(State0, Direction, From, To), + {ok, StateName, State}; +code_change(_OldVsn, StateName, State, _) -> {ok, StateName, State}. +format_status(Type, Data) -> + ssl_connection:format_status(Type, Data). + %%-------------------------------------------------------------------- %%% Internal functions %%-------------------------------------------------------------------- @@ -362,22 +348,13 @@ encode_handshake(Handshake, Version, ConnectionStates0, Hist0) -> ssl_record:encode_handshake(Frag, Version, ConnectionStates0), {Encoded, ConnectionStates, Hist}. - encode_change_cipher(#change_cipher_spec{}, Version, ConnectionStates) -> ssl_record:encode_change_cipher_spec(Version, ConnectionStates). - - decode_alerts(Bin) -> - decode_alerts(Bin, []). + ssl_alert:decode(Bin). -decode_alerts(<<?BYTE(Level), ?BYTE(Description), Rest/binary>>, Acc) -> - A = ?ALERT_REC(Level, Description), - decode_alerts(Rest, [A | Acc]); -decode_alerts(<<>>, Acc) -> - lists:reverse(Acc, []). - -initial_state(Role, Host, Port, Socket, {SSLOptions, SocketOptions}, User, +initial_state(Role, Host, Port, Socket, {SSLOptions, SocketOptions, Tracker}, User, {CbModule, DataTag, CloseTag, ErrorTag}) -> ConnectionStates = ssl_record:init_connection_states(Role), @@ -391,9 +368,7 @@ initial_state(Role, Host, Port, Socket, {SSLOptions, SocketOptions}, User, Monitor = erlang:monitor(process, User), #state{socket_options = SocketOptions, - %% We do not want to save the password in the state so that - %% could be written in the clear into error logs. - ssl_options = SSLOptions#ssl_options{password = undefined}, + ssl_options = SSLOptions, session = #session{is_resumable = new}, transport_cb = CbModule, data_tag = DataTag, @@ -411,7 +386,8 @@ initial_state(Role, Host, Port, Socket, {SSLOptions, SocketOptions}, User, renegotiation = {false, first}, start_or_recv_from = undefined, send_queue = queue:new(), - protocol_cb = ?MODULE + protocol_cb = ?MODULE, + tracker = Tracker }. next_state(Current,_, #alert{} = Alert, #state{negotiated_version = Version} = State) -> @@ -420,10 +396,13 @@ next_state(Current,_, #alert{} = Alert, #state{negotiated_version = Version} = S next_state(_,Next, no_record, State) -> {next_state, Next, State, get_timeout(State)}; -next_state(_,Next, #ssl_tls{type = ?ALERT, fragment = EncAlerts}, State) -> - Alerts = decode_alerts(EncAlerts), - handle_alerts(Alerts, {next_state, Next, State, get_timeout(State)}); - +next_state(Current, Next, #ssl_tls{type = ?ALERT, fragment = EncAlerts}, #state{negotiated_version = Version} = State) -> + case decode_alerts(EncAlerts) of + Alerts = [_|_] -> + handle_alerts(Alerts, {next_state, Next, State, get_timeout(State)}); + #alert{} = Alert -> + handle_own_alert(Alert, Version, Current, State) + end; next_state(Current, Next, #ssl_tls{type = ?HANDSHAKE, fragment = Data}, State0 = #state{protocol_buffers = #protocol_buffers{tls_handshake_buffer = Buf0} = Buffers, @@ -468,12 +447,16 @@ next_state(_, StateName, #ssl_tls{type = ?APPLICATION_DATA, fragment = Data}, St next_state(StateName, StateName, Record, State) end; next_state(Current, Next, #ssl_tls{type = ?CHANGE_CIPHER_SPEC, fragment = <<1>>} = - _ChangeCipher, - #state{connection_states = ConnectionStates0} = State0) -> + _ChangeCipher, + #state{connection_states = ConnectionStates0} = State0) + when Next == cipher; Next == abbreviated -> ConnectionStates1 = ssl_record:activate_pending_connection_state(ConnectionStates0, read), {Record, State} = next_record(State0#state{connection_states = ConnectionStates1}), - next_state(Current, Next, Record, State); + next_state(Current, Next, Record, State#state{expecting_finished = true}); +next_state(Current, _Next, #ssl_tls{type = ?CHANGE_CIPHER_SPEC, fragment = <<1>>} = + _ChangeCipher, #state{negotiated_version = Version} = State) -> + handle_own_alert(?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE), Version, Current, State); next_state(Current, Next, #ssl_tls{type = _Unknown}, State0) -> %% Ignore unknown type {Record, State} = next_record(State0), @@ -513,7 +496,7 @@ next_record(State) -> next_record_if_active(State = #state{socket_options = - #socket_options{active = false}}) -> + #socket_options{active = false}}) -> {no_record ,State}; next_record_if_active(State) -> @@ -577,7 +560,8 @@ read_application_data(Data, #state{user_application = {_Mon, Pid}, bytes_to_read = BytesToRead, start_or_recv_from = RecvFrom, timer = Timer, - user_data_buffer = Buffer0} = State0) -> + user_data_buffer = Buffer0, + tracker = Tracker} = State0) -> Buffer1 = if Buffer0 =:= <<>> -> Data; Data =:= <<>> -> Buffer0; @@ -585,7 +569,7 @@ read_application_data(Data, #state{user_application = {_Mon, Pid}, end, case get_data(SOpts, BytesToRead, Buffer1) of {ok, ClientData, Buffer} -> % Send data - SocketOpt = deliver_app_data(Transport, Socket, SOpts, ClientData, Pid, RecvFrom), + SocketOpt = deliver_app_data(Transport, Socket, SOpts, ClientData, Pid, RecvFrom, Tracker), cancel_timer(Timer), State = State0#state{user_data_buffer = Buffer, start_or_recv_from = undefined, @@ -606,7 +590,7 @@ read_application_data(Data, #state{user_application = {_Mon, Pid}, {passive, Buffer} -> next_record_if_active(State0#state{user_data_buffer = Buffer}); {error,_Reason} -> %% Invalid packet in packet mode - deliver_packet_error(Transport, Socket, SOpts, Buffer1, Pid, RecvFrom), + deliver_packet_error(Transport, Socket, SOpts, Buffer1, Pid, RecvFrom, Tracker), {stop, normal, State0} end. @@ -661,8 +645,8 @@ decode_packet(Type, Buffer, PacketOpts) -> %% HTTP headers using the {packet, httph} option, we don't do any automatic %% switching of states. deliver_app_data(Transport, Socket, SOpts = #socket_options{active=Active, packet=Type}, - Data, Pid, From) -> - send_or_reply(Active, Pid, From, format_reply(Transport, Socket, SOpts, Data)), + Data, Pid, From, Tracker) -> + send_or_reply(Active, Pid, From, format_reply(Transport, Socket, SOpts, Data, Tracker)), SO = case Data of {P, _, _, _} when ((P =:= http_request) or (P =:= http_response)), ((Type =:= http) or (Type =:= http_bin)) -> @@ -682,20 +666,20 @@ deliver_app_data(Transport, Socket, SOpts = #socket_options{active=Active, packe end. format_reply(_, _,#socket_options{active = false, mode = Mode, packet = Packet, - header = Header}, Data) -> + header = Header}, Data, _) -> {ok, do_format_reply(Mode, Packet, Header, Data)}; format_reply(Transport, Socket, #socket_options{active = _, mode = Mode, packet = Packet, - header = Header}, Data) -> - {ssl, ssl_socket:socket(self(), Transport, Socket, ?MODULE), + header = Header}, Data, Tracker) -> + {ssl, ssl_socket:socket(self(), Transport, Socket, ?MODULE, Tracker), do_format_reply(Mode, Packet, Header, Data)}. -deliver_packet_error(Transport, Socket, SO= #socket_options{active = Active}, Data, Pid, From) -> - send_or_reply(Active, Pid, From, format_packet_error(Transport, Socket, SO, Data)). +deliver_packet_error(Transport, Socket, SO= #socket_options{active = Active}, Data, Pid, From, Tracker) -> + send_or_reply(Active, Pid, From, format_packet_error(Transport, Socket, SO, Data, Tracker)). -format_packet_error(_, _,#socket_options{active = false, mode = Mode}, Data) -> +format_packet_error(_, _,#socket_options{active = false, mode = Mode}, Data, _) -> {error, {invalid_packet, do_format_reply(Mode, raw, 0, Data)}}; -format_packet_error(Transport, Socket, #socket_options{active = _, mode = Mode}, Data) -> - {ssl_error, ssl_socket:socket(self(), Transport, Socket, ?MODULE), +format_packet_error(Transport, Socket, #socket_options{active = _, mode = Mode}, Data, Tracker) -> + {ssl_error, ssl_socket:socket(self(), Transport, Socket, ?MODULE, Tracker), {invalid_packet, do_format_reply(Mode, raw, 0, Data)}}. do_format_reply(binary, _, N, Data) when N > 0 -> % Header mode @@ -751,7 +735,11 @@ handle_tls_handshake(Handle, StateName, handle_tls_handshake(Handle, NextStateName, State); {stop, _,_} = Stop -> Stop - end. + end; + +handle_tls_handshake(_Handle, _StateName, #state{}) -> + throw(?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE)). + write_application_data(Data0, From, #state{socket = Socket, negotiated_version = Version, @@ -835,10 +823,10 @@ handle_alert(#alert{level = ?FATAL} = Alert, StateName, #state{socket = Socket, transport_cb = Transport, ssl_options = SslOpts, start_or_recv_from = From, host = Host, port = Port, session = Session, user_application = {_Mon, Pid}, - role = Role, socket_options = Opts} = State) -> + role = Role, socket_options = Opts, tracker = Tracker} = State) -> invalidate_session(Role, Host, Port, Session), log_alert(SslOpts#ssl_options.log_alert, StateName, Alert), - alert_user(Transport, Socket, StateName, Opts, Pid, From, Alert, Role), + alert_user(Transport, Tracker, Socket, StateName, Opts, Pid, From, Alert, Role), {stop, normal, State}; handle_alert(#alert{level = ?WARNING, description = ?CLOSE_NOTIFY} = Alert, @@ -859,36 +847,37 @@ handle_alert(#alert{level = ?WARNING, description = ?NO_RENEGOTIATION} = Alert, {Record, State} = next_record(State0), next_state(StateName, connection, Record, State); -handle_alert(#alert{level = ?WARNING, description = ?USER_CANCELED} = Alert, StateName, +%% Gracefully log and ignore all other warning alerts +handle_alert(#alert{level = ?WARNING} = Alert, StateName, #state{ssl_options = SslOpts} = State0) -> log_alert(SslOpts#ssl_options.log_alert, StateName, Alert), {Record, State} = next_record(State0), next_state(StateName, StateName, Record, State). -alert_user(Transport, Socket, connection, Opts, Pid, From, Alert, Role) -> - alert_user(Transport,Socket, Opts#socket_options.active, Pid, From, Alert, Role); -alert_user(Transport, Socket,_, _, _, From, Alert, Role) -> - alert_user(Transport, Socket, From, Alert, Role). +alert_user(Transport, Tracker, Socket, connection, Opts, Pid, From, Alert, Role) -> + alert_user(Transport, Tracker, Socket, Opts#socket_options.active, Pid, From, Alert, Role); +alert_user(Transport, Tracker, Socket,_, _, _, From, Alert, Role) -> + alert_user(Transport, Tracker, Socket, From, Alert, Role). -alert_user(Transport, Socket, From, Alert, Role) -> - alert_user(Transport, Socket, false, no_pid, From, Alert, Role). +alert_user(Transport, Tracker, Socket, From, Alert, Role) -> + alert_user(Transport, Tracker, Socket, false, no_pid, From, Alert, Role). -alert_user(_,_, false = Active, Pid, From, Alert, Role) -> +alert_user(_, _, _, false = Active, Pid, From, Alert, Role) -> %% If there is an outstanding ssl_accept | recv %% From will be defined and send_or_reply will %% send the appropriate error message. ReasonCode = ssl_alert:reason_code(Alert, Role), send_or_reply(Active, Pid, From, {error, ReasonCode}); -alert_user(Transport, Socket, Active, Pid, From, Alert, Role) -> +alert_user(Transport, Tracker, Socket, Active, Pid, From, Alert, Role) -> case ssl_alert:reason_code(Alert, Role) of closed -> send_or_reply(Active, Pid, From, {ssl_closed, ssl_socket:socket(self(), - Transport, Socket, ?MODULE)}); + Transport, Socket, ?MODULE, Tracker)}); ReasonCode -> send_or_reply(Active, Pid, From, {ssl_error, ssl_socket:socket(self(), - Transport, Socket, ?MODULE), ReasonCode}) + Transport, Socket, ?MODULE, Tracker), ReasonCode}) end. log_alert(true, Info, Alert) -> @@ -921,15 +910,17 @@ handle_own_alert(Alert, Version, StateName, handle_normal_shutdown(Alert, _, #state{socket = Socket, transport_cb = Transport, start_or_recv_from = StartFrom, + tracker = Tracker, role = Role, renegotiation = {false, first}}) -> - alert_user(Transport, Socket, StartFrom, Alert, Role); + alert_user(Transport, Tracker,Socket, StartFrom, Alert, Role); handle_normal_shutdown(Alert, StateName, #state{socket = Socket, socket_options = Opts, transport_cb = Transport, user_application = {_Mon, Pid}, + tracker = Tracker, start_or_recv_from = RecvFrom, role = Role}) -> - alert_user(Transport, Socket, StateName, Opts, Pid, RecvFrom, Alert, Role). + alert_user(Transport, Tracker, Socket, StateName, Opts, Pid, RecvFrom, Alert, Role). handle_unexpected_message(Msg, Info, #state{negotiated_version = Version} = State) -> Alert = ?ALERT_REC(?FATAL,?UNEXPECTED_MESSAGE), @@ -970,3 +961,14 @@ workaround_transport_delivery_problems(Socket, gen_tcp = Transport) -> Transport:recv(Socket, 0, 30000); workaround_transport_delivery_problems(Socket, Transport) -> Transport:close(Socket). + +convert_state(#state{ssl_options = Options} = State, up, "5.3.5", "5.3.6") -> + State#state{ssl_options = convert_options_partial_chain(Options, up)}; +convert_state(#state{ssl_options = Options} = State, down, "5.3.6", "5.3.5") -> + State#state{ssl_options = convert_options_partial_chain(Options, down)}. + +convert_options_partial_chain(Options, up) -> + {Head, Tail} = lists:split(5, tuple_to_list(Options)), + list_to_tuple(Head ++ [{partial_chain, fun(_) -> unknown_ca end}] ++ Tail); +convert_options_partial_chain(Options, down) -> + list_to_tuple(proplists:delete(partial_chain, tuple_to_list(Options))). diff --git a/lib/ssl/src/tls_connection_sup.erl b/lib/ssl/src/tls_connection_sup.erl index 6f0d8a7262..7a637c212a 100644 --- a/lib/ssl/src/tls_connection_sup.erl +++ b/lib/ssl/src/tls_connection_sup.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2013. All Rights Reserved. +%% Copyright Ericsson AB 2007-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -59,7 +59,7 @@ init(_O) -> StartFunc = {tls_connection, start_link, []}, Restart = temporary, % E.g. should not be restarted Shutdown = 4000, - Modules = [tls_connection], + Modules = [tls_connection, ssl_connection], Type = worker, ChildSpec = {Name, StartFunc, Restart, Shutdown, Type, Modules}, diff --git a/lib/ssl/src/tls_handshake.erl b/lib/ssl/src/tls_handshake.erl index 01abefca46..183cabcfcd 100644 --- a/lib/ssl/src/tls_handshake.erl +++ b/lib/ssl/src/tls_handshake.erl @@ -33,6 +33,8 @@ -export([client_hello/8, hello/4, get_tls_handshake/3, encode_handshake/2, decode_handshake/3]). +-type tls_handshake() :: #client_hello{} | ssl_handshake:ssl_handshake(). + %%==================================================================== %% Internal application API %%==================================================================== @@ -71,11 +73,11 @@ client_hello(Host, Port, ConnectionStates, #connection_states{} | {inet:port_number(), #session{}, db_handle(), atom(), #connection_states{}, binary() | undefined}, boolean()) -> - {tls_version(), session_id(), #connection_states{}, binary() | undefined}| - {tls_version(), {resumed | new, #session{}}, #connection_states{}, - [binary()] | undefined, - [oid()] | undefined, [oid()] | undefined} | - #alert{}. + {tls_record:tls_version(), session_id(), #connection_states{}, binary() | undefined}| + {tls_record:tls_version(), {resumed | new, #session{}}, #connection_states{}, + [binary()] | undefined, + [ssl_handshake:oid()] | undefined, [ssl_handshake:oid()] | undefined} | + #alert{}. %% %% Description: Handles a recieved hello message %%-------------------------------------------------------------------- @@ -122,7 +124,7 @@ hello(#client_hello{client_version = ClientVersion, end. %%-------------------------------------------------------------------- --spec encode_handshake(tls_handshake(), tls_version()) -> iolist(). +-spec encode_handshake(tls_handshake(), tls_record:tls_version()) -> iolist(). %% %% Description: Encode a handshake packet %%--------------------------------------------------------------------x @@ -132,7 +134,7 @@ encode_handshake(Package, Version) -> [MsgType, ?uint24(Len), Bin]. %%-------------------------------------------------------------------- --spec get_tls_handshake(tls_version(), binary(), binary() | iolist()) -> +-spec get_tls_handshake(tls_record:tls_version(), binary(), binary() | iolist()) -> {[tls_handshake()], binary()}. %% %% Description: Given buffered and new data from ssl_record, collects diff --git a/lib/ssl/src/tls_handshake.hrl b/lib/ssl/src/tls_handshake.hrl index dbe930cb90..1646e5b6f2 100644 --- a/lib/ssl/src/tls_handshake.hrl +++ b/lib/ssl/src/tls_handshake.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2013-2013. All Rights Reserved. +%% Copyright Ericsson AB 2013-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -37,6 +37,4 @@ extensions }). --type tls_handshake() :: #client_hello{} | ssl_handshake(). - -endif. % -ifdef(tls_handshake). diff --git a/lib/ssl/src/tls_record.erl b/lib/ssl/src/tls_record.erl index 8c0c4f3c91..f50ea22f39 100644 --- a/lib/ssl/src/tls_record.erl +++ b/lib/ssl/src/tls_record.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2013. All Rights Reserved. +%% Copyright Ericsson AB 2007-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -44,6 +44,11 @@ highest_protocol_version/1, supported_protocol_versions/0, is_acceptable_version/1, is_acceptable_version/2]). +-export_type([tls_version/0, tls_atom_version/0]). + +-type tls_version() :: ssl_record:ssl_version(). +-type tls_atom_version() :: sslv3 | tlsv1 | 'tlsv1.1' | 'tlsv1.2'. + -compile(inline). %%==================================================================== @@ -149,21 +154,24 @@ decode_cipher_text(#ssl_tls{type = Type, version = Version, sequence_number = Seq, security_parameters = SecParams} = ReadState0, CompressAlg = SecParams#security_parameters.compression_algorithm, - {PlainFragment, Mac, ReadState1} = ssl_record:decipher(Version, CipherFragment, ReadState0), - MacHash = calc_mac_hash(Type, Version, PlainFragment, ReadState1), - case ssl_record:is_correct_mac(Mac, MacHash) of - true -> - {Plain, CompressionS1} = ssl_record:uncompress(CompressAlg, - PlainFragment, CompressionS0), - ConnnectionStates = ConnnectionStates0#connection_states{ - current_read = ReadState1#connection_state{ - sequence_number = Seq + 1, - compression_state = CompressionS1}}, - {CipherText#ssl_tls{fragment = Plain}, ConnnectionStates}; - false -> - ?ALERT_REC(?FATAL, ?BAD_RECORD_MAC) - end. - + case ssl_record:decipher(Version, CipherFragment, ReadState0) of + {PlainFragment, Mac, ReadState1} -> + MacHash = calc_mac_hash(Type, Version, PlainFragment, ReadState1), + case ssl_record:is_correct_mac(Mac, MacHash) of + true -> + {Plain, CompressionS1} = ssl_record:uncompress(CompressAlg, + PlainFragment, CompressionS0), + ConnnectionStates = ConnnectionStates0#connection_states{ + current_read = ReadState1#connection_state{ + sequence_number = Seq + 1, + compression_state = CompressionS1}}, + {CipherText#ssl_tls{fragment = Plain}, ConnnectionStates}; + false -> + ?ALERT_REC(?FATAL, ?BAD_RECORD_MAC) + end; + #alert{} = Alert -> + Alert + end. %%-------------------------------------------------------------------- -spec protocol_version(tls_atom_version() | tls_version()) -> tls_version() | tls_atom_version(). diff --git a/lib/ssl/src/tls_v1.erl b/lib/ssl/src/tls_v1.erl index 7b1f53b969..7a5f9c1b38 100644 --- a/lib/ssl/src/tls_v1.erl +++ b/lib/ssl/src/tls_v1.erl @@ -166,7 +166,7 @@ setup_keys(Version, PrfAlgo, MasterSecret, ServerRandom, ClientRandom, HashSize, {ClientWriteMacSecret, ServerWriteMacSecret, ClientWriteKey, ServerWriteKey, ClientIV, ServerIV}. --spec mac_hash(integer(), binary(), integer(), integer(), tls_version(), +-spec mac_hash(integer(), binary(), integer(), integer(), tls_record:tls_version(), integer(), binary()) -> binary(). mac_hash(Method, Mac_write_secret, Seq_num, Type, {Major, Minor}, @@ -181,25 +181,9 @@ mac_hash(Method, Mac_write_secret, Seq_num, Type, {Major, Minor}, Fragment]), Mac. --spec suites(1|2|3) -> [cipher_suite()]. - -suites(Minor) when Minor == 1; Minor == 2-> - case sufficent_ec_support() of - true -> - all_suites(Minor); - false -> - no_ec_suites(Minor) - end; - -suites(Minor) when Minor == 3 -> - case sufficent_ec_support() of - true -> - all_suites(3) ++ all_suites(2); - false -> - no_ec_suites(3) ++ no_ec_suites(2) - end. - -all_suites(Minor) when Minor == 1; Minor == 2-> +-spec suites(1|2|3) -> [ssl_cipher:cipher_suite()]. + +suites(Minor) when Minor == 1; Minor == 2 -> [ ?TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, ?TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, @@ -235,7 +219,7 @@ all_suites(Minor) when Minor == 1; Minor == 2-> ?TLS_RSA_WITH_DES_CBC_SHA ]; -all_suites(3) -> +suites(3) -> [ ?TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, ?TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, @@ -254,33 +238,7 @@ all_suites(3) -> ?TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, ?TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, ?TLS_RSA_WITH_AES_128_CBC_SHA256 - ]. - -no_ec_suites(Minor) when Minor == 1; Minor == 2-> - [ - ?TLS_DHE_RSA_WITH_AES_256_CBC_SHA, - ?TLS_DHE_DSS_WITH_AES_256_CBC_SHA, - ?TLS_RSA_WITH_AES_256_CBC_SHA, - ?TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA, - ?TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA, - ?TLS_RSA_WITH_3DES_EDE_CBC_SHA, - ?TLS_DHE_RSA_WITH_AES_128_CBC_SHA, - ?TLS_DHE_DSS_WITH_AES_128_CBC_SHA, - ?TLS_RSA_WITH_AES_128_CBC_SHA, - ?TLS_RSA_WITH_RC4_128_SHA, - ?TLS_RSA_WITH_RC4_128_MD5, - ?TLS_DHE_RSA_WITH_DES_CBC_SHA, - ?TLS_RSA_WITH_DES_CBC_SHA - ]; -no_ec_suites(3) -> - [ - ?TLS_DHE_RSA_WITH_AES_256_CBC_SHA256, - ?TLS_DHE_DSS_WITH_AES_256_CBC_SHA256, - ?TLS_RSA_WITH_AES_256_CBC_SHA256, - ?TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, - ?TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, - ?TLS_RSA_WITH_AES_128_CBC_SHA256 - ]. + ] ++ suites(2). %%-------------------------------------------------------------------- %%% Internal functions @@ -442,7 +400,3 @@ enum_to_oid(27) -> ?brainpoolP384r1; enum_to_oid(28) -> ?brainpoolP512r1; enum_to_oid(_) -> undefined. - -sufficent_ec_support() -> - CryptoSupport = crypto:supports(), - proplists:get_bool(ecdh, proplists:get_value(public_keys, CryptoSupport)). diff --git a/lib/ssl/test/Makefile b/lib/ssl/test/Makefile index 244eb5ce0a..2f8ff6f04e 100644 --- a/lib/ssl/test/Makefile +++ b/lib/ssl/test/Makefile @@ -39,6 +39,7 @@ MODULES = \ ssl_basic_SUITE \ ssl_cipher_SUITE \ ssl_certificate_verify_SUITE\ + ssl_crl_SUITE\ ssl_dist_SUITE \ ssl_handshake_SUITE \ ssl_npn_hello_SUITE \ diff --git a/lib/ssl/test/erl_make_certs.erl b/lib/ssl/test/erl_make_certs.erl index 22dc951ac1..daf4466f11 100644 --- a/lib/ssl/test/erl_make_certs.erl +++ b/lib/ssl/test/erl_make_certs.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2011-2013. All Rights Reserved. +%% Copyright Ericsson AB 2011-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -325,14 +325,14 @@ sign_algorithm(#'RSAPrivateKey'{}, Opts) -> {Type, 'NULL'}; sign_algorithm(#'DSAPrivateKey'{p=P, q=Q, g=G}, _Opts) -> {?'id-dsa-with-sha1', {params,#'Dss-Parms'{p=P, q=Q, g=G}}}; -sign_algorithm(#'ECPrivateKey'{}, Opts) -> +sign_algorithm(#'ECPrivateKey'{parameters = Parms}, Opts) -> Type = case proplists:get_value(digest, Opts, sha1) of sha1 -> ?'ecdsa-with-SHA1'; sha512 -> ?'ecdsa-with-SHA512'; sha384 -> ?'ecdsa-with-SHA384'; sha256 -> ?'ecdsa-with-SHA256' end, - {Type, 'NULL'}. + {Type, Parms}. make_key(rsa, _Opts) -> %% (OBS: for testing only) diff --git a/lib/ssl/test/make_certs.erl b/lib/ssl/test/make_certs.erl index 4603a9f846..15a7e118ff 100644 --- a/lib/ssl/test/make_certs.erl +++ b/lib/ssl/test/make_certs.erl @@ -18,23 +18,71 @@ %% -module(make_certs). +-compile([export_all]). --export([all/2]). +%-export([all/1, all/2, rootCA/2, intermediateCA/3, endusers/3, enduser/3, revoke/3, gencrl/2, verify/3]). --record(dn, {commonName, +-record(config, {commonName, organizationalUnitName = "Erlang OTP", organizationName = "Ericsson AB", localityName = "Stockholm", countryName = "SE", - emailAddress = "[email protected]"}). + emailAddress = "[email protected]", + default_bits = 2048, + v2_crls = true, + ecc_certs = false, + issuing_distribution_point = false, + crl_port = 8000, + openssl_cmd = "openssl"}). + + +default_config() -> + #config{}. + +make_config(Args) -> + make_config(Args, #config{}). + +make_config([], C) -> + C; +make_config([{organizationalUnitName, Name}|T], C) when is_list(Name) -> + make_config(T, C#config{organizationalUnitName = Name}); +make_config([{organizationName, Name}|T], C) when is_list(Name) -> + make_config(T, C#config{organizationName = Name}); +make_config([{localityName, Name}|T], C) when is_list(Name) -> + make_config(T, C#config{localityName = Name}); +make_config([{countryName, Name}|T], C) when is_list(Name) -> + make_config(T, C#config{countryName = Name}); +make_config([{emailAddress, Name}|T], C) when is_list(Name) -> + make_config(T, C#config{emailAddress = Name}); +make_config([{default_bits, Bits}|T], C) when is_integer(Bits) -> + make_config(T, C#config{default_bits = Bits}); +make_config([{v2_crls, Bool}|T], C) when is_boolean(Bool) -> + make_config(T, C#config{v2_crls = Bool}); +make_config([{crl_port, Port}|T], C) when is_integer(Port) -> + make_config(T, C#config{crl_port = Port}); +make_config([{ecc_certs, Bool}|T], C) when is_boolean(Bool) -> + make_config(T, C#config{ecc_certs = Bool}); +make_config([{issuing_distribution_point, Bool}|T], C) when is_boolean(Bool) -> + make_config(T, C#config{issuing_distribution_point = Bool}); +make_config([{openssl_cmd, Cmd}|T], C) when is_list(Cmd) -> + make_config(T, C#config{openssl_cmd = Cmd}). + + +all([DataDir, PrivDir]) -> + all(DataDir, PrivDir). all(DataDir, PrivDir) -> - OpenSSLCmd = "openssl", + all(DataDir, PrivDir, #config{}). + +all(DataDir, PrivDir, C) when is_list(C) -> + all(DataDir, PrivDir, make_config(C)); +all(DataDir, PrivDir, C = #config{}) -> + ok = filelib:ensure_dir(filename:join(PrivDir, "erlangCA")), create_rnd(DataDir, PrivDir), % For all requests - rootCA(PrivDir, OpenSSLCmd, "erlangCA"), - intermediateCA(PrivDir, OpenSSLCmd, "otpCA", "erlangCA"), - endusers(PrivDir, OpenSSLCmd, "otpCA", ["client", "server"]), - collect_certs(PrivDir, ["erlangCA", "otpCA"], ["client", "server"]), + rootCA(PrivDir, "erlangCA", C), + intermediateCA(PrivDir, "otpCA", "erlangCA", C), + endusers(PrivDir, "otpCA", ["client", "server", "revoked"], C), + endusers(PrivDir, "erlangCA", ["localhost"], C), %% Create keycert files SDir = filename:join([PrivDir, "server"]), SC = filename:join([SDir, "cert.pem"]), @@ -46,7 +94,14 @@ all(DataDir, PrivDir) -> CK = filename:join([CDir, "key.pem"]), CKC = filename:join([CDir, "keycert.pem"]), append_files([CK, CC], CKC), - remove_rnd(PrivDir). + RDir = filename:join([PrivDir, "revoked"]), + RC = filename:join([RDir, "cert.pem"]), + RK = filename:join([RDir, "key.pem"]), + RKC = filename:join([RDir, "keycert.pem"]), + revoke(PrivDir, "otpCA", "revoked", C), + append_files([RK, RC], RKC), + remove_rnd(PrivDir), + {ok, C}. append_files(FileNames, ResultFileName) -> {ok, ResultFile} = file:open(ResultFileName, [write]), @@ -59,111 +114,176 @@ do_append_files([F|Fs], RF) -> ok = file:write(RF, Data), do_append_files(Fs, RF). -rootCA(Root, OpenSSLCmd, Name) -> - create_ca_dir(Root, Name, ca_cnf(Name)), - DN = #dn{commonName = Name}, - create_self_signed_cert(Root, OpenSSLCmd, Name, req_cnf(DN)), - ok. +rootCA(Root, Name, C) -> + create_ca_dir(Root, Name, ca_cnf(C#config{commonName = Name})), + create_self_signed_cert(Root, Name, req_cnf(C#config{commonName = Name}), C), + file:copy(filename:join([Root, Name, "cert.pem"]), filename:join([Root, Name, "cacerts.pem"])), + gencrl(Root, Name, C). -intermediateCA(Root, OpenSSLCmd, CA, ParentCA) -> - CA = "otpCA", - create_ca_dir(Root, CA, ca_cnf(CA)), +intermediateCA(Root, CA, ParentCA, C) -> + create_ca_dir(Root, CA, ca_cnf(C#config{commonName = CA})), CARoot = filename:join([Root, CA]), - DN = #dn{commonName = CA}, CnfFile = filename:join([CARoot, "req.cnf"]), - file:write_file(CnfFile, req_cnf(DN)), + file:write_file(CnfFile, req_cnf(C#config{commonName = CA})), KeyFile = filename:join([CARoot, "private", "key.pem"]), ReqFile = filename:join([CARoot, "req.pem"]), - create_req(Root, OpenSSLCmd, CnfFile, KeyFile, ReqFile), + create_req(Root, CnfFile, KeyFile, ReqFile, C), CertFile = filename:join([CARoot, "cert.pem"]), - sign_req(Root, OpenSSLCmd, ParentCA, "ca_cert", ReqFile, CertFile). - -endusers(Root, OpenSSLCmd, CA, Users) -> - lists:foreach(fun(User) -> enduser(Root, OpenSSLCmd, CA, User) end, Users). - -enduser(Root, OpenSSLCmd, CA, User) -> + sign_req(Root, ParentCA, "ca_cert", ReqFile, CertFile, C), + CACertsFile = filename:join(CARoot, "cacerts.pem"), + file:copy(filename:join([Root, ParentCA, "cacerts.pem"]), CACertsFile), + %% append this CA's cert to the cacerts file + {ok, Bin} = file:read_file(CertFile), + {ok, FD} = file:open(CACertsFile, [append]), + file:write(FD, ["\n", Bin]), + file:close(FD), + gencrl(Root, CA, C). + +endusers(Root, CA, Users, C) -> + [enduser(Root, CA, User, C) || User <- Users]. + +enduser(Root, CA, User, C) -> UsrRoot = filename:join([Root, User]), file:make_dir(UsrRoot), CnfFile = filename:join([UsrRoot, "req.cnf"]), - DN = #dn{commonName = User}, - file:write_file(CnfFile, req_cnf(DN)), + file:write_file(CnfFile, req_cnf(C#config{commonName = User})), KeyFile = filename:join([UsrRoot, "key.pem"]), ReqFile = filename:join([UsrRoot, "req.pem"]), - create_req(Root, OpenSSLCmd, CnfFile, KeyFile, ReqFile), + create_req(Root, CnfFile, KeyFile, ReqFile, C), + %create_req(Root, CnfFile, KeyFile, ReqFile), CertFileAllUsage = filename:join([UsrRoot, "cert.pem"]), - sign_req(Root, OpenSSLCmd, CA, "user_cert", ReqFile, CertFileAllUsage), + sign_req(Root, CA, "user_cert", ReqFile, CertFileAllUsage, C), CertFileDigitalSigOnly = filename:join([UsrRoot, "digital_signature_only_cert.pem"]), - sign_req(Root, OpenSSLCmd, CA, "user_cert_digital_signature_only", ReqFile, CertFileDigitalSigOnly). - -collect_certs(Root, CAs, Users) -> - Bins = lists:foldr( - fun(CA, Acc) -> - File = filename:join([Root, CA, "cert.pem"]), - {ok, Bin} = file:read_file(File), - [Bin, "\n" | Acc] - end, [], CAs), - lists:foreach( - fun(User) -> - File = filename:join([Root, User, "cacerts.pem"]), - file:write_file(File, Bins) - end, Users). + sign_req(Root, CA, "user_cert_digital_signature_only", ReqFile, CertFileDigitalSigOnly, C), + CACertsFile = filename:join(UsrRoot, "cacerts.pem"), + file:copy(filename:join([Root, CA, "cacerts.pem"]), CACertsFile), + ok. + +revoke(Root, CA, User, C) -> + UsrCert = filename:join([Root, User, "cert.pem"]), + CACnfFile = filename:join([Root, CA, "ca.cnf"]), + Cmd = [C#config.openssl_cmd, " ca" + " -revoke ", UsrCert, + [" -crl_reason keyCompromise" || C#config.v2_crls ], + " -config ", CACnfFile], + Env = [{"ROOTDIR", filename:absname(Root)}], + cmd(Cmd, Env), + gencrl(Root, CA, C). + +gencrl(Root, CA, C) -> + CACnfFile = filename:join([Root, CA, "ca.cnf"]), + CACRLFile = filename:join([Root, CA, "crl.pem"]), + Cmd = [C#config.openssl_cmd, " ca" + " -gencrl ", + " -crlhours 24", + " -out ", CACRLFile, + " -config ", CACnfFile], + Env = [{"ROOTDIR", filename:absname(Root)}], + cmd(Cmd, Env). -create_self_signed_cert(Root, OpenSSLCmd, CAName, Cnf) -> +verify(Root, CA, User, C) -> + CAFile = filename:join([Root, User, "cacerts.pem"]), + CACRLFile = filename:join([Root, CA, "crl.pem"]), + CertFile = filename:join([Root, User, "cert.pem"]), + Cmd = [C#config.openssl_cmd, " verify" + " -CAfile ", CAFile, + " -CRLfile ", CACRLFile, %% this is undocumented, but seems to work + " -crl_check ", + CertFile], + Env = [{"ROOTDIR", filename:absname(Root)}], + try cmd(Cmd, Env) catch + exit:{eval_cmd, _, _} -> + invalid + end. + +create_self_signed_cert(Root, CAName, Cnf, C = #config{ecc_certs = true}) -> CARoot = filename:join([Root, CAName]), CnfFile = filename:join([CARoot, "req.cnf"]), file:write_file(CnfFile, Cnf), KeyFile = filename:join([CARoot, "private", "key.pem"]), CertFile = filename:join([CARoot, "cert.pem"]), - Cmd = [OpenSSLCmd, " req" + Cmd = [C#config.openssl_cmd, " ecparam" + " -out ", KeyFile, + " -name secp521r1 ", + %" -name sect283k1 ", + " -genkey "], + Env = [{"ROOTDIR", filename:absname(Root)}], + cmd(Cmd, Env), + + Cmd2 = [C#config.openssl_cmd, " req" " -new" " -x509" " -config ", CnfFile, - " -keyout ", KeyFile, + " -key ", KeyFile, + " -outform PEM ", " -out ", CertFile], - Env = [{"ROOTDIR", Root}], - cmd(Cmd, Env), - fix_key_file(OpenSSLCmd, KeyFile). - -% openssl 1.0 generates key files in pkcs8 format by default and we don't handle this format -fix_key_file(OpenSSLCmd, KeyFile) -> - KeyFileTmp = KeyFile ++ ".tmp", - Cmd = [OpenSSLCmd, " rsa", - " -in ", - KeyFile, - " -out ", - KeyFileTmp], - cmd(Cmd, []), - ok = file:rename(KeyFileTmp, KeyFile). + cmd(Cmd2, Env); +create_self_signed_cert(Root, CAName, Cnf, C) -> + CARoot = filename:join([Root, CAName]), + CnfFile = filename:join([CARoot, "req.cnf"]), + file:write_file(CnfFile, Cnf), + KeyFile = filename:join([CARoot, "private", "key.pem"]), + CertFile = filename:join([CARoot, "cert.pem"]), + Cmd = [C#config.openssl_cmd, " req" + " -new" + " -x509" + " -config ", CnfFile, + " -keyout ", KeyFile, + " -outform PEM", + " -out ", CertFile], + Env = [{"ROOTDIR", filename:absname(Root)}], + cmd(Cmd, Env). + create_ca_dir(Root, CAName, Cnf) -> CARoot = filename:join([Root, CAName]), + ok = filelib:ensure_dir(CARoot), file:make_dir(CARoot), create_dirs(CARoot, ["certs", "crl", "newcerts", "private"]), create_rnd(Root, filename:join([CAName, "private"])), create_files(CARoot, [{"serial", "01\n"}, + {"crlnumber", "01"}, {"index.txt", ""}, {"ca.cnf", Cnf}]). -create_req(Root, OpenSSLCmd, CnfFile, KeyFile, ReqFile) -> - Cmd = [OpenSSLCmd, " req" +create_req(Root, CnfFile, KeyFile, ReqFile, C = #config{ecc_certs = true}) -> + Cmd = [C#config.openssl_cmd, " ecparam" + " -out ", KeyFile, + " -name secp521r1 ", + %" -name sect283k1 ", + " -genkey "], + Env = [{"ROOTDIR", filename:absname(Root)}], + cmd(Cmd, Env), + Cmd2 = [C#config.openssl_cmd, " req" + " -new ", + " -key ", KeyFile, + " -outform PEM ", + " -out ", ReqFile, + " -config ", CnfFile], + cmd(Cmd2, Env); + %fix_key_file(KeyFile). +create_req(Root, CnfFile, KeyFile, ReqFile, C) -> + Cmd = [C#config.openssl_cmd, " req" " -new" " -config ", CnfFile, + " -outform PEM ", " -keyout ", KeyFile, " -out ", ReqFile], - Env = [{"ROOTDIR", Root}], - cmd(Cmd, Env), - fix_key_file(OpenSSLCmd, KeyFile). + Env = [{"ROOTDIR", filename:absname(Root)}], + cmd(Cmd, Env). + %fix_key_file(KeyFile). + -sign_req(Root, OpenSSLCmd, CA, CertType, ReqFile, CertFile) -> +sign_req(Root, CA, CertType, ReqFile, CertFile, C) -> CACnfFile = filename:join([Root, CA, "ca.cnf"]), - Cmd = [OpenSSLCmd, " ca" + Cmd = [C#config.openssl_cmd, " ca" " -batch" " -notext" " -config ", CACnfFile, " -extensions ", CertType, " -in ", ReqFile, " -out ", CertFile], - Env = [{"ROOTDIR", Root}], + Env = [{"ROOTDIR", filename:absname(Root)}], cmd(Cmd, Env). %% @@ -194,19 +314,19 @@ cmd(Cmd, Env) -> FCmd = lists:flatten(Cmd), Port = open_port({spawn, FCmd}, [stream, eof, exit_status, stderr_to_stdout, {env, Env}]), - eval_cmd(Port). + eval_cmd(Port, FCmd). -eval_cmd(Port) -> +eval_cmd(Port, Cmd) -> receive {Port, {data, _}} -> - eval_cmd(Port); + eval_cmd(Port, Cmd); {Port, eof} -> ok end, receive {Port, {exit_status, Status}} when Status /= 0 -> %% io:fwrite("exit status: ~w~n", [Status]), - exit({eval_cmd, Status}) + exit({eval_cmd, Cmd, Status}) after 0 -> ok end. @@ -215,7 +335,7 @@ eval_cmd(Port) -> %% Contents of configuration files %% -req_cnf(DN) -> +req_cnf(C) -> ["# Purpose: Configuration for requests (end users and CAs)." "\n" "ROOTDIR = $ENV::ROOTDIR\n" @@ -224,10 +344,10 @@ req_cnf(DN) -> "[req]\n" "input_password = secret\n" "output_password = secret\n" - "default_bits = 1024\n" + "default_bits = ", integer_to_list(C#config.default_bits), "\n" "RANDFILE = $ROOTDIR/RAND\n" "encrypt_key = no\n" - "default_md = sha1\n" + "default_md = md5\n" "#string_mask = pkix\n" "x509_extensions = ca_ext\n" "prompt = no\n" @@ -235,12 +355,12 @@ req_cnf(DN) -> "\n" "[name]\n" - "commonName = ", DN#dn.commonName, "\n" - "organizationalUnitName = ", DN#dn.organizationalUnitName, "\n" - "organizationName = ", DN#dn.organizationName, "\n" - "localityName = ", DN#dn.localityName, "\n" - "countryName = ", DN#dn.countryName, "\n" - "emailAddress = ", DN#dn.emailAddress, "\n" + "commonName = ", C#config.commonName, "\n" + "organizationalUnitName = ", C#config.organizationalUnitName, "\n" + "organizationName = ", C#config.organizationName, "\n" + "localityName = ", C#config.localityName, "\n" + "countryName = ", C#config.countryName, "\n" + "emailAddress = ", C#config.emailAddress, "\n" "\n" "[ca_ext]\n" @@ -249,8 +369,7 @@ req_cnf(DN) -> "subjectKeyIdentifier = hash\n" "subjectAltName = email:copy\n"]. - -ca_cnf(CA) -> +ca_cnf(C) -> ["# Purpose: Configuration for CAs.\n" "\n" "ROOTDIR = $ENV::ROOTDIR\n" @@ -258,21 +377,23 @@ ca_cnf(CA) -> "\n" "[ca]\n" - "dir = $ROOTDIR/", CA, "\n" + "dir = $ROOTDIR/", C#config.commonName, "\n" "certs = $dir/certs\n" "crl_dir = $dir/crl\n" "database = $dir/index.txt\n" "new_certs_dir = $dir/newcerts\n" "certificate = $dir/cert.pem\n" "serial = $dir/serial\n" - "crl = $dir/crl.pem\n" + "crl = $dir/crl.pem\n", + ["crlnumber = $dir/crlnumber\n" || C#config.v2_crls], "private_key = $dir/private/key.pem\n" "RANDFILE = $dir/private/RAND\n" "\n" - "x509_extensions = user_cert\n" + "x509_extensions = user_cert\n", + ["crl_extensions = crl_ext\n" || C#config.v2_crls], "unique_subject = no\n" "default_days = 3600\n" - "default_md = sha1\n" + "default_md = md5\n" "preserve = no\n" "policy = policy_match\n" "\n" @@ -286,6 +407,13 @@ ca_cnf(CA) -> "emailAddress = supplied\n" "\n" + "[crl_ext]\n" + "authorityKeyIdentifier=keyid:always,issuer:always\n", + ["issuingDistributionPoint=critical, @idpsec\n" || C#config.issuing_distribution_point], + + "[idpsec]\n" + "fullname=URI:http://localhost:8000/",C#config.commonName,"/crl.pem\n" + "[user_cert]\n" "basicConstraints = CA:false\n" "keyUsage = nonRepudiation, digitalSignature, keyEncipherment\n" @@ -293,6 +421,12 @@ ca_cnf(CA) -> "authorityKeyIdentifier = keyid,issuer:always\n" "subjectAltName = email:copy\n" "issuerAltName = issuer:copy\n" + "crlDistributionPoints=@crl_section\n" + + "[crl_section]\n" + %% intentionally invalid + "URI.1=http://localhost/",C#config.commonName,"/crl.pem\n" + "URI.2=http://localhost:",integer_to_list(C#config.crl_port),"/",C#config.commonName,"/crl.pem\n" "\n" "[user_cert_digital_signature_only]\n" @@ -310,4 +444,7 @@ ca_cnf(CA) -> "subjectKeyIdentifier = hash\n" "authorityKeyIdentifier = keyid:always,issuer:always\n" "subjectAltName = email:copy\n" - "issuerAltName = issuer:copy\n"]. + "issuerAltName = issuer:copy\n" + "crlDistributionPoints=@crl_section\n" + ]. + diff --git a/lib/ssl/test/ssl_ECC_SUITE.erl b/lib/ssl/test/ssl_ECC_SUITE.erl index 608f2f11c3..3566a8a0a5 100644 --- a/lib/ssl/test/ssl_ECC_SUITE.erl +++ b/lib/ssl/test/ssl_ECC_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2013. All Rights Reserved. +%% Copyright Ericsson AB 2007-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -57,41 +57,51 @@ all_versions_groups ()-> ]. key_cert_combinations() -> - [client_ec_server_ec, - client_rsa_server_ec, - client_ec_server_rsa, - client_rsa_server_rsa]. + [client_ecdh_server_ecdh, + client_rsa_server_ecdh, + client_ecdh_server_rsa, + client_rsa_server_rsa, + client_ecdsa_server_ecdsa, + client_ecdsa_server_rsa, + client_rsa_server_ecdsa + ]. %%-------------------------------------------------------------------- -init_per_suite(Config) -> - catch crypto:stop(), +init_per_suite(Config0) -> + end_per_suite(Config0), try crypto:start() of ok -> - ssl:start(), - Config + %% make rsa certs using oppenssl + Result = + (catch make_certs:all(?config(data_dir, Config0), + ?config(priv_dir, Config0))), + ct:log("Make certs ~p~n", [Result]), + Config1 = ssl_test_lib:make_ecdsa_cert(Config0), + Config2 = ssl_test_lib:make_ecdh_rsa_cert(Config1), + ssl_test_lib:cert_options(Config2) catch _:_ -> {skip, "Crypto did not start"} end. end_per_suite(_Config) -> - ssl:stop(), + application:stop(ssl), application:stop(crypto). %%-------------------------------------------------------------------- -init_per_group(erlang_client, Config) -> +init_per_group(erlang_client = Group, Config) -> case ssl_test_lib:is_sane_ecc(openssl) of true -> - common_init_per_group(erlang_client, [{server_type, openssl}, - {client_type, erlang} | Config]); + common_init_per_group(Group, [{server_type, openssl}, + {client_type, erlang} | Config]); false -> {skip, "Known ECC bug in openssl"} end; -init_per_group(erlang_server, Config) -> +init_per_group(erlang_server = Group, Config) -> case ssl_test_lib:is_sane_ecc(openssl) of true -> - common_init_per_group(erlang_client, [{server_type, erlang}, - {client_type, openssl} | Config]); + common_init_per_group(Group, [{server_type, erlang}, + {client_type, openssl} | Config]); false -> {skip, "Known ECC bug in openssl"} end; @@ -99,11 +109,21 @@ init_per_group(erlang_server, Config) -> init_per_group(erlang = Group, Config) -> case ssl_test_lib:sufficient_crypto_support(Group) of true -> - common_init_per_group(erlang, [{server_type, erlang}, - {client_type, erlang} | Config]); + common_init_per_group(Group, [{server_type, erlang}, + {client_type, erlang} | Config]); + false -> + {skip, "Crypto does not support ECC"} + end; + +init_per_group(openssl = Group, Config) -> + case ssl_test_lib:sufficient_crypto_support(Group) of + true -> + common_init_per_group(Group, [{server_type, openssl}, + {client_type, openssl} | Config]); false -> {skip, "Crypto does not support ECC"} - end; + end; + init_per_group(Group, Config) -> common_init_per_group(Group, Config). @@ -121,76 +141,118 @@ end_per_group(_GroupName, Config) -> %%-------------------------------------------------------------------- -init_per_testcase(_TestCase, Config) -> +init_per_testcase(TestCase, Config) -> ct:log("TLS/SSL version ~p~n ", [tls_record:supported_protocol_versions()]), ct:log("Ciphers: ~p~n ", [ ssl:cipher_suites()]), + end_per_testcase(TestCase, Config), + ssl:start(), Config. -end_per_testcase(_TestCase, Config) -> +end_per_testcase(_TestCase, Config) -> + application:stop(ssl), Config. %%-------------------------------------------------------------------- %% Test Cases -------------------------------------------------------- %%-------------------------------------------------------------------- -client_ec_server_ec(Config) when is_list(Config) -> - basic_test("ec1.crt", "ec1.key", "ec2.crt", "ec2.key", Config). - -client_ec_server_rsa(Config) when is_list(Config) -> - basic_test("ec1.crt", "ec1.key", "rsa1.crt", "rsa1.key", Config). +client_ecdh_server_ecdh(Config) when is_list(Config) -> + COpts = ?config(client_ecdh_rsa_opts, Config), + SOpts = ?config(server_ecdh_rsa_verify_opts, Config), + basic_test(COpts, SOpts, Config). + +client_ecdh_server_rsa(Config) when is_list(Config) -> + COpts = ?config(client_ecdh_rsa_opts, Config), + SOpts = ?config(server_ecdh_rsa_verify_opts, Config), + basic_test(COpts, SOpts, Config). + +client_rsa_server_ecdh(Config) when is_list(Config) -> + COpts = ?config(client_ecdh_rsa_opts, Config), + SOpts = ?config(server_ecdh_rsa_verify_opts, Config), + basic_test(COpts, SOpts, Config). + +client_rsa_server_rsa(Config) when is_list(Config) -> + COpts = ?config(client_verification_opts, Config), + SOpts = ?config(server_verification_opts, Config), + basic_test(COpts, SOpts, Config). + +client_ecdsa_server_ecdsa(Config) when is_list(Config) -> + COpts = ?config(client_ecdsa_opts, Config), + SOpts = ?config(server_ecdsa_verify_opts, Config), + basic_test(COpts, SOpts, Config). -client_rsa_server_ec(Config) when is_list(Config) -> - basic_test("rsa1.crt", "rsa1.key", "ec2.crt", "ec2.key", Config). +client_ecdsa_server_rsa(Config) when is_list(Config) -> + COpts = ?config(client_ecdsa_opts, Config), + SOpts = ?config(server_ecdsa_verify_opts, Config), + basic_test(COpts, SOpts, Config). -client_rsa_server_rsa(Config) when is_list(Config) -> - basic_test("rsa1.crt", "rsa1.key", "rsa2.crt", "rsa2.key", Config). +client_rsa_server_ecdsa(Config) when is_list(Config) -> + COpts = ?config(client_ecdsa_opts, Config), + SOpts = ?config(server_ecdsa_verify_opts, Config), + basic_test(COpts, SOpts, Config). %%-------------------------------------------------------------------- %% Internal functions ------------------------------------------------ %%-------------------------------------------------------------------- -basic_test(ClientCert, ClientKey, ServerCert, ServerKey, Config) -> - DataDir = ?config(data_dir, Config), +basic_test(COpts, SOpts, Config) -> + basic_test(proplists:get_value(certfile, COpts), + proplists:get_value(keyfile, COpts), + proplists:get_value(cacertfile, COpts), + proplists:get_value(certfile, SOpts), + proplists:get_value(keyfile, SOpts), + proplists:get_value(cacertfile, SOpts), + Config). + +basic_test(ClientCert, ClientKey, ClientCA, ServerCert, ServerKey, ServerCA, Config) -> SType = ?config(server_type, Config), CType = ?config(client_type, Config), {Server, Port} = start_server(SType, - filename:join(DataDir, "CA.pem"), - filename:join(DataDir, ServerCert), - filename:join(DataDir, ServerKey), + ClientCA, ServerCA, + ServerCert, + ServerKey, Config), - Client = start_client(CType, Port, filename:join(DataDir, "CA.pem"), - filename:join(DataDir, ClientCert), - filename:join(DataDir, ClientKey), Config), - check_result(Server, SType, Client, CType). + Client = start_client(CType, Port, ServerCA, ClientCA, + ClientCert, + ClientKey, Config), + check_result(Server, SType, Client, CType), + close(Server, Client). -start_client(openssl, Port, CA, Cert, Key, _) -> +start_client(openssl, Port, CA, OwnCa, Cert, Key, Config) -> + PrivDir = ?config(priv_dir, Config), + NewCA = new_ca(filename:join(PrivDir, "new_ca.pem"), CA, OwnCa), Version = tls_record:protocol_version(tls_record:highest_protocol_version([])), - Cmd = "openssl s_client -port " ++ integer_to_list(Port) ++ ssl_test_lib:version_flag(Version) ++ - " -cert " ++ Cert ++ " -CAfile " ++ CA - ++ " -key " ++ Key ++ " -host localhost -msg", + Cmd = "openssl s_client -verify 2 -port " ++ integer_to_list(Port) ++ ssl_test_lib:version_flag(Version) ++ + " -cert " ++ Cert ++ " -CAfile " ++ NewCA + ++ " -key " ++ Key ++ " -host localhost -msg -debug", OpenSslPort = open_port({spawn, Cmd}, [stderr_to_stdout]), true = port_command(OpenSslPort, "Hello world"), OpenSslPort; -start_client(erlang, Port, CA, Cert, Key, Config) -> +start_client(erlang, Port, CA, _, Cert, Key, Config) -> {ClientNode, _, Hostname} = ssl_test_lib:run_where(Config), ssl_test_lib:start_client([{node, ClientNode}, {port, Port}, {host, Hostname}, {from, self()}, {mfa, {ssl_test_lib, send_recv_result_active, []}}, - {options, [{verify, verify_peer}, {cacertfile, CA}, + {options, [{verify, verify_peer}, + {cacertfile, CA}, {certfile, Cert}, {keyfile, Key}]}]). -start_server(openssl, CA, Cert, Key, _) -> +start_server(openssl, CA, OwnCa, Cert, Key, Config) -> + PrivDir = ?config(priv_dir, Config), + NewCA = new_ca(filename:join(PrivDir, "new_ca.pem"), CA, OwnCa), + Port = ssl_test_lib:inet_port(node()), Version = tls_record:protocol_version(tls_record:highest_protocol_version([])), Cmd = "openssl s_server -accept " ++ integer_to_list(Port) ++ ssl_test_lib:version_flag(Version) ++ - " -cert " ++ Cert ++ " -CAfile " ++ CA - ++ " -key " ++ Key ++ " -Verify 2 -msg", + " -verify 2 -cert " ++ Cert ++ " -CAfile " ++ NewCA + ++ " -key " ++ Key ++ " -msg -debug", OpenSslPort = open_port({spawn, Cmd}, [stderr_to_stdout]), ssl_test_lib:wait_for_openssl_server(), true = port_command(OpenSslPort, "Hello world"), {OpenSslPort, Port}; -start_server(erlang, CA, Cert, Key, Config) -> +start_server(erlang, CA, _, Cert, Key, Config) -> + {_, ServerNode, _} = ssl_test_lib:run_where(Config), Server = ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, {from, self()}, @@ -217,9 +279,31 @@ openssl_check(_, Config) -> TLSVersion = ?config(tls_version, Config), case ssl_test_lib:check_sane_openssl_version(TLSVersion) of true -> - ssl:start(), Config; false -> {skip, "TLS version not supported by openssl"} end. +close(Port1, Port2) when is_port(Port1), is_port(Port2) -> + ssl_test_lib:close_port(Port1), + ssl_test_lib:close_port(Port2); +close(Port, Pid) when is_port(Port) -> + ssl_test_lib:close_port(Port), + ssl_test_lib:close(Pid); +close(Pid, Port) when is_port(Port) -> + ssl_test_lib:close_port(Port), + ssl_test_lib:close(Pid); +close(Client, Server) -> + ssl_test_lib:close(Server), + ssl_test_lib:close(Client). + +%% Work around OpenSSL bug, apparently the same bug as we had fixed in +%% 11629690ba61f8e0c93ef9b2b6102fd279825977 +new_ca(FileName, CA, OwnCa) -> + {ok, P1} = file:read_file(CA), + E1 = public_key:pem_decode(P1), + {ok, P2} = file:read_file(OwnCa), + E2 = public_key:pem_decode(P2), + Pem = public_key:pem_encode(E2 ++E1), + file:write_file(FileName, Pem), + FileName. diff --git a/lib/ssl/test/ssl_ECC_SUITE_data/CA.pem b/lib/ssl/test/ssl_ECC_SUITE_data/CA.pem deleted file mode 100644 index f82efdefc5..0000000000 --- a/lib/ssl/test/ssl_ECC_SUITE_data/CA.pem +++ /dev/null @@ -1,14 +0,0 @@ ------BEGIN CERTIFICATE----- -MIICGjCCAYegAwIBAgIQZIIqq4RXfpBKJXV69Jc4BjAJBgUrDgMCHQUAMB0xGzAZ -BgNVBAMTEklTQSBUZXN0IEF1dGhvcml0eTAeFw0xMjAzMjAxNzEzMjFaFw0zOTEy -MzEyMzU5NTlaMB0xGzAZBgNVBAMTEklTQSBUZXN0IEF1dGhvcml0eTCBnzANBgkq -hkiG9w0BAQEFAAOBjQAwgYkCgYEAqnt6FSyFQVSDyP7mY63IhCzgysTxBEg1qDb8 -nBHj9REReZA5UQ5iyEOdTbdLyOaSk2rJyA2wdTjYkNnLzK49nZFlpf89r3/bakAM -wZv69S3FJi9W2z9m4JPv/5+QCYnFNRSnnHw3maNElwoQyknx96I3W7EuVOvKtKhh -4DaD0WsCAwEAAaNjMGEwDwYDVR0TAQH/BAUwAwEB/zBOBgNVHQEERzBFgBBCHwn2 -8AmbN+cvJl1iJ1bsoR8wHTEbMBkGA1UEAxMSSVNBIFRlc3QgQXV0aG9yaXR5ghBk -giqrhFd+kEoldXr0lzgGMAkGBSsOAwIdBQADgYEAIlVecua5Cr1z/cdwQ8znlgOU -U+y/uzg0nupKkopzVnRYhwV4hxZt3izAz4C/SJZB7eL0bUKlg1ceGjbQsGEm0fzF -LEV3vym4G51bxv03Iecwo96G4NgjJ7+9/7ciBVzfxZyfuCpYG1M2LyrbOyuevtTy -2+vIueT0lv6UftgBfIE= ------END CERTIFICATE----- diff --git a/lib/ssl/test/ssl_ECC_SUITE_data/ec1.crt b/lib/ssl/test/ssl_ECC_SUITE_data/ec1.crt deleted file mode 100644 index 7d2b9cde9d..0000000000 --- a/lib/ssl/test/ssl_ECC_SUITE_data/ec1.crt +++ /dev/null @@ -1,11 +0,0 @@ ------BEGIN CERTIFICATE----- -MIIBhjCB8AIBBjANBgkqhkiG9w0BAQUFADAdMRswGQYDVQQDExJJU0EgVGVzdCBB -dXRob3JpdHkwHhcNMTMwODA4MTAxNDI3WhcNMjMwODA2MTAxNDI3WjBFMQswCQYD -VQQGEwJVUzERMA8GA1UECBMIVmlyZ2luaWExFTATBgNVBAcTDEZvcnQgQmVsdm9p -cjEMMAoGA1UEAxMDZWMxMFYwEAYHKoZIzj0CAQYFK4EEAAoDQgAEpiRIxUCESROR -P8IByg+vBv1fDdAg7yXfAh95GxFtvhBqZs6ATwaRKyLmZYgUm/4NUAyUeqmTBb7s -2msKo5mnNzANBgkqhkiG9w0BAQUFAAOBgQAmwzoB1DVO69FQOUdBVnyups4t0c1c -8h+1z/5P4EtPltk4o3mRn0AZogqdXCpNbuSGbSJh+dep5xW30VLxNHdc+tZSLK6j -pT7A3hymMk8qbi13hxeH/VpEP25y1EjHowow9Wmb6ebtT/v7qFQ9AAHD9ONcIM4I -FCC8vdFo7M5GgQ== ------END CERTIFICATE----- diff --git a/lib/ssl/test/ssl_ECC_SUITE_data/ec1.key b/lib/ssl/test/ssl_ECC_SUITE_data/ec1.key deleted file mode 100644 index 2dc9508b3c..0000000000 --- a/lib/ssl/test/ssl_ECC_SUITE_data/ec1.key +++ /dev/null @@ -1,8 +0,0 @@ ------BEGIN EC PARAMETERS----- -BgUrgQQACg== ------END EC PARAMETERS----- ------BEGIN EC PRIVATE KEY----- -MHQCAQEEIOO0WK8znNzLyZIoGRIlaKnCNr2Wy8uk9i+GGFIhDGNAoAcGBSuBBAAK -oUQDQgAEpiRIxUCESRORP8IByg+vBv1fDdAg7yXfAh95GxFtvhBqZs6ATwaRKyLm -ZYgUm/4NUAyUeqmTBb7s2msKo5mnNw== ------END EC PRIVATE KEY----- diff --git a/lib/ssl/test/ssl_ECC_SUITE_data/ec2.crt b/lib/ssl/test/ssl_ECC_SUITE_data/ec2.crt deleted file mode 100644 index b0558a0ebc..0000000000 --- a/lib/ssl/test/ssl_ECC_SUITE_data/ec2.crt +++ /dev/null @@ -1,11 +0,0 @@ ------BEGIN CERTIFICATE----- -MIIBhjCB8AIBBzANBgkqhkiG9w0BAQUFADAdMRswGQYDVQQDExJJU0EgVGVzdCBB -dXRob3JpdHkwHhcNMTMwODA4MTAxNDM0WhcNMjMwODA2MTAxNDM0WjBFMQswCQYD -VQQGEwJVUzERMA8GA1UECBMIVmlyZ2luaWExFTATBgNVBAcTDEZvcnQgQmVsdm9p -cjEMMAoGA1UEAxMDZWMyMFYwEAYHKoZIzj0CAQYFK4EEAAoDQgAEzXaYReUyvoYl -FwGOe0MJEXWCUncMfr2xG4GMjGYlfZsvLGEokefsJIvW+I+9jgUT2UFjxFXYNAvm -uD1A1iWVWjANBgkqhkiG9w0BAQUFAAOBgQBFa6iIlrT9DWptIdB8uSYvp7qwiHxN -hiVH5YhGIHHqjGZqtRHrSxqNEYMXXrgH9Hxc6gDbk9PsHZyVVoh/HgVWddqW1inh -tStZm420PAKCuH4T6Cfsk76GE2m7FRzJvw9TM1f2A5nIy9abyrpup8lZGcIL4Kmq -1Fix1LRtrmLNTA== ------END CERTIFICATE----- diff --git a/lib/ssl/test/ssl_ECC_SUITE_data/ec2.key b/lib/ssl/test/ssl_ECC_SUITE_data/ec2.key deleted file mode 100644 index 366d13648b..0000000000 --- a/lib/ssl/test/ssl_ECC_SUITE_data/ec2.key +++ /dev/null @@ -1,8 +0,0 @@ ------BEGIN EC PARAMETERS----- -BgUrgQQACg== ------END EC PARAMETERS----- ------BEGIN EC PRIVATE KEY----- -MHQCAQEEIPR3ORUpAFMTQhUJ0jllN38LKWziG8yP2H54Y/9vh1PwoAcGBSuBBAAK -oUQDQgAEzXaYReUyvoYlFwGOe0MJEXWCUncMfr2xG4GMjGYlfZsvLGEokefsJIvW -+I+9jgUT2UFjxFXYNAvmuD1A1iWVWg== ------END EC PRIVATE KEY----- diff --git a/lib/ssl/test/ssl_ECC_SUITE_data/rsa1.crt b/lib/ssl/test/ssl_ECC_SUITE_data/rsa1.crt deleted file mode 100644 index ed9beacf68..0000000000 --- a/lib/ssl/test/ssl_ECC_SUITE_data/rsa1.crt +++ /dev/null @@ -1,20 +0,0 @@ ------BEGIN CERTIFICATE----- -MIIDVjCCAr8CAQkwDQYJKoZIhvcNAQEFBQAwHTEbMBkGA1UEAxMSSVNBIFRlc3Qg -QXV0aG9yaXR5MB4XDTEzMDgwODEwMTUzNFoXDTQwMTIyNDEwMTUzNFowRjELMAkG -A1UEBhMCVVMxETAPBgNVBAgTCFZpcmdpbmlhMRUwEwYDVQQHEwxGb3J0IEJlbHZv -aXIxDTALBgNVBAMTBHJzYTEwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoIC -AQC62v40w1AjV3oJuyYC2Fw6XhTOi1il6xZFnB9J1WhCmuxAB/VMhBcNypx38mNk -eQ7a/ERQ5ddhZey29DYeFYU8oqfDURgWx5USHufb90xBen9KPmX3VNuQ8ZFP2q8Q -b01/oRHBJQRBuaCtFHzpGIVBjC6dD5yeQgJsYaF4u+PBbonsIGROXMybcvUzXmjU -dwpy2NhjGQL5sWcOdIeRP43APSyRYvq4tuBUZk2XxWfBcvA8LpcoYPMlRTf6jGL1 -/fAAcCYJ9lh3h92w0NZ/7ZRa/ebTplxK6yqCftuSKui1KdL69m0WZqHl79AUSfs9 -lsOwx9lHkyYvJeMofyeDbZ+3OYLmVqEBG1fza2aV2XVh9zJ8fAwmXy/c2IDhw/oD -HAe/rSg/Sgt03ydIKqtZHbl3v0EexQQRlJRULIzdtON02dJMUd4EFUgQ9OUtEmC2 -Psj9Jdu1g5cevU7Mymu8Ot+fjHiGTcBUsXNuXFCbON3Gw7cIDl4+iv+cpDHHVC9L -HK3PMEq3vu3qOGXSz+LDOoqkfROcLG7BclBuN2zoVSsMHFkB4aJhwy7eHhGz0z2W -c6LTVd+GAApdY80kmjOjT//QxHEsX/n1useHza3OszQqZiArr4ub4rtq+l1DxAS/ -DWrZ/JGsbKL8cjWso6qBF94xTi8WhjkKuUYhsm+qLAbNOQIDAQABMA0GCSqGSIb3 -DQEBBQUAA4GBAIcuzqRkfypV/9Z85ZQCCoejPm5Urhv7dfg1/B3QtazogPBZLgL5 -e60fG1uAw5GmqTViHLvW06z73oQvJrFkrCLVvadDNtrKYKXnXqdkgVyk36F/B737 -A43HGnMfSxCfRhIOuKZB9clP5PiNlhw36yi3DratqT6TUvI69hg8a7jA ------END CERTIFICATE----- diff --git a/lib/ssl/test/ssl_ECC_SUITE_data/rsa1.key b/lib/ssl/test/ssl_ECC_SUITE_data/rsa1.key deleted file mode 100644 index 6e0d913d79..0000000000 --- a/lib/ssl/test/ssl_ECC_SUITE_data/rsa1.key +++ /dev/null @@ -1,51 +0,0 @@ ------BEGIN RSA PRIVATE KEY----- -MIIJKAIBAAKCAgEAutr+NMNQI1d6CbsmAthcOl4UzotYpesWRZwfSdVoQprsQAf1 -TIQXDcqcd/JjZHkO2vxEUOXXYWXstvQ2HhWFPKKnw1EYFseVEh7n2/dMQXp/Sj5l -91TbkPGRT9qvEG9Nf6ERwSUEQbmgrRR86RiFQYwunQ+cnkICbGGheLvjwW6J7CBk -TlzMm3L1M15o1HcKctjYYxkC+bFnDnSHkT+NwD0skWL6uLbgVGZNl8VnwXLwPC6X -KGDzJUU3+oxi9f3wAHAmCfZYd4fdsNDWf+2UWv3m06ZcSusqgn7bkirotSnS+vZt -Fmah5e/QFEn7PZbDsMfZR5MmLyXjKH8ng22ftzmC5lahARtX82tmldl1YfcyfHwM -Jl8v3NiA4cP6AxwHv60oP0oLdN8nSCqrWR25d79BHsUEEZSUVCyM3bTjdNnSTFHe -BBVIEPTlLRJgtj7I/SXbtYOXHr1OzMprvDrfn4x4hk3AVLFzblxQmzjdxsO3CA5e -Por/nKQxx1QvSxytzzBKt77t6jhl0s/iwzqKpH0TnCxuwXJQbjds6FUrDBxZAeGi -YcMu3h4Rs9M9lnOi01XfhgAKXWPNJJozo0//0MRxLF/59brHh82tzrM0KmYgK6+L -m+K7avpdQ8QEvw1q2fyRrGyi/HI1rKOqgRfeMU4vFoY5CrlGIbJvqiwGzTkCAwEA -AQKCAgBkXyaWKSRvF5pSh9lPRfGk2MzMdkXUOofoNIkKHDy5KocljiDSTVIk8mVC -eU2ytuSn9UKtQgmEJEAXtu8rEdxUSftcC7+o3OTSqw9ZNWoc8jRWKVaUmVyoa1rn -Tk0jwuYaXOcwnTXAKHqK/qpqe+V45FhVvgEfcc3jcj5OoH8jdMFZubyn62ltRz83 -rMsa9icCskDqWpEil40IUshP2ZfHYBUEs+qCNpoiPCIKGNw3KgqqCUzhP9LcfmYn -jCnMge/eDGAikdXLv4vyYvwWFATRK/pGTuLcy542IvbHeY0vY5wVezH2CoOFBGD9 -xQ/UcZwE5hVtQToNsYhoRIVxL/3Of0qDk1M6W2Plh2MAstyejIHE3ct0pPfW3rsu -j/9Z/H0P9Q5ghSjarwOp2qGrrz6/4LVbbTDY8V1L928l4SqbUMtEQxcxTBN8YFoD -mPV3Jc3zls9wiiEX53RcH8MK5tjrcRwWqurTZvi/pkLfXlGDgKGCOaa3HgWVQyU+ -L6jVZM+u1nwN+jNXQYGeLEro/6tvG8WQbRMHQoxLG+rm4V3/SwH0DcfrVFDTg+i6 -3wMU1GC/aQEdTFWXvHAkpwrf4M9QWvjtheiaSxtBUoAY6l+ixCVHKrIk6glKLEjx -92HxmcJdopQScFETAyg8eVKV0kOGfVeFEpIqwq7hVedmTflpQQKCAQEA44h4dAta -cYeBqBr8eljWcgs79gmgwBEQxQUnwE/zuzLKn5NxAW324Kh25V/n/MupUzBlLPWn -91UHfw9PCXT8/HvgYQ4S5sXbKRbGmuPSsTmz4Rfe2ix6RggVNUOwORVNDyM7SQh7 -USdzZH5dMxKfF5L/b4Byx7eQZaoeKlfaXcqgikNZZ6pkhVCNxUKi9vvjS9r2wwCd -xtgu5MfTpdEci0zH1+uuRisVRcEbcRX9umUTCiZrmEeddZXNiwTAS3FtX7qGzuq9 -LKIeETwcOZrWj0E48UvbSfK4Axn7sf5J0n7/Qo7I089S5QQEI6ZDP501i71dNFhn -qfcY30c1k3TC7QKCAQEA0juuVHExKNLLNmQejNPfuHYoH0Uk2BH/8x96/Mkj6k6K -SUCHDS3iWOljXGw8YtpS8v5mGBGgMhJ+s/vCRM6R9eXYTc8u2ktY/kjyW0PgW8/Z -vb9VrQpn5svTNwj2Q8qYsTqXnQKO7YuL+hnQpQNAcID6FTeOASVLGObEf810qRfN -4y3RqCWUnYXXTyXj+cJdbXTxfF7HVZPIAQKqE7J5Qo9ynYILY62oSmUGC6m8VKyE -rrvDMK1IVi0X4w+Jx4HX0IC2+DBKxCaLWT69bE1IwjB06Q5zoTQPVi6c6qQp7K0H -kqSyLJ/ctwcEubu0DPNmvMlgWtAbAsoESA5GbIit/QKCAQEAxRzp9OYNAUM6AK74 -QOmLRZsT4+6tUxa1p2jy6fiZlnfG731kra9c630mG0n9iJPK6aWIUO20CGGiL+HM -P84YiIaseIgfucp4NV1kyrRJR31MptjuF6Xme5ru/IjaNmmMq2uDJZ7ybfi2T73k -8aTVLDANl8P4K6qLrnc00MvxAcXTVFRKNLN5h8CkQNqcoUjPvVxA3+g9xxBrd4jh -gsnoZ4kpq5WiEWmrcRV8t3gsqfh8CRQFrBOGhmIzgZapG/J0pTTLKqBTKEJ9t8KS -VRkdfVcshGWJ4MMjxJQS5zz7KR8Z9cgKlOwLzRiwmU2k/owr4hY3k2xuyeClrHBd -KpRBdQKCAQBvDk/dE55gbloi9WieBB6eluxC+IeqDHgkunCBsM9kKvEqGQg+kgqL -5V4zqImNvr8q1fCgrk7tpI+CDHBnYKgCOdS15cheUIdGbMp6I7UVSws/DR/5NRIF -/Y4p+HX/Abr/hHAq5PsTyS+8gn6RbNJRnBB/vMUrHcQ5902+JY6G9KgyZjXmmVOU -kutWSDHR8jbgZ3JZvMeYEWUKA5pMpW8hFh35zoStt0K7afpzlsqCAFBm7ZEC2cbo -nxGLRN4HojObVSNSoFAepi3eiyINYBYbXvWjV5sFgTbI0/7YhLgQ6qahdJcas6go -l3CLnPhUDxAqkkZwMpbSNl1kowXYt6sRAoIBAAOWnXgf9Bdb9OWKGgt42gVfC4cz -zj2JoLpbDTtbEdHNn8XQvPhGbpdtgnsuEMijIMy1UTlmv17jbFWdZTDeN31EUJrC -smgKX0OlVFKD90AI0BiIREK0hJUBV0pV4JoUjwnQBHGvranD06/wAtHEqgqF1Ipp -DCAKwxggM7qtB1R1vkrc/aLQej+mlwA8N6q92rnEsg+EnEbhtLDDZQcV/q5cSDCN -MMcnM+QdyjKwEeCVXHaqNfeSqKg/Ab2eZbS9VxA+XZD73+eUY/JeJsg7LfZrRz0T -ij5LCS7A+nVB5/B5tGkk4fcNhk2n356be6l46S98BEgtuwGLC9pqXf7zyp4= ------END RSA PRIVATE KEY----- diff --git a/lib/ssl/test/ssl_ECC_SUITE_data/rsa2.crt b/lib/ssl/test/ssl_ECC_SUITE_data/rsa2.crt deleted file mode 100644 index 06ca92dda3..0000000000 --- a/lib/ssl/test/ssl_ECC_SUITE_data/rsa2.crt +++ /dev/null @@ -1,20 +0,0 @@ ------BEGIN CERTIFICATE----- -MIIDVjCCAr8CAQowDQYJKoZIhvcNAQEFBQAwHTEbMBkGA1UEAxMSSVNBIFRlc3Qg -QXV0aG9yaXR5MB4XDTEzMDgwODEwMTYwMloXDTQwMTIyNDEwMTYwMlowRjELMAkG -A1UEBhMCVVMxETAPBgNVBAgTCFZpcmdpbmlhMRUwEwYDVQQHEwxGb3J0IEJlbHZv -aXIxDTALBgNVBAMTBHJzYTIwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoIC -AQCjQUe0BGOpULjOAmLbXM4SSQzJvxJbCFi3tryyd+OARq6Fdp6/fslVhsr0PhWE -X8yRbAugIjseTpLwz+1OC6LavOGV1ixzGTI/9HDXGKbf8qoCrSdh28sqQJnmqGT4 -UCKLn6Rqjg2iyBBcSK3LrtKEPI4C7NaSOZUtANkppvziEMwm+0r16sgHh2Xx6mxd -22q01kq1lJqwEnIDPMSz3+ESUVQQ4T3ka7yFIhc9PYmILIXkZi0x7AiDeRkIILul -GQrduTWSPGY3prXeDAbmQNazxrHp8fcR2AfFSI6HYxMALq9jWxc4xDIkss6BO2Et -riJOIgXFpbyVsYCbkI1kXhEWFDt3uJBIcmtJKGzro4xv+XLG6BbUeTJgSHXMc7Cb -fX87+CBIFR5a/aqkEKh/mcvsDdaV+kpNKdr7q4wAuIQb8g7IyXEDuAm1VZjQs9WC -KFRGSq9sergEw9gna0iThRZjD+dzNzB17XmlAK4wa98a7MntwqpAt/GsCFOiPM8E -c+8gpuo8WqC0kP8OpImyw9cQhlZ3dca1qkr2cyKyAOGxUxyA67FgiHSsxJJ2Xhse -o49qeKTjMZd8zhSokM2TH6qEf7YfOePU51YRfAHUhzRmE31N/MExqDjFjklksEtM -iHhbPo+cOoxV8x1u13umdUvtTaAUSBA/DpvzWdnORvnaqQIDAQABMA0GCSqGSIb3 -DQEBBQUAA4GBAFD+O7h+5R5S1rIN9eC+oEGpvRhMG4v4G3pJp+c7bbtO7ifFx1WP -bta1b5YtiQYcKP0ORABm/3Kcvsb3VbaMH/zkxWEbASZsmIcBY3ml4f2kkn6WT2hD -Wc6VMIAR3N6Mj1b30yI1qYVIid+zIouiykMB+zqllm+Uar0SPNjKxDU/ ------END CERTIFICATE----- diff --git a/lib/ssl/test/ssl_ECC_SUITE_data/rsa2.key b/lib/ssl/test/ssl_ECC_SUITE_data/rsa2.key deleted file mode 100644 index d415ef0391..0000000000 --- a/lib/ssl/test/ssl_ECC_SUITE_data/rsa2.key +++ /dev/null @@ -1,51 +0,0 @@ ------BEGIN RSA PRIVATE KEY----- -MIIJJwIBAAKCAgEAo0FHtARjqVC4zgJi21zOEkkMyb8SWwhYt7a8snfjgEauhXae -v37JVYbK9D4VhF/MkWwLoCI7Hk6S8M/tTgui2rzhldYscxkyP/Rw1xim3/KqAq0n -YdvLKkCZ5qhk+FAii5+kao4NosgQXEity67ShDyOAuzWkjmVLQDZKab84hDMJvtK -9erIB4dl8epsXdtqtNZKtZSasBJyAzzEs9/hElFUEOE95Gu8hSIXPT2JiCyF5GYt -MewIg3kZCCC7pRkK3bk1kjxmN6a13gwG5kDWs8ax6fH3EdgHxUiOh2MTAC6vY1sX -OMQyJLLOgTthLa4iTiIFxaW8lbGAm5CNZF4RFhQ7d7iQSHJrSShs66OMb/lyxugW -1HkyYEh1zHOwm31/O/ggSBUeWv2qpBCof5nL7A3WlfpKTSna+6uMALiEG/IOyMlx -A7gJtVWY0LPVgihURkqvbHq4BMPYJ2tIk4UWYw/nczcwde15pQCuMGvfGuzJ7cKq -QLfxrAhTojzPBHPvIKbqPFqgtJD/DqSJssPXEIZWd3XGtapK9nMisgDhsVMcgOux -YIh0rMSSdl4bHqOPanik4zGXfM4UqJDNkx+qhH+2Hznj1OdWEXwB1Ic0ZhN9TfzB -Mag4xY5JZLBLTIh4Wz6PnDqMVfMdbtd7pnVL7U2gFEgQPw6b81nZzkb52qkCAwEA -AQKCAgBORLHXwHL3bdfsDIDQooG5ioQzBQQL2MiP63A0L/5GNZzeJ6ycKnDkLCeJ -SWqPeE5fOemo8EBfm1QfV9BxpmqBbCTK7U+KLv5EYzDmLs9ydqjDd7h11iZlL2uZ -hgpCckjdn7/3xfsLm9ccJ0wLZtlOxKlhBaMpn6nBVbLHoWOEDoGR/tBFbjZQRb2+ -aaFirhtOb56Jx6ER4QYAP1Ye1qrVWWBwZ0yBApXzThDOL36MZqwagFISqRK71YcG -uoq78HGhM3ZXkdV/wNFYj3OPWG6W6h/KBVNqnqO7FbofdoRZhghYHgfYE1fm+ELA -+nLwr5eK1gzmYTs0mVELRBZFlEOkCfYNOnuRgysFezEklS+ICp3HzIhYXza3kyTf -B2ZBwZZVCv/94MKyibyANErmv1a5ugY5Hsn9/WKC8qTto+qLYoyFCvBjzj0PSaVX -/3cty2DY0SK16K1Y4AOPtJMYTXYB3tVX8Akgjz1F6REBtZSOXrSQ3Vhy1ORl3Hzf -WCBYDqL8K0hJiBVgkvneIyIjmFHsdM60Nr7EldBEnJ/UrPzsl2VuWFPZlnasfUaW -x+vq1H4Dfz+bHt8coBRHDjKgUvwkfFeBQOBR5DG3vMrxguVRA1EYYMRR5C3yxk2m -ARAtdh4VxUQDQjjrmr7Dl/y1rU34aInXIrrFWpuvIhl8Ht09sQKCAQEA1pXKK5f0 -HkKfM/qk5xzF+WdHClBrPXi0XwLN6UQ+WWMMNhkGZ+FMPXl/6IJDT91s6DA3tPhr -OZF64n9ZFaGgHNBXNiB+Txjv5vZeSBMFt3hSonqt42aijx6gXfmLnkA+TYpa6Wex -YCeEgdH8LocJa7Gj2vzrYliPYk3deh6SnZZ6N8bI+ciwK3ZGF/pkWaTX83dIFq3w -YyZ+0dEpNGbA9812wNVourPg3OfqG3/CdnTfvY1M9KCC3JalpyzQL4Zm5soXF0wj -36C2yTxA02AyFz3TvUIBrvsN6i0gmGfE79+UIp29JYrFRsIgBDt+ze2vQWUz2MX5 -GeX6/yCBgiTXtwKCAQEAwsNf6k2m5Cw+WtuLzzUfBBJCN+t1lrnYJ6lF0HubW6TZ -vX1kBWyc+Rpo4ljr/+f4R9aC/gTEQOmV/hNVZy1RU2dAI8cH+r6JWG9lgif+8h// -5R81txE7gnuK1Na7PmvnQPPN661zsQZ5e1ENPXS3TJmUW/M01JxAMqEQjvAPa/II -H2KjL5NX28k9Hiw9rP6n+qXAfG/LEwXgoVCcehPwfANqQ1l95UgOdKDmjG94dipI -h2DEK70ZbrsgQbT60Wd8I5h0yhiQsik2/bVkqLmcG4SSg0/5cf2vZMApgoH/adUz -rJFdthm7iGPLhwS6fbhXew17Af96FvzfkifUV+cgnwKCAQBNUlYyFSQKz1jMgxFu -kciokNVhWw75bIgaAEmwNz38OZuJ1sSfI+iz8hbr8hxNJ+15UP6RwD3q1YghG2A/ -Uij+mPgD8ftxhvvTDo10jR4vOTUVhP0phq8mwRNqKWRs1ptcl3Egz5NzoWm22bJ0 -FYaIfs8bNq2el2i7NHGM8n1EOZe6h2+dyfno/0pMk5YbUzHZce7Q9UY8g/+InUSq -tCfuYuPaokuFkxGAqDSMSiIJSx3gEI1dTIU69TGlppkxts1XdhSR+YanqyKSKpr1 -T6FdDJNCjAlNQvuFmVM4d5PYF4kqXApu/60MTSD6RXHwxCe1ecEP6G5VLbCew9jG -y33LAoIBAGsWyC9pwQEm/qYwn4AwYjx32acrtX1J9HtiTLvkqzjJvNu/DXcaEHm7 -tr32TNVp9A9z+JS5hDt49Hs+oC/aMCRe2lqRvmZ1y8kvfy4A1eLGC4stDPj65bDK -QzziURRyejYxmCElPz6wI63VlCUdfwgEThn88SiSPY5ZF2SwxJoC+8peDwJCzwVP -cmabxtHPOAfOibciNRPhoHCyhUdunUVjD1O26k1ewGwKaJoBVMgMWdLuNw8hq9FB -3OukGmF3uD9OPbE9rpn3pX/89Dr9y8MpsvG20J6H8Z/BNVHILus/SmlxiIhvP7kv -viIgTHaCHL/RWrhvg+8N3dRcSBqJQFsCggEAFe2TMEq2AlnBn4gsuAOIuZPYKQCg -2a+tl1grQzmNth6AGGQcIqShadICD6SnVMIS64HHV/m18Cuz7GhJ06ZVjXJsHueG -UpTE9wAmI2LxnNkupkLJu+SVcW3N86PujWmQBFpHkd+IRPLS51xjD9W5zLJ7HL4/ -fnKO+B+ZK6Imxbe5C5vJezkGfeOSyQoVtt6MT/XtSKNEGPBX+M6fLKgUMMg2H2Mt -/SsD7DkOzFteKXzaEg/K8oOTpsOPkVDwNl2KErlEqbJv0k7yEVw50mYmsn/OLjh8 -+9EibISwCODbPxB+PhV6u2ue1IvGLRqtsN60lFOvbGn+kSewy9EUVHHQDQ== ------END RSA PRIVATE KEY----- diff --git a/lib/ssl/test/ssl_basic_SUITE.erl b/lib/ssl/test/ssl_basic_SUITE.erl index 64a93440c7..1da4e88077 100644 --- a/lib/ssl/test/ssl_basic_SUITE.erl +++ b/lib/ssl/test/ssl_basic_SUITE.erl @@ -115,7 +115,10 @@ options_tests() -> reuseaddr, tcp_reuseaddr, honor_server_cipher_order, - honor_client_cipher_order + honor_client_cipher_order, + ciphersuite_vs_version, + unordered_protocol_versions_server, + unordered_protocol_versions_client ]. api_tests() -> @@ -137,7 +140,9 @@ api_tests() -> ssl_accept_timeout, ssl_recv_timeout, versions_option, - server_name_indication_option + server_name_indication_option, + accept_pool, + new_options_in_accept ]. session_tests() -> @@ -186,7 +191,11 @@ error_handling_tests()-> tcp_error_propagation_in_active_mode, tcp_connect, tcp_connect_big, - close_transport_accept + close_transport_accept, + recv_active, + recv_active_once, + recv_error_handling, + dont_crash_on_handshake_garbage ]. rizzo_tests() -> @@ -239,6 +248,14 @@ end_per_group(_GroupName, Config) -> Config. %%-------------------------------------------------------------------- +init_per_testcase(Case, Config) when Case == unordered_protocol_versions_client; + Case == unordered_protocol_versions_server-> + case proplists:get_value(supported, ssl:versions()) of + ['tlsv1.2' | _] -> + Config; + _ -> + {skip, "TLS 1.2 need but not supported on this platform"} + end; init_per_testcase(no_authority_key_identifier, Config) -> %% Clear cach so that root cert will not %% be found. @@ -309,7 +326,11 @@ alerts(Config) when is_list(Config) -> ?ILLEGAL_PARAMETER, ?UNKNOWN_CA, ?ACCESS_DENIED, ?DECODE_ERROR, ?DECRYPT_ERROR, ?EXPORT_RESTRICTION, ?PROTOCOL_VERSION, ?INSUFFICIENT_SECURITY, ?INTERNAL_ERROR, ?USER_CANCELED, - ?NO_RENEGOTIATION], + ?NO_RENEGOTIATION, ?UNSUPPORTED_EXTENSION, ?CERTIFICATE_UNOBTAINABLE, + ?UNRECOGNISED_NAME, ?BAD_CERTIFICATE_STATUS_RESPONSE, + ?BAD_CERTIFICATE_HASH_VALUE, ?UNKNOWN_PSK_IDENTITY, + 255 %% Unsupported/unknow alert will result in a description too + ], Alerts = [?ALERT_REC(?WARNING, ?CLOSE_NOTIFY) | [?ALERT_REC(?FATAL, Desc) || Desc <- Descriptions]], lists:foreach(fun(Alert) -> @@ -321,6 +342,38 @@ alerts(Config) when is_list(Config) -> end end, Alerts). %%-------------------------------------------------------------------- +new_options_in_accept() -> + [{doc,"Test that you can set ssl options in ssl_accept/3 and not tcp upgrade"}]. +new_options_in_accept(Config) when is_list(Config) -> + ClientOpts = ?config(client_opts, Config), + ServerOpts0 = ?config(server_dsa_opts, Config), + [_ , _ | ServerSslOpts] = ?config(server_opts, Config), %% Remove non ssl opts + {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), + Server = ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, + {from, self()}, + {ssl_extra_opts, [{versions, [sslv3]}, + {ciphers,[{rsa,rc4_128,sha}]} | ServerSslOpts]}, %% To be set in ssl_accept/3 + {mfa, {?MODULE, connection_info_result, []}}, + {options, proplists:delete(cacertfile, ServerOpts0)}]), + + Port = ssl_test_lib:inet_port(Server), + Client = ssl_test_lib:start_client([{node, ClientNode}, {port, Port}, + {host, Hostname}, + {from, self()}, + {mfa, {?MODULE, connection_info_result, []}}, + {options, [{versions, [sslv3]} | ClientOpts]}]), + + ct:log("Testcase ~p, Client ~p Server ~p ~n", + [self(), Client, Server]), + + ServerMsg = ClientMsg = {ok, {sslv3, {rsa, rc4_128, sha}}}, + + ssl_test_lib:check_result(Server, ServerMsg, Client, ClientMsg), + + ssl_test_lib:close(Server), + ssl_test_lib:close(Client). +%%-------------------------------------------------------------------- + connection_info() -> [{doc,"Test the API function ssl:connection_info/1"}]. connection_info(Config) when is_list(Config) -> @@ -360,6 +413,7 @@ protocol_versions() -> protocol_versions(Config) when is_list(Config) -> basic_test(Config). + %%-------------------------------------------------------------------- empty_protocol_versions() -> [{doc,"Test to set an empty list of protocol versions in app environment."}]. @@ -1118,6 +1172,57 @@ close_transport_accept(Config) when is_list(Config) -> Other -> exit({?LINE, Other}) end. +%%-------------------------------------------------------------------- +recv_active() -> + [{doc,"Test recv on active socket"}]. + +recv_active(Config) when is_list(Config) -> + ClientOpts = ?config(client_opts, Config), + ServerOpts = ?config(server_opts, Config), + {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), + Server = + ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, + {from, self()}, + {mfa, {?MODULE, try_recv_active, []}}, + {options, [{active, true} | ServerOpts]}]), + Port = ssl_test_lib:inet_port(Server), + Client = + ssl_test_lib:start_client([{node, ClientNode}, {port, Port}, + {host, Hostname}, + {from, self()}, + {mfa, {?MODULE, try_recv_active, []}}, + {options, [{active, true} | ClientOpts]}]), + + ssl_test_lib:check_result(Server, ok, Client, ok), + + ssl_test_lib:close(Server), + ssl_test_lib:close(Client). + +%%-------------------------------------------------------------------- +recv_active_once() -> + [{doc,"Test recv on active socket"}]. + +recv_active_once(Config) when is_list(Config) -> + ClientOpts = ?config(client_opts, Config), + ServerOpts = ?config(server_opts, Config), + {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), + Server = + ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, + {from, self()}, + {mfa, {?MODULE, try_recv_active_once, []}}, + {options, [{active, once} | ServerOpts]}]), + Port = ssl_test_lib:inet_port(Server), + Client = + ssl_test_lib:start_client([{node, ClientNode}, {port, Port}, + {host, Hostname}, + {from, self()}, + {mfa, {?MODULE, try_recv_active_once, []}}, + {options, [{active, once} | ClientOpts]}]), + + ssl_test_lib:check_result(Server, ok, Client, ok), + + ssl_test_lib:close(Server), + ssl_test_lib:close(Client). %%-------------------------------------------------------------------- dh_params() -> @@ -1141,7 +1246,7 @@ dh_params(Config) when is_list(Config) -> {from, self()}, {mfa, {ssl_test_lib, send_recv_result_active, []}}, {options, - [{ciphers,[{dhe_rsa,aes_256_cbc,sha,ignore}]} | + [{ciphers,[{dhe_rsa,aes_256_cbc,sha}]} | ClientOpts]}]), ssl_test_lib:check_result(Server, ok, Client, ok), @@ -1240,7 +1345,7 @@ tcp_connect() -> tcp_connect(Config) when is_list(Config) -> ServerOpts = ?config(server_opts, Config), {_, ServerNode, Hostname} = ssl_test_lib:run_where(Config), - TcpOpts = [binary, {reuseaddr, true}], + TcpOpts = [binary, {reuseaddr, true}, {active, false}], Server = ssl_test_lib:start_upgrade_server_error([{node, ServerNode}, {port, 0}, {from, self()}, @@ -1266,6 +1371,7 @@ tcp_connect_big() -> [{doc,"Test what happens when a tcp tries to connect, i,e. a bad big (ssl) packet is sent first"}]. tcp_connect_big(Config) when is_list(Config) -> + process_flag(trap_exit, true), ServerOpts = ?config(server_opts, Config), {_, ServerNode, Hostname} = ssl_test_lib:run_where(Config), TcpOpts = [binary, {reuseaddr, true}], @@ -1291,7 +1397,9 @@ tcp_connect_big(Config) when is_list(Config) -> {Server, {error, timeout}} -> ct:fail("hangs"); {Server, {error, Error}} -> - ct:log("Error ~p", [Error]) + ct:log("Error ~p", [Error]); + {'EXIT', Server, _} -> + ok end end. @@ -2523,6 +2631,81 @@ honor_cipher_order(Config, Honor, ServerCiphers, ClientCiphers, Expected) -> ssl_test_lib:close(Client). %%-------------------------------------------------------------------- +ciphersuite_vs_version(Config) when is_list(Config) -> + + {_ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), + ServerOpts = ?config(server_opts, Config), + + Server = ssl_test_lib:start_server_error([{node, ServerNode}, {port, 0}, + {from, self()}, + {options, ServerOpts}]), + Port = ssl_test_lib:inet_port(Server), + + {ok, Socket} = gen_tcp:connect(Hostname, Port, [binary, {active, false}]), + ok = gen_tcp:send(Socket, + <<22, 3,0, 49:16, % handshake, SSL 3.0, length + 1, 45:24, % client_hello, length + 3,0, % SSL 3.0 + 16#deadbeef:256, % 32 'random' bytes = 256 bits + 0, % no session ID + %% three cipher suites -- null, one with sha256 hash and one with sha hash + 6:16, 0,255, 0,61, 0,57, + 1, 0 % no compression + >>), + {ok, <<22, RecMajor:8, RecMinor:8, _RecLen:16, 2, HelloLen:24>>} = gen_tcp:recv(Socket, 9, 10000), + {ok, <<HelloBin:HelloLen/binary>>} = gen_tcp:recv(Socket, HelloLen, 5000), + ServerHello = tls_handshake:decode_handshake({RecMajor, RecMinor}, 2, HelloBin), + case ServerHello of + #server_hello{server_version = {3,0}, cipher_suite = <<0,57>>} -> + ok; + _ -> + ct:fail({unexpected_server_hello, ServerHello}) + end. + +%%-------------------------------------------------------------------- + +dont_crash_on_handshake_garbage() -> + [{doc, "Ensure SSL server worker thows an alert on garbage during handshake " + "instead of crashing and exposing state to user code"}]. + +dont_crash_on_handshake_garbage(Config) -> + ServerOpts = ?config(server_opts, Config), + + {_ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), + + Server = ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, + {from, self()}, + {mfa, {ssl_test_lib, send_recv_result_active, []}}, + {options, ServerOpts}]), + unlink(Server), monitor(process, Server), + Port = ssl_test_lib:inet_port(Server), + + {ok, Socket} = gen_tcp:connect(Hostname, Port, [binary, {active, false}]), + + % Send hello and garbage record + ok = gen_tcp:send(Socket, + [<<22, 3,3, 49:16, 1, 45:24, 3,3, % client_hello + 16#deadbeef:256, % 32 'random' bytes = 256 bits + 0, 6:16, 0,255, 0,61, 0,57, 1, 0 >>, % some hello values + + <<22, 3,3, 5:16, 92,64,37,228,209>> % garbage + ]), + % Send unexpected change_cipher_spec + ok = gen_tcp:send(Socket, <<20, 0,0,12, 111,40,244,7,137,224,16,109,197,110,249,152>>), + + % Ensure we receive an alert, not sudden disconnect + {ok, <<21, _/binary>>} = drop_handshakes(Socket, 1000). + +drop_handshakes(Socket, Timeout) -> + {ok, <<RecType:8, _RecMajor:8, _RecMinor:8, RecLen:16>> = Header} = gen_tcp:recv(Socket, 5, Timeout), + {ok, <<Frag:RecLen/binary>>} = gen_tcp:recv(Socket, RecLen, Timeout), + case RecType of + 22 -> drop_handshakes(Socket, Timeout); + _ -> {ok, <<Header/binary, Frag/binary>>} + end. + + +%%-------------------------------------------------------------------- hibernate() -> [{doc,"Check that an SSL connection that is started with option " @@ -2921,6 +3104,57 @@ versions_option(Config) when is_list(Config) -> %%-------------------------------------------------------------------- +unordered_protocol_versions_server() -> + [{doc,"Test that the highest protocol is selected even" + " when it is not first in the versions list."}]. + +unordered_protocol_versions_server(Config) when is_list(Config) -> + ClientOpts = ?config(client_opts, Config), + ServerOpts = ?config(server_opts, Config), + + {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), + Server = ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, + {from, self()}, + {mfa, {?MODULE, connection_info_result, []}}, + {options, [{versions, ['tlsv1.1', 'tlsv1.2']} | ServerOpts]}]), + Port = ssl_test_lib:inet_port(Server), + + Client = ssl_test_lib:start_client([{node, ClientNode}, {port, Port}, + {host, Hostname}, + {from, self()}, + {mfa, {?MODULE, connection_info_result, []}}, + {options, ClientOpts}]), + CipherSuite = first_rsa_suite(ssl:cipher_suites()), + ServerMsg = ClientMsg = {ok, {'tlsv1.2', CipherSuite}}, + ssl_test_lib:check_result(Server, ServerMsg, Client, ClientMsg). + +%%-------------------------------------------------------------------- +unordered_protocol_versions_client() -> + [{doc,"Test that the highest protocol is selected even" + " when it is not first in the versions list."}]. + +unordered_protocol_versions_client(Config) when is_list(Config) -> + ClientOpts = ?config(client_opts, Config), + ServerOpts = ?config(server_opts, Config), + + {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), + Server = ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, + {from, self()}, + {mfa, {?MODULE, connection_info_result, []}}, + {options, ServerOpts }]), + Port = ssl_test_lib:inet_port(Server), + + Client = ssl_test_lib:start_client([{node, ClientNode}, {port, Port}, + {host, Hostname}, + {from, self()}, + {mfa, {?MODULE, connection_info_result, []}}, + {options, [{versions, ['tlsv1.1', 'tlsv1.2']} | ClientOpts]}]), + + CipherSuite = first_rsa_suite(ssl:cipher_suites()), + ServerMsg = ClientMsg = {ok, {'tlsv1.2', CipherSuite}}, + ssl_test_lib:check_result(Server, ServerMsg, Client, ClientMsg). + +%%-------------------------------------------------------------------- server_name_indication_option() -> [{doc,"Test API server_name_indication option to connect."}]. @@ -2958,6 +3192,53 @@ server_name_indication_option(Config) when is_list(Config) -> ssl_test_lib:close(Server), ssl_test_lib:close(Client0), ssl_test_lib:close(Client1). +%%-------------------------------------------------------------------- + +accept_pool() -> + [{doc,"Test having an accept pool."}]. +accept_pool(Config) when is_list(Config) -> + ClientOpts = ?config(client_opts, Config), + ServerOpts = ?config(server_opts, Config), + + {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), + Server0 = ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, + {from, self()}, + {accepters, 3}, + {mfa, {ssl_test_lib, send_recv_result_active, []}}, + {options, ServerOpts}]), + Port = ssl_test_lib:inet_port(Server0), + [Server1, Server2] = ssl_test_lib:accepters(2), + + Client0 = ssl_test_lib:start_client([{node, ClientNode}, {port, Port}, + {host, Hostname}, + {from, self()}, + {mfa, {ssl_test_lib, send_recv_result_active, []}}, + {options, ClientOpts} + ]), + + Client1 = ssl_test_lib:start_client([{node, ClientNode}, {port, Port}, + {host, Hostname}, + {from, self()}, + {mfa, {ssl_test_lib, send_recv_result_active, []}}, + {options, ClientOpts} + ]), + + Client2 = ssl_test_lib:start_client([{node, ClientNode}, {port, Port}, + {host, Hostname}, + {from, self()}, + {mfa, {ssl_test_lib, send_recv_result_active, []}}, + {options, ClientOpts} + ]), + + ssl_test_lib:check_ok([Server0, Server1, Server2, Client0, Client1, Client2]), + + ssl_test_lib:close(Server0), + ssl_test_lib:close(Server1), + ssl_test_lib:close(Server2), + ssl_test_lib:close(Client0), + ssl_test_lib:close(Client1), + ssl_test_lib:close(Client2). + %%-------------------------------------------------------------------- %% Internal functions ------------------------------------------------ @@ -3418,7 +3699,7 @@ run_suites(Ciphers, Version, Config, Type) -> Result = lists:map(fun(Cipher) -> cipher(Cipher, Version, Config, ClientOpts, ServerOpts) end, - Ciphers), + ssl_test_lib:filter_suites(Ciphers)), case lists:flatten(Result) of [] -> ok; @@ -3469,6 +3750,10 @@ cipher(CipherSuite, Version, Config, ClientOpts, ServerOpts) -> connection_info_result(Socket) -> ssl:connection_info(Socket). +version_info_result(Socket) -> + {ok, {Version, _}} = ssl:connection_info(Socket), + {ok, Version}. + connect_dist_s(S) -> Msg = term_to_binary({erlang,term}), ok = ssl:send(S, Msg). @@ -3546,3 +3831,22 @@ version_option_test(Config, Version) -> ssl_test_lib:close(Server), ssl_test_lib:close(Client). + +try_recv_active(Socket) -> + ssl:send(Socket, "Hello world"), + {error, einval} = ssl:recv(Socket, 11), + ok. +try_recv_active_once(Socket) -> + {error, einval} = ssl:recv(Socket, 11), + ok. + +first_rsa_suite([{ecdhe_rsa, _, _} = Suite | _]) -> + Suite; +first_rsa_suite([{dhe_rsa, _, _} = Suite| _]) -> + Suite; +first_rsa_suite([{rsa, _, _} = Suite| _]) -> + Suite; +first_rsa_suite([_ | Rest]) -> + first_rsa_suite(Rest). + + diff --git a/lib/ssl/test/ssl_certificate_verify_SUITE.erl b/lib/ssl/test/ssl_certificate_verify_SUITE.erl index 14047c6e9c..b7864ba6e7 100644 --- a/lib/ssl/test/ssl_certificate_verify_SUITE.erl +++ b/lib/ssl/test/ssl_certificate_verify_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2012-2013. All Rights Reserved. +%% Copyright Ericsson AB 2012-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -58,6 +58,10 @@ tests() -> server_verify_none, server_require_peer_cert_ok, server_require_peer_cert_fail, + server_require_peer_cert_partial_chain, + server_require_peer_cert_allow_partial_chain, + server_require_peer_cert_do_not_allow_partial_chain, + server_require_peer_cert_partial_chain_fun_fail, verify_fun_always_run_client, verify_fun_always_run_server, cert_expired, @@ -143,8 +147,8 @@ server_verify_none() -> [{doc,"Test server option verify_none"}]. server_verify_none(Config) when is_list(Config) -> - ClientOpts = ?config(client_opts, Config), - ServerOpts = ?config(server_opts, Config), + ClientOpts = ?config(client_verification_opts, Config), + ServerOpts = ?config(server_verification_opts, Config), Active = ?config(active, Config), ReceiveFunction = ?config(receive_function, Config), @@ -261,6 +265,163 @@ server_require_peer_cert_fail(Config) when is_list(Config) -> end. %%-------------------------------------------------------------------- + +server_require_peer_cert_partial_chain() -> + [{doc, "Client sends an incompleate chain, by default not acceptable."}]. + +server_require_peer_cert_partial_chain(Config) when is_list(Config) -> + ServerOpts = [{verify, verify_peer}, {fail_if_no_peer_cert, true} + | ?config(server_verification_opts, Config)], + ClientOpts = ?config(client_verification_opts, Config), + {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), + + {ok, ClientCAs} = file:read_file(proplists:get_value(cacertfile, ClientOpts)), + [{_,RootCA,_}, {_, _, _}] = public_key:pem_decode(ClientCAs), + + + Server = ssl_test_lib:start_server_error([{node, ServerNode}, {port, 0}, + {from, self()}, + {mfa, {ssl_test_lib, no_result, []}}, + {options, [{active, false} | ServerOpts]}]), + Port = ssl_test_lib:inet_port(Server), + Client = ssl_test_lib:start_client_error([{node, ClientNode}, {port, Port}, + {host, Hostname}, + {from, self()}, + {mfa, {ssl_test_lib, no_result, []}}, + {options, [{active, false}, + {cacerts, [RootCA]} | + proplists:delete(cacertfile, ClientOpts)]}]), + receive + {Server, {error, {tls_alert, "unknown ca"}}} -> + receive + {Client, {error, {tls_alert, "unknown ca"}}} -> + ok; + {Client, {error, closed}} -> + ok + end + end. +%%-------------------------------------------------------------------- +server_require_peer_cert_allow_partial_chain() -> + [{doc, "Server trusts intermediat CA and accepts a partial chain. (partial_chain option)"}]. + +server_require_peer_cert_allow_partial_chain(Config) when is_list(Config) -> + ServerOpts = [{verify, verify_peer}, {fail_if_no_peer_cert, true} + | ?config(server_verification_opts, Config)], + ClientOpts = ?config(client_verification_opts, Config), + {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), + + {ok, ServerCAs} = file:read_file(proplists:get_value(cacertfile, ServerOpts)), + [{_,_,_}, {_, IntermidiateCA, _}] = public_key:pem_decode(ServerCAs), + + PartialChain = fun(CertChain) -> + case lists:member(IntermidiateCA, CertChain) of + true -> + {trusted_ca, IntermidiateCA}; + false -> + unknown_ca + end + end, + + Server = ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, + {from, self()}, + {mfa, {ssl_test_lib, send_recv_result_active, []}}, + {options, [{cacerts, [IntermidiateCA]}, + {partial_chain, PartialChain} | + proplists:delete(cacertfile, ServerOpts)]}]), + Port = ssl_test_lib:inet_port(Server), + Client = ssl_test_lib:start_client([{node, ClientNode}, {port, Port}, + {host, Hostname}, + {from, self()}, + {mfa, {ssl_test_lib, send_recv_result_active, []}}, + {options, ClientOpts}]), + ssl_test_lib:check_result(Server, ok, Client, ok), + ssl_test_lib:close(Server), + ssl_test_lib:close(Client). + + %%-------------------------------------------------------------------- +server_require_peer_cert_do_not_allow_partial_chain() -> + [{doc, "Server does not accept the chain sent by the client as ROOT CA is unkown, " + "and we do not choose to trust the intermediate CA. (partial_chain option)"}]. + +server_require_peer_cert_do_not_allow_partial_chain(Config) when is_list(Config) -> + ServerOpts = [{verify, verify_peer}, {fail_if_no_peer_cert, true} + | ?config(server_verification_opts, Config)], + ClientOpts = ?config(client_verification_opts, Config), + {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), + + {ok, ServerCAs} = file:read_file(proplists:get_value(cacertfile, ServerOpts)), + [{_,_,_}, {_, IntermidiateCA, _}] = public_key:pem_decode(ServerCAs), + + PartialChain = fun(_CertChain) -> + unknown_ca + end, + + Server = ssl_test_lib:start_server_error([{node, ServerNode}, {port, 0}, + {from, self()}, + {mfa, {ssl_test_lib, no_result, []}}, + {options, [{cacerts, [IntermidiateCA]}, + {partial_chain, PartialChain} | + proplists:delete(cacertfile, ServerOpts)]}]), + + Port = ssl_test_lib:inet_port(Server), + Client = ssl_test_lib:start_client_error([{node, ClientNode}, {port, Port}, + {host, Hostname}, + {from, self()}, + {mfa, {ssl_test_lib, no_result, []}}, + {options, ClientOpts}]), + + receive + {Server, {error, {tls_alert, "unknown ca"}}} -> + receive + {Client, {error, {tls_alert, "unknown ca"}}} -> + ok; + {Client, {error, closed}} -> + ok + end + end. + + %%-------------------------------------------------------------------- +server_require_peer_cert_partial_chain_fun_fail() -> + [{doc, "If parial_chain fun crashes, treat it as if it returned unkown_ca"}]. + +server_require_peer_cert_partial_chain_fun_fail(Config) when is_list(Config) -> + ServerOpts = [{verify, verify_peer}, {fail_if_no_peer_cert, true} + | ?config(server_verification_opts, Config)], + ClientOpts = ?config(client_verification_opts, Config), + {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), + + {ok, ServerCAs} = file:read_file(proplists:get_value(cacertfile, ServerOpts)), + [{_,_,_}, {_, IntermidiateCA, _}] = public_key:pem_decode(ServerCAs), + + PartialChain = fun(_CertChain) -> + ture = false %% crash on purpose + end, + + Server = ssl_test_lib:start_server_error([{node, ServerNode}, {port, 0}, + {from, self()}, + {mfa, {ssl_test_lib, no_result, []}}, + {options, [{cacerts, [IntermidiateCA]}, + {partial_chain, PartialChain} | + proplists:delete(cacertfile, ServerOpts)]}]), + + Port = ssl_test_lib:inet_port(Server), + Client = ssl_test_lib:start_client_error([{node, ClientNode}, {port, Port}, + {host, Hostname}, + {from, self()}, + {mfa, {ssl_test_lib, no_result, []}}, + {options, ClientOpts}]), + + receive + {Server, {error, {tls_alert, "unknown ca"}}} -> + receive + {Client, {error, {tls_alert, "unknown ca"}}} -> + ok; + {Client, {error, closed}} -> + ok + end + end. + +%%-------------------------------------------------------------------- verify_fun_always_run_client() -> [{doc,"Verify that user verify_fun is always run (for valid and valid_peer not only unknown_extension)"}]. @@ -434,10 +595,16 @@ cert_expired(Config) when is_list(Config) -> Client = ssl_test_lib:start_client_error([{node, ClientNode}, {port, Port}, {host, Hostname}, {from, self()}, - {options, [{verify, verify_peer} | ClientOpts]}]), - - ssl_test_lib:check_result(Server, {error, {tls_alert, "certificate expired"}}, - Client, {error, {tls_alert, "certificate expired"}}). + {options, [{verify, verify_peer} | ClientOpts]}]), + receive + {Client, {error, {tls_alert, "certificate expired"}}} -> + receive + {Server, {error, {tls_alert, "certificate expired"}}} -> + ok; + {Server, {error, closed}} -> + ok + end + end. two_digits_str(N) when N < 10 -> lists:flatten(io_lib:format("0~p", [N])); @@ -632,7 +799,7 @@ no_authority_key_identifier() -> no_authority_key_identifier(Config) when is_list(Config) -> ClientOpts = ?config(client_verification_opts, Config), - ServerOpts = ?config(server_opts, Config), + ServerOpts = ?config(server_verification_opts, Config), PrivDir = ?config(priv_dir, Config), KeyFile = filename:join(PrivDir, "otpCA/private/key.pem"), @@ -804,7 +971,7 @@ unknown_server_ca_fail() -> [{doc,"Test that the client fails if the ca is unknown in verify_peer mode"}]. unknown_server_ca_fail(Config) when is_list(Config) -> ClientOpts = ?config(client_opts, Config), - ServerOpts = ?config(server_opts, Config), + ServerOpts = ?config(server_verification_opts, Config), {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), Server = ssl_test_lib:start_server_error([{node, ServerNode}, {port, 0}, {from, self()}, @@ -833,11 +1000,11 @@ unknown_server_ca_fail(Config) when is_list(Config) -> {verify_fun, FunAndState} | ClientOpts]}]), receive - {Server, {error, {tls_alert, "unknown ca"}}} -> + {Client, {error, {tls_alert, "unknown ca"}}} -> receive - {Client, {error, {tls_alert, "unknown ca"}}} -> + {Server, {error, {tls_alert, "unknown ca"}}} -> ok; - {Client, {error, closed}} -> + {Server, {error, closed}} -> ok end end. @@ -848,7 +1015,7 @@ unknown_server_ca_accept_verify_none() -> [{doc,"Test that the client succeds if the ca is unknown in verify_none mode"}]. unknown_server_ca_accept_verify_none(Config) when is_list(Config) -> ClientOpts = ?config(client_opts, Config), - ServerOpts = ?config(server_opts, Config), + ServerOpts = ?config(server_verification_opts, Config), {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), Server = ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, {from, self()}, @@ -873,7 +1040,7 @@ unknown_server_ca_accept_verify_peer() -> " with a verify_fun that accepts the unknown ca error"}]. unknown_server_ca_accept_verify_peer(Config) when is_list(Config) -> ClientOpts = ?config(client_opts, Config), - ServerOpts = ?config(server_opts, Config), + ServerOpts = ?config(server_verification_opts, Config), {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), Server = ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, {from, self()}, @@ -912,7 +1079,7 @@ unknown_server_ca_accept_backwardscompatibility() -> [{doc,"Test that old style verify_funs will work"}]. unknown_server_ca_accept_backwardscompatibility(Config) when is_list(Config) -> ClientOpts = ?config(client_opts, Config), - ServerOpts = ?config(server_opts, Config), + ServerOpts = ?config(server_verification_opts, Config), {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), Server = ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, {from, self()}, diff --git a/lib/ssl/test/ssl_crl_SUITE.erl b/lib/ssl/test/ssl_crl_SUITE.erl new file mode 100644 index 0000000000..bad0949ec4 --- /dev/null +++ b/lib/ssl/test/ssl_crl_SUITE.erl @@ -0,0 +1,542 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2008-2013. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% + +-module(ssl_crl_SUITE). + +%% Note: This directive should only be used in test suites. +-compile(export_all). + +-include_lib("common_test/include/ct.hrl"). +-include_lib("public_key/include/public_key.hrl"). + +-define(TIMEOUT, 120000). +-define(LONG_TIMEOUT, 600000). +-define(SLEEP, 1000). +-define(OPENSSL_RENEGOTIATE, "R\n"). +-define(OPENSSL_QUIT, "Q\n"). +-define(OPENSSL_GARBAGE, "P\n"). +-define(EXPIRE, 10). + +%%-------------------------------------------------------------------- +%% Common Test interface functions ----------------------------------- +%%-------------------------------------------------------------------- + +suite() -> [{ct_hooks,[ts_install_cth]}]. + +all() -> + [ + {group, basic}, + {group, v1_crl}, + {group, idp_crl} + ]. + +groups() -> + [{basic, [], basic_tests()}, + {v1_crl, [], v1_crl_tests()}, + {idp_crl, [], idp_crl_tests()}]. + +basic_tests() -> + [crl_verify_valid, crl_verify_revoked]. + +v1_crl_tests() -> + [crl_verify_valid, crl_verify_revoked]. + +idp_crl_tests() -> + [crl_verify_valid, crl_verify_revoked]. + +%%%================================================================ +%%% Suite init/end + +init_per_suite(Config0) -> + Dog = ct:timetrap(?LONG_TIMEOUT *2), + case os:find_executable("openssl") of + false -> + {skip, "Openssl not found"}; + _ -> + TLSVersion = ?config(tls_version, Config0), + OpenSSL_version = (catch os:cmd("openssl version")), + ct:log("TLS version: ~p~nOpenSSL version: ~p~n~n~p:module_info(): ~p~n~nssl:module_info(): ~p~n", + [TLSVersion, OpenSSL_version, ?MODULE, ?MODULE:module_info(), ssl:module_info()]), + case ssl_test_lib:enough_openssl_crl_support(OpenSSL_version) of + false -> + {skip, io_lib:format("Bad openssl version: ~p",[OpenSSL_version])}; + _ -> + catch crypto:stop(), + try crypto:start() of + ok -> + ssl:start(), + {ok, Hostname0} = inet:gethostname(), + IPfamily = + case lists:member(list_to_atom(Hostname0), ct:get_config(ipv6_hosts,[])) of + true -> inet6; + false -> inet + end, + [{ipfamily,IPfamily}, {watchdog, Dog}, {openssl_version,OpenSSL_version} | Config0] + catch _C:_E -> + ct:log("crypto:start() caught ~p:~p",[_C,_E]), + {skip, "Crypto did not start"} + end + end + end. + +end_per_suite(_Config) -> + ssl:stop(), + application:stop(crypto). + +%%%================================================================ +%%% Group init/end + +init_per_group(Group, Config) -> + ssl:start(), + inets:start(), + CertDir = filename:join(?config(priv_dir, Config), Group), + DataDir = ?config(data_dir, Config), + ServerRoot = make_dir_path([?config(priv_dir,Config), Group, tmp]), + %% start a HTTP server to serve the CRLs + {ok, Httpd} = inets:start(httpd, [{ipfamily, ?config(ipfamily,Config)}, + {server_name, "localhost"}, {port, 0}, + {server_root, ServerRoot}, + {document_root, CertDir}, + {modules, [mod_get]} + ]), + [{port,Port}] = httpd:info(Httpd, [port]), + ct:log("~p:~p~nHTTPD IP family=~p, port=~p~n", [?MODULE, ?LINE, ?config(ipfamily,Config), Port]), + CertOpts = [{crl_port,Port}|cert_opts(Group)], + Result = make_certs:all(DataDir, CertDir, CertOpts), + ct:log("~p:~p~nmake_certs:all(~n DataDir=~p,~n CertDir=~p,~n ServerRoot=~p~n Opts=~p~n) returned ~p~n", [?MODULE,?LINE,DataDir, CertDir, ServerRoot, CertOpts, Result]), + [{make_cert_result, Result}, {cert_dir, CertDir}, {httpd, Httpd} | Config]. + +cert_opts(v1_crl) -> [{v2_crls, false}]; +cert_opts(idp_crl) -> [{issuing_distribution_point, true}]; +cert_opts(_) -> []. + +make_dir_path(PathComponents) -> + lists:foldl(fun(F,P0) -> file:make_dir(P=filename:join(P0,F)), P end, + "", + PathComponents). + + +end_per_group(_GroupName, Config) -> + case ?config(httpd, Config) of + undefined -> ok; + Pid -> + ct:log("Stop httpd ~p",[Pid]), + ok = inets:stop(httpd, Pid) + ,ct:log("Stopped",[]) + end, + inets:stop(), + Config. + +%%%================================================================ +%%% Test cases + +crl_verify_valid() -> + [{doc,"Verify a simple valid CRL chain"}]. +crl_verify_valid(Config) when is_list(Config) -> + process_flag(trap_exit, true), + PrivDir = ?config(cert_dir, Config), + ServerOpts = [{keyfile, filename:join([PrivDir, "server", "key.pem"])}, + {certfile, filename:join([PrivDir, "server", "cert.pem"])}, + {cacertfile, filename:join([PrivDir, "server", "cacerts.pem"])}], + + {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), + Data = "From openssl to erlang", + + Server = ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, + {from, self()}, + {mfa, {?MODULE, erlang_ssl_receive, [Data]}}, + %{mfa, {ssl_test_lib, no_result, []}}, + {options, ServerOpts}]), + ct:log("~p:~p~nreturn from ssl_test_lib:start_server:~n~p",[?MODULE,?LINE,Server]), + Port = ssl_test_lib:inet_port(Server), + + CACerts = load_cert(filename:join([PrivDir, "erlangCA", "cacerts.pem"])), + + ClientOpts = [{cacerts, CACerts}, + {verify, verify_peer}, + {verify_fun, {fun validate_function/3, {CACerts, []}}}], + + + ct:log("~p:~p~ncalling ssl_test_lib:start_client",[?MODULE,?LINE]), + Client = ssl_test_lib:start_client([{node, ClientNode}, {port, Port}, + {host, Hostname}, + {from, self()}, + {mfa, {?MODULE, + erlang_ssl_send, [Data]}}, + %{mfa, {ssl_test_lib, no_result, []}}, + {options, ClientOpts}]), + ct:log("~p:~p~nreturn from ssl_test_lib:start_client:~n~p",[?MODULE,?LINE,Client]), + + ssl_test_lib:check_result(Client, ok, Server, ok), + + %% Clean close down! Server needs to be closed first !! + ssl_test_lib:close(Server), + ssl_test_lib:close(Client), + process_flag(trap_exit, false). + +crl_verify_revoked() -> + [{doc,"Verify a simple valid CRL chain"}]. +crl_verify_revoked(Config) when is_list(Config) -> + process_flag(trap_exit, true), + PrivDir = ?config(cert_dir, Config), + ServerOpts = [{keyfile, filename:join([PrivDir, "revoked", "key.pem"])}, + {certfile, filename:join([PrivDir, "revoked", "cert.pem"])}, + {cacertfile, filename:join([PrivDir, "revoked", "cacerts.pem"])}], + ct:log("~p:~p~nserver opts ~p~n", [?MODULE,?LINE, ServerOpts]), + + {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), + + Server = ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, + {from, self()}, + %{mfa, {?MODULE, erlang_ssl_receive, [Data]}}, + {mfa, {ssl_test_lib, no_result, []}}, + {options, ServerOpts}]), + Port = ssl_test_lib:inet_port(Server), + + CACerts = load_cert(filename:join([PrivDir, "erlangCA", "cacerts.pem"])), + ClientOpts = [{cacerts, CACerts}, + {verify, verify_peer}, + {verify_fun, {fun validate_function/3, {CACerts, []}}}], + + {connect_failed, _} = ssl_test_lib:start_client([{node, ClientNode}, {port, Port}, + {host, Hostname}, + {from, self()}, + %{mfa, {?MODULE, + %erlang_ssl_receive, [Data]}}, + {mfa, {ssl_test_lib, no_result, []}}, + {options, ClientOpts}]), + + %% Clean close down! Server needs to be closed first !! + ssl_test_lib:close(Server), + process_flag(trap_exit, false). + +%%%================================================================ +%%% Lib + +erlang_ssl_receive(Socket, Data) -> + ct:log("~p:~p~nConnection info: ~p~n", + [?MODULE,?LINE, ssl:connection_info(Socket)]), + receive + {ssl, Socket, Data} -> + ct:log("~p:~p~nReceived ~p~n",[?MODULE,?LINE, Data]), + %% open_ssl server sometimes hangs waiting in blocking read + ssl:send(Socket, "Got it"), + ok; + {ssl, Socket, Byte} when length(Byte) == 1 -> + erlang_ssl_receive(Socket, tl(Data)); + {Port, {data,Debug}} when is_port(Port) -> + ct:log("~p:~p~nopenssl ~s~n",[?MODULE,?LINE, Debug]), + erlang_ssl_receive(Socket,Data); + Other -> + ct:fail({unexpected_message, Other}) + after 4000 -> + ct:fail({did_not_get, Data}) + end. + + +erlang_ssl_send(Socket, Data) -> + ct:log("~p:~p~nConnection info: ~p~n", + [?MODULE,?LINE, ssl:connection_info(Socket)]), + ssl:send(Socket, Data), + ok. + +load_certs(undefined) -> + undefined; +load_certs(CertDir) -> + case file:list_dir(CertDir) of + {ok, Certs} -> + load_certs(lists:map(fun(Cert) -> filename:join(CertDir, Cert) + end, Certs), []); + {error, _} -> + undefined + end. + +load_certs([], Acc) -> + ct:log("~p:~p~nSuccessfully loaded ~p CA certificates~n", [?MODULE,?LINE, length(Acc)]), + Acc; +load_certs([Cert|Certs], Acc) -> + case filelib:is_dir(Cert) of + true -> + load_certs(Certs, Acc); + _ -> + %ct:log("~p:~p~nLoading certificate ~p~n", [?MODULE,?LINE, Cert]), + load_certs(Certs, load_cert(Cert) ++ Acc) + end. + +load_cert(Cert) -> + {ok, Bin} = file:read_file(Cert), + case filename:extension(Cert) of + ".der" -> + %% no decoding necessary + [Bin]; + _ -> + %% assume PEM otherwise + Contents = public_key:pem_decode(Bin), + [DER || {Type, DER, Cipher} <- Contents, Type == 'Certificate', Cipher == 'not_encrypted'] + end. + +%% @doc Validator function for SSL negotiation. +%% +validate_function(Cert, valid_peer, State) -> + ct:log("~p:~p~nvaliding peer ~p with ~p intermediate certs~n", + [?MODULE,?LINE, get_common_name(Cert), + length(element(2, State))]), + %% peer certificate validated, now check the CRL + Res = (catch check_crl(Cert, State)), + ct:log("~p:~p~nCRL validate result for ~p: ~p~n", + [?MODULE,?LINE, get_common_name(Cert), Res]), + {Res, State}; +validate_function(Cert, valid, {TrustedCAs, IntermediateCerts}=State) -> + case public_key:pkix_is_self_signed(Cert) of + true -> + ct:log("~p:~p~nroot certificate~n",[?MODULE,?LINE]), + %% this is a root cert, no CRL + {valid, {TrustedCAs, [Cert|IntermediateCerts]}}; + false -> + %% check is valid CA certificate, add to the list of + %% intermediates + Res = (catch check_crl(Cert, State)), + ct:log("~p:~p~nCRL intermediate CA validate result for ~p: ~p~n", + [?MODULE,?LINE, get_common_name(Cert), Res]), + {Res, {TrustedCAs, [Cert|IntermediateCerts]}} + end; +validate_function(_Cert, _Event, State) -> + %ct:log("~p:~p~nignoring event ~p~n", [?MODULE,?LINE, _Event]), + {valid, State}. + +%% @doc Given a certificate, find CRL distribution points for the given +%% certificate, fetch, and attempt to validate each CRL through +%% issuer_function/4. +%% +check_crl(Cert, State) -> + %% pull the CRL distribution point(s) out of the certificate, if any + ct:log("~p:~p~ncheck_crl(~n Cert=~p,~nState=~p~n)",[?MODULE,?LINE,Cert,State]), + case pubkey_cert:select_extension( + ?'id-ce-cRLDistributionPoints', + pubkey_cert:extensions_list(Cert#'OTPCertificate'.tbsCertificate#'OTPTBSCertificate'.extensions)) of + undefined -> + ct:log("~p:~p~nno CRL distribution points for ~p~n", + [?MODULE,?LINE, get_common_name(Cert)]), + %% fail; we can't validate if there's no CRL + no_crl; + CRLExtension -> + ct:log("~p:~p~nCRLExtension=~p)",[?MODULE,?LINE,CRLExtension]), + CRLDistPoints = CRLExtension#'Extension'.extnValue, + DPointsAndCRLs = lists:foldl(fun(Point, Acc) -> + %% try to read the CRL over http or from a + %% local file + case fetch_point(Point) of + not_available -> + ct:log("~p:~p~nfetch_point returned~n~p~n)",[?MODULE,?LINE,not_available]), + Acc; + Res -> + ct:log("~p:~p~nfetch_point returned~n~p~n)",[?MODULE,?LINE,Res]), + [{Point, Res} | Acc] + end + end, [], CRLDistPoints), + public_key:pkix_crls_validate(Cert, + DPointsAndCRLs, + [{issuer_fun, + {fun issuer_function/4, State}}]) + end. + +%% @doc Given a list of distribution points for CRLs, certificates and +%% both trusted and intermediary certificates, attempt to build and +%% authority chain back via build_chain to verify that it is valid. +%% +issuer_function(_DP, CRL, _Issuer, {TrustedCAs, IntermediateCerts}) -> + %% XXX the 'Issuer' we get passed here is the AuthorityKeyIdentifier, + %% which we are not currently smart enough to understand + %% Read the CA certs out of the file + ct:log("~p:~p~nissuer_function(~nCRL=~p,~nLast param=~p)",[?MODULE,?LINE,CRL, {TrustedCAs, IntermediateCerts}]), + Certs = [public_key:pkix_decode_cert(DER, otp) || DER <- TrustedCAs], + %% get the real issuer out of the CRL + Issuer = public_key:pkix_normalize_name( + pubkey_cert_records:transform( + CRL#'CertificateList'.tbsCertList#'TBSCertList'.issuer, decode)), + %% assume certificates are ordered from root to tip + case find_issuer(Issuer, IntermediateCerts ++ Certs) of + undefined -> + ct:log("~p:~p~nunable to find certificate matching CRL issuer ~p~n", + [?MODULE,?LINE, Issuer]), + error; + IssuerCert -> + ct:log("~p:~p~nIssuerCert=~p~n)",[?MODULE,?LINE,IssuerCert]), + case build_chain({public_key:pkix_encode('OTPCertificate', + IssuerCert, + otp), + IssuerCert}, IntermediateCerts, Certs, []) of + undefined -> + error; + {OTPCert, Path} -> + {ok, OTPCert, Path} + end + end. + +%% @doc Attempt to build authority chain back using intermediary +%% certificates, falling back on trusted certificates if the +%% intermediary chain of certificates does not fully extend to the +%% root. +%% +%% Returns: {RootCA :: #OTPCertificate{}, Chain :: [der_encoded()]} +%% +build_chain({DER, Cert}, IntCerts, TrustedCerts, Acc) -> + %% check if this cert is self-signed, if it is, we've reached the + %% root of the chain + Issuer = public_key:pkix_normalize_name( + Cert#'OTPCertificate'.tbsCertificate#'OTPTBSCertificate'.issuer), + Subject = public_key:pkix_normalize_name( + Cert#'OTPCertificate'.tbsCertificate#'OTPTBSCertificate'.subject), + case Issuer == Subject of + true -> + case find_issuer(Issuer, TrustedCerts) of + undefined -> + ct:log("~p:~p~nself-signed certificate is NOT trusted~n",[?MODULE,?LINE]), + undefined; + TrustedCert -> + %% return the cert from the trusted list, to prevent + %% issuer spoofing + {TrustedCert, + [public_key:pkix_encode( + 'OTPCertificate', TrustedCert, otp)|Acc]} + end; + false -> + Match = lists:foldl( + fun(C, undefined) -> + S = public_key:pkix_normalize_name(C#'OTPCertificate'.tbsCertificate#'OTPTBSCertificate'.subject), + %% compare the subject to the current issuer + case Issuer == S of + true -> + %% we've found our man + {public_key:pkix_encode('OTPCertificate', C, otp), C}; + false -> + undefined + end; + (_E, A) -> + %% already matched + A + end, undefined, IntCerts), + case Match of + undefined when IntCerts /= TrustedCerts -> + %% continue the chain by using the trusted CAs + ct:log("~p:~p~nRan out of intermediate certs, switching to trusted certs~n",[?MODULE,?LINE]), + build_chain({DER, Cert}, TrustedCerts, TrustedCerts, Acc); + undefined -> + ct:log("Can't construct chain of trust beyond ~p~n", + [?MODULE,?LINE, get_common_name(Cert)]), + %% can't find the current cert's issuer + undefined; + Match -> + build_chain(Match, IntCerts, TrustedCerts, [DER|Acc]) + end + end. + +%% @doc Given a certificate and a list of trusted or intermediary +%% certificates, attempt to find a match in the list or bail with +%% undefined. +find_issuer(Issuer, Certs) -> + lists:foldl( + fun(OTPCert, undefined) -> + %% check if this certificate matches the issuer + Normal = public_key:pkix_normalize_name( + OTPCert#'OTPCertificate'.tbsCertificate#'OTPTBSCertificate'.subject), + case Normal == Issuer of + true -> + OTPCert; + false -> + undefined + end; + (_E, Acc) -> + %% already found a match + Acc + end, undefined, Certs). + +%% @doc Find distribution points for a given CRL and then attempt to +%% fetch the CRL from the first available. +fetch_point(#'DistributionPoint'{distributionPoint={fullName, Names}}) -> + Decoded = [{NameType, + pubkey_cert_records:transform(Name, decode)} + || {NameType, Name} <- Names], + ct:log("~p:~p~ncall fetch(~nDecoded=~p~n)",[?MODULE,?LINE,Decoded]), + fetch(Decoded). + +%% @doc Given a list of locations to retrieve a CRL from, attempt to +%% retrieve either from a file or http resource and bail as soon as +%% it can be found. +%% +%% Currently, only hand a armored PEM or DER encoded file, with +%% defaulting to DER. +%% +fetch([]) -> + not_available; +fetch([{uniformResourceIdentifier, "http"++_=URL}|Rest]) -> + ct:log("~p:~p~ngetting CRL from ~p~n", [?MODULE,?LINE, URL]), + case httpc:request(get, {URL, []}, [], [{body_format, binary}]) of + {ok, {_Status, _Headers, Body}} -> + case Body of + <<"-----BEGIN", _/binary>> -> + ct:log("~p:~p~npublic_key:pem_decode,~nBody=~p~n)",[?MODULE,?LINE,Body]), + [{'CertificateList', + DER, _}=CertList] = public_key:pem_decode(Body), + ct:log("~p:~p~npublic_key:pem_entry_decode,~nCertList=~p~n)",[?MODULE,?LINE,CertList]), + {DER, public_key:pem_entry_decode(CertList)}; + _ -> + ct:log("~p:~p~npublic_key:pem_entry_decode,~nBody=~p~n)",[?MODULE,?LINE,{'CertificateList', Body, not_encrypted}]), + %% assume DER encoded + try + public_key:pem_entry_decode({'CertificateList', Body, not_encrypted}) + of + CertList -> {Body, CertList} + catch + _C:_E -> + ct:log("~p:~p~nfailed DER assumption~nRest=~p", [?MODULE,?LINE,Rest]), + fetch(Rest) + end + end; + {error, _Reason} -> + ct:log("~p:~p~nfailed to get CRL ~p~n", [?MODULE,?LINE, _Reason]), + fetch(Rest); + Other -> + ct:log("~p:~p~nreally failed to get CRL ~p~n", [?MODULE,?LINE, Other]), + fetch(Rest) + end; +fetch([Loc|Rest]) -> + %% unsupported CRL location + ct:log("~p:~p~nunable to fetch CRL from unsupported location ~p~n", + [?MODULE,?LINE, Loc]), + fetch(Rest). + +%% get the common name attribute out of an OTPCertificate record +get_common_name(OTPCert) -> + %% You'd think there'd be an easier way than this giant mess, but I + %% couldn't find one. + {rdnSequence, Subject} = OTPCert#'OTPCertificate'.tbsCertificate#'OTPTBSCertificate'.subject, + case [Attribute#'AttributeTypeAndValue'.value || [Attribute] <- Subject, + Attribute#'AttributeTypeAndValue'.type == ?'id-at-commonName'] of + [Att] -> + case Att of + {teletexString, Str} -> Str; + {printableString, Str} -> Str; + {utf8String, Bin} -> binary_to_list(Bin) + end; + _ -> + unknown + end. + diff --git a/lib/ssl/test/ssl_handshake_SUITE.erl b/lib/ssl/test/ssl_handshake_SUITE.erl index 6d020c472b..e5e942ce1b 100644 --- a/lib/ssl/test/ssl_handshake_SUITE.erl +++ b/lib/ssl/test/ssl_handshake_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2013. All Rights Reserved. +%% Copyright Ericsson AB 2008-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -26,6 +26,7 @@ -include_lib("common_test/include/ct.hrl"). -include("ssl_internal.hrl"). -include("tls_handshake.hrl"). +-include_lib("public_key/include/public_key.hrl"). %%-------------------------------------------------------------------- %% Common Test interface functions ----------------------------------- @@ -36,7 +37,9 @@ all() -> [decode_hello_handshake, decode_single_hello_extension_correctly, decode_supported_elliptic_curves_hello_extension_correctly, decode_unknown_hello_extension_correctly, - encode_single_hello_sni_extension_correctly]. + encode_single_hello_sni_extension_correctly, + decode_single_hello_sni_extension_correctly, + select_proper_tls_1_2_rsa_default_hashsign]. %%-------------------------------------------------------------------- %% Test Cases -------------------------------------------------------- @@ -95,3 +98,18 @@ encode_single_hello_sni_extension_correctly(_Config) -> HelloExt = <<ExtSize:16/unsigned-big-integer, SNI/binary>>, Encoded = ssl_handshake:encode_hello_extensions(Exts), HelloExt = Encoded. + +decode_single_hello_sni_extension_correctly(_Config) -> + Exts = #hello_extensions{sni = #sni{hostname = "test.com"}}, + SNI = <<16#00, 16#00, 16#00, 16#0d, 16#00, 16#0b, 16#00, 16#00, 16#08, + $t, $e, $s, $t, $., $c, $o, $m>>, + Decoded = ssl_handshake:decode_hello_extensions(SNI), + Exts = Decoded. + +select_proper_tls_1_2_rsa_default_hashsign(_Config) -> + % RFC 5246 section 7.4.1.4.1 tells to use {sha1,rsa} as default signature_algorithm for RSA key exchanges + {sha, rsa} = ssl_handshake:select_hashsign_algs(undefined, ?rsaEncryption, {3,3}), + % Older versions use MD5/SHA1 combination + {md5sha, rsa} = ssl_handshake:select_hashsign_algs(undefined, ?rsaEncryption, {3,2}), + {md5sha, rsa} = ssl_handshake:select_hashsign_algs(undefined, ?rsaEncryption, {3,0}). + diff --git a/lib/ssl/test/ssl_test_lib.erl b/lib/ssl/test/ssl_test_lib.erl index 7ed9adfcd9..74d71263de 100644 --- a/lib/ssl/test/ssl_test_lib.erl +++ b/lib/ssl/test/ssl_test_lib.erl @@ -60,14 +60,23 @@ run_server(Opts) -> Options = proplists:get_value(options, Opts), Pid = proplists:get_value(from, Opts), Transport = proplists:get_value(transport, Opts, ssl), - ct:log("ssl:listen(~p, ~p)~n", [Port, Options]), + ct:log("~p:~p~nssl:listen(~p, ~p)~n", [?MODULE,?LINE, Port, Options]), {ok, ListenSocket} = rpc:call(Node, Transport, listen, [Port, Options]), Pid ! {listen, up}, send_selected_port(Pid, Port, ListenSocket), run_server(ListenSocket, Opts). run_server(ListenSocket, Opts) -> - do_run_server(ListenSocket, connect(ListenSocket, Opts), Opts). + Accepters = proplists:get_value(accepters, Opts, 1), + run_server(ListenSocket, Opts, Accepters). + +run_server(ListenSocket, Opts, 1) -> + do_run_server(ListenSocket, connect(ListenSocket, Opts), Opts); +run_server(ListenSocket, Opts, N) -> + Pid = proplists:get_value(from, Opts), + Server = spawn(?MODULE, run_server, [ListenSocket, Opts, 1]), + Pid ! {accepter, N, Server}, + run_server(ListenSocket, Opts, N-1). do_run_server(_, {error, timeout} = Result, Opts) -> Pid = proplists:get_value(from, Opts), @@ -78,13 +87,13 @@ do_run_server(ListenSocket, AcceptSocket, Opts) -> Pid = proplists:get_value(from, Opts), Transport = proplists:get_value(transport, Opts, ssl), {Module, Function, Args} = proplists:get_value(mfa, Opts), - ct:log("Server: apply(~p,~p,~p)~n", - [Module, Function, [AcceptSocket | Args]]), + ct:log("~p:~p~nServer: apply(~p,~p,~p)~n", + [?MODULE,?LINE, Module, Function, [AcceptSocket | Args]]), case rpc:call(Node, Module, Function, [AcceptSocket | Args]) of no_result_msg -> ok; Msg -> - ct:log("Server Msg: ~p ~n", [Msg]), + ct:log("~p:~p~nServer Msg: ~p ~n", [?MODULE,?LINE, Msg]), Pid ! {self(), Msg} end, receive @@ -93,10 +102,10 @@ do_run_server(ListenSocket, AcceptSocket, Opts) -> {listen, MFA} -> run_server(ListenSocket, [MFA | proplists:delete(mfa, Opts)]); close -> - ct:log("Server closing ~p ~n", [self()]), + ct:log("~p:~p~nServer closing ~p ~n", [?MODULE,?LINE, self()]), Result = rpc:call(Node, Transport, close, [AcceptSocket], 500), Result1 = rpc:call(Node, Transport, close, [ListenSocket], 500), - ct:log("Result ~p : ~p ~n", [Result, Result1]); + ct:log("~p:~p~nResult ~p : ~p ~n", [?MODULE,?LINE, Result, Result1]); {ssl_closed, _} -> ok end. @@ -106,7 +115,8 @@ connect(#sslsocket{} = ListenSocket, Opts) -> Node = proplists:get_value(node, Opts), ReconnectTimes = proplists:get_value(reconnect_times, Opts, 0), Timeout = proplists:get_value(timeout, Opts, infinity), - AcceptSocket = connect(ListenSocket, Node, 1 + ReconnectTimes, dummy, Timeout), + SslOpts = proplists:get_value(ssl_extra_opts, Opts, []), + AcceptSocket = connect(ListenSocket, Node, 1 + ReconnectTimes, dummy, Timeout, SslOpts), case ReconnectTimes of 0 -> AcceptSocket; @@ -116,27 +126,35 @@ connect(#sslsocket{} = ListenSocket, Opts) -> end; connect(ListenSocket, Opts) -> Node = proplists:get_value(node, Opts), - ct:log("gen_tcp:accept(~p)~n", [ListenSocket]), + ct:log("~p:~p~ngen_tcp:accept(~p)~n", [?MODULE,?LINE, ListenSocket]), {ok, AcceptSocket} = rpc:call(Node, gen_tcp, accept, [ListenSocket]), AcceptSocket. -connect(_, _, 0, AcceptSocket, _) -> +connect(_, _, 0, AcceptSocket, _, _) -> AcceptSocket; -connect(ListenSocket, Node, N, _, Timeout) -> + +connect(ListenSocket, Node, N, _, Timeout, []) -> ct:log("ssl:transport_accept(~p)~n", [ListenSocket]), {ok, AcceptSocket} = rpc:call(Node, ssl, transport_accept, [ListenSocket]), - ct:log("ssl:ssl_accept(~p, ~p)~n", [AcceptSocket, Timeout]), + ct:log("~p:~p~nssl:ssl_accept(~p, ~p)~n", [?MODULE,?LINE, AcceptSocket, Timeout]), case rpc:call(Node, ssl, ssl_accept, [AcceptSocket, Timeout]) of ok -> - connect(ListenSocket, Node, N-1, AcceptSocket, Timeout); + connect(ListenSocket, Node, N-1, AcceptSocket, Timeout, []); Result -> + ct:log("~p:~p~nssl:ssl_accept@~p ret ~p",[?MODULE,?LINE, Node,Result]), Result - end. + end; +connect(ListenSocket, Node, _, _, Timeout, Opts) -> + ct:log("ssl:transport_accept(~p)~n", [ListenSocket]), + {ok, AcceptSocket} = rpc:call(Node, ssl, transport_accept, + [ListenSocket]), + ct:log("ssl:ssl_accept(~p,~p, ~p)~n", [AcceptSocket, Opts, Timeout]), + rpc:call(Node, ssl, ssl_accept, [AcceptSocket, Opts, Timeout]), + AcceptSocket. - remove_close_msg(0) -> ok; remove_close_msg(ReconnectTimes) -> @@ -148,11 +166,13 @@ remove_close_msg(ReconnectTimes) -> start_client(Args) -> Result = spawn_link(?MODULE, run_client_init, [lists:delete(return_socket, Args)]), receive - { connected, Socket } -> - case lists:member(return_socket, Args) of - true -> { Result, Socket }; - false -> Result - end + {connected, Socket} -> + case lists:member(return_socket, Args) of + true -> {Result, Socket}; + false -> Result + end; + {connect_failed, Reason} -> + {connect_failed, Reason} end. run_client_init(Opts) -> @@ -166,27 +186,27 @@ run_client(Opts) -> Pid = proplists:get_value(from, Opts), Transport = proplists:get_value(transport, Opts, ssl), Options = proplists:get_value(options, Opts), - ct:log("ssl:connect(~p, ~p, ~p)~n", [Host, Port, Options]), + ct:log("~p:~p~n~p:connect(~p, ~p)@~p~n", [?MODULE,?LINE, Transport, Host, Port, Node]), case rpc:call(Node, Transport, connect, [Host, Port, Options]) of {ok, Socket} -> - Pid ! { connected, Socket }, - ct:log("Client: connected~n", []), + Pid ! {connected, Socket}, + ct:log("~p:~p~nClient: connected~n", [?MODULE,?LINE]), %% In special cases we want to know the client port, it will %% be indicated by sending {port, 0} in options list! send_selected_port(Pid, proplists:get_value(port, Options), Socket), {Module, Function, Args} = proplists:get_value(mfa, Opts), - ct:log("Client: apply(~p,~p,~p)~n", - [Module, Function, [Socket | Args]]), + ct:log("~p:~p~nClient: apply(~p,~p,~p)~n", + [?MODULE,?LINE, Module, Function, [Socket | Args]]), case rpc:call(Node, Module, Function, [Socket | Args]) of no_result_msg -> ok; Msg -> - ct:log("Client Msg: ~p ~n", [Msg]), + ct:log("~p:~p~nClient Msg: ~p ~n", [?MODULE,?LINE, Msg]), Pid ! {self(), Msg} end, receive close -> - ct:log("Client closing~n", []), + ct:log("~p:~p~nClient closing~n", [?MODULE,?LINE]), rpc:call(Node, Transport, close, [Socket]); {ssl_closed, Socket} -> ok; @@ -196,50 +216,42 @@ run_client(Opts) -> {error, econnrefused = Reason} -> case get(retries) of N when N < 5 -> + ct:log("~p:~p~neconnrefused retries=~p sleep ~p",[?MODULE,?LINE, N,?SLEEP]), put(retries, N+1), ct:sleep(?SLEEP), run_client(Opts); _ -> - ct:log("Client faild several times: connection failed: ~p ~n", [Reason]), + ct:log("~p:~p~nClient faild several times: connection failed: ~p ~n", [?MODULE,?LINE, Reason]), Pid ! {self(), {error, Reason}} end; {error, Reason} -> - ct:log("Client: connection failed: ~p ~n", [Reason]), - Pid ! {self(), {error, Reason}} + ct:log("~p:~p~nClient: connection failed: ~p ~n", [?MODULE,?LINE, Reason]), + Pid ! {connect_failed, Reason}; + {badrpc,BadRPC} -> + ct:log("~p:~p~nBad rpc: ~p",[?MODULE,?LINE, BadRPC]), + Pid ! {connect_failed, {badrpc,BadRPC}} end. close(Pid) -> - ct:log("Close ~p ~n", [Pid]), + ct:log("~p:~p~nClose ~p ~n", [?MODULE,?LINE, Pid]), Monitor = erlang:monitor(process, Pid), Pid ! close, receive {'DOWN', Monitor, process, Pid, Reason} -> erlang:demonitor(Monitor), - ct:log("Pid: ~p down due to:~p ~n", [Pid, Reason]) + ct:log("~p:~p~nPid: ~p down due to:~p ~n", [?MODULE,?LINE, Pid, Reason]) end. check_result(Server, ServerMsg, Client, ClientMsg) -> receive - {Server, ServerMsg} -> - receive - {Client, ClientMsg} -> - ok; - Unexpected -> - Reason = {{expected, {Client, ClientMsg}}, - {got, Unexpected}}, - ct:fail(Reason) - end; - {Client, ClientMsg} -> - receive - {Server, ServerMsg} -> - ok; - Unexpected -> - Reason = {{expected, {Server, ClientMsg}}, - {got, Unexpected}}, - ct:fail(Reason) - end; + {Server, ServerMsg} -> + check_result(Client, ClientMsg); + + {Client, ClientMsg} -> + check_result(Server, ServerMsg); + {Port, {data,Debug}} when is_port(Port) -> - io:format("openssl ~s~n",[Debug]), + ct:log("~p:~p~nopenssl ~s~n",[?MODULE,?LINE, Debug]), check_result(Server, ServerMsg, Client, ClientMsg); Unexpected -> @@ -253,7 +265,7 @@ check_result(Pid, Msg) -> {Pid, Msg} -> ok; {Port, {data,Debug}} when is_port(Port) -> - io:format("openssl ~s~n",[Debug]), + ct:log("~p:~p~nopenssl ~s~n",[?MODULE,?LINE, Debug]), check_result(Pid,Msg); Unexpected -> Reason = {{expected, {Pid, Msg}}, @@ -278,19 +290,28 @@ wait_for_result(Server, ServerMsg, Client, ClientMsg) -> %% Unexpected end; {Port, {data,Debug}} when is_port(Port) -> - io:format("openssl ~s~n",[Debug]), + ct:log("~p:~p~nopenssl ~s~n",[?MODULE,?LINE, Debug]), wait_for_result(Server, ServerMsg, Client, ClientMsg) %% Unexpected -> %% Unexpected end. - +check_ok([]) -> + ok; +check_ok(Pids) -> + receive + {Pid, ok} -> + check_ok(lists:delete(Pid, Pids)); + Other -> + ct:fail({expected, {"pid()", ok}, got, Other}) + end. + wait_for_result(Pid, Msg) -> receive {Pid, Msg} -> ok; {Port, {data,Debug}} when is_port(Port) -> - io:format("openssl ~s~n",[Debug]), + ct:log("~p:~p~nopenssl ~s~n",[?MODULE,?LINE, Debug]), wait_for_result(Pid,Msg) %% Unexpected -> %% Unexpected @@ -429,7 +450,7 @@ make_ecdsa_cert(Config) -> {cacertfile, ServerCaCertFile}, {certfile, ServerCertFile}, {keyfile, ServerKeyFile}]}, {server_ecdsa_verify_opts, [{ssl_imp, new},{reuseaddr, true}, - {cacertfile, ClientCaCertFile}, + {cacertfile, ServerCaCertFile}, {certfile, ServerCertFile}, {keyfile, ServerKeyFile}, {verify, verify_peer}]}, {client_ecdsa_opts, [{ssl_imp, new},{reuseaddr, true}, @@ -454,7 +475,7 @@ make_ecdh_rsa_cert(Config) -> {cacertfile, ServerCaCertFile}, {certfile, ServerCertFile}, {keyfile, ServerKeyFile}]}, {server_ecdh_rsa_verify_opts, [{ssl_imp, new},{reuseaddr, true}, - {cacertfile, ClientCaCertFile}, + {cacertfile, ServerCaCertFile}, {certfile, ServerCertFile}, {keyfile, ServerKeyFile}, {verify, verify_peer}]}, {client_ecdh_rsa_opts, [{ssl_imp, new},{reuseaddr, true}, @@ -515,33 +536,33 @@ run_upgrade_server(Opts) -> SslOptions = proplists:get_value(ssl_options, Opts), Pid = proplists:get_value(from, Opts), - ct:log("gen_tcp:listen(~p, ~p)~n", [Port, TcpOptions]), + ct:log("~p:~p~ngen_tcp:listen(~p, ~p)~n", [?MODULE,?LINE, Port, TcpOptions]), {ok, ListenSocket} = rpc:call(Node, gen_tcp, listen, [Port, TcpOptions]), Pid ! {listen, up}, send_selected_port(Pid, Port, ListenSocket), - ct:log("gen_tcp:accept(~p)~n", [ListenSocket]), + ct:log("~p:~p~ngen_tcp:accept(~p)~n", [?MODULE,?LINE, ListenSocket]), {ok, AcceptSocket} = rpc:call(Node, gen_tcp, accept, [ListenSocket]), try {ok, SslAcceptSocket} = case TimeOut of infinity -> - ct:log("ssl:ssl_accept(~p, ~p)~n", - [AcceptSocket, SslOptions]), + ct:log("~p:~p~nssl:ssl_accept(~p, ~p)~n", + [?MODULE,?LINE, AcceptSocket, SslOptions]), rpc:call(Node, ssl, ssl_accept, [AcceptSocket, SslOptions]); _ -> - ct:log("ssl:ssl_accept(~p, ~p, ~p)~n", - [AcceptSocket, SslOptions, TimeOut]), + ct:log("~p:~p~nssl:ssl_accept(~p, ~p, ~p)~n", + [?MODULE,?LINE, AcceptSocket, SslOptions, TimeOut]), rpc:call(Node, ssl, ssl_accept, [AcceptSocket, SslOptions, TimeOut]) end, {Module, Function, Args} = proplists:get_value(mfa, Opts), Msg = rpc:call(Node, Module, Function, [SslAcceptSocket | Args]), - ct:log("Upgrade Server Msg: ~p ~n", [Msg]), + ct:log("~p:~p~nUpgrade Server Msg: ~p ~n", [?MODULE,?LINE, Msg]), Pid ! {self(), Msg}, receive close -> - ct:log("Upgrade Server closing~n", []), + ct:log("~p:~p~nUpgrade Server closing~n", [?MODULE,?LINE]), rpc:call(Node, ssl, close, [SslAcceptSocket]) end catch error:{badmatch, Error} -> @@ -559,24 +580,24 @@ run_upgrade_client(Opts) -> TcpOptions = proplists:get_value(tcp_options, Opts), SslOptions = proplists:get_value(ssl_options, Opts), - ct:log("gen_tcp:connect(~p, ~p, ~p)~n", - [Host, Port, TcpOptions]), + ct:log("~p:~p~ngen_tcp:connect(~p, ~p, ~p)~n", + [?MODULE,?LINE, Host, Port, TcpOptions]), {ok, Socket} = rpc:call(Node, gen_tcp, connect, [Host, Port, TcpOptions]), send_selected_port(Pid, Port, Socket), - ct:log("ssl:connect(~p, ~p)~n", [Socket, SslOptions]), + ct:log("~p:~p~nssl:connect(~p, ~p)~n", [?MODULE,?LINE, Socket, SslOptions]), {ok, SslSocket} = rpc:call(Node, ssl, connect, [Socket, SslOptions]), {Module, Function, Args} = proplists:get_value(mfa, Opts), - ct:log("apply(~p, ~p, ~p)~n", - [Module, Function, [SslSocket | Args]]), + ct:log("~p:~p~napply(~p, ~p, ~p)~n", + [?MODULE,?LINE, Module, Function, [SslSocket | Args]]), Msg = rpc:call(Node, Module, Function, [SslSocket | Args]), - ct:log("Upgrade Client Msg: ~p ~n", [Msg]), + ct:log("~p:~p~nUpgrade Client Msg: ~p ~n", [?MODULE,?LINE, Msg]), Pid ! {self(), Msg}, receive close -> - ct:log("Upgrade Client closing~n", []), + ct:log("~p:~p~nUpgrade Client closing~n", [?MODULE,?LINE]), rpc:call(Node, ssl, close, [SslSocket]) end. @@ -595,21 +616,21 @@ run_upgrade_server_error(Opts) -> SslOptions = proplists:get_value(ssl_options, Opts), Pid = proplists:get_value(from, Opts), - ct:log("gen_tcp:listen(~p, ~p)~n", [Port, TcpOptions]), + ct:log("~p:~p~ngen_tcp:listen(~p, ~p)~n", [?MODULE,?LINE, Port, TcpOptions]), {ok, ListenSocket} = rpc:call(Node, gen_tcp, listen, [Port, TcpOptions]), Pid ! {listen, up}, send_selected_port(Pid, Port, ListenSocket), - ct:log("gen_tcp:accept(~p)~n", [ListenSocket]), + ct:log("~p:~p~ngen_tcp:accept(~p)~n", [?MODULE,?LINE, ListenSocket]), {ok, AcceptSocket} = rpc:call(Node, gen_tcp, accept, [ListenSocket]), Error = case TimeOut of infinity -> - ct:log("ssl:ssl_accept(~p, ~p)~n", - [AcceptSocket, SslOptions]), + ct:log("~p:~p~nssl:ssl_accept(~p, ~p)~n", + [?MODULE,?LINE, AcceptSocket, SslOptions]), rpc:call(Node, ssl, ssl_accept, [AcceptSocket, SslOptions]); _ -> - ct:log("ssl:ssl_accept(~p, ~p, ~p)~n", - [AcceptSocket, SslOptions, TimeOut]), + ct:log("~p:~p~nssl:ssl_accept(~p, ~p, ~p)~n", + [?MODULE,?LINE, AcceptSocket, SslOptions, TimeOut]), rpc:call(Node, ssl, ssl_accept, [AcceptSocket, SslOptions, TimeOut]) end, @@ -628,26 +649,26 @@ run_server_error(Opts) -> Options = proplists:get_value(options, Opts), Pid = proplists:get_value(from, Opts), Transport = proplists:get_value(transport, Opts, ssl), - ct:log("ssl:listen(~p, ~p)~n", [Port, Options]), + ct:log("~p:~p~nssl:listen(~p, ~p)~n", [?MODULE,?LINE, Port, Options]), case rpc:call(Node, Transport, listen, [Port, Options]) of {ok, #sslsocket{} = ListenSocket} -> %% To make sure error_client will %% get {error, closed} and not {error, connection_refused} Pid ! {listen, up}, send_selected_port(Pid, Port, ListenSocket), - ct:log("ssl:transport_accept(~p)~n", [ListenSocket]), + ct:log("~p:~p~nssl:transport_accept(~p)~n", [?MODULE,?LINE, ListenSocket]), case rpc:call(Node, Transport, transport_accept, [ListenSocket]) of {error, _} = Error -> Pid ! {self(), Error}; {ok, AcceptSocket} -> - ct:log("ssl:ssl_accept(~p)~n", [AcceptSocket]), + ct:log("~p:~p~nssl:ssl_accept(~p)~n", [?MODULE,?LINE, AcceptSocket]), Error = rpc:call(Node, ssl, ssl_accept, [AcceptSocket]), Pid ! {self(), Error} end; {ok, ListenSocket} -> Pid ! {listen, up}, send_selected_port(Pid, Port, ListenSocket), - ct:log("~p:accept(~p)~n", [Transport, ListenSocket]), + ct:log("~p:~p~n~p:accept(~p)~n", [?MODULE,?LINE, Transport, ListenSocket]), case rpc:call(Node, Transport, accept, [ListenSocket]) of {error, _} = Error -> Pid ! {self(), Error} @@ -669,10 +690,21 @@ run_client_error(Opts) -> Pid = proplists:get_value(from, Opts), Transport = proplists:get_value(transport, Opts, ssl), Options = proplists:get_value(options, Opts), - ct:log("ssl:connect(~p, ~p, ~p)~n", [Host, Port, Options]), + ct:log("~p:~p~nssl:connect(~p, ~p, ~p)~n", [?MODULE,?LINE, Host, Port, Options]), Error = rpc:call(Node, Transport, connect, [Host, Port, Options]), Pid ! {self(), Error}. +accepters(N) -> + accepters([], N). + +accepters(Acc, 0) -> + Acc; +accepters(Acc, N) -> + receive + {accepter, _, Server} -> + accepters([Server| Acc], N-1) + end. + inet_port(Pid) when is_pid(Pid)-> receive {Pid, {port, Port}} -> @@ -840,25 +872,34 @@ psk_suites() -> {psk, '3des_ede_cbc', sha}, {psk, aes_128_cbc, sha}, {psk, aes_256_cbc, sha}, + {psk, aes_128_cbc, sha256}, + {psk, aes_256_cbc, sha384}, {dhe_psk, rc4_128, sha}, {dhe_psk, '3des_ede_cbc', sha}, {dhe_psk, aes_128_cbc, sha}, {dhe_psk, aes_256_cbc, sha}, + {dhe_psk, aes_128_cbc, sha256}, + {dhe_psk, aes_256_cbc, sha384}, {rsa_psk, rc4_128, sha}, {rsa_psk, '3des_ede_cbc', sha}, {rsa_psk, aes_128_cbc, sha}, - {rsa_psk, aes_256_cbc, sha}], + {rsa_psk, aes_256_cbc, sha}, + {rsa_psk, aes_128_cbc, sha256}, + {rsa_psk, aes_256_cbc, sha384} +], ssl_cipher:filter_suites(Suites). psk_anon_suites() -> - [{psk, rc4_128, sha}, - {psk, '3des_ede_cbc', sha}, - {psk, aes_128_cbc, sha}, - {psk, aes_256_cbc, sha}, - {dhe_psk, rc4_128, sha}, - {dhe_psk, '3des_ede_cbc', sha}, - {dhe_psk, aes_128_cbc, sha}, - {dhe_psk, aes_256_cbc, sha}]. + Suites = + [{psk, rc4_128, sha}, + {psk, '3des_ede_cbc', sha}, + {psk, aes_128_cbc, sha}, + {psk, aes_256_cbc, sha}, + {dhe_psk, rc4_128, sha}, + {dhe_psk, '3des_ede_cbc', sha}, + {dhe_psk, aes_128_cbc, sha}, + {dhe_psk, aes_256_cbc, sha}], + ssl_cipher:filter_suites(Suites). srp_suites() -> Suites = @@ -871,9 +912,11 @@ srp_suites() -> ssl_cipher:filter_suites(Suites). srp_anon_suites() -> - [{srp_anon, '3des_ede_cbc', sha}, - {srp_anon, aes_128_cbc, sha}, - {srp_anon, aes_256_cbc, sha}]. + Suites = + [{srp_anon, '3des_ede_cbc', sha}, + {srp_anon, aes_128_cbc, sha}, + {srp_anon, aes_256_cbc, sha}], + ssl_cipher:filter_suites(Suites). srp_dss_suites() -> Suites = @@ -892,7 +935,7 @@ der_to_pem(File, Entries) -> cipher_result(Socket, Result) -> Result = ssl:connection_info(Socket), - ct:log("Successfull connect: ~p~n", [Result]), + ct:log("~p:~p~nSuccessfull connect: ~p~n", [?MODULE,?LINE, Result]), %% Importante to send two packets here %% to properly test "cipher state" handling ssl:send(Socket, "Hello\n"), @@ -1061,10 +1104,13 @@ check_sane_openssl_version(Version) -> true end. +enough_openssl_crl_support("OpenSSL 0." ++ _) -> false; +enough_openssl_crl_support(_) -> true. + wait_for_openssl_server() -> receive {Port, {data, Debug}} when is_port(Port) -> - ct:log("openssl ~s~n",[Debug]), + ct:log("~p:~p~nopenssl ~s~n",[?MODULE,?LINE, Debug]), %% openssl has started make sure %% it will be in accept. Parsing %% output is too error prone. (Even @@ -1080,3 +1126,46 @@ version_flag('tlsv1.2') -> " -tls1_2 "; version_flag(sslv3) -> " -ssl3 ". + +filter_suites(Ciphers0) -> + Version = tls_record:highest_protocol_version([]), + Supported0 = ssl_cipher:suites(Version) + ++ ssl_cipher:anonymous_suites() + ++ ssl_cipher:psk_suites(Version) + ++ ssl_cipher:srp_suites(), + Supported1 = ssl_cipher:filter_suites(Supported0), + Supported2 = [ssl:suite_definition(S) || S <- Supported1], + [Cipher || Cipher <- Ciphers0, lists:member(Cipher, Supported2)]. + +-define(OPENSSL_QUIT, "Q\n"). +close_port(Port) -> + catch port_command(Port, ?OPENSSL_QUIT), + close_loop(Port, 500, false). + +close_loop(Port, Time, SentClose) -> + receive + {Port, {data,Debug}} when is_port(Port) -> + ct:log("openssl ~s~n",[Debug]), + close_loop(Port, Time, SentClose); + {ssl,_,Msg} -> + ct:log("ssl Msg ~s~n",[Msg]), + close_loop(Port, Time, SentClose); + {Port, closed} -> + ct:log("Port Closed~n",[]), + ok; + {'EXIT', Port, Reason} -> + ct:log("Port Closed ~p~n",[Reason]), + ok; + Msg -> + ct:log("Port Msg ~p~n",[Msg]), + close_loop(Port, Time, SentClose) + after Time -> + case SentClose of + false -> + ct:log("Closing port ~n",[]), + catch erlang:port_close(Port), + close_loop(Port, Time, true); + true -> + ct:log("Timeout~n",[]) + end + end. diff --git a/lib/ssl/test/ssl_to_openssl_SUITE.erl b/lib/ssl/test/ssl_to_openssl_SUITE.erl index a7361755e5..942c446ec4 100644 --- a/lib/ssl/test/ssl_to_openssl_SUITE.erl +++ b/lib/ssl/test/ssl_to_openssl_SUITE.erl @@ -226,7 +226,7 @@ basic_erlang_client_openssl_server(Config) when is_list(Config) -> ssl_test_lib:check_result(Client, ok), %% Clean close down! Server needs to be closed first !! - close_port(OpensslPort), + ssl_test_lib:close_port(OpensslPort), ssl_test_lib:close(Client), process_flag(trap_exit, false). @@ -259,7 +259,7 @@ basic_erlang_server_openssl_client(Config) when is_list(Config) -> %% Clean close down! Server needs to be closed first !! ssl_test_lib:close(Server), - close_port(OpenSslPort), + ssl_test_lib:close_port(OpenSslPort), process_flag(trap_exit, false), ok. %%-------------------------------------------------------------------- @@ -298,7 +298,7 @@ erlang_client_openssl_server(Config) when is_list(Config) -> ssl_test_lib:check_result(Client, ok), %% Clean close down! Server needs to be closed first !! - close_port(OpensslPort), + ssl_test_lib:close_port(OpensslPort), ssl_test_lib:close(Client), process_flag(trap_exit, false). @@ -332,11 +332,9 @@ erlang_server_openssl_client(Config) when is_list(Config) -> %% Clean close down! Server needs to be closed first !! ssl_test_lib:close(Server), - close_port(OpenSslPort), + ssl_test_lib:close_port(OpenSslPort), process_flag(trap_exit, false). -%%-------------------------------------------------------------------- - erlang_client_openssl_server_dsa_cert() -> [{doc,"Test erlang server with openssl client"}]. erlang_client_openssl_server_dsa_cert(Config) when is_list(Config) -> @@ -376,7 +374,7 @@ erlang_client_openssl_server_dsa_cert(Config) when is_list(Config) -> ssl_test_lib:check_result(Client, ok), %% Clean close down! Server needs to be closed first !! - close_port(OpensslPort), + ssl_test_lib:close_port(OpensslPort), ssl_test_lib:close(Client), process_flag(trap_exit, false), ok. @@ -414,7 +412,7 @@ erlang_server_openssl_client_dsa_cert(Config) when is_list(Config) -> %% Clean close down! Server needs to be closed first !! ssl_test_lib:close(Server), - close_port(OpenSslPort), + ssl_test_lib:close_port(OpenSslPort), process_flag(trap_exit, false). %%-------------------------------------------------------------------- @@ -450,7 +448,7 @@ erlang_server_openssl_client_reuse_session(Config) when is_list(Config) -> %% Clean close down! Server needs to be closed first !! ssl_test_lib:close(Server), - close_port(OpenSslPort), + ssl_test_lib:close_port(OpenSslPort), process_flag(trap_exit, false), ok. @@ -496,7 +494,7 @@ erlang_client_openssl_server_renegotiate(Config) when is_list(Config) -> ssl_test_lib:check_result(Client, ok), %% Clean close down! Server needs to be closed first !! - close_port(OpensslPort), + ssl_test_lib:close_port(OpensslPort), ssl_test_lib:close(Client), process_flag(trap_exit, false), ok. @@ -542,7 +540,7 @@ erlang_client_openssl_server_nowrap_seqnum(Config) when is_list(Config) -> ssl_test_lib:check_result(Client, ok), %% Clean close down! Server needs to be closed first !! - close_port(OpensslPort), + ssl_test_lib:close_port(OpensslPort), ssl_test_lib:close(Client), process_flag(trap_exit, false). %%-------------------------------------------------------------------- @@ -581,7 +579,7 @@ erlang_server_openssl_client_nowrap_seqnum(Config) when is_list(Config) -> %% Clean close down! Server needs to be closed first !! ssl_test_lib:close(Server), - close_port(OpenSslPort), + ssl_test_lib:close_port(OpenSslPort), process_flag(trap_exit, false). %%-------------------------------------------------------------------- @@ -624,7 +622,7 @@ erlang_client_openssl_server_no_server_ca_cert(Config) when is_list(Config) -> ssl_test_lib:check_result(Client, ok), %% Clean close down! Server needs to be closed first !! - close_port(OpensslPort), + ssl_test_lib:close_port(OpensslPort), ssl_test_lib:close(Client), process_flag(trap_exit, false). @@ -666,7 +664,7 @@ erlang_client_openssl_server_client_cert(Config) when is_list(Config) -> ssl_test_lib:check_result(Client, ok), %% Clean close down! Server needs to be closed first !! - close_port(OpensslPort), + ssl_test_lib:close_port(OpensslPort), ssl_test_lib:close(Client), process_flag(trap_exit, false). @@ -708,7 +706,7 @@ erlang_server_openssl_client_client_cert(Config) when is_list(Config) -> ssl_test_lib:check_result(Server, ok), %% Clean close down! Server needs to be closed first !! - close_port(OpenSslPort), + ssl_test_lib:close_port(OpenSslPort), ssl_test_lib:close(Server), process_flag(trap_exit, false). @@ -821,7 +819,7 @@ erlang_client_bad_openssl_server(Config) when is_list(Config) -> [{versions, [Version]} | ClientOpts]}]), %% Clean close down! Server needs to be closed first !! - close_port(OpensslPort), + ssl_test_lib:close_port(OpensslPort), ssl_test_lib:close(Client1), process_flag(trap_exit, false), ok. @@ -878,7 +876,7 @@ expired_session(Config) when is_list(Config) -> {from, self()}, {options, ClientOpts}]), %% Clean close down! Server needs to be closed first !! - close_port(OpensslPort), + ssl_test_lib:close_port(OpensslPort), ssl_test_lib:close(Client2), process_flag(trap_exit, false). @@ -1089,7 +1087,7 @@ cipher(CipherSuite, Version, Config, ClientOpts, ServerOpts) -> Result = ssl_test_lib:wait_for_result(Client, ok), %% Clean close down! Server needs to be closed first !! - close_port(OpenSslPort), + ssl_test_lib:close_port(OpenSslPort), ssl_test_lib:close(Client), Return = case Result of @@ -1136,7 +1134,7 @@ start_erlang_client_and_openssl_server_with_opts(Config, ErlangClientOpts, Opens Callback(Client, OpensslPort), %% Clean close down! Server needs to be closed first !! - close_port(OpensslPort), + ssl_test_lib:close_port(OpensslPort), ssl_test_lib:close(Client), process_flag(trap_exit, false). @@ -1175,7 +1173,7 @@ start_erlang_client_and_openssl_server_for_npn_negotiation(Config, Data, Callbac Callback(Client, OpensslPort), %% Clean close down! Server needs to be closed first !! - close_port(OpensslPort), + ssl_test_lib:close_port(OpensslPort), ssl_test_lib:close(Client), process_flag(trap_exit, false). @@ -1205,7 +1203,7 @@ start_erlang_server_and_openssl_client_for_npn_negotiation(Config, Data, Callbac ssl_test_lib:close(Server), - close_port(OpenSslPort), + ssl_test_lib:close_port(OpenSslPort), process_flag(trap_exit, false). @@ -1234,7 +1232,7 @@ start_erlang_server_and_openssl_client_with_opts(Config, ErlangServerOpts, OpenS ssl_test_lib:close(Server), - close_port(OpenSslPort), + ssl_test_lib:close_port(OpenSslPort), process_flag(trap_exit, false). @@ -1282,39 +1280,6 @@ delayed_send(Socket, [ErlData, OpenSslData]) -> ssl:send(Socket, ErlData), erlang_ssl_receive(Socket, OpenSslData). -close_port(Port) -> - catch port_command(Port, ?OPENSSL_QUIT), - close_loop(Port, 500, false). - -close_loop(Port, Time, SentClose) -> - receive - {Port, {data,Debug}} when is_port(Port) -> - ct:log("openssl ~s~n",[Debug]), - close_loop(Port, Time, SentClose); - {ssl,_,Msg} -> - ct:log("ssl Msg ~s~n",[Msg]), - close_loop(Port, Time, SentClose); - {Port, closed} -> - ct:log("Port Closed~n",[]), - ok; - {'EXIT', Port, Reason} -> - ct:log("Port Closed ~p~n",[Reason]), - ok; - Msg -> - ct:log("Port Msg ~p~n",[Msg]), - close_loop(Port, Time, SentClose) - after Time -> - case SentClose of - false -> - ct:log("Closing port ~n",[]), - catch erlang:port_close(Port), - close_loop(Port, Time, true); - true -> - ct:log("Timeout~n",[]) - end - end. - - server_sent_garbage(Socket) -> receive server_sent_garbage -> @@ -1341,7 +1306,7 @@ check_sane_openssl_renegotaite(Config, Version) when Version == 'tlsv1.1'; {skip, "Known renegotiation bug in OpenSSL"}; "OpenSSL 1.0.1a" ++ _ -> {skip, "Known renegotiation bug in OpenSSL"}; - "OpenSSL 1.0.1" ++ _ -> + "OpenSSL 1.0.1 " ++ _ -> {skip, "Known renegotiation bug in OpenSSL"}; _ -> check_sane_openssl_renegotaite(Config) diff --git a/lib/ssl/vsn.mk b/lib/ssl/vsn.mk index e08f5dff78..404b71374f 100644 --- a/lib/ssl/vsn.mk +++ b/lib/ssl/vsn.mk @@ -1 +1 @@ -SSL_VSN = 5.3.4 +SSL_VSN = 5.3.6 diff --git a/lib/stdlib/doc/src/erl_tar.xml b/lib/stdlib/doc/src/erl_tar.xml index f81e36f810..7f25f5b7bc 100644 --- a/lib/stdlib/doc/src/erl_tar.xml +++ b/lib/stdlib/doc/src/erl_tar.xml @@ -35,10 +35,11 @@ <modulesummary>Unix 'tar' utility for reading and writing tar archives</modulesummary> <description> <p>The <c>erl_tar</c> module archives and extract files to and from - a tar file. The tar file format is the POSIX extended tar file format - specified in IEEE Std 1003.1 and ISO/IEC 9945-1. That is the same - format as used by <c>tar</c> program on Solaris, but is not the same - as used by the GNU tar program.</p> + a tar file. <c>erl_tar</c> supports the <c>ustar</c> format + (IEEE Std 1003.1 and ISO/IEC 9945-1). All modern <c>tar</c> + programs (including GNU tar) can read this format. To ensure that + that GNU tar produces a tar file that <c>erl_tar</c> can read, + give the <c>--format=ustar</c> option to GNU tar.</p> <p>By convention, the name of a tar file should end in "<c>.tar</c>". To abide to the convention, you'll need to add "<c>.tar</c>" yourself to the name.</p> @@ -65,6 +66,20 @@ </description> <section> + <title>UNICODE SUPPORT</title> + <p>If <seealso + marker="kernel:file#native_name_encoding/0">file:native_name_encoding/0</seealso> + returns <c>utf8</c>, path names will be encoded in UTF-8 when + creating tar files and path names will be assumed to be encoded in + UTF-8 when extracting tar files.</p> + + <p>If <seealso + marker="kernel:file#native_name_encoding/0">file:native_name_encoding/0</seealso> + returns <c>latin1</c>, no translation of path names will be + done.</p> + </section> + + <section> <title>LIMITATIONS</title> <p>For maximum compatibility, it is safe to archive files with names up to 100 characters in length. Such tar files can generally be @@ -112,8 +127,8 @@ <fsummary>Add a file to an open tar file</fsummary> <type> <v>TarDescriptor = term()</v> - <v>FilenameOrBin = Filename()|binary()</v> - <v>Filename = filename()()</v> + <v>FilenameOrBin = filename()|binary()</v> + <v>Filename = filename()</v> <v>NameInArchive = filename()</v> <v>Options = [Option]</v> <v>Option = dereference|verbose</v> diff --git a/lib/stdlib/doc/src/gen_fsm.xml b/lib/stdlib/doc/src/gen_fsm.xml index 1713367bd8..848d57f3e6 100644 --- a/lib/stdlib/doc/src/gen_fsm.xml +++ b/lib/stdlib/doc/src/gen_fsm.xml @@ -115,7 +115,7 @@ gen_fsm:sync_send_all_state_event -----> Module:handle_sync_event/4 If <c>FsmName={global,GlobalName}</c>, the gen_fsm is registered globally as <c>GlobalName</c> using <c>global:register_name/2</c>. - If <c>EventMgrName={via,Module,ViaName}</c>, the event manager will + If <c>FsmName={via,Module,ViaName}</c>, the gen_fsm will register with the registry represented by <c>Module</c>. The <c>Module</c> callback should export the functions <c>register_name/2</c>, <c>unregister_name/1</c>, @@ -210,7 +210,7 @@ gen_fsm:sync_send_all_state_event -----> Module:handle_sync_event/4 registered at another node, or</item> <item><c>{global,GlobalName}</c>, if the gen_fsm is globally registered.</item> - <item><c>{via,Module,ViaName}</c>, if the event manager is registered + <item><c>{via,Module,ViaName}</c>, if the gen_fsm is registered through an alternative process registry.</item> </list> <p><c>Event</c> is an arbitrary term which is passed as one of diff --git a/lib/stdlib/doc/src/gen_server.xml b/lib/stdlib/doc/src/gen_server.xml index 4c83fde237..62c0394479 100644 --- a/lib/stdlib/doc/src/gen_server.xml +++ b/lib/stdlib/doc/src/gen_server.xml @@ -113,7 +113,7 @@ gen_server:abcast -----> Module:handle_cast/2 registered globally as <c>GlobalName</c> using <c>global:register_name/2</c>. If no name is provided, the gen_server is not registered. - If <c>EventMgrName={via,Module,ViaName}</c>, the event manager will + If <c>ServerName={via,Module,ViaName}</c>, the gen_server will register with the registry represented by <c>Module</c>. The <c>Module</c> callback should export the functions <c>register_name/2</c>, <c>unregister_name/1</c>, diff --git a/lib/stdlib/doc/src/maps.xml b/lib/stdlib/doc/src/maps.xml index 76137e3dee..64229fa8d3 100644 --- a/lib/stdlib/doc/src/maps.xml +++ b/lib/stdlib/doc/src/maps.xml @@ -108,6 +108,26 @@ </func> <func> + <name name="get" arity="3"/> + <fsummary></fsummary> + <desc> + <p> + Returns the value <c><anno>Value</anno></c> associated with <c><anno>Key</anno></c> if + <c><anno>Map</anno></c> contains <c><anno>Key</anno></c>. + If no value is associated with <c><anno>Key</anno></c> then returns <c><anno>Default</anno></c>. + </p> + <p>Example:</p> + <code type="none"> +> Map = #{ key1 => val1, key2 => val2 }. +#{key1 => val1,key2 => val2} +> maps:get(key1, Map, "Default value"). +val1 +> maps:get(key3, Map, "Default value"). +"Default value"</code> + </desc> + </func> + + <func> <name name="is_key" arity="2"/> <fsummary></fsummary> <desc> @@ -299,6 +319,23 @@ false</code> </func> <func> + <name name="with" arity="2"/> + <fsummary></fsummary> + <desc> + <p> + Returns a new map <c><anno>Map2</anno></c> with the keys <c>K1</c> through <c>Kn</c> and their associated values from map <c><anno>Map1</anno></c>. + Any key in <c><anno>Ks</anno></c> that does not exist in <c><anno>Map1</anno></c> are ignored. + </p> + <p>Example:</p> + <code type="none"> +> Map = #{42 => value_three,1337 => "value two","a" => 1}, + Ks = ["a",42,"other key"], + maps:without(Ks,Map). +#{42 => value_three,"a" => 1}</code> + </desc> + </func> + + <func> <name name="without" arity="2"/> <fsummary></fsummary> <desc> diff --git a/lib/stdlib/doc/src/notes.xml b/lib/stdlib/doc/src/notes.xml index e94a4d6a55..ebc750a399 100644 --- a/lib/stdlib/doc/src/notes.xml +++ b/lib/stdlib/doc/src/notes.xml @@ -30,6 +30,563 @@ </header> <p>This document describes the changes made to the STDLIB application.</p> +<section><title>STDLIB 2.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + The type spec of the FormFunc argument to + sys:handle_debug/4 was erroneously pointing to dbg_fun(). + This is now corrected and the new type is format_fun().</p> + <p> + Own Id: OTP-11800</p> + </item> + <item> + <p> + Behaviors such as gen_fsm and gen_server should always + invoke format_status/2 before printing the state to the + logs.</p> + <p> + Own Id: OTP-11967</p> + </item> + <item> + <p> The documentation of <c>dets:insert_new/2</c> has + been corrected. (Thanks to Alexei Sholik for reporting + the bug.) </p> + <p> + Own Id: OTP-12024</p> + </item> + <item> + <p> + Printing a term with io_lib:format and control sequence + w, precision P and field width F, where F< P would + fail in one of the two following ways:</p> + <p> + 1) If P < printed length of the term, an infinite loop + would be entered, consuming all available memory.</p> + <p> + 2) If P >= printed length of the term, an exception + would be raised.</p> + <p> + These two problems are now corrected.</p> + <p> + Own Id: OTP-12041</p> + </item> + <item> + <p> + The documentation of <c>maps:values/1</c> has been + corrected.</p> + <p> + Own Id: OTP-12055</p> + </item> + <item> + <p> + Expand shell functions in map expressions.</p> + <p> + Own Id: OTP-12063</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Add maps:with/2</p> + <p> + Own Id: OTP-12137</p> + </item> + </list> + </section> + +</section> + +<section><title>STDLIB 2.1.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + OTP-11850 fixed filelib:wildcard/1 to work with broken + symlinks. This correction, however, introduced problems + since symlinks were no longer followed for functions like + filelib:ensure_dir/1, filelib:is_dir/1, + filelib:file_size/1, etc. This is now corrected.</p> + <p> + Own Id: OTP-12054 Aux Id: seq12660 </p> + </item> + </list> + </section> + +</section> + +<section><title>STDLIB 2.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p><c>filelib:wildcard("broken_symlink")</c> would return + an empty list if "broken_symlink" was a symlink that did + not point to an existing file.</p> + <p> + Own Id: OTP-11850 Aux Id: seq12571 </p> + </item> + <item> + <p><c>erl_tar</c> can now handle files names that contain + Unicode characters. See "UNICODE SUPPORT" in the + documentation for <c>erl_tar</c>.</p> + <p>When creating a tar file, <c>erl_tar</c> would + sometime write a too short end of tape marker. GNU tar + would correctly extract files from such tar file, but + would complain about "A lone zero block at...".</p> + <p> + Own Id: OTP-11854</p> + </item> + <item> + <p> When redefining and exporting the type <c>map()</c> + the Erlang Code Linter (<c>erl_lint</c>) erroneously + emitted an error. This bug has been fixed. </p> + <p> + Own Id: OTP-11872</p> + </item> + <item> + <p> + Fix evaluation of map updates in the debugger and + erl_eval</p> + <p> + Reported-by: José Valim</p> + <p> + Own Id: OTP-11922</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p>The following native functions now bump an appropriate + amount of reductions and yield when out of + reductions:</p> <list> + <item><c>erlang:binary_to_list/1</c></item> + <item><c>erlang:binary_to_list/3</c></item> + <item><c>erlang:bitstring_to_list/1</c></item> + <item><c>erlang:list_to_binary/1</c></item> + <item><c>erlang:iolist_to_binary/1</c></item> + <item><c>erlang:list_to_bitstring/1</c></item> + <item><c>binary:list_to_bin/1</c></item> </list> + <p>Characteristics impact:</p> <taglist> + <tag>Performance</tag> <item>The functions converting + from lists got a performance loss for very small lists, + and a performance gain for very large lists.</item> + <tag>Priority</tag> <item>Previously a process executing + one of these functions effectively got an unfair priority + boost. This priority boost depended on the input size. + The larger the input was, the larger the priority boost + got. This unfair priority boost is now lost. </item> + </taglist> + <p> + Own Id: OTP-11888</p> + </item> + <item> + <p> + Add <c>maps:get/3</c> to maps module. The function will + return the supplied default value if the key does not + exist in the map.</p> + <p> + Own Id: OTP-11951</p> + </item> + </list> + </section> + +</section> + +<section><title>STDLIB 2.0</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + The option dupnames did not work as intended in re. When + looking for names with {capture, [Name, ...]}, re:run + returned a random instance of the match for that name, + instead of the leftmost matching instance, which was what + the documentation stated. This is now corrected to adhere + to the documentation. The option {capture,all_names} + along with a re:inspect/2 function is also added to + further help in using named subpatterns.</p> + <p> + Own Id: OTP-11205</p> + </item> + <item> + <p> + If option 'binary' was set for standard_input, then c:i() + would hang if the output was more than one page long - + i.e. then input after "(c)ontinue (q)uit -->" could + not be read. This has been corrected. (Thanks to José + Valim)</p> + <p> + Own Id: OTP-11589</p> + </item> + <item> + <p> + stdlib/lists: Add function droplast/1 This functions + drops the last element of a non-empty list. lists:last/1 + and lists:droplast/1 are the dual of hd/1 and tl/1 but + for the end of a list. (Thanks to Hans Svensson)</p> + <p> + Own Id: OTP-11677</p> + </item> + <item> + <p> + Allow all auto imports to be suppressed at once. + Introducing the no_auto_import attribute: + -compile(no_auto_import). Useful for code generation + tools that always use the qualified function names and + want to avoid the auto imported functions clashing with + local ones. (Thanks to José Valim.)</p> + <p> + Own Id: OTP-11682</p> + </item> + <item> + <p> + supervisor_bridge does no longer report normal + termination of children. The reason is that in some + cases, for instance when the restart strategy is + simple_one_for_one, the log could be completely + overloaded with reports about normally terminating + processes. (Thanks to Artem Ocheredko)</p> + <p> + Own Id: OTP-11685</p> + </item> + <item> + <p> The type annotations for alternative registries using + the {via, Module, Name} syntax for sup_name() and + sup_ref() in the supervisor module are now consistent + with the documentation. Dialyzer should no longer + complain about valid supervisor:start_link() and + supervisor:start_child() calls. (Thanks to Caleb + Helbling.) </p> + <p> + Own Id: OTP-11707</p> + </item> + <item> + <p> Two Dets bugs have been fixed. When trying to open a + short file that is not a Dets file, the file was deleted + even with just read access. Calling + <c>dets:is_dets_file/1</c> with a file that is not a Dets + file, a file descriptor was left open. (Thanks to Håkan + Mattsson for reporting the bugs.) </p> + <p> + Own Id: OTP-11709</p> + </item> + <item> + <p> + Fix race bug in <c>ets:all</c>. Concurrent creation of + tables could cause other tables to not be included in the + result. (Thanks to Florian Schintke for bug report)</p> + <p> + Own Id: OTP-11726</p> + </item> + <item> + <p> + erl_eval now properly evaluates '=='/2 when it is used in + guards. (Thanks to José Valim)</p> + <p> + Own Id: OTP-11747</p> + </item> + <item> + <p> + Calls to proplists:get_value/3 are replaced by the faster + lists:keyfind/3 in io_lib_pretty. Elements in the list + are always 2-tuples. (Thanks to Andrew Thompson)</p> + <p> + Own Id: OTP-11752</p> + </item> + <item> + <p> A qlc bug where filters were erroneously optimized + away has been fixed. Thanks to Sam Bobroff for reporting + the bug. </p> + <p> + Own Id: OTP-11758</p> + </item> + <item> + <p> + A number of compiler errors where unusual or nonsensical + code would crash the compiler have been reported by Ulf + Norell and corrected by Anthony Ramine.</p> + <p> + Own Id: OTP-11770</p> + </item> + <item> + <p> Since Erlang/OTP R16B the Erlang Core Linter + (<c>erl_lint</c>) has not emitted errors when built-in + types were re-defined. This bug has been fixed. (Thanks + to Roberto Aloi.) </p> + <p> + Own Id: OTP-11772</p> + </item> + <item> + <p> + The functions <c>sys:get_state/1,2</c> and + <c>sys:replace_state/2,3</c> are fixed so they can now be + run while the process is sys suspended. To accomplish + this, the new callbacks <c>Mod:system_get_state/1</c> and + <c>Mod:system_replace_state/2</c> are added, which are + also implemented by the generic behaviours + <c>gen_server</c>, <c>gen_event</c> and <c>gen_fsm</c>.</p> + <p> + The potential incompatibility refers to</p> + <p> + <list> <item>The previous behaviour of intercepting the + system message and passing a tuple of size 2 as the last + argument to <c>sys:handle_system_msg/6</c> is no longer + supported.</item> <item>The error handling when + <c>StateFun</c> in <c>sys:replace_state/2,3</c> fails is + changed from being totally silent to possibly (if the + callback module does not catch) throw an exception in the + client process.</item> </list></p> + <p> + (Thanks to James Fish and Steve Vinoski)</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-11817</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Options to set match_limit and match_limit_recursion are + added to re:run. The option report_errors is also added + to get more information when re:run fails due to limits + or compilation errors.</p> + <p> + Own Id: OTP-10285</p> + </item> + <item> + <p> The pre-defined types <c>array/0</c>, <c>dict/0</c>, + <c>digraph/0</c>, <c>gb_set/0</c>, <c>gb_tree/0</c>, + <c>queue/0</c>, <c>set/0</c>, and <c>tid/0</c> have been + deprecated. They will be removed in Erlang/OTP 18.0. </p> + <p> Instead the types <c>array:array/0</c>, + <c>dict:dict/0</c>, <c>digraph:graph/0</c>, + <c>gb_set:set/0</c>, <c>gb_tree:tree/0</c>, + <c>queue:queue/0</c>, <c>sets:set/0</c>, and + <c>ets:tid/0</c> can be used. (Note: it has always been + necessary to use <c>ets:tid/0</c>.) </p> <p> It is + allowed in Erlang/OTP 17.0 to locally re-define the types + <c>array/0</c>, <c>dict/0</c>, and so on. </p> <p> New + types <c>array:array/1</c>, <c>dict:dict/2</c>, + <c>gb_sets:set/1</c>, <c>gb_trees:tree/2</c>, + <c>queue:queue/1</c>, and <c>sets:set/1</c> have been + added. </p> <p> A compiler option, + <c>nowarn_deprecated_type</c>, has been introduced. By + including the attribute </p> <c> + -compile(nowarn_deprecated_type).</c> <p> in an Erlang + source file, warnings about deprecated types can be + avoided in Erlang/OTP 17.0. </p> <p> The option can also + be given as a compiler flag: </p> <c> erlc + +nowarn_deprecated_type file.erl</c> + <p> + Own Id: OTP-10342</p> + </item> + <item> + <p> + Calls to erlang:open_port/2 with 'spawn' are updated to + handle space in the command path.</p> + <p> + Own Id: OTP-10842</p> + </item> + <item> + <p> Dialyzer's <c>unmatched_return</c> warnings have been + corrected. </p> + <p> + Own Id: OTP-10908</p> + </item> + <item> + <p> + Forbid unsized fields in patterns of binary generators + and simplified v3_core's translation of bit string + generators. (Thanks to Anthony Ramine.)</p> + <p> + Own Id: OTP-11186</p> + </item> + <item> + <p> + The version of the PCRE library Used by Erlang's re + module is raised to 8.33 from 7.6. This means, among + other things, better Unicode and Unicode Character + Properties support. New options connected to PCRE 8.33 + are also added to the re module (ucd, notempty_atstart, + no_start_optimize). PCRE has extended the regular + expression syntax between 7.6 and 8.33, why this imposes + a potential incompatibility. Only very complicated + regular expressions may be affected, but if you know you + are using obscure features, please test run your regular + expressions and verify that their behavior has not + changed.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-11204</p> + </item> + <item> + <p> + Added dict:is_empty/1 and orddict:is_empty/1. (Thanks to + Magnus Henoch.)</p> + <p> + Own Id: OTP-11353</p> + </item> + <item> + <p> + A call to either the <c>garbage_collect/1</c> BIF or the + <c>check_process_code/2</c> BIF may trigger garbage + collection of another processes than the process calling + the BIF. The previous implementations performed these + kinds of garbage collections without considering the + internal state of the process being garbage collected. In + order to be able to more easily and more efficiently + implement yielding native code, these types of garbage + collections have been rewritten. A garbage collection + like this is now triggered by an asynchronous request + signal, the actual garbage collection is performed by the + process being garbage collected itself, and finalized by + a reply signal to the process issuing the request. Using + this approach processes can disable garbage collection + and yield without having to set up the heap in a state + that can be garbage collected.</p> + <p> + The <seealso + marker="erts:erlang#garbage_collect/2"><c>garbage_collect/2</c></seealso>, + and <seealso + marker="erts:erlang#check_process_code/3"><c>check_process_code/3</c></seealso> + BIFs have been introduced. Both taking an option list as + last argument. Using these, one can issue asynchronous + requests.</p> + <p> + <c>code:purge/1</c> and <c>code:soft_purge/1</c> have + been rewritten to utilize asynchronous + <c>check_process_code</c> requests in order to + parallelize work.</p> + <p> + Characteristics impact: A call to the + <c>garbage_collect/1</c> BIF or the + <c>check_process_code/2</c> BIF will normally take longer + time to complete while the system as a whole wont be as + much negatively effected by the operation as before. A + call to <c>code:purge/1</c> and <c>code:soft_purge/1</c> + may complete faster or slower depending on the state of + the system while the system as a whole wont be as much + negatively effected by the operation as before.</p> + <p> + Own Id: OTP-11388 Aux Id: OTP-11535, OTP-11648 </p> + </item> + <item> + <p> Improve the documentation of the supervisor's + <c>via</c> reference. (Thanks to MaximMinin.) </p> + <p> + Own Id: OTP-11399</p> + </item> + <item> + <p><c>orddict:from_list/1</c> now uses the optimized sort + routines in the <c>lists</c> module instead of + (essentially) an insertion sort. Depending on the input + data, the speed of the new <c>from_list/1</c> is anything + from slightly faster up to several orders of magnitude + faster than the old <c>from_list/1</c>.</p> (Thanks to + Steve Vinoski.) + <p> + Own Id: OTP-11552</p> + </item> + <item> + <p> + EEP43: New data type - Maps</p> + <p> + With Maps you may for instance: <taglist> <item><c>M0 = + #{ a => 1, b => 2}, % create + associations</c></item> <item><c>M1 = M0#{ a := 10 }, % + update values</c></item> <item><c>M2 = M1#{ "hi" => + "hello"}, % add new associations</c></item> <item><c>#{ + "hi" := V1, a := V2, b := V3} = M2. % match keys with + values</c></item> </taglist></p> + <p> + For information on how to use Maps please see Map Expressions in the + <seealso marker="doc/reference_manual:expressions#map_expressions"> + Reference Manual</seealso>.</p> + <p> + The current implementation is without the following + features: <taglist> <item>No variable keys</item> + <item>No single value access</item> <item>No map + comprehensions</item> </taglist></p> + <p> + Note that Maps is <em>experimental</em> during OTP 17.0.</p> + <p> + Own Id: OTP-11616</p> + </item> + <item> + <p> + When tab completing the erlang shell now expands + zero-arity functions all the way to closing parenthesis, + unless there is another function with the same name and a + different arity. (Thanks to Pierre Fenoll.)</p> + <p> + Own Id: OTP-11684</p> + </item> + <item> + <p> The Erlang Code Preprocessor (<c>epp</c>) could loop + when encountering a circular macro definition in an + included file. This bug has been fixed. </p> <p> Thanks + to Maruthavanan Subbarayan for reporting the bug, and to + Richard Carlsson for providing a bug fix. </p> + <p> + Own Id: OTP-11728</p> + </item> + <item> + <p> The Erlang Code Linter (<c>erl_lint</c>) has since + Erlang/OTP R13B emitted warnings whenever any of the + types <c>arity()</c>, <c>bitstring()</c>, + <c>iodata()</c>, or <c>boolean()</c> were re-defined. Now + errors are emitted instead. </p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-11771</p> + </item> + <item> + <p> The <c>encoding</c> option of + <c>erl_parse:abstract/2</c> has been extended to include + <c>none</c> and a callback function (a predicate). </p> + <p> + Own Id: OTP-11807</p> + </item> + <item> + <p> + Export zip option types to allow referal from other + modules.</p> + <p> + Thanks to Pierre Fenoll and Håkan Mattson</p> + <p> + Own Id: OTP-11828</p> + </item> + <item> + <p> + The module <c>pg</c> has been deprecated and will be + removed in Erlang/OTP 18.</p> + <p> + Own Id: OTP-11840</p> + </item> + </list> + </section> + +</section> + <section><title>STDLIB 1.19.4</title> <section><title>Fixed Bugs and Malfunctions</title> @@ -1809,7 +2366,7 @@ platforms than before. If <c>configure</c> warns about no atomic implementation available, try using the <c>libatomic_ops</c> library. Use the <seealso - marker="doc/installation_guide:INSTALL#How-to-Build-and-Install-ErlangOTP_A-Closer-Look-at-the-individual-Steps_Configuring">--with-libatomic_ops=PATH</seealso> + marker="doc/installation_guide:INSTALL#Advanced-configuration-and-build-of-ErlangOTP">--with-libatomic_ops=PATH</seealso> <c>configure</c> command line argument when specifying where the <c>libatomic_ops</c> installation is located. The <c>libatomic_ops</c> library can be downloaded from: @@ -1827,7 +2384,7 @@ the pentium 4 processor. If you want the runtime system to be compatible with older processors (back to 486) you need to pass the <seealso - marker="doc/installation_guide:INSTALL#How-to-Build-and-Install-ErlangOTP_A-Closer-Look-at-the-individual-Steps_Configuring">--enable-ethread-pre-pentium4-compatibility</seealso> + marker="doc/installation_guide:INSTALL#Advanced-configuration-and-build-of-ErlangOTP">--enable-ethread-pre-pentium4-compatibility</seealso> <c>configure</c> command line argument when configuring the system.</p> <p> diff --git a/lib/stdlib/doc/src/pg.xml b/lib/stdlib/doc/src/pg.xml index 7cc1b805b4..a3b69884b6 100644 --- a/lib/stdlib/doc/src/pg.xml +++ b/lib/stdlib/doc/src/pg.xml @@ -5,7 +5,7 @@ <header> <copyright> <year>1996</year> - <year>2013</year> + <year>2014</year> <holder>Ericsson AB, All Rights Reserved</holder> </copyright> <legalnotice> @@ -32,6 +32,9 @@ <module>pg</module> <modulesummary>Distributed, Named Process Groups</modulesummary> <description> + <warning> + <p>This module is deprecated and will be removed in Erlang/OTP 18.</p> + </warning> <p>This (experimental) module implements process groups. A process group is a group of processes that can be accessed by a common name. For example, a group named <c>foobar</c> can include a set diff --git a/lib/stdlib/doc/src/string.xml b/lib/stdlib/doc/src/string.xml index c96cc95a44..b05d5cbc08 100644 --- a/lib/stdlib/doc/src/string.xml +++ b/lib/stdlib/doc/src/string.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1996</year><year>2013</year> + <year>1996</year><year>2014</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -124,6 +124,10 @@ <code type="none"> > tokens("abc defxxghix jkl", "x "). ["abc", "def", "ghi", "jkl"] </code> + <p>Note that, as shown in the example above, two or more + adjacent separator characters in <c><anno>String</anno></c> + will be treated as one. That is, there will not be any empty + strings in the resulting list of tokens.</p> </desc> </func> <func> diff --git a/lib/stdlib/doc/src/sys.xml b/lib/stdlib/doc/src/sys.xml index a46fa1289f..19605f325b 100644 --- a/lib/stdlib/doc/src/sys.xml +++ b/lib/stdlib/doc/src/sys.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1996</year><year>2013</year> + <year>1996</year><year>2014</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -115,6 +115,9 @@ <datatype> <name name="dbg_fun"/> </datatype> + <datatype> + <name name="format_fun"/> + </datatype> </datatypes> <funcs> <func> diff --git a/lib/stdlib/doc/src/zip.xml b/lib/stdlib/doc/src/zip.xml index 66c21da193..48b376743d 100644 --- a/lib/stdlib/doc/src/zip.xml +++ b/lib/stdlib/doc/src/zip.xml @@ -123,6 +123,18 @@ </taglist> </desc> </datatype> + <datatype> + <name name="filename"/> + <p>The name of a zip file.</p> + </datatype> + <datatype><name name="extension"/></datatype> + <datatype><name name="extension_spec"/></datatype> + <datatype> + <name name="create_option"/> + <desc> + <p>These options are described in <seealso marker="#zip_options">create/3</seealso>.</p> + </desc> + </datatype> </datatypes> <funcs> <func> @@ -160,6 +172,7 @@ set to <c>["gif", "jpg"]</c> and <c>uncompress</c> is set to <c>["jpg"]</c>, only files with <c>"gif"</c> as extension will be compressed. No other files will be compressed.</p> + <marker id="zip_options"></marker> <p>The following options are available:</p> <taglist> <tag><c>cooked</c></tag> diff --git a/lib/stdlib/examples/erl_id_trans.erl b/lib/stdlib/examples/erl_id_trans.erl index 5fcb74310e..e71e26e51a 100644 --- a/lib/stdlib/examples/erl_id_trans.erl +++ b/lib/stdlib/examples/erl_id_trans.erl @@ -148,7 +148,7 @@ pattern({map,Line,Ps0}) -> Ps1 = pattern_list(Ps0), {map,Line,Ps1}; pattern({map_field_exact,Line,K,V}) -> - Ke = pattern(K), + Ke = expr(K), Ve = pattern(V), {map_field_exact,Line,Ke,Ve}; %%pattern({struct,Line,Tag,Ps0}) -> diff --git a/lib/stdlib/src/dets.erl b/lib/stdlib/src/dets.erl index c32da1624f..76e03bbfaa 100644 --- a/lib/stdlib/src/dets.erl +++ b/lib/stdlib/src/dets.erl @@ -440,9 +440,10 @@ insert(Tab, Objs) when is_list(Objs) -> insert(Tab, Obj) -> badarg(treq(Tab, {insert, [Obj]}), [Tab, Obj]). --spec insert_new(Name, Objects) -> boolean() when +-spec insert_new(Name, Objects) -> boolean() | {'error', Reason} when Name :: tab_name(), - Objects :: object() | [object()]. + Objects :: object() | [object()], + Reason :: term(). insert_new(Tab, Objs) when is_list(Objs) -> badarg(treq(Tab, {insert_new, Objs}), [Tab, Objs]); diff --git a/lib/stdlib/src/epp.erl b/lib/stdlib/src/epp.erl index 9b506b0a44..5f8637c118 100644 --- a/lib/stdlib/src/epp.erl +++ b/lib/stdlib/src/epp.erl @@ -1121,8 +1121,20 @@ skip_toks(From, St, [I|Sis]) -> skip_toks(From, St#epp{location=Cl}, Sis); {ok,_Toks,Cl} -> skip_toks(From, St#epp{location=Cl}, [I|Sis]); - {error,_E,Cl} -> - skip_toks(From, St#epp{location=Cl}, [I|Sis]); + {error,E,Cl} -> + case E of + {_,file_io_server,invalid_unicode} -> + %% The compiler needs to know that there was + %% invalid unicode characters in the file + %% (and there is no point in continuing anyway + %% since io server process has terminated). + epp_reply(From, {error,E}), + leave_file(wait_request(St), St); + _ -> + %% Some other invalid token, such as a bad floating + %% point number. Just ignore it. + skip_toks(From, St#epp{location=Cl}, [I|Sis]) + end; {eof,Cl} -> leave_file(From, St#epp{location=Cl,istk=[I|Sis]}); {error,_E} -> diff --git a/lib/stdlib/src/erl_eval.erl b/lib/stdlib/src/erl_eval.erl index acde3ad5d6..639ddfc214 100644 --- a/lib/stdlib/src/erl_eval.erl +++ b/lib/stdlib/src/erl_eval.erl @@ -77,7 +77,7 @@ %% Only exprs/2 checks the command by calling erl_lint. The reason is %% that if there is a function handler present, then it is possible %% that there are valid constructs in Expression to be taken care of -%% by a function handler but considerad errors by erl_lint. +%% by a function handler but considered errors by erl_lint. -spec(exprs(Expressions, Bindings) -> {value, Value, NewBindings} when Expressions :: expressions(), @@ -244,17 +244,17 @@ expr({record,_,_,Name,_}, _Bs, _Lf, _Ef, _RBs) -> erlang:raise(error, {undef_record,Name}, stacktrace()); %% map -expr({map,_, Binding,Es}, Bs0, Lf, Ef, RBs) -> - {value, Map0, Bs1} = expr(Binding, Bs0, Lf, Ef, RBs), +expr({map,_,Binding,Es}, Bs0, Lf, Ef, RBs) -> + {value, Map0, Bs1} = expr(Binding, Bs0, Lf, Ef, none), case Map0 of #{} -> - {Vs,Bs} = eval_map_fields(Es, Bs1, Lf, Ef), + {Vs,Bs2} = eval_map_fields(Es, Bs0, Lf, Ef), Map1 = lists:foldl(fun ({map_assoc,K,V}, Mi) -> maps:put(K, V, Mi); ({map_exact,K,V}, Mi) -> maps:update(K, V, Mi) end, Map0, Vs), - ret_expr(Map1, Bs, RBs); + ret_expr(Map1, merge_bindings(Bs2, Bs1), RBs); _ -> erlang:raise(error, {badarg,Map0}, stacktrace()) end; diff --git a/lib/stdlib/src/erl_expand_records.erl b/lib/stdlib/src/erl_expand_records.erl index f53c6e1278..c74f68647f 100644 --- a/lib/stdlib/src/erl_expand_records.erl +++ b/lib/stdlib/src/erl_expand_records.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2012. All Rights Reserved. +%% Copyright Ericsson AB 2005-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -38,6 +38,8 @@ checked_ra=[] % successfully accessed records }). +-define(REC_OFFSET, 100000000). % A hundred millions. Also in v3_core. + -spec(module(AbsForms, CompileOptions) -> AbsForms when AbsForms :: [erl_parse:abstract_form()], CompileOptions :: [compile:option()]). @@ -135,19 +137,20 @@ pattern({tuple,Line,Ps}, St0) -> pattern({map,Line,Ps}, St0) -> {TPs,St1} = pattern_list(Ps, St0), {{map,Line,TPs},St1}; -pattern({map_field_exact,Line,Key0,V0}, St0) -> - {Key,St1} = pattern(Key0, St0), +pattern({map_field_exact,Line,K0,V0}, St0) -> + {K,St1} = expr(K0, St0), {V,St2} = pattern(V0, St1), - {{map_field_exact,Line,Key,V},St2}; + {{map_field_exact,Line,K,V},St2}; %%pattern({struct,Line,Tag,Ps}, St0) -> %% {TPs,TPsvs,St1} = pattern_list(Ps, St0), %% {{struct,Line,Tag,TPs},TPsvs,St1}; pattern({record_index,Line,Name,Field}, St) -> {index_expr(Line, Field, Name, record_fields(Name, St)),St}; -pattern({record,Line,Name,Pfs}, St0) -> +pattern({record,Line0,Name,Pfs}, St0) -> Fs = record_fields(Name, St0), {TMs,St1} = pattern_list(pattern_fields(Fs, Pfs), St0), - {{tuple,Line,[{atom,Line,Name} | TMs]},St1}; + Line = record_offset(Line0, St1), + {{tuple,Line,[{atom,Line0,Name} | TMs]},St1}; pattern({bin,Line,Es0}, St0) -> {Es1,St1} = pattern_bin(Es0, St0), {{bin,Line,Es1},St1}; @@ -329,8 +332,9 @@ expr({map_field_exact,Line,K0,V0}, St0) -> expr({record_index,Line,Name,F}, St) -> I = index_expr(Line, F, Name, record_fields(Name, St)), expr(I, St); -expr({record,Line,Name,Is}, St) -> - expr({tuple,Line,[{atom,Line,Name} | +expr({record,Line0,Name,Is}, St) -> + Line = record_offset(Line0, St), + expr({tuple,Line,[{atom,Line0,Name} | record_inits(record_fields(Name, St), Is)]}, St); expr({record_field,Line,R,Name,F}, St) -> @@ -582,8 +586,9 @@ strict_get_record_field(Line, R, {atom,_,F}=Index, Name, St0) -> I = index_expr(F, Fs, 2), P = record_pattern(2, I, Var, length(Fs)+1, Line, [{atom,Line,Name}]), NLine = neg_line(Line), + RLine = record_offset(NLine, St), E = {'case',NLine,R, - [{clause,NLine,[{tuple,NLine,P}],[],[Var]}, + [{clause,NLine,[{tuple,RLine,P}],[],[Var]}, {clause,NLine,[{var,NLine,'_'}],[], [{call,NLine,{remote,NLine, {atom,NLine,erlang}, @@ -836,7 +841,7 @@ optimize_is_record(H0, G0, #exprec{compile=Opts}) -> [] -> {H0,G0}; Rs0 -> - case lists:member(no_is_record_optimization, Opts) of + case lists:member(dialyzer, Opts) of % no_is_record_optimization true -> {H0,G0}; false -> @@ -961,3 +966,10 @@ opt_remove_2(A, _) -> A. neg_line(L) -> erl_parse:set_line(L, fun(Line) -> -abs(Line) end). + +record_offset(L, St) -> + case lists:member(dialyzer, St#exprec.compile) of + true when L >= 0 -> L+?REC_OFFSET; + true when L < 0 -> L-?REC_OFFSET; + false -> L + end. diff --git a/lib/stdlib/src/erl_lint.erl b/lib/stdlib/src/erl_lint.erl index c4c94fbee4..39cc03cf7a 100644 --- a/lib/stdlib/src/erl_lint.erl +++ b/lib/stdlib/src/erl_lint.erl @@ -1046,9 +1046,10 @@ check_undefined_types(#lint{usage=Usage,types=Def}=St0) -> Used = Usage#usage.used_types, UTAs = dict:fetch_keys(Used), Undef = [{TA,dict:fetch(TA, Used)} || - TA <- UTAs, + {T,_}=TA <- UTAs, not dict:is_key(TA, Def), - not is_default_type(TA)], + not is_default_type(TA), + not is_newly_introduced_var_arity_type(T)], foldl(fun ({TA,L}, St) -> add_error(L, {undefined_type,TA}, St) end, St0, Undef). @@ -1407,7 +1408,7 @@ pattern({map,_Line,Ps}, Vt, Old, Bvt, St) -> ({map_field_assoc,L,_,_}, {Psvt,Bvt0,St0}) -> {Psvt,Bvt0,add_error(L, illegal_pattern, St0)}; ({map_field_exact,L,KP,VP}, {Psvt,Bvt0,St0}) -> - case is_valid_map_key(KP, St0) of + case is_valid_map_key(KP, pattern, St0) of true -> {Pvt,Bvt1,St1} = pattern(VP, Vt, Old, Bvt, St0), {vtmerge_pat(Pvt, Psvt),vtmerge_pat(Bvt0, Bvt1), St1}; @@ -2322,14 +2323,16 @@ is_valid_call(Call) -> %% check for value expression without variables is_valid_map_key(K,St) -> + is_valid_map_key(K,expr,St). +is_valid_map_key(K,Ctx,St) -> case expr(K,[],St) of {[],_} -> - is_valid_map_key_value(K); + is_valid_map_key_value(K,Ctx); {[Var|_],_} -> {false,variable,element(1,Var)} end. -is_valid_map_key_value(K) -> +is_valid_map_key_value(K,Ctx) -> case K of {char,_,_} -> true; {integer,_,_} -> true; @@ -2338,34 +2341,36 @@ is_valid_map_key_value(K) -> {nil,_} -> true; {atom,_,_} -> true; {cons,_,H,T} -> - is_valid_map_key_value(H) andalso - is_valid_map_key_value(T); + is_valid_map_key_value(H,Ctx) andalso + is_valid_map_key_value(T,Ctx); {tuple,_,Es} -> foldl(fun(E,B) -> - B andalso is_valid_map_key_value(E) + B andalso is_valid_map_key_value(E,Ctx) end,true,Es); {map,_,Arg,Ps} -> % only check for value expressions to be valid % invalid map expressions are later checked in % core and kernel - is_valid_map_key_value(Arg) andalso foldl(fun + is_valid_map_key_value(Arg,Ctx) andalso foldl(fun ({Tag,_,Ke,Ve},B) when Tag =:= map_field_assoc; - Tag =:= map_field_exact -> - B andalso is_valid_map_key_value(Ke) - andalso is_valid_map_key_value(Ve) + Tag =:= map_field_exact, Ctx =:= expr -> + B andalso is_valid_map_key_value(Ke,Ctx) + andalso is_valid_map_key_value(Ve,Ctx); + (_,_) -> false end,true,Ps); {map,_,Ps} -> foldl(fun ({Tag,_,Ke,Ve},B) when Tag =:= map_field_assoc; - Tag =:= map_field_exact -> - B andalso is_valid_map_key_value(Ke) - andalso is_valid_map_key_value(Ve) + Tag =:= map_field_exact, Ctx =:= expr -> + B andalso is_valid_map_key_value(Ke,Ctx) + andalso is_valid_map_key_value(Ve,Ctx); + (_,_) -> false end, true, Ps); {record,_,_,Fs} -> foldl(fun ({record_field,_,Ke,Ve},B) -> - B andalso is_valid_map_key_value(Ke) - andalso is_valid_map_key_value(Ve) + B andalso is_valid_map_key_value(Ke,Ctx) + andalso is_valid_map_key_value(Ve,Ctx) end,true,Fs); {bin,_,Es} -> % only check for value expressions to be valid @@ -2373,7 +2378,7 @@ is_valid_map_key_value(K) -> % core and kernel foldl(fun ({bin_element,_,E,_,_},B) -> - B andalso is_valid_map_key_value(E) + B andalso is_valid_map_key_value(E,Ctx) end,true,Es); _ -> false end. diff --git a/lib/stdlib/src/erl_parse.yrl b/lib/stdlib/src/erl_parse.yrl index 1dc5fc52a7..e1ae3b7aea 100644 --- a/lib/stdlib/src/erl_parse.yrl +++ b/lib/stdlib/src/erl_parse.yrl @@ -848,10 +848,12 @@ build_fun(Line, Cs) -> end. check_clauses(Cs, Name, Arity) -> - mapl(fun ({clause,L,N,As,G,B}) when N =:= Name, length(As) =:= Arity -> - {clause,L,As,G,B}; - ({clause,L,_N,_As,_G,_B}) -> - ret_err(L, "head mismatch") end, Cs). + [case C of + {clause,L,N,As,G,B} when N =:= Name, length(As) =:= Arity -> + {clause,L,As,G,B}; + {clause,L,_N,_As,_G,_B} -> + ret_err(L, "head mismatch") + end || C <- Cs]. build_try(L,Es,Scs,{Ccs,As}) -> {'try',L,Es,Scs,Ccs,As}. @@ -861,17 +863,6 @@ ret_err(L, S) -> {location,Location} = get_attribute(L, location), return_error(Location, S). -%% mapl(F,List) -%% an alternative map which always maps from left to right -%% and makes it possible to interrupt the mapping with throw on -%% the first occurence from left as expected. -%% can be removed when the jam machine (and all other machines) -%% uses the standardized (Erlang 5.0) evaluation order (from left to right) -mapl(F, [H|T]) -> - V = F(H), - [V | mapl(F,T)]; -mapl(_, []) -> - []. %% Convert between the abstract form of a term and a term. diff --git a/lib/stdlib/src/erl_pp.erl b/lib/stdlib/src/erl_pp.erl index 9dbe89da91..3dbb5ab64c 100644 --- a/lib/stdlib/src/erl_pp.erl +++ b/lib/stdlib/src/erl_pp.erl @@ -256,6 +256,10 @@ ltype({type,_Line,nonempty_list,[T]}) -> {seq,$[,$],[$,],[ltype(T),leaf("...")]}; ltype({type,Line,nil,[]}) -> lexpr({nil,Line}, 0, options(none)); +ltype({type,Line,map,any}) -> + simple_type({atom,Line,map}, []); +ltype({type,_Line,map,Pairs}) -> + map_type(Pairs); ltype({type,Line,tuple,any}) -> simple_type({atom,Line,tuple}, []); ltype({type,_Line,tuple,Ts}) -> @@ -289,6 +293,23 @@ binary_type(I1, I2) -> E2 = [[leaf("_:_*"),lexpr(I2, P, options(none))] || U], {seq,'<<','>>',[$,],E1++E2}. +map_type(Fs) -> + {first,[$#],map_pair_types(Fs)}. + +map_pair_types(Fs) -> + tuple_type(Fs, fun map_pair_type/1). + +map_pair_type({type,_Line,map_field_assoc,Ktype,Vtype}) -> + map_assoc_typed(lexpr(Ktype, options(none)), Vtype). + +map_assoc_typed(B, {type,_,union,Ts}) -> + {first,[B,$\s],{seq,[],[],[],map_assoc_union_type(Ts)}}; +map_assoc_typed(B, Type) -> + {list,[{cstep,[B," =>"],ltype(Type)}]}. + +map_assoc_union_type([T|Ts]) -> + [[leaf("=> "),ltype(T)] | ltypes(Ts, fun union_elem/1)]. + record_type(Name, Fields) -> {first,[record_name(Name)],field_types(Fields)}. diff --git a/lib/stdlib/src/erl_scan.erl b/lib/stdlib/src/erl_scan.erl index ae59d5f44f..6fd6bb888b 100644 --- a/lib/stdlib/src/erl_scan.erl +++ b/lib/stdlib/src/erl_scan.erl @@ -1075,7 +1075,7 @@ scan_number([$#|Cs]=Cs0, St, Line, Col, Toks, Ncs0) -> Ncs = lists:reverse(Ncs0), case catch list_to_integer(Ncs) of B when B >= 2, B =< 1+$Z-$A+10 -> - Bcs = ?STR(St, Ncs++[$#]), + Bcs = Ncs++[$#], scan_based_int(Cs, St, Line, Col, Toks, {B,[],Bcs}); B -> Len = length(Ncs), @@ -1108,7 +1108,7 @@ scan_based_int(Cs, St, Line, Col, Toks, {B,Ncs0,Bcs}) -> Ncs = lists:reverse(Ncs0), case catch erlang:list_to_integer(Ncs, B) of N when is_integer(N) -> - tok3(Cs, St, Line, Col, Toks, integer, ?STR(St, Bcs++Ncs), N); + tok3(Cs, St, Line, Col, Toks, integer, Bcs++Ncs, N); _ -> Len = length(Bcs)+length(Ncs), Ncol = incr_column(Col, Len), diff --git a/lib/stdlib/src/erl_tar.erl b/lib/stdlib/src/erl_tar.erl index 40b48d7999..acf7a5cd40 100644 --- a/lib/stdlib/src/erl_tar.erl +++ b/lib/stdlib/src/erl_tar.erl @@ -381,7 +381,12 @@ to_octal(Int, Count, Result) -> to_octal(Int div 8, Count-1, [Int rem 8 + $0|Result]). to_string(Str0, Count) -> - Str = list_to_binary(Str0), + Str = case file:native_name_encoding() of + utf8 -> + unicode:characters_to_binary(Str0); + latin1 -> + list_to_binary(Str0) + end, case byte_size(Str) of Size when Size < Count -> [Str|zeroes(Count-Size)]; @@ -392,9 +397,17 @@ to_string(Str0, Count) -> pad_file(File) -> {ok,Position} = file:position(File, {cur,0}), - %% There must be at least one empty record at the end of the file. - Zeros = zeroes(?block_size - (Position rem ?block_size)), - file:write(File, Zeros). + %% There must be at least two zero records at the end. + Fill = case ?block_size - (Position rem ?block_size) of + Fill0 when Fill0 < 2*?record_size -> + %% We need to another block here to ensure that there + %% are at least two zero records at the end. + Fill0 + ?block_size; + Fill0 -> + %% Large enough. + Fill0 + end, + file:write(File, zeroes(Fill)). split_filename(Name) when length(Name) =< ?th_name_len -> {"", Name}; @@ -608,7 +621,22 @@ typeflag(Bin) -> %% Get the name of the file from the prefix and name fields of the %% tar header. -get_name(Bin) -> +get_name(Bin0) -> + List0 = get_name_raw(Bin0), + case file:native_name_encoding() of + utf8 -> + Bin = list_to_binary(List0), + case unicode:characters_to_list(Bin) of + {error,_,_} -> + List0; + List when is_list(List) -> + List + end; + latin1 -> + List0 + end. + +get_name_raw(Bin) -> Name = from_string(Bin, ?th_name, ?th_name_len), case binary_to_list(Bin, ?th_prefix+1, ?th_prefix+1) of [0] -> diff --git a/lib/stdlib/src/escript.erl b/lib/stdlib/src/escript.erl index a8a82272d6..6bd0eb8a22 100644 --- a/lib/stdlib/src/escript.erl +++ b/lib/stdlib/src/escript.erl @@ -59,7 +59,6 @@ file:filename() | {file:filename(), binary()} | {file:filename(), binary(), file:file_info()}. --type zip_create_option() :: term(). -type section() :: shebang | {shebang, shebang() | default | undefined} @@ -68,8 +67,8 @@ | {emu_args, emu_args() | undefined} | {source, file:filename() | binary()} | {beam, file:filename() | binary()} - | {archive, file:filename() | binary()} - | {archive, [zip_file()], [zip_create_option()]}. + | {archive, zip:filename() | binary()} + | {archive, [zip_file()], [zip:create_option()]}. %%----------------------------------------------------------------------- diff --git a/lib/stdlib/src/filelib.erl b/lib/stdlib/src/filelib.erl index a266daa084..9efbe8da20 100644 --- a/lib/stdlib/src/filelib.erl +++ b/lib/stdlib/src/filelib.erl @@ -265,7 +265,7 @@ do_wildcard(Pattern, Cwd, Mod) -> lists:sort(Files). do_wildcard_1({exists,File}, Mod) -> - case eval_read_file_info(File, Mod) of + case eval_read_link_info(File, Mod) of {ok,_} -> [File]; _ -> [] end; @@ -497,6 +497,16 @@ eval_read_file_info(File, erl_prim_loader) -> eval_read_file_info(File, Mod) -> Mod:read_file_info(File). +eval_read_link_info(File, file) -> + file:read_link_info(File); +eval_read_link_info(File, erl_prim_loader) -> + case erl_prim_loader:read_link_info(File) of + error -> {error, erl_prim_loader}; + Res-> Res + end; +eval_read_link_info(File, Mod) -> + Mod:read_link_info(File). + eval_list_dir(Dir, file) -> file:list_dir(Dir); eval_list_dir(Dir, erl_prim_loader) -> diff --git a/lib/stdlib/src/gen_fsm.erl b/lib/stdlib/src/gen_fsm.erl index e914f7d0b2..5afe3e8b09 100644 --- a/lib/stdlib/src/gen_fsm.erl +++ b/lib/stdlib/src/gen_fsm.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2013. All Rights Reserved. +%% Copyright Ericsson AB 1996-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -594,7 +594,8 @@ reply(Name, {To, Tag}, Reply, Debug, StateName) -> terminate(Reason, Name, Msg, Mod, StateName, StateData, Debug) -> case catch Mod:terminate(Reason, StateName, StateData) of {'EXIT', R} -> - error_info(R, Name, Msg, StateName, StateData, Debug), + FmtStateData = format_status(terminate, Mod, get(), StateData), + error_info(R, Name, Msg, StateName, FmtStateData, Debug), exit(R); _ -> case Reason of @@ -605,17 +606,7 @@ terminate(Reason, Name, Msg, Mod, StateName, StateData, Debug) -> {shutdown,_}=Shutdown -> exit(Shutdown); _ -> - FmtStateData = - case erlang:function_exported(Mod, format_status, 2) of - true -> - Args = [get(), StateData], - case catch Mod:format_status(terminate, Args) of - {'EXIT', _} -> StateData; - Else -> Else - end; - _ -> - StateData - end, + FmtStateData = format_status(terminate, Mod, get(), StateData), error_info(Reason,Name,Msg,StateName,FmtStateData,Debug), exit(Reason) end @@ -680,21 +671,29 @@ format_status(Opt, StatusData) -> Header = gen:format_status_header("Status for state machine", Name), Log = sys:get_debug(log, Debug, []), - DefaultStatus = [{data, [{"StateData", StateData}]}], - Specfic = - case erlang:function_exported(Mod, format_status, 2) of - true -> - case catch Mod:format_status(Opt,[PDict,StateData]) of - {'EXIT', _} -> DefaultStatus; - StatusList when is_list(StatusList) -> StatusList; - Else -> [Else] - end; - _ -> - DefaultStatus - end, + Specfic = format_status(Opt, Mod, PDict, StateData), + Specfic = case format_status(Opt, Mod, PDict, StateData) of + S when is_list(S) -> S; + S -> [S] + end, [{header, Header}, {data, [{"Status", SysState}, {"Parent", Parent}, {"Logged events", Log}, {"StateName", StateName}]} | Specfic]. + +format_status(Opt, Mod, PDict, State) -> + DefStatus = case Opt of + terminate -> State; + _ -> [{data, [{"StateData", State}]}] + end, + case erlang:function_exported(Mod, format_status, 2) of + true -> + case catch Mod:format_status(Opt, [PDict, State]) of + {'EXIT', _} -> DefStatus; + Else -> Else + end; + _ -> + DefStatus + end. diff --git a/lib/stdlib/src/gen_server.erl b/lib/stdlib/src/gen_server.erl index 202a931fae..dadfe56b3d 100644 --- a/lib/stdlib/src/gen_server.erl +++ b/lib/stdlib/src/gen_server.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2013. All Rights Reserved. +%% Copyright Ericsson AB 1996-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -720,7 +720,8 @@ print_event(Dev, Event, Name) -> terminate(Reason, Name, Msg, Mod, State, Debug) -> case catch Mod:terminate(Reason, State) of {'EXIT', R} -> - error_info(R, Name, Msg, State, Debug), + FmtState = format_status(terminate, Mod, get(), State), + error_info(R, Name, Msg, FmtState, Debug), exit(R); _ -> case Reason of @@ -731,17 +732,7 @@ terminate(Reason, Name, Msg, Mod, State, Debug) -> {shutdown,_}=Shutdown -> exit(Shutdown); _ -> - FmtState = - case erlang:function_exported(Mod, format_status, 2) of - true -> - Args = [get(), State], - case catch Mod:format_status(terminate, Args) of - {'EXIT', _} -> State; - Else -> Else - end; - _ -> - State - end, + FmtState = format_status(terminate, Mod, get(), State), error_info(Reason, Name, Msg, FmtState, Debug), exit(Reason) end @@ -875,23 +866,29 @@ name_to_pid(Name) -> %%----------------------------------------------------------------- format_status(Opt, StatusData) -> [PDict, SysState, Parent, Debug, [Name, State, Mod, _Time]] = StatusData, - Header = gen:format_status_header("Status for generic server", - Name), + Header = gen:format_status_header("Status for generic server", Name), Log = sys:get_debug(log, Debug, []), - DefaultStatus = [{data, [{"State", State}]}], - Specfic = - case erlang:function_exported(Mod, format_status, 2) of - true -> - case catch Mod:format_status(Opt, [PDict, State]) of - {'EXIT', _} -> DefaultStatus; - StatusList when is_list(StatusList) -> StatusList; - Else -> [Else] - end; - _ -> - DefaultStatus - end, + Specfic = case format_status(Opt, Mod, PDict, State) of + S when is_list(S) -> S; + S -> [S] + end, [{header, Header}, {data, [{"Status", SysState}, {"Parent", Parent}, {"Logged events", Log}]} | Specfic]. + +format_status(Opt, Mod, PDict, State) -> + DefStatus = case Opt of + terminate -> State; + _ -> [{data, [{"State", State}]}] + end, + case erlang:function_exported(Mod, format_status, 2) of + true -> + case catch Mod:format_status(Opt, [PDict, State]) of + {'EXIT', _} -> DefStatus; + Else -> Else + end; + _ -> + DefStatus + end. diff --git a/lib/stdlib/src/io_lib_format.erl b/lib/stdlib/src/io_lib_format.erl index 56e15a17ec..89ae6fb187 100644 --- a/lib/stdlib/src/io_lib_format.erl +++ b/lib/stdlib/src/io_lib_format.erl @@ -255,7 +255,7 @@ term(T, none, _Adj, none, _Pad) -> T; term(T, none, Adj, P, Pad) -> term(T, P, Adj, P, Pad); term(T, F, Adj, P0, Pad) -> L = lists:flatlength(T), - P = case P0 of none -> erlang:min(L, F); _ -> P0 end, + P = erlang:min(L, case P0 of none -> F; _ -> min(P0, F) end), if L > P -> adjust(chars($*, P), chars(Pad, F-P), Adj); diff --git a/lib/stdlib/src/io_lib_pretty.erl b/lib/stdlib/src/io_lib_pretty.erl index 4057abd8d5..aece06afa6 100644 --- a/lib/stdlib/src/io_lib_pretty.erl +++ b/lib/stdlib/src/io_lib_pretty.erl @@ -25,8 +25,6 @@ -export([print/1,print/2,print/3,print/4,print/5,print/6]). --compile(no_native). - %%% %%% Exported functions %%% diff --git a/lib/stdlib/src/maps.erl b/lib/stdlib/src/maps.erl index 1f94d9e69d..ba4d6a5c87 100644 --- a/lib/stdlib/src/maps.erl +++ b/lib/stdlib/src/maps.erl @@ -23,7 +23,9 @@ fold/3, map/2, size/1, - without/2 + without/2, + with/2, + get/3 ]). @@ -43,8 +45,6 @@ values/1 ]). --compile(no_native). - -spec get(Key,Map) -> Value when Key :: term(), Map :: map(), @@ -134,16 +134,31 @@ to_list(_) -> erlang:nif_error(undef). update(_,_,_) -> erlang:nif_error(undef). --spec values(Map) -> Keys when +-spec values(Map) -> Values when Map :: map(), - Keys :: [Key], - Key :: term(). + Values :: [Value], + Value :: term(). values(_) -> erlang:nif_error(undef). %%% End of BIFs +-spec get(Key, Map, Default) -> Value | Default when + Key :: term(), + Map :: map(), + Value :: term(), + Default :: term(). + +get(Key, Map, Default) -> + case maps:find(Key, Map) of + {ok, Value} -> + Value; + error -> + Default + end. + + -spec fold(Fun,Init,Map) -> Acc when Fun :: fun((K, V, AccIn) -> AccOut), Init :: term(), @@ -187,3 +202,13 @@ size(Map) when is_map(Map) -> without(Ks, M) when is_list(Ks), is_map(M) -> maps:from_list([{K,V}||{K,V} <- maps:to_list(M), not lists:member(K, Ks)]). + + +-spec with(Ks, Map1) -> Map2 when + Ks :: [K], + Map1 :: map(), + Map2 :: map(), + K :: term(). + +with(Ks, M) when is_list(Ks), is_map(M) -> + maps:from_list([{K,V}||{K,V} <- maps:to_list(M), lists:member(K, Ks)]). diff --git a/lib/stdlib/src/otp_internal.erl b/lib/stdlib/src/otp_internal.erl index 380bc3eccc..6c25beabe9 100644 --- a/lib/stdlib/src/otp_internal.erl +++ b/lib/stdlib/src/otp_internal.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2013. All Rights Reserved. +%% Copyright Ericsson AB 1999-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -83,18 +83,18 @@ obsolete_1(crypto, sha_init, 0) -> {deprecated, {crypto, hash_init, 1}}; obsolete_1(crypto, md4_update, 2) -> - {deprecated, {crypto, hash_update, 3}}; + {deprecated, {crypto, hash_update, 2}}; obsolete_1(crypto, md5_update, 2) -> - {deprecated, {crypto, hash_update, 3}}; + {deprecated, {crypto, hash_update, 2}}; obsolete_1(crypto, sha_update, 2) -> - {deprecated, {crypto, hash_update, 3}}; + {deprecated, {crypto, hash_update, 2}}; obsolete_1(crypto, md4_final, 1) -> - {deprecated, {crypto, hash_final, 2}}; + {deprecated, {crypto, hash_final, 1}}; obsolete_1(crypto, md5_final, 1) -> - {deprecated, {crypto, hash_final, 2}}; + {deprecated, {crypto, hash_final, 1}}; obsolete_1(crypto, sha_final, 1) -> - {deprecated, {crypto, hash_final, 2}}; + {deprecated, {crypto, hash_final, 1}}; obsolete_1(crypto, md5_mac, 2) -> {deprecated, {crypto, hmac, 3}}; @@ -104,9 +104,9 @@ obsolete_1(crypto, sha_mac, 3) -> {deprecated, {crypto, hmac, 4}}; obsolete_1(crypto, sha_mac_96, 2) -> - {deprecated, {crypto, hmac_n, 3}}; + {deprecated, {crypto, hmac, 4}}; obsolete_1(crypto, md5_mac_96, 2) -> - {deprecated, {crypto, hmac_n, 3}}; + {deprecated, {crypto, hmac, 4}}; obsolete_1(crypto, rsa_sign, 2) -> {deprecated, {crypto, sign, 4}}; @@ -123,9 +123,9 @@ obsolete_1(crypto, dss_sign, 3) -> {deprecated, {crypto, sign, 4}}; obsolete_1(crypto, dss_verify, 3) -> - {deprecated, {crypto, verify, 4}}; + {deprecated, {crypto, verify, 5}}; obsolete_1(crypto, dss_verify, 4) -> - {deprecated, {crypto, verify, 4}}; + {deprecated, {crypto, verify, 5}}; obsolete_1(crypto, mod_exp, 3) -> {deprecated, {crypto, mod_pow, 3}}; @@ -133,7 +133,7 @@ obsolete_1(crypto, mod_exp, 3) -> obsolete_1(crypto, dh_compute_key, 3) -> {deprecated, {crypto, compute_key, 4}}; obsolete_1(crypto, dh_generate_key, 1) -> - {deprecated, {crypto, generate_key, 3}}; + {deprecated, {crypto, generate_key, 2}}; obsolete_1(crypto, dh_generate_key, 2) -> {deprecated, {crypto, generate_key, 3}}; @@ -250,12 +250,12 @@ obsolete_1(snmp, N, A) -> false -> no; true -> - {deprecated, "Deprecated (will be removed in R17B); use snmpa:"++atom_to_list(N)++"/"++ + {deprecated, "Deprecated (will be removed in OTP 18); use snmpa:"++atom_to_list(N)++"/"++ integer_to_list(A)++" instead"} end; obsolete_1(snmpa, old_info_format, 1) -> - {deprecated, "Deprecated; (will be removed in R17B); use \"new\" format instead"}; + {deprecated, "Deprecated; (will be removed in OTP 18); use \"new\" format instead"}; obsolete_1(snmpm, agent_info, 3) -> {removed, {snmpm, agent_info, 2}, "R16B"}; obsolete_1(snmpm, update_agent_info, 5) -> @@ -366,23 +366,6 @@ obsolete_1(auth, node_cookie, 1) -> obsolete_1(auth, node_cookie, 2) -> {deprecated, "Deprecated; use erlang:set_cookie/2 and net_adm:ping/1 instead"}; -obsolete_1(erlang, is_constant, 1) -> - {removed, "Removed in R13B"}; - -%% Added in R12B-0. -obsolete_1(ssl, port, 1) -> - {removed, {ssl, sockname, 1}, "R13B"}; -obsolete_1(ssl, accept, A) when A =:= 1; A =:= 2 -> - {removed, "deprecated; use ssl:transport_accept/1,2 and ssl:ssl_accept/1,2"}; -obsolete_1(erlang, fault, 1) -> - {removed, {erlang,error,1}, "R13B"}; -obsolete_1(erlang, fault, 2) -> - {removed, {erlang,error,2}, "R13B"}; - -%% Added in R12B-2. -obsolete_1(file, rawopen, 2) -> - {removed, "deprecated (will be removed in R13B); use file:open/2 with the raw option"}; - obsolete_1(http, request, 1) -> {removed,{httpc,request,1},"R15B"}; obsolete_1(http, request, 2) -> {removed,{httpc,request,2},"R15B"}; obsolete_1(http, request, 4) -> {removed,{httpc,request,4},"R15B"}; @@ -438,13 +421,13 @@ obsolete_1(ssh_cm, stop_listener, 1) -> obsolete_1(ssh_cm, session_open, A) when A =:= 2; A =:= 4 -> {removed,{ssh_connection,session_channel,A},"R14B"}; obsolete_1(ssh_cm, direct_tcpip, A) when A =:= 6; A =:= 8 -> - {removed,{ssh_connection,direct_tcpip,A}}; + {removed,{ssh_connection,direct_tcpip,A},"R14B"}; obsolete_1(ssh_cm, tcpip_forward, 3) -> {removed,{ssh_connection,tcpip_forward,3},"R14B"}; obsolete_1(ssh_cm, cancel_tcpip_forward, 3) -> {removed,{ssh_connection,cancel_tcpip_forward,3},"R14B"}; obsolete_1(ssh_cm, open_pty, A) when A =:= 3; A =:= 7; A =:= 9 -> - {removed,{ssh_connection,open_pty,A},"R14"}; + {removed,{ssh_connection,open_pty,A},"R14B"}; obsolete_1(ssh_cm, setenv, 5) -> {removed,{ssh_connection,setenv,5},"R14B"}; obsolete_1(ssh_cm, shell, 2) -> @@ -458,11 +441,11 @@ obsolete_1(ssh_cm, winch, A) when A =:= 4; A =:= 6 -> obsolete_1(ssh_cm, signal, 3) -> {removed,{ssh_connection,signal,3},"R14B"}; obsolete_1(ssh_cm, attach, A) when A =:= 2; A =:= 3 -> - {removed,{ssh,attach,A}}; + {removed,"no longer useful; removed in R14B"}; obsolete_1(ssh_cm, detach, 2) -> - {removed,"no longer useful; will be removed in R14B"}; + {removed,"no longer useful; removed in R14B"}; obsolete_1(ssh_cm, set_user_ack, 4) -> - {removed,"no longer useful; will be removed in R14B"}; + {removed,"no longer useful; removed in R14B"}; obsolete_1(ssh_cm, adjust_window, 3) -> {removed,{ssh_connection,adjust_window,3},"R14B"}; obsolete_1(ssh_cm, close, 2) -> @@ -478,9 +461,9 @@ obsolete_1(ssh_cm, send_ack, A) when 3 =< A, A =< 5 -> obsolete_1(ssh_ssh, connect, A) when 1 =< A, A =< 3 -> {removed,{ssh,shell,A},"R14B"}; obsolete_1(ssh_sshd, listen, A) when 0 =< A, A =< 3 -> - {removed,{ssh,daemon,[1,2,3]},"R14"}; + {removed,{ssh,daemon,[1,2,3]},"R14B"}; obsolete_1(ssh_sshd, stop, 1) -> - {removed,{ssh,stop_listener,1}}; + {removed,{ssh,stop_listener,1},"R14B"}; %% Added in R13A. obsolete_1(regexp, _, _) -> @@ -524,7 +507,7 @@ obsolete_1(docb_xml_check, _, _) -> %% Added in R15B obsolete_1(asn1rt, F, _) when F == load_driver; F == unload_driver -> - {deprecated,"deprecated (will be removed in R16A); has no effect as drivers are no longer used."}; + {deprecated,"deprecated (will be removed in OTP 18); has no effect as drivers are no longer used"}; obsolete_1(ssl, pid, 1) -> {removed,"was removed in R16; is no longer needed"}; obsolete_1(inviso, _, _) -> @@ -532,7 +515,7 @@ obsolete_1(inviso, _, _) -> %% Added in R15B01. obsolete_1(gs, _, _) -> - {deprecated,"the gs application has been deprecated and will be removed in R17; use the wx application instead"}; + {deprecated,"the gs application has been deprecated and will be removed in OTP 18; use the wx application instead"}; obsolete_1(ssh, sign_data, 2) -> {deprecated,"deprecated (will be removed in R16A); use public_key:pem_decode/1, public_key:pem_entry_decode/1 " "and public_key:sign/3 instead"}; @@ -594,6 +577,8 @@ obsolete_1(asn1rt, utf8_binary_to_list, 1) -> {deprecated,{unicode,characters_to_list,1}}; obsolete_1(asn1rt, utf8_list_to_binary, 1) -> {deprecated,{unicode,characters_to_binary,1}}; +obsolete_1(pg, _, _) -> + {deprecated,"deprecated; will be removed in OTP 18"}; obsolete_1(_, _, _) -> no. diff --git a/lib/stdlib/src/pg.erl b/lib/stdlib/src/pg.erl index ee177e4e0b..a41fd329c2 100644 --- a/lib/stdlib/src/pg.erl +++ b/lib/stdlib/src/pg.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2011. All Rights Reserved. +%% Copyright Ericsson AB 1996-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -17,6 +17,7 @@ %% %CopyrightEnd% %% -module(pg). +-deprecated(module). %% pg provides a process group facility. Messages %% can be multicasted to all members in the group diff --git a/lib/stdlib/src/proc_lib.erl b/lib/stdlib/src/proc_lib.erl index 1eb6fc2e86..bf2a4e7ac5 100644 --- a/lib/stdlib/src/proc_lib.erl +++ b/lib/stdlib/src/proc_lib.erl @@ -216,10 +216,8 @@ ensure_link(SpawnOpts) -> init_p(Parent, Ancestors, Fun) when is_function(Fun) -> put('$ancestors', [Parent|Ancestors]), - {module,Mod} = erlang:fun_info(Fun, module), - {name,Name} = erlang:fun_info(Fun, name), - {arity,Arity} = erlang:fun_info(Fun, arity), - put('$initial_call', {Mod,Name,Arity}), + Mfa = erlang:fun_info_mfa(Fun), + put('$initial_call', Mfa), try Fun() catch diff --git a/lib/stdlib/src/shell.erl b/lib/stdlib/src/shell.erl index 3b90542452..679c13f0cf 100644 --- a/lib/stdlib/src/shell.erl +++ b/lib/stdlib/src/shell.erl @@ -371,6 +371,14 @@ expand_expr({bc,L,E,Qs}, C) -> {bc,L,expand_expr(E, C),expand_quals(Qs, C)}; expand_expr({tuple,L,Elts}, C) -> {tuple,L,expand_exprs(Elts, C)}; +expand_expr({map,L,Es}, C) -> + {map,L,expand_exprs(Es, C)}; +expand_expr({map,L,Arg,Es}, C) -> + {map,L,expand_expr(Arg, C),expand_exprs(Es, C)}; +expand_expr({map_field_assoc,L,K,V}, C) -> + {map_field_assoc,L,expand_expr(K, C),expand_expr(V, C)}; +expand_expr({map_field_exact,L,K,V}, C) -> + {map_field_exact,L,expand_expr(K, C),expand_expr(V, C)}; expand_expr({record_index,L,Name,F}, C) -> {record_index,L,Name,expand_expr(F, C)}; expand_expr({record,L,Name,Is}, C) -> diff --git a/lib/stdlib/src/slave.erl b/lib/stdlib/src/slave.erl index 3e647635bc..1898dc8aba 100644 --- a/lib/stdlib/src/slave.erl +++ b/lib/stdlib/src/slave.erl @@ -290,7 +290,10 @@ register_unique_name(Number) -> %% no need to use rsh. mk_cmd(Host, Name, Args, Waiter, Prog0) -> - Prog = quote_progname(Prog0), + Prog = case os:type() of + {ose,_} -> mk_ose_prog(Prog0); + _ -> quote_progname(Prog0) + end, BasicCmd = lists:concat([Prog, " -detached -noinput -master ", node(), " ", long_or_short(), Name, "@", Host, @@ -310,6 +313,24 @@ mk_cmd(Host, Name, Args, Waiter, Prog0) -> end end. +%% On OSE we have to pass the beam arguments directory to the slave +%% process. To find out what arguments that should be passed on we +%% make an assumption. All arguments after the last "--" should be +%% skipped. So given these arguments: +%% -Muycs256 -A 1 -- -root /mst/ -progname beam.debug.smp -- -home /mst/ -- -kernel inetrc '"/mst/inetrc.conf"' -- -name test@localhost +%% we send +%% -Muycs256 -A 1 -- -root /mst/ -progname beam.debug.smp -- -home /mst/ -- -kernel inetrc '"/mst/inetrc.conf"' -- +%% to the slave with whatever other args that are added in mk_cmd. +mk_ose_prog(Prog) -> + SkipTail = fun("--",[]) -> + ["--"]; + (_,[]) -> + []; + (Arg,Args) -> + [Arg," "|Args] + end, + [Prog,tl(lists:foldr(SkipTail,[],erlang:system_info(emu_args)))]. + %% This is an attempt to distinguish between spaces in the program %% path and spaces that separate arguments. The program is quoted to %% allow spaces in the path. diff --git a/lib/stdlib/src/stdlib.app.src b/lib/stdlib/src/stdlib.app.src index d388410de0..aa9899da3b 100644 --- a/lib/stdlib/src/stdlib.app.src +++ b/lib/stdlib/src/stdlib.app.src @@ -103,7 +103,7 @@ dets]}, {applications, [kernel]}, {env, []}, - {runtime_dependencies, ["sasl-2.4","kernel-3.0","erts-6.0","crypto-3.3", + {runtime_dependencies, ["sasl-2.4","kernel-3.0.2","erts-6.2","crypto-3.3", "compiler-5.0"]} ]}. diff --git a/lib/stdlib/src/stdlib.appup.src b/lib/stdlib/src/stdlib.appup.src index 22eefb2514..99d9b8b431 100644 --- a/lib/stdlib/src/stdlib.appup.src +++ b/lib/stdlib/src/stdlib.appup.src @@ -17,9 +17,11 @@ %% %CopyrightEnd% {"%VSN%", %% Up from - max one major revision back - [{<<"2\\.0(\\.[0-9]+)*">>,[restart_new_emulator]}, %% R17 + [{<<"2\\.1(\\.[0-9]+)*">>,[restart_new_emulator]}, %% 17.1 + {<<"2\\.0(\\.[0-9]+)*">>,[restart_new_emulator]}, %% 17.0 {<<"1\\.19(\\.[0-9]+)*">>,[restart_new_emulator]}],%% R16 %% Down to - max one major revision back - [{<<"2\\.0(\\.[0-9]+)*">>,[restart_new_emulator]}, %% R17 + [{<<"2\\.1(\\.[0-9]+)*">>,[restart_new_emulator]}, %% 17.1 + {<<"2\\.0(\\.[0-9]+)*">>,[restart_new_emulator]}, %% 17.0 {<<"1\\.19(\\.[0-9]+)*">>,[restart_new_emulator]}] %% R16 }. diff --git a/lib/stdlib/src/sys.erl b/lib/stdlib/src/sys.erl index e25cc25f57..d3ba09ce82 100644 --- a/lib/stdlib/src/sys.erl +++ b/lib/stdlib/src/sys.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2013. All Rights Reserved. +%% Copyright Ericsson AB 1996-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -46,7 +46,7 @@ {N :: non_neg_integer(), [{Event :: system_event(), FuncState :: _, - FormFunc :: dbg_fun()}]}} + FormFunc :: format_fun()}]}} | {'statistics', {file:date_time(), {'reductions', non_neg_integer()}, MessagesIn :: non_neg_integer(), @@ -57,6 +57,10 @@ Event :: system_event(), ProcState :: _) -> 'done' | (NewFuncState :: _)). +-type format_fun() :: fun((Device :: io:device() | file:io_device(), + Event :: system_event(), + Extra :: term()) -> any()). + %%----------------------------------------------------------------- %% System messages %%----------------------------------------------------------------- @@ -346,7 +350,7 @@ handle_system_msg(SysState, Msg, From, Parent, Mod, Debug, Misc, Hib) -> %%----------------------------------------------------------------- -spec handle_debug(Debug, FormFunc, Extra, Event) -> [dbg_opt()] when Debug :: [dbg_opt()], - FormFunc :: dbg_fun(), + FormFunc :: format_fun(), Extra :: term(), Event :: system_event(). handle_debug([{trace, true} | T], FormFunc, State, Event) -> diff --git a/lib/stdlib/src/zip.erl b/lib/stdlib/src/zip.erl index c40ce8e203..b768c6d0b9 100644 --- a/lib/stdlib/src/zip.erl +++ b/lib/stdlib/src/zip.erl @@ -203,8 +203,18 @@ zip_comment_length}). --type zip_file() :: #zip_file{}. +-type create_option() :: memory | cooked | verbose | {comment, string()} + | {cwd, file:filename()} + | {compress, extension_spec()} + | {uncompress, extension_spec()}. +-type extension() :: string(). +-type extension_spec() :: all | [extension()] | {add, [extension()]} | {del, [extension()]}. +-type filename() :: file:filename(). + -type zip_comment() :: #zip_comment{}. +-type zip_file() :: #zip_file{}. + +-export_type([create_option/0, filename/0]). %% Open a zip archive with options %% @@ -340,13 +350,13 @@ unzip(F) -> unzip(F, []). -spec(unzip(Archive, Options) -> RetValue when Archive :: file:name() | binary(), Options :: [Option], - Option :: {file_list, FileList} + Option :: {file_list, FileList} | cooked | keep_old_files | verbose | memory | {file_filter, FileFilter} | {cwd, CWD}, FileList :: [file:name()], FileBinList :: [{file:name(),binary()}], FileFilter :: fun((ZipFile) -> boolean()), - CWD :: string(), + CWD :: file:filename(), ZipFile :: zip_file(), RetValue :: {ok, FileList} | {ok, FileBinList} @@ -430,7 +440,7 @@ zip(F, Files) -> zip(F, Files, []). What :: all | [Extension] | {add, [Extension]} | {del, [Extension]}, Extension :: string(), Comment :: string(), - CWD :: string(), + CWD :: file:filename(), RetValue :: {ok, FileName :: file:name()} | {ok, {FileName :: file:name(), binary()}} | {error, Reason :: term()}). @@ -712,8 +722,8 @@ table(F, O) -> list_dir(F, O). FileList :: [FileSpec], FileSpec :: file:name() | {file:name(), binary()} | {file:name(), binary(), file:file_info()}, - RetValue :: {ok, FileName :: file:name()} - | {ok, {FileName :: file:name(), binary()}} + RetValue :: {ok, FileName :: filename()} + | {ok, {FileName :: filename(), binary()}} | {error, Reason :: term()}). create(F, Fs) -> zip(F, Fs). @@ -724,14 +734,9 @@ create(F, Fs) -> zip(F, Fs). FileSpec :: file:name() | {file:name(), binary()} | {file:name(), binary(), file:file_info()}, Options :: [Option], - Option :: memory | cooked | verbose | {comment, Comment} - | {cwd, CWD} | {compress, What} | {uncompress, What}, - What :: all | [Extension] | {add, [Extension]} | {del, [Extension]}, - Extension :: string(), - Comment :: string(), - CWD :: string(), - RetValue :: {ok, FileName :: file:name()} - | {ok, {FileName :: file:name(), binary()}} + Option :: create_option(), + RetValue :: {ok, FileName :: filename()} + | {ok, {FileName :: filename(), binary()}} | {error, Reason :: term()}). create(F, Fs, O) -> zip(F, Fs, O). @@ -755,7 +760,7 @@ extract(F) -> unzip(F). FileList :: [file:name()], FileBinList :: [{file:name(),binary()}], FileFilter :: fun((ZipFile) -> boolean()), - CWD :: string(), + CWD :: file:filename(), ZipFile :: zip_file(), RetValue :: {ok, FileList} | {ok, FileBinList} @@ -1153,7 +1158,7 @@ zip_open(Archive) -> zip_open(Archive, []). Archive :: file:name() | binary(), ZipHandle :: pid(), Options :: [Option], - Option :: cooked | memory | {cwd, CWD :: string()}, + Option :: cooked | memory | {cwd, CWD :: file:filename()}, Reason :: term()). zip_open(Archive, Options) -> diff --git a/lib/stdlib/test/Makefile b/lib/stdlib/test/Makefile index 39f6ce423a..a271229c59 100644 --- a/lib/stdlib/test/Makefile +++ b/lib/stdlib/test/Makefile @@ -85,7 +85,8 @@ MODULES= \ zip_SUITE \ random_unicode_list \ random_iolist \ - error_logger_forwarder + error_logger_forwarder \ + maps_SUITE ERL_FILES= $(MODULES:%=%.erl) diff --git a/lib/stdlib/test/binary_module_SUITE.erl b/lib/stdlib/test/binary_module_SUITE.erl index d5a0fe21b4..32cec0db6f 100644 --- a/lib/stdlib/test/binary_module_SUITE.erl +++ b/lib/stdlib/test/binary_module_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2013. All Rights Reserved. +%% Copyright Ericsson AB 1997-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -40,7 +40,7 @@ -export([init_per_testcase/2, end_per_testcase/2]). % Default timetrap timeout (set in init_per_testcase). % Some of these testcases are really heavy... --define(default_timeout, ?t:minutes(20)). +-define(default_timeout, ?t:minutes(30)). -endif. diff --git a/lib/stdlib/test/dets_SUITE.erl b/lib/stdlib/test/dets_SUITE.erl index 00a5da42ad..119b4dc7cb 100644 --- a/lib/stdlib/test/dets_SUITE.erl +++ b/lib/stdlib/test/dets_SUITE.erl @@ -2032,6 +2032,12 @@ match(Config, Version) -> CrashPos = if Version =:= 8 -> 5; Version =:= 9 -> 1 end, crash(Fname, ObjPos2+CrashPos), {ok, _} = dets:open_file(T, Args), + case dets:insert_new(T, Obj) of % OTP-12024 + ok -> + bad_object(dets:sync(T), Fname); + Else3 -> + bad_object(Else3, Fname) + end, io:format("Expect corrupt table:~n"), case ins(T, N) of ok -> @@ -3953,15 +3959,6 @@ otp_11709(Config) when is_list(Config) -> %% Parts common to several test cases %% -start_node_rel(Name, Rel, How) -> - Release = [{release, atom_to_list(Rel)}], - Pa = filename:dirname(code:which(?MODULE)), - test_server:start_node(Name, How, - [{args, - " -kernel net_setuptime 100 " - " -pa " ++ Pa}, - {erl, Release}]). - crash(File, Where) -> crash(File, Where, 10). @@ -4352,7 +4349,7 @@ check_badarg({'EXIT', {badarg, [{M,F,Args,_} | _]}}, M, F, Args) -> check_badarg({'EXIT', {badarg, [{M,F,A,_} | _]}}, M, F, Args) -> true = test_server:is_native(M) andalso length(Args) =:= A. -check_pps(P0) -> +check_pps({Ports0,Procs0} = P0) -> case pps() of P0 -> ok; @@ -4364,22 +4361,28 @@ check_pps(P0) -> case pps() of P0 -> ok; - P1 -> - io:format("failure, got ~p~n, expected ~p\n", [P1, P0]), - {Ports0,Procs0} = P0, - {Ports1,Procs1} = P1, - show("Old ports", Ports0 -- Ports1), - show("New ports", Ports1 -- Ports0), - show("Old procs", Procs0 -- Procs1), - show("New procs", Procs1 -- Procs0), - ?t:fail() - end + {Ports1,Procs1} = P1 -> + case {Ports1 -- Ports0, Procs1 -- Procs0} of + {[], []} -> ok; + {PortsDiff,ProcsDiff} -> + io:format("failure, got ~p~n, expected ~p\n", [P1, P0]), + show("Old port", Ports0 -- Ports1), + show("New port", PortsDiff), + show("Old proc", Procs0 -- Procs1), + show("New proc", ProcsDiff), + ?t:fail() + end + end end. show(_S, []) -> ok; -show(S, L) -> - io:format("~s: ~p~n", [S, L]). +show(S, [Pid|Pids]) when is_pid(Pid) -> + io:format("~s: ~p~n", [S, erlang:process_info(Pid)]), + show(S, Pids); +show(S, [Port|Ports]) when is_port(Port)-> + io:format("~s: ~p~n", [S, erlang:port_info(Port)]), + show(S, Ports). pps() -> dets:start(), diff --git a/lib/stdlib/test/erl_eval_SUITE.erl b/lib/stdlib/test/erl_eval_SUITE.erl index b91d14b5b8..b55324161b 100644 --- a/lib/stdlib/test/erl_eval_SUITE.erl +++ b/lib/stdlib/test/erl_eval_SUITE.erl @@ -1451,6 +1451,13 @@ eep43(Config) when is_list(Config) -> " {Map#{a := B},Map#{a => c},Map#{d => e}} " "end.", {#{a => b},#{a => c},#{a => b,d => e}}), + check(fun () -> + lists:map(fun (X) -> X#{price := 0} end, + [#{hello => 0, price => nil}]) + end, + "lists:map(fun (X) -> X#{price := 0} end, + [#{hello => 0, price => nil}]).", + [#{hello => 0, price => 0}]), error_check("[camembert]#{}.", {badarg,[camembert]}), error_check("#{} = 1.", {badmatch,1}), ok. diff --git a/lib/stdlib/test/erl_lint_SUITE.erl b/lib/stdlib/test/erl_lint_SUITE.erl index f822986981..ea61b2082b 100644 --- a/lib/stdlib/test/erl_lint_SUITE.erl +++ b/lib/stdlib/test/erl_lint_SUITE.erl @@ -52,7 +52,7 @@ guard/1, otp_4886/1, otp_4988/1, otp_5091/1, otp_5276/1, otp_5338/1, otp_5362/1, otp_5371/1, otp_7227/1, otp_5494/1, otp_5644/1, otp_5878/1, otp_5917/1, otp_6585/1, otp_6885/1, otp_10436/1, otp_11254/1, - otp_11772/1, otp_11771/1, + otp_11772/1, otp_11771/1, otp_11872/1, export_all/1, bif_clash/1, behaviour_basic/1, behaviour_multiple/1, @@ -88,7 +88,7 @@ all() -> otp_4886, otp_4988, otp_5091, otp_5276, otp_5338, otp_5362, otp_5371, otp_7227, otp_5494, otp_5644, otp_5878, otp_5917, otp_6585, otp_6885, otp_10436, otp_11254, - otp_11772, otp_11771, export_all, + otp_11772, otp_11771, otp_11872, export_all, bif_clash, behaviour_basic, behaviour_multiple, otp_7550, otp_8051, format_warn, {group, on_load}, too_many_arguments, basic_errors, bin_syntax_errors, predef, @@ -1279,10 +1279,9 @@ guard(Config) when is_list(Config) -> tuple. ">>, [nowarn_obsolete_guard], - {error, + {errors, [{6,erl_lint,illegal_guard_expr},{18,erl_lint,illegal_guard_expr}], - [{18,erl_lint,{removed,{erlang,is_constant,1}, - "Removed in R13B"}}]}}, + []}}, {guard2, <<"-record(apa,{}). t1(A) when atom(A), atom(A) -> @@ -1341,14 +1340,11 @@ guard(Config) when is_list(Config) -> tuple. ">>, [nowarn_obsolete_guard], - {error,[{6,erl_lint,illegal_guard_expr}, - {6,erl_lint,illegal_guard_expr}, - {18,erl_lint,illegal_guard_expr}, - {18,erl_lint,illegal_guard_expr}], - [{18,erl_lint,{removed,{erlang,is_constant,1}, - "Removed in R13B"}}, - {18,erl_lint,{removed,{erlang,is_constant,1}, - "Removed in R13B"}}]}}, + {errors,[{6,erl_lint,illegal_guard_expr}, + {6,erl_lint,illegal_guard_expr}, + {18,erl_lint,illegal_guard_expr}, + {18,erl_lint,illegal_guard_expr}], + []}}, {guard3, <<"-record(apa,{}). t2(A) when atom(A); atom(A) -> @@ -2634,6 +2630,29 @@ otp_11771(Config) when is_list(Config) -> []} = run_test2(Config, Ts, []), ok. +otp_11872(doc) -> + "OTP-11872. The type map() undefined when exported."; +otp_11872(suite) -> []; +otp_11872(Config) when is_list(Config) -> + Ts = <<" + -module(map). + + -compile(export_all). + + -export_type([map/0, product/0]). + + -opaque map() :: dict(). + + -spec t() -> map(). + + t() -> + 1. + ">>, + {error,[{6,erl_lint,{undefined_type,{product,0}}}], + [{8,erl_lint,{new_var_arity_type,map}}]} = + run_test2(Config, Ts, []), + ok. + export_all(doc) -> "OTP-7392. Warning for export_all."; export_all(Config) when is_list(Config) -> @@ -3410,7 +3429,19 @@ maps(Config) -> {4,erl_lint,illegal_map_key}, {6,erl_lint,illegal_map_key}, {8,erl_lint,illegal_map_key}, - {10,erl_lint,illegal_map_key}],[]}}], + {10,erl_lint,illegal_map_key}],[]}}, + {errors_in_map_keys_pattern, + <<"t(#{ a := 2, + #{} := A, + #{ 3 => 33 } := hi, + #{ 3 := 33 } := hi, + #{ hi => 54, \"hello\" => 45 } := hi, + #{ V => 33 } := hi }) -> + A. + ">>, + [], + {errors,[{4,erl_lint,illegal_map_key}, + {6,erl_lint,{illegal_map_key_variable,'V'}}],[]}}], [] = run(Config, Ts), ok. diff --git a/lib/stdlib/test/erl_pp_SUITE.erl b/lib/stdlib/test/erl_pp_SUITE.erl index 390322a5fa..babf3a49eb 100644 --- a/lib/stdlib/test/erl_pp_SUITE.erl +++ b/lib/stdlib/test/erl_pp_SUITE.erl @@ -993,6 +993,16 @@ maps_syntax(Config) when is_list(Config) -> ok = pp_expr(<<"#{ a => 1, <<\"hi\">> => \"world\", 33 => 1.0 }">>), ok = pp_expr(<<"#{ a := V1, <<\"hi\">> := V2 } = M">>), ok = pp_expr(<<"M#{ a => V1, <<\"hi\">> := V2 }">>), + F = <<"-module(maps_type_syntax).\n" + "-compile(export_all).\n" + "-type t1() :: map().\n" + "-type t2() :: #{ atom() => integer(), atom() => float() }.\n" + "-spec f1(t1()) -> 'true'.\n" + "f1(M) when is_map(M) -> true.\n" + "-spec f2(t2()) -> integer().\n" + "f2(#{a := V1,b := V2}) -> V1 + V2.\n" + "\n">>, + ok = pp_forms(F), ok. diff --git a/lib/stdlib/test/erl_scan_SUITE.erl b/lib/stdlib/test/erl_scan_SUITE.erl index 35067e8116..9be9f641c8 100644 --- a/lib/stdlib/test/erl_scan_SUITE.erl +++ b/lib/stdlib/test/erl_scan_SUITE.erl @@ -204,20 +204,20 @@ reserved_words() -> [begin ?line {RW, true} = {RW, erl_scan:reserved_word(RW)}, S = atom_to_list(RW), - Ts = [{RW,1}], + Ts = [{RW,{1,1}}], ?line test_string(S, Ts) end || RW <- L], ok. atoms() -> - ?line test_string("a - b", [{atom,1,a},{atom,2,b}]), - ?line test_string("'a b'", [{atom,1,'a b'}]), - ?line test_string("a", [{atom,1,a}]), - ?line test_string("a@2", [{atom,1,a@2}]), - ?line test_string([39,65,200,39], [{atom,1,'AÈ'}]), - ?line test_string("ärlig östen", [{atom,1,ärlig},{atom,1,östen}]), + test_string("a + b", [{atom,{1,1},a},{atom,{2,18},b}]), + test_string("'a b'", [{atom,{1,1},'a b'}]), + test_string("a", [{atom,{1,1},a}]), + test_string("a@2", [{atom,{1,1},a@2}]), + test_string([39,65,200,39], [{atom,{1,1},'AÈ'}]), + test_string("ärlig östen", [{atom,{1,1},ärlig},{atom,{1,7},östen}]), ?line {ok,[{atom,_,'$a'}],{1,6}} = erl_scan:string("'$\\a'", {1,1}), ?line test("'$\\a'"), @@ -230,7 +230,7 @@ punctuations() -> %% One token at a time: [begin W = list_to_atom(S), - Ts = [{W,1}], + Ts = [{W,{1,1}}], ?line test_string(S, Ts) end || S <- L], Three = ["/=:=", "<=:=", "==:=", ">=:="], % three tokens... @@ -246,53 +246,60 @@ punctuations() -> [begin W1 = list_to_atom(S1), W2 = list_to_atom(S2), - Ts = [{W1,1},{W2,1}], + Ts = [{W1,{1,1}},{W2,{1,-L2+1}}], ?line test_string(S, Ts) - end || {S,[{_,S1,S2}|_]} <- SL], + end || {S,[{L2,S1,S2}|_]} <- SL], - PTs1 = [{'!',1},{'(',1},{')',1},{',',1},{';',1},{'=',1},{'[',1}, - {']',1},{'{',1},{'|',1},{'}',1}], + PTs1 = [{'!',{1,1}},{'(',{1,2}},{')',{1,3}},{',',{1,4}},{';',{1,5}}, + {'=',{1,6}},{'[',{1,7}},{']',{1,8}},{'{',{1,9}},{'|',{1,10}}, + {'}',{1,11}}], ?line test_string("!(),;=[]{|}", PTs1), - PTs2 = [{'#',1},{'&',1},{'*',1},{'+',1},{'/',1}, - {':',1},{'<',1},{'>',1},{'?',1},{'@',1}, - {'\\',1},{'^',1},{'`',1},{'~',1}], + PTs2 = [{'#',{1,1}},{'&',{1,2}},{'*',{1,3}},{'+',{1,4}},{'/',{1,5}}, + {':',{1,6}},{'<',{1,7}},{'>',{1,8}},{'?',{1,9}},{'@',{1,10}}, + {'\\',{1,11}},{'^',{1,12}},{'`',{1,13}},{'~',{1,14}}], ?line test_string("#&*+/:<>?@\\^`~", PTs2), - ?line test_string(".. ", [{'..',1}]), - ?line test("1 .. 2"), - ?line test_string("...", [{'...',1}]), + test_string(".. ", [{'..',{1,1}}]), + test_string("1 .. 2", + [{integer,{1,1},1},{'..',{1,3}},{integer,{1,6},2}]), + test_string("...", [{'...',{1,1}}]), ok. comments() -> ?line test("a %%\n b"), ?line {ok,[],1} = erl_scan:string("%"), ?line test("a %%\n b"), - ?line {ok,[{atom,_,a},{atom,_,b}],{2,3}} = + {ok,[{atom,{1,1},a},{atom,{2,2},b}],{2,3}} = erl_scan:string("a %%\n b",{1,1}), - ?line {ok,[{atom,_,a},{comment,_,"%%"},{atom,_,b}],{2,3}} = + {ok,[{atom,{1,1},a},{comment,{1,3},"%%"},{atom,{2,2},b}],{2,3}} = erl_scan:string("a %%\n b",{1,1}, [return_comments]), - ?line {ok,[{atom,_,a}, - {white_space,_," "}, - {white_space,_,"\n "}, - {atom,_,b}], - {2,3}} = + {ok,[{atom,{1,1},a}, + {white_space,{1,2}," "}, + {white_space,{1,5},"\n "}, + {atom,{2,2},b}], + {2,3}} = erl_scan:string("a %%\n b",{1,1},[return_white_spaces]), - ?line {ok,[{atom,_,a}, - {white_space,_," "}, - {comment,_,"%%"}, - {white_space,_,"\n "}, - {atom,_,b}], - {2,3}} = erl_scan:string("a %%\n b",{1,1},[return]), + {ok,[{atom,{1,1},a}, + {white_space,{1,2}," "}, + {comment,{1,3},"%%"}, + {white_space,{1,5},"\n "}, + {atom,{2,2},b}], + {2,3}} = erl_scan:string("a %%\n b",{1,1},[return]), ok. errors() -> ?line {error,{1,erl_scan,{string,$',"qa"}},1} = erl_scan:string("'qa"), %' + {error,{{1,1},erl_scan,{string,$',"qa"}},{1,4}} = %' + erl_scan:string("'qa", {1,1}, []), %' ?line {error,{1,erl_scan,{string,$","str"}},1} = %" erl_scan:string("\"str"), %" + {error,{{1,1},erl_scan,{string,$","str"}},{1,5}} = %" + erl_scan:string("\"str", {1,1}, []), %" ?line {error,{1,erl_scan,char},1} = erl_scan:string("$"), - ?line test_string([34,65,200,34], [{string,1,"AÈ"}]), - ?line test_string("\\", [{'\\',1}]), + {error,{{1,1},erl_scan,char},{1,2}} = erl_scan:string("$", {1,1}, []), + test_string([34,65,200,34], [{string,{1,1},"AÈ"}]), + test_string("\\", [{'\\',{1,1}}]), ?line {'EXIT',_} = (catch {foo, erl_scan:string('$\\a', {1,1})}), % type error ?line {'EXIT',_} = @@ -304,7 +311,7 @@ errors() -> integers() -> [begin I = list_to_integer(S), - Ts = [{integer,1,I}], + Ts = [{integer,{1,1},I}], ?line test_string(S, Ts) end || S <- [[N] || N <- lists:seq($0, $9)] ++ ["2323","000"] ], ok. @@ -313,14 +320,16 @@ base_integers() -> [begin B = list_to_integer(BS), I = erlang:list_to_integer(S, B), - Ts = [{integer,1,I}], + Ts = [{integer,{1,1},I}], ?line test_string(BS++"#"++S, Ts) end || {BS,S} <- [{"2","11"}, {"5","23234"}, {"12","05a"}, {"16","abcdef"}, {"16","ABCDEF"}] ], ?line {error,{1,erl_scan,{base,1}},1} = erl_scan:string("1#000"), + {error,{{1,1},erl_scan,{base,1}},{1,2}} = + erl_scan:string("1#000", {1,1}, []), - ?line test_string("12#bc", [{integer,1,11},{atom,1,c}]), + test_string("12#bc", [{integer,{1,1},11},{atom,{1,5},c}]), [begin Str = BS ++ "#" ++ S, @@ -329,40 +338,53 @@ base_integers() -> end || {BS,S} <- [{"3","3"},{"15","f"}, {"12","c"}] ], ?line {ok,[{integer,1,239},{'@',1}],1} = erl_scan:string("16#ef@"), - ?line {ok,[{integer,1,14},{atom,1,g@}],1} = erl_scan:string("16#eg@"), + {ok,[{integer,{1,1},239},{'@',{1,6}}],{1,7}} = + erl_scan:string("16#ef@", {1,1}, []), + {ok,[{integer,{1,1},14},{atom,{1,5},g@}],{1,7}} = + erl_scan:string("16#eg@", {1,1}, []), ok. floats() -> [begin F = list_to_float(FS), - Ts = [{float,1,F}], + Ts = [{float,{1,1},F}], ?line test_string(FS, Ts) end || FS <- ["1.0","001.17","3.31200","1.0e0","1.0E17", "34.21E-18", "17.0E+14"]], - ?line test_string("1.e2", [{integer,1,1},{'.',1},{atom,1,e2}]), + test_string("1.e2", [{integer,{1,1},1},{'.',{1,2}},{atom,{1,3},e2}]), ?line {error,{1,erl_scan,{illegal,float}},1} = erl_scan:string("1.0e400"), + {error,{{1,1},erl_scan,{illegal,float}},{1,8}} = + erl_scan:string("1.0e400", {1,1}, []), [begin - ?line {error,{1,erl_scan,{illegal,float}},1} = erl_scan:string(S) + {error,{1,erl_scan,{illegal,float}},1} = erl_scan:string(S), + {error,{{1,1},erl_scan,{illegal,float}},{1,_}} = + erl_scan:string(S, {1,1}, []) end || S <- ["1.14Ea"]], ok. dots() -> - Dot = [{".", {ok,[{dot,1}],1}}, - {". ", {ok,[{dot,1}],1}}, - {".\n", {ok,[{dot,1}],2}}, - {".%", {ok,[{dot,1}],1}}, - {".\210",{ok,[{dot,1}],1}}, - {".% öh",{ok,[{dot,1}],1}}, - {".%\n", {ok,[{dot,1}],2}}, - {".$", {error,{1,erl_scan,char},1}}, - {".$\\", {error,{1,erl_scan,char},1}}, - {".a", {ok,[{'.',1},{atom,1,a}],1}} + Dot = [{".", {ok,[{dot,1}],1}, {ok,[{dot,{1,1}}],{1,2}}}, + {". ", {ok,[{dot,1}],1}, {ok,[{dot,{1,1}}],{1,3}}}, + {".\n", {ok,[{dot,1}],2}, {ok,[{dot,{1,1}}],{2,1}}}, + {".%", {ok,[{dot,1}],1}, {ok,[{dot,{1,1}}],{1,3}}}, + {".\210",{ok,[{dot,1}],1}, {ok,[{dot,{1,1}}],{1,3}}}, + {".% öh",{ok,[{dot,1}],1}, {ok,[{dot,{1,1}}],{1,6}}}, + {".%\n", {ok,[{dot,1}],2}, {ok,[{dot,{1,1}}],{2,1}}}, + {".$", {error,{1,erl_scan,char},1}, + {error,{{1,2},erl_scan,char},{1,3}}}, + {".$\\", {error,{1,erl_scan,char},1}, + {error,{{1,2},erl_scan,char},{1,4}}}, + {".a", {ok,[{'.',1},{atom,1,a}],1}, + {ok,[{'.',{1,1}},{atom,{1,2},a}],{1,3}}} ], - ?line [R = erl_scan:string(S) || {S, R} <- Dot], + [begin + R = erl_scan:string(S), + R2 = erl_scan:string(S, {1,1}, []) + end || {S, R, R2} <- Dot], ?line {ok,[{dot,_}=T1],{1,2}} = erl_scan:string(".", {1,1}, text), ?line [{column,1},{length,1},{line,1},{text,"."}] = @@ -379,55 +401,55 @@ dots() -> ?line {error,{{1,2},erl_scan,char},{1,4}} = erl_scan:string(".$\\", {1,1}), - ?line test(". "), - ?line test(". "), - ?line test(".\n"), - ?line test(".\n\n"), - ?line test(".\n\r"), - ?line test(".\n\n\n"), - ?line test(".\210"), - ?line test(".%\n"), - ?line test(".a"), - - ?line test("%. \n. "), + test_string(". ", [{dot,{1,1}}]), + test_string(". ", [{dot,{1,1}}]), + test_string(".\n", [{dot,{1,1}}]), + test_string(".\n\n", [{dot,{1,1}}]), + test_string(".\n\r", [{dot,{1,1}}]), + test_string(".\n\n\n", [{dot,{1,1}}]), + test_string(".\210", [{dot,{1,1}}]), + test_string(".%\n", [{dot,{1,1}}]), + test_string(".a", [{'.',{1,1}},{atom,{1,2},a}]), + + test_string("%. \n. ", [{dot,{2,1}}]), ?line {more,C} = erl_scan:tokens([], "%. ",{1,1}, return), - ?line {done,{ok,[{comment,_,"%. "}, - {white_space,_,"\n"}, - {dot,_}], - {2,3}}, ""} = + {done,{ok,[{comment,{1,1},"%. "}, + {white_space,{1,4},"\n"}, + {dot,{2,1}}], + {2,3}}, ""} = erl_scan:tokens(C, "\n. ", {1,1}, return), % any loc, any options ?line [test_string(S, R) || - {S, R} <- [{".$\n", [{'.',1},{char,1,$\n}]}, - {"$\\\n", [{char,1,$\n}]}, - {"'\\\n'", [{atom,1,'\n'}]}, - {"$\n", [{char,1,$\n}]}] ], + {S, R} <- [{".$\n", [{'.',{1,1}},{char,{1,2},$\n}]}, + {"$\\\n", [{char,{1,1},$\n}]}, + {"'\\\n'", [{atom,{1,1},'\n'}]}, + {"$\n", [{char,{1,1},$\n}]}] ], ok. chars() -> [begin L = lists:flatten(io_lib:format("$\\~.8b", [C])), - Ts = [{char,1,C}], + Ts = [{char,{1,1},C}], ?line test_string(L, Ts) end || C <- lists:seq(0, 255)], %% Leading zeroes... [begin L = lists:flatten(io_lib:format("$\\~3.8.0b", [C])), - Ts = [{char,1,C}], + Ts = [{char,{1,1},C}], ?line test_string(L, Ts) end || C <- lists:seq(0, 255)], %% $\^\n now increments the line... [begin L = "$\\^" ++ [C], - Ts = [{char,1,C band 2#11111}], + Ts = [{char,{1,1},C band 2#11111}], ?line test_string(L, Ts) end || C <- lists:seq(0, 255)], [begin L = "$\\" ++ [C], - Ts = [{char,1,V}], + Ts = [{char,{1,1},V}], ?line test_string(L, Ts) end || {C,V} <- [{$n,$\n}, {$r,$\r}, {$t,$\t}, {$v,$\v}, {$b,$\b}, {$f,$\f}, {$e,$\e}, {$s,$\s}, @@ -440,45 +462,45 @@ chars() -> No = EC ++ Ds ++ X ++ New, [begin L = "$\\" ++ [C], - Ts = [{char,1,C}], + Ts = [{char,{1,1},C}], ?line test_string(L, Ts) end || C <- lists:seq(0, 255) -- No], [begin L = "'$\\" ++ [C] ++ "'", - Ts = [{atom,1,list_to_atom("$"++[C])}], + Ts = [{atom,{1,1},list_to_atom("$"++[C])}], ?line test_string(L, Ts) end || C <- lists:seq(0, 255) -- No], - ?line test_string("\"\\013a\\\n\"", [{string,1,"\va\n"}]), + test_string("\"\\013a\\\n\"", [{string,{1,1},"\va\n"}]), - ?line test_string("'\n'", [{atom,1,'\n'}]), - ?line test_string("\"\n\a\"", [{string,1,"\na"}]), + test_string("'\n'", [{atom,{1,1},'\n'}]), + test_string("\"\n\a\"", [{string,{1,1},"\na"}]), %% No escape [begin L = "$" ++ [C], - Ts = [{char,1,C}], + Ts = [{char,{1,1},C}], ?line test_string(L, Ts) end || C <- lists:seq(0, 255) -- (No ++ [$\\])], - ?line test_string("$\n", [{char,1,$\n}]), + test_string("$\n", [{char,{1,1},$\n}]), ?line {error,{{1,1},erl_scan,char},{1,4}} = erl_scan:string("$\\^",{1,1}), - ?line test_string("$\\\n", [{char,1,$\n}]), + test_string("$\\\n", [{char,{1,1},$\n}]), %% Robert's scanner returns line 1: - ?line test_string("$\\\n", [{char,1,$\n}]), - ?line test_string("$\n\n", [{char,1,$\n}]), + test_string("$\\\n", [{char,{1,1},$\n}]), + test_string("$\n\n", [{char,{1,1},$\n}]), ?line test("$\n\n"), ok. variables() -> - ?line test_string(" \237_Aouåeiyäö", [{var,1,'_Aouåeiyäö'}]), - ?line test_string("A_b_c@", [{var,1,'A_b_c@'}]), - ?line test_string("V@2", [{var,1,'V@2'}]), - ?line test_string("ABDÀ", [{var,1,'ABDÀ'}]), - ?line test_string("Ärlig Östen", [{var,1,'Ärlig'},{var,1,'Östen'}]), + test_string(" \237_Aouåeiyäö", [{var,{1,7},'_Aouåeiyäö'}]), + test_string("A_b_c@", [{var,{1,1},'A_b_c@'}]), + test_string("V@2", [{var,{1,1},'V@2'}]), + test_string("ABDÀ", [{var,{1,1},'ABDÀ'}]), + test_string("Ärlig Östen", [{var,{1,1},'Ärlig'},{var,{1,7},'Östen'}]), ok. eof() -> @@ -508,11 +530,25 @@ eof() -> ?line {done,{ok,[{atom,1,a}],1},eof} = erl_scan:tokens(C5,eof,1), + %% With column. + {more, C6} = erl_scan:tokens([], "a", {1,1}), + %% An error before R13A. + %% {done,{error,{1,erl_scan,scan},1},eof} = + {done,{ok,[{atom,{1,1},a}],{1,2}},eof} = + erl_scan:tokens(C6,eof,1), + %% A dot followed by eof is special: ?line {more, C} = erl_scan:tokens([], "a.", 1), ?line {done,{ok,[{atom,1,a},{dot,1}],1},eof} = erl_scan:tokens(C,eof,1), ?line {ok,[{atom,1,foo},{dot,1}],1} = erl_scan:string("foo."), + %% With column. + {more, CCol} = erl_scan:tokens([], "a.", {1,1}), + {done,{ok,[{atom,{1,1},a},{dot,{1,2}}],{1,3}},eof} = + erl_scan:tokens(CCol,eof,1), + {ok,[{atom,{1,1},foo},{dot,{1,4}}],{1,5}} = + erl_scan:string("foo.", {1,1}, []), + ok. illegal() -> @@ -816,34 +852,34 @@ unicode() -> erl_scan:string([1089]), ?line {error,{{1,1},erl_scan,{illegal,character}},{1,2}} = erl_scan:string([1089], {1,1}), - ?line {error,{1,erl_scan,{illegal,atom}},1} = + {error,{1,erl_scan,{illegal,atom}},1} = erl_scan:string("'a"++[1089]++"b'", 1), - ?line {error,{{1,1},erl_scan,{illegal,atom}},{1,6}} = + {error,{{1,1},erl_scan,{illegal,atom}},{1,6}} = erl_scan:string("'a"++[1089]++"b'", {1,1}), ?line test("\"a"++[1089]++"b\""), - ?line {ok,[{char,1,1}],1} = + {ok,[{char,1,1}],1} = erl_scan:string([$$,$\\,$^,1089], 1), - ?line {error,{1,erl_scan,Error},1} = + {error,{1,erl_scan,Error},1} = erl_scan:string("\"qa\x{aaa}", 1), - ?line "unterminated string starting with \"qa"++[2730]++"\"" = + "unterminated string starting with \"qa"++[2730]++"\"" = erl_scan:format_error(Error), ?line {error,{{1,1},erl_scan,_},{1,11}} = erl_scan:string("\"qa\\x{aaa}",{1,1}), - ?line {error,{{1,1},erl_scan,{illegal,atom}},{1,12}} = + {error,{{1,1},erl_scan,{illegal,atom}},{1,12}} = erl_scan:string("'qa\\x{aaa}'",{1,1}), - ?line {ok,[{char,1,1089}],1} = + {ok,[{char,1,1089}],1} = erl_scan:string([$$,1089], 1), - ?line {ok,[{char,1,1089}],1} = + {ok,[{char,1,1089}],1} = erl_scan:string([$$,$\\,1089], 1), Qs = "$\\x{aaa}", - ?line {ok,[{char,1,$\x{aaa}}],1} = + {ok,[{char,1,$\x{aaa}}],1} = erl_scan:string(Qs, 1), - ?line {ok,[Q2],{1,9}} = + {ok,[Q2],{1,9}} = erl_scan:string("$\\x{aaa}", {1,1}, [text]), - ?line [{category,char},{column,1},{length,8}, + [{category,char},{column,1},{length,8}, {line,1},{symbol,16#aaa},{text,Qs}] = erl_scan:token_info(Q2), @@ -1164,7 +1200,13 @@ otp_11807(Config) when is_list(Config) -> (catch erl_parse:abstract("string", [{encoding,bad}])), ok. -test_string(String, Expected) -> +test_string(String, ExpectedWithCol) -> + {ok, ExpectedWithCol, _EndWithCol} = erl_scan:string(String, {1, 1}, []), + Expected = [ begin + {L,_C} = element(2, T), + setelement(2, T, L) + end + || T <- ExpectedWithCol ], {ok, Expected, _End} = erl_scan:string(String), test(String). diff --git a/lib/stdlib/test/filelib_SUITE.erl b/lib/stdlib/test/filelib_SUITE.erl index 4a67d68428..040ae1effc 100644 --- a/lib/stdlib/test/filelib_SUITE.erl +++ b/lib/stdlib/test/filelib_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2013. All Rights Reserved. +%% Copyright Ericsson AB 2005-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -23,7 +23,8 @@ init_per_group/2,end_per_group/2, init_per_testcase/2,end_per_testcase/2, wildcard_one/1,wildcard_two/1,wildcard_errors/1, - fold_files/1,otp_5960/1,ensure_dir_eexist/1]). + fold_files/1,otp_5960/1,ensure_dir_eexist/1,ensure_dir_symlink/1, + wildcard_symlink/1, is_file_symlink/1, file_props_symlink/1]). -import(lists, [foreach/2]). @@ -43,7 +44,8 @@ suite() -> [{ct_hooks,[ts_install_cth]}]. all() -> [wildcard_one, wildcard_two, wildcard_errors, - fold_files, otp_5960, ensure_dir_eexist]. + fold_files, otp_5960, ensure_dir_eexist, ensure_dir_symlink, + wildcard_symlink, is_file_symlink, file_props_symlink]. groups() -> []. @@ -366,3 +368,139 @@ ensure_dir_eexist(Config) when is_list(Config) -> ?line {error, eexist} = filelib:ensure_dir(NeedFile), ?line {error, eexist} = filelib:ensure_dir(NeedFileB), ok. + +ensure_dir_symlink(Config) when is_list(Config) -> + PrivDir = ?config(priv_dir, Config), + Dir = filename:join(PrivDir, "ensure_dir_symlink"), + Name = filename:join(Dir, "same_name_as_file_and_dir"), + ok = filelib:ensure_dir(Name), + ok = file:write_file(Name, <<"some string\n">>), + %% With a symlink to the directory. + Symlink = filename:join(PrivDir, "ensure_dir_symlink_link"), + case file:make_symlink(Dir, Symlink) of + {error,enotsup} -> + {skip,"Symlinks not supported on this platform"}; + {error,eperm} -> + {win32,_} = os:type(), + {skip,"Windows user not privileged to create symlinks"}; + ok -> + SymlinkedName = filename:join(Symlink, "same_name_as_file_and_dir"), + ok = filelib:ensure_dir(SymlinkedName) + end. + +wildcard_symlink(Config) when is_list(Config) -> + PrivDir = ?config(priv_dir, Config), + Dir = filename:join(PrivDir, ?MODULE_STRING++"_wildcard_symlink"), + SubDir = filename:join(Dir, "sub"), + AFile = filename:join(SubDir, "a_file"), + Alias = filename:join(Dir, "symlink"), + ok = file:make_dir(Dir), + ok = file:make_dir(SubDir), + ok = file:write_file(AFile, "not that big\n"), + case file:make_symlink(AFile, Alias) of + {error, enotsup} -> + {skip, "Links not supported on this platform"}; + {error, eperm} -> + {win32,_} = os:type(), + {skip, "Windows user not privileged to create symlinks"}; + ok -> + ["sub","symlink"] = + basenames(Dir, filelib:wildcard(filename:join(Dir, "*"))), + ["symlink"] = + basenames(Dir, filelib:wildcard(filename:join(Dir, "symlink"))), + ["sub","symlink"] = + basenames(Dir, filelib:wildcard(filename:join(Dir, "*"), + erl_prim_loader)), + ["symlink"] = + basenames(Dir, filelib:wildcard(filename:join(Dir, "symlink"), + erl_prim_loader)), + ["sub","symlink"] = + basenames(Dir, filelib:wildcard(filename:join(Dir, "*"), + prim_file)), + ["symlink"] = + basenames(Dir, filelib:wildcard(filename:join(Dir, "symlink"), + prim_file)), + ok = file:delete(AFile), + %% The symlink should still be visible even when its target + %% has been deleted. + ["sub","symlink"] = + basenames(Dir, filelib:wildcard(filename:join(Dir, "*"))), + ["symlink"] = + basenames(Dir, filelib:wildcard(filename:join(Dir, "symlink"))), + ["sub","symlink"] = + basenames(Dir, filelib:wildcard(filename:join(Dir, "*"), + erl_prim_loader)), + ["symlink"] = + basenames(Dir, filelib:wildcard(filename:join(Dir, "symlink"), + erl_prim_loader)), + ["sub","symlink"] = + basenames(Dir, filelib:wildcard(filename:join(Dir, "*"), + prim_file)), + ["symlink"] = + basenames(Dir, filelib:wildcard(filename:join(Dir, "symlink"), + prim_file)), + ok + end. + +basenames(Dir, Files) -> + [begin + Dir = filename:dirname(F), + filename:basename(F) + end || F <- Files]. + +is_file_symlink(Config) -> + PrivDir = ?config(priv_dir, Config), + Dir = filename:join(PrivDir, ?MODULE_STRING++"_is_file_symlink"), + SubDir = filename:join(Dir, "sub"), + AFile = filename:join(SubDir, "a_file"), + DirAlias = filename:join(Dir, "dir_symlink"), + FileAlias = filename:join(Dir, "file_symlink"), + ok = file:make_dir(Dir), + ok = file:make_dir(SubDir), + ok = file:write_file(AFile, "not that big\n"), + case file:make_symlink(SubDir, DirAlias) of + {error, enotsup} -> + {skip, "Links not supported on this platform"}; + {error, eperm} -> + {win32,_} = os:type(), + {skip, "Windows user not privileged to create symlinks"}; + ok -> + true = filelib:is_dir(DirAlias), + true = filelib:is_dir(DirAlias, erl_prim_loader), + true = filelib:is_dir(DirAlias, prim_file), + true = filelib:is_file(DirAlias), + true = filelib:is_file(DirAlias, erl_prim_loader), + true = filelib:is_file(DirAlias, prim_file), + ok = file:make_symlink(AFile,FileAlias), + true = filelib:is_file(FileAlias), + true = filelib:is_file(FileAlias, erl_prim_loader), + true = filelib:is_file(FileAlias, prim_file), + true = filelib:is_regular(FileAlias), + true = filelib:is_regular(FileAlias, erl_prim_loader), + true = filelib:is_regular(FileAlias, prim_file), + ok + end. + +file_props_symlink(Config) -> + PrivDir = ?config(priv_dir, Config), + Dir = filename:join(PrivDir, ?MODULE_STRING++"_file_props_symlink"), + AFile = filename:join(Dir, "a_file"), + Alias = filename:join(Dir, "symlink"), + ok = file:make_dir(Dir), + ok = file:write_file(AFile, "not that big\n"), + case file:make_symlink(AFile, Alias) of + {error, enotsup} -> + {skip, "Links not supported on this platform"}; + {error, eperm} -> + {win32,_} = os:type(), + {skip, "Windows user not privileged to create symlinks"}; + ok -> + {_,_} = LastMod = filelib:last_modified(AFile), + LastMod = filelib:last_modified(Alias), + LastMod = filelib:last_modified(Alias, erl_prim_loader), + LastMod = filelib:last_modified(Alias, prim_file), + FileSize = filelib:file_size(AFile), + FileSize = filelib:file_size(Alias), + FileSize = filelib:file_size(Alias, erl_prim_loader), + FileSize = filelib:file_size(Alias, prim_file) + end. diff --git a/lib/stdlib/test/gen_fsm_SUITE.erl b/lib/stdlib/test/gen_fsm_SUITE.erl index 8aeec07ae8..336065b258 100644 --- a/lib/stdlib/test/gen_fsm_SUITE.erl +++ b/lib/stdlib/test/gen_fsm_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2013. All Rights Reserved. +%% Copyright Ericsson AB 1996-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -31,7 +31,9 @@ -export([shutdown/1]). --export([ sys1/1, call_format_status/1, error_format_status/1, get_state/1, replace_state/1]). +-export([ sys1/1, + call_format_status/1, error_format_status/1, terminate_crash_format/1, + get_state/1, replace_state/1]). -export([hibernate/1,hiber_idle/3,hiber_wakeup/3,hiber_idle/2,hiber_wakeup/2]). @@ -66,7 +68,8 @@ groups() -> start8, start9, start10, start11, start12]}, {abnormal, [], [abnormal1, abnormal2]}, {sys, [], - [sys1, call_format_status, error_format_status, get_state, replace_state]}]. + [sys1, call_format_status, error_format_status, terminate_crash_format, + get_state, replace_state]}]. init_per_suite(Config) -> Config. @@ -403,7 +406,7 @@ error_format_status(Config) when is_list(Config) -> receive {error,_GroupLeader,{Pid, "** State machine"++_, - [Pid,{_,_,badreturn},idle,StateData,_]}} -> + [Pid,{_,_,badreturn},idle,{formatted,StateData},_]}} -> ok; Other -> ?line io:format("Unexpected: ~p", [Other]), @@ -413,6 +416,29 @@ error_format_status(Config) when is_list(Config) -> process_flag(trap_exit, OldFl), ok. +terminate_crash_format(Config) when is_list(Config) -> + error_logger_forwarder:register(), + OldFl = process_flag(trap_exit, true), + StateData = crash_terminate, + {ok, Pid} = gen_fsm:start(gen_fsm_SUITE, {state_data, StateData}, []), + stop_it(Pid), + receive + {error,_GroupLeader,{Pid, + "** State machine"++_, + [Pid,{_,_,_},idle,{formatted, StateData},_]}} -> + ok; + Other -> + io:format("Unexpected: ~p", [Other]), + ?t:fail() + after 5000 -> + io:format("Timeout: expected error logger msg", []), + ?t:fail() + end, + [] = ?t:messages_get(), + process_flag(trap_exit, OldFl), + ok. + + get_state(Config) when is_list(Config) -> State = self(), {ok, Pid} = gen_fsm:start(?MODULE, {state_data, State}, []), @@ -867,7 +893,8 @@ init({state_data, StateData}) -> init(_) -> {ok, idle, state_data}. - +terminate(_, _State, crash_terminate) -> + exit({crash, terminate}); terminate({From, stopped}, State, _Data) -> From ! {self(), {stopped, State}}, ok; @@ -1005,6 +1032,6 @@ handle_sync_event({get, _Pid}, _From, State, Data) -> {reply, {state, State, Data}, State, Data}. format_status(terminate, [_Pdict, StateData]) -> - StateData; + {formatted, StateData}; format_status(normal, [_Pdict, _StateData]) -> [format_status_called]. diff --git a/lib/stdlib/test/gen_server_SUITE.erl b/lib/stdlib/test/gen_server_SUITE.erl index 960e7f60e7..42694d8b5d 100644 --- a/lib/stdlib/test/gen_server_SUITE.erl +++ b/lib/stdlib/test/gen_server_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2013. All Rights Reserved. +%% Copyright Ericsson AB 1996-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -32,7 +32,8 @@ spec_init_local_registered_parent/1, spec_init_global_registered_parent/1, otp_5854/1, hibernate/1, otp_7669/1, call_format_status/1, - error_format_status/1, get_state/1, replace_state/1, call_with_huge_message_queue/1 + error_format_status/1, terminate_crash_format/1, + get_state/1, replace_state/1, call_with_huge_message_queue/1 ]). % spawn export @@ -56,7 +57,8 @@ all() -> call_remote_n3, spec_init, spec_init_local_registered_parent, spec_init_global_registered_parent, otp_5854, hibernate, - otp_7669, call_format_status, error_format_status, + otp_7669, + call_format_status, error_format_status, terminate_crash_format, get_state, replace_state, call_with_huge_message_queue]. @@ -273,7 +275,7 @@ crash(Config) when is_list(Config) -> receive {error,_GroupLeader4,{Pid4, "** Generic server"++_, - [Pid4,crash,state4,crashed]}} -> + [Pid4,crash,{formatted, state4},crashed]}} -> ok; Other4a -> ?line io:format("Unexpected: ~p", [Other4a]), @@ -1024,7 +1026,7 @@ error_format_status(Config) when is_list(Config) -> receive {error,_GroupLeader,{Pid, "** Generic server"++_, - [Pid,crash,State,crashed]}} -> + [Pid,crash,{formatted, State},crashed]}} -> ok; Other -> ?line io:format("Unexpected: ~p", [Other]), @@ -1034,6 +1036,31 @@ error_format_status(Config) when is_list(Config) -> process_flag(trap_exit, OldFl), ok. +%% Verify that error when terminating correctly calls our format_status/2 fun +%% +terminate_crash_format(Config) when is_list(Config) -> + error_logger_forwarder:register(), + OldFl = process_flag(trap_exit, true), + State = crash_terminate, + {ok, Pid} = gen_server:start_link(?MODULE, {state, State}, []), + gen_server:call(Pid, stop), + receive {'EXIT', Pid, {crash, terminate}} -> ok end, + receive + {error,_GroupLeader,{Pid, + "** Generic server"++_, + [Pid,stop, {formatted, State},{crash, terminate}]}} -> + ok; + Other -> + io:format("Unexpected: ~p", [Other]), + ?t:fail() + after 5000 -> + io:format("Timeout: expected error logger msg", []), + ?t:fail() + end, + ?t:messages_get(), + process_flag(trap_exit, OldFl), + ok. + %% Verify that sys:get_state correctly returns gen_server state %% get_state(suite) -> @@ -1323,10 +1350,12 @@ terminate({From, stopped}, _State) -> terminate({From, stopped_info}, _State) -> From ! {self(), stopped_info}, ok; +terminate(_, crash_terminate) -> + exit({crash, terminate}); terminate(_Reason, _State) -> ok. format_status(terminate, [_PDict, State]) -> - State; + {formatted, State}; format_status(normal, [_PDict, _State]) -> format_status_called. diff --git a/lib/stdlib/test/io_SUITE.erl b/lib/stdlib/test/io_SUITE.erl index 5a8971c071..3a76275f31 100644 --- a/lib/stdlib/test/io_SUITE.erl +++ b/lib/stdlib/test/io_SUITE.erl @@ -30,7 +30,7 @@ io_fread_newlines/1, otp_8989/1, io_lib_fread_literal/1, printable_range/1, io_lib_print_binary_depth_one/1, otp_10302/1, otp_10755/1, - otp_10836/1]). + otp_10836/1, io_lib_width_too_small/1]). -export([pretty/2]). @@ -69,7 +69,8 @@ all() -> io_lib_collect_line_3_wb, cr_whitespace_in_string, io_fread_newlines, otp_8989, io_lib_fread_literal, printable_range, - io_lib_print_binary_depth_one, otp_10302, otp_10755, otp_10836]. + io_lib_print_binary_depth_one, otp_10302, otp_10755, otp_10836, + io_lib_width_too_small]. groups() -> []. @@ -2213,3 +2214,8 @@ compile_file(File, Text, Config) -> try compile:file(Fname, [return]) after ok %file:delete(Fname) end. + +io_lib_width_too_small(Config) -> + "**" = lists:flatten(io_lib:format("~2.3w", [3.14])), + "**" = lists:flatten(io_lib:format("~2.5w", [3.14])), + ok. diff --git a/lib/stdlib/test/maps_SUITE.erl b/lib/stdlib/test/maps_SUITE.erl new file mode 100644 index 0000000000..dda20a615b --- /dev/null +++ b/lib/stdlib/test/maps_SUITE.erl @@ -0,0 +1,80 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 1997-2014. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%%%---------------------------------------------------------------- +%%% Purpose: Test suite for the 'maps' module. +%%%----------------------------------------------------------------- + +-module(maps_SUITE). + +-include_lib("test_server/include/test_server.hrl"). + +-define(default_timeout, ?t:minutes(1)). + +% Test server specific exports +-export([all/0]). +-export([suite/0]). +-export([init_per_suite/1]). +-export([end_per_suite/1]). +-export([init_per_testcase/2]). +-export([end_per_testcase/2]). + +-export([t_get_3/1,t_with_2/1,t_without_2/1]). + +suite() -> + [{ct_hooks, [ts_install_cth]}]. + +all() -> + [t_get_3,t_with_2,t_without_2]. + +init_per_suite(Config) -> + Config. + +end_per_suite(_Config) -> + ok. + +init_per_testcase(_Case, Config) -> + Dog=test_server:timetrap(?default_timeout), + [{watchdog, Dog}|Config]. + +end_per_testcase(_Case, Config) -> + Dog=?config(watchdog, Config), + test_server:timetrap_cancel(Dog), + ok. + +t_get_3(Config) when is_list(Config) -> + Map = #{ key1 => value1, key2 => value2 }, + DefaultValue = "Default value", + value1 = maps:get(key1, Map, DefaultValue), + value2 = maps:get(key2, Map, DefaultValue), + DefaultValue = maps:get(key3, Map, DefaultValue), + ok. + +t_without_2(_Config) -> + Ki = [11,22,33,44,55,66,77,88,99], + M0 = maps:from_list([{{k,I},{v,I}}||I<-lists:seq(1,100)]), + M1 = maps:from_list([{{k,I},{v,I}}||I<-lists:seq(1,100) -- Ki]), + M1 = maps:without([{k,I}||I <- Ki],M0), + ok. + +t_with_2(_Config) -> + Ki = [11,22,33,44,55,66,77,88,99], + M0 = maps:from_list([{{k,I},{v,I}}||I<-lists:seq(1,100)]), + M1 = maps:from_list([{{k,I},{v,I}}||I<-Ki]), + M1 = maps:with([{k,I}||I <- Ki],M0), + ok. diff --git a/lib/stdlib/test/shell_SUITE.erl b/lib/stdlib/test/shell_SUITE.erl index e016432f4d..f841e2c4a6 100644 --- a/lib/stdlib/test/shell_SUITE.erl +++ b/lib/stdlib/test/shell_SUITE.erl @@ -2532,6 +2532,11 @@ otp_6554(Config) when is_list(Config) -> "\n end.\nok.\n" = t(<<"begin F = fun() -> foo end, 1 end. B = F(). C = 17. b().">>), + ?line "3: command not found" = comm_err(<<"#{v(3) => v}.">>), + ?line "3: command not found" = comm_err(<<"#{k => v(3)}.">>), + ?line "3: command not found" = comm_err(<<"#{v(3) := v}.">>), + ?line "3: command not found" = comm_err(<<"#{k := v(3)}.">>), + ?line "3: command not found" = comm_err(<<"(v(3))#{}.">>), %% Tests I'd like to do: (you should try them manually) %% "catch spawn_link(fun() -> timer:sleep(1000), exit(foo) end)." %% "** exception error: foo" should be output after 1 second diff --git a/lib/stdlib/test/stdlib_SUITE.erl b/lib/stdlib/test/stdlib_SUITE.erl index 53a34511d9..59821220b4 100644 --- a/lib/stdlib/test/stdlib_SUITE.erl +++ b/lib/stdlib/test/stdlib_SUITE.erl @@ -94,10 +94,10 @@ appup_tests(App,{OkVsns,NokVsns}) -> ok. create_test_vsns(App) -> - This = erlang:system_info(otp_release), - FirstMajor = previous_major(This), + ThisMajor = erlang:system_info(otp_release), + FirstMajor = previous_major(ThisMajor), SecondMajor = previous_major(FirstMajor), - Ok = app_vsn(App,[FirstMajor]), + Ok = app_vsn(App,[ThisMajor,FirstMajor]), Nok0 = app_vsn(App,[SecondMajor]), Nok = case Ok of [Ok1|_] -> @@ -108,9 +108,9 @@ create_test_vsns(App) -> {Ok,Nok}. previous_major("17") -> - "r16"; -previous_major("r"++Rel) -> - "r"++previous_major(Rel); + "r16b"; +previous_major("r16b") -> + "r15b"; previous_major(Rel) -> integer_to_list(list_to_integer(Rel)-1). diff --git a/lib/stdlib/test/supervisor_SUITE.erl b/lib/stdlib/test/supervisor_SUITE.erl index ac5a34c3bc..836ea7c030 100644 --- a/lib/stdlib/test/supervisor_SUITE.erl +++ b/lib/stdlib/test/supervisor_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2013. All Rights Reserved. +%% Copyright Ericsson AB 1996-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -60,7 +60,7 @@ simple_one_for_one_extra/1, simple_one_for_one_shutdown/1]). %% Misc tests --export([child_unlink/1, tree/1, count_children_memory/1, +-export([child_unlink/1, tree/1, count_children/1, do_not_save_start_parameters_for_temporary_children/1, do_not_save_child_specs_for_temporary_children/1, simple_one_for_one_scale_many_temporary_children/1, @@ -82,7 +82,7 @@ all() -> {group, normal_termination}, {group, shutdown_termination}, {group, abnormal_termination}, child_unlink, tree, - count_children_memory, do_not_save_start_parameters_for_temporary_children, + count_children, do_not_save_start_parameters_for_temporary_children, do_not_save_child_specs_for_temporary_children, simple_one_for_one_scale_many_temporary_children, temporary_bystander, simple_global_supervisor, hanging_restart_loop, hanging_restart_loop_simple]. @@ -129,23 +129,10 @@ init_per_group(_GroupName, Config) -> end_per_group(_GroupName, Config) -> Config. -init_per_testcase(count_children_memory, Config) -> - try erlang:memory() of - _ -> - erts_debug:set_internal_state(available_internal_state, true), - Dog = ?t:timetrap(?TIMEOUT), - [{watchdog,Dog}|Config] - catch error:notsup -> - {skip, "+Meamin used during test; erlang:memory/1 not available"} - end; init_per_testcase(_Case, Config) -> Dog = ?t:timetrap(?TIMEOUT), [{watchdog,Dog}|Config]. -end_per_testcase(count_children_memory, Config) -> - catch erts_debug:set_internal_state(available_internal_state, false), - ?t:timetrap_cancel(?config(watchdog,Config)), - ok; end_per_testcase(_Case, Config) -> ?t:timetrap_cancel(?config(watchdog,Config)), ok. @@ -1249,34 +1236,24 @@ tree(Config) when is_list(Config) -> [0,0,0,0] = get_child_counts(NewSup2). %%------------------------------------------------------------------------- -%% Test that count_children does not eat memory. -count_children_memory(Config) when is_list(Config) -> +%% Test count_children +count_children(Config) when is_list(Config) -> process_flag(trap_exit, true), Child = {child, {supervisor_1, start_child, []}, temporary, 1000, worker, []}, {ok, SupPid} = start_link({ok, {{simple_one_for_one, 2, 3600}, [Child]}}), [supervisor:start_child(sup_test, []) || _Ignore <- lists:seq(1,1000)], - garbage_collect(), - _Size1 = proc_memory(), Children = supervisor:which_children(sup_test), - _Size2 = proc_memory(), ChildCount = get_child_counts(sup_test), - _Size3 = proc_memory(), [supervisor:start_child(sup_test, []) || _Ignore2 <- lists:seq(1,1000)], - garbage_collect(), - Children2 = supervisor:which_children(sup_test), - Size4 = proc_memory(), ChildCount2 = get_child_counts(sup_test), - Size5 = proc_memory(), + Children2 = supervisor:which_children(sup_test), - garbage_collect(), - Children3 = supervisor:which_children(sup_test), - Size6 = proc_memory(), ChildCount3 = get_child_counts(sup_test), - Size7 = proc_memory(), + Children3 = supervisor:which_children(sup_test), 1000 = length(Children), [1,1000,0,1000] = ChildCount, @@ -1285,27 +1262,9 @@ count_children_memory(Config) when is_list(Config) -> Children3 = Children2, ChildCount3 = ChildCount2, - %% count_children consumes memory using an accumulator function, - %% but the space can be reclaimed incrementally, - %% which_children may generate garbage that will be reclaimed later. - case (Size5 =< Size4) of - true -> ok; - false -> - test_server:fail({count_children, used_more_memory,Size4,Size5}) - end, - case Size7 =< Size6 of - true -> ok; - false -> - test_server:fail({count_children, used_more_memory,Size6,Size7}) - end, - [terminate(SupPid, Pid, child, kill) || {undefined, Pid, worker, _Modules} <- Children3], [1,0,0,0] = get_child_counts(sup_test). -proc_memory() -> - erts_debug:set_internal_state(wait, deallocations), - erlang:memory(processes_used). - %%------------------------------------------------------------------------- %% Temporary children shall not be restarted so they should not save %% start parameters, as it potentially can take up a huge amount of @@ -1483,7 +1442,7 @@ simple_one_for_one_scale_many_temporary_children(_Config) -> if T1 > 0 -> Scaling = T2 div T1, - if Scaling > 20 -> + if Scaling > 50 -> %% The scaling shoul be linear (i.e.10, really), but we %% give some extra here to avoid failing the test %% unecessarily. diff --git a/lib/stdlib/test/tar_SUITE.erl b/lib/stdlib/test/tar_SUITE.erl index 5bc34e35af..6349139925 100644 --- a/lib/stdlib/test/tar_SUITE.erl +++ b/lib/stdlib/test/tar_SUITE.erl @@ -23,7 +23,7 @@ create_long_names/1, bad_tar/1, errors/1, extract_from_binary/1, extract_from_binary_compressed/1, extract_from_open_file/1, symlinks/1, open_add_close/1, cooked_compressed/1, - memory/1]). + memory/1,unicode/1]). -include_lib("test_server/include/test_server.hrl"). -include_lib("kernel/include/file.hrl"). @@ -34,7 +34,7 @@ all() -> [borderline, atomic, long_names, create_long_names, bad_tar, errors, extract_from_binary, extract_from_binary_compressed, extract_from_open_file, - symlinks, open_add_close, cooked_compressed, memory]. + symlinks, open_add_close, cooked_compressed, memory, unicode]. groups() -> []. @@ -73,6 +73,7 @@ borderline(Config) when is_list(Config) -> ?line lists:foreach(fun(Size) -> borderline_test(Size, TempDir) end, [0, 1, 10, 13, 127, 333, Record-1, Record, Record+1, + Block-2*Record-1, Block-2*Record, Block-2*Record+1, Block-Record-1, Block-Record, Block-Record+1, Block-1, Block, Block+1, Block+Record-1, Block+Record, Block+Record+1]), @@ -726,6 +727,56 @@ memory(Config) when is_list(Config) -> ?line ok = delete_files([Name1,Name2]), ok. +%% Test filenames with characters outside the US ASCII range. +unicode(Config) when is_list(Config) -> + PrivDir = ?config(priv_dir, Config), + do_unicode(PrivDir), + case has_transparent_naming() of + true -> + Pa = filename:dirname(code:which(?MODULE)), + Node = start_node(unicode, "+fnl -pa "++Pa), + ok = rpc:call(Node, erlang, apply, + [fun() -> do_unicode(PrivDir) end,[]]), + true = test_server:stop_node(Node), + ok; + false -> + ok + end. + +has_transparent_naming() -> + case os:type() of + {unix,darwin} -> false; + {unix,_} -> true; + _ -> false + end. + +do_unicode(PrivDir) -> + ok = file:set_cwd(PrivDir), + ok = file:make_dir("unicöde"), + + Names = unicode_create_files(), + Tar = "unicöde.tar", + ok = erl_tar:create(Tar, ["unicöde"], []), + {ok,Names} = erl_tar:table(Tar, []), + _ = [ok = file:delete(Name) || Name <- Names], + ok = erl_tar:extract(Tar), + _ = [{ok,_} = file:read_file(Name) || Name <- Names], + _ = [ok = file:delete(Name) || Name <- Names], + ok = file:del_dir("unicöde"), + ok. + +unicode_create_files() -> + FileA = "unicöde/smörgåsbord", + ok = file:write_file(FileA, "yum!\n"), + [FileA|case file:native_name_encoding() of + utf8 -> + FileB = "unicöde/Хороший файл!", + ok = file:write_file(FileB, "But almost empty.\n"), + [FileB]; + latin1 -> + [] + end]. + %% Delete the given list of files. delete_files([]) -> ok; delete_files([Item|Rest]) -> @@ -791,3 +842,14 @@ make_temp_dir(Base, I) -> ok -> Name; {error,eexist} -> make_temp_dir(Base, I+1) end. + +start_node(Name, Args) -> + [_,Host] = string:tokens(atom_to_list(node()), "@"), + ct:log("Trying to start ~w@~s~n", [Name,Host]), + case test_server:start_node(Name, peer, [{args,Args}]) of + {error,Reason} -> + test_server:fail(Reason); + {ok,Node} -> + ct:log("Node ~p started~n", [Node]), + Node + end. diff --git a/lib/stdlib/test/unicode_SUITE.erl b/lib/stdlib/test/unicode_SUITE.erl index e2d789bbe6..10b29d0d28 100644 --- a/lib/stdlib/test/unicode_SUITE.erl +++ b/lib/stdlib/test/unicode_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2013. All Rights Reserved. +%% Copyright Ericsson AB 2008-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -98,7 +98,7 @@ ex_binaries_errors_utf8(Config) when is_list(Config) -> unicode:characters_to_list(Chomped), BrokenPart = iolist_to_binary(DeepBrokenPart2) end || X <- lists:seq(1,OKLen) ] - end || N <- lists:seq(1,20) ], + end || N <- lists:seq(1,21,4) ], ok. ex_binaries_errors_utf16_little(Config) when is_list(Config) -> @@ -124,7 +124,7 @@ ex_binaries_errors_utf16_little(Config) when is_list(Config) -> unicode:characters_to_list(Chomped,{utf16,little}), BrokenPart = iolist_to_binary(DeepBrokenPart2) end || X <- lists:seq(1,OKLen) ] - end || N <- lists:seq(1,15) ], + end || N <- lists:seq(1,16,3) ], ok. ex_binaries_errors_utf16_big(Config) when is_list(Config) -> BrokenPart = << <<X:16/big>> || X <- lists:seq(16#DC00,16#DFFF) >>, @@ -149,7 +149,7 @@ ex_binaries_errors_utf16_big(Config) when is_list(Config) -> unicode:characters_to_list(Chomped,{utf16,big}), BrokenPart = iolist_to_binary(DeepBrokenPart2) end || X <- lists:seq(1,OKLen) ] - end || N <- lists:seq(1,15) ], + end || N <- lists:seq(1,16,3) ], ok. ex_binaries_errors_utf32_big(Config) when is_list(Config) -> @@ -175,7 +175,7 @@ ex_binaries_errors_utf32_big(Config) when is_list(Config) -> unicode:characters_to_list(Chomped,{utf32,big}), BrokenPart = iolist_to_binary(DeepBrokenPart2) end || X <- lists:seq(1,OKLen) ] - end || N <- lists:seq(1,15) ], + end || N <- lists:seq(1,16,3) ], ok. ex_binaries_errors_utf32_little(Config) when is_list(Config) -> @@ -201,7 +201,7 @@ ex_binaries_errors_utf32_little(Config) when is_list(Config) -> unicode:characters_to_list(Chomped,{utf32,little}), BrokenPart = iolist_to_binary(DeepBrokenPart2) end || X <- lists:seq(1,OKLen) ] - end || N <- lists:seq(1,15) ], + end || N <- lists:seq(1,16,3) ], ok. diff --git a/lib/stdlib/vsn.mk b/lib/stdlib/vsn.mk index 37a6590b06..b522c3ea3c 100644 --- a/lib/stdlib/vsn.mk +++ b/lib/stdlib/vsn.mk @@ -1 +1 @@ -STDLIB_VSN = 2.0 +STDLIB_VSN = 2.2 diff --git a/lib/syntax_tools/doc/src/notes.xml b/lib/syntax_tools/doc/src/notes.xml index a9d3f68d1d..8384af53b0 100644 --- a/lib/syntax_tools/doc/src/notes.xml +++ b/lib/syntax_tools/doc/src/notes.xml @@ -31,6 +31,95 @@ <p>This document describes the changes made to the Syntax_Tools application.</p> +<section><title>Syntax_Tools 1.6.16</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> The default encoding for Erlang source files is now + UTF-8. As a temporary measure to ease the transition from + the old default of Latin-1, if EDoc encounters byte + sequences that are not valid UTF-8 sequences, EDoc will + re-try in Latin-1 mode. This workaround will be removed + in a future release. </p> + <p> + Own Id: OTP-12008</p> + </item> + </list> + </section> + +</section> + +<section><title>Syntax_Tools 1.6.15</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fix reverting map in syntax_tools</p> + <p> + There was a bug in erl_syntax when running e.g. + erl_syntax:revert_forms, affecting maps. Instead of + getting Key/Value you got Key/Key in the resulting + abstract form.</p> + <p> + Own Id: OTP-11930</p> + </item> + </list> + </section> + +</section> + +<section><title>Syntax_Tools 1.6.14</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Application upgrade (appup) files are corrected for the + following applications: </p> + <p> + <c>asn1, common_test, compiler, crypto, debugger, + dialyzer, edoc, eldap, erl_docgen, et, eunit, gs, hipe, + inets, observer, odbc, os_mon, otp_mibs, parsetools, + percept, public_key, reltool, runtime_tools, ssh, + syntax_tools, test_server, tools, typer, webtool, wx, + xmerl</c></p> + <p> + A new test utility for testing appup files is added to + test_server. This is now used by most applications in + OTP.</p> + <p> + (Thanks to Tobias Schlager)</p> + <p> + Own Id: OTP-11744</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Add implementation of having erl_tidy print to screen + instead of writing to the file provided. (Thanks to Aaron + France)</p> + <p> + Own Id: OTP-11632</p> + </item> + <item> + <p> + Support Maps syntax in syntax_tools (Thanks to Anthony + Ramine).</p> + <p> + Own Id: OTP-11663</p> + </item> + </list> + </section> + +</section> + <section><title>Syntax_Tools 1.6.13</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/syntax_tools/src/epp_dodger.erl b/lib/syntax_tools/src/epp_dodger.erl index 131be4e8e4..7e12eab1b5 100644 --- a/lib/syntax_tools/src/epp_dodger.erl +++ b/lib/syntax_tools/src/epp_dodger.erl @@ -184,9 +184,27 @@ quick_parse_file(File, Options) -> parse_file(File, fun quick_parse/3, Options ++ [no_fail]). parse_file(File, Parser, Options) -> + case do_parse_file(utf8, File, Parser, Options) of + {ok, Forms}=Ret -> + case find_invalid_unicode(Forms) of + none -> + Ret; + invalid_unicode -> + case epp:read_encoding(File) of + utf8 -> + Ret; + _ -> + do_parse_file(latin1, File, Parser, Options) + end + end; + Else -> + Else + end. + +do_parse_file(DefEncoding, File, Parser, Options) -> case file:open(File, [read]) of {ok, Dev} -> - _ = epp:set_encoding(Dev), + _ = epp:set_encoding(Dev, DefEncoding), try Parser(Dev, 1, Options) after ok = file:close(Dev) end; @@ -194,6 +212,14 @@ parse_file(File, Parser, Options) -> Error end. +find_invalid_unicode([H|T]) -> + case H of + {error, {_Line, file_io_server, invalid_unicode}} -> + invalid_unicode; + _Other -> + find_invalid_unicode(T) + end; +find_invalid_unicode([]) -> none. %% ===================================================================== %% @spec parse(IODevice) -> {ok, Forms} | {error, errorinfo()} diff --git a/lib/syntax_tools/src/erl_comment_scan.erl b/lib/syntax_tools/src/erl_comment_scan.erl index dae7530ce7..03429d4d42 100644 --- a/lib/syntax_tools/src/erl_comment_scan.erl +++ b/lib/syntax_tools/src/erl_comment_scan.erl @@ -72,13 +72,24 @@ file(Name) -> {ok, V} -> case V of {ok, B} -> - Enc = case epp:read_encoding(Name) of + Encoding = epp:read_encoding_from_binary(B), + Enc = case Encoding of none -> epp:default_encoding(); Enc0 -> Enc0 end, case catch unicode:characters_to_list(B, Enc) of String when is_list(String) -> string(String); + R when Encoding =:= none -> + case + catch unicode:characters_to_list(B, latin1) + of + String when is_list(String) -> + string(String); + _ -> + error_read_file(Name1), + exit(R) + end; R -> error_read_file(Name1), exit(R) diff --git a/lib/syntax_tools/src/erl_syntax.erl b/lib/syntax_tools/src/erl_syntax.erl index c9996c954e..46a5ca48df 100644 --- a/lib/syntax_tools/src/erl_syntax.erl +++ b/lib/syntax_tools/src/erl_syntax.erl @@ -2071,7 +2071,7 @@ map_field_assoc_value(Node) -> {map_field_assoc, _, _, Value} -> Value; _ -> - (data(Node))#map_field_assoc.name + (data(Node))#map_field_assoc.value end. @@ -2129,7 +2129,7 @@ map_field_exact_value(Node) -> {map_field_exact, _, _, Value} -> Value; _ -> - (data(Node))#map_field_exact.name + (data(Node))#map_field_exact.value end. diff --git a/lib/syntax_tools/test/syntax_tools_SUITE.erl b/lib/syntax_tools/test/syntax_tools_SUITE.erl index d4c54a72aa..6fb3e5ccfb 100644 --- a/lib/syntax_tools/test/syntax_tools_SUITE.erl +++ b/lib/syntax_tools/test/syntax_tools_SUITE.erl @@ -24,12 +24,12 @@ init_per_group/2,end_per_group/2]). %% Test cases --export([app_test/1,appup_test/1,smoke_test/1,revert/1]). +-export([app_test/1,appup_test/1,smoke_test/1,revert/1,revert_map/1]). suite() -> [{ct_hooks,[ts_install_cth]}]. all() -> - [app_test,appup_test,smoke_test,revert]. + [app_test,appup_test,smoke_test,revert,revert_map]. groups() -> []. @@ -109,6 +109,16 @@ revert_file(File, Path) -> ok end. +%% Testing bug fix for reverting map_field_assoc +revert_map(Config) -> + Dog = ?t:timetrap(?t:minutes(1)), + ?line [{map_field_assoc,16,{atom,17,name},{var,18,'Value'}}] = + erl_syntax:revert_forms([{tree,map_field_assoc, + {attr,16,[],none}, + {map_field_assoc, + {atom,17,name},{var,18,'Value'}}}]), + ?line ?t:timetrap_cancel(Dog). + p_run(Test, List) -> N = erlang:system_info(schedulers), p_run_loop(Test, List, N, [], 0). diff --git a/lib/syntax_tools/vsn.mk b/lib/syntax_tools/vsn.mk index cf396ce636..6a80734f83 100644 --- a/lib/syntax_tools/vsn.mk +++ b/lib/syntax_tools/vsn.mk @@ -1 +1 @@ -SYNTAX_TOOLS_VSN = 1.6.14 +SYNTAX_TOOLS_VSN = 1.6.16 diff --git a/lib/test_server/doc/src/notes.xml b/lib/test_server/doc/src/notes.xml index d05626094a..a801a87725 100644 --- a/lib/test_server/doc/src/notes.xml +++ b/lib/test_server/doc/src/notes.xml @@ -32,6 +32,87 @@ <file>notes.xml</file> </header> +<section><title>Test_Server 3.7.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + The mechanism for running code cover analysis with + common_test has been improved. Earlier, if a test run + consisted of multiple tests, cover would be started and + stopped for each test. This would give "intermediate" + cover logs available from the "Coverage log" link on the + test suite result pages. To accumulate cover data over + all tests, the 'export' option had to be used in the + cover spec file. This was not well documented, and the + functionality was quite confusing.</p> + <p> + Using the 'nodes' option in the cover spec file would + fail when the test run consisted of multiple tests, since + the specified nodes would only be included in the cover + analysis of the first test.</p> + <p> + The repeated compilation and analysis of the same modules + was also very time consuming.</p> + <p> + To overcome these problems, ct will now only cover + compile and analyze modules once per test run, i.e. once + for each cover spec file. The log file is available via a + new button on the top level index page. The old "Coverage + log" links on the test suite result pages still exist, + but they all point to the same log containing the + accumulated result.</p> + <p> + Own Id: OTP-11971</p> + </item> + </list> + </section> + +</section> + +<section><title>Test_Server 3.7</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Application upgrade (appup) files are corrected for the + following applications: </p> + <p> + <c>asn1, common_test, compiler, crypto, debugger, + dialyzer, edoc, eldap, erl_docgen, et, eunit, gs, hipe, + inets, observer, odbc, os_mon, otp_mibs, parsetools, + percept, public_key, reltool, runtime_tools, ssh, + syntax_tools, test_server, tools, typer, webtool, wx, + xmerl</c></p> + <p> + A new test utility for testing appup files is added to + test_server. This is now used by most applications in + OTP.</p> + <p> + (Thanks to Tobias Schlager)</p> + <p> + Own Id: OTP-11744</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Calls to erlang:open_port/2 with 'spawn' are updated to + handle space in the command path.</p> + <p> + Own Id: OTP-10842</p> + </item> + </list> + </section> + +</section> + <section><title>Test_Server 3.6.4</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/test_server/doc/src/test_server_ctrl.xml b/lib/test_server/doc/src/test_server_ctrl.xml index f4aae724e0..0cda531716 100644 --- a/lib/test_server/doc/src/test_server_ctrl.xml +++ b/lib/test_server/doc/src/test_server_ctrl.xml @@ -773,11 +773,14 @@ test_server_ctrl:cross_cover_analyse(Level,[{s1,S1LogDir},{s2,S2LogDir}]) <p><c>What = tests_start, Data = {Name,NumCases}</c><br></br> <c>What = loginfo, Data = [{topdir,TestRootDir},{rundir,CurrLogDir}]</c><br></br> <c>What = tests_done, Data = {Ok,Failed,{UserSkipped,AutoSkipped}}</c><br></br> - <c>What = tc_start, Data = {{Mod,Func},TCLogFile}</c><br></br> - <c>What = tc_done, Data = {Mod,Func,Result}</c><br></br> - <c>What = tc_user_skip, Data = {Mod,Func,Comment}</c><br></br> - <c>What = tc_auto_skip, Data = {Mod,Func,Comment}</c><br></br> + <c>What = tc_start, Data = {{Mod,{Func,GroupName}},TCLogFile}</c><br></br> + <c>What = tc_done, Data = {Mod,{Func,GroupName},Result}</c><br></br> + <c>What = tc_user_skip, Data = {Mod,{Func,GroupName},Comment}</c><br></br> + <c>What = tc_auto_skip, Data = {Mod,{Func,GroupName},Comment}</c><br></br> <c>What = framework_error, Data = {{FWMod,FWFunc},Error}</c></p> + <p>Note that for a test case function that doesn't belong to a group, + <c>GroupName</c> has value <c>undefined</c>, otherwise the name of the test + case group.</p> </desc> </func> <func> diff --git a/lib/test_server/src/Makefile b/lib/test_server/src/Makefile index ab4dd4d95d..35bbad3c22 100644 --- a/lib/test_server/src/Makefile +++ b/lib/test_server/src/Makefile @@ -124,7 +124,7 @@ include $(ERL_TOP)/make/otp_release_targets.mk release_spec: opt $(INSTALL_DIR) "$(RELSYSDIR)/src" $(INSTALL_DATA) $(ERL_FILES) "$(RELSYSDIR)/src" - $(INSTALL_DATA) $(INTERNAL_HRL_FILES) "$(RELSYSDIR)/src" + $(INSTALL_DATA) $(INTERNAL_HRL_FILES) $(TS_HRL_FILES) "$(RELSYSDIR)/src" $(INSTALL_DIR) "$(RELSYSDIR)/include" $(INSTALL_DATA) $(HRL_FILES) "$(RELSYSDIR)/include" $(INSTALL_DIR) "$(RELSYSDIR)/ebin" diff --git a/lib/test_server/src/test_server.erl b/lib/test_server/src/test_server.erl index 82672521f7..9192a76a17 100644 --- a/lib/test_server/src/test_server.erl +++ b/lib/test_server/src/test_server.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2013. All Rights Reserved. +%% Copyright Ericsson AB 1996-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -22,7 +22,7 @@ %%% TEST_SERVER_CTRL INTERFACE %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -export([run_test_case_apply/1,init_target_info/0,init_purify/0]). --export([cover_compile/1,cover_analyse/3]). +-export([cover_compile/1,cover_analyse/2]). %%% TEST_SERVER_SUP INTERFACE %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -export([get_loc/1,set_tc_state/1]). @@ -80,8 +80,8 @@ init_purify() -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% cover_compile({App,Include,Exclude,Cross}) -> -%% {ok,AnalyseModules} | {error,Reason} +%% cover_compile(#cover{app=App,incl=Include,excl=Exclude,cross=Cross}) -> +%% {ok,#cover{mods=AnalyseModules}} | {error,Reason} %% %% App = atom() , name of application to be compiled %% Exclude = [atom()], list of modules to exclude @@ -90,33 +90,35 @@ init_purify() -> %% Cross = [atoms()], list of modules outside of App shat should be included %% in the cover compilation, but that shall not be part of %% the cover analysis for this application. +%% AnalyseModules = [atom()], list of successfully compiled modules %% -%% Cover compile the given application. Return {ok,AnalyseMods} if application -%% is found, else {error,application_not_found}. +%% Cover compile the given application. Return {ok,CoverInfo} if +%% compilation succeeds, else (if application is not found and there +%% are no modules to compile) {error,application_not_found}. -cover_compile({none,_Exclude,Include,Cross}) -> +cover_compile(CoverInfo=#cover{app=none,incl=Include,cross=Cross}) -> CrossMods = lists:flatmap(fun({_,M}) -> M end,Cross), CompileMods = Include++CrossMods, case length(CompileMods) of 0 -> io:fwrite("WARNING: No modules to cover compile!\n\n",[]), cover:start(), % start cover server anyway - {ok,[]}; + {ok,CoverInfo#cover{mods=[]}}; N -> io:fwrite("Cover compiling ~w modules - " "this may take some time... ",[N]), do_cover_compile(CompileMods), io:fwrite("done\n\n",[]), - {ok,Include} + {ok,CoverInfo#cover{mods=Include}} end; -cover_compile({App,all,Include,Cross}) -> +cover_compile(CoverInfo=#cover{app=App,excl=all,incl=Include,cross=Cross}) -> CrossMods = lists:flatmap(fun({_,M}) -> M end,Cross), CompileMods = Include++CrossMods, case length(CompileMods) of 0 -> io:fwrite("WARNING: No modules to cover compile!\n\n",[]), cover:start(), % start cover server anyway - {ok,[]}; + {ok,CoverInfo#cover{mods=[]}}; N -> io:fwrite("Cover compiling '~w' (~w files) - " "this may take some time... ",[App,N]), @@ -126,9 +128,9 @@ cover_compile({App,all,Include,Cross}) -> "~tp\n", [App,CompileMods]), do_cover_compile(CompileMods), io:fwrite("done\n\n",[]), - {ok,Include} + {ok,CoverInfo#cover{mods=Include}} end; -cover_compile({App,Exclude,Include,Cross}) -> +cover_compile(CoverInfo=#cover{app=App,excl=Exclude,incl=Include,cross=Cross}) -> CrossMods = lists:flatmap(fun({_,M}) -> M end,Cross), case code:lib_dir(App) of {error,bad_name} -> @@ -146,7 +148,7 @@ cover_compile({App,Exclude,Include,Cross}) -> "~tp\n", [App,Include]), do_cover_compile(CompileMods), io:fwrite("done\n\n",[]), - {ok,Include} + {ok,CoverInfo#cover{mods=Include}} end; LibDir -> EbinDir = filename:join([LibDir,"ebin"]), @@ -158,13 +160,13 @@ cover_compile({App,Exclude,Include,Cross}) -> 0 -> io:fwrite("WARNING: No modules to cover compile!\n\n",[]), cover:start(), % start cover server anyway - {ok,[]}; + {ok,CoverInfo#cover{mods=[]}}; N -> io:fwrite("Cover compiling '~w' (~w files) - " "this may take some time... ",[App,N]), do_cover_compile(CompileMods), io:fwrite("done\n\n",[]), - {ok,AnalyseMods} + {ok,CoverInfo#cover{mods=AnalyseMods}} end end. @@ -174,9 +176,11 @@ module_names(Beams) -> do_cover_compile(Modules) -> - do_cover_compile1(lists:usort(Modules)). % remove duplicates + cover:start(), + pmap1(fun(M) -> do_cover_compile1(M) end,lists:usort(Modules)), + ok. -do_cover_compile1([M|Rest]) -> +do_cover_compile1(M) -> case {code:is_sticky(M),code:is_loaded(M)} of {true,_} -> code:unstick_mod(M), @@ -187,15 +191,13 @@ do_cover_compile1([M|Rest]) -> io:fwrite("\nWARNING: Could not cover compile ~w: ~p\n", [M,Error]) end, - code:stick_mod(M), - do_cover_compile1(Rest); + code:stick_mod(M); {false,false} -> case code:load_file(M) of {module,_} -> - do_cover_compile1([M|Rest]); + do_cover_compile1(M); Error -> - io:fwrite("\nWARNING: Could not load ~w: ~p\n",[M,Error]), - do_cover_compile1(Rest) + io:fwrite("\nWARNING: Could not load ~w: ~p\n",[M,Error]) end; {false,_} -> case cover:compile_beam(M) of @@ -204,23 +206,52 @@ do_cover_compile1([M|Rest]) -> Error -> io:fwrite("\nWARNING: Could not cover compile ~w: ~p\n", [M,Error]) - end, - do_cover_compile1(Rest) - end; -do_cover_compile1([]) -> - ok. + end + end. + +pmap1(Fun,List) -> + NTot = length(List), + NProcs = erlang:system_info(schedulers) * 2, + NPerProc = (NTot div NProcs) + 1, + + {[],Pids} = + lists:foldr( + fun(_,{L,Ps}) -> + {L1,L2} = if length(L)>=NPerProc -> lists:split(NPerProc,L); + true -> {L,[]} % last chunk + end, + {P,_Ref} = + spawn_monitor(fun() -> + exit(lists:map(Fun,L1)) + end), + {L2,[P|Ps]} + end, + {List,[]}, + lists:seq(1,NProcs)), + collect(Pids,[]). + +collect([],Acc) -> + lists:append(Acc); +collect([Pid|Pids],Acc) -> + receive + {'DOWN', _Ref, process, Pid, Result} -> + %% collect(lists:delete(Pid,Pids),[Result|Acc]) + collect(Pids,[Result|Acc]) + end. + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% cover_analyse(Analyse,Modules,Stop) -> [{M,{Cov,NotCov,Details}}] +%% cover_analyse(Dir,#cover{level=Analyse,mods=Modules,stop=Stop) -> +%% [{M,{Cov,NotCov,Details}}] %% -%% Analyse = {details,Dir} | details | {overview,void()} | overview +%% Dir = string() +%% Analyse = details | overview %% Modules = [atom()], the modules to analyse %% -%% Cover analysis. If Analyse=={details,Dir} analyse_to_file is used. +%% Cover analysis. If Analyse==details analyse_to_file is used. %% -%% If Analyse=={overview,Dir} analyse_to_file is not used, only an -%% overview containing the number of covered/not covered lines in each -%% module. +%% If Analyse==overview analyse_to_file is not used, only an overview +%% containing the number of covered/not covered lines in each module. %% %% Also, cover data will be exported to a file called all.coverdata in %% the given directory. @@ -235,11 +266,11 @@ do_cover_compile1([]) -> %% which means that the modules will stay cover compiled. Note that %% this is only recommended if the erlang node is being terminated %% after the test is completed. -cover_analyse(Analyse,Modules,Stop) -> - print(stdout, "Cover analysing...\n", []), +cover_analyse(Dir,#cover{level=Analyse,mods=Modules,stop=Stop}) -> + io:fwrite(user, "Cover analysing... ", []), DetailsFun = case Analyse of - {details,Dir} -> + details -> case cover:export(filename:join(Dir,"all.coverdata")) of ok -> fun(M) -> @@ -256,7 +287,7 @@ cover_analyse(Analyse,Modules,Stop) -> Error -> fun(_) -> Error end end; - {overview,Dir} -> + overview -> case cover:export(filename:join(Dir,"all.coverdata")) of ok -> fun(_) -> undefined end; @@ -264,17 +295,19 @@ cover_analyse(Analyse,Modules,Stop) -> fun(_) -> Error end end end, - R = pmap( + R = pmap2( fun(M) -> case cover:analyse(M,module) of {ok,{M,{Cov,NotCov}}} -> {M,{Cov,NotCov,DetailsFun(M)}}; Err -> - io:fwrite("WARNING: Analysis failed for ~w. Reason: ~p\n", + io:fwrite(user, + "\nWARNING: Analysis failed for ~w. Reason: ~p\n", [M,Err]), {M,Err} end end, Modules), + io:fwrite(user, "done\n\n", []), case Stop of true -> @@ -286,12 +319,12 @@ cover_analyse(Analyse,Modules,Stop) -> end, R. -pmap(Fun,List) -> +pmap2(Fun,List) -> Collector = self(), Pids = lists:map(fun(E) -> spawn(fun() -> - Collector ! {res,self(),Fun(E)} - end) + Collector ! {res,self(),Fun(E)} + end) end, List), lists:map(fun(Pid) -> receive @@ -300,7 +333,6 @@ pmap(Fun,List) -> end end, Pids). - do_cover_for_node(Node,CoverFunc) -> do_cover_for_node(Node,CoverFunc,true). do_cover_for_node(Node,CoverFunc,StickUnstick) -> @@ -444,7 +476,7 @@ run_test_case_apply(Mod, Func, Args, Name, RunInit, TimetrapData) -> %% If this process (group leader of the test case) terminates before %% all messages have been replied back to the io server, the io server %% hangs. Fixed by the 20 milli timeout check here, and by using monitor in -%% io.erl (livrem OCH hangslen mao :) +%% io.erl. %% %% A test case is known to have failed if it returns {'EXIT', _} tuple, %% or sends a message {failed, File, Line} to it's group_leader @@ -673,7 +705,7 @@ handle_tc_exit({testcase_aborted,{user_timetrap_error,_}=Msg,_}, St) -> spawn_fw_call(Mod, Func, Config, Pid, Msg, unknown, self()), St; handle_tc_exit(Reason, #st{status={framework,FwMod,FwFunc}, - config=Config,pid=Pid}=St) -> + config=Config,pid=Pid}=St) -> R = case Reason of {timetrap_timeout,TVal,_} -> {timetrap,TVal}; @@ -1208,6 +1240,10 @@ do_init_per_testcase(Mod, Args) -> "a Config list.\n",[]}, {skip,{failed,{Mod,init_per_testcase,bad_return}}} catch + throw:{Skip,Reason} when Skip =:= skip; Skip =:= skipped -> + {skip,Reason}; + exit:{Skip,Reason} when Skip =:= skip; Skip =:= skipped -> + {skip,Reason}; throw:Other -> set_loc(erlang:get_stacktrace()), Line = get_loc(), @@ -1378,6 +1414,10 @@ ts_tc(M, F, A) -> Result = try apply(M, F, A) catch + throw:{skip, Reason} -> {skip, Reason}; + throw:{skipped, Reason} -> {skip, Reason}; + exit:{skip, Reason} -> {skip, Reason}; + exit:{skipped, Reason} -> {skip, Reason}; Type:Reason -> Stk = erlang:get_stacktrace(), set_loc(Stk), diff --git a/lib/test_server/src/test_server_ctrl.erl b/lib/test_server/src/test_server_ctrl.erl index 5fbc47a813..af8921fe75 100644 --- a/lib/test_server/src/test_server_ctrl.erl +++ b/lib/test_server/src/test_server_ctrl.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2002-2013. All Rights Reserved. +%% Copyright Ericsson AB 2002-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -52,7 +52,9 @@ -export([reject_io_reqs/1, get_levels/0, set_levels/3]). -export([multiply_timetraps/1, scale_timetraps/1, get_timetrap_parameters/0]). -export([create_priv_dir/1]). --export([cover/2, cover/3, cover/8, cross_cover_analyse/2, trc/1, stop_trace/0]). +-export([cover/1, cover/2, cover/3, + cover_compile/7, cover_analyse/2, cross_cover_analyse/2, + trc/1, stop_trace/0]). -export([testcase_callback/1]). -export([set_random_seed/1]). -export([kill_slavenodes/0]). @@ -409,11 +411,26 @@ cover(App, Analyse) when is_atom(App) -> cover(CoverFile, Analyse) -> cover(none, CoverFile, Analyse). cover(App, CoverFile, Analyse) -> - controller_call({cover,{App,CoverFile},Analyse,true}). -cover(App, CoverFile, Exclude, Include, Cross, Export, Analyse, Stop) -> - controller_call({cover, - {App,{CoverFile,Exclude,Include,Cross,Export}}, - Analyse,Stop}). + {Excl,Incl,Cross} = read_cover_file(CoverFile), + CoverInfo = #cover{app=App, + file=CoverFile, + excl=Excl, + incl=Incl, + cross=Cross, + level=Analyse}, + controller_call({cover,CoverInfo}). + +cover(CoverInfo) -> + controller_call({cover,CoverInfo}). + +cover_compile(App,File,Excl,Incl,Cross,Analyse,Stop) -> + cover_compile(#cover{app=App, + file=File, + excl=Excl, + incl=Incl, + cross=Cross, + level=Analyse, + stop=Stop}). testcase_callback(ModFunc) -> controller_call({testcase_callback,ModFunc}). @@ -563,7 +580,7 @@ handle_call({add_job,Dir,Name,TopCase,Skip}, _From, State) -> ExtraTools = case State#state.cover of false -> []; - {App,Analyse,Stop} -> [{cover,App,Analyse,Stop}] + CoverInfo -> [{cover,CoverInfo}] end, ExtraTools1 = case State#state.random_seed of @@ -816,13 +833,13 @@ handle_call(stop_trace, _From, State) -> {reply,R,State#state{trc=false}}; %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% handle_call({cover,App,Analyse,Stop}, _, State) -> ok | {error,Reason} +%% handle_call({cover,CoverInfo}, _, State) -> ok | {error,Reason} %% -%% All modules inn application App are cover compiled -%% Analyse indicates on which level the coverage should be analysed +%% Set specification of cover analysis to be used when running tests +%% (see start_extra_tools/1 and stop_extra_tools/1) -handle_call({cover,App,Analyse,Stop}, _From, State) -> - {reply,ok,State#state{cover={App,Analyse,Stop}}}; +handle_call({cover,CoverInfo}, _From, State) -> + {reply,ok,State#state{cover=CoverInfo}}; %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% handle_call({create_priv_dir,Value}, _, State) -> ok | {error,Reason} @@ -1203,11 +1220,10 @@ elapsed_time(Before, After) -> start_extra_tools(ExtraTools) -> start_extra_tools(ExtraTools, []). -start_extra_tools([{cover,App,Analyse,Stop} | ExtraTools], Started) -> - case cover_compile(App) of - {ok,AnalyseMods} -> - start_extra_tools(ExtraTools, - [{cover,App,Analyse,AnalyseMods,Stop}|Started]); +start_extra_tools([{cover,CoverInfo} | ExtraTools], Started) -> + case start_cover(CoverInfo) of + {ok,NewCoverInfo} -> + start_extra_tools(ExtraTools,[{cover,NewCoverInfo}|Started]); {error,_} -> start_extra_tools(ExtraTools, Started) end; @@ -1226,8 +1242,8 @@ stop_extra_tools(ExtraTools) -> end, stop_extra_tools(ExtraTools, TestDir). -stop_extra_tools([{cover,App,Analyse,AnalyseMods,Stop}|ExtraTools], TestDir) -> - cover_analyse(App, Analyse, AnalyseMods, Stop, TestDir), +stop_extra_tools([{cover,CoverInfo}|ExtraTools], TestDir) -> + stop_cover(CoverInfo,TestDir), stop_extra_tools(ExtraTools, TestDir); %%stop_extra_tools([_ | ExtraTools], TestDir) -> %% stop_extra_tools(ExtraTools, TestDir); @@ -1569,16 +1585,24 @@ do_test_cases(TopCases, SkipCases, ok end end, - + CoverLog = + case get(test_server_cover_log_dir) of + undefined -> + ?coverlog_name; + AbsLogDir -> + AbsLog = filename:join(AbsLogDir,?coverlog_name), + make_relative(AbsLog, TestDir) + end, print(html, "<p><ul>\n" "<li><a href=\"~ts\">Full textual log</a></li>\n" "<li><a href=\"~ts\">Coverage log</a></li>\n" "<li><a href=\"~ts\">Unexpected I/O log</a></li>\n</ul></p>\n", - [?suitelog_name,?coverlog_name,?unexpected_io_log]), + [?suitelog_name,CoverLog,?unexpected_io_log]), print(html, "<p>~ts</p>\n" ++ - xhtml("<table bgcolor=\"white\" border=\"3\" cellpadding=\"5\">", + xhtml(["<table bgcolor=\"white\" border=\"3\" cellpadding=\"5\">\n", + "<thead>\n"], ["<table id=\"",?sortable_table_name,"\">\n", "<thead>\n"]) ++ "<tr><th>Num</th><th>Module</th><th>Group</th>" ++ @@ -5087,15 +5111,15 @@ pinfo(P) -> %% Cover compilation %% The compilation is executed on the target node -cover_compile({App,{_File,Exclude,Include,Cross,_Export}}) -> - cover_compile1({App,Exclude,Include,Cross}); +start_cover(#cover{}=CoverInfo) -> + cover_compile(CoverInfo); +start_cover({log,CoverLogDir}=CoverInfo) -> + %% Cover is controlled by the framework - here's the log + put(test_server_cover_log_dir,CoverLogDir), + {ok,CoverInfo}. -cover_compile({App,CoverFile}) -> - {Exclude,Include,Cross} = read_cover_file(CoverFile), - cover_compile1({App,Exclude,Include,Cross}). - -cover_compile1(What) -> - test_server:cover_compile(What). +cover_compile(CoverInfo) -> + test_server:cover_compile(CoverInfo). %% Read the coverfile for an application and return a list of modules %% that are members of the application but shall not be compiled @@ -5163,25 +5187,45 @@ check_cross([]) -> %% %% This per application analysis writes the file cover.html in the %% application's run.<timestamp> directory. -cover_analyse({App,CoverInfo}, Analyse, AnalyseMods, Stop, TestDir) -> +stop_cover(#cover{}=CoverInfo, TestDir) -> + cover_analyse(CoverInfo, TestDir); +stop_cover(_CoverInfo, _TestDir) -> + %% Cover is probably controlled by the framework + ok. + +make_relative(AbsDir, VsDir) -> + DirTokens = filename:split(AbsDir), + VsTokens = filename:split(VsDir), + filename:join(make_relative1(DirTokens, VsTokens)). + +make_relative1([T | DirTs], [T | VsTs]) -> + make_relative1(DirTs, VsTs); +make_relative1(Last = [_File], []) -> + Last; +make_relative1(Last = [_File], VsTs) -> + Ups = ["../" || _ <- VsTs], + Ups ++ Last; +make_relative1(DirTs, []) -> + DirTs; +make_relative1(DirTs, VsTs) -> + Ups = ["../" || _ <- VsTs], + Ups ++ DirTs. + + +cover_analyse(CoverInfo, TestDir) -> write_default_cross_coverlog(TestDir), {ok,CoverLog} = open_html_file(filename:join(TestDir, ?coverlog_name)), write_coverlog_header(CoverLog), + #cover{app=App, + file=CoverFile, + excl=Excluded, + cross=Cross} = CoverInfo, io:fwrite(CoverLog, "<h1>Coverage for application '~w'</h1>\n", [App]), io:fwrite(CoverLog, "<p><a href=\"~ts\">Coverdata collected over all tests</a></p>", [?cross_coverlog_name]), - {CoverFile,_Included,Excluded,Cross} = - case CoverInfo of - {File,Excl,Incl,Cr,Export} -> - cover:export(Export), - {File,Incl,Excl,Cr}; - File -> - {Excl,Incl,Cr} = read_cover_file(File), - {File,Incl,Excl,Cr} - end, io:fwrite(CoverLog, "<p>CoverFile: <code>~tp</code>\n", [CoverFile]), write_cross_cover_info(TestDir,Cross), @@ -5196,7 +5240,7 @@ cover_analyse({App,CoverInfo}, Analyse, AnalyseMods, Stop, TestDir) -> io:fwrite(CoverLog, "<p>Excluded module(s): <code>~tp</code>\n", [Excluded]), - Coverage = cover_analyse(Analyse, AnalyseMods, Stop), + Coverage = test_server:cover_analyse(TestDir, CoverInfo), write_binary_file(filename:join(TestDir,?raw_coverlog_name), term_to_binary(Coverage)), @@ -5215,11 +5259,6 @@ cover_analyse({App,CoverInfo}, Analyse, AnalyseMods, Stop, TestDir) -> write_binary_file(filename:join(TestDir, ?cover_total), term_to_binary(TotPercent)). -cover_analyse(Analyse, AnalyseMods, Stop) -> - TestDir = get(test_server_log_dir_base), - test_server:cover_analyse({Analyse,TestDir}, AnalyseMods, Stop). - - %% Cover analysis - accumulated over multiple tests %% This can be executed on any node after all tests are finished. %% Analyse = overview | details diff --git a/lib/test_server/src/test_server_internal.hrl b/lib/test_server/src/test_server_internal.hrl index 4e734a330b..bafeeaabfe 100644 --- a/lib/test_server/src/test_server_internal.hrl +++ b/lib/test_server/src/test_server_internal.hrl @@ -49,3 +49,12 @@ master, cookie}). + +-record(cover, {app, % application; Name | none + file, % cover spec file + incl, % explicitly include modules + excl, % explicitly exclude modules + level, % analyse level; details | overview + mods, % actually cover compiled modules + stop=true, % stop cover after analyse; boolean() + cross}).% cross cover analyse info diff --git a/lib/test_server/src/test_server_node.erl b/lib/test_server/src/test_server_node.erl index 582abb2153..acd47788db 100644 --- a/lib/test_server/src/test_server_node.erl +++ b/lib/test_server/src/test_server_node.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2002-2013. All Rights Reserved. +%% Copyright Ericsson AB 2002-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -653,7 +653,7 @@ find_rel_linux(Rel) -> end. find_rel_suse(Rel, SuseRel) -> - Root = "/usr/local/otp/releases/otp_beam_linux_sles", + Root = "/usr/local/otp/releases/sles", case SuseRel of "11" -> %% Try both SuSE 11, SuSE 10 and SuSe 9 in that order. @@ -673,10 +673,10 @@ find_rel_suse(Rel, SuseRel) -> find_rel_suse_1(Rel, RootWc) -> case erlang:system_info(wordsize) of 4 -> - find_rel_suse_2(Rel, RootWc++"_i386"); + find_rel_suse_2(Rel, RootWc++"_32"); 8 -> - find_rel_suse_2(Rel, RootWc++"_x64") ++ - find_rel_suse_2(Rel, RootWc++"_i386") + find_rel_suse_2(Rel, RootWc++"_64") ++ + find_rel_suse_2(Rel, RootWc++"_32") end. find_rel_suse_2(Rel, RootWc) -> diff --git a/lib/test_server/src/ts.erl b/lib/test_server/src/ts.erl index bc7d244c7c..d6d2e865e2 100644 --- a/lib/test_server/src/ts.erl +++ b/lib/test_server/src/ts.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2013. All Rights Reserved. +%% Copyright Ericsson AB 1997-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -641,16 +641,17 @@ get_last_app_tests([Dir|Dirs],RE,Acc) -> NewAcc = case re:run(Dir,RE,[{capture,all,list}]) of {match,[Dir,AppStr]} -> + Dir1 = filename:dirname(Dir), % cover logs in ct_run.<t> dir App = list_to_atom(AppStr), case lists:keytake(App,1,Acc) of {value,{App,LastDir},Rest} -> - if Dir > LastDir -> - [{App,Dir}|Rest]; + if Dir1 > LastDir -> + [{App,Dir1}|Rest]; true -> Acc end; false -> - [{App,Dir} | Acc] + [{App,Dir1} | Acc] end; _ -> Acc diff --git a/lib/test_server/test/test_server_SUITE_data/test_server_parallel01_SUITE.erl b/lib/test_server/test/test_server_SUITE_data/test_server_parallel01_SUITE.erl index f38f768f3b..9658191289 100644 --- a/lib/test_server/test/test_server_SUITE_data/test_server_parallel01_SUITE.erl +++ b/lib/test_server/test/test_server_SUITE_data/test_server_parallel01_SUITE.erl @@ -191,7 +191,7 @@ conf1_end(Config) -> %% check 2s & 3s < 4s Ms = timer:now_diff(now(),?config(t0,Config)), test_server:comment(io_lib:format("~p",[now()])), - if Ms > 3500000 -> exit({bad_parallel_exec,Ms}); + if Ms > 4000000 -> exit({bad_parallel_exec,Ms}); Ms < 3000000 -> exit({bad_parallel_exec,Ms}); true -> ok end. @@ -204,7 +204,7 @@ conf2_end(Config) -> %% check 3s & 2s < 4s Ms = timer:now_diff(now(),?config(t0,Config)), test_server:comment(io_lib:format("~p",[now()])), - if Ms > 3500000 -> exit({bad_parallel_exec,Ms}); + if Ms > 4000000 -> exit({bad_parallel_exec,Ms}); Ms < 3000000 -> exit({bad_parallel_exec,Ms}); true -> ok end. @@ -217,7 +217,7 @@ conf3_end(Config) -> %% check 6s & 6s & (2s & 3s) & 1s = ~6s Ms = timer:now_diff(now(),?config(t0,Config)), test_server:comment(io_lib:format("~p",[now()])), - if Ms > 6500000 -> exit({bad_parallel_exec,Ms}); + if Ms > 7000000 -> exit({bad_parallel_exec,Ms}); Ms < 6000000 -> exit({bad_parallel_exec,Ms}); true -> ok end. @@ -230,7 +230,7 @@ conf4_end(Config) -> %% check 2s & 3s >= 5s Ms = timer:now_diff(now(),?config(t0,Config)), test_server:comment(io_lib:format("~p",[now()])), - if Ms > 5500000 -> exit({bad_parallel_exec,Ms}); + if Ms > 6000000 -> exit({bad_parallel_exec,Ms}); Ms < 5000000 -> exit({bad_parallel_exec,Ms}); true -> ok end. @@ -243,7 +243,7 @@ conf5_end(Config) -> %% check 1s & 1s & (3s & 2s) & 1s = ~6s Ms = timer:now_diff(now(),?config(t0,Config)), test_server:comment(io_lib:format("~p",[now()])), - if Ms > 7000000 -> exit({bad_parallel_exec,Ms}); + if Ms > 7500000 -> exit({bad_parallel_exec,Ms}); Ms < 6000000 -> exit({bad_parallel_exec,Ms}); true -> ok end. @@ -257,7 +257,7 @@ conf6_end(Config) -> %% check 3s & 2s < 5s Ms = timer:now_diff(now(),?config(t0,Config)), test_server:comment(io_lib:format("~p",[now()])), - if Ms > 3500000 -> exit({bad_parallel_exec,Ms}); + if Ms > 4500000 -> exit({bad_parallel_exec,Ms}); Ms < 3000000 -> exit({bad_parallel_exec,Ms}); true -> ok end. @@ -277,7 +277,7 @@ conf7_end(Config) -> %% check 1s & 1s & (2s & 2s) & 1s = ~3s Ms = timer:now_diff(now(),?config(t0,Config)), test_server:comment(io_lib:format("~p",[now()])), - if Ms > 3500000 -> exit({bad_parallel_exec,Ms}); + if Ms > 4000000 -> exit({bad_parallel_exec,Ms}); Ms < 3000000 -> exit({bad_parallel_exec,Ms}); true -> ok end. @@ -291,7 +291,7 @@ conf8_end(Config) -> %% check 2s & 2s < 4s Ms = timer:now_diff(now(),?config(t0,Config)), test_server:comment(io_lib:format("~p",[now()])), - if Ms > 2500000 -> exit({bad_parallel_exec,Ms}); + if Ms > 3000000 -> exit({bad_parallel_exec,Ms}); Ms < 2000000 -> exit({bad_parallel_exec,Ms}); true -> ok end. diff --git a/lib/test_server/vsn.mk b/lib/test_server/vsn.mk index 4eb70aa2cd..9e1ac8fd12 100644 --- a/lib/test_server/vsn.mk +++ b/lib/test_server/vsn.mk @@ -1 +1 @@ -TEST_SERVER_VSN = 3.7 +TEST_SERVER_VSN = 3.7.1 diff --git a/lib/tools/doc/src/notes.xml b/lib/tools/doc/src/notes.xml index 2e4c354fbd..faee5efd43 100644 --- a/lib/tools/doc/src/notes.xml +++ b/lib/tools/doc/src/notes.xml @@ -30,6 +30,115 @@ </header> <p>This document describes the changes made to the Tools application.</p> +<section><title>Tools 2.7</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Add log2 histogram to lcnt for lock wait time</p> + <p> + Own Id: OTP-12059</p> + </item> + </list> + </section> + +</section> + +<section><title>Tools 2.6.15</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Removed <c>erlang:bitstr_to_list/1</c> and + <c>erlang:list_to_bitstr/1</c>. They were added by + mistake, and have always raised an <c>undefined</c> + exception when called.</p> + <p> + Own Id: OTP-11942</p> + </item> + </list> + </section> + +</section> + +<section><title>Tools 2.6.14</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Removed the support for the query keyword from emacs mode + (Thanks to Paul Oliver)</p> + <p> + Own Id: OTP-11568</p> + </item> + <item> + <p> + Emacs mode improvements (Thanks to Steve Vinoski)</p> + <p> + Own Id: OTP-11601</p> + </item> + <item> + <p> + Application upgrade (appup) files are corrected for the + following applications: </p> + <p> + <c>asn1, common_test, compiler, crypto, debugger, + dialyzer, edoc, eldap, erl_docgen, et, eunit, gs, hipe, + inets, observer, odbc, os_mon, otp_mibs, parsetools, + percept, public_key, reltool, runtime_tools, ssh, + syntax_tools, test_server, tools, typer, webtool, wx, + xmerl</c></p> + <p> + A new test utility for testing appup files is added to + test_server. This is now used by most applications in + OTP.</p> + <p> + (Thanks to Tobias Schlager)</p> + <p> + Own Id: OTP-11744</p> + </item> + <item> + <p> + The emacs erlang mode now match erlang keywords more + carefully (Thanks to Steve Vinoski)</p> + <p> + Own Id: OTP-11786</p> + </item> + <item> + <p> + The emacs erlang-mode now auto loads for more file types + (Thanks to Phil Hagelberg)</p> + <p> + Own Id: OTP-11788</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + <c>cover</c> can run on itself. Also, support for reading + BEAM files produced by ancient OTP versions before R9C + has been removed.</p> + <p> + Own Id: OTP-11692</p> + </item> + <item> + <p> + Support maps in cover</p> + <p> + Own Id: OTP-11764</p> + </item> + </list> + </section> + +</section> + <section><title>Tools 2.6.13</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/tools/emacs/erlang.el b/lib/tools/emacs/erlang.el index ec5a1f4bc5..c56759ebb9 100644 --- a/lib/tools/emacs/erlang.el +++ b/lib/tools/emacs/erlang.el @@ -7,7 +7,7 @@ ;; %CopyrightBegin% ;; -;; Copyright Ericsson AB 1996-2013. All Rights Reserved. +;; Copyright Ericsson AB 1996-2014. All Rights Reserved. ;; ;; The contents of this file are subject to the Erlang Public License, ;; Version 1.1, (the "License"); you may not use this file except in @@ -853,7 +853,6 @@ resulting regexp is surrounded by \\_< and \\_>." "append_element" "await_proc_exit" "await_sched_wall_time_modifications" - "bitstr_to_list" "bump_reductions" "call_on_load_function" "cancel_timer" @@ -887,6 +886,7 @@ resulting regexp is surrounded by \\_< and \\_>." "flush_monitor_message" "format_cpu_topology" "fun_info" + "fun_info_mfa" "fun_to_list" "function_exported" "garbage_collect_message_area" @@ -899,7 +899,6 @@ resulting regexp is surrounded by \\_< and \\_>." "hibernate" "insert_element" "is_builtin" - "list_to_bitstr" "load_nif" "loaded" "localtime" diff --git a/lib/tools/src/lcnt.erl b/lib/tools/src/lcnt.erl index 20ee32c861..f1251fddab 100644 --- a/lib/tools/src/lcnt.erl +++ b/lib/tools/src/lcnt.erl @@ -61,6 +61,8 @@ locations/1, inspect/1, inspect/2, + histogram/1, + histogram/2, information/0, swap_pid_keys/0, % set options @@ -89,14 +91,14 @@ duration = 0 }). - -record(stats, { - file, - line, - tries, - colls, - time, % us - nt % #timings collected + file :: atom(), + line :: non_neg_integer(), + tries :: non_neg_integer(), + colls :: non_neg_integer(), + time :: non_neg_integer(), % us + nt :: non_neg_integer(), % #timings collected + hist :: tuple() % histogram }). -record(lock, { @@ -115,7 +117,9 @@ colls, cr, % collision ratio time, - dtr % time duration ratio + dtr, % time duration ratio + %% new + hist % log2 histogram of lock wait_time }). @@ -127,7 +131,7 @@ %% -------------------------------------------------------------------- %% start() -> gen_server:start({local, ?MODULE}, ?MODULE, [], []). -stop() -> gen_server:cast(?MODULE, stop). +stop() -> gen_server:call(?MODULE, stop, infinity). init([]) -> {ok, #state{ locks = [], duration = 0 } }. %% -------------------------------------------------------------------- %% @@ -171,6 +175,8 @@ conflicts() -> call({conflicts, []}). conflicts(Opts) -> call({conflicts, Opts}). inspect(Lock) -> call({inspect, Lock, []}). inspect(Lock, Opts) -> call({inspect, Lock, Opts}). +histogram(Lock) -> call({histogram, Lock, []}). +histogram(Lock, Opts)-> call({histogram, Lock, Opts}). information() -> call(information). swap_pid_keys() -> call(swap_pid_keys). raw() -> call(raw). @@ -283,14 +289,14 @@ handle_call({locations, InOpts}, _From, #state{ locks = Locks } = State) when is {reply, ok, State}; -handle_call({inspect, Lockname, InOpts}, _From, #state{ duration = Duration, locks = Locks } = State) when is_list(InOpts) -> +handle_call({inspect, Lockname, InOpts}, _From, #state{ duration=Duration, locks=Locks } = State) when is_list(InOpts) -> Default = [ {sort, time}, {reverse, false}, - {print, [name,id,tries,colls,ratio,time,duration]}, + {print, [name,id,tries,colls,ratio,time,duration,histogram]}, {max_locks, 20}, {combine, false}, - {thresholds, [] }, + {thresholds, []}, {locations, false}], Opts = options(InOpts, Default), @@ -299,7 +305,7 @@ handle_call({inspect, Lockname, InOpts}, _From, #state{ duration = Duration, loc {true, true} -> locks_ids(Filtered); _ -> [] end, - Combos = combine_classes(Filtered, proplists:get_value(combine, Opts)), + Combos = combine_classes(Filtered, proplists:get_value(combine, Opts)), case proplists:get_value(locations, Opts) of true -> lists:foreach(fun @@ -313,17 +319,14 @@ handle_call({inspect, Lockname, InOpts}, _From, #state{ duration = Duration, loc [] -> ok; _ -> - %io:format("Combined ~p~n", [Combined]), print("lock: " ++ term2string(Name)), print("id: " ++ IdString), print("type: " ++ term2string(Type)), Ps = stats2print(Combined, Duration), - Opts1 = options([{print, [entry, tries,colls,ratio,time,duration]}, + Opts1 = options([{print, [entry, tries,colls,ratio,time,duration,histogram]}, {thresholds, [{tries, -1}, {colls, -1}, {time, -1}]}], Opts), print_lock_information(filter_print(Ps, Opts1), proplists:get_value(print, Opts1)) end - % (#lock{ name = Name, id = Id}) -> - % io:format("Empty lock ~p ~p~n", [Name, Id]) end, Combos); _ -> Print1 = locks2print(Combos, Duration), @@ -332,6 +335,34 @@ handle_call({inspect, Lockname, InOpts}, _From, #state{ duration = Duration, loc end, {reply, ok, State}; +%% histogram + +handle_call({histogram, Lockname, InOpts}, _From, #state{ duration=Duration, locks=Locks} = State)-> + Default = [ + {sort, time}, + {reverse, false}, + {print, [name,id,tries,colls,ratio,time,duration,histogram]}, + {max_locks, 20}, + {combine, true}, + {thresholds, []}, + {locations, false}], + + Opts = options(InOpts, Default), + Filtered = filter_locks(Locks, Lockname), + Combos = combine_classes(Filtered, proplists:get_value(combine, Opts)), + lists:foreach(fun + (#lock{ stats = Stats }=L) -> + SumStats = summate_stats(Stats), + Opts1 = options([{print, [name,id,tries,colls,ratio,time,duration]}, + {thresholds, [{tries, -1}, {colls, -1}, {time, -1}]}], Opts), + Prints = locks2print([L], Duration), + print_lock_information(Prints, proplists:get_value(print, Opts1)), + print_full_histogram(SumStats#stats.hist), + io:format("~n") + end, Combos), + + {reply, ok, State}; + handle_call(raw, _From, #state{ locks = Locks} = State)-> {reply, Locks, State}; @@ -347,7 +378,6 @@ handle_call(swap_pid_keys, _From, #state{ locks = Locks } = State)-> (L) -> L end, Locks), - {reply, ok, State#state{ locks = SwappedLocks}}; % settings @@ -380,6 +410,8 @@ handle_call({save, Filename}, _From, State) -> {reply, {error, Error}, State} end; +handle_call(stop, _From, State) -> + {stop, normal, ok, State}; handle_call(Command, _From, State) -> {reply, {error, {undefined, Command}}, State}. @@ -390,8 +422,6 @@ handle_call(Command, _From, State) -> %% %% -------------------------------------------------------------------- %% -handle_cast(stop, State) -> - {stop, normal, State}; handle_cast(_, State) -> {noreply, State}. @@ -432,15 +462,32 @@ code_change(_OldVsn, State, _Extra) -> summate_locks(Locks) -> summate_locks(Locks, #stats{ tries = 0, colls = 0, time = 0, nt = 0}). summate_locks([], Stats) -> Stats; -summate_locks([L|Ls], #stats{ tries = Tries, colls = Colls, time = Time, nt = Nt}) -> +summate_locks([L|Ls], #stats{ tries = Tries, colls = Colls, time = Time, nt = Nt, hist = Hist}) -> S = summate_stats(L#lock.stats), - summate_locks(Ls, #stats{ tries = Tries + S#stats.tries, colls = Colls + S#stats.colls, time = Time + S#stats.time, nt = Nt + S#stats.nt}). + summate_locks(Ls, #stats{ + tries = Tries + S#stats.tries, + colls = Colls + S#stats.colls, + time = Time + S#stats.time, + nt = Nt + S#stats.nt, + hist = summate_histogram(Hist, S#stats.hist) + }). summate_stats(Stats) -> summate_stats(Stats, #stats{ tries = 0, colls = 0, time = 0, nt = 0}). summate_stats([], Stats) -> Stats; -summate_stats([S|Ss], #stats{ tries = Tries, colls = Colls, time = Time, nt = Nt}) -> - summate_stats(Ss, #stats{ tries = Tries + S#stats.tries, colls = Colls + S#stats.colls, time = Time + S#stats.time, nt = Nt + S#stats.nt}). - +summate_stats([S|Ss], #stats{ tries = Tries, colls = Colls, time = Time, nt = Nt, hist = Hist}) -> + summate_stats(Ss, #stats{ + tries = Tries + S#stats.tries, + colls = Colls + S#stats.colls, + time = Time + S#stats.time, + nt = Nt + S#stats.nt, + hist = summate_histogram(Hist, S#stats.hist) + }). + +%% first call is undefined +summate_histogram(Tup,undefined) when is_tuple(Tup) -> Tup; +summate_histogram(undefined,Tup) when is_tuple(Tup) -> Tup; +summate_histogram(Hs1,Hs2) -> + list_to_tuple([ A + B || {A,B} <- lists:zip(tuple_to_list(Hs1),tuple_to_list(Hs2))]). %% manipulators filter_locks_type(Locks, undefined) -> Locks; @@ -465,17 +512,16 @@ filter_print(PLs, Opts) -> TLs = threshold_locks(PLs, proplists:get_value(thresholds, Opts, [])), SLs = sort_locks(TLs, proplists:get_value(sort, Opts, time)), CLs = cut_locks(SLs, proplists:get_value(max_locks, Opts, none)), - reverse_locks(CLs, proplists:get_value(reverse, Opts, false)). - -sort_locks(Locks, Type) -> lists:reverse(sort_locks0(Locks, Type)). -sort_locks0(Locks, name) -> lists:keysort(#print.name, Locks); -sort_locks0(Locks, id) -> lists:keysort(#print.id, Locks); -sort_locks0(Locks, type) -> lists:keysort(#print.type, Locks); -sort_locks0(Locks, tries) -> lists:keysort(#print.tries, Locks); -sort_locks0(Locks, colls) -> lists:keysort(#print.colls, Locks); -sort_locks0(Locks, ratio) -> lists:keysort(#print.cr, Locks); -sort_locks0(Locks, time) -> lists:keysort(#print.time, Locks); -sort_locks0(Locks, _) -> sort_locks0(Locks, time). + reverse_locks(CLs, not proplists:get_value(reverse,Opts, false)). + +sort_locks(Locks, name) -> lists:keysort(#print.name, Locks); +sort_locks(Locks, id) -> lists:keysort(#print.id, Locks); +sort_locks(Locks, type) -> lists:keysort(#print.type, Locks); +sort_locks(Locks, tries) -> lists:keysort(#print.tries, Locks); +sort_locks(Locks, colls) -> lists:keysort(#print.colls, Locks); +sort_locks(Locks, ratio) -> lists:keysort(#print.cr, Locks); +sort_locks(Locks, time) -> lists:keysort(#print.time, Locks); +sort_locks(Locks, _) -> sort_locks(Locks, time). % cut locks not above certain thresholds threshold_locks(Locks, Thresholds) -> @@ -556,45 +602,61 @@ locks_ids(Locks) -> locks_ids(Locks, []). locks_ids([], Out) -> Out; locks_ids([#lock{ name = Key } = L|Ls], Out) -> case proplists:get_value(Key, Out) of - undefined -> - locks_ids(Ls, [{Key, [L#lock.id] } | Out]); - Ids -> - locks_ids(Ls, [{Key, [L#lock.id | Ids] } | proplists:delete(Key,Out)]) + undefined -> locks_ids(Ls, [{Key, [L#lock.id]}|Out]); + Ids -> locks_ids(Ls, [{Key, [L#lock.id|Ids]}|proplists:delete(Key,Out)]) end. stats2print(Stats, Duration) -> lists:map(fun (S) -> - #print{ - entry = term2string("~tp:~p", [S#stats.file, S#stats.line]), - colls = S#stats.colls, - tries = S#stats.tries, - cr = percent(S#stats.colls, S#stats.tries), - time = S#stats.time, - dtr = percent(S#stats.time, Duration) - } + #print{entry = term2string("~tp:~p", [S#stats.file, S#stats.line]), + colls = S#stats.colls, + tries = S#stats.tries, + cr = percent(S#stats.colls, S#stats.tries), + time = S#stats.time, + dtr = percent(S#stats.time, Duration), + hist = format_histogram(S#stats.hist)} end, Stats). locks2print(Locks, Duration) -> lists:map( fun (L) -> - Tries = lists:sum([T || #stats{ tries = T} <- L#lock.stats]), - Colls = lists:sum([C || #stats{ colls = C} <- L#lock.stats]), - Time = lists:sum([T || #stats{ time = T} <- L#lock.stats]), - Cr = percent(Colls, Tries), - Dtr = percent(Time, Duration), - #print{ - name = L#lock.name, - id = L#lock.id, - type = L#lock.type, - tries = Tries, - colls = Colls, - cr = Cr, - time = Time, - dtr = Dtr - } + #stats{tries = Tries, + colls = Colls, + time = Time, + hist = Hist} = summate_stats(L#lock.stats), + Cr = percent(Colls, Tries), + Dtr = percent(Time, Duration), + #print{name = L#lock.name, + id = L#lock.id, + type = L#lock.type, + tries = Tries, + colls = Colls, + hist = format_histogram(Hist), + cr = Cr, + time = Time, + dtr = Dtr} end, Locks). + +format_histogram(Tup) when is_tuple(Tup) -> + Vs = tuple_to_list(Tup), + Max = lists:max(Vs), + case Max of + 0 -> string_histogram(Vs); + _ -> string_histogram([case V of 0 -> 0; _ -> V/Max end || V <- Vs]) + end. + +string_histogram([0|Vs]) -> + [$\s|string_histogram(Vs)]; +string_histogram([V|Vs]) when V > 0.66 -> + [$X|string_histogram(Vs)]; +string_histogram([V|Vs]) when V > 0.33 -> + [$x|string_histogram(Vs)]; +string_histogram([_|Vs]) -> + [$.|string_histogram(Vs)]; +string_histogram([]) -> []. + %% state making data2state(Data, State) -> @@ -606,22 +668,32 @@ data2state(Data, State) -> locks = Locks }. -locks2records(Locks) -> locks2records(Locks, []). -locks2records([], Out) -> Out; -locks2records([{Name, Id, Type, Stats}|Locks], Out) -> - Lock = #lock{ - name = Name, - id = clean_id_creation(Id), - type = Type, - stats = [ #stats{ - file = File, - line = Line, - tries = Tries, - colls = Colls, - time = time2us({S, Ns}), - nt = N - } || {{File, Line}, {Tries, Colls, {S, Ns, N}}} <- Stats] }, - locks2records(Locks, [Lock|Out]). +locks2records([{Name, Id, Type, Stats}|Locks]) -> + [#lock{name = Name, + id = clean_id_creation(Id), + type = Type, + stats = stats2record(Stats)}|locks2records(Locks)]; +locks2records([]) -> []. + +%% new stats with histogram +stats2record([{{File,Line},{Tries,Colls,{S,Ns,N}},Hist}|Stats]) -> + [#stats{file = File, + line = Line, + hist = Hist, + tries = Tries, + colls = Colls, + time = time2us({S, Ns}), + nt = N} | stats2record(Stats)]; +%% old stats without histogram +stats2record([{{File,Line},{Tries,Colls,{S,Ns,N}}}|Stats]) -> + [#stats{file = File, + line = Line, + hist = {}, + tries = Tries, + colls = Colls, + time = time2us({S, Ns}), + nt = N} | stats2record(Stats)]; +stats2record([]) -> []. clean_id_creation(Id) when is_pid(Id) -> Bin = term_to_binary(Id), @@ -647,22 +719,45 @@ state2list(State) -> (X, Y) -> {X,Y} end, record_info(fields, state), Values). -list2state(List) -> list2state(record_info(fields, state), List, [state]). -list2state([], _, Out) -> list_to_tuple(lists:reverse(Out)); -list2state([locks|Fs], List, Out) -> - Locks = [ list2lock(Lock) || Lock <- proplists:get_value(locks, List, [])], - list2state(Fs, List, [Locks|Out]); -list2state([F|Fs], List, Out) -> list2state(Fs, List, [proplists:get_value(F, List, state_default(F))|Out]). - lock_default(Field) -> proplists:get_value(Field, lock2list(#lock{})). lock2list(Lock) -> [_|Values] = tuple_to_list(Lock), lists:zip(record_info(fields, lock), Values). -list2lock(List) -> list2lock(record_info(fields, lock), List, [lock]). -list2lock([], _, Out) -> list_to_tuple(lists:reverse(Out)); -list2lock([F|Fs], List, Out) -> list2lock(Fs, List, [proplists:get_value(F, List, lock_default(F))|Out]). + +list2state(List) -> + list_to_tuple([state|list2state(record_info(fields, state), List)]). +list2state([], _) -> []; +list2state([locks|Fs], List) -> + Locks = [list2lock(Lock) || Lock <- proplists:get_value(locks, List, [])], + [Locks|list2state(Fs,List)]; +list2state([F|Fs], List) -> + [proplists:get_value(F, List, state_default(F))|list2state(Fs, List)]. + +list2lock(Ls) -> + list_to_tuple([lock|list2lock(record_info(fields, lock), Ls)]). + +list2lock([],_) -> []; +list2lock([stats=F|Fs], Ls) -> + Stats = stats2stats(proplists:get_value(F, Ls, lock_default(F))), + [Stats|list2lock(Fs, Ls)]; +list2lock([F|Fs], Ls) -> + [proplists:get_value(F, Ls, lock_default(F))|list2lock(Fs, Ls)]. + +%% process old stats (hack) +%% old stats had no histograms +%% in future versions stats should be serialized as a list, not a record + +stats2stats([]) -> []; +stats2stats([Stat|Stats]) -> + Sz = tuple_size(#stats{}), + [stat2stat(Stat,Sz)|stats2stats(Stats)]. + +stat2stat(Stat,Sz) when tuple_size(Stat) =:= Sz -> Stat; +stat2stat(Stat,_) -> + %% assume no histogram at the end + list_to_tuple(tuple_to_list(Stat) ++ [{0}]). %% printing @@ -683,7 +778,7 @@ auto_print_width(Locks, Print) -> ({print,print}, Out) -> [print|Out]; ({Str, Len}, Out) -> [erlang:min(erlang:max(length(s(Str))+1,Len),80)|Out] end, [], lists:zip(tuple_to_list(L), tuple_to_list(Max))))) - end, #print{ id = 4, type = 5, entry = 5, name = 6, tries = 8, colls = 13, cr = 16, time = 11, dtr = 14 }, + end, #print{ id = 4, type = 5, entry = 5, name = 6, tries = 8, colls = 13, cr = 16, time = 11, dtr = 14, hist=20 }, Locks), % Setup the offsets for later pruning Offsets = [ @@ -695,7 +790,9 @@ auto_print_width(Locks, Print) -> {colls, R#print.colls}, {ratio, R#print.cr}, {time, R#print.time}, - {duration, R#print.dtr}], + {duration, R#print.dtr}, + {histogram, R#print.hist} + ], % Prune offsets to only allow specified print options lists:foldr(fun ({Type, W}, Out) -> [{Type, W}|Out]; @@ -705,9 +802,7 @@ auto_print_width(Locks, Print) -> print_lock_information(Locks, Print) -> % remake Print to autosize entries AutoPrint = auto_print_width(Locks, Print), - print_header(AutoPrint), - lists:foreach(fun (L) -> print_lock(L, AutoPrint) @@ -724,7 +819,8 @@ print_header(Opts) -> colls = "#collisions", cr = "collisions [%]", time = "time [us]", - dtr = "duration [%]" + dtr = "duration [%]", + hist = "histogram" }, Divider = #print{ name = lists:duplicate(1 + length(Header#print.name), 45), @@ -735,39 +831,44 @@ print_header(Opts) -> colls = lists:duplicate(1 + length(Header#print.colls), 45), cr = lists:duplicate(1 + length(Header#print.cr), 45), time = lists:duplicate(1 + length(Header#print.time), 45), - dtr = lists:duplicate(1 + length(Header#print.dtr), 45) + dtr = lists:duplicate(1 + length(Header#print.dtr), 45), + hist = lists:duplicate(1 + length(Header#print.hist), 45) }, print_lock(Header, Opts), print_lock(Divider, Opts), ok. -print_lock(L, Opts) -> print_lock(L, Opts, []). -print_lock(_, [], Formats) -> print(strings(lists:reverse(Formats))); -print_lock(L, [Opt|Opts], Formats) -> +print_lock(L, Opts) -> + print(strings(format_lock(L, Opts))). + +format_lock(_, []) -> []; +format_lock(L, [Opt|Opts]) -> case Opt of - id -> print_lock(L, Opts, [{space, 25, s(L#print.id) } | Formats]); - {id, W} -> print_lock(L, Opts, [{space, W, s(L#print.id) } | Formats]); - type -> print_lock(L, Opts, [{space, 18, s(L#print.type) } | Formats]); - {type, W} -> print_lock(L, Opts, [{space, W, s(L#print.type) } | Formats]); - entry -> print_lock(L, Opts, [{space, 30, s(L#print.entry)} | Formats]); - {entry, W} -> print_lock(L, Opts, [{space, W, s(L#print.entry)} | Formats]); - name -> print_lock(L, Opts, [{space, 22, s(L#print.name) } | Formats]); - {name, W} -> print_lock(L, Opts, [{space, W, s(L#print.name) } | Formats]); - tries -> print_lock(L, Opts, [{space, 12, s(L#print.tries)} | Formats]); - {tries, W} -> print_lock(L, Opts, [{space, W, s(L#print.tries)} | Formats]); - colls -> print_lock(L, Opts, [{space, 14, s(L#print.colls)} | Formats]); - {colls, W} -> print_lock(L, Opts, [{space, W, s(L#print.colls)} | Formats]); - ratio -> print_lock(L, Opts, [{space, 20, s(L#print.cr) } | Formats]); - {ratio, W} -> print_lock(L, Opts, [{space, W, s(L#print.cr) } | Formats]); - time -> print_lock(L, Opts, [{space, 15, s(L#print.time) } | Formats]); - {time, W} -> print_lock(L, Opts, [{space, W, s(L#print.time) } | Formats]); - duration -> print_lock(L, Opts, [{space, 20, s(L#print.dtr) } | Formats]); - {duration, W} -> print_lock(L, Opts, [{space, W, s(L#print.dtr) } | Formats]); - _ -> print_lock(L, Opts, Formats) + id -> [{space, 25, s(L#print.id) } | format_lock(L, Opts)]; + {id, W} -> [{space, W, s(L#print.id) } | format_lock(L, Opts)]; + type -> [{space, 18, s(L#print.type) } | format_lock(L, Opts)]; + {type, W} -> [{space, W, s(L#print.type) } | format_lock(L, Opts)]; + entry -> [{space, 30, s(L#print.entry)} | format_lock(L, Opts)]; + {entry, W} -> [{space, W, s(L#print.entry)} | format_lock(L, Opts)]; + name -> [{space, 22, s(L#print.name) } | format_lock(L, Opts)]; + {name, W} -> [{space, W, s(L#print.name) } | format_lock(L, Opts)]; + tries -> [{space, 12, s(L#print.tries)} | format_lock(L, Opts)]; + {tries, W} -> [{space, W, s(L#print.tries)} | format_lock(L, Opts)]; + colls -> [{space, 14, s(L#print.colls)} | format_lock(L, Opts)]; + {colls, W} -> [{space, W, s(L#print.colls)} | format_lock(L, Opts)]; + ratio -> [{space, 20, s(L#print.cr) } | format_lock(L, Opts)]; + {ratio, W} -> [{space, W, s(L#print.cr) } | format_lock(L, Opts)]; + time -> [{space, 15, s(L#print.time) } | format_lock(L, Opts)]; + {time, W} -> [{space, W, s(L#print.time) } | format_lock(L, Opts)]; + duration -> [{space, 20, s(L#print.dtr) } | format_lock(L, Opts)]; + {duration, W} -> [{space, W, s(L#print.dtr) } | format_lock(L, Opts)]; + histogram -> [{space, 0, s(L#print.hist) } | format_lock(L, Opts)]; + {histogram, W} -> [{space, W, s(L#print.hist) } | format_lock(L, Opts)]; + _ -> format_lock(L, Opts) end. -print_state_information(#state{ locks = Locks} = State) -> +print_state_information(#state{locks = Locks} = State) -> Stats = summate_locks(Locks), print("information:"), print(kv("#locks", s(length(Locks)))), @@ -779,9 +880,25 @@ print_state_information(#state{ locks = Locks} = State) -> print(kv("percent of duration", s(Stats#stats.time/State#state.duration*100) ++ " %")), ok. + +print_full_histogram(T) when is_tuple(T) -> + Vs = tuple_to_list(T), + Max = lists:max(Vs), + W = 60, + print_full_histogram(0,Vs,Max,W). + +print_full_histogram(_,[],_,_) -> ok; +print_full_histogram(Ix,[V|Vs],0,W) -> + io:format("~2w = log2 : ~8w |~n", [Ix,V]), + print_full_histogram(Ix+1,Vs,0,W); +print_full_histogram(Ix,[V|Vs],Max,W) -> + io:format("~2w = log2 : ~8w | ~s~n", [Ix,V,lists:duplicate(trunc(W*(V/Max)), $#)]), + print_full_histogram(Ix+1,Vs,Max,W). + + %% AUX -time2us({S, Ns}) -> round(S*1000000 + Ns/1000). +time2us({S, Ns}) -> S*1000000 + (Ns div 1000). percent(_,0) -> 0.0; percent(T,N) -> T/N*100. @@ -808,7 +925,7 @@ s(T) -> term2string(T). strings(Strings) -> strings(Strings, []). strings([], Out) -> Out; -strings([{space, N, S} | Ss], Out) -> strings(Ss, Out ++ term2string(term2string("~~~ps", [N]), [S])); +strings([{space, N, S} | Ss], Out) -> strings(Ss, Out ++ term2string(term2string("~~~ws", [N]), [S])); strings([{format, Format, S} | Ss], Out) -> strings(Ss, Out ++ term2string(Format, [S])); strings([S|Ss], Out) -> strings(Ss, Out ++ term2string("~ts", [S])). @@ -825,7 +942,7 @@ term2string(Term) when is_pid(Term) -> term2string(Term) -> term2string("~w", [Term]). term2string(Format, Terms) -> lists:flatten(io_lib:format(Format, Terms)). -%%% AUD id binary +%%% AUX id binary bytes16(Value) -> B0 = Value band 255, diff --git a/lib/tools/test/eprof_SUITE.erl b/lib/tools/test/eprof_SUITE.erl index 1227d5b841..04b522de4a 100644 --- a/lib/tools/test/eprof_SUITE.erl +++ b/lib/tools/test/eprof_SUITE.erl @@ -104,7 +104,7 @@ basic(Config) when is_list(Config) -> profiling = eprof:profile([A]), true = exit(A, kill_it), profiling_stopped = eprof:stop_profiling(), - {error,_} = eprof:profile(fun() -> a = b end), + {error,_} = eprof:profile(fun() -> a = id(b) end), %% with mfa @@ -149,8 +149,7 @@ basic_option_1(Config) -> % vanilla {ok, _} = eprof:profile(fun() -> eprof_test:do(10) end, [{set_on_spawn, true}]), - [{_, MfasDo1},{_, MfasLists1}] = eprof:dump(), - Mfas1 = MfasDo1 ++ MfasLists1, + Mfas1 = lists:foldl(fun({_,Mfas},Out) -> Mfas ++ Out end, [], eprof:dump()), {value, {_, {11, _}}} = lists:keysearch({eprof_test,dec,1}, 1, Mfas1), {value, {_, { 1, _}}} = lists:keysearch({eprof_test, go,1}, 1, Mfas1), @@ -159,8 +158,7 @@ basic_option_1(Config) -> {ok, _} = eprof:profile(fun() -> eprof_test:do(10) end, [set_on_spawn]), - [{_, MfasDo2},{_, MfasLists2}] = eprof:dump(), - Mfas2 = MfasDo2 ++ MfasLists2, + Mfas2 = lists:foldl(fun({_,Mfas},Out) -> Mfas ++ Out end, [], eprof:dump()), {value, {_, {11, _}}} = lists:keysearch({eprof_test,dec,1}, 1, Mfas2), {value, {_, { 1, _}}} = lists:keysearch({eprof_test, go,1}, 1, Mfas2), {value, {_, { 9, _}}} = lists:keysearch({lists, split_2,5}, 1, Mfas2), @@ -255,3 +253,5 @@ ensure_eprof_stopped() -> Pid -> stopped=eprof:stop() end. + +id(I) -> I. diff --git a/lib/tools/test/lcnt_SUITE.erl b/lib/tools/test/lcnt_SUITE.erl index 1bee6021ab..010dffe138 100644 --- a/lib/tools/test/lcnt_SUITE.erl +++ b/lib/tools/test/lcnt_SUITE.erl @@ -27,11 +27,11 @@ %% Test cases -export([ - load_v1/1, - conflicts/1, - locations/1, - swap_keys/1 - ]). + t_load/1, + t_conflicts/1, + t_locations/1, + t_swap_keys/1 + ]). %% Default timetrap timeout (set in init_per_testcase) -define(default_timeout, ?t:minutes(4)). @@ -54,48 +54,52 @@ end_per_testcase(_Case, Config) -> suite() -> [{ct_hooks,[ts_install_cth]}]. -all() -> - [load_v1, conflicts, locations, swap_keys]. +all() -> [t_load, t_conflicts, t_locations, t_swap_keys]. -groups() -> - []. +groups() -> []. -init_per_group(_GroupName, Config) -> - Config. +init_per_group(_GroupName, Config) -> Config. -end_per_group(_GroupName, Config) -> - Config. +end_per_group(_GroupName, Config) -> Config. %%---------------------------------------------------------------------- %% Tests %%---------------------------------------------------------------------- -load_v1(suite) -> - []; -load_v1(doc) -> - ["Load data from file."]; -load_v1(Config) when is_list(Config) -> - ?line {ok, _} = lcnt:start(), - ?line Path = ?config(data_dir, Config), - ?line File = filename:join([Path,"big_bang_40.lcnt"]), - ?line ok = lcnt:load(File), - ?line ok = lcnt:stop(), +t_load(suite) -> []; +t_load(doc) -> ["Load data from file."]; +t_load(Config) when is_list(Config) -> + Path = ?config(data_dir, Config), + Files = [filename:join([Path,"big_bang_40.lcnt"]), + filename:join([Path,"ehb_3_3_hist.lcnt"])], + ok = t_load_file(Files), ok. -conflicts(suite) -> - []; -conflicts(doc) -> - ["API: conflicts"]; -conflicts(Config) when is_list(Config) -> - ?line {ok, _} = lcnt:start(), - ?line Path = ?config(data_dir, Config), - ?line File = filename:join([Path,"big_bang_40.lcnt"]), - ?line ok = lcnt:load(File), - ?line ok = lcnt:conflicts(), - THs = [-1, 0, 100, 1000], - Print = [name , id , type , entry , tries , colls , ratio , time , duration], - Opts = [ +t_load_file([]) -> ok; +t_load_file([File|Files]) -> + {ok, _} = lcnt:start(), + ok = lcnt:load(File), + ok = lcnt:stop(), + t_load_file(Files). + +t_conflicts(suite) -> []; +t_conflicts(doc) -> ["API: conflicts"]; +t_conflicts(Config) when is_list(Config) -> + Path = ?config(data_dir, Config), + Files = [filename:join([Path,"big_bang_40.lcnt"]), + filename:join([Path,"ehb_3_3_hist.lcnt"])], + ok = t_conflicts_file(Files), + ok. + +t_conflicts_file([]) -> ok; +t_conflicts_file([File|Files]) -> + {ok, _} = lcnt:start(), + ok = lcnt:load(File), + ok = lcnt:conflicts(), + THs = [-1, 0, 100, 1000], + Print = [name , id , type , entry , tries , colls , ratio , time , duration], + Opts = [ [{sort, Sort}, {reverse, Rev}, {max_locks, ML}, {combine, Combine}, {thresholds, [TH]}, {print, [Print]}] || Sort <- [name , id , type , tries , colls , ratio , time , entry], ML <- [none, 1 , 32, 4096], @@ -103,28 +107,33 @@ conflicts(Config) when is_list(Config) -> TH <- [{tries, Tries} || Tries <- THs] ++ [{colls, Colls} || Colls <- THs] ++ [{time, Time} || Time <- THs], Rev <- [true, false] ], - ?line ok = test_conflicts_opts(Opts), - ?line ok = lcnt:stop(), - ok. + ok = test_conflicts_opts(Opts), + ok = lcnt:stop(), + t_conflicts_file(Files). + test_conflicts_opts([]) -> ok; test_conflicts_opts([Opt|Opts]) -> - ?line ok = lcnt:conflicts(Opt), + ok = lcnt:conflicts(Opt), test_conflicts_opts(Opts). -locations(suite) -> - []; -locations(doc) -> - ["API: locations"]; -locations(Config) when is_list(Config) -> - ?line {ok, _} = lcnt:start(), - ?line Path = ?config(data_dir, Config), - ?line File = filename:join([Path,"big_bang_40.lcnt"]), - ?line ok = lcnt:load(File), - ?line ok = lcnt:locations(), - THs = [-1, 0, 100, 1000], - Print = [name , id , type , entry , tries , colls , ratio , time , duration], - Opts = [ +t_locations(suite) -> []; +t_locations(doc) -> ["API: locations"]; +t_locations(Config) when is_list(Config) -> + Path = ?config(data_dir, Config), + Files = [filename:join([Path,"big_bang_40.lcnt"]), + filename:join([Path,"ehb_3_3_hist.lcnt"])], + ok = t_locations_file(Files), + ok. + +t_locations_file([]) -> ok; +t_locations_file([File|Files]) -> + {ok, _} = lcnt:start(), + ok = lcnt:load(File), + ok = lcnt:locations(), + THs = [-1, 0, 100, 1000], + Print = [name , id , type , entry , tries , colls , ratio , time , duration], + Opts = [ [{full_id, Id}, {sort, Sort}, {max_locks, ML}, {combine, Combine}, {thresholds, [TH]}, {print, Print}] || Sort <- [name , id , type , tries , colls , ratio , time , entry], ML <- [none, 1 , 64], @@ -132,30 +141,34 @@ locations(Config) when is_list(Config) -> TH <- [{tries, Tries} || Tries <- THs] ++ [{colls, Colls} || Colls <- THs] ++ [{time, Time} || Time <- THs], Id <- [true, false] ], - ?line ok = test_locations_opts(Opts), - ?line ok = lcnt:stop(), - ok. + ok = test_locations_opts(Opts), + ok = lcnt:stop(), + t_locations_file(Files). test_locations_opts([]) -> ok; test_locations_opts([Opt|Opts]) -> - ?line ok = lcnt:locations(Opt), + ok = lcnt:locations(Opt), test_locations_opts(Opts). -swap_keys(suite) -> - []; -swap_keys(doc) -> - ["Test interchanging port/process id with class"]; -swap_keys(Config) when is_list(Config) -> - ?line {ok, _} = lcnt:start(), - ?line Path = ?config(data_dir, Config), - ?line File = filename:join([Path,"big_bang_40.lcnt"]), - ?line ok = lcnt:load(File), - ?line ok = lcnt:conflicts(), - ?line ok = lcnt:swap_pid_keys(), - ?line ok = lcnt:conflicts(), - ?line ok = lcnt:stop(), +t_swap_keys(suite) -> []; +t_swap_keys(doc) -> ["Test interchanging port/process id with class"]; +t_swap_keys(Config) when is_list(Config) -> + Path = ?config(data_dir, Config), + Files = [filename:join([Path,"big_bang_40.lcnt"]), + filename:join([Path,"ehb_3_3_hist.lcnt"])], + ok = t_swap_keys_file(Files), ok. +t_swap_keys_file([]) -> ok; +t_swap_keys_file([File|Files]) -> + {ok, _} = lcnt:start(), + ok = lcnt:load(File), + ok = lcnt:conflicts(), + ok = lcnt:swap_pid_keys(), + ok = lcnt:conflicts(), + ok = lcnt:stop(), + t_swap_keys_file(Files). + %%---------------------------------------------------------------------- %% Auxiliary tests diff --git a/lib/tools/test/lcnt_SUITE_data/ehb_3_3_hist.lcnt b/lib/tools/test/lcnt_SUITE_data/ehb_3_3_hist.lcnt Binary files differnew file mode 100644 index 0000000000..ff5bdcbdaa --- /dev/null +++ b/lib/tools/test/lcnt_SUITE_data/ehb_3_3_hist.lcnt diff --git a/lib/tools/test/xref_SUITE.erl b/lib/tools/test/xref_SUITE.erl index 353275ae3b..6870aefe5c 100644 --- a/lib/tools/test/xref_SUITE.erl +++ b/lib/tools/test/xref_SUITE.erl @@ -1098,7 +1098,6 @@ read_expected(Version) -> {POS1+1,{FF,{mod17,fun17,0}}}, {POS1+2,{FF,{erlang,spawn,1}}}, {POS1+2,{FF,{read,local,0}}}, - {POS1+3,{FF,{erlang,binary_to_term,1}}}, {POS1+3,{FF,{erlang,spawn,1}}}, {POS1+4,{FF,{dist,func,0}}}, {POS1+4,{FF,{erlang,spawn,1}}}, @@ -1207,6 +1206,7 @@ read_expected(Version) -> OKB1 = [{POS13+1,{FF,{erts_debug,apply,4}}}, {POS13+2,{FF,{erts_debug,apply,4}}}, {POS13+3,{FF,{erts_debug,apply,4}}}, + {POS1+3, {FF,{erlang,binary_to_term,1}}}, {POS3+1, {FF,{erlang,spawn,3}}}, {POS3+2, {FF,{erlang,spawn,3}}}, {POS3+3, {FF,{erlang,spawn_link,3}}}, diff --git a/lib/tools/vsn.mk b/lib/tools/vsn.mk index 2d2970de3a..3acb8d38e2 100644 --- a/lib/tools/vsn.mk +++ b/lib/tools/vsn.mk @@ -1 +1 @@ -TOOLS_VSN = 2.6.14 +TOOLS_VSN = 2.7 diff --git a/lib/typer/Makefile b/lib/typer/Makefile index 40a82e9bba..d4396abc9d 100644 --- a/lib/typer/Makefile +++ b/lib/typer/Makefile @@ -29,7 +29,7 @@ include $(ERL_TOP)/make/$(TARGET)/otp.mk # Macros # -SUB_DIRECTORIES = src +SUB_DIRECTORIES = src doc/src include vsn.mk VSN = $(TYPER_VSN) diff --git a/lib/typer/doc/Makefile b/lib/typer/doc/Makefile new file mode 100644 index 0000000000..4ea0137202 --- /dev/null +++ b/lib/typer/doc/Makefile @@ -0,0 +1,39 @@ +# +# %CopyrightBegin% +# +# Copyright Ericsson AB 2006-2012. All Rights Reserved. +# +# The contents of this file are subject to the Erlang Public License, +# Version 1.1, (the "License"); you may not use this file except in +# compliance with the License. You should have received a copy of the +# Erlang Public License along with this software. If not, it can be +# retrieved online at http://www.erlang.org/. +# +# Software distributed under the License is distributed on an "AS IS" +# basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +# the License for the specific language governing rights and limitations +# under the License. +# +# %CopyrightEnd% +# +SHELL=/bin/sh + +include $(ERL_TOP)/make/target.mk +include $(ERL_TOP)/make/$(TARGET)/otp.mk + +clean: + -rm -f *.html edoc-info stylesheet.css erlang.png + +distclean: clean +realclean: clean + +# ---------------------------------------------------- +# Special Build Targets +# ---------------------------------------------------- + + + +# ---------------------------------------------------- +# Release Target +# ---------------------------------------------------- +include $(ERL_TOP)/make/otp_release_targets.mk diff --git a/lib/typer/doc/html/.gitignore b/lib/typer/doc/html/.gitignore new file mode 100644 index 0000000000..e69de29bb2 --- /dev/null +++ b/lib/typer/doc/html/.gitignore diff --git a/lib/typer/doc/pdf/.gitignore b/lib/typer/doc/pdf/.gitignore new file mode 100644 index 0000000000..e69de29bb2 --- /dev/null +++ b/lib/typer/doc/pdf/.gitignore diff --git a/lib/typer/doc/src/Makefile b/lib/typer/doc/src/Makefile new file mode 100644 index 0000000000..2683c08679 --- /dev/null +++ b/lib/typer/doc/src/Makefile @@ -0,0 +1,117 @@ +# +# %CopyrightBegin% +# +# Copyright Ericsson AB 2006-2012. All Rights Reserved. +# +# The contents of this file are subject to the Erlang Public License, +# Version 1.1, (the "License"); you may not use this file except in +# compliance with the License. You should have received a copy of the +# Erlang Public License along with this software. If not, it can be +# retrieved online at http://www.erlang.org/. +# +# Software distributed under the License is distributed on an "AS IS" +# basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +# the License for the specific language governing rights and limitations +# under the License. +# +# %CopyrightEnd% +# +include $(ERL_TOP)/make/target.mk +include $(ERL_TOP)/make/$(TARGET)/otp.mk + +# ---------------------------------------------------- +# Application version +# ---------------------------------------------------- +include ../../vsn.mk +VSN=$(TYPER_VSN) +APPLICATION=typer + +# ---------------------------------------------------- +# Release directory specification +# ---------------------------------------------------- +RELSYSDIR = $(RELEASE_PATH)/lib/$(APPLICATION)-$(VSN) + +# ---------------------------------------------------- +# Target Specs +# ---------------------------------------------------- +XML_APPLICATION_FILES = ref_man.xml +XML_REF3_FILES = + +XML_PART_FILES = part_notes.xml +XML_CHAPTER_FILES = notes.xml + +BOOK_FILES = book.xml + +XML_FILES = \ + $(BOOK_FILES) $(XML_CHAPTER_FILES) \ + $(XML_PART_FILES) $(XML_REF3_FILES) $(XML_APPLICATION_FILES) + +GIF_FILES = + +# ---------------------------------------------------- + +HTML_FILES = $(XML_APPLICATION_FILES:%.xml=$(HTMLDIR)/%.html) \ + $(XML_PART_FILES:%.xml=$(HTMLDIR)/%.html) + +INFO_FILE = ../../info +EXTRA_FILES = \ + $(DEFAULT_GIF_FILES) \ + $(DEFAULT_HTML_FILES) \ + $(XML_REF3_FILES:%.xml=$(HTMLDIR)/%.html) \ + $(XML_CHAPTER_FILES:%.xml=$(HTMLDIR)/%.html) + +MAN3_FILES = $(XML_REF3_FILES:%.xml=$(MAN3DIR)/%.3) + +HTML_REF_MAN_FILE = $(HTMLDIR)/index.html + +TOP_PDF_FILE = $(PDFDIR)/$(APPLICATION)-$(VSN).pdf + +# ---------------------------------------------------- +# FLAGS +# ---------------------------------------------------- +XML_FLAGS += + +# ---------------------------------------------------- +# Targets +# ---------------------------------------------------- +$(HTMLDIR)/%.gif: %.gif + $(INSTALL_DATA) $< $@ + +docs: pdf html man + +$(TOP_PDF_FILE): $(XML_FILES) + +pdf: $(TOP_PDF_FILE) + +html: gifs $(HTML_REF_MAN_FILE) + +man: $(MAN3_FILES) + +gifs: $(GIF_FILES:%=$(HTMLDIR)/%) + +debug opt: + +clean clean_docs: + rm -rf $(HTMLDIR)/* + rm -f $(MAN3DIR)/* + rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) + rm -f errs core *~ + +distclean: clean +realclean: clean + +# ---------------------------------------------------- +# Release Target +# ---------------------------------------------------- +include $(ERL_TOP)/make/otp_release_targets.mk + +release_docs_spec: docs + $(INSTALL_DIR) "$(RELSYSDIR)/doc/pdf" + $(INSTALL_DATA) $(TOP_PDF_FILE) "$(RELSYSDIR)/doc/pdf" + $(INSTALL_DIR) "$(RELSYSDIR)/doc/html" + $(INSTALL_DATA) $(HTMLDIR)/* \ + "$(RELSYSDIR)/doc/html" + $(INSTALL_DATA) $(INFO_FILE) "$(RELSYSDIR)" + + +release_spec: diff --git a/lib/typer/doc/src/book.xml b/lib/typer/doc/src/book.xml new file mode 100644 index 0000000000..5cc85a3022 --- /dev/null +++ b/lib/typer/doc/src/book.xml @@ -0,0 +1,41 @@ +<?xml version="1.0" encoding="utf-8" ?> +<!DOCTYPE book SYSTEM "book.dtd"> + +<book xmlns:xi="http://www.w3.org/2001/XInclude"> + <header titlestyle="normal"> + <copyright> + <year>2006</year><year>2013</year> + <holder>Ericsson AB. All Rights Reserved.</holder> + </copyright> + <legalnotice> + The contents of this file are subject to the Erlang Public License, + Version 1.1, (the "License"); you may not use this file except in + compliance with the License. You should have received a copy of the + Erlang Public License along with this software. If not, it can be + retrieved online at http://www.erlang.org/. + + Software distributed under the License is distributed on an "AS IS" + basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + the License for the specific language governing rights and limitations + under the License. + + </legalnotice> + + <title>TypEr</title> + <prepared></prepared> + <docno></docno> + <date></date> + <rev></rev> + </header> + <pagetext></pagetext> + <preamble> + </preamble> + <pagetext>TypEr</pagetext> + <applications> + <xi:include href="ref_man.xml"/> + </applications> + <releasenotes> + <xi:include href="notes.xml"/> + </releasenotes> +</book> + diff --git a/lib/typer/doc/src/fascicules.xml b/lib/typer/doc/src/fascicules.xml new file mode 100644 index 0000000000..b15610fa8b --- /dev/null +++ b/lib/typer/doc/src/fascicules.xml @@ -0,0 +1,12 @@ +<?xml version="1.0" encoding="utf-8" ?> +<!DOCTYPE fascicules SYSTEM "fascicules.dtd"> + +<fascicules> + <fascicule file="part_notes" href="part_notes_frame.html" entry="yes"> + Release Notes + </fascicule> + <fascicule file="" href="../../../../doc/print.html" entry="no"> + Off-Print + </fascicule> +</fascicules> + diff --git a/lib/typer/doc/src/notes.xml b/lib/typer/doc/src/notes.xml new file mode 100644 index 0000000000..23e22759d6 --- /dev/null +++ b/lib/typer/doc/src/notes.xml @@ -0,0 +1,66 @@ +<?xml version="1.0" encoding="utf-8" ?> +<!DOCTYPE chapter SYSTEM "chapter.dtd"> + +<chapter> + <header> + <copyright> + <year>2014</year> + <holder>Ericsson AB. All Rights Reserved.</holder> + </copyright> + <legalnotice> + The contents of this file are subject to the Erlang Public License, + Version 1.1, (the "License"); you may not use this file except in + compliance with the License. You should have received a copy of the + Erlang Public License along with this software. If not, it can be + retrieved online at http://www.erlang.org/. + + Software distributed under the License is distributed on an "AS IS" + basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + the License for the specific language governing rights and limitations + under the License. + + </legalnotice> + + <title>TypEr Release Notes</title> + <prepared>otp_appnotes</prepared> + <docno>nil</docno> + <date>nil</date> + <rev>nil</rev> + <file>notes.xml</file> + </header> + <p>This document describes the changes made to TypEr.</p> + +<section><title>TypEr 0.9.8</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> The name of a compiler option has been fixed in the + Makefile. </p> + <p> + Own Id: OTP-11996</p> + </item> + </list> + </section> + +</section> + +<section><title>TypEr 0.9.7</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Added initial documentation framework for TypEr.</p> + <p> + Own Id: OTP-11860</p> + </item> + </list> + </section> + +</section> + + + +</chapter> + diff --git a/lib/typer/doc/src/part_notes.xml b/lib/typer/doc/src/part_notes.xml new file mode 100644 index 0000000000..b4ccd3ed77 --- /dev/null +++ b/lib/typer/doc/src/part_notes.xml @@ -0,0 +1,35 @@ +<?xml version="1.0" encoding="utf-8" ?> +<!DOCTYPE part SYSTEM "part.dtd"> + +<part xmlns:xi="http://www.w3.org/2001/XInclude"> + <header> + <copyright> + <year>2006</year><year>2013</year> + <holder>Ericsson AB. All Rights Reserved.</holder> + </copyright> + <legalnotice> + The contents of this file are subject to the Erlang Public License, + Version 1.1, (the "License"); you may not use this file except in + compliance with the License. You should have received a copy of the + Erlang Public License along with this software. If not, it can be + retrieved online at http://www.erlang.org/. + + Software distributed under the License is distributed on an "AS IS" + basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + the License for the specific language governing rights and limitations + under the License. + + </legalnotice> + + <title>TypEr Release Notes</title> + <prepared></prepared> + <docno></docno> + <date></date> + <rev></rev> + </header> + <description> + <p><em>TypEr</em></p> + </description> + <xi:include href="notes.xml"/> +</part> + diff --git a/lib/typer/doc/src/ref_man.xml b/lib/typer/doc/src/ref_man.xml new file mode 100644 index 0000000000..b54a5f5947 --- /dev/null +++ b/lib/typer/doc/src/ref_man.xml @@ -0,0 +1,35 @@ +<?xml version="1.0" encoding="utf-8" ?> +<!DOCTYPE application SYSTEM "application.dtd"> + +<application xmlns:xi="http://www.w3.org/2001/XInclude"> + <header> + <copyright> + <year>2014</year> + <holder>Ericsson AB. All Rights Reserved.</holder> + </copyright> + <legalnotice> + The contents of this file are subject to the Erlang Public License, + Version 1.1, (the "License"); you may not use this file except in + compliance with the License. You should have received a copy of the + Erlang Public License along with this software. If not, it can be + retrieved online at http://www.erlang.org/. + + Software distributed under the License is distributed on an "AS IS" + basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + the License for the specific language governing rights and limitations + under the License. + + </legalnotice> + + <title>TypEr</title> + <prepared></prepared> + <docno></docno> + <date></date> + <rev></rev> + <file>ref_man.xml</file> + </header> + <description> + </description> + <xi:include href="typer_app.xml"/> +</application> + diff --git a/lib/typer/doc/src/typer_app.xml b/lib/typer/doc/src/typer_app.xml new file mode 100644 index 0000000000..469a9be108 --- /dev/null +++ b/lib/typer/doc/src/typer_app.xml @@ -0,0 +1,43 @@ +<?xml version="1.0" encoding="utf-8" ?> +<!DOCTYPE appref SYSTEM "appref.dtd"> + +<appref> + <header> + <copyright> + <year>2014</year> + <holder>Ericsson AB. All Rights Reserved.</holder> + </copyright> + <legalnotice> + The contents of this file are subject to the Erlang Public License, + Version 1.1, (the "License"); you may not use this file except in + compliance with the License. You should have received a copy of the + Erlang Public License along with this software. If not, it can be + retrieved online at http://www.erlang.org/. + + Software distributed under the License is distributed on an "AS IS" + basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + the License for the specific language governing rights and limitations + under the License. + + </legalnotice> + + <title>TypEr</title> + <prepared></prepared> + <responsible></responsible> + <docno></docno> + <approved></approved> + <checked></checked> + <date></date> + <rev></rev> + <file>typer.xml</file> + </header> + <app>TypEr</app> + <appsummary>The TypEr Application</appsummary> + <description> + <p>An Erlang/OTP application that shows type information + for Erlang modules to the user. Additionally, it can + annotate the code of files with such type information.</p> + </description> + +</appref> + diff --git a/lib/typer/info b/lib/typer/info new file mode 100644 index 0000000000..5145fbcfff --- /dev/null +++ b/lib/typer/info @@ -0,0 +1,2 @@ +group: tools +short: TypEr diff --git a/lib/typer/src/Makefile b/lib/typer/src/Makefile index 13af466755..a7059de971 100644 --- a/lib/typer/src/Makefile +++ b/lib/typer/src/Makefile @@ -63,7 +63,7 @@ APPUP_TARGET= $(EBIN)/$(APPUP_FILE) # ---------------------------------------------------- # FLAGS # ---------------------------------------------------- -ERL_COMPILE_FLAGS += +warn_exported_vars +warn_untyped_record +warn_missing_spec +ERL_COMPILE_FLAGS += +warn_export_vars +warn_untyped_record +warn_missing_spec # ---------------------------------------------------- # Targets diff --git a/lib/typer/vsn.mk b/lib/typer/vsn.mk index 49fdda756e..ce658e257b 100644 --- a/lib/typer/vsn.mk +++ b/lib/typer/vsn.mk @@ -1 +1 @@ -TYPER_VSN = 0.9.6 +TYPER_VSN = 0.9.8 diff --git a/lib/webtool/doc/src/notes.xml b/lib/webtool/doc/src/notes.xml index 74e5fd88c1..e571668c91 100644 --- a/lib/webtool/doc/src/notes.xml +++ b/lib/webtool/doc/src/notes.xml @@ -31,6 +31,35 @@ <p>This document describes the changes made to the Webtool application.</p> +<section><title>WebTool 0.8.10</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Application upgrade (appup) files are corrected for the + following applications: </p> + <p> + <c>asn1, common_test, compiler, crypto, debugger, + dialyzer, edoc, eldap, erl_docgen, et, eunit, gs, hipe, + inets, observer, odbc, os_mon, otp_mibs, parsetools, + percept, public_key, reltool, runtime_tools, ssh, + syntax_tools, test_server, tools, typer, webtool, wx, + xmerl</c></p> + <p> + A new test utility for testing appup files is added to + test_server. This is now used by most applications in + OTP.</p> + <p> + (Thanks to Tobias Schlager)</p> + <p> + Own Id: OTP-11744</p> + </item> + </list> + </section> + +</section> + <section><title>WebTool 0.8.9.2</title> <section><title>Improvements and New Features</title> diff --git a/lib/wx/aclocal.m4 b/lib/wx/aclocal.m4 index c51c26794a..ed492d55ff 100644 --- a/lib/wx/aclocal.m4 +++ b/lib/wx/aclocal.m4 @@ -83,8 +83,10 @@ AC_ARG_VAR(erl_xcomp_ose_LM_POST_LINK, [OSE postlink tool (only used when cross AC_ARG_VAR(erl_xcomp_ose_LM_SET_CONF, [Sets the configuration for an OSE load module (only used when cross compiling for OSE)]) AC_ARG_VAR(erl_xcomp_ose_LM_ELF_SIZE, [Prints the section size information for an OSE load module (only used when cross compiling for OSE)]) AC_ARG_VAR(erl_xcomp_ose_LM_LCF, [OSE load module linker configuration file (only used when cross compiling for OSE)]) -AC_ARG_VAR(erl_xcomp_ose_LM_CONF, [OSE load module default configuration file (only used when cross compiling for OSE)]) -AC_ARG_VAR(erl_xcomp_ose_CONFD, [OSE OSE confd source file]) +AC_ARG_VAR(erl_xcomp_ose_BEAM_LM_CONF, [BEAM OSE load module default configuration file (only used when cross compiling for OSE)]) +AC_ARG_VAR(erl_xcomp_ose_EPMD_LM_CONF, [EPMD OSE load module default configuration file (only used when cross compiling for OSE)]) +AC_ARG_VAR(erl_xcomp_ose_RUN_ERL_LM_CONF, [run_erl_lm OSE load module default configuration file (only used when cross compiling for OSE)]) +AC_ARG_VAR(erl_xcomp_ose_CONFD, [OSE confd source file]) AC_ARG_VAR(erl_xcomp_ose_CRT0_LM, [OSE crt0 lm source file]) ]) @@ -1116,7 +1118,7 @@ case "$THR_LIB_NAME" in [Define if you have the "ose_spi/ose_spi.h" header file.])) ;; esac - if test "x$THR_LIB_NAME" == "xpthread"; then + if test "x$THR_LIB_NAME" = "xpthread"; then case $host_os in openbsd*) # The default stack size is insufficient for our needs @@ -1220,7 +1222,7 @@ case "$THR_LIB_NAME" in dnl dnl Check for functions dnl - if test "x$THR_LIB_NAME" == "xpthread"; then + if test "x$THR_LIB_NAME" = "xpthread"; then AC_CHECK_FUNC(pthread_spin_lock, \ [ethr_have_native_spinlock=yes \ AC_DEFINE(ETHR_HAVE_PTHREAD_SPIN_LOCK, 1, \ diff --git a/lib/wx/api_gen/wx_doxygen.conf b/lib/wx/api_gen/wx_doxygen.conf index 829702cbbf..f4d3c99ec0 100644 --- a/lib/wx/api_gen/wx_doxygen.conf +++ b/lib/wx/api_gen/wx_doxygen.conf @@ -57,7 +57,6 @@ GENERATE_DEPRECATEDLIST= NO ENABLED_SECTIONS = MAX_INITIALIZER_LINES = 30 SHOW_USED_FILES = YES -SHOW_DIRECTORIES = YES FILE_VERSION_FILTER = #--------------------------------------------------------------------------- # configuration options related to warning and progress messages @@ -110,7 +109,6 @@ HTML_FILE_EXTENSION = .html HTML_HEADER = HTML_FOOTER = HTML_STYLESHEET = -HTML_ALIGN_MEMBERS = YES GENERATE_HTMLHELP = NO CHM_FILE = HHC_LOCATION = @@ -251,7 +249,9 @@ PREDEFINED = \ wxUSE_DATAOBJ=1 \ wxUSE_SLIDER=1 \ wxUSE_CLIPBOARD=1 \ + wxUSE_POPUPWIN=1 \ wxUSE_SYSTEM_OPTIONS=1 \ + wxUSE_INTL=1 \ wxABI_VERSION=20809 \ __WXGTK24__=1 \ __WXGTK20__=1 \ diff --git a/lib/wx/api_gen/wx_extra/wxListCtrl.c_src b/lib/wx/api_gen/wx_extra/wxListCtrl.c_src index 8fa31e512e..d6196d11a2 100644 --- a/lib/wx/api_gen/wx_extra/wxListCtrl.c_src +++ b/lib/wx/api_gen/wx_extra/wxListCtrl.c_src @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2011-2013. All Rights Reserved. +%% Copyright Ericsson AB 2011-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -165,7 +165,7 @@ case ~s: { // wxListCtrl::SortItems taylormade callbackInfo* cb = new callbackInfo(); cb->port = Ecmd.port; cb->callbackID = sortCallback; - bool Result = This->SortItems(wxEListCtrlCompare, (long)cb); + bool Result = This->SortItems(wxEListCtrlCompare, (wxeIntPtr)cb); delete cb; /* Destroy the callback, see wxEPrintout::clear_cb */ diff --git a/lib/wx/api_gen/wx_gen.erl b/lib/wx/api_gen/wx_gen.erl index 0f28b3dd5e..a60a9a93d5 100644 --- a/lib/wx/api_gen/wx_gen.erl +++ b/lib/wx/api_gen/wx_gen.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2013. All Rights Reserved. +%% Copyright Ericsson AB 2008-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -743,7 +743,14 @@ parse_type2([N="wxTreeItemData"|R],Info,Opts,T) -> parse_type2(R,Info,Opts,T#type{name="wxETreeItemData",base={term,N}}); parse_type2([N="wxClientData"|R],Info,Opts,T) -> parse_type2(R,Info,Opts,T#type{name="wxeErlTerm",base={term,N}}); -parse_type2([N="wxChar"|R],Info,Opts,T) -> +parse_type2([N="wxChar",{by_ref,_}|R],Info,Opts,T = #type{mod=[const]}) -> + case get(current_class) of + "wxLocale" -> %% Special since changed between 2.8 and 3.0 + parse_type2(R,Info,Opts,T#type{name="wxeLocaleC",base=string}); + _ -> + parse_type2(R,Info,Opts,T#type{name=N,base=int,single=false}) + end; +parse_type2([N="wxChar"|R],Info,Opts,T) -> parse_type2(R,Info,Opts,T#type{name=N,base=int}); parse_type2(["wxUint32"|R],Info,Opts,T=#type{mod=Mod}) -> parse_type2(R,Info,Opts,T#type{name=int,base=int,mod=[unsigned|Mod]}); @@ -1274,6 +1281,7 @@ parse_enums([File|Files], Parsed) -> %%io:format("Parse Enums in ~s ~n", [FileName]), case xmerl_scan:file(FileName, [{space, normalize}]) of {error, enoent} -> + %% io:format("Ignore ~p~n", [FileName]), parse_enums(Files, gb_sets:add(File,Parsed)); {Doc, _} -> ES = "./compounddef/sectiondef/memberdef[@kind=\"enum\"]", diff --git a/lib/wx/api_gen/wx_gen_cpp.erl b/lib/wx/api_gen/wx_gen_cpp.erl index ea5d89be72..107d064f4a 100644 --- a/lib/wx/api_gen/wx_gen_cpp.erl +++ b/lib/wx/api_gen/wx_gen_cpp.erl @@ -71,7 +71,8 @@ gen_derived_dest_2(C=#class{name=Class, options=Opts}) -> if Derived andalso (TaylorMade =:= false) -> case lists:keysearch(ifdef,1,Opts) of - {value, {ifdef, What}} -> w("#if ~p~n",[What]); + {value, {ifdef, What}} when is_list(What)-> w("#if ~s~n",[What]); + {value, {ifdef, What}} when is_atom(What) -> w("#if ~p~n",[What]); _ -> ok end, w("class E~s : public ~s {~n",[Class,Class]), @@ -190,13 +191,14 @@ gen_funcs(Defs) -> %% w(" case WXE_REMOVE_PORT:~n", []), %% w(" { destroyMemEnv(Ecmd.port); } break;~n", []), w(" case DESTROY_OBJECT: {~n"), - w(" wxObject *This = (wxObject *) getPtr(bp,memenv);~n"), - w(" if(This) {~n"), - w(" if(recurse_level > 1) {~n"), + w(" void *This = getPtr(bp,memenv);~n"), + w(" wxeRefData *refd = getRefData(This);~n"), + w(" if(This && refd) {~n"), + w(" if(recurse_level > 1 && refd->type != 4) {~n"), w(" delayed_delete->Append(Ecmd.Save());~n"), w(" } else {~n"), - w(" ((WxeApp *) wxTheApp)->clearPtr((void *) This);~n"), - w(" delete This; }~n"), + w(" ((WxeApp *) wxTheApp)->clearPtr(This);~n"), + w(" delete_object(This, refd); }~n"), w(" } } break;~n"), w(" case WXE_REGISTER_OBJECT: {~n" " registerPid(bp, Ecmd.caller, memenv);~n" @@ -235,24 +237,27 @@ gen_funcs(Defs) -> "wxFileDataObject", "wxTextDataObject", "wxBitmapDataObject" ], - w("void WxeApp::delete_object(void *ptr, wxeRefData *refd) {~n", []), + w("bool WxeApp::delete_object(void *ptr, wxeRefData *refd) {~n", []), w(" switch(refd->type) {~n", []), - Case = fun(#class{name=Class, id=Id, abstract=IsAbs, parent=P}) when P /= "static" -> + Case = fun(C=#class{name=Class, id=Id, abstract=IsAbs, parent=P}) when P /= "static" -> UglyWorkaround = lists:member(Class, UglySkipList), + HaveVirtual = virtual_dest(C), case hd(reverse(wx_gen_erl:parents(Class))) of - root when IsAbs == false, UglyWorkaround == false -> - w(" case ~p: delete (~s *) ptr; break;~n", [Id, Class]); root when IsAbs == false, UglyWorkaround == true -> w(" case ~p: /* delete (~s *) ptr;" "These objects must be deleted by owner object */ " "break;~n", [Id, Class]); + root when IsAbs == false, HaveVirtual == true -> + w(" case ~p: delete (E~s *) ptr; return false;~n", [Id, Class]); + root when IsAbs == false, UglyWorkaround == false -> + w(" case ~p: delete (~s *) ptr; break;~n", [Id, Class]); _ -> ok end; (_) -> ok end, [Case(Class) || Class <- Defs], - w(" default: delete (wxObject *) ptr;~n", []), - w("}}~n~n", []), + w(" default: delete (wxObject *) ptr; return false;~n", []), + w(" }~n return true;~n}~n~n", []), Res. gen_class(C=#class{name=Name,methods=Ms,options=Opts}) -> @@ -267,7 +272,8 @@ gen_class(C=#class{name=Name,methods=Ms,options=Opts}) -> false -> case lists:keysearch(ifdef,1,Opts) of {value, {ifdef, What}} -> - w("#if ~p~n",[What]), + is_atom(What) andalso w("#if ~p~n",[What]), + is_list(What) andalso w("#if ~s~n",[What]), Methods = lists:flatten(Ms), MsR = [gen_method(Name,M) || M <- lists:keysort(#method.id, Methods)], @@ -401,6 +407,8 @@ declare_type(N,true,Def,#type{base=Base,single=true,name=Type,by_val=false,ref={ w(" ~s *~s=~s;~n", [Type,N,Def]); declare_type(N,true,Def,#type{single=true,name="wxArtClient"}) -> w(" wxArtClient ~s= ~s;~n", [N,Def]); +declare_type(N,true,_Def,#type{name="wxeLocaleC", single=true,base=string}) -> + w(" wxString ~s= wxEmptyString;~n", [N]); declare_type(N,true,Def,#type{single=true,base=string}) -> w(" wxString ~s= ~s;~n", [N,Def]); %% declare_type(N,true,_Def,#type{name="wxString"}) -> @@ -730,9 +738,13 @@ call_wx(_N,{constructor,_},#type{base={class,RClass}},Ps) -> false -> 0 end; false -> - case hd(reverse(wx_gen_erl:parents(RClass))) of - root -> Id; - _ -> 1 + case is_dc(RClass) of + true -> 4; + false -> + case hd(reverse(wx_gen_erl:parents(RClass))) of + root -> Id; + _ -> 1 + end end end, case virtual_dest(ClassDef) orelse (CType =/= 0) of @@ -894,6 +906,10 @@ is_window(Class) -> is_dialog(Class) -> lists:member("wxDialog", wx_gen_erl:parents(Class)). +is_dc(Class) -> + Parents = wx_gen_erl:parents(Class), + lists:member("wxDC", Parents) orelse lists:member("wxGraphicsContext", Parents). + build_return_vals(Type,Ps) -> HaveType = case Type of void -> 0; _ -> 1 end, NoOut = lists:sum([1 || #param{in=In} <- Ps, In =/= true]) + HaveType, @@ -997,6 +1013,8 @@ build_ret(Name,_,#type{base=float,single=true}) -> w(" rt.addFloat(~s);~n",[Name]); build_ret(Name,_,#type{base=double,single=true}) -> w(" rt.addFloat(~s);~n",[Name]); +build_ret(Name,_,#type{name="wxeLocaleC"}) -> + w(" rt.add(wxeLocaleC2String(~s));~n",[Name]); build_ret(Name,_,#type{base=string,single=true}) -> w(" rt.add(~s);~n",[Name]); build_ret(Name,_,#type{name="wxArrayString", single=array}) -> @@ -1090,6 +1108,7 @@ gen_macros() -> w("#include <wx/listbook.h>~n"), w("#include <wx/treebook.h>~n"), w("#include <wx/taskbar.h>~n"), + w("#include <wx/popupwin.h>~n"), w("#include <wx/html/htmlwin.h>~n"), w("#include <wx/html/htmlcell.h>~n"), w("#include <wx/filename.h>~n"), diff --git a/lib/wx/api_gen/wx_gen_erl.erl b/lib/wx/api_gen/wx_gen_erl.erl index a4b03d3fd1..5ac781b40c 100644 --- a/lib/wx/api_gen/wx_gen_erl.erl +++ b/lib/wx/api_gen/wx_gen_erl.erl @@ -44,7 +44,11 @@ gen(Defs) -> gen_unique_names(Defs), gen_event_recs(), gen_enums_ints(), - [gen_class(Class) || Class <- Defs], + Static = gen_static([C || C=#class{parent="static"} <- Defs]), + Replace = fun(C=#class{name=Name}, Dfs) -> + lists:keyreplace(Name, #class.name, Dfs, C) + end, + [gen_class(Class) || Class <- lists:foldl(Replace, Defs, Static)], gen_funcnames(). gen_class(Class) -> @@ -54,9 +58,8 @@ gen_class(Class) -> Class end. -gen_class1(C=#class{name=Name,parent="static",methods=Ms,options=_Opts}) -> +gen_static(Files) -> open_write("../src/gen/wx_misc.erl"), - put(current_class, Name), erl_copyright(), w("", []), w("%% This file is generated DO NOT EDIT~n~n", []), @@ -67,17 +70,27 @@ gen_class1(C=#class{name=Name,parent="static",methods=Ms,options=_Opts}) -> w("-module(wx_misc).~n", []), w("-include(\"wxe.hrl\").~n",[]), %% w("-compile(export_all).~n~n", []), %% XXXX remove ??? + [gen_static_exports(C) || C <- Files], + Classes = [gen_static_methods(C) || C <- Files], + close(), + Classes. + +gen_static_exports(C=#class{parent="static",methods=Ms}) -> Exp = fun(M) -> gen_export(C,M) end, ExportList = lists:usort(lists:append(lists:map(Exp,reverse(Ms)))), w("-export([~s]).~n~n", [args(fun({EF,_}) -> EF end, ",", ExportList, 60)]), + ok. +gen_static_methods(C=#class{name=Name, parent="static",methods=Ms}) -> + put(current_class, Name), Gen = fun(M) -> gen_method(Name,M) end, NewMs = lists:map(Gen,reverse(Ms)), - close(), erase(current_class), - C#class{methods=NewMs}; + C#class{methods=NewMs}. +gen_class1(C=#class{parent="static"}) -> + C; gen_class1(C=#class{name=Name,parent=Parent,methods=Ms,options=Opts}) -> case Opts of ["ignore"] -> throw(skipped); @@ -755,7 +768,7 @@ write_spec(Args, Optional, {complex, Res}, Eol) -> optional_type(Opts, Eol) -> "Option :: " ++ args(fun optional_type2/1, Eol++"\t\t | ", Opts). -optional_type2(#param{name=Name, def=Def, type=T}) -> +optional_type2(#param{name=Name, def=_Def, type=T}) -> "{" ++ erl_option_name(Name) ++ ", " ++ doc_arg_type2(T) ++ "}". %% %% Default: " ++ Def. doc_link("utils", Func) -> @@ -1357,4 +1370,3 @@ split_list(F, Keep, [M|Ms], Acc) -> split_list(_, _, [], []) -> []; split_list(_, _, [], Acc) -> [lists:reverse(Acc)]. - diff --git a/lib/wx/api_gen/wxapi.conf b/lib/wx/api_gen/wxapi.conf index 73c5af43d8..2e961cce98 100644 --- a/lib/wx/api_gen/wxapi.conf +++ b/lib/wx/api_gen/wxapi.conf @@ -31,7 +31,8 @@ %% wxALWAYS_NATIVE_DOUBLE_BUFFER, wxGAUGE_EMULATE_INDETERMINATE_MODE, - wxTR_DEFAULT_STYLE + wxTR_DEFAULT_STYLE, + wxSL_LABELS ]}. {gvars, @@ -877,6 +878,7 @@ {class, wxTextCtrl, wxControl, [], ['wxTextCtrl','~wxTextCtrl','AppendText','CanCopy','CanCut','CanPaste', 'CanRedo','CanUndo','Clear','Copy','Create','Cut','DiscardEdits', + 'ChangeValue', 'EmulateKeyPress','GetDefaultStyle','GetInsertionPoint','GetLastPosition', 'GetLineLength','GetLineText','GetNumberOfLines','GetRange','GetSelection', 'GetStringSelection','GetStyle','GetValue',%'HitTest', %no Mac @@ -1577,9 +1579,14 @@ 'wxShutdown', {'wxShell', 1}, 'wxLaunchDefaultBrowser', {'wxGetEmailAddress',0}, {'wxGetUserId',0}, {'wxGetHomeDir',0}, 'wxNewId', 'wxRegisterId', 'wxGetCurrentId', - 'wxGetOsDescription', 'wxIsPlatformLittleEndian', 'wxIsPlatform64Bit' + 'wxGetOsDescription', 'wxIsPlatformLittleEndian', 'wxIsPlatform64Bit' ]}. +{class, gdicmn, static, [], + [ {'wxDisplaySize', [{"width", [out]}, {"height", [out]}]}, 'wxSetCursor' + ]}. + + {class, wxPrintout, object, [{alias, [{wxePrintout, wxePrintout}]}], [{'wxPrintout', [{where, taylormade}]},'~wxPrintout', 'GetDC', %% 'GetPageInfo',Callback @@ -1885,3 +1892,26 @@ wxEVT_TASKBAR_LEFT_DCLICK,wxEVT_TASKBAR_RIGHT_DCLICK]}],[]}. {class, wxInitDialogEvent, wxEvent, [{event,[wxEVT_INIT_DIALOG]}], []}. + +{class, wxLocale, root, [{skip, [{'wxLocale', 5}, {'Init', 6}]}], + ['wxLocale', '~wxLocale', 'Init', + 'AddCatalog','AddCatalogLookupPathPrefix', + %%'AddLanguage','FindLanguageInfo', 'GetLanguageInfo', + 'GetCanonicalName','GetLanguage', + 'GetLanguageName','GetLocale','GetName','GetString', + 'GetHeaderValue', + 'GetSysName', + 'GetSystemEncoding','GetSystemEncodingName', + 'GetSystemLanguage', + 'IsLoaded','IsOk']}. + +{class, wxActivateEvent, wxEvent, + [{acc, [{m_active, "GetActive()"}]}, + {event, [wxEVT_ACTIVATE, wxEVT_ACTIVATE_APP, wxEVT_HIBERNATE]}], + ['GetActive']}. + +{class, wxPopupWindow, wxWindow, [{ifdef, wxUSE_POPUPWIN}], + ['wxPopupWindow', '~wxPopupWindow', 'Create', 'Position']}. + +{class, wxPopupTransientWindow, wxPopupWindow, [{ifdef, wxUSE_POPUPWIN}], + ['wxPopupTransientWindow', '~wxPopupTransientWindow', 'Popup', 'Dismiss']}. diff --git a/lib/wx/c_src/gen/wxe_derived_dest.h b/lib/wx/c_src/gen/wxe_derived_dest.h index 7e2d4524cb..0a3765a910 100644 --- a/lib/wx/c_src/gen/wxe_derived_dest.h +++ b/lib/wx/c_src/gen/wxe_derived_dest.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2008-2013. All Rights Reserved. + * Copyright Ericsson AB 2008-2014. All Rights Reserved. * * The contents of this file are subject to the Erlang Public License, * Version 1.1, (the "License"); you may not use this file except in @@ -764,3 +764,25 @@ class EwxTaskBarIcon : public wxTaskBarIcon { EwxTaskBarIcon() : wxTaskBarIcon() {}; }; +class EwxLocale : public wxLocale { + public: ~EwxLocale() {((WxeApp *)wxTheApp)->clearPtr(this);}; + EwxLocale(int language,int flags) : wxLocale(language,flags) {}; + EwxLocale() : wxLocale() {}; +}; + +#if wxUSE_POPUPWIN +class EwxPopupWindow : public wxPopupWindow { + public: ~EwxPopupWindow() {((WxeApp *)wxTheApp)->clearPtr(this);}; + EwxPopupWindow(wxWindow * parent,int flags) : wxPopupWindow(parent,flags) {}; + EwxPopupWindow() : wxPopupWindow() {}; +}; +#endif // wxUSE_POPUPWIN + +#if wxUSE_POPUPWIN +class EwxPopupTransientWindow : public wxPopupTransientWindow { + public: ~EwxPopupTransientWindow() {((WxeApp *)wxTheApp)->clearPtr(this);}; + EwxPopupTransientWindow(wxWindow * parent,int style) : wxPopupTransientWindow(parent,style) {}; + EwxPopupTransientWindow() : wxPopupTransientWindow() {}; +}; +#endif // wxUSE_POPUPWIN + diff --git a/lib/wx/c_src/gen/wxe_events.cpp b/lib/wx/c_src/gen/wxe_events.cpp index 1bd17366a2..255b36c2fa 100644 --- a/lib/wx/c_src/gen/wxe_events.cpp +++ b/lib/wx/c_src/gen/wxe_events.cpp @@ -220,84 +220,87 @@ void initEventTable() {wxEVT_STC_HOTSPOT_DCLICK, 203, "stc_hotspot_dclick"}, {wxEVT_STC_CALLTIP_CLICK, 203, "stc_calltip_click"}, {wxEVT_STC_AUTOCOMP_SELECTION, 203, "stc_autocomp_selection"}, - {wxEVT_COMMAND_TREE_BEGIN_DRAG, 208, "command_tree_begin_drag"}, - {wxEVT_COMMAND_TREE_BEGIN_RDRAG, 208, "command_tree_begin_rdrag"}, - {wxEVT_COMMAND_TREE_BEGIN_LABEL_EDIT, 208, "command_tree_begin_label_edit"}, - {wxEVT_COMMAND_TREE_END_LABEL_EDIT, 208, "command_tree_end_label_edit"}, - {wxEVT_COMMAND_TREE_DELETE_ITEM, 208, "command_tree_delete_item"}, - {wxEVT_COMMAND_TREE_GET_INFO, 208, "command_tree_get_info"}, - {wxEVT_COMMAND_TREE_SET_INFO, 208, "command_tree_set_info"}, - {wxEVT_COMMAND_TREE_ITEM_EXPANDED, 208, "command_tree_item_expanded"}, - {wxEVT_COMMAND_TREE_ITEM_EXPANDING, 208, "command_tree_item_expanding"}, - {wxEVT_COMMAND_TREE_ITEM_COLLAPSED, 208, "command_tree_item_collapsed"}, - {wxEVT_COMMAND_TREE_ITEM_COLLAPSING, 208, "command_tree_item_collapsing"}, - {wxEVT_COMMAND_TREE_SEL_CHANGED, 208, "command_tree_sel_changed"}, - {wxEVT_COMMAND_TREE_SEL_CHANGING, 208, "command_tree_sel_changing"}, - {wxEVT_COMMAND_TREE_KEY_DOWN, 208, "command_tree_key_down"}, - {wxEVT_COMMAND_TREE_ITEM_ACTIVATED, 208, "command_tree_item_activated"}, - {wxEVT_COMMAND_TREE_ITEM_RIGHT_CLICK, 208, "command_tree_item_right_click"}, - {wxEVT_COMMAND_TREE_ITEM_MIDDLE_CLICK, 208, "command_tree_item_middle_click"}, - {wxEVT_COMMAND_TREE_END_DRAG, 208, "command_tree_end_drag"}, - {wxEVT_COMMAND_TREE_STATE_IMAGE_CLICK, 208, "command_tree_state_image_click"}, - {wxEVT_COMMAND_TREE_ITEM_GETTOOLTIP, 208, "command_tree_item_gettooltip"}, - {wxEVT_COMMAND_TREE_ITEM_MENU, 208, "command_tree_item_menu"}, - {wxEVT_COMMAND_NOTEBOOK_PAGE_CHANGED, 209, "command_notebook_page_changed"}, - {wxEVT_COMMAND_NOTEBOOK_PAGE_CHANGING, 209, "command_notebook_page_changing"}, - {wxEVT_COMMAND_TEXT_COPY, 215, "command_text_copy"}, - {wxEVT_COMMAND_TEXT_CUT, 215, "command_text_cut"}, - {wxEVT_COMMAND_TEXT_PASTE, 215, "command_text_paste"}, - {wxEVT_COMMAND_SPINCTRL_UPDATED, 216, "command_spinctrl_updated"}, + {wxEVT_COMMAND_TREE_BEGIN_DRAG, 209, "command_tree_begin_drag"}, + {wxEVT_COMMAND_TREE_BEGIN_RDRAG, 209, "command_tree_begin_rdrag"}, + {wxEVT_COMMAND_TREE_BEGIN_LABEL_EDIT, 209, "command_tree_begin_label_edit"}, + {wxEVT_COMMAND_TREE_END_LABEL_EDIT, 209, "command_tree_end_label_edit"}, + {wxEVT_COMMAND_TREE_DELETE_ITEM, 209, "command_tree_delete_item"}, + {wxEVT_COMMAND_TREE_GET_INFO, 209, "command_tree_get_info"}, + {wxEVT_COMMAND_TREE_SET_INFO, 209, "command_tree_set_info"}, + {wxEVT_COMMAND_TREE_ITEM_EXPANDED, 209, "command_tree_item_expanded"}, + {wxEVT_COMMAND_TREE_ITEM_EXPANDING, 209, "command_tree_item_expanding"}, + {wxEVT_COMMAND_TREE_ITEM_COLLAPSED, 209, "command_tree_item_collapsed"}, + {wxEVT_COMMAND_TREE_ITEM_COLLAPSING, 209, "command_tree_item_collapsing"}, + {wxEVT_COMMAND_TREE_SEL_CHANGED, 209, "command_tree_sel_changed"}, + {wxEVT_COMMAND_TREE_SEL_CHANGING, 209, "command_tree_sel_changing"}, + {wxEVT_COMMAND_TREE_KEY_DOWN, 209, "command_tree_key_down"}, + {wxEVT_COMMAND_TREE_ITEM_ACTIVATED, 209, "command_tree_item_activated"}, + {wxEVT_COMMAND_TREE_ITEM_RIGHT_CLICK, 209, "command_tree_item_right_click"}, + {wxEVT_COMMAND_TREE_ITEM_MIDDLE_CLICK, 209, "command_tree_item_middle_click"}, + {wxEVT_COMMAND_TREE_END_DRAG, 209, "command_tree_end_drag"}, + {wxEVT_COMMAND_TREE_STATE_IMAGE_CLICK, 209, "command_tree_state_image_click"}, + {wxEVT_COMMAND_TREE_ITEM_GETTOOLTIP, 209, "command_tree_item_gettooltip"}, + {wxEVT_COMMAND_TREE_ITEM_MENU, 209, "command_tree_item_menu"}, + {wxEVT_COMMAND_NOTEBOOK_PAGE_CHANGED, 210, "command_notebook_page_changed"}, + {wxEVT_COMMAND_NOTEBOOK_PAGE_CHANGING, 210, "command_notebook_page_changing"}, + {wxEVT_COMMAND_TEXT_COPY, 216, "command_text_copy"}, + {wxEVT_COMMAND_TEXT_CUT, 216, "command_text_cut"}, + {wxEVT_COMMAND_TEXT_PASTE, 216, "command_text_paste"}, + {wxEVT_COMMAND_SPINCTRL_UPDATED, 217, "command_spinctrl_updated"}, {wxEVT_SCROLL_LINEUP + wxEVT_USER_FIRST, 165, "spin_up"}, {wxEVT_SCROLL_LINEDOWN + wxEVT_USER_FIRST, 165, "spin_down"}, {wxEVT_SCROLL_THUMBTRACK + wxEVT_USER_FIRST, 165, "spin"}, - {wxEVT_COMMAND_SPLITTER_SASH_POS_CHANGED, 218, "command_splitter_sash_pos_changed"}, - {wxEVT_COMMAND_SPLITTER_SASH_POS_CHANGING, 218, "command_splitter_sash_pos_changing"}, - {wxEVT_COMMAND_SPLITTER_DOUBLECLICKED, 218, "command_splitter_doubleclicked"}, - {wxEVT_COMMAND_SPLITTER_UNSPLIT, 218, "command_splitter_unsplit"}, - {wxEVT_COMMAND_HTML_LINK_CLICKED, 220, "command_html_link_clicked"}, - {wxEVT_COMMAND_AUINOTEBOOK_PAGE_CLOSE, 223, "command_auinotebook_page_close"}, - {wxEVT_COMMAND_AUINOTEBOOK_PAGE_CHANGED, 223, "command_auinotebook_page_changed"}, - {wxEVT_COMMAND_AUINOTEBOOK_PAGE_CHANGING, 223, "command_auinotebook_page_changing"}, - {wxEVT_COMMAND_AUINOTEBOOK_BUTTON, 223, "command_auinotebook_button"}, - {wxEVT_COMMAND_AUINOTEBOOK_BEGIN_DRAG, 223, "command_auinotebook_begin_drag"}, - {wxEVT_COMMAND_AUINOTEBOOK_END_DRAG, 223, "command_auinotebook_end_drag"}, - {wxEVT_COMMAND_AUINOTEBOOK_DRAG_MOTION, 223, "command_auinotebook_drag_motion"}, - {wxEVT_COMMAND_AUINOTEBOOK_ALLOW_DND, 223, "command_auinotebook_allow_dnd"}, + {wxEVT_COMMAND_SPLITTER_SASH_POS_CHANGED, 219, "command_splitter_sash_pos_changed"}, + {wxEVT_COMMAND_SPLITTER_SASH_POS_CHANGING, 219, "command_splitter_sash_pos_changing"}, + {wxEVT_COMMAND_SPLITTER_DOUBLECLICKED, 219, "command_splitter_doubleclicked"}, + {wxEVT_COMMAND_SPLITTER_UNSPLIT, 219, "command_splitter_unsplit"}, + {wxEVT_COMMAND_HTML_LINK_CLICKED, 221, "command_html_link_clicked"}, + {wxEVT_COMMAND_AUINOTEBOOK_PAGE_CLOSE, 224, "command_auinotebook_page_close"}, + {wxEVT_COMMAND_AUINOTEBOOK_PAGE_CHANGED, 224, "command_auinotebook_page_changed"}, + {wxEVT_COMMAND_AUINOTEBOOK_PAGE_CHANGING, 224, "command_auinotebook_page_changing"}, + {wxEVT_COMMAND_AUINOTEBOOK_BUTTON, 224, "command_auinotebook_button"}, + {wxEVT_COMMAND_AUINOTEBOOK_BEGIN_DRAG, 224, "command_auinotebook_begin_drag"}, + {wxEVT_COMMAND_AUINOTEBOOK_END_DRAG, 224, "command_auinotebook_end_drag"}, + {wxEVT_COMMAND_AUINOTEBOOK_DRAG_MOTION, 224, "command_auinotebook_drag_motion"}, + {wxEVT_COMMAND_AUINOTEBOOK_ALLOW_DND, 224, "command_auinotebook_allow_dnd"}, #if wxCHECK_VERSION(2,8,5) - {wxEVT_COMMAND_AUINOTEBOOK_TAB_MIDDLE_DOWN, 223, "command_auinotebook_tab_middle_down"}, + {wxEVT_COMMAND_AUINOTEBOOK_TAB_MIDDLE_DOWN, 224, "command_auinotebook_tab_middle_down"}, #endif #if wxCHECK_VERSION(2,8,5) - {wxEVT_COMMAND_AUINOTEBOOK_TAB_MIDDLE_UP, 223, "command_auinotebook_tab_middle_up"}, + {wxEVT_COMMAND_AUINOTEBOOK_TAB_MIDDLE_UP, 224, "command_auinotebook_tab_middle_up"}, #endif #if wxCHECK_VERSION(2,8,5) - {wxEVT_COMMAND_AUINOTEBOOK_TAB_RIGHT_DOWN, 223, "command_auinotebook_tab_right_down"}, + {wxEVT_COMMAND_AUINOTEBOOK_TAB_RIGHT_DOWN, 224, "command_auinotebook_tab_right_down"}, #endif #if wxCHECK_VERSION(2,8,5) - {wxEVT_COMMAND_AUINOTEBOOK_TAB_RIGHT_UP, 223, "command_auinotebook_tab_right_up"}, + {wxEVT_COMMAND_AUINOTEBOOK_TAB_RIGHT_UP, 224, "command_auinotebook_tab_right_up"}, #endif #if wxCHECK_VERSION(2,8,5) - {wxEVT_COMMAND_AUINOTEBOOK_PAGE_CLOSED, 223, "command_auinotebook_page_closed"}, + {wxEVT_COMMAND_AUINOTEBOOK_PAGE_CLOSED, 224, "command_auinotebook_page_closed"}, #endif #if wxCHECK_VERSION(2,8,5) - {wxEVT_COMMAND_AUINOTEBOOK_DRAG_DONE, 223, "command_auinotebook_drag_done"}, + {wxEVT_COMMAND_AUINOTEBOOK_DRAG_DONE, 224, "command_auinotebook_drag_done"}, #endif #if wxCHECK_VERSION(2,8,5) - {wxEVT_COMMAND_AUINOTEBOOK_BG_DCLICK, 223, "command_auinotebook_bg_dclick"}, + {wxEVT_COMMAND_AUINOTEBOOK_BG_DCLICK, 224, "command_auinotebook_bg_dclick"}, #endif - {wxEVT_AUI_PANE_BUTTON, 224, "aui_pane_button"}, - {wxEVT_AUI_PANE_CLOSE, 224, "aui_pane_close"}, - {wxEVT_AUI_PANE_MAXIMIZE, 224, "aui_pane_maximize"}, - {wxEVT_AUI_PANE_RESTORE, 224, "aui_pane_restore"}, - {wxEVT_AUI_RENDER, 224, "aui_render"}, - {wxEVT_AUI_FIND_MANAGER, 224, "aui_find_manager"}, - {wxEVT_TASKBAR_MOVE, 227, "taskbar_move"}, - {wxEVT_TASKBAR_LEFT_DOWN, 227, "taskbar_left_down"}, - {wxEVT_TASKBAR_LEFT_UP, 227, "taskbar_left_up"}, - {wxEVT_TASKBAR_RIGHT_DOWN, 227, "taskbar_right_down"}, - {wxEVT_TASKBAR_RIGHT_UP, 227, "taskbar_right_up"}, - {wxEVT_TASKBAR_LEFT_DCLICK, 227, "taskbar_left_dclick"}, - {wxEVT_TASKBAR_RIGHT_DCLICK, 227, "taskbar_right_dclick"}, - {wxEVT_INIT_DIALOG, 228, "init_dialog"}, + {wxEVT_AUI_PANE_BUTTON, 225, "aui_pane_button"}, + {wxEVT_AUI_PANE_CLOSE, 225, "aui_pane_close"}, + {wxEVT_AUI_PANE_MAXIMIZE, 225, "aui_pane_maximize"}, + {wxEVT_AUI_PANE_RESTORE, 225, "aui_pane_restore"}, + {wxEVT_AUI_RENDER, 225, "aui_render"}, + {wxEVT_AUI_FIND_MANAGER, 225, "aui_find_manager"}, + {wxEVT_TASKBAR_MOVE, 228, "taskbar_move"}, + {wxEVT_TASKBAR_LEFT_DOWN, 228, "taskbar_left_down"}, + {wxEVT_TASKBAR_LEFT_UP, 228, "taskbar_left_up"}, + {wxEVT_TASKBAR_RIGHT_DOWN, 228, "taskbar_right_down"}, + {wxEVT_TASKBAR_RIGHT_UP, 228, "taskbar_right_up"}, + {wxEVT_TASKBAR_LEFT_DCLICK, 228, "taskbar_left_dclick"}, + {wxEVT_TASKBAR_RIGHT_DCLICK, 228, "taskbar_right_dclick"}, + {wxEVT_INIT_DIALOG, 229, "init_dialog"}, + {wxEVT_ACTIVATE, 231, "activate"}, + {wxEVT_ACTIVATE_APP, 231, "activate_app"}, + {wxEVT_HIBERNATE, 231, "hibernate"}, {-1, 0, } }; for(int i=0; event_types[i].ev_type != -1; i++) { @@ -719,7 +722,7 @@ case 203: {// wxStyledTextEvent rt.addTupleCount(22); break; } -case 208: {// wxTreeEvent +case 209: {// wxTreeEvent wxTreeEvent * ev = (wxTreeEvent *) event; evClass = (char*)"wxTreeEvent"; rt.addAtom((char*)"wxTree"); @@ -730,21 +733,21 @@ case 208: {// wxTreeEvent rt.addTupleCount(5); break; } -case 209: {// wxNotebookEvent +case 210: {// wxNotebookEvent evClass = (char*)"wxNotebookEvent"; rt.addAtom((char*)"wxNotebook"); rt.addAtom(Etype->eName); rt.addTupleCount(2); break; } -case 215: {// wxClipboardTextEvent +case 216: {// wxClipboardTextEvent evClass = (char*)"wxClipboardTextEvent"; rt.addAtom((char*)"wxClipboardText"); rt.addAtom(Etype->eName); rt.addTupleCount(2); break; } -case 216: {// wxSpinEvent +case 217: {// wxSpinEvent wxSpinEvent * ev = (wxSpinEvent *) event; evClass = (char*)"wxSpinEvent"; rt.addAtom((char*)"wxSpin"); @@ -753,14 +756,14 @@ case 216: {// wxSpinEvent rt.addTupleCount(3); break; } -case 218: {// wxSplitterEvent +case 219: {// wxSplitterEvent evClass = (char*)"wxSplitterEvent"; rt.addAtom((char*)"wxSplitter"); rt.addAtom(Etype->eName); rt.addTupleCount(2); break; } -case 220: {// wxHtmlLinkEvent +case 221: {// wxHtmlLinkEvent wxHtmlLinkEvent * ev = (wxHtmlLinkEvent *) event; evClass = (char*)"wxHtmlLinkEvent"; rt.addAtom((char*)"wxHtmlLink"); @@ -769,7 +772,7 @@ case 220: {// wxHtmlLinkEvent rt.addTupleCount(3); break; } -case 223: {// wxAuiNotebookEvent +case 224: {// wxAuiNotebookEvent wxAuiNotebookEvent * ev = (wxAuiNotebookEvent *) event; wxAuiNotebook * GetDragSource = ev->GetDragSource(); evClass = (char*)"wxAuiNotebookEvent"; @@ -781,7 +784,7 @@ case 223: {// wxAuiNotebookEvent rt.addTupleCount(5); break; } -case 224: {// wxAuiManagerEvent +case 225: {// wxAuiManagerEvent wxAuiManagerEvent * ev = (wxAuiManagerEvent *) event; wxAuiManager * GetManager = ev->GetManager(); wxAuiPaneInfo * GetPane = ev->GetPane(); @@ -798,20 +801,29 @@ case 224: {// wxAuiManagerEvent rt.addTupleCount(8); break; } -case 227: {// wxTaskBarIconEvent +case 228: {// wxTaskBarIconEvent evClass = (char*)"wxTaskBarIconEvent"; rt.addAtom((char*)"wxTaskBarIcon"); rt.addAtom(Etype->eName); rt.addTupleCount(2); break; } -case 228: {// wxInitDialogEvent +case 229: {// wxInitDialogEvent evClass = (char*)"wxInitDialogEvent"; rt.addAtom((char*)"wxInitDialog"); rt.addAtom(Etype->eName); rt.addTupleCount(2); break; } +case 231: {// wxActivateEvent + wxActivateEvent * ev = (wxActivateEvent *) event; + evClass = (char*)"wxActivateEvent"; + rt.addAtom((char*)"wxActivate"); + rt.addAtom(Etype->eName); + rt.addBool(ev->GetActive()); + rt.addTupleCount(3); + break; +} } rt.addTupleCount(5); diff --git a/lib/wx/c_src/gen/wxe_funcs.cpp b/lib/wx/c_src/gen/wxe_funcs.cpp index 3f5cb4c0f5..91ce5d810c 100644 --- a/lib/wx/c_src/gen/wxe_funcs.cpp +++ b/lib/wx/c_src/gen/wxe_funcs.cpp @@ -45,13 +45,14 @@ void WxeApp::wxe_dispatch(wxeCommand& Ecmd) switch (Ecmd.op) { case DESTROY_OBJECT: { - wxObject *This = (wxObject *) getPtr(bp,memenv); - if(This) { - if(recurse_level > 1) { + void *This = getPtr(bp,memenv); + wxeRefData *refd = getRefData(This); + if(This && refd) { + if(recurse_level > 1 && refd->type != 4) { delayed_delete->Append(Ecmd.Save()); } else { - ((WxeApp *) wxTheApp)->clearPtr((void *) This); - delete This; } + ((WxeApp *) wxTheApp)->clearPtr(This); + delete_object(This, refd); } } } break; case WXE_REGISTER_OBJECT: { registerPid(bp, Ecmd.caller, memenv); @@ -5843,26 +5844,26 @@ case wxMirrorDC_new: { // wxMirrorDC::wxMirrorDC wxDC *dc = (wxDC *) getPtr(bp,memenv); bp += 4; bool * mirror = (bool *) bp; bp += 4; wxMirrorDC * Result = new EwxMirrorDC(*dc,*mirror); - newPtr((void *) Result, 1, memenv); + newPtr((void *) Result, 4, memenv); rt.addRef(getRef((void *)Result,memenv), "wxMirrorDC"); break; } case wxScreenDC_new: { // wxScreenDC::wxScreenDC wxScreenDC * Result = new EwxScreenDC(); - newPtr((void *) Result, 1, memenv); + newPtr((void *) Result, 4, memenv); rt.addRef(getRef((void *)Result,memenv), "wxScreenDC"); break; } case wxPostScriptDC_new_0: { // wxPostScriptDC::wxPostScriptDC wxPostScriptDC * Result = new EwxPostScriptDC(); - newPtr((void *) Result, 1, memenv); + newPtr((void *) Result, 4, memenv); rt.addRef(getRef((void *)Result,memenv), "wxPostScriptDC"); break; } case wxPostScriptDC_new_1: { // wxPostScriptDC::wxPostScriptDC wxPrintData *printData = (wxPrintData *) getPtr(bp,memenv); bp += 4; wxPostScriptDC * Result = new EwxPostScriptDC(*printData); - newPtr((void *) Result, 1, memenv); + newPtr((void *) Result, 4, memenv); rt.addRef(getRef((void *)Result,memenv), "wxPostScriptDC"); break; } @@ -5883,7 +5884,7 @@ case wxPostScriptDC_GetResolution: { // wxPostScriptDC::GetResolution #if !wxCHECK_VERSION(2,9,0) case wxWindowDC_new_0: { // wxWindowDC::wxWindowDC wxWindowDC * Result = new EwxWindowDC(); - newPtr((void *) Result, 1, memenv); + newPtr((void *) Result, 4, memenv); rt.addRef(getRef((void *)Result,memenv), "wxWindowDC"); break; } @@ -5891,14 +5892,14 @@ case wxWindowDC_new_0: { // wxWindowDC::wxWindowDC case wxWindowDC_new_1: { // wxWindowDC::wxWindowDC wxWindow *win = (wxWindow *) getPtr(bp,memenv); bp += 4; wxWindowDC * Result = new EwxWindowDC(win); - newPtr((void *) Result, 1, memenv); + newPtr((void *) Result, 4, memenv); rt.addRef(getRef((void *)Result,memenv), "wxWindowDC"); break; } #if !wxCHECK_VERSION(2,9,0) case wxClientDC_new_0: { // wxClientDC::wxClientDC wxClientDC * Result = new EwxClientDC(); - newPtr((void *) Result, 1, memenv); + newPtr((void *) Result, 4, memenv); rt.addRef(getRef((void *)Result,memenv), "wxClientDC"); break; } @@ -5906,14 +5907,14 @@ case wxClientDC_new_0: { // wxClientDC::wxClientDC case wxClientDC_new_1: { // wxClientDC::wxClientDC wxWindow *win = (wxWindow *) getPtr(bp,memenv); bp += 4; wxClientDC * Result = new EwxClientDC(win); - newPtr((void *) Result, 1, memenv); + newPtr((void *) Result, 4, memenv); rt.addRef(getRef((void *)Result,memenv), "wxClientDC"); break; } #if !wxCHECK_VERSION(2,9,0) case wxPaintDC_new_0: { // wxPaintDC::wxPaintDC wxPaintDC * Result = new EwxPaintDC(); - newPtr((void *) Result, 1, memenv); + newPtr((void *) Result, 4, memenv); rt.addRef(getRef((void *)Result,memenv), "wxPaintDC"); break; } @@ -5921,27 +5922,27 @@ case wxPaintDC_new_0: { // wxPaintDC::wxPaintDC case wxPaintDC_new_1: { // wxPaintDC::wxPaintDC wxWindow *win = (wxWindow *) getPtr(bp,memenv); bp += 4; wxPaintDC * Result = new EwxPaintDC(win); - newPtr((void *) Result, 1, memenv); + newPtr((void *) Result, 4, memenv); rt.addRef(getRef((void *)Result,memenv), "wxPaintDC"); break; } case wxMemoryDC_new_1_0: { // wxMemoryDC::wxMemoryDC wxBitmap *bitmap = (wxBitmap *) getPtr(bp,memenv); bp += 4; wxMemoryDC * Result = new EwxMemoryDC(*bitmap); - newPtr((void *) Result, 1, memenv); + newPtr((void *) Result, 4, memenv); rt.addRef(getRef((void *)Result,memenv), "wxMemoryDC"); break; } case wxMemoryDC_new_1_1: { // wxMemoryDC::wxMemoryDC wxDC * dc = (wxDC *) getPtr(bp,memenv); bp += 4; wxMemoryDC * Result = new EwxMemoryDC(dc); - newPtr((void *) Result, 1, memenv); + newPtr((void *) Result, 4, memenv); rt.addRef(getRef((void *)Result,memenv), "wxMemoryDC"); break; } case wxMemoryDC_new_0: { // wxMemoryDC::wxMemoryDC wxMemoryDC * Result = new EwxMemoryDC(); - newPtr((void *) Result, 1, memenv); + newPtr((void *) Result, 4, memenv); rt.addRef(getRef((void *)Result,memenv), "wxMemoryDC"); break; } @@ -5961,7 +5962,7 @@ case wxMemoryDC_SelectObjectAsSource: { // wxMemoryDC::SelectObjectAsSource } case wxBufferedDC_new_0: { // wxBufferedDC::wxBufferedDC wxBufferedDC * Result = new EwxBufferedDC(); - newPtr((void *) Result, 1, memenv); + newPtr((void *) Result, 4, memenv); rt.addRef(getRef((void *)Result,memenv), "wxBufferedDC"); break; } @@ -5979,7 +5980,7 @@ buffer = (wxBitmap *) getPtr(bp,memenv); bp += 4; } break; }}; wxBufferedDC * Result = new EwxBufferedDC(dc,*buffer,style); - newPtr((void *) Result, 1, memenv); + newPtr((void *) Result, 4, memenv); rt.addRef(getRef((void *)Result,memenv), "wxBufferedDC"); break; } @@ -5996,7 +5997,7 @@ case wxBufferedDC_new_3: { // wxBufferedDC::wxBufferedDC } break; }}; wxBufferedDC * Result = new EwxBufferedDC(dc,area,style); - newPtr((void *) Result, 1, memenv); + newPtr((void *) Result, 4, memenv); rt.addRef(getRef((void *)Result,memenv), "wxBufferedDC"); break; } @@ -6043,7 +6044,7 @@ case wxBufferedPaintDC_new_3: { // wxBufferedPaintDC::wxBufferedPaintDC } break; }}; wxBufferedPaintDC * Result = new EwxBufferedPaintDC(window,*buffer,style); - newPtr((void *) Result, 1, memenv); + newPtr((void *) Result, 4, memenv); rt.addRef(getRef((void *)Result,memenv), "wxBufferedPaintDC"); break; } @@ -6057,7 +6058,7 @@ case wxBufferedPaintDC_new_2: { // wxBufferedPaintDC::wxBufferedPaintDC } break; }}; wxBufferedPaintDC * Result = new EwxBufferedPaintDC(window,style); - newPtr((void *) Result, 1, memenv); + newPtr((void *) Result, 4, memenv); rt.addRef(getRef((void *)Result,memenv), "wxBufferedPaintDC"); break; } @@ -15875,7 +15876,7 @@ case wxListCtrl_SortItems: { // wxListCtrl::SortItems taylormade callbackInfo* cb = new callbackInfo(); cb->port = Ecmd.port; cb->callbackID = sortCallback; - bool Result = This->SortItems(wxEListCtrlCompare, (long)cb); + bool Result = This->SortItems(wxEListCtrlCompare, (wxeIntPtr)cb); delete cb; /* Destroy the callback, see wxEPrintout::clear_cb */ @@ -16774,6 +16775,15 @@ case wxTextCtrl_DiscardEdits: { // wxTextCtrl::DiscardEdits This->DiscardEdits(); break; } +case wxTextCtrl_ChangeValue: { // wxTextCtrl::ChangeValue + wxTextCtrl *This = (wxTextCtrl *) getPtr(bp,memenv); bp += 4; + int * valueLen = (int *) bp; bp += 4; + wxString value = wxString(bp, wxConvUTF8); + bp += *valueLen+((8-((0+ *valueLen) & 7)) & 7); + if(!This) throw wxe_badarg(0); + This->ChangeValue(value); + break; +} case wxTextCtrl_EmulateKeyPress: { // wxTextCtrl::EmulateKeyPress wxTextCtrl *This = (wxTextCtrl *) getPtr(bp,memenv); bp += 4; wxKeyEvent *event = (wxKeyEvent *) getPtr(bp,memenv); bp += 4; @@ -26730,6 +26740,20 @@ case utils_wxIsPlatform64Bit: { // utils::wxIsPlatform64Bit rt.addBool(Result); break; } +case gdicmn_wxDisplaySize: { // gdicmn::wxDisplaySize + int width; + int height; + ::wxDisplaySize(&width,&height); + rt.addInt(width); + rt.addInt(height); + rt.addTupleCount(2); + break; +} +case gdicmn_wxSetCursor: { // gdicmn::wxSetCursor + wxCursor *cursor = (wxCursor *) getPtr(bp,memenv); bp += 4; + ::wxSetCursor(*cursor); + break; +} case wxPrintout_new: { // wxPrintout::wxPrintout taylormade int onPreparePrinting=0,onBeginPrinting=0,onEndPrinting=0,onBeginDocument=0, @@ -30442,7 +30466,7 @@ case wxNotebookEvent_SetSelection: { // wxNotebookEvent::SetSelection } case wxFileDataObject_new: { // wxFileDataObject::wxFileDataObject wxFileDataObject * Result = new wxFileDataObject(); - newPtr((void *) Result, 211, memenv); + newPtr((void *) Result, 212, memenv); rt.addRef(getRef((void *)Result,memenv), "wxFileDataObject"); break; } @@ -30478,7 +30502,7 @@ case wxTextDataObject_new: { // wxTextDataObject::wxTextDataObject } break; }}; wxTextDataObject * Result = new wxTextDataObject(text); - newPtr((void *) Result, 212, memenv); + newPtr((void *) Result, 213, memenv); rt.addRef(getRef((void *)Result,memenv), "wxTextDataObject"); break; } @@ -30514,7 +30538,7 @@ case wxTextDataObject_destroy: { // wxTextDataObject::destroy case wxBitmapDataObject_new_1_1: { // wxBitmapDataObject::wxBitmapDataObject wxBitmap *bitmap = (wxBitmap *) getPtr(bp,memenv); bp += 4; wxBitmapDataObject * Result = new wxBitmapDataObject(*bitmap); - newPtr((void *) Result, 213, memenv); + newPtr((void *) Result, 214, memenv); rt.addRef(getRef((void *)Result,memenv), "wxBitmapDataObject"); break; } @@ -30526,7 +30550,7 @@ bitmap = (wxBitmap *) getPtr(bp,memenv); bp += 4; } break; }}; wxBitmapDataObject * Result = new wxBitmapDataObject(*bitmap); - newPtr((void *) Result, 213, memenv); + newPtr((void *) Result, 214, memenv); rt.addRef(getRef((void *)Result,memenv), "wxBitmapDataObject"); break; } @@ -31359,7 +31383,7 @@ case wxAuiManagerEvent_CanVeto: { // wxAuiManagerEvent::CanVeto } case wxLogNull_new: { // wxLogNull::wxLogNull wxLogNull * Result = new wxLogNull(); - newPtr((void *) Result, 225, memenv); + newPtr((void *) Result, 226, memenv); rt.addRef(getRef((void *)Result,memenv), "wxLogNull"); break; } @@ -31406,6 +31430,309 @@ case wxTaskBarIcon_SetIcon: { // wxTaskBarIcon::SetIcon rt.addBool(Result); break; } +case wxLocale_new_0: { // wxLocale::wxLocale + wxLocale * Result = new EwxLocale(); + newPtr((void *) Result, 230, memenv); + rt.addRef(getRef((void *)Result,memenv), "wxLocale"); + break; +} +case wxLocale_new_2: { // wxLocale::wxLocale + int flags=wxLOCALE_LOAD_DEFAULT|wxLOCALE_CONV_ENCODING; + int * language = (int *) bp; bp += 4; + bp += 4; /* Align */ + while( * (int*) bp) { switch (* (int*) bp) { + case 1: {bp += 4; + flags = (int)*(int *) bp; bp += 4; + } break; + }}; + wxLocale * Result = new EwxLocale(*language,flags); + newPtr((void *) Result, 230, memenv); + rt.addRef(getRef((void *)Result,memenv), "wxLocale"); + break; +} +case wxLocale_destruct: { // wxLocale::~wxLocale + wxLocale *This = (wxLocale *) getPtr(bp,memenv); bp += 4; + if(This) { ((WxeApp *) wxTheApp)->clearPtr((void *) This); + delete This;} + break; +} +case wxLocale_Init: { // wxLocale::Init + int language=wxLANGUAGE_DEFAULT; + int flags=wxLOCALE_LOAD_DEFAULT|wxLOCALE_CONV_ENCODING; + wxLocale *This = (wxLocale *) getPtr(bp,memenv); bp += 4; + bp += 4; /* Align */ + while( * (int*) bp) { switch (* (int*) bp) { + case 1: {bp += 4; + language = (int)*(int *) bp; bp += 4; + } break; + case 2: {bp += 4; + flags = (int)*(int *) bp; bp += 4; + } break; + }}; + if(!This) throw wxe_badarg(0); + bool Result = This->Init(language,flags); + rt.addBool(Result); + break; +} +case wxLocale_AddCatalog_1: { // wxLocale::AddCatalog + wxLocale *This = (wxLocale *) getPtr(bp,memenv); bp += 4; + int * szDomainLen = (int *) bp; bp += 4; + wxString szDomain = wxString(bp, wxConvUTF8); + bp += *szDomainLen+((8-((0+ *szDomainLen) & 7)) & 7); + if(!This) throw wxe_badarg(0); + bool Result = This->AddCatalog(szDomain); + rt.addBool(Result); + break; +} +case wxLocale_AddCatalog_3: { // wxLocale::AddCatalog + wxLocale *This = (wxLocale *) getPtr(bp,memenv); bp += 4; + int * szDomainLen = (int *) bp; bp += 4; + wxString szDomain = wxString(bp, wxConvUTF8); + bp += *szDomainLen+((8-((0+ *szDomainLen) & 7)) & 7); + wxLanguage msgIdLanguage = *(wxLanguage *) bp; bp += 4;; + int * msgIdCharsetLen = (int *) bp; bp += 4; + wxString msgIdCharset = wxString(bp, wxConvUTF8); + bp += *msgIdCharsetLen+((8-((0+ *msgIdCharsetLen) & 7)) & 7); + if(!This) throw wxe_badarg(0); + bool Result = This->AddCatalog(szDomain,msgIdLanguage,msgIdCharset); + rt.addBool(Result); + break; +} +case wxLocale_AddCatalogLookupPathPrefix: { // wxLocale::AddCatalogLookupPathPrefix + int * prefixLen = (int *) bp; bp += 4; + wxString prefix = wxString(bp, wxConvUTF8); + bp += *prefixLen+((8-((4+ *prefixLen) & 7)) & 7); + wxLocale::AddCatalogLookupPathPrefix(prefix); + break; +} +case wxLocale_GetCanonicalName: { // wxLocale::GetCanonicalName + wxLocale *This = (wxLocale *) getPtr(bp,memenv); bp += 4; + if(!This) throw wxe_badarg(0); + wxString Result = This->GetCanonicalName(); + rt.add(Result); + break; +} +case wxLocale_GetLanguage: { // wxLocale::GetLanguage + wxLocale *This = (wxLocale *) getPtr(bp,memenv); bp += 4; + if(!This) throw wxe_badarg(0); + int Result = This->GetLanguage(); + rt.addInt(Result); + break; +} +case wxLocale_GetLanguageName: { // wxLocale::GetLanguageName + int * lang = (int *) bp; bp += 4; + wxString Result = wxLocale::GetLanguageName(*lang); + rt.add(Result); + break; +} +case wxLocale_GetLocale: { // wxLocale::GetLocale + wxLocale *This = (wxLocale *) getPtr(bp,memenv); bp += 4; + if(!This) throw wxe_badarg(0); + const wxeLocaleC Result = This->GetLocale(); + rt.add(wxeLocaleC2String(Result)); + break; +} +case wxLocale_GetName: { // wxLocale::GetName + wxLocale *This = (wxLocale *) getPtr(bp,memenv); bp += 4; + if(!This) throw wxe_badarg(0); + const wxString * Result = &This->GetName(); + rt.add(Result); + break; +} +case wxLocale_GetString_2: { // wxLocale::GetString + wxString szDomain= wxEmptyString; + wxLocale *This = (wxLocale *) getPtr(bp,memenv); bp += 4; + int * szOrigStringLen = (int *) bp; bp += 4; + wxString szOrigString = wxString(bp, wxConvUTF8); + bp += *szOrigStringLen+((8-((0+ *szOrigStringLen) & 7)) & 7); + while( * (int*) bp) { switch (* (int*) bp) { + case 1: {bp += 4; + int * szDomainLen = (int *) bp; bp += 4; + szDomain = wxString(bp, wxConvUTF8); + bp += *szDomainLen+((8-((0+ *szDomainLen) & 7)) & 7); + } break; + }}; + if(!This) throw wxe_badarg(0); + const wxeLocaleC Result = This->GetString(szOrigString,szDomain); + rt.add(wxeLocaleC2String(Result)); + break; +} +case wxLocale_GetString_4: { // wxLocale::GetString + wxString szDomain= wxEmptyString; + wxLocale *This = (wxLocale *) getPtr(bp,memenv); bp += 4; + int * szOrigStringLen = (int *) bp; bp += 4; + wxString szOrigString = wxString(bp, wxConvUTF8); + bp += *szOrigStringLen+((8-((0+ *szOrigStringLen) & 7)) & 7); + int * szOrigString2Len = (int *) bp; bp += 4; + wxString szOrigString2 = wxString(bp, wxConvUTF8); + bp += *szOrigString2Len+((8-((4+ *szOrigString2Len) & 7)) & 7); + int * n = (int *) bp; bp += 4; + bp += 4; /* Align */ + while( * (int*) bp) { switch (* (int*) bp) { + case 1: {bp += 4; + int * szDomainLen = (int *) bp; bp += 4; + szDomain = wxString(bp, wxConvUTF8); + bp += *szDomainLen+((8-((0+ *szDomainLen) & 7)) & 7); + } break; + }}; + if(!This) throw wxe_badarg(0); + const wxeLocaleC Result = This->GetString(szOrigString,szOrigString2,*n,szDomain); + rt.add(wxeLocaleC2String(Result)); + break; +} +case wxLocale_GetHeaderValue: { // wxLocale::GetHeaderValue + wxString szDomain= wxEmptyString; + wxLocale *This = (wxLocale *) getPtr(bp,memenv); bp += 4; + int * szHeaderLen = (int *) bp; bp += 4; + wxString szHeader = wxString(bp, wxConvUTF8); + bp += *szHeaderLen+((8-((0+ *szHeaderLen) & 7)) & 7); + while( * (int*) bp) { switch (* (int*) bp) { + case 1: {bp += 4; + int * szDomainLen = (int *) bp; bp += 4; + szDomain = wxString(bp, wxConvUTF8); + bp += *szDomainLen+((8-((0+ *szDomainLen) & 7)) & 7); + } break; + }}; + if(!This) throw wxe_badarg(0); + wxString Result = This->GetHeaderValue(szHeader,szDomain); + rt.add(Result); + break; +} +case wxLocale_GetSysName: { // wxLocale::GetSysName + wxLocale *This = (wxLocale *) getPtr(bp,memenv); bp += 4; + if(!This) throw wxe_badarg(0); + wxString Result = This->GetSysName(); + rt.add(Result); + break; +} +case wxLocale_GetSystemEncoding: { // wxLocale::GetSystemEncoding + int Result = wxLocale::GetSystemEncoding(); + rt.addInt(Result); + break; +} +case wxLocale_GetSystemEncodingName: { // wxLocale::GetSystemEncodingName + wxString Result = wxLocale::GetSystemEncodingName(); + rt.add(Result); + break; +} +case wxLocale_GetSystemLanguage: { // wxLocale::GetSystemLanguage + int Result = wxLocale::GetSystemLanguage(); + rt.addInt(Result); + break; +} +case wxLocale_IsLoaded: { // wxLocale::IsLoaded + wxLocale *This = (wxLocale *) getPtr(bp,memenv); bp += 4; + int * szDomainLen = (int *) bp; bp += 4; + wxString szDomain = wxString(bp, wxConvUTF8); + bp += *szDomainLen+((8-((0+ *szDomainLen) & 7)) & 7); + if(!This) throw wxe_badarg(0); + bool Result = This->IsLoaded(szDomain); + rt.addBool(Result); + break; +} +case wxLocale_IsOk: { // wxLocale::IsOk + wxLocale *This = (wxLocale *) getPtr(bp,memenv); bp += 4; + if(!This) throw wxe_badarg(0); + bool Result = This->IsOk(); + rt.addBool(Result); + break; +} +case wxActivateEvent_GetActive: { // wxActivateEvent::GetActive + wxActivateEvent *This = (wxActivateEvent *) getPtr(bp,memenv); bp += 4; + if(!This) throw wxe_badarg(0); + bool Result = This->GetActive(); + rt.addBool(Result); + break; +} +#if wxUSE_POPUPWIN +case wxPopupWindow_new_2: { // wxPopupWindow::wxPopupWindow + int flags=wxBORDER_NONE; + wxWindow *parent = (wxWindow *) getPtr(bp,memenv); bp += 4; + bp += 4; /* Align */ + while( * (int*) bp) { switch (* (int*) bp) { + case 1: {bp += 4; + flags = (int)*(int *) bp; bp += 4; + } break; + }}; + wxPopupWindow * Result = new EwxPopupWindow(parent,flags); + newPtr((void *) Result, 0, memenv); + rt.addRef(getRef((void *)Result,memenv), "wxPopupWindow"); + break; +} +case wxPopupWindow_new_0: { // wxPopupWindow::wxPopupWindow + wxPopupWindow * Result = new EwxPopupWindow(); + newPtr((void *) Result, 0, memenv); + rt.addRef(getRef((void *)Result,memenv), "wxPopupWindow"); + break; +} +case wxPopupWindow_Create: { // wxPopupWindow::Create + int flags=wxBORDER_NONE; + wxPopupWindow *This = (wxPopupWindow *) getPtr(bp,memenv); bp += 4; + wxWindow *parent = (wxWindow *) getPtr(bp,memenv); bp += 4; + while( * (int*) bp) { switch (* (int*) bp) { + case 1: {bp += 4; + flags = (int)*(int *) bp; bp += 4; + } break; + }}; + if(!This) throw wxe_badarg(0); + bool Result = This->Create(parent,flags); + rt.addBool(Result); + break; +} +case wxPopupWindow_Position: { // wxPopupWindow::Position + wxPopupWindow *This = (wxPopupWindow *) getPtr(bp,memenv); bp += 4; + int * ptOriginX = (int *) bp; bp += 4; + int * ptOriginY = (int *) bp; bp += 4; + wxPoint ptOrigin = wxPoint(*ptOriginX,*ptOriginY); + int * sizeW = (int *) bp; bp += 4; + int * sizeH = (int *) bp; bp += 4; + wxSize size = wxSize(*sizeW,*sizeH); + if(!This) throw wxe_badarg(0); + This->Position(ptOrigin,size); + break; +} +#endif // wxUSE_POPUPWIN +#if wxUSE_POPUPWIN +case wxPopupTransientWindow_new_0: { // wxPopupTransientWindow::wxPopupTransientWindow + wxPopupTransientWindow * Result = new EwxPopupTransientWindow(); + newPtr((void *) Result, 0, memenv); + rt.addRef(getRef((void *)Result,memenv), "wxPopupTransientWindow"); + break; +} +case wxPopupTransientWindow_new_2: { // wxPopupTransientWindow::wxPopupTransientWindow + int style=wxBORDER_NONE; + wxWindow *parent = (wxWindow *) getPtr(bp,memenv); bp += 4; + bp += 4; /* Align */ + while( * (int*) bp) { switch (* (int*) bp) { + case 1: {bp += 4; + style = (int)*(int *) bp; bp += 4; + } break; + }}; + wxPopupTransientWindow * Result = new EwxPopupTransientWindow(parent,style); + newPtr((void *) Result, 0, memenv); + rt.addRef(getRef((void *)Result,memenv), "wxPopupTransientWindow"); + break; +} +case wxPopupTransientWindow_Popup: { // wxPopupTransientWindow::Popup + wxWindow * focus=NULL; + wxPopupTransientWindow *This = (wxPopupTransientWindow *) getPtr(bp,memenv); bp += 4; + bp += 4; /* Align */ + while( * (int*) bp) { switch (* (int*) bp) { + case 1: {bp += 4; +focus = (wxWindow *) getPtr(bp,memenv); bp += 4; + } break; + }}; + if(!This) throw wxe_badarg(0); + This->Popup(focus); + break; +} +case wxPopupTransientWindow_Dismiss: { // wxPopupTransientWindow::Dismiss + wxPopupTransientWindow *This = (wxPopupTransientWindow *) getPtr(bp,memenv); bp += 4; + if(!This) throw wxe_badarg(0); + This->Dismiss(); + break; +} +#endif // wxUSE_POPUPWIN default: { wxeReturn error = wxeReturn(WXE_DRV_PORT, Ecmd.caller, false); error.addAtom("_wxe_error_"); error.addInt((int) Ecmd.op); @@ -31427,7 +31754,7 @@ case wxTaskBarIcon_SetIcon: { // wxTaskBarIcon::SetIcon }} /* The End */ -void WxeApp::delete_object(void *ptr, wxeRefData *refd) { +bool WxeApp::delete_object(void *ptr, wxeRefData *refd) { switch(refd->type) { case 24: delete (wxGridCellBoolRenderer *) ptr; break; case 25: delete (wxGridCellBoolEditor *) ptr; break; @@ -31446,10 +31773,13 @@ void WxeApp::delete_object(void *ptr, wxeRefData *refd) { case 101: delete (wxListItemAttr *) ptr; break; case 103: delete (wxTextAttr *) ptr; break; case 155: delete (wxAuiPaneInfo *) ptr; break; - case 211: /* delete (wxFileDataObject *) ptr;These objects must be deleted by owner object */ break; - case 212: /* delete (wxTextDataObject *) ptr;These objects must be deleted by owner object */ break; - case 213: /* delete (wxBitmapDataObject *) ptr;These objects must be deleted by owner object */ break; - case 225: delete (wxLogNull *) ptr; break; - default: delete (wxObject *) ptr; -}} + case 212: /* delete (wxFileDataObject *) ptr;These objects must be deleted by owner object */ break; + case 213: /* delete (wxTextDataObject *) ptr;These objects must be deleted by owner object */ break; + case 214: /* delete (wxBitmapDataObject *) ptr;These objects must be deleted by owner object */ break; + case 226: delete (wxLogNull *) ptr; break; + case 230: delete (EwxLocale *) ptr; return false; + default: delete (wxObject *) ptr; return false; + } + return true; +} diff --git a/lib/wx/c_src/gen/wxe_init.cpp b/lib/wx/c_src/gen/wxe_init.cpp index a75298392b..3a4bced790 100644 --- a/lib/wx/c_src/gen/wxe_init.cpp +++ b/lib/wx/c_src/gen/wxe_init.cpp @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2008-2011. All Rights Reserved. + * Copyright Ericsson AB 2008-2014. All Rights Reserved. * * The contents of this file are subject to the Erlang Public License, * Version 1.1, (the "License"); you may not use this file except in @@ -36,6 +36,8 @@ void WxeApp::init_nonconsts(wxeMemEnv *memenv, ErlDrvTermData caller) { rt.addTupleCount(2); rt.addAtom("wxGAUGE_EMULATE_INDETERMINATE_MODE"); rt.addInt(wxGAUGE_EMULATE_INDETERMINATE_MODE); rt.addTupleCount(2); + rt.addAtom("wxSL_LABELS"); rt.addInt(wxSL_LABELS); + rt.addTupleCount(2); rt.addAtom("wxTR_DEFAULT_STYLE"); rt.addInt(wxTR_DEFAULT_STYLE); rt.addTupleCount(2); rt.addAtom("wxBETA_NUMBER"); rt.addInt(wxBETA_NUMBER); @@ -136,7 +138,7 @@ void WxeApp::init_nonconsts(wxeMemEnv *memenv, ErlDrvTermData caller) { rt.addTupleCount(2); rt.addAtom("wxWHITE_PEN"); rt.addRef(getRef((void *)wxWHITE_PEN,memenv),"wxPen"); rt.addTupleCount(2); - rt.endList(56); + rt.endList(57); rt.addTupleCount(2); rt.send(); } diff --git a/lib/wx/c_src/gen/wxe_macros.h b/lib/wx/c_src/gen/wxe_macros.h index a1da6500d5..c8ca9bfe5b 100644 --- a/lib/wx/c_src/gen/wxe_macros.h +++ b/lib/wx/c_src/gen/wxe_macros.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2008-2013. All Rights Reserved. + * Copyright Ericsson AB 2008-2014. All Rights Reserved. * * The contents of this file are subject to the Erlang Public License, * Version 1.1, (the "License"); you may not use this file except in @@ -58,6 +58,7 @@ #include <wx/listbook.h> #include <wx/treebook.h> #include <wx/taskbar.h> +#include <wx/popupwin.h> #include <wx/html/htmlwin.h> #include <wx/html/htmlcell.h> #include <wx/filename.h> @@ -1716,1641 +1717,1676 @@ #define wxTextCtrl_Create 1824 #define wxTextCtrl_Cut 1825 #define wxTextCtrl_DiscardEdits 1826 -#define wxTextCtrl_EmulateKeyPress 1827 -#define wxTextCtrl_GetDefaultStyle 1828 -#define wxTextCtrl_GetInsertionPoint 1829 -#define wxTextCtrl_GetLastPosition 1830 -#define wxTextCtrl_GetLineLength 1831 -#define wxTextCtrl_GetLineText 1832 -#define wxTextCtrl_GetNumberOfLines 1833 -#define wxTextCtrl_GetRange 1834 -#define wxTextCtrl_GetSelection 1835 -#define wxTextCtrl_GetStringSelection 1836 -#define wxTextCtrl_GetStyle 1837 -#define wxTextCtrl_GetValue 1838 -#define wxTextCtrl_IsEditable 1839 -#define wxTextCtrl_IsModified 1840 -#define wxTextCtrl_IsMultiLine 1841 -#define wxTextCtrl_IsSingleLine 1842 -#define wxTextCtrl_LoadFile 1843 -#define wxTextCtrl_MarkDirty 1844 -#define wxTextCtrl_Paste 1845 -#define wxTextCtrl_PositionToXY 1846 -#define wxTextCtrl_Redo 1847 -#define wxTextCtrl_Remove 1848 -#define wxTextCtrl_Replace 1849 -#define wxTextCtrl_SaveFile 1850 -#define wxTextCtrl_SetDefaultStyle 1851 -#define wxTextCtrl_SetEditable 1852 -#define wxTextCtrl_SetInsertionPoint 1853 -#define wxTextCtrl_SetInsertionPointEnd 1854 -#define wxTextCtrl_SetMaxLength 1856 -#define wxTextCtrl_SetSelection 1857 -#define wxTextCtrl_SetStyle 1858 -#define wxTextCtrl_SetValue 1859 -#define wxTextCtrl_ShowPosition 1860 -#define wxTextCtrl_Undo 1861 -#define wxTextCtrl_WriteText 1862 -#define wxTextCtrl_XYToPosition 1863 -#define wxNotebook_new_0 1866 -#define wxNotebook_new_3 1867 -#define wxNotebook_destruct 1868 -#define wxNotebook_AddPage 1869 -#define wxNotebook_AdvanceSelection 1870 -#define wxNotebook_AssignImageList 1871 -#define wxNotebook_Create 1872 -#define wxNotebook_DeleteAllPages 1873 -#define wxNotebook_DeletePage 1874 -#define wxNotebook_RemovePage 1875 -#define wxNotebook_GetCurrentPage 1876 -#define wxNotebook_GetImageList 1877 -#define wxNotebook_GetPage 1879 -#define wxNotebook_GetPageCount 1880 -#define wxNotebook_GetPageImage 1881 -#define wxNotebook_GetPageText 1882 -#define wxNotebook_GetRowCount 1883 -#define wxNotebook_GetSelection 1884 -#define wxNotebook_GetThemeBackgroundColour 1885 -#define wxNotebook_HitTest 1887 -#define wxNotebook_InsertPage 1889 -#define wxNotebook_SetImageList 1890 -#define wxNotebook_SetPadding 1891 -#define wxNotebook_SetPageSize 1892 -#define wxNotebook_SetPageImage 1893 -#define wxNotebook_SetPageText 1894 -#define wxNotebook_SetSelection 1895 -#define wxNotebook_ChangeSelection 1896 -#define wxChoicebook_new_0 1897 -#define wxChoicebook_new_3 1898 -#define wxChoicebook_AddPage 1899 -#define wxChoicebook_AdvanceSelection 1900 -#define wxChoicebook_AssignImageList 1901 -#define wxChoicebook_Create 1902 -#define wxChoicebook_DeleteAllPages 1903 -#define wxChoicebook_DeletePage 1904 -#define wxChoicebook_RemovePage 1905 -#define wxChoicebook_GetCurrentPage 1906 -#define wxChoicebook_GetImageList 1907 -#define wxChoicebook_GetPage 1909 -#define wxChoicebook_GetPageCount 1910 -#define wxChoicebook_GetPageImage 1911 -#define wxChoicebook_GetPageText 1912 -#define wxChoicebook_GetSelection 1913 -#define wxChoicebook_HitTest 1914 -#define wxChoicebook_InsertPage 1915 -#define wxChoicebook_SetImageList 1916 -#define wxChoicebook_SetPageSize 1917 -#define wxChoicebook_SetPageImage 1918 -#define wxChoicebook_SetPageText 1919 -#define wxChoicebook_SetSelection 1920 -#define wxChoicebook_ChangeSelection 1921 -#define wxChoicebook_destroy 1922 -#define wxToolbook_new_0 1923 -#define wxToolbook_new_3 1924 -#define wxToolbook_AddPage 1925 -#define wxToolbook_AdvanceSelection 1926 -#define wxToolbook_AssignImageList 1927 -#define wxToolbook_Create 1928 -#define wxToolbook_DeleteAllPages 1929 -#define wxToolbook_DeletePage 1930 -#define wxToolbook_RemovePage 1931 -#define wxToolbook_GetCurrentPage 1932 -#define wxToolbook_GetImageList 1933 -#define wxToolbook_GetPage 1935 -#define wxToolbook_GetPageCount 1936 -#define wxToolbook_GetPageImage 1937 -#define wxToolbook_GetPageText 1938 -#define wxToolbook_GetSelection 1939 -#define wxToolbook_HitTest 1941 -#define wxToolbook_InsertPage 1942 -#define wxToolbook_SetImageList 1943 -#define wxToolbook_SetPageSize 1944 -#define wxToolbook_SetPageImage 1945 -#define wxToolbook_SetPageText 1946 -#define wxToolbook_SetSelection 1947 -#define wxToolbook_ChangeSelection 1948 -#define wxToolbook_destroy 1949 -#define wxListbook_new_0 1950 -#define wxListbook_new_3 1951 -#define wxListbook_AddPage 1952 -#define wxListbook_AdvanceSelection 1953 -#define wxListbook_AssignImageList 1954 -#define wxListbook_Create 1955 -#define wxListbook_DeleteAllPages 1956 -#define wxListbook_DeletePage 1957 -#define wxListbook_RemovePage 1958 -#define wxListbook_GetCurrentPage 1959 -#define wxListbook_GetImageList 1960 -#define wxListbook_GetPage 1962 -#define wxListbook_GetPageCount 1963 -#define wxListbook_GetPageImage 1964 -#define wxListbook_GetPageText 1965 -#define wxListbook_GetSelection 1966 -#define wxListbook_HitTest 1968 -#define wxListbook_InsertPage 1969 -#define wxListbook_SetImageList 1970 -#define wxListbook_SetPageSize 1971 -#define wxListbook_SetPageImage 1972 -#define wxListbook_SetPageText 1973 -#define wxListbook_SetSelection 1974 -#define wxListbook_ChangeSelection 1975 -#define wxListbook_destroy 1976 -#define wxTreebook_new_0 1977 -#define wxTreebook_new_3 1978 -#define wxTreebook_AddPage 1979 -#define wxTreebook_AdvanceSelection 1980 -#define wxTreebook_AssignImageList 1981 -#define wxTreebook_Create 1982 -#define wxTreebook_DeleteAllPages 1983 -#define wxTreebook_DeletePage 1984 -#define wxTreebook_RemovePage 1985 -#define wxTreebook_GetCurrentPage 1986 -#define wxTreebook_GetImageList 1987 -#define wxTreebook_GetPage 1989 -#define wxTreebook_GetPageCount 1990 -#define wxTreebook_GetPageImage 1991 -#define wxTreebook_GetPageText 1992 -#define wxTreebook_GetSelection 1993 -#define wxTreebook_ExpandNode 1994 -#define wxTreebook_IsNodeExpanded 1995 -#define wxTreebook_HitTest 1997 -#define wxTreebook_InsertPage 1998 -#define wxTreebook_InsertSubPage 1999 -#define wxTreebook_SetImageList 2000 -#define wxTreebook_SetPageSize 2001 -#define wxTreebook_SetPageImage 2002 -#define wxTreebook_SetPageText 2003 -#define wxTreebook_SetSelection 2004 -#define wxTreebook_ChangeSelection 2005 -#define wxTreebook_destroy 2006 -#define wxTreeCtrl_new_2 2009 -#define wxTreeCtrl_new_0 2010 -#define wxTreeCtrl_destruct 2012 -#define wxTreeCtrl_AddRoot 2013 -#define wxTreeCtrl_AppendItem 2014 -#define wxTreeCtrl_AssignImageList 2015 -#define wxTreeCtrl_AssignStateImageList 2016 -#define wxTreeCtrl_Collapse 2017 -#define wxTreeCtrl_CollapseAndReset 2018 -#define wxTreeCtrl_Create 2019 -#define wxTreeCtrl_Delete 2020 -#define wxTreeCtrl_DeleteAllItems 2021 -#define wxTreeCtrl_DeleteChildren 2022 -#define wxTreeCtrl_EditLabel 2023 -#define wxTreeCtrl_EnsureVisible 2024 -#define wxTreeCtrl_Expand 2025 -#define wxTreeCtrl_GetBoundingRect 2026 -#define wxTreeCtrl_GetChildrenCount 2028 -#define wxTreeCtrl_GetCount 2029 -#define wxTreeCtrl_GetEditControl 2030 -#define wxTreeCtrl_GetFirstChild 2031 -#define wxTreeCtrl_GetNextChild 2032 -#define wxTreeCtrl_GetFirstVisibleItem 2033 -#define wxTreeCtrl_GetImageList 2034 -#define wxTreeCtrl_GetIndent 2035 -#define wxTreeCtrl_GetItemBackgroundColour 2036 -#define wxTreeCtrl_GetItemData 2037 -#define wxTreeCtrl_GetItemFont 2038 -#define wxTreeCtrl_GetItemImage_1 2039 -#define wxTreeCtrl_GetItemImage_2 2040 -#define wxTreeCtrl_GetItemText 2041 -#define wxTreeCtrl_GetItemTextColour 2042 -#define wxTreeCtrl_GetLastChild 2043 -#define wxTreeCtrl_GetNextSibling 2044 -#define wxTreeCtrl_GetNextVisible 2045 -#define wxTreeCtrl_GetItemParent 2046 -#define wxTreeCtrl_GetPrevSibling 2047 -#define wxTreeCtrl_GetPrevVisible 2048 -#define wxTreeCtrl_GetRootItem 2049 -#define wxTreeCtrl_GetSelection 2050 -#define wxTreeCtrl_GetSelections 2051 -#define wxTreeCtrl_GetStateImageList 2052 -#define wxTreeCtrl_HitTest 2053 -#define wxTreeCtrl_InsertItem 2055 -#define wxTreeCtrl_IsBold 2056 -#define wxTreeCtrl_IsExpanded 2057 -#define wxTreeCtrl_IsSelected 2058 -#define wxTreeCtrl_IsVisible 2059 -#define wxTreeCtrl_ItemHasChildren 2060 -#define wxTreeCtrl_IsTreeItemIdOk 2061 -#define wxTreeCtrl_PrependItem 2062 -#define wxTreeCtrl_ScrollTo 2063 -#define wxTreeCtrl_SelectItem_1 2064 -#define wxTreeCtrl_SelectItem_2 2065 -#define wxTreeCtrl_SetIndent 2066 -#define wxTreeCtrl_SetImageList 2067 -#define wxTreeCtrl_SetItemBackgroundColour 2068 -#define wxTreeCtrl_SetItemBold 2069 -#define wxTreeCtrl_SetItemData 2070 -#define wxTreeCtrl_SetItemDropHighlight 2071 -#define wxTreeCtrl_SetItemFont 2072 -#define wxTreeCtrl_SetItemHasChildren 2073 -#define wxTreeCtrl_SetItemImage_2 2074 -#define wxTreeCtrl_SetItemImage_3 2075 -#define wxTreeCtrl_SetItemText 2076 -#define wxTreeCtrl_SetItemTextColour 2077 -#define wxTreeCtrl_SetStateImageList 2078 -#define wxTreeCtrl_SetWindowStyle 2079 -#define wxTreeCtrl_SortChildren 2080 -#define wxTreeCtrl_Toggle 2081 -#define wxTreeCtrl_ToggleItemSelection 2082 -#define wxTreeCtrl_Unselect 2083 -#define wxTreeCtrl_UnselectAll 2084 -#define wxTreeCtrl_UnselectItem 2085 -#define wxScrollBar_new_0 2086 -#define wxScrollBar_new_3 2087 -#define wxScrollBar_destruct 2088 -#define wxScrollBar_Create 2089 -#define wxScrollBar_GetRange 2090 -#define wxScrollBar_GetPageSize 2091 -#define wxScrollBar_GetThumbPosition 2092 -#define wxScrollBar_GetThumbSize 2093 -#define wxScrollBar_SetThumbPosition 2094 -#define wxScrollBar_SetScrollbar 2095 -#define wxSpinButton_new_2 2097 -#define wxSpinButton_new_0 2098 -#define wxSpinButton_Create 2099 -#define wxSpinButton_GetMax 2100 -#define wxSpinButton_GetMin 2101 -#define wxSpinButton_GetValue 2102 -#define wxSpinButton_SetRange 2103 -#define wxSpinButton_SetValue 2104 -#define wxSpinButton_destroy 2105 -#define wxSpinCtrl_new_0 2106 -#define wxSpinCtrl_new_2 2107 -#define wxSpinCtrl_Create 2109 -#define wxSpinCtrl_SetValue_1_1 2112 -#define wxSpinCtrl_SetValue_1_0 2113 -#define wxSpinCtrl_GetValue 2115 -#define wxSpinCtrl_SetRange 2117 -#define wxSpinCtrl_SetSelection 2118 -#define wxSpinCtrl_GetMin 2120 -#define wxSpinCtrl_GetMax 2122 -#define wxSpinCtrl_destroy 2123 -#define wxStaticText_new_0 2124 -#define wxStaticText_new_4 2125 -#define wxStaticText_Create 2126 -#define wxStaticText_GetLabel 2127 -#define wxStaticText_SetLabel 2128 -#define wxStaticText_Wrap 2129 -#define wxStaticText_destroy 2130 -#define wxStaticBitmap_new_0 2131 -#define wxStaticBitmap_new_4 2132 -#define wxStaticBitmap_Create 2133 -#define wxStaticBitmap_GetBitmap 2134 -#define wxStaticBitmap_SetBitmap 2135 -#define wxStaticBitmap_destroy 2136 -#define wxRadioBox_new 2137 -#define wxRadioBox_destruct 2139 -#define wxRadioBox_Create 2140 -#define wxRadioBox_Enable_2 2141 -#define wxRadioBox_Enable_1 2142 -#define wxRadioBox_GetSelection 2143 -#define wxRadioBox_GetString 2144 -#define wxRadioBox_SetSelection 2145 -#define wxRadioBox_Show_2 2146 -#define wxRadioBox_Show_1 2147 -#define wxRadioBox_GetColumnCount 2148 -#define wxRadioBox_GetItemHelpText 2149 -#define wxRadioBox_GetItemToolTip 2150 -#define wxRadioBox_GetItemFromPoint 2152 -#define wxRadioBox_GetRowCount 2153 -#define wxRadioBox_IsItemEnabled 2154 -#define wxRadioBox_IsItemShown 2155 -#define wxRadioBox_SetItemHelpText 2156 -#define wxRadioBox_SetItemToolTip 2157 -#define wxRadioButton_new_0 2158 -#define wxRadioButton_new_4 2159 -#define wxRadioButton_Create 2160 -#define wxRadioButton_GetValue 2161 -#define wxRadioButton_SetValue 2162 -#define wxRadioButton_destroy 2163 -#define wxSlider_new_6 2165 -#define wxSlider_new_0 2166 -#define wxSlider_Create 2167 -#define wxSlider_GetLineSize 2168 -#define wxSlider_GetMax 2169 -#define wxSlider_GetMin 2170 -#define wxSlider_GetPageSize 2171 -#define wxSlider_GetThumbLength 2172 -#define wxSlider_GetValue 2173 -#define wxSlider_SetLineSize 2174 -#define wxSlider_SetPageSize 2175 -#define wxSlider_SetRange 2176 -#define wxSlider_SetThumbLength 2177 -#define wxSlider_SetValue 2178 -#define wxSlider_destroy 2179 -#define wxDialog_new_4 2181 -#define wxDialog_new_0 2182 -#define wxDialog_destruct 2184 -#define wxDialog_Create 2185 -#define wxDialog_CreateButtonSizer 2186 -#define wxDialog_CreateStdDialogButtonSizer 2187 -#define wxDialog_EndModal 2188 -#define wxDialog_GetAffirmativeId 2189 -#define wxDialog_GetReturnCode 2190 -#define wxDialog_IsModal 2191 -#define wxDialog_SetAffirmativeId 2192 -#define wxDialog_SetReturnCode 2193 -#define wxDialog_Show 2194 -#define wxDialog_ShowModal 2195 -#define wxColourDialog_new_0 2196 -#define wxColourDialog_new_2 2197 -#define wxColourDialog_destruct 2198 -#define wxColourDialog_Create 2199 -#define wxColourDialog_GetColourData 2200 -#define wxColourData_new_0 2201 -#define wxColourData_new_1 2202 -#define wxColourData_destruct 2203 -#define wxColourData_GetChooseFull 2204 -#define wxColourData_GetColour 2205 -#define wxColourData_GetCustomColour 2207 -#define wxColourData_SetChooseFull 2208 -#define wxColourData_SetColour 2209 -#define wxColourData_SetCustomColour 2210 -#define wxPalette_new_0 2211 -#define wxPalette_new_4 2212 -#define wxPalette_destruct 2214 -#define wxPalette_Create 2215 -#define wxPalette_GetColoursCount 2216 -#define wxPalette_GetPixel 2217 -#define wxPalette_GetRGB 2218 -#define wxPalette_IsOk 2219 -#define wxDirDialog_new 2223 -#define wxDirDialog_destruct 2224 -#define wxDirDialog_GetPath 2225 -#define wxDirDialog_GetMessage 2226 -#define wxDirDialog_SetMessage 2227 -#define wxDirDialog_SetPath 2228 -#define wxFileDialog_new 2232 -#define wxFileDialog_destruct 2233 -#define wxFileDialog_GetDirectory 2234 -#define wxFileDialog_GetFilename 2235 -#define wxFileDialog_GetFilenames 2236 -#define wxFileDialog_GetFilterIndex 2237 -#define wxFileDialog_GetMessage 2238 -#define wxFileDialog_GetPath 2239 -#define wxFileDialog_GetPaths 2240 -#define wxFileDialog_GetWildcard 2241 -#define wxFileDialog_SetDirectory 2242 -#define wxFileDialog_SetFilename 2243 -#define wxFileDialog_SetFilterIndex 2244 -#define wxFileDialog_SetMessage 2245 -#define wxFileDialog_SetPath 2246 -#define wxFileDialog_SetWildcard 2247 -#define wxPickerBase_SetInternalMargin 2248 -#define wxPickerBase_GetInternalMargin 2249 -#define wxPickerBase_SetTextCtrlProportion 2250 -#define wxPickerBase_SetPickerCtrlProportion 2251 -#define wxPickerBase_GetTextCtrlProportion 2252 -#define wxPickerBase_GetPickerCtrlProportion 2253 -#define wxPickerBase_HasTextCtrl 2254 -#define wxPickerBase_GetTextCtrl 2255 -#define wxPickerBase_IsTextCtrlGrowable 2256 -#define wxPickerBase_SetPickerCtrlGrowable 2257 -#define wxPickerBase_SetTextCtrlGrowable 2258 -#define wxPickerBase_IsPickerCtrlGrowable 2259 -#define wxFilePickerCtrl_new_0 2260 -#define wxFilePickerCtrl_new_3 2261 -#define wxFilePickerCtrl_Create 2262 -#define wxFilePickerCtrl_GetPath 2263 -#define wxFilePickerCtrl_SetPath 2264 -#define wxFilePickerCtrl_destroy 2265 -#define wxDirPickerCtrl_new_0 2266 -#define wxDirPickerCtrl_new_3 2267 -#define wxDirPickerCtrl_Create 2268 -#define wxDirPickerCtrl_GetPath 2269 -#define wxDirPickerCtrl_SetPath 2270 -#define wxDirPickerCtrl_destroy 2271 -#define wxColourPickerCtrl_new_0 2272 -#define wxColourPickerCtrl_new_3 2273 -#define wxColourPickerCtrl_Create 2274 -#define wxColourPickerCtrl_GetColour 2275 -#define wxColourPickerCtrl_SetColour_1_1 2276 -#define wxColourPickerCtrl_SetColour_1_0 2277 -#define wxColourPickerCtrl_destroy 2278 -#define wxDatePickerCtrl_new_0 2279 -#define wxDatePickerCtrl_new_3 2280 -#define wxDatePickerCtrl_GetRange 2281 -#define wxDatePickerCtrl_GetValue 2282 -#define wxDatePickerCtrl_SetRange 2283 -#define wxDatePickerCtrl_SetValue 2284 -#define wxDatePickerCtrl_destroy 2285 -#define wxFontPickerCtrl_new_0 2286 -#define wxFontPickerCtrl_new_3 2287 -#define wxFontPickerCtrl_Create 2288 -#define wxFontPickerCtrl_GetSelectedFont 2289 -#define wxFontPickerCtrl_SetSelectedFont 2290 -#define wxFontPickerCtrl_GetMaxPointSize 2291 -#define wxFontPickerCtrl_SetMaxPointSize 2292 -#define wxFontPickerCtrl_destroy 2293 -#define wxFindReplaceDialog_new_0 2296 -#define wxFindReplaceDialog_new_4 2297 -#define wxFindReplaceDialog_destruct 2298 -#define wxFindReplaceDialog_Create 2299 -#define wxFindReplaceDialog_GetData 2300 -#define wxFindReplaceData_new_0 2301 -#define wxFindReplaceData_new_1 2302 -#define wxFindReplaceData_GetFindString 2303 -#define wxFindReplaceData_GetReplaceString 2304 -#define wxFindReplaceData_GetFlags 2305 -#define wxFindReplaceData_SetFlags 2306 -#define wxFindReplaceData_SetFindString 2307 -#define wxFindReplaceData_SetReplaceString 2308 -#define wxFindReplaceData_destroy 2309 -#define wxMultiChoiceDialog_new_0 2310 -#define wxMultiChoiceDialog_new_5 2312 -#define wxMultiChoiceDialog_GetSelections 2313 -#define wxMultiChoiceDialog_SetSelections 2314 -#define wxMultiChoiceDialog_destroy 2315 -#define wxSingleChoiceDialog_new_0 2316 -#define wxSingleChoiceDialog_new_5 2318 -#define wxSingleChoiceDialog_GetSelection 2319 -#define wxSingleChoiceDialog_GetStringSelection 2320 -#define wxSingleChoiceDialog_SetSelection 2321 -#define wxSingleChoiceDialog_destroy 2322 -#define wxTextEntryDialog_new 2323 -#define wxTextEntryDialog_GetValue 2324 -#define wxTextEntryDialog_SetValue 2325 -#define wxTextEntryDialog_destroy 2326 -#define wxPasswordEntryDialog_new 2327 -#define wxPasswordEntryDialog_destroy 2328 -#define wxFontData_new_0 2329 -#define wxFontData_new_1 2330 -#define wxFontData_destruct 2331 -#define wxFontData_EnableEffects 2332 -#define wxFontData_GetAllowSymbols 2333 -#define wxFontData_GetColour 2334 -#define wxFontData_GetChosenFont 2335 -#define wxFontData_GetEnableEffects 2336 -#define wxFontData_GetInitialFont 2337 -#define wxFontData_GetShowHelp 2338 -#define wxFontData_SetAllowSymbols 2339 -#define wxFontData_SetChosenFont 2340 -#define wxFontData_SetColour 2341 -#define wxFontData_SetInitialFont 2342 -#define wxFontData_SetRange 2343 -#define wxFontData_SetShowHelp 2344 -#define wxFontDialog_new_0 2348 -#define wxFontDialog_new_2 2350 -#define wxFontDialog_Create 2352 -#define wxFontDialog_GetFontData 2353 -#define wxFontDialog_destroy 2355 -#define wxProgressDialog_new 2356 -#define wxProgressDialog_destruct 2357 -#define wxProgressDialog_Resume 2358 -#define wxProgressDialog_Update_2 2359 -#define wxProgressDialog_Update_0 2360 -#define wxMessageDialog_new 2361 -#define wxMessageDialog_destruct 2362 -#define wxPageSetupDialog_new 2363 -#define wxPageSetupDialog_destruct 2364 -#define wxPageSetupDialog_GetPageSetupData 2365 -#define wxPageSetupDialog_ShowModal 2366 -#define wxPageSetupDialogData_new_0 2367 -#define wxPageSetupDialogData_new_1_0 2368 -#define wxPageSetupDialogData_new_1_1 2369 -#define wxPageSetupDialogData_destruct 2370 -#define wxPageSetupDialogData_EnableHelp 2371 -#define wxPageSetupDialogData_EnableMargins 2372 -#define wxPageSetupDialogData_EnableOrientation 2373 -#define wxPageSetupDialogData_EnablePaper 2374 -#define wxPageSetupDialogData_EnablePrinter 2375 -#define wxPageSetupDialogData_GetDefaultMinMargins 2376 -#define wxPageSetupDialogData_GetEnableMargins 2377 -#define wxPageSetupDialogData_GetEnableOrientation 2378 -#define wxPageSetupDialogData_GetEnablePaper 2379 -#define wxPageSetupDialogData_GetEnablePrinter 2380 -#define wxPageSetupDialogData_GetEnableHelp 2381 -#define wxPageSetupDialogData_GetDefaultInfo 2382 -#define wxPageSetupDialogData_GetMarginTopLeft 2383 -#define wxPageSetupDialogData_GetMarginBottomRight 2384 -#define wxPageSetupDialogData_GetMinMarginTopLeft 2385 -#define wxPageSetupDialogData_GetMinMarginBottomRight 2386 -#define wxPageSetupDialogData_GetPaperId 2387 -#define wxPageSetupDialogData_GetPaperSize 2388 -#define wxPageSetupDialogData_GetPrintData 2390 -#define wxPageSetupDialogData_IsOk 2391 -#define wxPageSetupDialogData_SetDefaultInfo 2392 -#define wxPageSetupDialogData_SetDefaultMinMargins 2393 -#define wxPageSetupDialogData_SetMarginTopLeft 2394 -#define wxPageSetupDialogData_SetMarginBottomRight 2395 -#define wxPageSetupDialogData_SetMinMarginTopLeft 2396 -#define wxPageSetupDialogData_SetMinMarginBottomRight 2397 -#define wxPageSetupDialogData_SetPaperId 2398 -#define wxPageSetupDialogData_SetPaperSize_1_1 2399 -#define wxPageSetupDialogData_SetPaperSize_1_0 2400 -#define wxPageSetupDialogData_SetPrintData 2401 -#define wxPrintDialog_new_2_0 2402 -#define wxPrintDialog_new_2_1 2403 -#define wxPrintDialog_destruct 2404 -#define wxPrintDialog_GetPrintDialogData 2405 -#define wxPrintDialog_GetPrintDC 2406 -#define wxPrintDialogData_new_0 2407 -#define wxPrintDialogData_new_1_1 2408 -#define wxPrintDialogData_new_1_0 2409 -#define wxPrintDialogData_destruct 2410 -#define wxPrintDialogData_EnableHelp 2411 -#define wxPrintDialogData_EnablePageNumbers 2412 -#define wxPrintDialogData_EnablePrintToFile 2413 -#define wxPrintDialogData_EnableSelection 2414 -#define wxPrintDialogData_GetAllPages 2415 -#define wxPrintDialogData_GetCollate 2416 -#define wxPrintDialogData_GetFromPage 2417 -#define wxPrintDialogData_GetMaxPage 2418 -#define wxPrintDialogData_GetMinPage 2419 -#define wxPrintDialogData_GetNoCopies 2420 -#define wxPrintDialogData_GetPrintData 2421 -#define wxPrintDialogData_GetPrintToFile 2422 -#define wxPrintDialogData_GetSelection 2423 -#define wxPrintDialogData_GetToPage 2424 -#define wxPrintDialogData_IsOk 2425 -#define wxPrintDialogData_SetCollate 2426 -#define wxPrintDialogData_SetFromPage 2427 -#define wxPrintDialogData_SetMaxPage 2428 -#define wxPrintDialogData_SetMinPage 2429 -#define wxPrintDialogData_SetNoCopies 2430 -#define wxPrintDialogData_SetPrintData 2431 -#define wxPrintDialogData_SetPrintToFile 2432 -#define wxPrintDialogData_SetSelection 2433 -#define wxPrintDialogData_SetToPage 2434 -#define wxPrintData_new_0 2435 -#define wxPrintData_new_1 2436 -#define wxPrintData_destruct 2437 -#define wxPrintData_GetCollate 2438 -#define wxPrintData_GetBin 2439 -#define wxPrintData_GetColour 2440 -#define wxPrintData_GetDuplex 2441 -#define wxPrintData_GetNoCopies 2442 -#define wxPrintData_GetOrientation 2443 -#define wxPrintData_GetPaperId 2444 -#define wxPrintData_GetPrinterName 2445 -#define wxPrintData_GetQuality 2446 -#define wxPrintData_IsOk 2447 -#define wxPrintData_SetBin 2448 -#define wxPrintData_SetCollate 2449 -#define wxPrintData_SetColour 2450 -#define wxPrintData_SetDuplex 2451 -#define wxPrintData_SetNoCopies 2452 -#define wxPrintData_SetOrientation 2453 -#define wxPrintData_SetPaperId 2454 -#define wxPrintData_SetPrinterName 2455 -#define wxPrintData_SetQuality 2456 -#define wxPrintPreview_new_2 2459 -#define wxPrintPreview_new_3 2460 -#define wxPrintPreview_destruct 2462 -#define wxPrintPreview_GetCanvas 2463 -#define wxPrintPreview_GetCurrentPage 2464 -#define wxPrintPreview_GetFrame 2465 -#define wxPrintPreview_GetMaxPage 2466 -#define wxPrintPreview_GetMinPage 2467 -#define wxPrintPreview_GetPrintout 2468 -#define wxPrintPreview_GetPrintoutForPrinting 2469 -#define wxPrintPreview_IsOk 2470 -#define wxPrintPreview_PaintPage 2471 -#define wxPrintPreview_Print 2472 -#define wxPrintPreview_RenderPage 2473 -#define wxPrintPreview_SetCanvas 2474 -#define wxPrintPreview_SetCurrentPage 2475 -#define wxPrintPreview_SetFrame 2476 -#define wxPrintPreview_SetPrintout 2477 -#define wxPrintPreview_SetZoom 2478 -#define wxPreviewFrame_new 2479 -#define wxPreviewFrame_destruct 2480 -#define wxPreviewFrame_CreateControlBar 2481 -#define wxPreviewFrame_CreateCanvas 2482 -#define wxPreviewFrame_Initialize 2483 -#define wxPreviewFrame_OnCloseWindow 2484 -#define wxPreviewControlBar_new 2485 -#define wxPreviewControlBar_destruct 2486 -#define wxPreviewControlBar_CreateButtons 2487 -#define wxPreviewControlBar_GetPrintPreview 2488 -#define wxPreviewControlBar_GetZoomControl 2489 -#define wxPreviewControlBar_SetZoomControl 2490 -#define wxPrinter_new 2492 -#define wxPrinter_CreateAbortWindow 2493 -#define wxPrinter_GetAbort 2494 -#define wxPrinter_GetLastError 2495 -#define wxPrinter_GetPrintDialogData 2496 -#define wxPrinter_Print 2497 -#define wxPrinter_PrintDialog 2498 -#define wxPrinter_ReportError 2499 -#define wxPrinter_Setup 2500 -#define wxPrinter_destroy 2501 -#define wxXmlResource_new_1 2502 -#define wxXmlResource_new_2 2503 -#define wxXmlResource_destruct 2504 -#define wxXmlResource_AttachUnknownControl 2505 -#define wxXmlResource_ClearHandlers 2506 -#define wxXmlResource_CompareVersion 2507 -#define wxXmlResource_Get 2508 -#define wxXmlResource_GetFlags 2509 -#define wxXmlResource_GetVersion 2510 -#define wxXmlResource_GetXRCID 2511 -#define wxXmlResource_InitAllHandlers 2512 -#define wxXmlResource_Load 2513 -#define wxXmlResource_LoadBitmap 2514 -#define wxXmlResource_LoadDialog_2 2515 -#define wxXmlResource_LoadDialog_3 2516 -#define wxXmlResource_LoadFrame_2 2517 -#define wxXmlResource_LoadFrame_3 2518 -#define wxXmlResource_LoadIcon 2519 -#define wxXmlResource_LoadMenu 2520 -#define wxXmlResource_LoadMenuBar_2 2521 -#define wxXmlResource_LoadMenuBar_1 2522 -#define wxXmlResource_LoadPanel_2 2523 -#define wxXmlResource_LoadPanel_3 2524 -#define wxXmlResource_LoadToolBar 2525 -#define wxXmlResource_Set 2526 -#define wxXmlResource_SetFlags 2527 -#define wxXmlResource_Unload 2528 -#define wxXmlResource_xrcctrl 2529 -#define wxHtmlEasyPrinting_new 2530 -#define wxHtmlEasyPrinting_destruct 2531 -#define wxHtmlEasyPrinting_GetPrintData 2532 -#define wxHtmlEasyPrinting_GetPageSetupData 2533 -#define wxHtmlEasyPrinting_PreviewFile 2534 -#define wxHtmlEasyPrinting_PreviewText 2535 -#define wxHtmlEasyPrinting_PrintFile 2536 -#define wxHtmlEasyPrinting_PrintText 2537 -#define wxHtmlEasyPrinting_PageSetup 2538 -#define wxHtmlEasyPrinting_SetFonts 2539 -#define wxHtmlEasyPrinting_SetHeader 2540 -#define wxHtmlEasyPrinting_SetFooter 2541 -#define wxGLCanvas_new_2 2543 -#define wxGLCanvas_new_3_1 2544 -#define wxGLCanvas_new_3_0 2545 -#define wxGLCanvas_GetContext 2546 -#define wxGLCanvas_SetCurrent 2548 -#define wxGLCanvas_SwapBuffers 2549 -#define wxGLCanvas_destroy 2550 -#define wxAuiManager_new 2551 -#define wxAuiManager_destruct 2552 -#define wxAuiManager_AddPane_2_1 2553 -#define wxAuiManager_AddPane_3 2554 -#define wxAuiManager_AddPane_2_0 2555 -#define wxAuiManager_DetachPane 2556 -#define wxAuiManager_GetAllPanes 2557 -#define wxAuiManager_GetArtProvider 2558 -#define wxAuiManager_GetDockSizeConstraint 2559 -#define wxAuiManager_GetFlags 2560 -#define wxAuiManager_GetManagedWindow 2561 -#define wxAuiManager_GetManager 2562 -#define wxAuiManager_GetPane_1_1 2563 -#define wxAuiManager_GetPane_1_0 2564 -#define wxAuiManager_HideHint 2565 -#define wxAuiManager_InsertPane 2566 -#define wxAuiManager_LoadPaneInfo 2567 -#define wxAuiManager_LoadPerspective 2568 -#define wxAuiManager_SavePaneInfo 2569 -#define wxAuiManager_SavePerspective 2570 -#define wxAuiManager_SetArtProvider 2571 -#define wxAuiManager_SetDockSizeConstraint 2572 -#define wxAuiManager_SetFlags 2573 -#define wxAuiManager_SetManagedWindow 2574 -#define wxAuiManager_ShowHint 2575 -#define wxAuiManager_UnInit 2576 -#define wxAuiManager_Update 2577 -#define wxAuiPaneInfo_new_0 2578 -#define wxAuiPaneInfo_new_1 2579 -#define wxAuiPaneInfo_destruct 2580 -#define wxAuiPaneInfo_BestSize_1 2581 -#define wxAuiPaneInfo_BestSize_2 2582 -#define wxAuiPaneInfo_Bottom 2583 -#define wxAuiPaneInfo_BottomDockable 2584 -#define wxAuiPaneInfo_Caption 2585 -#define wxAuiPaneInfo_CaptionVisible 2586 -#define wxAuiPaneInfo_Centre 2587 -#define wxAuiPaneInfo_CentrePane 2588 -#define wxAuiPaneInfo_CloseButton 2589 -#define wxAuiPaneInfo_DefaultPane 2590 -#define wxAuiPaneInfo_DestroyOnClose 2591 -#define wxAuiPaneInfo_Direction 2592 -#define wxAuiPaneInfo_Dock 2593 -#define wxAuiPaneInfo_Dockable 2594 -#define wxAuiPaneInfo_Fixed 2595 -#define wxAuiPaneInfo_Float 2596 -#define wxAuiPaneInfo_Floatable 2597 -#define wxAuiPaneInfo_FloatingPosition_1 2598 -#define wxAuiPaneInfo_FloatingPosition_2 2599 -#define wxAuiPaneInfo_FloatingSize_1 2600 -#define wxAuiPaneInfo_FloatingSize_2 2601 -#define wxAuiPaneInfo_Gripper 2602 -#define wxAuiPaneInfo_GripperTop 2603 -#define wxAuiPaneInfo_HasBorder 2604 -#define wxAuiPaneInfo_HasCaption 2605 -#define wxAuiPaneInfo_HasCloseButton 2606 -#define wxAuiPaneInfo_HasFlag 2607 -#define wxAuiPaneInfo_HasGripper 2608 -#define wxAuiPaneInfo_HasGripperTop 2609 -#define wxAuiPaneInfo_HasMaximizeButton 2610 -#define wxAuiPaneInfo_HasMinimizeButton 2611 -#define wxAuiPaneInfo_HasPinButton 2612 -#define wxAuiPaneInfo_Hide 2613 -#define wxAuiPaneInfo_IsBottomDockable 2614 -#define wxAuiPaneInfo_IsDocked 2615 -#define wxAuiPaneInfo_IsFixed 2616 -#define wxAuiPaneInfo_IsFloatable 2617 -#define wxAuiPaneInfo_IsFloating 2618 -#define wxAuiPaneInfo_IsLeftDockable 2619 -#define wxAuiPaneInfo_IsMovable 2620 -#define wxAuiPaneInfo_IsOk 2621 -#define wxAuiPaneInfo_IsResizable 2622 -#define wxAuiPaneInfo_IsRightDockable 2623 -#define wxAuiPaneInfo_IsShown 2624 -#define wxAuiPaneInfo_IsToolbar 2625 -#define wxAuiPaneInfo_IsTopDockable 2626 -#define wxAuiPaneInfo_Layer 2627 -#define wxAuiPaneInfo_Left 2628 -#define wxAuiPaneInfo_LeftDockable 2629 -#define wxAuiPaneInfo_MaxSize_1 2630 -#define wxAuiPaneInfo_MaxSize_2 2631 -#define wxAuiPaneInfo_MaximizeButton 2632 -#define wxAuiPaneInfo_MinSize_1 2633 -#define wxAuiPaneInfo_MinSize_2 2634 -#define wxAuiPaneInfo_MinimizeButton 2635 -#define wxAuiPaneInfo_Movable 2636 -#define wxAuiPaneInfo_Name 2637 -#define wxAuiPaneInfo_PaneBorder 2638 -#define wxAuiPaneInfo_PinButton 2639 -#define wxAuiPaneInfo_Position 2640 -#define wxAuiPaneInfo_Resizable 2641 -#define wxAuiPaneInfo_Right 2642 -#define wxAuiPaneInfo_RightDockable 2643 -#define wxAuiPaneInfo_Row 2644 -#define wxAuiPaneInfo_SafeSet 2645 -#define wxAuiPaneInfo_SetFlag 2646 -#define wxAuiPaneInfo_Show 2647 -#define wxAuiPaneInfo_ToolbarPane 2648 -#define wxAuiPaneInfo_Top 2649 -#define wxAuiPaneInfo_TopDockable 2650 -#define wxAuiPaneInfo_Window 2651 -#define wxAuiNotebook_new_0 2652 -#define wxAuiNotebook_new_2 2653 -#define wxAuiNotebook_AddPage 2654 -#define wxAuiNotebook_Create 2655 -#define wxAuiNotebook_DeletePage 2656 -#define wxAuiNotebook_GetArtProvider 2657 -#define wxAuiNotebook_GetPage 2658 -#define wxAuiNotebook_GetPageBitmap 2659 -#define wxAuiNotebook_GetPageCount 2660 -#define wxAuiNotebook_GetPageIndex 2661 -#define wxAuiNotebook_GetPageText 2662 -#define wxAuiNotebook_GetSelection 2663 -#define wxAuiNotebook_InsertPage 2664 -#define wxAuiNotebook_RemovePage 2665 -#define wxAuiNotebook_SetArtProvider 2666 -#define wxAuiNotebook_SetFont 2667 -#define wxAuiNotebook_SetPageBitmap 2668 -#define wxAuiNotebook_SetPageText 2669 -#define wxAuiNotebook_SetSelection 2670 -#define wxAuiNotebook_SetTabCtrlHeight 2671 -#define wxAuiNotebook_SetUniformBitmapSize 2672 -#define wxAuiNotebook_destroy 2673 -#define wxMDIParentFrame_new_0 2674 -#define wxMDIParentFrame_new_4 2675 -#define wxMDIParentFrame_destruct 2676 -#define wxMDIParentFrame_ActivateNext 2677 -#define wxMDIParentFrame_ActivatePrevious 2678 -#define wxMDIParentFrame_ArrangeIcons 2679 -#define wxMDIParentFrame_Cascade 2680 -#define wxMDIParentFrame_Create 2681 -#define wxMDIParentFrame_GetActiveChild 2682 -#define wxMDIParentFrame_GetClientWindow 2683 -#define wxMDIParentFrame_Tile 2684 -#define wxMDIChildFrame_new_0 2685 -#define wxMDIChildFrame_new_4 2686 -#define wxMDIChildFrame_destruct 2687 -#define wxMDIChildFrame_Activate 2688 -#define wxMDIChildFrame_Create 2689 -#define wxMDIChildFrame_Maximize 2690 -#define wxMDIChildFrame_Restore 2691 -#define wxMDIClientWindow_new_0 2692 -#define wxMDIClientWindow_new_2 2693 -#define wxMDIClientWindow_destruct 2694 -#define wxMDIClientWindow_CreateClient 2695 -#define wxLayoutAlgorithm_new 2696 -#define wxLayoutAlgorithm_LayoutFrame 2697 -#define wxLayoutAlgorithm_LayoutMDIFrame 2698 -#define wxLayoutAlgorithm_LayoutWindow 2699 -#define wxLayoutAlgorithm_destroy 2700 -#define wxEvent_GetId 2701 -#define wxEvent_GetSkipped 2702 -#define wxEvent_GetTimestamp 2703 -#define wxEvent_IsCommandEvent 2704 -#define wxEvent_ResumePropagation 2705 -#define wxEvent_ShouldPropagate 2706 -#define wxEvent_Skip 2707 -#define wxEvent_StopPropagation 2708 -#define wxCommandEvent_getClientData 2709 -#define wxCommandEvent_GetExtraLong 2710 -#define wxCommandEvent_GetInt 2711 -#define wxCommandEvent_GetSelection 2712 -#define wxCommandEvent_GetString 2713 -#define wxCommandEvent_IsChecked 2714 -#define wxCommandEvent_IsSelection 2715 -#define wxCommandEvent_SetInt 2716 -#define wxCommandEvent_SetString 2717 -#define wxScrollEvent_GetOrientation 2718 -#define wxScrollEvent_GetPosition 2719 -#define wxScrollWinEvent_GetOrientation 2720 -#define wxScrollWinEvent_GetPosition 2721 -#define wxMouseEvent_AltDown 2722 -#define wxMouseEvent_Button 2723 -#define wxMouseEvent_ButtonDClick 2724 -#define wxMouseEvent_ButtonDown 2725 -#define wxMouseEvent_ButtonUp 2726 -#define wxMouseEvent_CmdDown 2727 -#define wxMouseEvent_ControlDown 2728 -#define wxMouseEvent_Dragging 2729 -#define wxMouseEvent_Entering 2730 -#define wxMouseEvent_GetButton 2731 -#define wxMouseEvent_GetPosition 2734 -#define wxMouseEvent_GetLogicalPosition 2735 -#define wxMouseEvent_GetLinesPerAction 2736 -#define wxMouseEvent_GetWheelRotation 2737 -#define wxMouseEvent_GetWheelDelta 2738 -#define wxMouseEvent_GetX 2739 -#define wxMouseEvent_GetY 2740 -#define wxMouseEvent_IsButton 2741 -#define wxMouseEvent_IsPageScroll 2742 -#define wxMouseEvent_Leaving 2743 -#define wxMouseEvent_LeftDClick 2744 -#define wxMouseEvent_LeftDown 2745 -#define wxMouseEvent_LeftIsDown 2746 -#define wxMouseEvent_LeftUp 2747 -#define wxMouseEvent_MetaDown 2748 -#define wxMouseEvent_MiddleDClick 2749 -#define wxMouseEvent_MiddleDown 2750 -#define wxMouseEvent_MiddleIsDown 2751 -#define wxMouseEvent_MiddleUp 2752 -#define wxMouseEvent_Moving 2753 -#define wxMouseEvent_RightDClick 2754 -#define wxMouseEvent_RightDown 2755 -#define wxMouseEvent_RightIsDown 2756 -#define wxMouseEvent_RightUp 2757 -#define wxMouseEvent_ShiftDown 2758 -#define wxSetCursorEvent_GetCursor 2759 -#define wxSetCursorEvent_GetX 2760 -#define wxSetCursorEvent_GetY 2761 -#define wxSetCursorEvent_HasCursor 2762 -#define wxSetCursorEvent_SetCursor 2763 -#define wxKeyEvent_AltDown 2764 -#define wxKeyEvent_CmdDown 2765 -#define wxKeyEvent_ControlDown 2766 -#define wxKeyEvent_GetKeyCode 2767 -#define wxKeyEvent_GetModifiers 2768 -#define wxKeyEvent_GetPosition 2771 -#define wxKeyEvent_GetRawKeyCode 2772 -#define wxKeyEvent_GetRawKeyFlags 2773 -#define wxKeyEvent_GetUnicodeKey 2774 -#define wxKeyEvent_GetX 2775 -#define wxKeyEvent_GetY 2776 -#define wxKeyEvent_HasModifiers 2777 -#define wxKeyEvent_MetaDown 2778 -#define wxKeyEvent_ShiftDown 2779 -#define wxSizeEvent_GetSize 2780 -#define wxMoveEvent_GetPosition 2781 -#define wxEraseEvent_GetDC 2782 -#define wxFocusEvent_GetWindow 2783 -#define wxChildFocusEvent_GetWindow 2784 -#define wxMenuEvent_GetMenu 2785 -#define wxMenuEvent_GetMenuId 2786 -#define wxMenuEvent_IsPopup 2787 -#define wxCloseEvent_CanVeto 2788 -#define wxCloseEvent_GetLoggingOff 2789 -#define wxCloseEvent_SetCanVeto 2790 -#define wxCloseEvent_SetLoggingOff 2791 -#define wxCloseEvent_Veto 2792 -#define wxShowEvent_SetShow 2793 -#define wxShowEvent_GetShow 2794 -#define wxIconizeEvent_Iconized 2795 -#define wxJoystickEvent_ButtonDown 2796 -#define wxJoystickEvent_ButtonIsDown 2797 -#define wxJoystickEvent_ButtonUp 2798 -#define wxJoystickEvent_GetButtonChange 2799 -#define wxJoystickEvent_GetButtonState 2800 -#define wxJoystickEvent_GetJoystick 2801 -#define wxJoystickEvent_GetPosition 2802 -#define wxJoystickEvent_GetZPosition 2803 -#define wxJoystickEvent_IsButton 2804 -#define wxJoystickEvent_IsMove 2805 -#define wxJoystickEvent_IsZMove 2806 -#define wxUpdateUIEvent_CanUpdate 2807 -#define wxUpdateUIEvent_Check 2808 -#define wxUpdateUIEvent_Enable 2809 -#define wxUpdateUIEvent_Show 2810 -#define wxUpdateUIEvent_GetChecked 2811 -#define wxUpdateUIEvent_GetEnabled 2812 -#define wxUpdateUIEvent_GetShown 2813 -#define wxUpdateUIEvent_GetSetChecked 2814 -#define wxUpdateUIEvent_GetSetEnabled 2815 -#define wxUpdateUIEvent_GetSetShown 2816 -#define wxUpdateUIEvent_GetSetText 2817 -#define wxUpdateUIEvent_GetText 2818 -#define wxUpdateUIEvent_GetMode 2819 -#define wxUpdateUIEvent_GetUpdateInterval 2820 -#define wxUpdateUIEvent_ResetUpdateTime 2821 -#define wxUpdateUIEvent_SetMode 2822 -#define wxUpdateUIEvent_SetText 2823 -#define wxUpdateUIEvent_SetUpdateInterval 2824 -#define wxMouseCaptureChangedEvent_GetCapturedWindow 2825 -#define wxPaletteChangedEvent_SetChangedWindow 2826 -#define wxPaletteChangedEvent_GetChangedWindow 2827 -#define wxQueryNewPaletteEvent_SetPaletteRealized 2828 -#define wxQueryNewPaletteEvent_GetPaletteRealized 2829 -#define wxNavigationKeyEvent_GetDirection 2830 -#define wxNavigationKeyEvent_SetDirection 2831 -#define wxNavigationKeyEvent_IsWindowChange 2832 -#define wxNavigationKeyEvent_SetWindowChange 2833 -#define wxNavigationKeyEvent_IsFromTab 2834 -#define wxNavigationKeyEvent_SetFromTab 2835 -#define wxNavigationKeyEvent_GetCurrentFocus 2836 -#define wxNavigationKeyEvent_SetCurrentFocus 2837 -#define wxHelpEvent_GetOrigin 2838 -#define wxHelpEvent_GetPosition 2839 -#define wxHelpEvent_SetOrigin 2840 -#define wxHelpEvent_SetPosition 2841 -#define wxContextMenuEvent_GetPosition 2842 -#define wxContextMenuEvent_SetPosition 2843 -#define wxIdleEvent_CanSend 2844 -#define wxIdleEvent_GetMode 2845 -#define wxIdleEvent_RequestMore 2846 -#define wxIdleEvent_MoreRequested 2847 -#define wxIdleEvent_SetMode 2848 -#define wxGridEvent_AltDown 2849 -#define wxGridEvent_ControlDown 2850 -#define wxGridEvent_GetCol 2851 -#define wxGridEvent_GetPosition 2852 -#define wxGridEvent_GetRow 2853 -#define wxGridEvent_MetaDown 2854 -#define wxGridEvent_Selecting 2855 -#define wxGridEvent_ShiftDown 2856 -#define wxNotifyEvent_Allow 2857 -#define wxNotifyEvent_IsAllowed 2858 -#define wxNotifyEvent_Veto 2859 -#define wxSashEvent_GetEdge 2860 -#define wxSashEvent_GetDragRect 2861 -#define wxSashEvent_GetDragStatus 2862 -#define wxListEvent_GetCacheFrom 2863 -#define wxListEvent_GetCacheTo 2864 -#define wxListEvent_GetKeyCode 2865 -#define wxListEvent_GetIndex 2866 -#define wxListEvent_GetColumn 2867 -#define wxListEvent_GetPoint 2868 -#define wxListEvent_GetLabel 2869 -#define wxListEvent_GetText 2870 -#define wxListEvent_GetImage 2871 -#define wxListEvent_GetData 2872 -#define wxListEvent_GetMask 2873 -#define wxListEvent_GetItem 2874 -#define wxListEvent_IsEditCancelled 2875 -#define wxDateEvent_GetDate 2876 -#define wxCalendarEvent_GetWeekDay 2877 -#define wxFileDirPickerEvent_GetPath 2878 -#define wxColourPickerEvent_GetColour 2879 -#define wxFontPickerEvent_GetFont 2880 -#define wxStyledTextEvent_GetPosition 2881 -#define wxStyledTextEvent_GetKey 2882 -#define wxStyledTextEvent_GetModifiers 2883 -#define wxStyledTextEvent_GetModificationType 2884 -#define wxStyledTextEvent_GetText 2885 -#define wxStyledTextEvent_GetLength 2886 -#define wxStyledTextEvent_GetLinesAdded 2887 -#define wxStyledTextEvent_GetLine 2888 -#define wxStyledTextEvent_GetFoldLevelNow 2889 -#define wxStyledTextEvent_GetFoldLevelPrev 2890 -#define wxStyledTextEvent_GetMargin 2891 -#define wxStyledTextEvent_GetMessage 2892 -#define wxStyledTextEvent_GetWParam 2893 -#define wxStyledTextEvent_GetLParam 2894 -#define wxStyledTextEvent_GetListType 2895 -#define wxStyledTextEvent_GetX 2896 -#define wxStyledTextEvent_GetY 2897 -#define wxStyledTextEvent_GetDragText 2898 -#define wxStyledTextEvent_GetDragAllowMove 2899 -#define wxStyledTextEvent_GetDragResult 2900 -#define wxStyledTextEvent_GetShift 2901 -#define wxStyledTextEvent_GetControl 2902 -#define wxStyledTextEvent_GetAlt 2903 -#define utils_wxGetKeyState 2904 -#define utils_wxGetMousePosition 2905 -#define utils_wxGetMouseState 2906 -#define utils_wxSetDetectableAutoRepeat 2907 -#define utils_wxBell 2908 -#define utils_wxFindMenuItemId 2909 -#define utils_wxGenericFindWindowAtPoint 2910 -#define utils_wxFindWindowAtPoint 2911 -#define utils_wxBeginBusyCursor 2912 -#define utils_wxEndBusyCursor 2913 -#define utils_wxIsBusy 2914 -#define utils_wxShutdown 2915 -#define utils_wxShell 2916 -#define utils_wxLaunchDefaultBrowser 2917 -#define utils_wxGetEmailAddress 2918 -#define utils_wxGetUserId 2919 -#define utils_wxGetHomeDir 2920 -#define utils_wxNewId 2921 -#define utils_wxRegisterId 2922 -#define utils_wxGetCurrentId 2923 -#define utils_wxGetOsDescription 2924 -#define utils_wxIsPlatformLittleEndian 2925 -#define utils_wxIsPlatform64Bit 2926 -#define wxPrintout_new 2927 -#define wxPrintout_destruct 2928 -#define wxPrintout_GetDC 2929 -#define wxPrintout_GetPageSizeMM 2930 -#define wxPrintout_GetPageSizePixels 2931 -#define wxPrintout_GetPaperRectPixels 2932 -#define wxPrintout_GetPPIPrinter 2933 -#define wxPrintout_GetPPIScreen 2934 -#define wxPrintout_GetTitle 2935 -#define wxPrintout_IsPreview 2936 -#define wxPrintout_FitThisSizeToPaper 2937 -#define wxPrintout_FitThisSizeToPage 2938 -#define wxPrintout_FitThisSizeToPageMargins 2939 -#define wxPrintout_MapScreenSizeToPaper 2940 -#define wxPrintout_MapScreenSizeToPage 2941 -#define wxPrintout_MapScreenSizeToPageMargins 2942 -#define wxPrintout_MapScreenSizeToDevice 2943 -#define wxPrintout_GetLogicalPaperRect 2944 -#define wxPrintout_GetLogicalPageRect 2945 -#define wxPrintout_GetLogicalPageMarginsRect 2946 -#define wxPrintout_SetLogicalOrigin 2947 -#define wxPrintout_OffsetLogicalOrigin 2948 -#define wxStyledTextCtrl_new_2 2949 -#define wxStyledTextCtrl_new_0 2950 -#define wxStyledTextCtrl_destruct 2951 -#define wxStyledTextCtrl_Create 2952 -#define wxStyledTextCtrl_AddText 2953 -#define wxStyledTextCtrl_AddStyledText 2954 -#define wxStyledTextCtrl_InsertText 2955 -#define wxStyledTextCtrl_ClearAll 2956 -#define wxStyledTextCtrl_ClearDocumentStyle 2957 -#define wxStyledTextCtrl_GetLength 2958 -#define wxStyledTextCtrl_GetCharAt 2959 -#define wxStyledTextCtrl_GetCurrentPos 2960 -#define wxStyledTextCtrl_GetAnchor 2961 -#define wxStyledTextCtrl_GetStyleAt 2962 -#define wxStyledTextCtrl_Redo 2963 -#define wxStyledTextCtrl_SetUndoCollection 2964 -#define wxStyledTextCtrl_SelectAll 2965 -#define wxStyledTextCtrl_SetSavePoint 2966 -#define wxStyledTextCtrl_GetStyledText 2967 -#define wxStyledTextCtrl_CanRedo 2968 -#define wxStyledTextCtrl_MarkerLineFromHandle 2969 -#define wxStyledTextCtrl_MarkerDeleteHandle 2970 -#define wxStyledTextCtrl_GetUndoCollection 2971 -#define wxStyledTextCtrl_GetViewWhiteSpace 2972 -#define wxStyledTextCtrl_SetViewWhiteSpace 2973 -#define wxStyledTextCtrl_PositionFromPoint 2974 -#define wxStyledTextCtrl_PositionFromPointClose 2975 -#define wxStyledTextCtrl_GotoLine 2976 -#define wxStyledTextCtrl_GotoPos 2977 -#define wxStyledTextCtrl_SetAnchor 2978 -#define wxStyledTextCtrl_GetCurLine 2979 -#define wxStyledTextCtrl_GetEndStyled 2980 -#define wxStyledTextCtrl_ConvertEOLs 2981 -#define wxStyledTextCtrl_GetEOLMode 2982 -#define wxStyledTextCtrl_SetEOLMode 2983 -#define wxStyledTextCtrl_StartStyling 2984 -#define wxStyledTextCtrl_SetStyling 2985 -#define wxStyledTextCtrl_GetBufferedDraw 2986 -#define wxStyledTextCtrl_SetBufferedDraw 2987 -#define wxStyledTextCtrl_SetTabWidth 2988 -#define wxStyledTextCtrl_GetTabWidth 2989 -#define wxStyledTextCtrl_SetCodePage 2990 -#define wxStyledTextCtrl_MarkerDefine 2991 -#define wxStyledTextCtrl_MarkerSetForeground 2992 -#define wxStyledTextCtrl_MarkerSetBackground 2993 -#define wxStyledTextCtrl_MarkerAdd 2994 -#define wxStyledTextCtrl_MarkerDelete 2995 -#define wxStyledTextCtrl_MarkerDeleteAll 2996 -#define wxStyledTextCtrl_MarkerGet 2997 -#define wxStyledTextCtrl_MarkerNext 2998 -#define wxStyledTextCtrl_MarkerPrevious 2999 -#define wxStyledTextCtrl_MarkerDefineBitmap 3000 -#define wxStyledTextCtrl_MarkerAddSet 3001 -#define wxStyledTextCtrl_MarkerSetAlpha 3002 -#define wxStyledTextCtrl_SetMarginType 3003 -#define wxStyledTextCtrl_GetMarginType 3004 -#define wxStyledTextCtrl_SetMarginWidth 3005 -#define wxStyledTextCtrl_GetMarginWidth 3006 -#define wxStyledTextCtrl_SetMarginMask 3007 -#define wxStyledTextCtrl_GetMarginMask 3008 -#define wxStyledTextCtrl_SetMarginSensitive 3009 -#define wxStyledTextCtrl_GetMarginSensitive 3010 -#define wxStyledTextCtrl_StyleClearAll 3011 -#define wxStyledTextCtrl_StyleSetForeground 3012 -#define wxStyledTextCtrl_StyleSetBackground 3013 -#define wxStyledTextCtrl_StyleSetBold 3014 -#define wxStyledTextCtrl_StyleSetItalic 3015 -#define wxStyledTextCtrl_StyleSetSize 3016 -#define wxStyledTextCtrl_StyleSetFaceName 3017 -#define wxStyledTextCtrl_StyleSetEOLFilled 3018 -#define wxStyledTextCtrl_StyleResetDefault 3019 -#define wxStyledTextCtrl_StyleSetUnderline 3020 -#define wxStyledTextCtrl_StyleSetCase 3021 -#define wxStyledTextCtrl_StyleSetHotSpot 3022 -#define wxStyledTextCtrl_SetSelForeground 3023 -#define wxStyledTextCtrl_SetSelBackground 3024 -#define wxStyledTextCtrl_GetSelAlpha 3025 -#define wxStyledTextCtrl_SetSelAlpha 3026 -#define wxStyledTextCtrl_SetCaretForeground 3027 -#define wxStyledTextCtrl_CmdKeyAssign 3028 -#define wxStyledTextCtrl_CmdKeyClear 3029 -#define wxStyledTextCtrl_CmdKeyClearAll 3030 -#define wxStyledTextCtrl_SetStyleBytes 3031 -#define wxStyledTextCtrl_StyleSetVisible 3032 -#define wxStyledTextCtrl_GetCaretPeriod 3033 -#define wxStyledTextCtrl_SetCaretPeriod 3034 -#define wxStyledTextCtrl_SetWordChars 3035 -#define wxStyledTextCtrl_BeginUndoAction 3036 -#define wxStyledTextCtrl_EndUndoAction 3037 -#define wxStyledTextCtrl_IndicatorSetStyle 3038 -#define wxStyledTextCtrl_IndicatorGetStyle 3039 -#define wxStyledTextCtrl_IndicatorSetForeground 3040 -#define wxStyledTextCtrl_IndicatorGetForeground 3041 -#define wxStyledTextCtrl_SetWhitespaceForeground 3042 -#define wxStyledTextCtrl_SetWhitespaceBackground 3043 -#define wxStyledTextCtrl_GetStyleBits 3044 -#define wxStyledTextCtrl_SetLineState 3045 -#define wxStyledTextCtrl_GetLineState 3046 -#define wxStyledTextCtrl_GetMaxLineState 3047 -#define wxStyledTextCtrl_GetCaretLineVisible 3048 -#define wxStyledTextCtrl_SetCaretLineVisible 3049 -#define wxStyledTextCtrl_GetCaretLineBackground 3050 -#define wxStyledTextCtrl_SetCaretLineBackground 3051 -#define wxStyledTextCtrl_AutoCompShow 3052 -#define wxStyledTextCtrl_AutoCompCancel 3053 -#define wxStyledTextCtrl_AutoCompActive 3054 -#define wxStyledTextCtrl_AutoCompPosStart 3055 -#define wxStyledTextCtrl_AutoCompComplete 3056 -#define wxStyledTextCtrl_AutoCompStops 3057 -#define wxStyledTextCtrl_AutoCompSetSeparator 3058 -#define wxStyledTextCtrl_AutoCompGetSeparator 3059 -#define wxStyledTextCtrl_AutoCompSelect 3060 -#define wxStyledTextCtrl_AutoCompSetCancelAtStart 3061 -#define wxStyledTextCtrl_AutoCompGetCancelAtStart 3062 -#define wxStyledTextCtrl_AutoCompSetFillUps 3063 -#define wxStyledTextCtrl_AutoCompSetChooseSingle 3064 -#define wxStyledTextCtrl_AutoCompGetChooseSingle 3065 -#define wxStyledTextCtrl_AutoCompSetIgnoreCase 3066 -#define wxStyledTextCtrl_AutoCompGetIgnoreCase 3067 -#define wxStyledTextCtrl_UserListShow 3068 -#define wxStyledTextCtrl_AutoCompSetAutoHide 3069 -#define wxStyledTextCtrl_AutoCompGetAutoHide 3070 -#define wxStyledTextCtrl_AutoCompSetDropRestOfWord 3071 -#define wxStyledTextCtrl_AutoCompGetDropRestOfWord 3072 -#define wxStyledTextCtrl_RegisterImage 3073 -#define wxStyledTextCtrl_ClearRegisteredImages 3074 -#define wxStyledTextCtrl_AutoCompGetTypeSeparator 3075 -#define wxStyledTextCtrl_AutoCompSetTypeSeparator 3076 -#define wxStyledTextCtrl_AutoCompSetMaxWidth 3077 -#define wxStyledTextCtrl_AutoCompGetMaxWidth 3078 -#define wxStyledTextCtrl_AutoCompSetMaxHeight 3079 -#define wxStyledTextCtrl_AutoCompGetMaxHeight 3080 -#define wxStyledTextCtrl_SetIndent 3081 -#define wxStyledTextCtrl_GetIndent 3082 -#define wxStyledTextCtrl_SetUseTabs 3083 -#define wxStyledTextCtrl_GetUseTabs 3084 -#define wxStyledTextCtrl_SetLineIndentation 3085 -#define wxStyledTextCtrl_GetLineIndentation 3086 -#define wxStyledTextCtrl_GetLineIndentPosition 3087 -#define wxStyledTextCtrl_GetColumn 3088 -#define wxStyledTextCtrl_SetUseHorizontalScrollBar 3089 -#define wxStyledTextCtrl_GetUseHorizontalScrollBar 3090 -#define wxStyledTextCtrl_SetIndentationGuides 3091 -#define wxStyledTextCtrl_GetIndentationGuides 3092 -#define wxStyledTextCtrl_SetHighlightGuide 3093 -#define wxStyledTextCtrl_GetHighlightGuide 3094 -#define wxStyledTextCtrl_GetLineEndPosition 3095 -#define wxStyledTextCtrl_GetCodePage 3096 -#define wxStyledTextCtrl_GetCaretForeground 3097 -#define wxStyledTextCtrl_GetReadOnly 3098 -#define wxStyledTextCtrl_SetCurrentPos 3099 -#define wxStyledTextCtrl_SetSelectionStart 3100 -#define wxStyledTextCtrl_GetSelectionStart 3101 -#define wxStyledTextCtrl_SetSelectionEnd 3102 -#define wxStyledTextCtrl_GetSelectionEnd 3103 -#define wxStyledTextCtrl_SetPrintMagnification 3104 -#define wxStyledTextCtrl_GetPrintMagnification 3105 -#define wxStyledTextCtrl_SetPrintColourMode 3106 -#define wxStyledTextCtrl_GetPrintColourMode 3107 -#define wxStyledTextCtrl_FindText 3108 -#define wxStyledTextCtrl_FormatRange 3109 -#define wxStyledTextCtrl_GetFirstVisibleLine 3110 -#define wxStyledTextCtrl_GetLine 3111 -#define wxStyledTextCtrl_GetLineCount 3112 -#define wxStyledTextCtrl_SetMarginLeft 3113 -#define wxStyledTextCtrl_GetMarginLeft 3114 -#define wxStyledTextCtrl_SetMarginRight 3115 -#define wxStyledTextCtrl_GetMarginRight 3116 -#define wxStyledTextCtrl_GetModify 3117 -#define wxStyledTextCtrl_SetSelection 3118 -#define wxStyledTextCtrl_GetSelectedText 3119 -#define wxStyledTextCtrl_GetTextRange 3120 -#define wxStyledTextCtrl_HideSelection 3121 -#define wxStyledTextCtrl_LineFromPosition 3122 -#define wxStyledTextCtrl_PositionFromLine 3123 -#define wxStyledTextCtrl_LineScroll 3124 -#define wxStyledTextCtrl_EnsureCaretVisible 3125 -#define wxStyledTextCtrl_ReplaceSelection 3126 -#define wxStyledTextCtrl_SetReadOnly 3127 -#define wxStyledTextCtrl_CanPaste 3128 -#define wxStyledTextCtrl_CanUndo 3129 -#define wxStyledTextCtrl_EmptyUndoBuffer 3130 -#define wxStyledTextCtrl_Undo 3131 -#define wxStyledTextCtrl_Cut 3132 -#define wxStyledTextCtrl_Copy 3133 -#define wxStyledTextCtrl_Paste 3134 -#define wxStyledTextCtrl_Clear 3135 -#define wxStyledTextCtrl_SetText 3136 -#define wxStyledTextCtrl_GetText 3137 -#define wxStyledTextCtrl_GetTextLength 3138 -#define wxStyledTextCtrl_GetOvertype 3139 -#define wxStyledTextCtrl_SetCaretWidth 3140 -#define wxStyledTextCtrl_GetCaretWidth 3141 -#define wxStyledTextCtrl_SetTargetStart 3142 -#define wxStyledTextCtrl_GetTargetStart 3143 -#define wxStyledTextCtrl_SetTargetEnd 3144 -#define wxStyledTextCtrl_GetTargetEnd 3145 -#define wxStyledTextCtrl_ReplaceTarget 3146 -#define wxStyledTextCtrl_SearchInTarget 3147 -#define wxStyledTextCtrl_SetSearchFlags 3148 -#define wxStyledTextCtrl_GetSearchFlags 3149 -#define wxStyledTextCtrl_CallTipShow 3150 -#define wxStyledTextCtrl_CallTipCancel 3151 -#define wxStyledTextCtrl_CallTipActive 3152 -#define wxStyledTextCtrl_CallTipPosAtStart 3153 -#define wxStyledTextCtrl_CallTipSetHighlight 3154 -#define wxStyledTextCtrl_CallTipSetBackground 3155 -#define wxStyledTextCtrl_CallTipSetForeground 3156 -#define wxStyledTextCtrl_CallTipSetForegroundHighlight 3157 -#define wxStyledTextCtrl_CallTipUseStyle 3158 -#define wxStyledTextCtrl_VisibleFromDocLine 3159 -#define wxStyledTextCtrl_DocLineFromVisible 3160 -#define wxStyledTextCtrl_WrapCount 3161 -#define wxStyledTextCtrl_SetFoldLevel 3162 -#define wxStyledTextCtrl_GetFoldLevel 3163 -#define wxStyledTextCtrl_GetLastChild 3164 -#define wxStyledTextCtrl_GetFoldParent 3165 -#define wxStyledTextCtrl_ShowLines 3166 -#define wxStyledTextCtrl_HideLines 3167 -#define wxStyledTextCtrl_GetLineVisible 3168 -#define wxStyledTextCtrl_SetFoldExpanded 3169 -#define wxStyledTextCtrl_GetFoldExpanded 3170 -#define wxStyledTextCtrl_ToggleFold 3171 -#define wxStyledTextCtrl_EnsureVisible 3172 -#define wxStyledTextCtrl_SetFoldFlags 3173 -#define wxStyledTextCtrl_EnsureVisibleEnforcePolicy 3174 -#define wxStyledTextCtrl_SetTabIndents 3175 -#define wxStyledTextCtrl_GetTabIndents 3176 -#define wxStyledTextCtrl_SetBackSpaceUnIndents 3177 -#define wxStyledTextCtrl_GetBackSpaceUnIndents 3178 -#define wxStyledTextCtrl_SetMouseDwellTime 3179 -#define wxStyledTextCtrl_GetMouseDwellTime 3180 -#define wxStyledTextCtrl_WordStartPosition 3181 -#define wxStyledTextCtrl_WordEndPosition 3182 -#define wxStyledTextCtrl_SetWrapMode 3183 -#define wxStyledTextCtrl_GetWrapMode 3184 -#define wxStyledTextCtrl_SetWrapVisualFlags 3185 -#define wxStyledTextCtrl_GetWrapVisualFlags 3186 -#define wxStyledTextCtrl_SetWrapVisualFlagsLocation 3187 -#define wxStyledTextCtrl_GetWrapVisualFlagsLocation 3188 -#define wxStyledTextCtrl_SetWrapStartIndent 3189 -#define wxStyledTextCtrl_GetWrapStartIndent 3190 -#define wxStyledTextCtrl_SetLayoutCache 3191 -#define wxStyledTextCtrl_GetLayoutCache 3192 -#define wxStyledTextCtrl_SetScrollWidth 3193 -#define wxStyledTextCtrl_GetScrollWidth 3194 -#define wxStyledTextCtrl_TextWidth 3195 -#define wxStyledTextCtrl_GetEndAtLastLine 3196 -#define wxStyledTextCtrl_TextHeight 3197 -#define wxStyledTextCtrl_SetUseVerticalScrollBar 3198 -#define wxStyledTextCtrl_GetUseVerticalScrollBar 3199 -#define wxStyledTextCtrl_AppendText 3200 -#define wxStyledTextCtrl_GetTwoPhaseDraw 3201 -#define wxStyledTextCtrl_SetTwoPhaseDraw 3202 -#define wxStyledTextCtrl_TargetFromSelection 3203 -#define wxStyledTextCtrl_LinesJoin 3204 -#define wxStyledTextCtrl_LinesSplit 3205 -#define wxStyledTextCtrl_SetFoldMarginColour 3206 -#define wxStyledTextCtrl_SetFoldMarginHiColour 3207 -#define wxStyledTextCtrl_LineDown 3208 -#define wxStyledTextCtrl_LineDownExtend 3209 -#define wxStyledTextCtrl_LineUp 3210 -#define wxStyledTextCtrl_LineUpExtend 3211 -#define wxStyledTextCtrl_CharLeft 3212 -#define wxStyledTextCtrl_CharLeftExtend 3213 -#define wxStyledTextCtrl_CharRight 3214 -#define wxStyledTextCtrl_CharRightExtend 3215 -#define wxStyledTextCtrl_WordLeft 3216 -#define wxStyledTextCtrl_WordLeftExtend 3217 -#define wxStyledTextCtrl_WordRight 3218 -#define wxStyledTextCtrl_WordRightExtend 3219 -#define wxStyledTextCtrl_Home 3220 -#define wxStyledTextCtrl_HomeExtend 3221 -#define wxStyledTextCtrl_LineEnd 3222 -#define wxStyledTextCtrl_LineEndExtend 3223 -#define wxStyledTextCtrl_DocumentStart 3224 -#define wxStyledTextCtrl_DocumentStartExtend 3225 -#define wxStyledTextCtrl_DocumentEnd 3226 -#define wxStyledTextCtrl_DocumentEndExtend 3227 -#define wxStyledTextCtrl_PageUp 3228 -#define wxStyledTextCtrl_PageUpExtend 3229 -#define wxStyledTextCtrl_PageDown 3230 -#define wxStyledTextCtrl_PageDownExtend 3231 -#define wxStyledTextCtrl_EditToggleOvertype 3232 -#define wxStyledTextCtrl_Cancel 3233 -#define wxStyledTextCtrl_DeleteBack 3234 -#define wxStyledTextCtrl_Tab 3235 -#define wxStyledTextCtrl_BackTab 3236 -#define wxStyledTextCtrl_NewLine 3237 -#define wxStyledTextCtrl_FormFeed 3238 -#define wxStyledTextCtrl_VCHome 3239 -#define wxStyledTextCtrl_VCHomeExtend 3240 -#define wxStyledTextCtrl_ZoomIn 3241 -#define wxStyledTextCtrl_ZoomOut 3242 -#define wxStyledTextCtrl_DelWordLeft 3243 -#define wxStyledTextCtrl_DelWordRight 3244 -#define wxStyledTextCtrl_LineCut 3245 -#define wxStyledTextCtrl_LineDelete 3246 -#define wxStyledTextCtrl_LineTranspose 3247 -#define wxStyledTextCtrl_LineDuplicate 3248 -#define wxStyledTextCtrl_LowerCase 3249 -#define wxStyledTextCtrl_UpperCase 3250 -#define wxStyledTextCtrl_LineScrollDown 3251 -#define wxStyledTextCtrl_LineScrollUp 3252 -#define wxStyledTextCtrl_DeleteBackNotLine 3253 -#define wxStyledTextCtrl_HomeDisplay 3254 -#define wxStyledTextCtrl_HomeDisplayExtend 3255 -#define wxStyledTextCtrl_LineEndDisplay 3256 -#define wxStyledTextCtrl_LineEndDisplayExtend 3257 -#define wxStyledTextCtrl_HomeWrapExtend 3258 -#define wxStyledTextCtrl_LineEndWrap 3259 -#define wxStyledTextCtrl_LineEndWrapExtend 3260 -#define wxStyledTextCtrl_VCHomeWrap 3261 -#define wxStyledTextCtrl_VCHomeWrapExtend 3262 -#define wxStyledTextCtrl_LineCopy 3263 -#define wxStyledTextCtrl_MoveCaretInsideView 3264 -#define wxStyledTextCtrl_LineLength 3265 -#define wxStyledTextCtrl_BraceHighlight 3266 -#define wxStyledTextCtrl_BraceBadLight 3267 -#define wxStyledTextCtrl_BraceMatch 3268 -#define wxStyledTextCtrl_GetViewEOL 3269 -#define wxStyledTextCtrl_SetViewEOL 3270 -#define wxStyledTextCtrl_SetModEventMask 3271 -#define wxStyledTextCtrl_GetEdgeColumn 3272 -#define wxStyledTextCtrl_SetEdgeColumn 3273 -#define wxStyledTextCtrl_SetEdgeMode 3274 -#define wxStyledTextCtrl_GetEdgeMode 3275 -#define wxStyledTextCtrl_GetEdgeColour 3276 -#define wxStyledTextCtrl_SetEdgeColour 3277 -#define wxStyledTextCtrl_SearchAnchor 3278 -#define wxStyledTextCtrl_SearchNext 3279 -#define wxStyledTextCtrl_SearchPrev 3280 -#define wxStyledTextCtrl_LinesOnScreen 3281 -#define wxStyledTextCtrl_UsePopUp 3282 -#define wxStyledTextCtrl_SelectionIsRectangle 3283 -#define wxStyledTextCtrl_SetZoom 3284 -#define wxStyledTextCtrl_GetZoom 3285 -#define wxStyledTextCtrl_GetModEventMask 3286 -#define wxStyledTextCtrl_SetSTCFocus 3287 -#define wxStyledTextCtrl_GetSTCFocus 3288 -#define wxStyledTextCtrl_SetStatus 3289 -#define wxStyledTextCtrl_GetStatus 3290 -#define wxStyledTextCtrl_SetMouseDownCaptures 3291 -#define wxStyledTextCtrl_GetMouseDownCaptures 3292 -#define wxStyledTextCtrl_SetSTCCursor 3293 -#define wxStyledTextCtrl_GetSTCCursor 3294 -#define wxStyledTextCtrl_SetControlCharSymbol 3295 -#define wxStyledTextCtrl_GetControlCharSymbol 3296 -#define wxStyledTextCtrl_WordPartLeft 3297 -#define wxStyledTextCtrl_WordPartLeftExtend 3298 -#define wxStyledTextCtrl_WordPartRight 3299 -#define wxStyledTextCtrl_WordPartRightExtend 3300 -#define wxStyledTextCtrl_SetVisiblePolicy 3301 -#define wxStyledTextCtrl_DelLineLeft 3302 -#define wxStyledTextCtrl_DelLineRight 3303 -#define wxStyledTextCtrl_GetXOffset 3304 -#define wxStyledTextCtrl_ChooseCaretX 3305 -#define wxStyledTextCtrl_SetXCaretPolicy 3306 -#define wxStyledTextCtrl_SetYCaretPolicy 3307 -#define wxStyledTextCtrl_GetPrintWrapMode 3308 -#define wxStyledTextCtrl_SetHotspotActiveForeground 3309 -#define wxStyledTextCtrl_SetHotspotActiveBackground 3310 -#define wxStyledTextCtrl_SetHotspotActiveUnderline 3311 -#define wxStyledTextCtrl_SetHotspotSingleLine 3312 -#define wxStyledTextCtrl_ParaDownExtend 3313 -#define wxStyledTextCtrl_ParaUp 3314 -#define wxStyledTextCtrl_ParaUpExtend 3315 -#define wxStyledTextCtrl_PositionBefore 3316 -#define wxStyledTextCtrl_PositionAfter 3317 -#define wxStyledTextCtrl_CopyRange 3318 -#define wxStyledTextCtrl_CopyText 3319 -#define wxStyledTextCtrl_SetSelectionMode 3320 -#define wxStyledTextCtrl_GetSelectionMode 3321 -#define wxStyledTextCtrl_LineDownRectExtend 3322 -#define wxStyledTextCtrl_LineUpRectExtend 3323 -#define wxStyledTextCtrl_CharLeftRectExtend 3324 -#define wxStyledTextCtrl_CharRightRectExtend 3325 -#define wxStyledTextCtrl_HomeRectExtend 3326 -#define wxStyledTextCtrl_VCHomeRectExtend 3327 -#define wxStyledTextCtrl_LineEndRectExtend 3328 -#define wxStyledTextCtrl_PageUpRectExtend 3329 -#define wxStyledTextCtrl_PageDownRectExtend 3330 -#define wxStyledTextCtrl_StutteredPageUp 3331 -#define wxStyledTextCtrl_StutteredPageUpExtend 3332 -#define wxStyledTextCtrl_StutteredPageDown 3333 -#define wxStyledTextCtrl_StutteredPageDownExtend 3334 -#define wxStyledTextCtrl_WordLeftEnd 3335 -#define wxStyledTextCtrl_WordLeftEndExtend 3336 -#define wxStyledTextCtrl_WordRightEnd 3337 -#define wxStyledTextCtrl_WordRightEndExtend 3338 -#define wxStyledTextCtrl_SetWhitespaceChars 3339 -#define wxStyledTextCtrl_SetCharsDefault 3340 -#define wxStyledTextCtrl_AutoCompGetCurrent 3341 -#define wxStyledTextCtrl_Allocate 3342 -#define wxStyledTextCtrl_FindColumn 3343 -#define wxStyledTextCtrl_GetCaretSticky 3344 -#define wxStyledTextCtrl_SetCaretSticky 3345 -#define wxStyledTextCtrl_ToggleCaretSticky 3346 -#define wxStyledTextCtrl_SetPasteConvertEndings 3347 -#define wxStyledTextCtrl_GetPasteConvertEndings 3348 -#define wxStyledTextCtrl_SelectionDuplicate 3349 -#define wxStyledTextCtrl_SetCaretLineBackAlpha 3350 -#define wxStyledTextCtrl_GetCaretLineBackAlpha 3351 -#define wxStyledTextCtrl_StartRecord 3352 -#define wxStyledTextCtrl_StopRecord 3353 -#define wxStyledTextCtrl_SetLexer 3354 -#define wxStyledTextCtrl_GetLexer 3355 -#define wxStyledTextCtrl_Colourise 3356 -#define wxStyledTextCtrl_SetProperty 3357 -#define wxStyledTextCtrl_SetKeyWords 3358 -#define wxStyledTextCtrl_SetLexerLanguage 3359 -#define wxStyledTextCtrl_GetProperty 3360 -#define wxStyledTextCtrl_GetStyleBitsNeeded 3361 -#define wxStyledTextCtrl_GetCurrentLine 3362 -#define wxStyledTextCtrl_StyleSetSpec 3363 -#define wxStyledTextCtrl_StyleSetFont 3364 -#define wxStyledTextCtrl_StyleSetFontAttr 3365 -#define wxStyledTextCtrl_StyleSetCharacterSet 3366 -#define wxStyledTextCtrl_StyleSetFontEncoding 3367 -#define wxStyledTextCtrl_CmdKeyExecute 3368 -#define wxStyledTextCtrl_SetMargins 3369 -#define wxStyledTextCtrl_GetSelection 3370 -#define wxStyledTextCtrl_PointFromPosition 3371 -#define wxStyledTextCtrl_ScrollToLine 3372 -#define wxStyledTextCtrl_ScrollToColumn 3373 -#define wxStyledTextCtrl_SetVScrollBar 3374 -#define wxStyledTextCtrl_SetHScrollBar 3375 -#define wxStyledTextCtrl_GetLastKeydownProcessed 3376 -#define wxStyledTextCtrl_SetLastKeydownProcessed 3377 -#define wxStyledTextCtrl_SaveFile 3378 -#define wxStyledTextCtrl_LoadFile 3379 -#define wxStyledTextCtrl_DoDragOver 3380 -#define wxStyledTextCtrl_DoDropText 3381 -#define wxStyledTextCtrl_GetUseAntiAliasing 3382 -#define wxStyledTextCtrl_AddTextRaw 3383 -#define wxStyledTextCtrl_InsertTextRaw 3384 -#define wxStyledTextCtrl_GetCurLineRaw 3385 -#define wxStyledTextCtrl_GetLineRaw 3386 -#define wxStyledTextCtrl_GetSelectedTextRaw 3387 -#define wxStyledTextCtrl_GetTextRangeRaw 3388 -#define wxStyledTextCtrl_SetTextRaw 3389 -#define wxStyledTextCtrl_GetTextRaw 3390 -#define wxStyledTextCtrl_AppendTextRaw 3391 -#define wxArtProvider_GetBitmap 3392 -#define wxArtProvider_GetIcon 3393 -#define wxTreeEvent_GetKeyCode 3394 -#define wxTreeEvent_GetItem 3395 -#define wxTreeEvent_GetKeyEvent 3396 -#define wxTreeEvent_GetLabel 3397 -#define wxTreeEvent_GetOldItem 3398 -#define wxTreeEvent_GetPoint 3399 -#define wxTreeEvent_IsEditCancelled 3400 -#define wxTreeEvent_SetToolTip 3401 -#define wxNotebookEvent_GetOldSelection 3402 -#define wxNotebookEvent_GetSelection 3403 -#define wxNotebookEvent_SetOldSelection 3404 -#define wxNotebookEvent_SetSelection 3405 -#define wxFileDataObject_new 3406 -#define wxFileDataObject_AddFile 3407 -#define wxFileDataObject_GetFilenames 3408 -#define wxFileDataObject_destroy 3409 -#define wxTextDataObject_new 3410 -#define wxTextDataObject_GetTextLength 3411 -#define wxTextDataObject_GetText 3412 -#define wxTextDataObject_SetText 3413 -#define wxTextDataObject_destroy 3414 -#define wxBitmapDataObject_new_1_1 3415 -#define wxBitmapDataObject_new_1_0 3416 -#define wxBitmapDataObject_GetBitmap 3417 -#define wxBitmapDataObject_SetBitmap 3418 -#define wxBitmapDataObject_destroy 3419 -#define wxClipboard_new 3421 -#define wxClipboard_destruct 3422 -#define wxClipboard_AddData 3423 -#define wxClipboard_Clear 3424 -#define wxClipboard_Close 3425 -#define wxClipboard_Flush 3426 -#define wxClipboard_GetData 3427 -#define wxClipboard_IsOpened 3428 -#define wxClipboard_Open 3429 -#define wxClipboard_SetData 3430 -#define wxClipboard_UsePrimarySelection 3432 -#define wxClipboard_IsSupported 3433 -#define wxClipboard_Get 3434 -#define wxSpinEvent_GetPosition 3435 -#define wxSpinEvent_SetPosition 3436 -#define wxSplitterWindow_new_0 3437 -#define wxSplitterWindow_new_2 3438 -#define wxSplitterWindow_destruct 3439 -#define wxSplitterWindow_Create 3440 -#define wxSplitterWindow_GetMinimumPaneSize 3441 -#define wxSplitterWindow_GetSashGravity 3442 -#define wxSplitterWindow_GetSashPosition 3443 -#define wxSplitterWindow_GetSplitMode 3444 -#define wxSplitterWindow_GetWindow1 3445 -#define wxSplitterWindow_GetWindow2 3446 -#define wxSplitterWindow_Initialize 3447 -#define wxSplitterWindow_IsSplit 3448 -#define wxSplitterWindow_ReplaceWindow 3449 -#define wxSplitterWindow_SetSashGravity 3450 -#define wxSplitterWindow_SetSashPosition 3451 -#define wxSplitterWindow_SetSashSize 3452 -#define wxSplitterWindow_SetMinimumPaneSize 3453 -#define wxSplitterWindow_SetSplitMode 3454 -#define wxSplitterWindow_SplitHorizontally 3455 -#define wxSplitterWindow_SplitVertically 3456 -#define wxSplitterWindow_Unsplit 3457 -#define wxSplitterWindow_UpdateSize 3458 -#define wxSplitterEvent_GetSashPosition 3459 -#define wxSplitterEvent_GetX 3460 -#define wxSplitterEvent_GetY 3461 -#define wxSplitterEvent_GetWindowBeingRemoved 3462 -#define wxSplitterEvent_SetSashPosition 3463 -#define wxHtmlWindow_new_0 3464 -#define wxHtmlWindow_new_2 3465 -#define wxHtmlWindow_AppendToPage 3466 -#define wxHtmlWindow_GetOpenedAnchor 3467 -#define wxHtmlWindow_GetOpenedPage 3468 -#define wxHtmlWindow_GetOpenedPageTitle 3469 -#define wxHtmlWindow_GetRelatedFrame 3470 -#define wxHtmlWindow_HistoryBack 3471 -#define wxHtmlWindow_HistoryCanBack 3472 -#define wxHtmlWindow_HistoryCanForward 3473 -#define wxHtmlWindow_HistoryClear 3474 -#define wxHtmlWindow_HistoryForward 3475 -#define wxHtmlWindow_LoadFile 3476 -#define wxHtmlWindow_LoadPage 3477 -#define wxHtmlWindow_SelectAll 3478 -#define wxHtmlWindow_SelectionToText 3479 -#define wxHtmlWindow_SelectLine 3480 -#define wxHtmlWindow_SelectWord 3481 -#define wxHtmlWindow_SetBorders 3482 -#define wxHtmlWindow_SetFonts 3483 -#define wxHtmlWindow_SetPage 3484 -#define wxHtmlWindow_SetRelatedFrame 3485 -#define wxHtmlWindow_SetRelatedStatusBar 3486 -#define wxHtmlWindow_ToText 3487 -#define wxHtmlWindow_destroy 3488 -#define wxHtmlLinkEvent_GetLinkInfo 3489 -#define wxSystemSettings_GetColour 3490 -#define wxSystemSettings_GetFont 3491 -#define wxSystemSettings_GetMetric 3492 -#define wxSystemSettings_GetScreenType 3493 -#define wxSystemOptions_GetOption 3494 -#define wxSystemOptions_GetOptionInt 3495 -#define wxSystemOptions_HasOption 3496 -#define wxSystemOptions_IsFalse 3497 -#define wxSystemOptions_SetOption_2_1 3498 -#define wxSystemOptions_SetOption_2_0 3499 -#define wxAuiNotebookEvent_SetSelection 3500 -#define wxAuiNotebookEvent_GetSelection 3501 -#define wxAuiNotebookEvent_SetOldSelection 3502 -#define wxAuiNotebookEvent_GetOldSelection 3503 -#define wxAuiNotebookEvent_SetDragSource 3504 -#define wxAuiNotebookEvent_GetDragSource 3505 -#define wxAuiManagerEvent_SetManager 3506 -#define wxAuiManagerEvent_GetManager 3507 -#define wxAuiManagerEvent_SetPane 3508 -#define wxAuiManagerEvent_GetPane 3509 -#define wxAuiManagerEvent_SetButton 3510 -#define wxAuiManagerEvent_GetButton 3511 -#define wxAuiManagerEvent_SetDC 3512 -#define wxAuiManagerEvent_GetDC 3513 -#define wxAuiManagerEvent_Veto 3514 -#define wxAuiManagerEvent_GetVeto 3515 -#define wxAuiManagerEvent_SetCanVeto 3516 -#define wxAuiManagerEvent_CanVeto 3517 -#define wxLogNull_new 3518 -#define wxLogNull_destroy 3519 -#define wxTaskBarIcon_new 3520 -#define wxTaskBarIcon_destruct 3521 -#define wxTaskBarIcon_PopupMenu 3522 -#define wxTaskBarIcon_RemoveIcon 3523 -#define wxTaskBarIcon_SetIcon 3524 +#define wxTextCtrl_ChangeValue 1827 +#define wxTextCtrl_EmulateKeyPress 1828 +#define wxTextCtrl_GetDefaultStyle 1829 +#define wxTextCtrl_GetInsertionPoint 1830 +#define wxTextCtrl_GetLastPosition 1831 +#define wxTextCtrl_GetLineLength 1832 +#define wxTextCtrl_GetLineText 1833 +#define wxTextCtrl_GetNumberOfLines 1834 +#define wxTextCtrl_GetRange 1835 +#define wxTextCtrl_GetSelection 1836 +#define wxTextCtrl_GetStringSelection 1837 +#define wxTextCtrl_GetStyle 1838 +#define wxTextCtrl_GetValue 1839 +#define wxTextCtrl_IsEditable 1840 +#define wxTextCtrl_IsModified 1841 +#define wxTextCtrl_IsMultiLine 1842 +#define wxTextCtrl_IsSingleLine 1843 +#define wxTextCtrl_LoadFile 1844 +#define wxTextCtrl_MarkDirty 1845 +#define wxTextCtrl_Paste 1846 +#define wxTextCtrl_PositionToXY 1847 +#define wxTextCtrl_Redo 1848 +#define wxTextCtrl_Remove 1849 +#define wxTextCtrl_Replace 1850 +#define wxTextCtrl_SaveFile 1851 +#define wxTextCtrl_SetDefaultStyle 1852 +#define wxTextCtrl_SetEditable 1853 +#define wxTextCtrl_SetInsertionPoint 1854 +#define wxTextCtrl_SetInsertionPointEnd 1855 +#define wxTextCtrl_SetMaxLength 1857 +#define wxTextCtrl_SetSelection 1858 +#define wxTextCtrl_SetStyle 1859 +#define wxTextCtrl_SetValue 1860 +#define wxTextCtrl_ShowPosition 1861 +#define wxTextCtrl_Undo 1862 +#define wxTextCtrl_WriteText 1863 +#define wxTextCtrl_XYToPosition 1864 +#define wxNotebook_new_0 1867 +#define wxNotebook_new_3 1868 +#define wxNotebook_destruct 1869 +#define wxNotebook_AddPage 1870 +#define wxNotebook_AdvanceSelection 1871 +#define wxNotebook_AssignImageList 1872 +#define wxNotebook_Create 1873 +#define wxNotebook_DeleteAllPages 1874 +#define wxNotebook_DeletePage 1875 +#define wxNotebook_RemovePage 1876 +#define wxNotebook_GetCurrentPage 1877 +#define wxNotebook_GetImageList 1878 +#define wxNotebook_GetPage 1880 +#define wxNotebook_GetPageCount 1881 +#define wxNotebook_GetPageImage 1882 +#define wxNotebook_GetPageText 1883 +#define wxNotebook_GetRowCount 1884 +#define wxNotebook_GetSelection 1885 +#define wxNotebook_GetThemeBackgroundColour 1886 +#define wxNotebook_HitTest 1888 +#define wxNotebook_InsertPage 1890 +#define wxNotebook_SetImageList 1891 +#define wxNotebook_SetPadding 1892 +#define wxNotebook_SetPageSize 1893 +#define wxNotebook_SetPageImage 1894 +#define wxNotebook_SetPageText 1895 +#define wxNotebook_SetSelection 1896 +#define wxNotebook_ChangeSelection 1897 +#define wxChoicebook_new_0 1898 +#define wxChoicebook_new_3 1899 +#define wxChoicebook_AddPage 1900 +#define wxChoicebook_AdvanceSelection 1901 +#define wxChoicebook_AssignImageList 1902 +#define wxChoicebook_Create 1903 +#define wxChoicebook_DeleteAllPages 1904 +#define wxChoicebook_DeletePage 1905 +#define wxChoicebook_RemovePage 1906 +#define wxChoicebook_GetCurrentPage 1907 +#define wxChoicebook_GetImageList 1908 +#define wxChoicebook_GetPage 1910 +#define wxChoicebook_GetPageCount 1911 +#define wxChoicebook_GetPageImage 1912 +#define wxChoicebook_GetPageText 1913 +#define wxChoicebook_GetSelection 1914 +#define wxChoicebook_HitTest 1915 +#define wxChoicebook_InsertPage 1916 +#define wxChoicebook_SetImageList 1917 +#define wxChoicebook_SetPageSize 1918 +#define wxChoicebook_SetPageImage 1919 +#define wxChoicebook_SetPageText 1920 +#define wxChoicebook_SetSelection 1921 +#define wxChoicebook_ChangeSelection 1922 +#define wxChoicebook_destroy 1923 +#define wxToolbook_new_0 1924 +#define wxToolbook_new_3 1925 +#define wxToolbook_AddPage 1926 +#define wxToolbook_AdvanceSelection 1927 +#define wxToolbook_AssignImageList 1928 +#define wxToolbook_Create 1929 +#define wxToolbook_DeleteAllPages 1930 +#define wxToolbook_DeletePage 1931 +#define wxToolbook_RemovePage 1932 +#define wxToolbook_GetCurrentPage 1933 +#define wxToolbook_GetImageList 1934 +#define wxToolbook_GetPage 1936 +#define wxToolbook_GetPageCount 1937 +#define wxToolbook_GetPageImage 1938 +#define wxToolbook_GetPageText 1939 +#define wxToolbook_GetSelection 1940 +#define wxToolbook_HitTest 1942 +#define wxToolbook_InsertPage 1943 +#define wxToolbook_SetImageList 1944 +#define wxToolbook_SetPageSize 1945 +#define wxToolbook_SetPageImage 1946 +#define wxToolbook_SetPageText 1947 +#define wxToolbook_SetSelection 1948 +#define wxToolbook_ChangeSelection 1949 +#define wxToolbook_destroy 1950 +#define wxListbook_new_0 1951 +#define wxListbook_new_3 1952 +#define wxListbook_AddPage 1953 +#define wxListbook_AdvanceSelection 1954 +#define wxListbook_AssignImageList 1955 +#define wxListbook_Create 1956 +#define wxListbook_DeleteAllPages 1957 +#define wxListbook_DeletePage 1958 +#define wxListbook_RemovePage 1959 +#define wxListbook_GetCurrentPage 1960 +#define wxListbook_GetImageList 1961 +#define wxListbook_GetPage 1963 +#define wxListbook_GetPageCount 1964 +#define wxListbook_GetPageImage 1965 +#define wxListbook_GetPageText 1966 +#define wxListbook_GetSelection 1967 +#define wxListbook_HitTest 1969 +#define wxListbook_InsertPage 1970 +#define wxListbook_SetImageList 1971 +#define wxListbook_SetPageSize 1972 +#define wxListbook_SetPageImage 1973 +#define wxListbook_SetPageText 1974 +#define wxListbook_SetSelection 1975 +#define wxListbook_ChangeSelection 1976 +#define wxListbook_destroy 1977 +#define wxTreebook_new_0 1978 +#define wxTreebook_new_3 1979 +#define wxTreebook_AddPage 1980 +#define wxTreebook_AdvanceSelection 1981 +#define wxTreebook_AssignImageList 1982 +#define wxTreebook_Create 1983 +#define wxTreebook_DeleteAllPages 1984 +#define wxTreebook_DeletePage 1985 +#define wxTreebook_RemovePage 1986 +#define wxTreebook_GetCurrentPage 1987 +#define wxTreebook_GetImageList 1988 +#define wxTreebook_GetPage 1990 +#define wxTreebook_GetPageCount 1991 +#define wxTreebook_GetPageImage 1992 +#define wxTreebook_GetPageText 1993 +#define wxTreebook_GetSelection 1994 +#define wxTreebook_ExpandNode 1995 +#define wxTreebook_IsNodeExpanded 1996 +#define wxTreebook_HitTest 1998 +#define wxTreebook_InsertPage 1999 +#define wxTreebook_InsertSubPage 2000 +#define wxTreebook_SetImageList 2001 +#define wxTreebook_SetPageSize 2002 +#define wxTreebook_SetPageImage 2003 +#define wxTreebook_SetPageText 2004 +#define wxTreebook_SetSelection 2005 +#define wxTreebook_ChangeSelection 2006 +#define wxTreebook_destroy 2007 +#define wxTreeCtrl_new_2 2010 +#define wxTreeCtrl_new_0 2011 +#define wxTreeCtrl_destruct 2013 +#define wxTreeCtrl_AddRoot 2014 +#define wxTreeCtrl_AppendItem 2015 +#define wxTreeCtrl_AssignImageList 2016 +#define wxTreeCtrl_AssignStateImageList 2017 +#define wxTreeCtrl_Collapse 2018 +#define wxTreeCtrl_CollapseAndReset 2019 +#define wxTreeCtrl_Create 2020 +#define wxTreeCtrl_Delete 2021 +#define wxTreeCtrl_DeleteAllItems 2022 +#define wxTreeCtrl_DeleteChildren 2023 +#define wxTreeCtrl_EditLabel 2024 +#define wxTreeCtrl_EnsureVisible 2025 +#define wxTreeCtrl_Expand 2026 +#define wxTreeCtrl_GetBoundingRect 2027 +#define wxTreeCtrl_GetChildrenCount 2029 +#define wxTreeCtrl_GetCount 2030 +#define wxTreeCtrl_GetEditControl 2031 +#define wxTreeCtrl_GetFirstChild 2032 +#define wxTreeCtrl_GetNextChild 2033 +#define wxTreeCtrl_GetFirstVisibleItem 2034 +#define wxTreeCtrl_GetImageList 2035 +#define wxTreeCtrl_GetIndent 2036 +#define wxTreeCtrl_GetItemBackgroundColour 2037 +#define wxTreeCtrl_GetItemData 2038 +#define wxTreeCtrl_GetItemFont 2039 +#define wxTreeCtrl_GetItemImage_1 2040 +#define wxTreeCtrl_GetItemImage_2 2041 +#define wxTreeCtrl_GetItemText 2042 +#define wxTreeCtrl_GetItemTextColour 2043 +#define wxTreeCtrl_GetLastChild 2044 +#define wxTreeCtrl_GetNextSibling 2045 +#define wxTreeCtrl_GetNextVisible 2046 +#define wxTreeCtrl_GetItemParent 2047 +#define wxTreeCtrl_GetPrevSibling 2048 +#define wxTreeCtrl_GetPrevVisible 2049 +#define wxTreeCtrl_GetRootItem 2050 +#define wxTreeCtrl_GetSelection 2051 +#define wxTreeCtrl_GetSelections 2052 +#define wxTreeCtrl_GetStateImageList 2053 +#define wxTreeCtrl_HitTest 2054 +#define wxTreeCtrl_InsertItem 2056 +#define wxTreeCtrl_IsBold 2057 +#define wxTreeCtrl_IsExpanded 2058 +#define wxTreeCtrl_IsSelected 2059 +#define wxTreeCtrl_IsVisible 2060 +#define wxTreeCtrl_ItemHasChildren 2061 +#define wxTreeCtrl_IsTreeItemIdOk 2062 +#define wxTreeCtrl_PrependItem 2063 +#define wxTreeCtrl_ScrollTo 2064 +#define wxTreeCtrl_SelectItem_1 2065 +#define wxTreeCtrl_SelectItem_2 2066 +#define wxTreeCtrl_SetIndent 2067 +#define wxTreeCtrl_SetImageList 2068 +#define wxTreeCtrl_SetItemBackgroundColour 2069 +#define wxTreeCtrl_SetItemBold 2070 +#define wxTreeCtrl_SetItemData 2071 +#define wxTreeCtrl_SetItemDropHighlight 2072 +#define wxTreeCtrl_SetItemFont 2073 +#define wxTreeCtrl_SetItemHasChildren 2074 +#define wxTreeCtrl_SetItemImage_2 2075 +#define wxTreeCtrl_SetItemImage_3 2076 +#define wxTreeCtrl_SetItemText 2077 +#define wxTreeCtrl_SetItemTextColour 2078 +#define wxTreeCtrl_SetStateImageList 2079 +#define wxTreeCtrl_SetWindowStyle 2080 +#define wxTreeCtrl_SortChildren 2081 +#define wxTreeCtrl_Toggle 2082 +#define wxTreeCtrl_ToggleItemSelection 2083 +#define wxTreeCtrl_Unselect 2084 +#define wxTreeCtrl_UnselectAll 2085 +#define wxTreeCtrl_UnselectItem 2086 +#define wxScrollBar_new_0 2087 +#define wxScrollBar_new_3 2088 +#define wxScrollBar_destruct 2089 +#define wxScrollBar_Create 2090 +#define wxScrollBar_GetRange 2091 +#define wxScrollBar_GetPageSize 2092 +#define wxScrollBar_GetThumbPosition 2093 +#define wxScrollBar_GetThumbSize 2094 +#define wxScrollBar_SetThumbPosition 2095 +#define wxScrollBar_SetScrollbar 2096 +#define wxSpinButton_new_2 2098 +#define wxSpinButton_new_0 2099 +#define wxSpinButton_Create 2100 +#define wxSpinButton_GetMax 2101 +#define wxSpinButton_GetMin 2102 +#define wxSpinButton_GetValue 2103 +#define wxSpinButton_SetRange 2104 +#define wxSpinButton_SetValue 2105 +#define wxSpinButton_destroy 2106 +#define wxSpinCtrl_new_0 2107 +#define wxSpinCtrl_new_2 2108 +#define wxSpinCtrl_Create 2110 +#define wxSpinCtrl_SetValue_1_1 2113 +#define wxSpinCtrl_SetValue_1_0 2114 +#define wxSpinCtrl_GetValue 2116 +#define wxSpinCtrl_SetRange 2118 +#define wxSpinCtrl_SetSelection 2119 +#define wxSpinCtrl_GetMin 2121 +#define wxSpinCtrl_GetMax 2123 +#define wxSpinCtrl_destroy 2124 +#define wxStaticText_new_0 2125 +#define wxStaticText_new_4 2126 +#define wxStaticText_Create 2127 +#define wxStaticText_GetLabel 2128 +#define wxStaticText_SetLabel 2129 +#define wxStaticText_Wrap 2130 +#define wxStaticText_destroy 2131 +#define wxStaticBitmap_new_0 2132 +#define wxStaticBitmap_new_4 2133 +#define wxStaticBitmap_Create 2134 +#define wxStaticBitmap_GetBitmap 2135 +#define wxStaticBitmap_SetBitmap 2136 +#define wxStaticBitmap_destroy 2137 +#define wxRadioBox_new 2138 +#define wxRadioBox_destruct 2140 +#define wxRadioBox_Create 2141 +#define wxRadioBox_Enable_2 2142 +#define wxRadioBox_Enable_1 2143 +#define wxRadioBox_GetSelection 2144 +#define wxRadioBox_GetString 2145 +#define wxRadioBox_SetSelection 2146 +#define wxRadioBox_Show_2 2147 +#define wxRadioBox_Show_1 2148 +#define wxRadioBox_GetColumnCount 2149 +#define wxRadioBox_GetItemHelpText 2150 +#define wxRadioBox_GetItemToolTip 2151 +#define wxRadioBox_GetItemFromPoint 2153 +#define wxRadioBox_GetRowCount 2154 +#define wxRadioBox_IsItemEnabled 2155 +#define wxRadioBox_IsItemShown 2156 +#define wxRadioBox_SetItemHelpText 2157 +#define wxRadioBox_SetItemToolTip 2158 +#define wxRadioButton_new_0 2159 +#define wxRadioButton_new_4 2160 +#define wxRadioButton_Create 2161 +#define wxRadioButton_GetValue 2162 +#define wxRadioButton_SetValue 2163 +#define wxRadioButton_destroy 2164 +#define wxSlider_new_6 2166 +#define wxSlider_new_0 2167 +#define wxSlider_Create 2168 +#define wxSlider_GetLineSize 2169 +#define wxSlider_GetMax 2170 +#define wxSlider_GetMin 2171 +#define wxSlider_GetPageSize 2172 +#define wxSlider_GetThumbLength 2173 +#define wxSlider_GetValue 2174 +#define wxSlider_SetLineSize 2175 +#define wxSlider_SetPageSize 2176 +#define wxSlider_SetRange 2177 +#define wxSlider_SetThumbLength 2178 +#define wxSlider_SetValue 2179 +#define wxSlider_destroy 2180 +#define wxDialog_new_4 2182 +#define wxDialog_new_0 2183 +#define wxDialog_destruct 2185 +#define wxDialog_Create 2186 +#define wxDialog_CreateButtonSizer 2187 +#define wxDialog_CreateStdDialogButtonSizer 2188 +#define wxDialog_EndModal 2189 +#define wxDialog_GetAffirmativeId 2190 +#define wxDialog_GetReturnCode 2191 +#define wxDialog_IsModal 2192 +#define wxDialog_SetAffirmativeId 2193 +#define wxDialog_SetReturnCode 2194 +#define wxDialog_Show 2195 +#define wxDialog_ShowModal 2196 +#define wxColourDialog_new_0 2197 +#define wxColourDialog_new_2 2198 +#define wxColourDialog_destruct 2199 +#define wxColourDialog_Create 2200 +#define wxColourDialog_GetColourData 2201 +#define wxColourData_new_0 2202 +#define wxColourData_new_1 2203 +#define wxColourData_destruct 2204 +#define wxColourData_GetChooseFull 2205 +#define wxColourData_GetColour 2206 +#define wxColourData_GetCustomColour 2208 +#define wxColourData_SetChooseFull 2209 +#define wxColourData_SetColour 2210 +#define wxColourData_SetCustomColour 2211 +#define wxPalette_new_0 2212 +#define wxPalette_new_4 2213 +#define wxPalette_destruct 2215 +#define wxPalette_Create 2216 +#define wxPalette_GetColoursCount 2217 +#define wxPalette_GetPixel 2218 +#define wxPalette_GetRGB 2219 +#define wxPalette_IsOk 2220 +#define wxDirDialog_new 2224 +#define wxDirDialog_destruct 2225 +#define wxDirDialog_GetPath 2226 +#define wxDirDialog_GetMessage 2227 +#define wxDirDialog_SetMessage 2228 +#define wxDirDialog_SetPath 2229 +#define wxFileDialog_new 2233 +#define wxFileDialog_destruct 2234 +#define wxFileDialog_GetDirectory 2235 +#define wxFileDialog_GetFilename 2236 +#define wxFileDialog_GetFilenames 2237 +#define wxFileDialog_GetFilterIndex 2238 +#define wxFileDialog_GetMessage 2239 +#define wxFileDialog_GetPath 2240 +#define wxFileDialog_GetPaths 2241 +#define wxFileDialog_GetWildcard 2242 +#define wxFileDialog_SetDirectory 2243 +#define wxFileDialog_SetFilename 2244 +#define wxFileDialog_SetFilterIndex 2245 +#define wxFileDialog_SetMessage 2246 +#define wxFileDialog_SetPath 2247 +#define wxFileDialog_SetWildcard 2248 +#define wxPickerBase_SetInternalMargin 2249 +#define wxPickerBase_GetInternalMargin 2250 +#define wxPickerBase_SetTextCtrlProportion 2251 +#define wxPickerBase_SetPickerCtrlProportion 2252 +#define wxPickerBase_GetTextCtrlProportion 2253 +#define wxPickerBase_GetPickerCtrlProportion 2254 +#define wxPickerBase_HasTextCtrl 2255 +#define wxPickerBase_GetTextCtrl 2256 +#define wxPickerBase_IsTextCtrlGrowable 2257 +#define wxPickerBase_SetPickerCtrlGrowable 2258 +#define wxPickerBase_SetTextCtrlGrowable 2259 +#define wxPickerBase_IsPickerCtrlGrowable 2260 +#define wxFilePickerCtrl_new_0 2261 +#define wxFilePickerCtrl_new_3 2262 +#define wxFilePickerCtrl_Create 2263 +#define wxFilePickerCtrl_GetPath 2264 +#define wxFilePickerCtrl_SetPath 2265 +#define wxFilePickerCtrl_destroy 2266 +#define wxDirPickerCtrl_new_0 2267 +#define wxDirPickerCtrl_new_3 2268 +#define wxDirPickerCtrl_Create 2269 +#define wxDirPickerCtrl_GetPath 2270 +#define wxDirPickerCtrl_SetPath 2271 +#define wxDirPickerCtrl_destroy 2272 +#define wxColourPickerCtrl_new_0 2273 +#define wxColourPickerCtrl_new_3 2274 +#define wxColourPickerCtrl_Create 2275 +#define wxColourPickerCtrl_GetColour 2276 +#define wxColourPickerCtrl_SetColour_1_1 2277 +#define wxColourPickerCtrl_SetColour_1_0 2278 +#define wxColourPickerCtrl_destroy 2279 +#define wxDatePickerCtrl_new_0 2280 +#define wxDatePickerCtrl_new_3 2281 +#define wxDatePickerCtrl_GetRange 2282 +#define wxDatePickerCtrl_GetValue 2283 +#define wxDatePickerCtrl_SetRange 2284 +#define wxDatePickerCtrl_SetValue 2285 +#define wxDatePickerCtrl_destroy 2286 +#define wxFontPickerCtrl_new_0 2287 +#define wxFontPickerCtrl_new_3 2288 +#define wxFontPickerCtrl_Create 2289 +#define wxFontPickerCtrl_GetSelectedFont 2290 +#define wxFontPickerCtrl_SetSelectedFont 2291 +#define wxFontPickerCtrl_GetMaxPointSize 2292 +#define wxFontPickerCtrl_SetMaxPointSize 2293 +#define wxFontPickerCtrl_destroy 2294 +#define wxFindReplaceDialog_new_0 2297 +#define wxFindReplaceDialog_new_4 2298 +#define wxFindReplaceDialog_destruct 2299 +#define wxFindReplaceDialog_Create 2300 +#define wxFindReplaceDialog_GetData 2301 +#define wxFindReplaceData_new_0 2302 +#define wxFindReplaceData_new_1 2303 +#define wxFindReplaceData_GetFindString 2304 +#define wxFindReplaceData_GetReplaceString 2305 +#define wxFindReplaceData_GetFlags 2306 +#define wxFindReplaceData_SetFlags 2307 +#define wxFindReplaceData_SetFindString 2308 +#define wxFindReplaceData_SetReplaceString 2309 +#define wxFindReplaceData_destroy 2310 +#define wxMultiChoiceDialog_new_0 2311 +#define wxMultiChoiceDialog_new_5 2313 +#define wxMultiChoiceDialog_GetSelections 2314 +#define wxMultiChoiceDialog_SetSelections 2315 +#define wxMultiChoiceDialog_destroy 2316 +#define wxSingleChoiceDialog_new_0 2317 +#define wxSingleChoiceDialog_new_5 2319 +#define wxSingleChoiceDialog_GetSelection 2320 +#define wxSingleChoiceDialog_GetStringSelection 2321 +#define wxSingleChoiceDialog_SetSelection 2322 +#define wxSingleChoiceDialog_destroy 2323 +#define wxTextEntryDialog_new 2324 +#define wxTextEntryDialog_GetValue 2325 +#define wxTextEntryDialog_SetValue 2326 +#define wxTextEntryDialog_destroy 2327 +#define wxPasswordEntryDialog_new 2328 +#define wxPasswordEntryDialog_destroy 2329 +#define wxFontData_new_0 2330 +#define wxFontData_new_1 2331 +#define wxFontData_destruct 2332 +#define wxFontData_EnableEffects 2333 +#define wxFontData_GetAllowSymbols 2334 +#define wxFontData_GetColour 2335 +#define wxFontData_GetChosenFont 2336 +#define wxFontData_GetEnableEffects 2337 +#define wxFontData_GetInitialFont 2338 +#define wxFontData_GetShowHelp 2339 +#define wxFontData_SetAllowSymbols 2340 +#define wxFontData_SetChosenFont 2341 +#define wxFontData_SetColour 2342 +#define wxFontData_SetInitialFont 2343 +#define wxFontData_SetRange 2344 +#define wxFontData_SetShowHelp 2345 +#define wxFontDialog_new_0 2349 +#define wxFontDialog_new_2 2351 +#define wxFontDialog_Create 2353 +#define wxFontDialog_GetFontData 2354 +#define wxFontDialog_destroy 2356 +#define wxProgressDialog_new 2357 +#define wxProgressDialog_destruct 2358 +#define wxProgressDialog_Resume 2359 +#define wxProgressDialog_Update_2 2360 +#define wxProgressDialog_Update_0 2361 +#define wxMessageDialog_new 2362 +#define wxMessageDialog_destruct 2363 +#define wxPageSetupDialog_new 2364 +#define wxPageSetupDialog_destruct 2365 +#define wxPageSetupDialog_GetPageSetupData 2366 +#define wxPageSetupDialog_ShowModal 2367 +#define wxPageSetupDialogData_new_0 2368 +#define wxPageSetupDialogData_new_1_0 2369 +#define wxPageSetupDialogData_new_1_1 2370 +#define wxPageSetupDialogData_destruct 2371 +#define wxPageSetupDialogData_EnableHelp 2372 +#define wxPageSetupDialogData_EnableMargins 2373 +#define wxPageSetupDialogData_EnableOrientation 2374 +#define wxPageSetupDialogData_EnablePaper 2375 +#define wxPageSetupDialogData_EnablePrinter 2376 +#define wxPageSetupDialogData_GetDefaultMinMargins 2377 +#define wxPageSetupDialogData_GetEnableMargins 2378 +#define wxPageSetupDialogData_GetEnableOrientation 2379 +#define wxPageSetupDialogData_GetEnablePaper 2380 +#define wxPageSetupDialogData_GetEnablePrinter 2381 +#define wxPageSetupDialogData_GetEnableHelp 2382 +#define wxPageSetupDialogData_GetDefaultInfo 2383 +#define wxPageSetupDialogData_GetMarginTopLeft 2384 +#define wxPageSetupDialogData_GetMarginBottomRight 2385 +#define wxPageSetupDialogData_GetMinMarginTopLeft 2386 +#define wxPageSetupDialogData_GetMinMarginBottomRight 2387 +#define wxPageSetupDialogData_GetPaperId 2388 +#define wxPageSetupDialogData_GetPaperSize 2389 +#define wxPageSetupDialogData_GetPrintData 2391 +#define wxPageSetupDialogData_IsOk 2392 +#define wxPageSetupDialogData_SetDefaultInfo 2393 +#define wxPageSetupDialogData_SetDefaultMinMargins 2394 +#define wxPageSetupDialogData_SetMarginTopLeft 2395 +#define wxPageSetupDialogData_SetMarginBottomRight 2396 +#define wxPageSetupDialogData_SetMinMarginTopLeft 2397 +#define wxPageSetupDialogData_SetMinMarginBottomRight 2398 +#define wxPageSetupDialogData_SetPaperId 2399 +#define wxPageSetupDialogData_SetPaperSize_1_1 2400 +#define wxPageSetupDialogData_SetPaperSize_1_0 2401 +#define wxPageSetupDialogData_SetPrintData 2402 +#define wxPrintDialog_new_2_0 2403 +#define wxPrintDialog_new_2_1 2404 +#define wxPrintDialog_destruct 2405 +#define wxPrintDialog_GetPrintDialogData 2406 +#define wxPrintDialog_GetPrintDC 2407 +#define wxPrintDialogData_new_0 2408 +#define wxPrintDialogData_new_1_1 2409 +#define wxPrintDialogData_new_1_0 2410 +#define wxPrintDialogData_destruct 2411 +#define wxPrintDialogData_EnableHelp 2412 +#define wxPrintDialogData_EnablePageNumbers 2413 +#define wxPrintDialogData_EnablePrintToFile 2414 +#define wxPrintDialogData_EnableSelection 2415 +#define wxPrintDialogData_GetAllPages 2416 +#define wxPrintDialogData_GetCollate 2417 +#define wxPrintDialogData_GetFromPage 2418 +#define wxPrintDialogData_GetMaxPage 2419 +#define wxPrintDialogData_GetMinPage 2420 +#define wxPrintDialogData_GetNoCopies 2421 +#define wxPrintDialogData_GetPrintData 2422 +#define wxPrintDialogData_GetPrintToFile 2423 +#define wxPrintDialogData_GetSelection 2424 +#define wxPrintDialogData_GetToPage 2425 +#define wxPrintDialogData_IsOk 2426 +#define wxPrintDialogData_SetCollate 2427 +#define wxPrintDialogData_SetFromPage 2428 +#define wxPrintDialogData_SetMaxPage 2429 +#define wxPrintDialogData_SetMinPage 2430 +#define wxPrintDialogData_SetNoCopies 2431 +#define wxPrintDialogData_SetPrintData 2432 +#define wxPrintDialogData_SetPrintToFile 2433 +#define wxPrintDialogData_SetSelection 2434 +#define wxPrintDialogData_SetToPage 2435 +#define wxPrintData_new_0 2436 +#define wxPrintData_new_1 2437 +#define wxPrintData_destruct 2438 +#define wxPrintData_GetCollate 2439 +#define wxPrintData_GetBin 2440 +#define wxPrintData_GetColour 2441 +#define wxPrintData_GetDuplex 2442 +#define wxPrintData_GetNoCopies 2443 +#define wxPrintData_GetOrientation 2444 +#define wxPrintData_GetPaperId 2445 +#define wxPrintData_GetPrinterName 2446 +#define wxPrintData_GetQuality 2447 +#define wxPrintData_IsOk 2448 +#define wxPrintData_SetBin 2449 +#define wxPrintData_SetCollate 2450 +#define wxPrintData_SetColour 2451 +#define wxPrintData_SetDuplex 2452 +#define wxPrintData_SetNoCopies 2453 +#define wxPrintData_SetOrientation 2454 +#define wxPrintData_SetPaperId 2455 +#define wxPrintData_SetPrinterName 2456 +#define wxPrintData_SetQuality 2457 +#define wxPrintPreview_new_2 2460 +#define wxPrintPreview_new_3 2461 +#define wxPrintPreview_destruct 2463 +#define wxPrintPreview_GetCanvas 2464 +#define wxPrintPreview_GetCurrentPage 2465 +#define wxPrintPreview_GetFrame 2466 +#define wxPrintPreview_GetMaxPage 2467 +#define wxPrintPreview_GetMinPage 2468 +#define wxPrintPreview_GetPrintout 2469 +#define wxPrintPreview_GetPrintoutForPrinting 2470 +#define wxPrintPreview_IsOk 2471 +#define wxPrintPreview_PaintPage 2472 +#define wxPrintPreview_Print 2473 +#define wxPrintPreview_RenderPage 2474 +#define wxPrintPreview_SetCanvas 2475 +#define wxPrintPreview_SetCurrentPage 2476 +#define wxPrintPreview_SetFrame 2477 +#define wxPrintPreview_SetPrintout 2478 +#define wxPrintPreview_SetZoom 2479 +#define wxPreviewFrame_new 2480 +#define wxPreviewFrame_destruct 2481 +#define wxPreviewFrame_CreateControlBar 2482 +#define wxPreviewFrame_CreateCanvas 2483 +#define wxPreviewFrame_Initialize 2484 +#define wxPreviewFrame_OnCloseWindow 2485 +#define wxPreviewControlBar_new 2486 +#define wxPreviewControlBar_destruct 2487 +#define wxPreviewControlBar_CreateButtons 2488 +#define wxPreviewControlBar_GetPrintPreview 2489 +#define wxPreviewControlBar_GetZoomControl 2490 +#define wxPreviewControlBar_SetZoomControl 2491 +#define wxPrinter_new 2493 +#define wxPrinter_CreateAbortWindow 2494 +#define wxPrinter_GetAbort 2495 +#define wxPrinter_GetLastError 2496 +#define wxPrinter_GetPrintDialogData 2497 +#define wxPrinter_Print 2498 +#define wxPrinter_PrintDialog 2499 +#define wxPrinter_ReportError 2500 +#define wxPrinter_Setup 2501 +#define wxPrinter_destroy 2502 +#define wxXmlResource_new_1 2503 +#define wxXmlResource_new_2 2504 +#define wxXmlResource_destruct 2505 +#define wxXmlResource_AttachUnknownControl 2506 +#define wxXmlResource_ClearHandlers 2507 +#define wxXmlResource_CompareVersion 2508 +#define wxXmlResource_Get 2509 +#define wxXmlResource_GetFlags 2510 +#define wxXmlResource_GetVersion 2511 +#define wxXmlResource_GetXRCID 2512 +#define wxXmlResource_InitAllHandlers 2513 +#define wxXmlResource_Load 2514 +#define wxXmlResource_LoadBitmap 2515 +#define wxXmlResource_LoadDialog_2 2516 +#define wxXmlResource_LoadDialog_3 2517 +#define wxXmlResource_LoadFrame_2 2518 +#define wxXmlResource_LoadFrame_3 2519 +#define wxXmlResource_LoadIcon 2520 +#define wxXmlResource_LoadMenu 2521 +#define wxXmlResource_LoadMenuBar_2 2522 +#define wxXmlResource_LoadMenuBar_1 2523 +#define wxXmlResource_LoadPanel_2 2524 +#define wxXmlResource_LoadPanel_3 2525 +#define wxXmlResource_LoadToolBar 2526 +#define wxXmlResource_Set 2527 +#define wxXmlResource_SetFlags 2528 +#define wxXmlResource_Unload 2529 +#define wxXmlResource_xrcctrl 2530 +#define wxHtmlEasyPrinting_new 2531 +#define wxHtmlEasyPrinting_destruct 2532 +#define wxHtmlEasyPrinting_GetPrintData 2533 +#define wxHtmlEasyPrinting_GetPageSetupData 2534 +#define wxHtmlEasyPrinting_PreviewFile 2535 +#define wxHtmlEasyPrinting_PreviewText 2536 +#define wxHtmlEasyPrinting_PrintFile 2537 +#define wxHtmlEasyPrinting_PrintText 2538 +#define wxHtmlEasyPrinting_PageSetup 2539 +#define wxHtmlEasyPrinting_SetFonts 2540 +#define wxHtmlEasyPrinting_SetHeader 2541 +#define wxHtmlEasyPrinting_SetFooter 2542 +#define wxGLCanvas_new_2 2544 +#define wxGLCanvas_new_3_1 2545 +#define wxGLCanvas_new_3_0 2546 +#define wxGLCanvas_GetContext 2547 +#define wxGLCanvas_SetCurrent 2549 +#define wxGLCanvas_SwapBuffers 2550 +#define wxGLCanvas_destroy 2551 +#define wxAuiManager_new 2552 +#define wxAuiManager_destruct 2553 +#define wxAuiManager_AddPane_2_1 2554 +#define wxAuiManager_AddPane_3 2555 +#define wxAuiManager_AddPane_2_0 2556 +#define wxAuiManager_DetachPane 2557 +#define wxAuiManager_GetAllPanes 2558 +#define wxAuiManager_GetArtProvider 2559 +#define wxAuiManager_GetDockSizeConstraint 2560 +#define wxAuiManager_GetFlags 2561 +#define wxAuiManager_GetManagedWindow 2562 +#define wxAuiManager_GetManager 2563 +#define wxAuiManager_GetPane_1_1 2564 +#define wxAuiManager_GetPane_1_0 2565 +#define wxAuiManager_HideHint 2566 +#define wxAuiManager_InsertPane 2567 +#define wxAuiManager_LoadPaneInfo 2568 +#define wxAuiManager_LoadPerspective 2569 +#define wxAuiManager_SavePaneInfo 2570 +#define wxAuiManager_SavePerspective 2571 +#define wxAuiManager_SetArtProvider 2572 +#define wxAuiManager_SetDockSizeConstraint 2573 +#define wxAuiManager_SetFlags 2574 +#define wxAuiManager_SetManagedWindow 2575 +#define wxAuiManager_ShowHint 2576 +#define wxAuiManager_UnInit 2577 +#define wxAuiManager_Update 2578 +#define wxAuiPaneInfo_new_0 2579 +#define wxAuiPaneInfo_new_1 2580 +#define wxAuiPaneInfo_destruct 2581 +#define wxAuiPaneInfo_BestSize_1 2582 +#define wxAuiPaneInfo_BestSize_2 2583 +#define wxAuiPaneInfo_Bottom 2584 +#define wxAuiPaneInfo_BottomDockable 2585 +#define wxAuiPaneInfo_Caption 2586 +#define wxAuiPaneInfo_CaptionVisible 2587 +#define wxAuiPaneInfo_Centre 2588 +#define wxAuiPaneInfo_CentrePane 2589 +#define wxAuiPaneInfo_CloseButton 2590 +#define wxAuiPaneInfo_DefaultPane 2591 +#define wxAuiPaneInfo_DestroyOnClose 2592 +#define wxAuiPaneInfo_Direction 2593 +#define wxAuiPaneInfo_Dock 2594 +#define wxAuiPaneInfo_Dockable 2595 +#define wxAuiPaneInfo_Fixed 2596 +#define wxAuiPaneInfo_Float 2597 +#define wxAuiPaneInfo_Floatable 2598 +#define wxAuiPaneInfo_FloatingPosition_1 2599 +#define wxAuiPaneInfo_FloatingPosition_2 2600 +#define wxAuiPaneInfo_FloatingSize_1 2601 +#define wxAuiPaneInfo_FloatingSize_2 2602 +#define wxAuiPaneInfo_Gripper 2603 +#define wxAuiPaneInfo_GripperTop 2604 +#define wxAuiPaneInfo_HasBorder 2605 +#define wxAuiPaneInfo_HasCaption 2606 +#define wxAuiPaneInfo_HasCloseButton 2607 +#define wxAuiPaneInfo_HasFlag 2608 +#define wxAuiPaneInfo_HasGripper 2609 +#define wxAuiPaneInfo_HasGripperTop 2610 +#define wxAuiPaneInfo_HasMaximizeButton 2611 +#define wxAuiPaneInfo_HasMinimizeButton 2612 +#define wxAuiPaneInfo_HasPinButton 2613 +#define wxAuiPaneInfo_Hide 2614 +#define wxAuiPaneInfo_IsBottomDockable 2615 +#define wxAuiPaneInfo_IsDocked 2616 +#define wxAuiPaneInfo_IsFixed 2617 +#define wxAuiPaneInfo_IsFloatable 2618 +#define wxAuiPaneInfo_IsFloating 2619 +#define wxAuiPaneInfo_IsLeftDockable 2620 +#define wxAuiPaneInfo_IsMovable 2621 +#define wxAuiPaneInfo_IsOk 2622 +#define wxAuiPaneInfo_IsResizable 2623 +#define wxAuiPaneInfo_IsRightDockable 2624 +#define wxAuiPaneInfo_IsShown 2625 +#define wxAuiPaneInfo_IsToolbar 2626 +#define wxAuiPaneInfo_IsTopDockable 2627 +#define wxAuiPaneInfo_Layer 2628 +#define wxAuiPaneInfo_Left 2629 +#define wxAuiPaneInfo_LeftDockable 2630 +#define wxAuiPaneInfo_MaxSize_1 2631 +#define wxAuiPaneInfo_MaxSize_2 2632 +#define wxAuiPaneInfo_MaximizeButton 2633 +#define wxAuiPaneInfo_MinSize_1 2634 +#define wxAuiPaneInfo_MinSize_2 2635 +#define wxAuiPaneInfo_MinimizeButton 2636 +#define wxAuiPaneInfo_Movable 2637 +#define wxAuiPaneInfo_Name 2638 +#define wxAuiPaneInfo_PaneBorder 2639 +#define wxAuiPaneInfo_PinButton 2640 +#define wxAuiPaneInfo_Position 2641 +#define wxAuiPaneInfo_Resizable 2642 +#define wxAuiPaneInfo_Right 2643 +#define wxAuiPaneInfo_RightDockable 2644 +#define wxAuiPaneInfo_Row 2645 +#define wxAuiPaneInfo_SafeSet 2646 +#define wxAuiPaneInfo_SetFlag 2647 +#define wxAuiPaneInfo_Show 2648 +#define wxAuiPaneInfo_ToolbarPane 2649 +#define wxAuiPaneInfo_Top 2650 +#define wxAuiPaneInfo_TopDockable 2651 +#define wxAuiPaneInfo_Window 2652 +#define wxAuiNotebook_new_0 2653 +#define wxAuiNotebook_new_2 2654 +#define wxAuiNotebook_AddPage 2655 +#define wxAuiNotebook_Create 2656 +#define wxAuiNotebook_DeletePage 2657 +#define wxAuiNotebook_GetArtProvider 2658 +#define wxAuiNotebook_GetPage 2659 +#define wxAuiNotebook_GetPageBitmap 2660 +#define wxAuiNotebook_GetPageCount 2661 +#define wxAuiNotebook_GetPageIndex 2662 +#define wxAuiNotebook_GetPageText 2663 +#define wxAuiNotebook_GetSelection 2664 +#define wxAuiNotebook_InsertPage 2665 +#define wxAuiNotebook_RemovePage 2666 +#define wxAuiNotebook_SetArtProvider 2667 +#define wxAuiNotebook_SetFont 2668 +#define wxAuiNotebook_SetPageBitmap 2669 +#define wxAuiNotebook_SetPageText 2670 +#define wxAuiNotebook_SetSelection 2671 +#define wxAuiNotebook_SetTabCtrlHeight 2672 +#define wxAuiNotebook_SetUniformBitmapSize 2673 +#define wxAuiNotebook_destroy 2674 +#define wxMDIParentFrame_new_0 2675 +#define wxMDIParentFrame_new_4 2676 +#define wxMDIParentFrame_destruct 2677 +#define wxMDIParentFrame_ActivateNext 2678 +#define wxMDIParentFrame_ActivatePrevious 2679 +#define wxMDIParentFrame_ArrangeIcons 2680 +#define wxMDIParentFrame_Cascade 2681 +#define wxMDIParentFrame_Create 2682 +#define wxMDIParentFrame_GetActiveChild 2683 +#define wxMDIParentFrame_GetClientWindow 2684 +#define wxMDIParentFrame_Tile 2685 +#define wxMDIChildFrame_new_0 2686 +#define wxMDIChildFrame_new_4 2687 +#define wxMDIChildFrame_destruct 2688 +#define wxMDIChildFrame_Activate 2689 +#define wxMDIChildFrame_Create 2690 +#define wxMDIChildFrame_Maximize 2691 +#define wxMDIChildFrame_Restore 2692 +#define wxMDIClientWindow_new_0 2693 +#define wxMDIClientWindow_new_2 2694 +#define wxMDIClientWindow_destruct 2695 +#define wxMDIClientWindow_CreateClient 2696 +#define wxLayoutAlgorithm_new 2697 +#define wxLayoutAlgorithm_LayoutFrame 2698 +#define wxLayoutAlgorithm_LayoutMDIFrame 2699 +#define wxLayoutAlgorithm_LayoutWindow 2700 +#define wxLayoutAlgorithm_destroy 2701 +#define wxEvent_GetId 2702 +#define wxEvent_GetSkipped 2703 +#define wxEvent_GetTimestamp 2704 +#define wxEvent_IsCommandEvent 2705 +#define wxEvent_ResumePropagation 2706 +#define wxEvent_ShouldPropagate 2707 +#define wxEvent_Skip 2708 +#define wxEvent_StopPropagation 2709 +#define wxCommandEvent_getClientData 2710 +#define wxCommandEvent_GetExtraLong 2711 +#define wxCommandEvent_GetInt 2712 +#define wxCommandEvent_GetSelection 2713 +#define wxCommandEvent_GetString 2714 +#define wxCommandEvent_IsChecked 2715 +#define wxCommandEvent_IsSelection 2716 +#define wxCommandEvent_SetInt 2717 +#define wxCommandEvent_SetString 2718 +#define wxScrollEvent_GetOrientation 2719 +#define wxScrollEvent_GetPosition 2720 +#define wxScrollWinEvent_GetOrientation 2721 +#define wxScrollWinEvent_GetPosition 2722 +#define wxMouseEvent_AltDown 2723 +#define wxMouseEvent_Button 2724 +#define wxMouseEvent_ButtonDClick 2725 +#define wxMouseEvent_ButtonDown 2726 +#define wxMouseEvent_ButtonUp 2727 +#define wxMouseEvent_CmdDown 2728 +#define wxMouseEvent_ControlDown 2729 +#define wxMouseEvent_Dragging 2730 +#define wxMouseEvent_Entering 2731 +#define wxMouseEvent_GetButton 2732 +#define wxMouseEvent_GetPosition 2735 +#define wxMouseEvent_GetLogicalPosition 2736 +#define wxMouseEvent_GetLinesPerAction 2737 +#define wxMouseEvent_GetWheelRotation 2738 +#define wxMouseEvent_GetWheelDelta 2739 +#define wxMouseEvent_GetX 2740 +#define wxMouseEvent_GetY 2741 +#define wxMouseEvent_IsButton 2742 +#define wxMouseEvent_IsPageScroll 2743 +#define wxMouseEvent_Leaving 2744 +#define wxMouseEvent_LeftDClick 2745 +#define wxMouseEvent_LeftDown 2746 +#define wxMouseEvent_LeftIsDown 2747 +#define wxMouseEvent_LeftUp 2748 +#define wxMouseEvent_MetaDown 2749 +#define wxMouseEvent_MiddleDClick 2750 +#define wxMouseEvent_MiddleDown 2751 +#define wxMouseEvent_MiddleIsDown 2752 +#define wxMouseEvent_MiddleUp 2753 +#define wxMouseEvent_Moving 2754 +#define wxMouseEvent_RightDClick 2755 +#define wxMouseEvent_RightDown 2756 +#define wxMouseEvent_RightIsDown 2757 +#define wxMouseEvent_RightUp 2758 +#define wxMouseEvent_ShiftDown 2759 +#define wxSetCursorEvent_GetCursor 2760 +#define wxSetCursorEvent_GetX 2761 +#define wxSetCursorEvent_GetY 2762 +#define wxSetCursorEvent_HasCursor 2763 +#define wxSetCursorEvent_SetCursor 2764 +#define wxKeyEvent_AltDown 2765 +#define wxKeyEvent_CmdDown 2766 +#define wxKeyEvent_ControlDown 2767 +#define wxKeyEvent_GetKeyCode 2768 +#define wxKeyEvent_GetModifiers 2769 +#define wxKeyEvent_GetPosition 2772 +#define wxKeyEvent_GetRawKeyCode 2773 +#define wxKeyEvent_GetRawKeyFlags 2774 +#define wxKeyEvent_GetUnicodeKey 2775 +#define wxKeyEvent_GetX 2776 +#define wxKeyEvent_GetY 2777 +#define wxKeyEvent_HasModifiers 2778 +#define wxKeyEvent_MetaDown 2779 +#define wxKeyEvent_ShiftDown 2780 +#define wxSizeEvent_GetSize 2781 +#define wxMoveEvent_GetPosition 2782 +#define wxEraseEvent_GetDC 2783 +#define wxFocusEvent_GetWindow 2784 +#define wxChildFocusEvent_GetWindow 2785 +#define wxMenuEvent_GetMenu 2786 +#define wxMenuEvent_GetMenuId 2787 +#define wxMenuEvent_IsPopup 2788 +#define wxCloseEvent_CanVeto 2789 +#define wxCloseEvent_GetLoggingOff 2790 +#define wxCloseEvent_SetCanVeto 2791 +#define wxCloseEvent_SetLoggingOff 2792 +#define wxCloseEvent_Veto 2793 +#define wxShowEvent_SetShow 2794 +#define wxShowEvent_GetShow 2795 +#define wxIconizeEvent_Iconized 2796 +#define wxJoystickEvent_ButtonDown 2797 +#define wxJoystickEvent_ButtonIsDown 2798 +#define wxJoystickEvent_ButtonUp 2799 +#define wxJoystickEvent_GetButtonChange 2800 +#define wxJoystickEvent_GetButtonState 2801 +#define wxJoystickEvent_GetJoystick 2802 +#define wxJoystickEvent_GetPosition 2803 +#define wxJoystickEvent_GetZPosition 2804 +#define wxJoystickEvent_IsButton 2805 +#define wxJoystickEvent_IsMove 2806 +#define wxJoystickEvent_IsZMove 2807 +#define wxUpdateUIEvent_CanUpdate 2808 +#define wxUpdateUIEvent_Check 2809 +#define wxUpdateUIEvent_Enable 2810 +#define wxUpdateUIEvent_Show 2811 +#define wxUpdateUIEvent_GetChecked 2812 +#define wxUpdateUIEvent_GetEnabled 2813 +#define wxUpdateUIEvent_GetShown 2814 +#define wxUpdateUIEvent_GetSetChecked 2815 +#define wxUpdateUIEvent_GetSetEnabled 2816 +#define wxUpdateUIEvent_GetSetShown 2817 +#define wxUpdateUIEvent_GetSetText 2818 +#define wxUpdateUIEvent_GetText 2819 +#define wxUpdateUIEvent_GetMode 2820 +#define wxUpdateUIEvent_GetUpdateInterval 2821 +#define wxUpdateUIEvent_ResetUpdateTime 2822 +#define wxUpdateUIEvent_SetMode 2823 +#define wxUpdateUIEvent_SetText 2824 +#define wxUpdateUIEvent_SetUpdateInterval 2825 +#define wxMouseCaptureChangedEvent_GetCapturedWindow 2826 +#define wxPaletteChangedEvent_SetChangedWindow 2827 +#define wxPaletteChangedEvent_GetChangedWindow 2828 +#define wxQueryNewPaletteEvent_SetPaletteRealized 2829 +#define wxQueryNewPaletteEvent_GetPaletteRealized 2830 +#define wxNavigationKeyEvent_GetDirection 2831 +#define wxNavigationKeyEvent_SetDirection 2832 +#define wxNavigationKeyEvent_IsWindowChange 2833 +#define wxNavigationKeyEvent_SetWindowChange 2834 +#define wxNavigationKeyEvent_IsFromTab 2835 +#define wxNavigationKeyEvent_SetFromTab 2836 +#define wxNavigationKeyEvent_GetCurrentFocus 2837 +#define wxNavigationKeyEvent_SetCurrentFocus 2838 +#define wxHelpEvent_GetOrigin 2839 +#define wxHelpEvent_GetPosition 2840 +#define wxHelpEvent_SetOrigin 2841 +#define wxHelpEvent_SetPosition 2842 +#define wxContextMenuEvent_GetPosition 2843 +#define wxContextMenuEvent_SetPosition 2844 +#define wxIdleEvent_CanSend 2845 +#define wxIdleEvent_GetMode 2846 +#define wxIdleEvent_RequestMore 2847 +#define wxIdleEvent_MoreRequested 2848 +#define wxIdleEvent_SetMode 2849 +#define wxGridEvent_AltDown 2850 +#define wxGridEvent_ControlDown 2851 +#define wxGridEvent_GetCol 2852 +#define wxGridEvent_GetPosition 2853 +#define wxGridEvent_GetRow 2854 +#define wxGridEvent_MetaDown 2855 +#define wxGridEvent_Selecting 2856 +#define wxGridEvent_ShiftDown 2857 +#define wxNotifyEvent_Allow 2858 +#define wxNotifyEvent_IsAllowed 2859 +#define wxNotifyEvent_Veto 2860 +#define wxSashEvent_GetEdge 2861 +#define wxSashEvent_GetDragRect 2862 +#define wxSashEvent_GetDragStatus 2863 +#define wxListEvent_GetCacheFrom 2864 +#define wxListEvent_GetCacheTo 2865 +#define wxListEvent_GetKeyCode 2866 +#define wxListEvent_GetIndex 2867 +#define wxListEvent_GetColumn 2868 +#define wxListEvent_GetPoint 2869 +#define wxListEvent_GetLabel 2870 +#define wxListEvent_GetText 2871 +#define wxListEvent_GetImage 2872 +#define wxListEvent_GetData 2873 +#define wxListEvent_GetMask 2874 +#define wxListEvent_GetItem 2875 +#define wxListEvent_IsEditCancelled 2876 +#define wxDateEvent_GetDate 2877 +#define wxCalendarEvent_GetWeekDay 2878 +#define wxFileDirPickerEvent_GetPath 2879 +#define wxColourPickerEvent_GetColour 2880 +#define wxFontPickerEvent_GetFont 2881 +#define wxStyledTextEvent_GetPosition 2882 +#define wxStyledTextEvent_GetKey 2883 +#define wxStyledTextEvent_GetModifiers 2884 +#define wxStyledTextEvent_GetModificationType 2885 +#define wxStyledTextEvent_GetText 2886 +#define wxStyledTextEvent_GetLength 2887 +#define wxStyledTextEvent_GetLinesAdded 2888 +#define wxStyledTextEvent_GetLine 2889 +#define wxStyledTextEvent_GetFoldLevelNow 2890 +#define wxStyledTextEvent_GetFoldLevelPrev 2891 +#define wxStyledTextEvent_GetMargin 2892 +#define wxStyledTextEvent_GetMessage 2893 +#define wxStyledTextEvent_GetWParam 2894 +#define wxStyledTextEvent_GetLParam 2895 +#define wxStyledTextEvent_GetListType 2896 +#define wxStyledTextEvent_GetX 2897 +#define wxStyledTextEvent_GetY 2898 +#define wxStyledTextEvent_GetDragText 2899 +#define wxStyledTextEvent_GetDragAllowMove 2900 +#define wxStyledTextEvent_GetDragResult 2901 +#define wxStyledTextEvent_GetShift 2902 +#define wxStyledTextEvent_GetControl 2903 +#define wxStyledTextEvent_GetAlt 2904 +#define utils_wxGetKeyState 2905 +#define utils_wxGetMousePosition 2906 +#define utils_wxGetMouseState 2907 +#define utils_wxSetDetectableAutoRepeat 2908 +#define utils_wxBell 2909 +#define utils_wxFindMenuItemId 2910 +#define utils_wxGenericFindWindowAtPoint 2911 +#define utils_wxFindWindowAtPoint 2912 +#define utils_wxBeginBusyCursor 2913 +#define utils_wxEndBusyCursor 2914 +#define utils_wxIsBusy 2915 +#define utils_wxShutdown 2916 +#define utils_wxShell 2917 +#define utils_wxLaunchDefaultBrowser 2918 +#define utils_wxGetEmailAddress 2919 +#define utils_wxGetUserId 2920 +#define utils_wxGetHomeDir 2921 +#define utils_wxNewId 2922 +#define utils_wxRegisterId 2923 +#define utils_wxGetCurrentId 2924 +#define utils_wxGetOsDescription 2925 +#define utils_wxIsPlatformLittleEndian 2926 +#define utils_wxIsPlatform64Bit 2927 +#define gdicmn_wxDisplaySize 2928 +#define gdicmn_wxSetCursor 2929 +#define wxPrintout_new 2930 +#define wxPrintout_destruct 2931 +#define wxPrintout_GetDC 2932 +#define wxPrintout_GetPageSizeMM 2933 +#define wxPrintout_GetPageSizePixels 2934 +#define wxPrintout_GetPaperRectPixels 2935 +#define wxPrintout_GetPPIPrinter 2936 +#define wxPrintout_GetPPIScreen 2937 +#define wxPrintout_GetTitle 2938 +#define wxPrintout_IsPreview 2939 +#define wxPrintout_FitThisSizeToPaper 2940 +#define wxPrintout_FitThisSizeToPage 2941 +#define wxPrintout_FitThisSizeToPageMargins 2942 +#define wxPrintout_MapScreenSizeToPaper 2943 +#define wxPrintout_MapScreenSizeToPage 2944 +#define wxPrintout_MapScreenSizeToPageMargins 2945 +#define wxPrintout_MapScreenSizeToDevice 2946 +#define wxPrintout_GetLogicalPaperRect 2947 +#define wxPrintout_GetLogicalPageRect 2948 +#define wxPrintout_GetLogicalPageMarginsRect 2949 +#define wxPrintout_SetLogicalOrigin 2950 +#define wxPrintout_OffsetLogicalOrigin 2951 +#define wxStyledTextCtrl_new_2 2952 +#define wxStyledTextCtrl_new_0 2953 +#define wxStyledTextCtrl_destruct 2954 +#define wxStyledTextCtrl_Create 2955 +#define wxStyledTextCtrl_AddText 2956 +#define wxStyledTextCtrl_AddStyledText 2957 +#define wxStyledTextCtrl_InsertText 2958 +#define wxStyledTextCtrl_ClearAll 2959 +#define wxStyledTextCtrl_ClearDocumentStyle 2960 +#define wxStyledTextCtrl_GetLength 2961 +#define wxStyledTextCtrl_GetCharAt 2962 +#define wxStyledTextCtrl_GetCurrentPos 2963 +#define wxStyledTextCtrl_GetAnchor 2964 +#define wxStyledTextCtrl_GetStyleAt 2965 +#define wxStyledTextCtrl_Redo 2966 +#define wxStyledTextCtrl_SetUndoCollection 2967 +#define wxStyledTextCtrl_SelectAll 2968 +#define wxStyledTextCtrl_SetSavePoint 2969 +#define wxStyledTextCtrl_GetStyledText 2970 +#define wxStyledTextCtrl_CanRedo 2971 +#define wxStyledTextCtrl_MarkerLineFromHandle 2972 +#define wxStyledTextCtrl_MarkerDeleteHandle 2973 +#define wxStyledTextCtrl_GetUndoCollection 2974 +#define wxStyledTextCtrl_GetViewWhiteSpace 2975 +#define wxStyledTextCtrl_SetViewWhiteSpace 2976 +#define wxStyledTextCtrl_PositionFromPoint 2977 +#define wxStyledTextCtrl_PositionFromPointClose 2978 +#define wxStyledTextCtrl_GotoLine 2979 +#define wxStyledTextCtrl_GotoPos 2980 +#define wxStyledTextCtrl_SetAnchor 2981 +#define wxStyledTextCtrl_GetCurLine 2982 +#define wxStyledTextCtrl_GetEndStyled 2983 +#define wxStyledTextCtrl_ConvertEOLs 2984 +#define wxStyledTextCtrl_GetEOLMode 2985 +#define wxStyledTextCtrl_SetEOLMode 2986 +#define wxStyledTextCtrl_StartStyling 2987 +#define wxStyledTextCtrl_SetStyling 2988 +#define wxStyledTextCtrl_GetBufferedDraw 2989 +#define wxStyledTextCtrl_SetBufferedDraw 2990 +#define wxStyledTextCtrl_SetTabWidth 2991 +#define wxStyledTextCtrl_GetTabWidth 2992 +#define wxStyledTextCtrl_SetCodePage 2993 +#define wxStyledTextCtrl_MarkerDefine 2994 +#define wxStyledTextCtrl_MarkerSetForeground 2995 +#define wxStyledTextCtrl_MarkerSetBackground 2996 +#define wxStyledTextCtrl_MarkerAdd 2997 +#define wxStyledTextCtrl_MarkerDelete 2998 +#define wxStyledTextCtrl_MarkerDeleteAll 2999 +#define wxStyledTextCtrl_MarkerGet 3000 +#define wxStyledTextCtrl_MarkerNext 3001 +#define wxStyledTextCtrl_MarkerPrevious 3002 +#define wxStyledTextCtrl_MarkerDefineBitmap 3003 +#define wxStyledTextCtrl_MarkerAddSet 3004 +#define wxStyledTextCtrl_MarkerSetAlpha 3005 +#define wxStyledTextCtrl_SetMarginType 3006 +#define wxStyledTextCtrl_GetMarginType 3007 +#define wxStyledTextCtrl_SetMarginWidth 3008 +#define wxStyledTextCtrl_GetMarginWidth 3009 +#define wxStyledTextCtrl_SetMarginMask 3010 +#define wxStyledTextCtrl_GetMarginMask 3011 +#define wxStyledTextCtrl_SetMarginSensitive 3012 +#define wxStyledTextCtrl_GetMarginSensitive 3013 +#define wxStyledTextCtrl_StyleClearAll 3014 +#define wxStyledTextCtrl_StyleSetForeground 3015 +#define wxStyledTextCtrl_StyleSetBackground 3016 +#define wxStyledTextCtrl_StyleSetBold 3017 +#define wxStyledTextCtrl_StyleSetItalic 3018 +#define wxStyledTextCtrl_StyleSetSize 3019 +#define wxStyledTextCtrl_StyleSetFaceName 3020 +#define wxStyledTextCtrl_StyleSetEOLFilled 3021 +#define wxStyledTextCtrl_StyleResetDefault 3022 +#define wxStyledTextCtrl_StyleSetUnderline 3023 +#define wxStyledTextCtrl_StyleSetCase 3024 +#define wxStyledTextCtrl_StyleSetHotSpot 3025 +#define wxStyledTextCtrl_SetSelForeground 3026 +#define wxStyledTextCtrl_SetSelBackground 3027 +#define wxStyledTextCtrl_GetSelAlpha 3028 +#define wxStyledTextCtrl_SetSelAlpha 3029 +#define wxStyledTextCtrl_SetCaretForeground 3030 +#define wxStyledTextCtrl_CmdKeyAssign 3031 +#define wxStyledTextCtrl_CmdKeyClear 3032 +#define wxStyledTextCtrl_CmdKeyClearAll 3033 +#define wxStyledTextCtrl_SetStyleBytes 3034 +#define wxStyledTextCtrl_StyleSetVisible 3035 +#define wxStyledTextCtrl_GetCaretPeriod 3036 +#define wxStyledTextCtrl_SetCaretPeriod 3037 +#define wxStyledTextCtrl_SetWordChars 3038 +#define wxStyledTextCtrl_BeginUndoAction 3039 +#define wxStyledTextCtrl_EndUndoAction 3040 +#define wxStyledTextCtrl_IndicatorSetStyle 3041 +#define wxStyledTextCtrl_IndicatorGetStyle 3042 +#define wxStyledTextCtrl_IndicatorSetForeground 3043 +#define wxStyledTextCtrl_IndicatorGetForeground 3044 +#define wxStyledTextCtrl_SetWhitespaceForeground 3045 +#define wxStyledTextCtrl_SetWhitespaceBackground 3046 +#define wxStyledTextCtrl_GetStyleBits 3047 +#define wxStyledTextCtrl_SetLineState 3048 +#define wxStyledTextCtrl_GetLineState 3049 +#define wxStyledTextCtrl_GetMaxLineState 3050 +#define wxStyledTextCtrl_GetCaretLineVisible 3051 +#define wxStyledTextCtrl_SetCaretLineVisible 3052 +#define wxStyledTextCtrl_GetCaretLineBackground 3053 +#define wxStyledTextCtrl_SetCaretLineBackground 3054 +#define wxStyledTextCtrl_AutoCompShow 3055 +#define wxStyledTextCtrl_AutoCompCancel 3056 +#define wxStyledTextCtrl_AutoCompActive 3057 +#define wxStyledTextCtrl_AutoCompPosStart 3058 +#define wxStyledTextCtrl_AutoCompComplete 3059 +#define wxStyledTextCtrl_AutoCompStops 3060 +#define wxStyledTextCtrl_AutoCompSetSeparator 3061 +#define wxStyledTextCtrl_AutoCompGetSeparator 3062 +#define wxStyledTextCtrl_AutoCompSelect 3063 +#define wxStyledTextCtrl_AutoCompSetCancelAtStart 3064 +#define wxStyledTextCtrl_AutoCompGetCancelAtStart 3065 +#define wxStyledTextCtrl_AutoCompSetFillUps 3066 +#define wxStyledTextCtrl_AutoCompSetChooseSingle 3067 +#define wxStyledTextCtrl_AutoCompGetChooseSingle 3068 +#define wxStyledTextCtrl_AutoCompSetIgnoreCase 3069 +#define wxStyledTextCtrl_AutoCompGetIgnoreCase 3070 +#define wxStyledTextCtrl_UserListShow 3071 +#define wxStyledTextCtrl_AutoCompSetAutoHide 3072 +#define wxStyledTextCtrl_AutoCompGetAutoHide 3073 +#define wxStyledTextCtrl_AutoCompSetDropRestOfWord 3074 +#define wxStyledTextCtrl_AutoCompGetDropRestOfWord 3075 +#define wxStyledTextCtrl_RegisterImage 3076 +#define wxStyledTextCtrl_ClearRegisteredImages 3077 +#define wxStyledTextCtrl_AutoCompGetTypeSeparator 3078 +#define wxStyledTextCtrl_AutoCompSetTypeSeparator 3079 +#define wxStyledTextCtrl_AutoCompSetMaxWidth 3080 +#define wxStyledTextCtrl_AutoCompGetMaxWidth 3081 +#define wxStyledTextCtrl_AutoCompSetMaxHeight 3082 +#define wxStyledTextCtrl_AutoCompGetMaxHeight 3083 +#define wxStyledTextCtrl_SetIndent 3084 +#define wxStyledTextCtrl_GetIndent 3085 +#define wxStyledTextCtrl_SetUseTabs 3086 +#define wxStyledTextCtrl_GetUseTabs 3087 +#define wxStyledTextCtrl_SetLineIndentation 3088 +#define wxStyledTextCtrl_GetLineIndentation 3089 +#define wxStyledTextCtrl_GetLineIndentPosition 3090 +#define wxStyledTextCtrl_GetColumn 3091 +#define wxStyledTextCtrl_SetUseHorizontalScrollBar 3092 +#define wxStyledTextCtrl_GetUseHorizontalScrollBar 3093 +#define wxStyledTextCtrl_SetIndentationGuides 3094 +#define wxStyledTextCtrl_GetIndentationGuides 3095 +#define wxStyledTextCtrl_SetHighlightGuide 3096 +#define wxStyledTextCtrl_GetHighlightGuide 3097 +#define wxStyledTextCtrl_GetLineEndPosition 3098 +#define wxStyledTextCtrl_GetCodePage 3099 +#define wxStyledTextCtrl_GetCaretForeground 3100 +#define wxStyledTextCtrl_GetReadOnly 3101 +#define wxStyledTextCtrl_SetCurrentPos 3102 +#define wxStyledTextCtrl_SetSelectionStart 3103 +#define wxStyledTextCtrl_GetSelectionStart 3104 +#define wxStyledTextCtrl_SetSelectionEnd 3105 +#define wxStyledTextCtrl_GetSelectionEnd 3106 +#define wxStyledTextCtrl_SetPrintMagnification 3107 +#define wxStyledTextCtrl_GetPrintMagnification 3108 +#define wxStyledTextCtrl_SetPrintColourMode 3109 +#define wxStyledTextCtrl_GetPrintColourMode 3110 +#define wxStyledTextCtrl_FindText 3111 +#define wxStyledTextCtrl_FormatRange 3112 +#define wxStyledTextCtrl_GetFirstVisibleLine 3113 +#define wxStyledTextCtrl_GetLine 3114 +#define wxStyledTextCtrl_GetLineCount 3115 +#define wxStyledTextCtrl_SetMarginLeft 3116 +#define wxStyledTextCtrl_GetMarginLeft 3117 +#define wxStyledTextCtrl_SetMarginRight 3118 +#define wxStyledTextCtrl_GetMarginRight 3119 +#define wxStyledTextCtrl_GetModify 3120 +#define wxStyledTextCtrl_SetSelection 3121 +#define wxStyledTextCtrl_GetSelectedText 3122 +#define wxStyledTextCtrl_GetTextRange 3123 +#define wxStyledTextCtrl_HideSelection 3124 +#define wxStyledTextCtrl_LineFromPosition 3125 +#define wxStyledTextCtrl_PositionFromLine 3126 +#define wxStyledTextCtrl_LineScroll 3127 +#define wxStyledTextCtrl_EnsureCaretVisible 3128 +#define wxStyledTextCtrl_ReplaceSelection 3129 +#define wxStyledTextCtrl_SetReadOnly 3130 +#define wxStyledTextCtrl_CanPaste 3131 +#define wxStyledTextCtrl_CanUndo 3132 +#define wxStyledTextCtrl_EmptyUndoBuffer 3133 +#define wxStyledTextCtrl_Undo 3134 +#define wxStyledTextCtrl_Cut 3135 +#define wxStyledTextCtrl_Copy 3136 +#define wxStyledTextCtrl_Paste 3137 +#define wxStyledTextCtrl_Clear 3138 +#define wxStyledTextCtrl_SetText 3139 +#define wxStyledTextCtrl_GetText 3140 +#define wxStyledTextCtrl_GetTextLength 3141 +#define wxStyledTextCtrl_GetOvertype 3142 +#define wxStyledTextCtrl_SetCaretWidth 3143 +#define wxStyledTextCtrl_GetCaretWidth 3144 +#define wxStyledTextCtrl_SetTargetStart 3145 +#define wxStyledTextCtrl_GetTargetStart 3146 +#define wxStyledTextCtrl_SetTargetEnd 3147 +#define wxStyledTextCtrl_GetTargetEnd 3148 +#define wxStyledTextCtrl_ReplaceTarget 3149 +#define wxStyledTextCtrl_SearchInTarget 3150 +#define wxStyledTextCtrl_SetSearchFlags 3151 +#define wxStyledTextCtrl_GetSearchFlags 3152 +#define wxStyledTextCtrl_CallTipShow 3153 +#define wxStyledTextCtrl_CallTipCancel 3154 +#define wxStyledTextCtrl_CallTipActive 3155 +#define wxStyledTextCtrl_CallTipPosAtStart 3156 +#define wxStyledTextCtrl_CallTipSetHighlight 3157 +#define wxStyledTextCtrl_CallTipSetBackground 3158 +#define wxStyledTextCtrl_CallTipSetForeground 3159 +#define wxStyledTextCtrl_CallTipSetForegroundHighlight 3160 +#define wxStyledTextCtrl_CallTipUseStyle 3161 +#define wxStyledTextCtrl_VisibleFromDocLine 3162 +#define wxStyledTextCtrl_DocLineFromVisible 3163 +#define wxStyledTextCtrl_WrapCount 3164 +#define wxStyledTextCtrl_SetFoldLevel 3165 +#define wxStyledTextCtrl_GetFoldLevel 3166 +#define wxStyledTextCtrl_GetLastChild 3167 +#define wxStyledTextCtrl_GetFoldParent 3168 +#define wxStyledTextCtrl_ShowLines 3169 +#define wxStyledTextCtrl_HideLines 3170 +#define wxStyledTextCtrl_GetLineVisible 3171 +#define wxStyledTextCtrl_SetFoldExpanded 3172 +#define wxStyledTextCtrl_GetFoldExpanded 3173 +#define wxStyledTextCtrl_ToggleFold 3174 +#define wxStyledTextCtrl_EnsureVisible 3175 +#define wxStyledTextCtrl_SetFoldFlags 3176 +#define wxStyledTextCtrl_EnsureVisibleEnforcePolicy 3177 +#define wxStyledTextCtrl_SetTabIndents 3178 +#define wxStyledTextCtrl_GetTabIndents 3179 +#define wxStyledTextCtrl_SetBackSpaceUnIndents 3180 +#define wxStyledTextCtrl_GetBackSpaceUnIndents 3181 +#define wxStyledTextCtrl_SetMouseDwellTime 3182 +#define wxStyledTextCtrl_GetMouseDwellTime 3183 +#define wxStyledTextCtrl_WordStartPosition 3184 +#define wxStyledTextCtrl_WordEndPosition 3185 +#define wxStyledTextCtrl_SetWrapMode 3186 +#define wxStyledTextCtrl_GetWrapMode 3187 +#define wxStyledTextCtrl_SetWrapVisualFlags 3188 +#define wxStyledTextCtrl_GetWrapVisualFlags 3189 +#define wxStyledTextCtrl_SetWrapVisualFlagsLocation 3190 +#define wxStyledTextCtrl_GetWrapVisualFlagsLocation 3191 +#define wxStyledTextCtrl_SetWrapStartIndent 3192 +#define wxStyledTextCtrl_GetWrapStartIndent 3193 +#define wxStyledTextCtrl_SetLayoutCache 3194 +#define wxStyledTextCtrl_GetLayoutCache 3195 +#define wxStyledTextCtrl_SetScrollWidth 3196 +#define wxStyledTextCtrl_GetScrollWidth 3197 +#define wxStyledTextCtrl_TextWidth 3198 +#define wxStyledTextCtrl_GetEndAtLastLine 3199 +#define wxStyledTextCtrl_TextHeight 3200 +#define wxStyledTextCtrl_SetUseVerticalScrollBar 3201 +#define wxStyledTextCtrl_GetUseVerticalScrollBar 3202 +#define wxStyledTextCtrl_AppendText 3203 +#define wxStyledTextCtrl_GetTwoPhaseDraw 3204 +#define wxStyledTextCtrl_SetTwoPhaseDraw 3205 +#define wxStyledTextCtrl_TargetFromSelection 3206 +#define wxStyledTextCtrl_LinesJoin 3207 +#define wxStyledTextCtrl_LinesSplit 3208 +#define wxStyledTextCtrl_SetFoldMarginColour 3209 +#define wxStyledTextCtrl_SetFoldMarginHiColour 3210 +#define wxStyledTextCtrl_LineDown 3211 +#define wxStyledTextCtrl_LineDownExtend 3212 +#define wxStyledTextCtrl_LineUp 3213 +#define wxStyledTextCtrl_LineUpExtend 3214 +#define wxStyledTextCtrl_CharLeft 3215 +#define wxStyledTextCtrl_CharLeftExtend 3216 +#define wxStyledTextCtrl_CharRight 3217 +#define wxStyledTextCtrl_CharRightExtend 3218 +#define wxStyledTextCtrl_WordLeft 3219 +#define wxStyledTextCtrl_WordLeftExtend 3220 +#define wxStyledTextCtrl_WordRight 3221 +#define wxStyledTextCtrl_WordRightExtend 3222 +#define wxStyledTextCtrl_Home 3223 +#define wxStyledTextCtrl_HomeExtend 3224 +#define wxStyledTextCtrl_LineEnd 3225 +#define wxStyledTextCtrl_LineEndExtend 3226 +#define wxStyledTextCtrl_DocumentStart 3227 +#define wxStyledTextCtrl_DocumentStartExtend 3228 +#define wxStyledTextCtrl_DocumentEnd 3229 +#define wxStyledTextCtrl_DocumentEndExtend 3230 +#define wxStyledTextCtrl_PageUp 3231 +#define wxStyledTextCtrl_PageUpExtend 3232 +#define wxStyledTextCtrl_PageDown 3233 +#define wxStyledTextCtrl_PageDownExtend 3234 +#define wxStyledTextCtrl_EditToggleOvertype 3235 +#define wxStyledTextCtrl_Cancel 3236 +#define wxStyledTextCtrl_DeleteBack 3237 +#define wxStyledTextCtrl_Tab 3238 +#define wxStyledTextCtrl_BackTab 3239 +#define wxStyledTextCtrl_NewLine 3240 +#define wxStyledTextCtrl_FormFeed 3241 +#define wxStyledTextCtrl_VCHome 3242 +#define wxStyledTextCtrl_VCHomeExtend 3243 +#define wxStyledTextCtrl_ZoomIn 3244 +#define wxStyledTextCtrl_ZoomOut 3245 +#define wxStyledTextCtrl_DelWordLeft 3246 +#define wxStyledTextCtrl_DelWordRight 3247 +#define wxStyledTextCtrl_LineCut 3248 +#define wxStyledTextCtrl_LineDelete 3249 +#define wxStyledTextCtrl_LineTranspose 3250 +#define wxStyledTextCtrl_LineDuplicate 3251 +#define wxStyledTextCtrl_LowerCase 3252 +#define wxStyledTextCtrl_UpperCase 3253 +#define wxStyledTextCtrl_LineScrollDown 3254 +#define wxStyledTextCtrl_LineScrollUp 3255 +#define wxStyledTextCtrl_DeleteBackNotLine 3256 +#define wxStyledTextCtrl_HomeDisplay 3257 +#define wxStyledTextCtrl_HomeDisplayExtend 3258 +#define wxStyledTextCtrl_LineEndDisplay 3259 +#define wxStyledTextCtrl_LineEndDisplayExtend 3260 +#define wxStyledTextCtrl_HomeWrapExtend 3261 +#define wxStyledTextCtrl_LineEndWrap 3262 +#define wxStyledTextCtrl_LineEndWrapExtend 3263 +#define wxStyledTextCtrl_VCHomeWrap 3264 +#define wxStyledTextCtrl_VCHomeWrapExtend 3265 +#define wxStyledTextCtrl_LineCopy 3266 +#define wxStyledTextCtrl_MoveCaretInsideView 3267 +#define wxStyledTextCtrl_LineLength 3268 +#define wxStyledTextCtrl_BraceHighlight 3269 +#define wxStyledTextCtrl_BraceBadLight 3270 +#define wxStyledTextCtrl_BraceMatch 3271 +#define wxStyledTextCtrl_GetViewEOL 3272 +#define wxStyledTextCtrl_SetViewEOL 3273 +#define wxStyledTextCtrl_SetModEventMask 3274 +#define wxStyledTextCtrl_GetEdgeColumn 3275 +#define wxStyledTextCtrl_SetEdgeColumn 3276 +#define wxStyledTextCtrl_SetEdgeMode 3277 +#define wxStyledTextCtrl_GetEdgeMode 3278 +#define wxStyledTextCtrl_GetEdgeColour 3279 +#define wxStyledTextCtrl_SetEdgeColour 3280 +#define wxStyledTextCtrl_SearchAnchor 3281 +#define wxStyledTextCtrl_SearchNext 3282 +#define wxStyledTextCtrl_SearchPrev 3283 +#define wxStyledTextCtrl_LinesOnScreen 3284 +#define wxStyledTextCtrl_UsePopUp 3285 +#define wxStyledTextCtrl_SelectionIsRectangle 3286 +#define wxStyledTextCtrl_SetZoom 3287 +#define wxStyledTextCtrl_GetZoom 3288 +#define wxStyledTextCtrl_GetModEventMask 3289 +#define wxStyledTextCtrl_SetSTCFocus 3290 +#define wxStyledTextCtrl_GetSTCFocus 3291 +#define wxStyledTextCtrl_SetStatus 3292 +#define wxStyledTextCtrl_GetStatus 3293 +#define wxStyledTextCtrl_SetMouseDownCaptures 3294 +#define wxStyledTextCtrl_GetMouseDownCaptures 3295 +#define wxStyledTextCtrl_SetSTCCursor 3296 +#define wxStyledTextCtrl_GetSTCCursor 3297 +#define wxStyledTextCtrl_SetControlCharSymbol 3298 +#define wxStyledTextCtrl_GetControlCharSymbol 3299 +#define wxStyledTextCtrl_WordPartLeft 3300 +#define wxStyledTextCtrl_WordPartLeftExtend 3301 +#define wxStyledTextCtrl_WordPartRight 3302 +#define wxStyledTextCtrl_WordPartRightExtend 3303 +#define wxStyledTextCtrl_SetVisiblePolicy 3304 +#define wxStyledTextCtrl_DelLineLeft 3305 +#define wxStyledTextCtrl_DelLineRight 3306 +#define wxStyledTextCtrl_GetXOffset 3307 +#define wxStyledTextCtrl_ChooseCaretX 3308 +#define wxStyledTextCtrl_SetXCaretPolicy 3309 +#define wxStyledTextCtrl_SetYCaretPolicy 3310 +#define wxStyledTextCtrl_GetPrintWrapMode 3311 +#define wxStyledTextCtrl_SetHotspotActiveForeground 3312 +#define wxStyledTextCtrl_SetHotspotActiveBackground 3313 +#define wxStyledTextCtrl_SetHotspotActiveUnderline 3314 +#define wxStyledTextCtrl_SetHotspotSingleLine 3315 +#define wxStyledTextCtrl_ParaDownExtend 3316 +#define wxStyledTextCtrl_ParaUp 3317 +#define wxStyledTextCtrl_ParaUpExtend 3318 +#define wxStyledTextCtrl_PositionBefore 3319 +#define wxStyledTextCtrl_PositionAfter 3320 +#define wxStyledTextCtrl_CopyRange 3321 +#define wxStyledTextCtrl_CopyText 3322 +#define wxStyledTextCtrl_SetSelectionMode 3323 +#define wxStyledTextCtrl_GetSelectionMode 3324 +#define wxStyledTextCtrl_LineDownRectExtend 3325 +#define wxStyledTextCtrl_LineUpRectExtend 3326 +#define wxStyledTextCtrl_CharLeftRectExtend 3327 +#define wxStyledTextCtrl_CharRightRectExtend 3328 +#define wxStyledTextCtrl_HomeRectExtend 3329 +#define wxStyledTextCtrl_VCHomeRectExtend 3330 +#define wxStyledTextCtrl_LineEndRectExtend 3331 +#define wxStyledTextCtrl_PageUpRectExtend 3332 +#define wxStyledTextCtrl_PageDownRectExtend 3333 +#define wxStyledTextCtrl_StutteredPageUp 3334 +#define wxStyledTextCtrl_StutteredPageUpExtend 3335 +#define wxStyledTextCtrl_StutteredPageDown 3336 +#define wxStyledTextCtrl_StutteredPageDownExtend 3337 +#define wxStyledTextCtrl_WordLeftEnd 3338 +#define wxStyledTextCtrl_WordLeftEndExtend 3339 +#define wxStyledTextCtrl_WordRightEnd 3340 +#define wxStyledTextCtrl_WordRightEndExtend 3341 +#define wxStyledTextCtrl_SetWhitespaceChars 3342 +#define wxStyledTextCtrl_SetCharsDefault 3343 +#define wxStyledTextCtrl_AutoCompGetCurrent 3344 +#define wxStyledTextCtrl_Allocate 3345 +#define wxStyledTextCtrl_FindColumn 3346 +#define wxStyledTextCtrl_GetCaretSticky 3347 +#define wxStyledTextCtrl_SetCaretSticky 3348 +#define wxStyledTextCtrl_ToggleCaretSticky 3349 +#define wxStyledTextCtrl_SetPasteConvertEndings 3350 +#define wxStyledTextCtrl_GetPasteConvertEndings 3351 +#define wxStyledTextCtrl_SelectionDuplicate 3352 +#define wxStyledTextCtrl_SetCaretLineBackAlpha 3353 +#define wxStyledTextCtrl_GetCaretLineBackAlpha 3354 +#define wxStyledTextCtrl_StartRecord 3355 +#define wxStyledTextCtrl_StopRecord 3356 +#define wxStyledTextCtrl_SetLexer 3357 +#define wxStyledTextCtrl_GetLexer 3358 +#define wxStyledTextCtrl_Colourise 3359 +#define wxStyledTextCtrl_SetProperty 3360 +#define wxStyledTextCtrl_SetKeyWords 3361 +#define wxStyledTextCtrl_SetLexerLanguage 3362 +#define wxStyledTextCtrl_GetProperty 3363 +#define wxStyledTextCtrl_GetStyleBitsNeeded 3364 +#define wxStyledTextCtrl_GetCurrentLine 3365 +#define wxStyledTextCtrl_StyleSetSpec 3366 +#define wxStyledTextCtrl_StyleSetFont 3367 +#define wxStyledTextCtrl_StyleSetFontAttr 3368 +#define wxStyledTextCtrl_StyleSetCharacterSet 3369 +#define wxStyledTextCtrl_StyleSetFontEncoding 3370 +#define wxStyledTextCtrl_CmdKeyExecute 3371 +#define wxStyledTextCtrl_SetMargins 3372 +#define wxStyledTextCtrl_GetSelection 3373 +#define wxStyledTextCtrl_PointFromPosition 3374 +#define wxStyledTextCtrl_ScrollToLine 3375 +#define wxStyledTextCtrl_ScrollToColumn 3376 +#define wxStyledTextCtrl_SetVScrollBar 3377 +#define wxStyledTextCtrl_SetHScrollBar 3378 +#define wxStyledTextCtrl_GetLastKeydownProcessed 3379 +#define wxStyledTextCtrl_SetLastKeydownProcessed 3380 +#define wxStyledTextCtrl_SaveFile 3381 +#define wxStyledTextCtrl_LoadFile 3382 +#define wxStyledTextCtrl_DoDragOver 3383 +#define wxStyledTextCtrl_DoDropText 3384 +#define wxStyledTextCtrl_GetUseAntiAliasing 3385 +#define wxStyledTextCtrl_AddTextRaw 3386 +#define wxStyledTextCtrl_InsertTextRaw 3387 +#define wxStyledTextCtrl_GetCurLineRaw 3388 +#define wxStyledTextCtrl_GetLineRaw 3389 +#define wxStyledTextCtrl_GetSelectedTextRaw 3390 +#define wxStyledTextCtrl_GetTextRangeRaw 3391 +#define wxStyledTextCtrl_SetTextRaw 3392 +#define wxStyledTextCtrl_GetTextRaw 3393 +#define wxStyledTextCtrl_AppendTextRaw 3394 +#define wxArtProvider_GetBitmap 3395 +#define wxArtProvider_GetIcon 3396 +#define wxTreeEvent_GetKeyCode 3397 +#define wxTreeEvent_GetItem 3398 +#define wxTreeEvent_GetKeyEvent 3399 +#define wxTreeEvent_GetLabel 3400 +#define wxTreeEvent_GetOldItem 3401 +#define wxTreeEvent_GetPoint 3402 +#define wxTreeEvent_IsEditCancelled 3403 +#define wxTreeEvent_SetToolTip 3404 +#define wxNotebookEvent_GetOldSelection 3405 +#define wxNotebookEvent_GetSelection 3406 +#define wxNotebookEvent_SetOldSelection 3407 +#define wxNotebookEvent_SetSelection 3408 +#define wxFileDataObject_new 3409 +#define wxFileDataObject_AddFile 3410 +#define wxFileDataObject_GetFilenames 3411 +#define wxFileDataObject_destroy 3412 +#define wxTextDataObject_new 3413 +#define wxTextDataObject_GetTextLength 3414 +#define wxTextDataObject_GetText 3415 +#define wxTextDataObject_SetText 3416 +#define wxTextDataObject_destroy 3417 +#define wxBitmapDataObject_new_1_1 3418 +#define wxBitmapDataObject_new_1_0 3419 +#define wxBitmapDataObject_GetBitmap 3420 +#define wxBitmapDataObject_SetBitmap 3421 +#define wxBitmapDataObject_destroy 3422 +#define wxClipboard_new 3424 +#define wxClipboard_destruct 3425 +#define wxClipboard_AddData 3426 +#define wxClipboard_Clear 3427 +#define wxClipboard_Close 3428 +#define wxClipboard_Flush 3429 +#define wxClipboard_GetData 3430 +#define wxClipboard_IsOpened 3431 +#define wxClipboard_Open 3432 +#define wxClipboard_SetData 3433 +#define wxClipboard_UsePrimarySelection 3435 +#define wxClipboard_IsSupported 3436 +#define wxClipboard_Get 3437 +#define wxSpinEvent_GetPosition 3438 +#define wxSpinEvent_SetPosition 3439 +#define wxSplitterWindow_new_0 3440 +#define wxSplitterWindow_new_2 3441 +#define wxSplitterWindow_destruct 3442 +#define wxSplitterWindow_Create 3443 +#define wxSplitterWindow_GetMinimumPaneSize 3444 +#define wxSplitterWindow_GetSashGravity 3445 +#define wxSplitterWindow_GetSashPosition 3446 +#define wxSplitterWindow_GetSplitMode 3447 +#define wxSplitterWindow_GetWindow1 3448 +#define wxSplitterWindow_GetWindow2 3449 +#define wxSplitterWindow_Initialize 3450 +#define wxSplitterWindow_IsSplit 3451 +#define wxSplitterWindow_ReplaceWindow 3452 +#define wxSplitterWindow_SetSashGravity 3453 +#define wxSplitterWindow_SetSashPosition 3454 +#define wxSplitterWindow_SetSashSize 3455 +#define wxSplitterWindow_SetMinimumPaneSize 3456 +#define wxSplitterWindow_SetSplitMode 3457 +#define wxSplitterWindow_SplitHorizontally 3458 +#define wxSplitterWindow_SplitVertically 3459 +#define wxSplitterWindow_Unsplit 3460 +#define wxSplitterWindow_UpdateSize 3461 +#define wxSplitterEvent_GetSashPosition 3462 +#define wxSplitterEvent_GetX 3463 +#define wxSplitterEvent_GetY 3464 +#define wxSplitterEvent_GetWindowBeingRemoved 3465 +#define wxSplitterEvent_SetSashPosition 3466 +#define wxHtmlWindow_new_0 3467 +#define wxHtmlWindow_new_2 3468 +#define wxHtmlWindow_AppendToPage 3469 +#define wxHtmlWindow_GetOpenedAnchor 3470 +#define wxHtmlWindow_GetOpenedPage 3471 +#define wxHtmlWindow_GetOpenedPageTitle 3472 +#define wxHtmlWindow_GetRelatedFrame 3473 +#define wxHtmlWindow_HistoryBack 3474 +#define wxHtmlWindow_HistoryCanBack 3475 +#define wxHtmlWindow_HistoryCanForward 3476 +#define wxHtmlWindow_HistoryClear 3477 +#define wxHtmlWindow_HistoryForward 3478 +#define wxHtmlWindow_LoadFile 3479 +#define wxHtmlWindow_LoadPage 3480 +#define wxHtmlWindow_SelectAll 3481 +#define wxHtmlWindow_SelectionToText 3482 +#define wxHtmlWindow_SelectLine 3483 +#define wxHtmlWindow_SelectWord 3484 +#define wxHtmlWindow_SetBorders 3485 +#define wxHtmlWindow_SetFonts 3486 +#define wxHtmlWindow_SetPage 3487 +#define wxHtmlWindow_SetRelatedFrame 3488 +#define wxHtmlWindow_SetRelatedStatusBar 3489 +#define wxHtmlWindow_ToText 3490 +#define wxHtmlWindow_destroy 3491 +#define wxHtmlLinkEvent_GetLinkInfo 3492 +#define wxSystemSettings_GetColour 3493 +#define wxSystemSettings_GetFont 3494 +#define wxSystemSettings_GetMetric 3495 +#define wxSystemSettings_GetScreenType 3496 +#define wxSystemOptions_GetOption 3497 +#define wxSystemOptions_GetOptionInt 3498 +#define wxSystemOptions_HasOption 3499 +#define wxSystemOptions_IsFalse 3500 +#define wxSystemOptions_SetOption_2_1 3501 +#define wxSystemOptions_SetOption_2_0 3502 +#define wxAuiNotebookEvent_SetSelection 3503 +#define wxAuiNotebookEvent_GetSelection 3504 +#define wxAuiNotebookEvent_SetOldSelection 3505 +#define wxAuiNotebookEvent_GetOldSelection 3506 +#define wxAuiNotebookEvent_SetDragSource 3507 +#define wxAuiNotebookEvent_GetDragSource 3508 +#define wxAuiManagerEvent_SetManager 3509 +#define wxAuiManagerEvent_GetManager 3510 +#define wxAuiManagerEvent_SetPane 3511 +#define wxAuiManagerEvent_GetPane 3512 +#define wxAuiManagerEvent_SetButton 3513 +#define wxAuiManagerEvent_GetButton 3514 +#define wxAuiManagerEvent_SetDC 3515 +#define wxAuiManagerEvent_GetDC 3516 +#define wxAuiManagerEvent_Veto 3517 +#define wxAuiManagerEvent_GetVeto 3518 +#define wxAuiManagerEvent_SetCanVeto 3519 +#define wxAuiManagerEvent_CanVeto 3520 +#define wxLogNull_new 3521 +#define wxLogNull_destroy 3522 +#define wxTaskBarIcon_new 3523 +#define wxTaskBarIcon_destruct 3524 +#define wxTaskBarIcon_PopupMenu 3525 +#define wxTaskBarIcon_RemoveIcon 3526 +#define wxTaskBarIcon_SetIcon 3527 +#define wxLocale_new_0 3528 +#define wxLocale_new_2 3530 +#define wxLocale_destruct 3531 +#define wxLocale_Init 3533 +#define wxLocale_AddCatalog_1 3534 +#define wxLocale_AddCatalog_3 3535 +#define wxLocale_AddCatalogLookupPathPrefix 3536 +#define wxLocale_GetCanonicalName 3537 +#define wxLocale_GetLanguage 3538 +#define wxLocale_GetLanguageName 3539 +#define wxLocale_GetLocale 3540 +#define wxLocale_GetName 3541 +#define wxLocale_GetString_2 3542 +#define wxLocale_GetString_4 3543 +#define wxLocale_GetHeaderValue 3544 +#define wxLocale_GetSysName 3545 +#define wxLocale_GetSystemEncoding 3546 +#define wxLocale_GetSystemEncodingName 3547 +#define wxLocale_GetSystemLanguage 3548 +#define wxLocale_IsLoaded 3549 +#define wxLocale_IsOk 3550 +#define wxActivateEvent_GetActive 3551 +#define wxPopupWindow_new_2 3553 +#define wxPopupWindow_new_0 3554 +#define wxPopupWindow_destruct 3556 +#define wxPopupWindow_Create 3557 +#define wxPopupWindow_Position 3558 +#define wxPopupTransientWindow_new_0 3559 +#define wxPopupTransientWindow_new_2 3560 +#define wxPopupTransientWindow_destruct 3561 +#define wxPopupTransientWindow_Popup 3562 +#define wxPopupTransientWindow_Dismiss 3563 diff --git a/lib/wx/c_src/wxe_callback_impl.cpp b/lib/wx/c_src/wxe_callback_impl.cpp index e06f68dcbf..3a59004eb0 100644 --- a/lib/wx/c_src/wxe_callback_impl.cpp +++ b/lib/wx/c_src/wxe_callback_impl.cpp @@ -284,7 +284,7 @@ EwxListCtrl::~EwxListCtrl() { * wxListCtrlCompare wrapper * ****************************************************************************/ -int wxCALLBACK wxEListCtrlCompare(long item1, long item2, long callbackInfoPtr) +int wxCALLBACK wxEListCtrlCompare(wxeIntPtr item1, wxeIntPtr item2, wxeIntPtr callbackInfoPtr) { callbackInfo * cb = (callbackInfo *)callbackInfoPtr; wxeMemEnv * memenv = ((WxeApp *) wxTheApp)->getMemEnv(cb->port); diff --git a/lib/wx/c_src/wxe_callback_impl.h b/lib/wx/c_src/wxe_callback_impl.h index 1c355e4d38..ecfcd3db41 100644 --- a/lib/wx/c_src/wxe_callback_impl.h +++ b/lib/wx/c_src/wxe_callback_impl.h @@ -23,6 +23,13 @@ void pre_callback(); void handle_event_callback(ErlDrvPort port, ErlDrvTermData process); +#if wxCHECK_VERSION(2,9,0) + #define wxeIntPtr wxIntPtr +#else + // This is bad but how it was in wx-2.8 + #define wxeIntPtr long +#endif + /* Fun Callback id */ class wxeEvtListener : public wxEvtHandler { @@ -88,6 +95,6 @@ struct callbackInfo { int callbackID; }; -int wxCALLBACK wxEListCtrlCompare(long item1, long item2, long callbackInfoPtr); +int wxCALLBACK wxEListCtrlCompare(wxeIntPtr item1, wxeIntPtr item2, wxeIntPtr callbackInfoPtr); #endif diff --git a/lib/wx/c_src/wxe_impl.cpp b/lib/wx/c_src/wxe_impl.cpp index 5964ccfd00..0ee52e3af2 100644 --- a/lib/wx/c_src/wxe_impl.cpp +++ b/lib/wx/c_src/wxe_impl.cpp @@ -104,7 +104,7 @@ void meta_command(int what, wxe_data *sd) { } } -void send_msg(const char * type, wxString * msg) { +void send_msg(const char * type, const wxString * msg) { wxeReturn rt = wxeReturn(WXE_DRV_PORT, init_caller); rt.addAtom((char *) "wxe_driver"); rt.addAtom((char *) type); @@ -160,6 +160,13 @@ bool WxeApp::OnInit() return TRUE; } + +#ifdef _MACOSX +void WxeApp::MacOpenFile(const wxString &filename) { + send_msg("open_file", &filename); +} +#endif + void WxeApp::shutdown(wxeMetaCommand& Ecmd) { ExitMainLoop(); } @@ -411,6 +418,13 @@ void WxeApp::destroyMemEnv(wxeMetaCommand& Ecmd) wxWindow *parent = NULL; wxeMemEnv * memenv = refmap[Ecmd.port]; + if(!memenv) { + wxString msg; + msg.Printf(wxT("MemEnv already deleted")); + send_msg("debug", &msg); + return; + } + if(wxe_debug) { wxString msg; msg.Printf(wxT("Destroying all memory ")); @@ -439,7 +453,6 @@ void WxeApp::destroyMemEnv(wxeMetaCommand& Ecmd) } if(recurse_level > 0) { // Delay delete until we are out of dispatch* - delayed_cleanup->Append(Ecmd.Clone()); } else { delete win; } @@ -448,9 +461,10 @@ void WxeApp::destroyMemEnv(wxeMetaCommand& Ecmd) } } - if(recurse_level > 0) + if(recurse_level > 0) { + delayed_cleanup->Append(Ecmd.Clone()); return; - + } // First pass, delete all top parents/windows of all linked objects // fprintf(stderr, "close port %x\r\n", Ecmd.port);fflush(stderr); @@ -486,20 +500,20 @@ void WxeApp::destroyMemEnv(wxeMetaCommand& Ecmd) if(it != ptr2ref.end()) { wxeRefData *refd = it->second; if(refd->alloc_in_erl) { - int type = refd->type; - if((refd->type == 1) && ((wxObject *)ptr)->IsKindOf(CLASSINFO(wxBufferedDC))) { + if((refd->type == 4) && ((wxObject *)ptr)->IsKindOf(CLASSINFO(wxBufferedDC))) { ((wxBufferedDC *)ptr)->m_dc = NULL; // Workaround } wxString msg; - if((refd->type == 0)) { // Maybe also class 1 + bool cleanup_ref=true; + if(refd->type == 0) { // Maybe also class 1 wxClassInfo *cinfo = ((wxObject *)ptr)->GetClassInfo(); msg.Printf(wxT("Memory leak: {wx_ref, %d, %s}"), refd->ref, cinfo->GetClassName()); send_msg("error", &msg); } else { - delete_object(ptr, refd); + cleanup_ref = delete_object(ptr, refd); } - if(type == 0 || type > 2) { + if(cleanup_ref) { // Delete refs for leaks and non overridden allocs delete refd; ptr2ref.erase(it); @@ -525,6 +539,17 @@ void WxeApp::destroyMemEnv(wxeMetaCommand& Ecmd) refmap.erase((ErlDrvTermData) Ecmd.port); } + +wxeRefData * WxeApp::getRefData(void *ptr) { + ptrMap::iterator it = ptr2ref.find(ptr); + if(it != ptr2ref.end()) { + wxeRefData *refd = it->second; + return refd; + } + return NULL; +} + + wxeMemEnv * WxeApp::getMemEnv(ErlDrvTermData port) { return refmap[port]; } @@ -607,7 +632,7 @@ void WxeApp::clearPtr(void * ptr) { if(((int) refd->pid) != -1) { // Send terminate pid to owner - wxeReturn rt = wxeReturn(WXE_DRV_PORT,refd->memenv->owner, false); + wxeReturn rt = wxeReturn(WXE_DRV_PORT,refd->pid, false); rt.addAtom("_wxe_destroy_"); rt.add(ERL_DRV_PID, refd->pid); rt.addTupleCount(2); diff --git a/lib/wx/c_src/wxe_impl.h b/lib/wx/c_src/wxe_impl.h index bb54961edd..57bf2e2dba 100644 --- a/lib/wx/c_src/wxe_impl.h +++ b/lib/wx/c_src/wxe_impl.h @@ -36,17 +36,28 @@ extern "C" { #include "wxe_callback_impl.h" #include "wxe_memory.h" +#if !wxCHECK_VERSION(2,9,0) +#define wxeLocaleC wxChar * +#define wxeLocaleC2String(Str) wxString(Str) +#else +typedef wxString wxeLocaleC; +#define wxeLocaleC2String(Str) Str +#endif + #define WXE_NOT_INITIATED 0 #define WXE_INITIATED 1 #define WXE_EXITED 2 #define WXE_ERROR -1 -void send_msg(const char *, wxString *); // For debugging and error msgs +void send_msg(const char *, const wxString *); // For debugging and error msgs class WxeApp : public wxApp { public: - virtual bool OnInit(); + virtual bool OnInit(); +#ifdef _MACOSX + virtual void MacOpenFile(const wxString &filename); +#endif void shutdown(wxeMetaCommand& event); int dispatch(wxList *, int, int); @@ -69,11 +80,12 @@ public: int getRef(void * ptr, wxeMemEnv *memenv); void * getPtr(char * bp, wxeMemEnv *memenv); void clearPtr(void *ptr); + wxeRefData * getRefData(void *ptr); void registerPid(char *ptr, ErlDrvTermData pid, wxeMemEnv *memenv); void init_nonconsts(wxeMemEnv *memenv, ErlDrvTermData caller); // Code found in gen/wxe_derived_dest.h - void delete_object(void *ptr, wxeRefData *refd); + bool delete_object(void *ptr, wxeRefData *refd); wxeMemMap refmap; ptrMap ptr2ref; diff --git a/lib/wx/c_src/wxe_memory.h b/lib/wx/c_src/wxe_memory.h index ec22183bfa..8a48c77154 100644 --- a/lib/wx/c_src/wxe_memory.h +++ b/lib/wx/c_src/wxe_memory.h @@ -47,7 +47,8 @@ class wxeRefData { int type; // 0 = wxWindow subclasses, 1 = wxObject subclasses // 2 = wxDialog subclasses, 3 = allocated wxObjects but not returned from new - // > 3 classes which lack virtual destr, or are supposed to be allocated on + // 4 = wxObjects that should always be deleted directly (wxDC derivates) + // > 4 classes which lack virtual destr, or are supposed to be allocated on // the stack bool alloc_in_erl; wxeMemEnv *memenv; diff --git a/lib/wx/configure.in b/lib/wx/configure.in index 3756786831..4c4d4f41a8 100755..100644 --- a/lib/wx/configure.in +++ b/lib/wx/configure.in @@ -441,12 +441,12 @@ else else CWXWIN_PROG=`cygpath -d "$PROGRAMFILES" | cygpath -f - 2>/dev/null` fi - CWXWIN3=$CWXWIN_PROG/wxWidgets-2.*.* - CWXWIN4=$CWXWIN_PROG/wxMSW-2.*.* - CWX_DOCUMENTED="/opt/local/pgm/wxMSW-2.*.* /opt/local/pgm/wxWidgets-2.*.*" + CWXWIN3=$CWXWIN_PROG/wxWidgets-?.*.* + CWXWIN4=$CWXWIN_PROG/wxMSW-?.*.* + CWX_DOCUMENTED="/opt/local/pgm/wxMSW-?.*.* /opt/local/pgm/wxWidgets-?.*.*" case $ac_cv_sizeof_void_p in 8) - CWX_DOCUMENTED="/opt/local64/pgm/wxMSW-2.*.* /opt/local64/pgm/wxWidgets-2.*.* $CWX_DOCUMENTED" + CWX_DOCUMENTED="/opt/local64/pgm/wxMSW-?.*.* /opt/local64/pgm/wxWidgets-?.*.* $CWX_DOCUMENTED" ;; *) true @@ -464,12 +464,14 @@ else WX_CFLAGS="-EHsc -D_UNICODE -DUNICODE -I$WXINCLUDE_MSVC -I$WXINCLUDE_PLAIN -I$WXINCLUDE_CONTRIB -D__WXMSW__" WX_CXXFLAGS="-TP $WX_CFLAGS" WX_LIBDIR=$dir/lib/vc_lib + WX_LIBDIR64=$dir/lib/vc_x64_lib WX_RESCOMP="rc.sh -I$WXINCLUDE_PLAIN -D __WIN32__" RC_FILE_TYPE=res - for lib in $WX_LIBDIR/wxbase*.lib $WX_LIBDIR2/wxbase*.lib; do - maybe=`echo $lib | egrep 'wxbase[[0-9]]*u\.lib'` + for lib in $WX_LIBDIR $WX_LIBDIR64; do + maybe=`ls $lib/wxbase*.lib | egrep 'wxbase[[0-9]]*u\.lib'` if test '!' -z "$maybe"; then corelib_number=`echo $maybe | sed 's,.*\([[0-9]].\)u\.lib,\1,'` + WX_LIBDIR=$lib break fi done @@ -675,6 +677,27 @@ if test "x$GCC" = xyes; then LM_TRY_ENABLE_CFLAG([-Werror=return-type], [CXXFLAGS]) fi +dnl ---------------------------------------------------------------------- +dnl Enable -fsanitize= flags. +dnl ---------------------------------------------------------------------- + +m4_define(DEFAULT_SANITIZERS, [address,undefined]) +AC_ARG_ENABLE( + sanitizers, + AS_HELP_STRING( + [--enable-sanitizers@<:@=comma-separated list of sanitizers@:>@], + [Default=DEFAULT_SANITIZERS]), +[ +case "$enableval" in + no) sanitizers= ;; + yes) sanitizers="-fsanitize=DEFAULT_SANITIZERS" ;; + *) sanitizers="-fsanitize=$enableval" ;; +esac +CFLAGS="$CFLAGS $sanitizers" +CXXFLAGS="$CXXFLAGS $sanitizers" +LDFLAGS="$LDFLAGS $sanitizers" +]) + ############################################################################# dnl diff --git a/lib/wx/doc/src/notes.xml b/lib/wx/doc/src/notes.xml index 6e653cf828..5a9c53e3b6 100644 --- a/lib/wx/doc/src/notes.xml +++ b/lib/wx/doc/src/notes.xml @@ -31,6 +31,106 @@ <p>This document describes the changes made to the wxErlang application.</p> +<section><title>Wx 1.3.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Implement --enable-sanitizers[=sanitizers]. Similar to + debugging with Valgrind, it's very useful to enable + -fsanitize= switches to catch bugs at runtime.</p> + <p> + Own Id: OTP-12153</p> + </item> + </list> + </section> + +</section> + +<section><title>Wx 1.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p>Fix delayed destroy for wxPaintDC objects which could + cause an eternal loop for modal dialogs.</p> <p>Fix + wxSL_LABELS compatibility between wxWidgets-2.8 and + wxWidgets-3.0 versions</p> + <p> + Own Id: OTP-11985</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Add missing classes wxPopup[Transient]Window, + wxActivateEvent and wxTextCtrl:cahngeValue/2 function.</p> + <p> + Own Id: OTP-11986</p> + </item> + </list> + </section> + +</section> + +<section><title>Wx 1.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Refactored C++ code, fixed crashes and a deadlock on + linux.</p> + <p> + Own Id: OTP-11586</p> + </item> + <item> + <p> + Some local implementations of removing the last element + from a list are replaced by <c>lists:droplast/1</c>. Note + that this requires at least <c>stdlib-2.0</c>, which is + the stdlib version delivered in OTP 17.0. (Thanks to Hans + Svensson)</p> + <p> + Own Id: OTP-11678</p> + </item> + <item> + <p> + Reworked the internal event handling to avoid crashes in + destroy objects. Thanks Tom for the bug report.</p> + <p> + Own Id: OTP-11699</p> + </item> + <item> + <p> + Application upgrade (appup) files are corrected for the + following applications: </p> + <p> + <c>asn1, common_test, compiler, crypto, debugger, + dialyzer, edoc, eldap, erl_docgen, et, eunit, gs, hipe, + inets, observer, odbc, os_mon, otp_mibs, parsetools, + percept, public_key, reltool, runtime_tools, ssh, + syntax_tools, test_server, tools, typer, webtool, wx, + xmerl</c></p> + <p> + A new test utility for testing appup files is added to + test_server. This is now used by most applications in + OTP.</p> + <p> + (Thanks to Tobias Schlager)</p> + <p> + Own Id: OTP-11744</p> + </item> + </list> + </section> + +</section> + <section><title>Wx 1.1.2</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/wx/include/wx.hrl b/lib/wx/include/wx.hrl index 2dc1791cce..348daf64ce 100644 --- a/lib/wx/include/wx.hrl +++ b/lib/wx/include/wx.hrl @@ -165,6 +165,11 @@ -type wxHelpEventType() :: help | detailed_help. -type wxHelp() :: #wxHelp{}. %% Callback event: {@link wxHelpEvent} +-record(wxActivate,{type :: wxActivateEventType(), %% Callback event: {@link wxActivateEvent} + active :: boolean()}). +-type wxActivateEventType() :: activate | activate_app | hibernate. +-type wxActivate() :: #wxActivate{}. %% Callback event: {@link wxActivateEvent} + -record(wxStyledText,{type :: wxStyledTextEventType(), %% Callback event: {@link wxStyledTextEvent} position :: integer(), key :: integer(), @@ -316,8 +321,8 @@ -type wxTreeEventType() :: command_tree_begin_drag | command_tree_begin_rdrag | command_tree_begin_label_edit | command_tree_end_label_edit | command_tree_delete_item | command_tree_get_info | command_tree_set_info | command_tree_item_expanded | command_tree_item_expanding | command_tree_item_collapsed | command_tree_item_collapsing | command_tree_sel_changed | command_tree_sel_changing | command_tree_key_down | command_tree_item_activated | command_tree_item_right_click | command_tree_item_middle_click | command_tree_end_drag | command_tree_state_image_click | command_tree_item_gettooltip | command_tree_item_menu. -type wxTree() :: #wxTree{}. %% Callback event: {@link wxTreeEvent} --type event() :: wxAuiManager() | wxAuiNotebook() | wxCalendar() | wxChildFocus() | wxClipboardText() | wxClose() | wxColourPicker() | wxCommand() | wxContextMenu() | wxDate() | wxDisplayChanged() | wxErase() | wxFileDirPicker() | wxFocus() | wxFontPicker() | wxGrid() | wxHelp() | wxHtmlLink() | wxIconize() | wxIdle() | wxInitDialog() | wxJoystick() | wxKey() | wxList() | wxMaximize() | wxMenu() | wxMouse() | wxMouseCaptureChanged() | wxMove() | wxNavigationKey() | wxNotebook() | wxPaint() | wxPaletteChanged() | wxQueryNewPalette() | wxSash() | wxScroll() | wxScrollWin() | wxSetCursor() | wxShow() | wxSize() | wxSpin() | wxSplitter() | wxStyledText() | wxSysColourChanged() | wxTaskBarIcon() | wxTree() | wxUpdateUI() | wxWindowCreate() | wxWindowDestroy(). --type wxEventType() :: wxAuiManagerEventType() | wxAuiNotebookEventType() | wxCalendarEventType() | wxChildFocusEventType() | wxClipboardTextEventType() | wxCloseEventType() | wxColourPickerEventType() | wxCommandEventType() | wxContextMenuEventType() | wxDateEventType() | wxDisplayChangedEventType() | wxEraseEventType() | wxFileDirPickerEventType() | wxFocusEventType() | wxFontPickerEventType() | wxGridEventType() | wxHelpEventType() | wxHtmlLinkEventType() | wxIconizeEventType() | wxIdleEventType() | wxInitDialogEventType() | wxJoystickEventType() | wxKeyEventType() | wxListEventType() | wxMaximizeEventType() | wxMenuEventType() | wxMouseCaptureChangedEventType() | wxMouseEventType() | wxMoveEventType() | wxNavigationKeyEventType() | wxNotebookEventType() | wxPaintEventType() | wxPaletteChangedEventType() | wxQueryNewPaletteEventType() | wxSashEventType() | wxScrollEventType() | wxScrollWinEventType() | wxSetCursorEventType() | wxShowEventType() | wxSizeEventType() | wxSpinEventType() | wxSplitterEventType() | wxStyledTextEventType() | wxSysColourChangedEventType() | wxTaskBarIconEventType() | wxTreeEventType() | wxUpdateUIEventType() | wxWindowCreateEventType() | wxWindowDestroyEventType(). +-type event() :: wxActivate() | wxAuiManager() | wxAuiNotebook() | wxCalendar() | wxChildFocus() | wxClipboardText() | wxClose() | wxColourPicker() | wxCommand() | wxContextMenu() | wxDate() | wxDisplayChanged() | wxErase() | wxFileDirPicker() | wxFocus() | wxFontPicker() | wxGrid() | wxHelp() | wxHtmlLink() | wxIconize() | wxIdle() | wxInitDialog() | wxJoystick() | wxKey() | wxList() | wxMaximize() | wxMenu() | wxMouse() | wxMouseCaptureChanged() | wxMove() | wxNavigationKey() | wxNotebook() | wxPaint() | wxPaletteChanged() | wxQueryNewPalette() | wxSash() | wxScroll() | wxScrollWin() | wxSetCursor() | wxShow() | wxSize() | wxSpin() | wxSplitter() | wxStyledText() | wxSysColourChanged() | wxTaskBarIcon() | wxTree() | wxUpdateUI() | wxWindowCreate() | wxWindowDestroy(). +-type wxEventType() :: wxActivateEventType() | wxAuiManagerEventType() | wxAuiNotebookEventType() | wxCalendarEventType() | wxChildFocusEventType() | wxClipboardTextEventType() | wxCloseEventType() | wxColourPickerEventType() | wxCommandEventType() | wxContextMenuEventType() | wxDateEventType() | wxDisplayChangedEventType() | wxEraseEventType() | wxFileDirPickerEventType() | wxFocusEventType() | wxFontPickerEventType() | wxGridEventType() | wxHelpEventType() | wxHtmlLinkEventType() | wxIconizeEventType() | wxIdleEventType() | wxInitDialogEventType() | wxJoystickEventType() | wxKeyEventType() | wxListEventType() | wxMaximizeEventType() | wxMenuEventType() | wxMouseCaptureChangedEventType() | wxMouseEventType() | wxMoveEventType() | wxNavigationKeyEventType() | wxNotebookEventType() | wxPaintEventType() | wxPaletteChangedEventType() | wxQueryNewPaletteEventType() | wxSashEventType() | wxScrollEventType() | wxScrollWinEventType() | wxSetCursorEventType() | wxShowEventType() | wxSizeEventType() | wxSpinEventType() | wxSplitterEventType() | wxStyledTextEventType() | wxSysColourChangedEventType() | wxTaskBarIconEventType() | wxTreeEventType() | wxUpdateUIEventType() | wxWindowCreateEventType() | wxWindowDestroyEventType(). %% Hardcoded Records -record(wxMouseState, {x :: integer(), y :: integer(), @@ -1980,10 +1985,255 @@ -define(wxIMAGELIST_DRAW_SELECTED, 4). -define(wxIMAGELIST_DRAW_TRANSPARENT, 2). -define(wxIMAGELIST_DRAW_NORMAL, 1). +% From "intl.h": wxLanguage +-define(wxLANGUAGE_DEFAULT, 0). +-define(wxLANGUAGE_UNKNOWN, 1). +-define(wxLANGUAGE_ABKHAZIAN, 2). +-define(wxLANGUAGE_AFAR, 3). +-define(wxLANGUAGE_AFRIKAANS, 4). +-define(wxLANGUAGE_ALBANIAN, 5). +-define(wxLANGUAGE_AMHARIC, 6). +-define(wxLANGUAGE_ARABIC, 7). +-define(wxLANGUAGE_ARABIC_ALGERIA, 8). +-define(wxLANGUAGE_ARABIC_BAHRAIN, 9). +-define(wxLANGUAGE_ARABIC_EGYPT, 10). +-define(wxLANGUAGE_ARABIC_IRAQ, 11). +-define(wxLANGUAGE_ARABIC_JORDAN, 12). +-define(wxLANGUAGE_ARABIC_KUWAIT, 13). +-define(wxLANGUAGE_ARABIC_LEBANON, 14). +-define(wxLANGUAGE_ARABIC_LIBYA, 15). +-define(wxLANGUAGE_ARABIC_MOROCCO, 16). +-define(wxLANGUAGE_ARABIC_OMAN, 17). +-define(wxLANGUAGE_ARABIC_QATAR, 18). +-define(wxLANGUAGE_ARABIC_SAUDI_ARABIA, 19). +-define(wxLANGUAGE_ARABIC_SUDAN, 20). +-define(wxLANGUAGE_ARABIC_SYRIA, 21). +-define(wxLANGUAGE_ARABIC_TUNISIA, 22). +-define(wxLANGUAGE_ARABIC_UAE, 23). +-define(wxLANGUAGE_ARABIC_YEMEN, 24). +-define(wxLANGUAGE_ARMENIAN, 25). +-define(wxLANGUAGE_ASSAMESE, 26). +-define(wxLANGUAGE_AYMARA, 27). +-define(wxLANGUAGE_AZERI, 28). +-define(wxLANGUAGE_AZERI_CYRILLIC, 29). +-define(wxLANGUAGE_AZERI_LATIN, 30). +-define(wxLANGUAGE_BASHKIR, 31). +-define(wxLANGUAGE_BASQUE, 32). +-define(wxLANGUAGE_BELARUSIAN, 33). +-define(wxLANGUAGE_BENGALI, 34). +-define(wxLANGUAGE_BHUTANI, 35). +-define(wxLANGUAGE_BIHARI, 36). +-define(wxLANGUAGE_BISLAMA, 37). +-define(wxLANGUAGE_BRETON, 38). +-define(wxLANGUAGE_BULGARIAN, 39). +-define(wxLANGUAGE_BURMESE, 40). +-define(wxLANGUAGE_CAMBODIAN, 41). +-define(wxLANGUAGE_CATALAN, 42). +-define(wxLANGUAGE_CHINESE, 43). +-define(wxLANGUAGE_CHINESE_SIMPLIFIED, 44). +-define(wxLANGUAGE_CHINESE_TRADITIONAL, 45). +-define(wxLANGUAGE_CHINESE_HONGKONG, 46). +-define(wxLANGUAGE_CHINESE_MACAU, 47). +-define(wxLANGUAGE_CHINESE_SINGAPORE, 48). +-define(wxLANGUAGE_CHINESE_TAIWAN, 49). +-define(wxLANGUAGE_CORSICAN, 50). +-define(wxLANGUAGE_CROATIAN, 51). +-define(wxLANGUAGE_CZECH, 52). +-define(wxLANGUAGE_DANISH, 53). +-define(wxLANGUAGE_DUTCH, 54). +-define(wxLANGUAGE_DUTCH_BELGIAN, 55). +-define(wxLANGUAGE_ENGLISH, 56). +-define(wxLANGUAGE_ENGLISH_UK, 57). +-define(wxLANGUAGE_ENGLISH_US, 58). +-define(wxLANGUAGE_ENGLISH_AUSTRALIA, 59). +-define(wxLANGUAGE_ENGLISH_BELIZE, 60). +-define(wxLANGUAGE_ENGLISH_BOTSWANA, 61). +-define(wxLANGUAGE_ENGLISH_CANADA, 62). +-define(wxLANGUAGE_ENGLISH_CARIBBEAN, 63). +-define(wxLANGUAGE_ENGLISH_DENMARK, 64). +-define(wxLANGUAGE_ENGLISH_EIRE, 65). +-define(wxLANGUAGE_ENGLISH_JAMAICA, 66). +-define(wxLANGUAGE_ENGLISH_NEW_ZEALAND, 67). +-define(wxLANGUAGE_ENGLISH_PHILIPPINES, 68). +-define(wxLANGUAGE_ENGLISH_SOUTH_AFRICA, 69). +-define(wxLANGUAGE_ENGLISH_TRINIDAD, 70). +-define(wxLANGUAGE_ENGLISH_ZIMBABWE, 71). +-define(wxLANGUAGE_ESPERANTO, 72). +-define(wxLANGUAGE_ESTONIAN, 73). +-define(wxLANGUAGE_FAEROESE, 74). +-define(wxLANGUAGE_FARSI, 75). +-define(wxLANGUAGE_FIJI, 76). +-define(wxLANGUAGE_FINNISH, 77). +-define(wxLANGUAGE_FRENCH, 78). +-define(wxLANGUAGE_FRENCH_BELGIAN, 79). +-define(wxLANGUAGE_FRENCH_CANADIAN, 80). +-define(wxLANGUAGE_FRENCH_LUXEMBOURG, 81). +-define(wxLANGUAGE_FRENCH_MONACO, 82). +-define(wxLANGUAGE_FRENCH_SWISS, 83). +-define(wxLANGUAGE_FRISIAN, 84). +-define(wxLANGUAGE_GALICIAN, 85). +-define(wxLANGUAGE_GEORGIAN, 86). +-define(wxLANGUAGE_GERMAN, 87). +-define(wxLANGUAGE_GERMAN_AUSTRIAN, 88). +-define(wxLANGUAGE_GERMAN_BELGIUM, 89). +-define(wxLANGUAGE_GERMAN_LIECHTENSTEIN, 90). +-define(wxLANGUAGE_GERMAN_LUXEMBOURG, 91). +-define(wxLANGUAGE_GERMAN_SWISS, 92). +-define(wxLANGUAGE_GREEK, 93). +-define(wxLANGUAGE_GREENLANDIC, 94). +-define(wxLANGUAGE_GUARANI, 95). +-define(wxLANGUAGE_GUJARATI, 96). +-define(wxLANGUAGE_HAUSA, 97). +-define(wxLANGUAGE_HEBREW, 98). +-define(wxLANGUAGE_HINDI, 99). +-define(wxLANGUAGE_HUNGARIAN, 100). +-define(wxLANGUAGE_ICELANDIC, 101). +-define(wxLANGUAGE_INDONESIAN, 102). +-define(wxLANGUAGE_INTERLINGUA, 103). +-define(wxLANGUAGE_INTERLINGUE, 104). +-define(wxLANGUAGE_INUKTITUT, 105). +-define(wxLANGUAGE_INUPIAK, 106). +-define(wxLANGUAGE_IRISH, 107). +-define(wxLANGUAGE_ITALIAN, 108). +-define(wxLANGUAGE_ITALIAN_SWISS, 109). +-define(wxLANGUAGE_JAPANESE, 110). +-define(wxLANGUAGE_JAVANESE, 111). +-define(wxLANGUAGE_KANNADA, 112). +-define(wxLANGUAGE_KASHMIRI, 113). +-define(wxLANGUAGE_KASHMIRI_INDIA, 114). +-define(wxLANGUAGE_KAZAKH, 115). +-define(wxLANGUAGE_KERNEWEK, 116). +-define(wxLANGUAGE_KINYARWANDA, 117). +-define(wxLANGUAGE_KIRGHIZ, 118). +-define(wxLANGUAGE_KIRUNDI, 119). +-define(wxLANGUAGE_KONKANI, 120). +-define(wxLANGUAGE_KOREAN, 121). +-define(wxLANGUAGE_KURDISH, 122). +-define(wxLANGUAGE_LAOTHIAN, 123). +-define(wxLANGUAGE_LATIN, 124). +-define(wxLANGUAGE_LATVIAN, 125). +-define(wxLANGUAGE_LINGALA, 126). +-define(wxLANGUAGE_LITHUANIAN, 127). +-define(wxLANGUAGE_MACEDONIAN, 128). +-define(wxLANGUAGE_MALAGASY, 129). +-define(wxLANGUAGE_MALAY, 130). +-define(wxLANGUAGE_MALAYALAM, 131). +-define(wxLANGUAGE_MALAY_BRUNEI_DARUSSALAM, 132). +-define(wxLANGUAGE_MALAY_MALAYSIA, 133). +-define(wxLANGUAGE_MALTESE, 134). +-define(wxLANGUAGE_MANIPURI, 135). +-define(wxLANGUAGE_MAORI, 136). +-define(wxLANGUAGE_MARATHI, 137). +-define(wxLANGUAGE_MOLDAVIAN, 138). +-define(wxLANGUAGE_MONGOLIAN, 139). +-define(wxLANGUAGE_NAURU, 140). +-define(wxLANGUAGE_NEPALI, 141). +-define(wxLANGUAGE_NEPALI_INDIA, 142). +-define(wxLANGUAGE_NORWEGIAN_BOKMAL, 143). +-define(wxLANGUAGE_NORWEGIAN_NYNORSK, 144). +-define(wxLANGUAGE_OCCITAN, 145). +-define(wxLANGUAGE_ORIYA, 146). +-define(wxLANGUAGE_OROMO, 147). +-define(wxLANGUAGE_PASHTO, 148). +-define(wxLANGUAGE_POLISH, 149). +-define(wxLANGUAGE_PORTUGUESE, 150). +-define(wxLANGUAGE_PORTUGUESE_BRAZILIAN, 151). +-define(wxLANGUAGE_PUNJABI, 152). +-define(wxLANGUAGE_QUECHUA, 153). +-define(wxLANGUAGE_RHAETO_ROMANCE, 154). +-define(wxLANGUAGE_ROMANIAN, 155). +-define(wxLANGUAGE_RUSSIAN, 156). +-define(wxLANGUAGE_RUSSIAN_UKRAINE, 157). +-define(wxLANGUAGE_SAMOAN, 158). +-define(wxLANGUAGE_SANGHO, 159). +-define(wxLANGUAGE_SANSKRIT, 160). +-define(wxLANGUAGE_SCOTS_GAELIC, 161). +-define(wxLANGUAGE_SERBIAN, 162). +-define(wxLANGUAGE_SERBIAN_CYRILLIC, 163). +-define(wxLANGUAGE_SERBIAN_LATIN, 164). +-define(wxLANGUAGE_SERBO_CROATIAN, 165). +-define(wxLANGUAGE_SESOTHO, 166). +-define(wxLANGUAGE_SETSWANA, 167). +-define(wxLANGUAGE_SHONA, 168). +-define(wxLANGUAGE_SINDHI, 169). +-define(wxLANGUAGE_SINHALESE, 170). +-define(wxLANGUAGE_SISWATI, 171). +-define(wxLANGUAGE_SLOVAK, 172). +-define(wxLANGUAGE_SLOVENIAN, 173). +-define(wxLANGUAGE_SOMALI, 174). +-define(wxLANGUAGE_SPANISH, 175). +-define(wxLANGUAGE_SPANISH_ARGENTINA, 176). +-define(wxLANGUAGE_SPANISH_BOLIVIA, 177). +-define(wxLANGUAGE_SPANISH_CHILE, 178). +-define(wxLANGUAGE_SPANISH_COLOMBIA, 179). +-define(wxLANGUAGE_SPANISH_COSTA_RICA, 180). +-define(wxLANGUAGE_SPANISH_DOMINICAN_REPUBLIC, 181). +-define(wxLANGUAGE_SPANISH_ECUADOR, 182). +-define(wxLANGUAGE_SPANISH_EL_SALVADOR, 183). +-define(wxLANGUAGE_SPANISH_GUATEMALA, 184). +-define(wxLANGUAGE_SPANISH_HONDURAS, 185). +-define(wxLANGUAGE_SPANISH_MEXICAN, 186). +-define(wxLANGUAGE_SPANISH_MODERN, 187). +-define(wxLANGUAGE_SPANISH_NICARAGUA, 188). +-define(wxLANGUAGE_SPANISH_PANAMA, 189). +-define(wxLANGUAGE_SPANISH_PARAGUAY, 190). +-define(wxLANGUAGE_SPANISH_PERU, 191). +-define(wxLANGUAGE_SPANISH_PUERTO_RICO, 192). +-define(wxLANGUAGE_SPANISH_URUGUAY, 193). +-define(wxLANGUAGE_SPANISH_US, 194). +-define(wxLANGUAGE_SPANISH_VENEZUELA, 195). +-define(wxLANGUAGE_SUNDANESE, 196). +-define(wxLANGUAGE_SWAHILI, 197). +-define(wxLANGUAGE_SWEDISH, 198). +-define(wxLANGUAGE_SWEDISH_FINLAND, 199). +-define(wxLANGUAGE_TAGALOG, 200). +-define(wxLANGUAGE_TAJIK, 201). +-define(wxLANGUAGE_TAMIL, 202). +-define(wxLANGUAGE_TATAR, 203). +-define(wxLANGUAGE_TELUGU, 204). +-define(wxLANGUAGE_THAI, 205). +-define(wxLANGUAGE_TIBETAN, 206). +-define(wxLANGUAGE_TIGRINYA, 207). +-define(wxLANGUAGE_TONGA, 208). +-define(wxLANGUAGE_TSONGA, 209). +-define(wxLANGUAGE_TURKISH, 210). +-define(wxLANGUAGE_TURKMEN, 211). +-define(wxLANGUAGE_TWI, 212). +-define(wxLANGUAGE_UIGHUR, 213). +-define(wxLANGUAGE_UKRAINIAN, 214). +-define(wxLANGUAGE_URDU, 215). +-define(wxLANGUAGE_URDU_INDIA, 216). +-define(wxLANGUAGE_URDU_PAKISTAN, 217). +-define(wxLANGUAGE_UZBEK, 218). +-define(wxLANGUAGE_UZBEK_CYRILLIC, 219). +-define(wxLANGUAGE_UZBEK_LATIN, 220). +-define(wxLANGUAGE_VIETNAMESE, 221). +-define(wxLANGUAGE_VOLAPUK, 222). +-define(wxLANGUAGE_WELSH, 223). +-define(wxLANGUAGE_WOLOF, 224). +-define(wxLANGUAGE_XHOSA, 225). +-define(wxLANGUAGE_YIDDISH, 226). +-define(wxLANGUAGE_YORUBA, 227). +-define(wxLANGUAGE_ZHUANG, 228). +-define(wxLANGUAGE_ZULU, 229). +-define(wxLANGUAGE_USER_DEFINED, 230). +-define(wxLANGUAGE_VALENCIAN, 536870911). +-define(wxLANGUAGE_SAMI, 536870912). % From "intl.h": wxLayoutDirection -define(wxLayout_Default, 0). -define(wxLayout_LeftToRight, 1). -define(wxLayout_RightToLeft, 2). +% From "intl.h": wxLocaleCategory +-define(wxLOCALE_CAT_NUMBER, 0). +-define(wxLOCALE_CAT_DATE, 1). +-define(wxLOCALE_CAT_MONEY, 2). +-define(wxLOCALE_CAT_MAX, 3). +% From "intl.h": wxLocaleInfo +-define(wxLOCALE_THOUSANDS_SEP, 0). +-define(wxLOCALE_DECIMAL_POINT, 1). +% From "intl.h": wxLocaleInitFlags +-define(wxLOCALE_LOAD_DEFAULT, 1). +-define(wxLOCALE_CONV_ENCODING, 2). % From "layout.h" -define(wxLAYOUT_DEFAULT_MARGIN, 0). % From "layout.h": wxEdge @@ -2270,7 +2520,7 @@ -define(wxSL_RIGHT, 256). -define(wxSL_TOP, 128). -define(wxSL_LEFT, 64). --define(wxSL_LABELS, 32). +-define(wxSL_LABELS, wxe_util:get_const(wxSL_LABELS)). -define(wxSL_AUTOTICKS, ?wxSL_TICKS). -define(wxSL_TICKS, 16). -define(wxSL_VERTICAL, ?wxVERTICAL). diff --git a/lib/wx/src/gen/wxActivateEvent.erl b/lib/wx/src/gen/wxActivateEvent.erl new file mode 100644 index 0000000000..dc03866027 --- /dev/null +++ b/lib/wx/src/gen/wxActivateEvent.erl @@ -0,0 +1,72 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2008-2014. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% This file is generated DO NOT EDIT + +%% @doc See external documentation: <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxactivateevent.html">wxActivateEvent</a>. +%% <dl><dt>Use {@link wxEvtHandler:connect/3.} with EventType:</dt> +%% <dd><em>activate</em>, <em>activate_app</em>, <em>hibernate</em></dd></dl> +%% See also the message variant {@link wxEvtHandler:wxActivate(). #wxActivate{}} event record type. +%% +%% <p>This class is derived (and can use functions) from: +%% <br />{@link wxEvent} +%% </p> +%% @type wxActivateEvent(). An object reference, The representation is internal +%% and can be changed without notice. It can't be used for comparsion +%% stored on disc or distributed for use on other nodes. + +-module(wxActivateEvent). +-include("wxe.hrl"). +-export([getActive/1]). + +%% inherited exports +-export([getId/1,getSkipped/1,getTimestamp/1,isCommandEvent/1,parent_class/1, + resumePropagation/2,shouldPropagate/1,skip/1,skip/2,stopPropagation/1]). + +-export_type([wxActivateEvent/0]). +%% @hidden +parent_class(wxEvent) -> true; +parent_class(_Class) -> erlang:error({badtype, ?MODULE}). + +-type wxActivateEvent() :: wx:wx_object(). +%% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxactivateevent.html#wxactivateeventgetactive">external documentation</a>. +-spec getActive(This) -> boolean() when + This::wxActivateEvent(). +getActive(#wx_ref{type=ThisT,ref=ThisRef}) -> + ?CLASS(ThisT,wxActivateEvent), + wxe_util:call(?wxActivateEvent_GetActive, + <<ThisRef:32/?UI>>). + + %% From wxEvent +%% @hidden +stopPropagation(This) -> wxEvent:stopPropagation(This). +%% @hidden +skip(This, Options) -> wxEvent:skip(This, Options). +%% @hidden +skip(This) -> wxEvent:skip(This). +%% @hidden +shouldPropagate(This) -> wxEvent:shouldPropagate(This). +%% @hidden +resumePropagation(This,PropagationLevel) -> wxEvent:resumePropagation(This,PropagationLevel). +%% @hidden +isCommandEvent(This) -> wxEvent:isCommandEvent(This). +%% @hidden +getTimestamp(This) -> wxEvent:getTimestamp(This). +%% @hidden +getSkipped(This) -> wxEvent:getSkipped(This). +%% @hidden +getId(This) -> wxEvent:getId(This). diff --git a/lib/wx/src/gen/wxLocale.erl b/lib/wx/src/gen/wxLocale.erl new file mode 100644 index 0000000000..a1a418bcee --- /dev/null +++ b/lib/wx/src/gen/wxLocale.erl @@ -0,0 +1,278 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2008-2014. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% This file is generated DO NOT EDIT + +%% @doc See external documentation: <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxlocale.html">wxLocale</a>. +%% @type wxLocale(). An object reference, The representation is internal +%% and can be changed without notice. It can't be used for comparsion +%% stored on disc or distributed for use on other nodes. + +-module(wxLocale). +-include("wxe.hrl"). +-export([addCatalog/2,addCatalog/4,addCatalogLookupPathPrefix/1,destroy/1, + getCanonicalName/1,getHeaderValue/2,getHeaderValue/3,getLanguage/1, + getLanguageName/1,getLocale/1,getName/1,getString/2,getString/3,getString/4, + getString/5,getSysName/1,getSystemEncoding/0,getSystemEncodingName/0, + getSystemLanguage/0,init/1,init/2,isLoaded/2,isOk/1,new/0,new/1,new/2]). + +%% inherited exports +-export([parent_class/1]). + +-export_type([wxLocale/0]). +%% @hidden +parent_class(_Class) -> erlang:error({badtype, ?MODULE}). + +-type wxLocale() :: wx:wx_object(). +%% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxlocale.html#wxlocalewxlocale">external documentation</a>. +-spec new() -> wxLocale(). +new() -> + wxe_util:construct(?wxLocale_new_0, + <<>>). + +%% @equiv new(Language, []) +-spec new(Language) -> wxLocale() when + Language::integer(). + +new(Language) + when is_integer(Language) -> + new(Language, []). + +%% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxlocale.html#wxlocalewxlocale">external documentation</a>. +-spec new(Language, [Option]) -> wxLocale() when + Language::integer(), + Option :: {flags, integer()}. +new(Language, Options) + when is_integer(Language),is_list(Options) -> + MOpts = fun({flags, Flags}, Acc) -> [<<1:32/?UI,Flags:32/?UI>>|Acc]; + (BadOpt, _) -> erlang:error({badoption, BadOpt}) end, + BinOpt = list_to_binary(lists:foldl(MOpts, [<<0:32>>], Options)), + wxe_util:construct(?wxLocale_new_2, + <<Language:32/?UI, 0:32,BinOpt/binary>>). + +%% @equiv init(This, []) +-spec init(This) -> boolean() when + This::wxLocale(). + +init(This) + when is_record(This, wx_ref) -> + init(This, []). + +%% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxlocale.html#wxlocaleinit">external documentation</a>. +-spec init(This, [Option]) -> boolean() when + This::wxLocale(), + Option :: {language, integer()} + | {flags, integer()}. +init(#wx_ref{type=ThisT,ref=ThisRef}, Options) + when is_list(Options) -> + ?CLASS(ThisT,wxLocale), + MOpts = fun({language, Language}, Acc) -> [<<1:32/?UI,Language:32/?UI>>|Acc]; + ({flags, Flags}, Acc) -> [<<2:32/?UI,Flags:32/?UI>>|Acc]; + (BadOpt, _) -> erlang:error({badoption, BadOpt}) end, + BinOpt = list_to_binary(lists:foldl(MOpts, [<<0:32>>], Options)), + wxe_util:call(?wxLocale_Init, + <<ThisRef:32/?UI, 0:32,BinOpt/binary>>). + +%% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxlocale.html#wxlocaleaddcatalog">external documentation</a>. +-spec addCatalog(This, SzDomain) -> boolean() when + This::wxLocale(), SzDomain::unicode:chardata(). +addCatalog(#wx_ref{type=ThisT,ref=ThisRef},SzDomain) + when is_list(SzDomain) -> + ?CLASS(ThisT,wxLocale), + SzDomain_UC = unicode:characters_to_binary([SzDomain,0]), + wxe_util:call(?wxLocale_AddCatalog_1, + <<ThisRef:32/?UI,(byte_size(SzDomain_UC)):32/?UI,(SzDomain_UC)/binary, 0:(((8- ((0+byte_size(SzDomain_UC)) band 16#7)) band 16#7))/unit:8>>). + +%% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxlocale.html#wxlocaleaddcatalog">external documentation</a>. +%%<br /> MsgIdLanguage = ?wxLANGUAGE_DEFAULT | ?wxLANGUAGE_UNKNOWN | ?wxLANGUAGE_ABKHAZIAN | ?wxLANGUAGE_AFAR | ?wxLANGUAGE_AFRIKAANS | ?wxLANGUAGE_ALBANIAN | ?wxLANGUAGE_AMHARIC | ?wxLANGUAGE_ARABIC | ?wxLANGUAGE_ARABIC_ALGERIA | ?wxLANGUAGE_ARABIC_BAHRAIN | ?wxLANGUAGE_ARABIC_EGYPT | ?wxLANGUAGE_ARABIC_IRAQ | ?wxLANGUAGE_ARABIC_JORDAN | ?wxLANGUAGE_ARABIC_KUWAIT | ?wxLANGUAGE_ARABIC_LEBANON | ?wxLANGUAGE_ARABIC_LIBYA | ?wxLANGUAGE_ARABIC_MOROCCO | ?wxLANGUAGE_ARABIC_OMAN | ?wxLANGUAGE_ARABIC_QATAR | ?wxLANGUAGE_ARABIC_SAUDI_ARABIA | ?wxLANGUAGE_ARABIC_SUDAN | ?wxLANGUAGE_ARABIC_SYRIA | ?wxLANGUAGE_ARABIC_TUNISIA | ?wxLANGUAGE_ARABIC_UAE | ?wxLANGUAGE_ARABIC_YEMEN | ?wxLANGUAGE_ARMENIAN | ?wxLANGUAGE_ASSAMESE | ?wxLANGUAGE_AYMARA | ?wxLANGUAGE_AZERI | ?wxLANGUAGE_AZERI_CYRILLIC | ?wxLANGUAGE_AZERI_LATIN | ?wxLANGUAGE_BASHKIR | ?wxLANGUAGE_BASQUE | ?wxLANGUAGE_BELARUSIAN | ?wxLANGUAGE_BENGALI | ?wxLANGUAGE_BHUTANI | ?wxLANGUAGE_BIHARI | ?wxLANGUAGE_BISLAMA | ?wxLANGUAGE_BRETON | ?wxLANGUAGE_BULGARIAN | ?wxLANGUAGE_BURMESE | ?wxLANGUAGE_CAMBODIAN | ?wxLANGUAGE_CATALAN | ?wxLANGUAGE_CHINESE | ?wxLANGUAGE_CHINESE_SIMPLIFIED | ?wxLANGUAGE_CHINESE_TRADITIONAL | ?wxLANGUAGE_CHINESE_HONGKONG | ?wxLANGUAGE_CHINESE_MACAU | ?wxLANGUAGE_CHINESE_SINGAPORE | ?wxLANGUAGE_CHINESE_TAIWAN | ?wxLANGUAGE_CORSICAN | ?wxLANGUAGE_CROATIAN | ?wxLANGUAGE_CZECH | ?wxLANGUAGE_DANISH | ?wxLANGUAGE_DUTCH | ?wxLANGUAGE_DUTCH_BELGIAN | ?wxLANGUAGE_ENGLISH | ?wxLANGUAGE_ENGLISH_UK | ?wxLANGUAGE_ENGLISH_US | ?wxLANGUAGE_ENGLISH_AUSTRALIA | ?wxLANGUAGE_ENGLISH_BELIZE | ?wxLANGUAGE_ENGLISH_BOTSWANA | ?wxLANGUAGE_ENGLISH_CANADA | ?wxLANGUAGE_ENGLISH_CARIBBEAN | ?wxLANGUAGE_ENGLISH_DENMARK | ?wxLANGUAGE_ENGLISH_EIRE | ?wxLANGUAGE_ENGLISH_JAMAICA | ?wxLANGUAGE_ENGLISH_NEW_ZEALAND | ?wxLANGUAGE_ENGLISH_PHILIPPINES | ?wxLANGUAGE_ENGLISH_SOUTH_AFRICA | ?wxLANGUAGE_ENGLISH_TRINIDAD | ?wxLANGUAGE_ENGLISH_ZIMBABWE | ?wxLANGUAGE_ESPERANTO | ?wxLANGUAGE_ESTONIAN | ?wxLANGUAGE_FAEROESE | ?wxLANGUAGE_FARSI | ?wxLANGUAGE_FIJI | ?wxLANGUAGE_FINNISH | ?wxLANGUAGE_FRENCH | ?wxLANGUAGE_FRENCH_BELGIAN | ?wxLANGUAGE_FRENCH_CANADIAN | ?wxLANGUAGE_FRENCH_LUXEMBOURG | ?wxLANGUAGE_FRENCH_MONACO | ?wxLANGUAGE_FRENCH_SWISS | ?wxLANGUAGE_FRISIAN | ?wxLANGUAGE_GALICIAN | ?wxLANGUAGE_GEORGIAN | ?wxLANGUAGE_GERMAN | ?wxLANGUAGE_GERMAN_AUSTRIAN | ?wxLANGUAGE_GERMAN_BELGIUM | ?wxLANGUAGE_GERMAN_LIECHTENSTEIN | ?wxLANGUAGE_GERMAN_LUXEMBOURG | ?wxLANGUAGE_GERMAN_SWISS | ?wxLANGUAGE_GREEK | ?wxLANGUAGE_GREENLANDIC | ?wxLANGUAGE_GUARANI | ?wxLANGUAGE_GUJARATI | ?wxLANGUAGE_HAUSA | ?wxLANGUAGE_HEBREW | ?wxLANGUAGE_HINDI | ?wxLANGUAGE_HUNGARIAN | ?wxLANGUAGE_ICELANDIC | ?wxLANGUAGE_INDONESIAN | ?wxLANGUAGE_INTERLINGUA | ?wxLANGUAGE_INTERLINGUE | ?wxLANGUAGE_INUKTITUT | ?wxLANGUAGE_INUPIAK | ?wxLANGUAGE_IRISH | ?wxLANGUAGE_ITALIAN | ?wxLANGUAGE_ITALIAN_SWISS | ?wxLANGUAGE_JAPANESE | ?wxLANGUAGE_JAVANESE | ?wxLANGUAGE_KANNADA | ?wxLANGUAGE_KASHMIRI | ?wxLANGUAGE_KASHMIRI_INDIA | ?wxLANGUAGE_KAZAKH | ?wxLANGUAGE_KERNEWEK | ?wxLANGUAGE_KINYARWANDA | ?wxLANGUAGE_KIRGHIZ | ?wxLANGUAGE_KIRUNDI | ?wxLANGUAGE_KONKANI | ?wxLANGUAGE_KOREAN | ?wxLANGUAGE_KURDISH | ?wxLANGUAGE_LAOTHIAN | ?wxLANGUAGE_LATIN | ?wxLANGUAGE_LATVIAN | ?wxLANGUAGE_LINGALA | ?wxLANGUAGE_LITHUANIAN | ?wxLANGUAGE_MACEDONIAN | ?wxLANGUAGE_MALAGASY | ?wxLANGUAGE_MALAY | ?wxLANGUAGE_MALAYALAM | ?wxLANGUAGE_MALAY_BRUNEI_DARUSSALAM | ?wxLANGUAGE_MALAY_MALAYSIA | ?wxLANGUAGE_MALTESE | ?wxLANGUAGE_MANIPURI | ?wxLANGUAGE_MAORI | ?wxLANGUAGE_MARATHI | ?wxLANGUAGE_MOLDAVIAN | ?wxLANGUAGE_MONGOLIAN | ?wxLANGUAGE_NAURU | ?wxLANGUAGE_NEPALI | ?wxLANGUAGE_NEPALI_INDIA | ?wxLANGUAGE_NORWEGIAN_BOKMAL | ?wxLANGUAGE_NORWEGIAN_NYNORSK | ?wxLANGUAGE_OCCITAN | ?wxLANGUAGE_ORIYA | ?wxLANGUAGE_OROMO | ?wxLANGUAGE_PASHTO | ?wxLANGUAGE_POLISH | ?wxLANGUAGE_PORTUGUESE | ?wxLANGUAGE_PORTUGUESE_BRAZILIAN | ?wxLANGUAGE_PUNJABI | ?wxLANGUAGE_QUECHUA | ?wxLANGUAGE_RHAETO_ROMANCE | ?wxLANGUAGE_ROMANIAN | ?wxLANGUAGE_RUSSIAN | ?wxLANGUAGE_RUSSIAN_UKRAINE | ?wxLANGUAGE_SAMOAN | ?wxLANGUAGE_SANGHO | ?wxLANGUAGE_SANSKRIT | ?wxLANGUAGE_SCOTS_GAELIC | ?wxLANGUAGE_SERBIAN | ?wxLANGUAGE_SERBIAN_CYRILLIC | ?wxLANGUAGE_SERBIAN_LATIN | ?wxLANGUAGE_SERBO_CROATIAN | ?wxLANGUAGE_SESOTHO | ?wxLANGUAGE_SETSWANA | ?wxLANGUAGE_SHONA | ?wxLANGUAGE_SINDHI | ?wxLANGUAGE_SINHALESE | ?wxLANGUAGE_SISWATI | ?wxLANGUAGE_SLOVAK | ?wxLANGUAGE_SLOVENIAN | ?wxLANGUAGE_SOMALI | ?wxLANGUAGE_SPANISH | ?wxLANGUAGE_SPANISH_ARGENTINA | ?wxLANGUAGE_SPANISH_BOLIVIA | ?wxLANGUAGE_SPANISH_CHILE | ?wxLANGUAGE_SPANISH_COLOMBIA | ?wxLANGUAGE_SPANISH_COSTA_RICA | ?wxLANGUAGE_SPANISH_DOMINICAN_REPUBLIC | ?wxLANGUAGE_SPANISH_ECUADOR | ?wxLANGUAGE_SPANISH_EL_SALVADOR | ?wxLANGUAGE_SPANISH_GUATEMALA | ?wxLANGUAGE_SPANISH_HONDURAS | ?wxLANGUAGE_SPANISH_MEXICAN | ?wxLANGUAGE_SPANISH_MODERN | ?wxLANGUAGE_SPANISH_NICARAGUA | ?wxLANGUAGE_SPANISH_PANAMA | ?wxLANGUAGE_SPANISH_PARAGUAY | ?wxLANGUAGE_SPANISH_PERU | ?wxLANGUAGE_SPANISH_PUERTO_RICO | ?wxLANGUAGE_SPANISH_URUGUAY | ?wxLANGUAGE_SPANISH_US | ?wxLANGUAGE_SPANISH_VENEZUELA | ?wxLANGUAGE_SUNDANESE | ?wxLANGUAGE_SWAHILI | ?wxLANGUAGE_SWEDISH | ?wxLANGUAGE_SWEDISH_FINLAND | ?wxLANGUAGE_TAGALOG | ?wxLANGUAGE_TAJIK | ?wxLANGUAGE_TAMIL | ?wxLANGUAGE_TATAR | ?wxLANGUAGE_TELUGU | ?wxLANGUAGE_THAI | ?wxLANGUAGE_TIBETAN | ?wxLANGUAGE_TIGRINYA | ?wxLANGUAGE_TONGA | ?wxLANGUAGE_TSONGA | ?wxLANGUAGE_TURKISH | ?wxLANGUAGE_TURKMEN | ?wxLANGUAGE_TWI | ?wxLANGUAGE_UIGHUR | ?wxLANGUAGE_UKRAINIAN | ?wxLANGUAGE_URDU | ?wxLANGUAGE_URDU_INDIA | ?wxLANGUAGE_URDU_PAKISTAN | ?wxLANGUAGE_UZBEK | ?wxLANGUAGE_UZBEK_CYRILLIC | ?wxLANGUAGE_UZBEK_LATIN | ?wxLANGUAGE_VIETNAMESE | ?wxLANGUAGE_VOLAPUK | ?wxLANGUAGE_WELSH | ?wxLANGUAGE_WOLOF | ?wxLANGUAGE_XHOSA | ?wxLANGUAGE_YIDDISH | ?wxLANGUAGE_YORUBA | ?wxLANGUAGE_ZHUANG | ?wxLANGUAGE_ZULU | ?wxLANGUAGE_USER_DEFINED | ?wxLANGUAGE_VALENCIAN | ?wxLANGUAGE_SAMI +-spec addCatalog(This, SzDomain, MsgIdLanguage, MsgIdCharset) -> boolean() when + This::wxLocale(), SzDomain::unicode:chardata(), MsgIdLanguage::wx:wx_enum(), MsgIdCharset::unicode:chardata(). +addCatalog(#wx_ref{type=ThisT,ref=ThisRef},SzDomain,MsgIdLanguage,MsgIdCharset) + when is_list(SzDomain),is_integer(MsgIdLanguage),is_list(MsgIdCharset) -> + ?CLASS(ThisT,wxLocale), + SzDomain_UC = unicode:characters_to_binary([SzDomain,0]), + MsgIdCharset_UC = unicode:characters_to_binary([MsgIdCharset,0]), + wxe_util:call(?wxLocale_AddCatalog_3, + <<ThisRef:32/?UI,(byte_size(SzDomain_UC)):32/?UI,(SzDomain_UC)/binary, 0:(((8- ((0+byte_size(SzDomain_UC)) band 16#7)) band 16#7))/unit:8,MsgIdLanguage:32/?UI,(byte_size(MsgIdCharset_UC)):32/?UI,(MsgIdCharset_UC)/binary, 0:(((8- ((0+byte_size(MsgIdCharset_UC)) band 16#7)) band 16#7))/unit:8>>). + +%% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxlocale.html#wxlocaleaddcataloglookuppathprefix">external documentation</a>. +-spec addCatalogLookupPathPrefix(Prefix) -> ok when + Prefix::unicode:chardata(). +addCatalogLookupPathPrefix(Prefix) + when is_list(Prefix) -> + Prefix_UC = unicode:characters_to_binary([Prefix,0]), + wxe_util:cast(?wxLocale_AddCatalogLookupPathPrefix, + <<(byte_size(Prefix_UC)):32/?UI,(Prefix_UC)/binary, 0:(((8- ((4+byte_size(Prefix_UC)) band 16#7)) band 16#7))/unit:8>>). + +%% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxlocale.html#wxlocalegetcanonicalname">external documentation</a>. +-spec getCanonicalName(This) -> unicode:charlist() when + This::wxLocale(). +getCanonicalName(#wx_ref{type=ThisT,ref=ThisRef}) -> + ?CLASS(ThisT,wxLocale), + wxe_util:call(?wxLocale_GetCanonicalName, + <<ThisRef:32/?UI>>). + +%% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxlocale.html#wxlocalegetlanguage">external documentation</a>. +-spec getLanguage(This) -> integer() when + This::wxLocale(). +getLanguage(#wx_ref{type=ThisT,ref=ThisRef}) -> + ?CLASS(ThisT,wxLocale), + wxe_util:call(?wxLocale_GetLanguage, + <<ThisRef:32/?UI>>). + +%% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxlocale.html#wxlocalegetlanguagename">external documentation</a>. +-spec getLanguageName(Lang) -> unicode:charlist() when + Lang::integer(). +getLanguageName(Lang) + when is_integer(Lang) -> + wxe_util:call(?wxLocale_GetLanguageName, + <<Lang:32/?UI>>). + +%% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxlocale.html#wxlocalegetlocale">external documentation</a>. +-spec getLocale(This) -> unicode:charlist() when + This::wxLocale(). +getLocale(#wx_ref{type=ThisT,ref=ThisRef}) -> + ?CLASS(ThisT,wxLocale), + wxe_util:call(?wxLocale_GetLocale, + <<ThisRef:32/?UI>>). + +%% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxlocale.html#wxlocalegetname">external documentation</a>. +-spec getName(This) -> unicode:charlist() when + This::wxLocale(). +getName(#wx_ref{type=ThisT,ref=ThisRef}) -> + ?CLASS(ThisT,wxLocale), + wxe_util:call(?wxLocale_GetName, + <<ThisRef:32/?UI>>). + +%% @equiv getString(This,SzOrigString, []) +-spec getString(This, SzOrigString) -> unicode:charlist() when + This::wxLocale(), SzOrigString::unicode:chardata(). + +getString(This,SzOrigString) + when is_record(This, wx_ref),is_list(SzOrigString) -> + getString(This,SzOrigString, []). + +%% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxlocale.html#wxlocalegetstring">external documentation</a>. +-spec getString(This, SzOrigString, [Option]) -> unicode:charlist() when + This::wxLocale(), SzOrigString::unicode:chardata(), + Option :: {szDomain, unicode:chardata()}. +getString(#wx_ref{type=ThisT,ref=ThisRef},SzOrigString, Options) + when is_list(SzOrigString),is_list(Options) -> + ?CLASS(ThisT,wxLocale), + SzOrigString_UC = unicode:characters_to_binary([SzOrigString,0]), + MOpts = fun({szDomain, SzDomain}, Acc) -> SzDomain_UC = unicode:characters_to_binary([SzDomain,0]),[<<1:32/?UI,(byte_size(SzDomain_UC)):32/?UI,(SzDomain_UC)/binary, 0:(((8- ((0+byte_size(SzDomain_UC)) band 16#7)) band 16#7))/unit:8>>|Acc]; + (BadOpt, _) -> erlang:error({badoption, BadOpt}) end, + BinOpt = list_to_binary(lists:foldl(MOpts, [<<0:32>>], Options)), + wxe_util:call(?wxLocale_GetString_2, + <<ThisRef:32/?UI,(byte_size(SzOrigString_UC)):32/?UI,(SzOrigString_UC)/binary, 0:(((8- ((0+byte_size(SzOrigString_UC)) band 16#7)) band 16#7))/unit:8, BinOpt/binary>>). + +%% @equiv getString(This,SzOrigString,SzOrigString2,N, []) +-spec getString(This, SzOrigString, SzOrigString2, N) -> unicode:charlist() when + This::wxLocale(), SzOrigString::unicode:chardata(), SzOrigString2::unicode:chardata(), N::integer(). + +getString(This,SzOrigString,SzOrigString2,N) + when is_record(This, wx_ref),is_list(SzOrigString),is_list(SzOrigString2),is_integer(N) -> + getString(This,SzOrigString,SzOrigString2,N, []). + +%% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxlocale.html#wxlocalegetstring">external documentation</a>. +-spec getString(This, SzOrigString, SzOrigString2, N, [Option]) -> unicode:charlist() when + This::wxLocale(), SzOrigString::unicode:chardata(), SzOrigString2::unicode:chardata(), N::integer(), + Option :: {szDomain, unicode:chardata()}. +getString(#wx_ref{type=ThisT,ref=ThisRef},SzOrigString,SzOrigString2,N, Options) + when is_list(SzOrigString),is_list(SzOrigString2),is_integer(N),is_list(Options) -> + ?CLASS(ThisT,wxLocale), + SzOrigString_UC = unicode:characters_to_binary([SzOrigString,0]), + SzOrigString2_UC = unicode:characters_to_binary([SzOrigString2,0]), + MOpts = fun({szDomain, SzDomain}, Acc) -> SzDomain_UC = unicode:characters_to_binary([SzDomain,0]),[<<1:32/?UI,(byte_size(SzDomain_UC)):32/?UI,(SzDomain_UC)/binary, 0:(((8- ((0+byte_size(SzDomain_UC)) band 16#7)) band 16#7))/unit:8>>|Acc]; + (BadOpt, _) -> erlang:error({badoption, BadOpt}) end, + BinOpt = list_to_binary(lists:foldl(MOpts, [<<0:32>>], Options)), + wxe_util:call(?wxLocale_GetString_4, + <<ThisRef:32/?UI,(byte_size(SzOrigString_UC)):32/?UI,(SzOrigString_UC)/binary, 0:(((8- ((0+byte_size(SzOrigString_UC)) band 16#7)) band 16#7))/unit:8,(byte_size(SzOrigString2_UC)):32/?UI,(SzOrigString2_UC)/binary, 0:(((8- ((4+byte_size(SzOrigString2_UC)) band 16#7)) band 16#7))/unit:8,N:32/?UI, 0:32,BinOpt/binary>>). + +%% @equiv getHeaderValue(This,SzHeader, []) +-spec getHeaderValue(This, SzHeader) -> unicode:charlist() when + This::wxLocale(), SzHeader::unicode:chardata(). + +getHeaderValue(This,SzHeader) + when is_record(This, wx_ref),is_list(SzHeader) -> + getHeaderValue(This,SzHeader, []). + +%% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxlocale.html#wxlocalegetheadervalue">external documentation</a>. +-spec getHeaderValue(This, SzHeader, [Option]) -> unicode:charlist() when + This::wxLocale(), SzHeader::unicode:chardata(), + Option :: {szDomain, unicode:chardata()}. +getHeaderValue(#wx_ref{type=ThisT,ref=ThisRef},SzHeader, Options) + when is_list(SzHeader),is_list(Options) -> + ?CLASS(ThisT,wxLocale), + SzHeader_UC = unicode:characters_to_binary([SzHeader,0]), + MOpts = fun({szDomain, SzDomain}, Acc) -> SzDomain_UC = unicode:characters_to_binary([SzDomain,0]),[<<1:32/?UI,(byte_size(SzDomain_UC)):32/?UI,(SzDomain_UC)/binary, 0:(((8- ((0+byte_size(SzDomain_UC)) band 16#7)) band 16#7))/unit:8>>|Acc]; + (BadOpt, _) -> erlang:error({badoption, BadOpt}) end, + BinOpt = list_to_binary(lists:foldl(MOpts, [<<0:32>>], Options)), + wxe_util:call(?wxLocale_GetHeaderValue, + <<ThisRef:32/?UI,(byte_size(SzHeader_UC)):32/?UI,(SzHeader_UC)/binary, 0:(((8- ((0+byte_size(SzHeader_UC)) band 16#7)) band 16#7))/unit:8, BinOpt/binary>>). + +%% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxlocale.html#wxlocalegetsysname">external documentation</a>. +-spec getSysName(This) -> unicode:charlist() when + This::wxLocale(). +getSysName(#wx_ref{type=ThisT,ref=ThisRef}) -> + ?CLASS(ThisT,wxLocale), + wxe_util:call(?wxLocale_GetSysName, + <<ThisRef:32/?UI>>). + +%% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxlocale.html#wxlocalegetsystemencoding">external documentation</a>. +%%<br /> Res = ?wxFONTENCODING_SYSTEM | ?wxFONTENCODING_DEFAULT | ?wxFONTENCODING_ISO8859_1 | ?wxFONTENCODING_ISO8859_2 | ?wxFONTENCODING_ISO8859_3 | ?wxFONTENCODING_ISO8859_4 | ?wxFONTENCODING_ISO8859_5 | ?wxFONTENCODING_ISO8859_6 | ?wxFONTENCODING_ISO8859_7 | ?wxFONTENCODING_ISO8859_8 | ?wxFONTENCODING_ISO8859_9 | ?wxFONTENCODING_ISO8859_10 | ?wxFONTENCODING_ISO8859_11 | ?wxFONTENCODING_ISO8859_12 | ?wxFONTENCODING_ISO8859_13 | ?wxFONTENCODING_ISO8859_14 | ?wxFONTENCODING_ISO8859_15 | ?wxFONTENCODING_ISO8859_MAX | ?wxFONTENCODING_KOI8 | ?wxFONTENCODING_KOI8_U | ?wxFONTENCODING_ALTERNATIVE | ?wxFONTENCODING_BULGARIAN | ?wxFONTENCODING_CP437 | ?wxFONTENCODING_CP850 | ?wxFONTENCODING_CP852 | ?wxFONTENCODING_CP855 | ?wxFONTENCODING_CP866 | ?wxFONTENCODING_CP874 | ?wxFONTENCODING_CP932 | ?wxFONTENCODING_CP936 | ?wxFONTENCODING_CP949 | ?wxFONTENCODING_CP950 | ?wxFONTENCODING_CP1250 | ?wxFONTENCODING_CP1251 | ?wxFONTENCODING_CP1252 | ?wxFONTENCODING_CP1253 | ?wxFONTENCODING_CP1254 | ?wxFONTENCODING_CP1255 | ?wxFONTENCODING_CP1256 | ?wxFONTENCODING_CP1257 | ?wxFONTENCODING_CP12_MAX | ?wxFONTENCODING_UTF7 | ?wxFONTENCODING_UTF8 | ?wxFONTENCODING_EUC_JP | ?wxFONTENCODING_UTF16BE | ?wxFONTENCODING_UTF16LE | ?wxFONTENCODING_UTF32BE | ?wxFONTENCODING_UTF32LE | ?wxFONTENCODING_MACROMAN | ?wxFONTENCODING_MACJAPANESE | ?wxFONTENCODING_MACCHINESETRAD | ?wxFONTENCODING_MACKOREAN | ?wxFONTENCODING_MACARABIC | ?wxFONTENCODING_MACHEBREW | ?wxFONTENCODING_MACGREEK | ?wxFONTENCODING_MACCYRILLIC | ?wxFONTENCODING_MACDEVANAGARI | ?wxFONTENCODING_MACGURMUKHI | ?wxFONTENCODING_MACGUJARATI | ?wxFONTENCODING_MACORIYA | ?wxFONTENCODING_MACBENGALI | ?wxFONTENCODING_MACTAMIL | ?wxFONTENCODING_MACTELUGU | ?wxFONTENCODING_MACKANNADA | ?wxFONTENCODING_MACMALAJALAM | ?wxFONTENCODING_MACSINHALESE | ?wxFONTENCODING_MACBURMESE | ?wxFONTENCODING_MACKHMER | ?wxFONTENCODING_MACTHAI | ?wxFONTENCODING_MACLAOTIAN | ?wxFONTENCODING_MACGEORGIAN | ?wxFONTENCODING_MACARMENIAN | ?wxFONTENCODING_MACCHINESESIMP | ?wxFONTENCODING_MACTIBETAN | ?wxFONTENCODING_MACMONGOLIAN | ?wxFONTENCODING_MACETHIOPIC | ?wxFONTENCODING_MACCENTRALEUR | ?wxFONTENCODING_MACVIATNAMESE | ?wxFONTENCODING_MACARABICEXT | ?wxFONTENCODING_MACSYMBOL | ?wxFONTENCODING_MACDINGBATS | ?wxFONTENCODING_MACTURKISH | ?wxFONTENCODING_MACCROATIAN | ?wxFONTENCODING_MACICELANDIC | ?wxFONTENCODING_MACROMANIAN | ?wxFONTENCODING_MACCELTIC | ?wxFONTENCODING_MACGAELIC | ?wxFONTENCODING_MACKEYBOARD | ?wxFONTENCODING_MAX | ?wxFONTENCODING_MACMIN | ?wxFONTENCODING_MACMAX | ?wxFONTENCODING_UTF16 | ?wxFONTENCODING_UTF32 | ?wxFONTENCODING_UNICODE | ?wxFONTENCODING_GB2312 | ?wxFONTENCODING_BIG5 | ?wxFONTENCODING_SHIFT_JIS +-spec getSystemEncoding() -> wx:wx_enum(). +getSystemEncoding() -> + wxe_util:call(?wxLocale_GetSystemEncoding, + <<>>). + +%% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxlocale.html#wxlocalegetsystemencodingname">external documentation</a>. +-spec getSystemEncodingName() -> unicode:charlist(). +getSystemEncodingName() -> + wxe_util:call(?wxLocale_GetSystemEncodingName, + <<>>). + +%% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxlocale.html#wxlocalegetsystemlanguage">external documentation</a>. +-spec getSystemLanguage() -> integer(). +getSystemLanguage() -> + wxe_util:call(?wxLocale_GetSystemLanguage, + <<>>). + +%% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxlocale.html#wxlocaleisloaded">external documentation</a>. +-spec isLoaded(This, SzDomain) -> boolean() when + This::wxLocale(), SzDomain::unicode:chardata(). +isLoaded(#wx_ref{type=ThisT,ref=ThisRef},SzDomain) + when is_list(SzDomain) -> + ?CLASS(ThisT,wxLocale), + SzDomain_UC = unicode:characters_to_binary([SzDomain,0]), + wxe_util:call(?wxLocale_IsLoaded, + <<ThisRef:32/?UI,(byte_size(SzDomain_UC)):32/?UI,(SzDomain_UC)/binary, 0:(((8- ((0+byte_size(SzDomain_UC)) band 16#7)) band 16#7))/unit:8>>). + +%% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxlocale.html#wxlocaleisok">external documentation</a>. +-spec isOk(This) -> boolean() when + This::wxLocale(). +isOk(#wx_ref{type=ThisT,ref=ThisRef}) -> + ?CLASS(ThisT,wxLocale), + wxe_util:call(?wxLocale_IsOk, + <<ThisRef:32/?UI>>). + +%% @doc Destroys this object, do not use object again +-spec destroy(This::wxLocale()) -> ok. +destroy(Obj=#wx_ref{type=Type}) -> + ?CLASS(Type,wxLocale), + wxe_util:destroy(?wxLocale_destruct,Obj), + ok. diff --git a/lib/wx/src/gen/wxPopupTransientWindow.erl b/lib/wx/src/gen/wxPopupTransientWindow.erl new file mode 100644 index 0000000000..253d33e5ac --- /dev/null +++ b/lib/wx/src/gen/wxPopupTransientWindow.erl @@ -0,0 +1,506 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2008-2014. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% This file is generated DO NOT EDIT + +%% @doc See external documentation: <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxpopuptransientwindow.html">wxPopupTransientWindow</a>. +%% <p>This class is derived (and can use functions) from: +%% <br />{@link wxPopupWindow} +%% <br />{@link wxWindow} +%% <br />{@link wxEvtHandler} +%% </p> +%% @type wxPopupTransientWindow(). An object reference, The representation is internal +%% and can be changed without notice. It can't be used for comparsion +%% stored on disc or distributed for use on other nodes. + +-module(wxPopupTransientWindow). +-include("wxe.hrl"). +-export([destroy/1,dismiss/1,new/0,new/1,new/2,popup/1,popup/2]). + +%% inherited exports +-export([cacheBestSize/2,captureMouse/1,center/1,center/2,centerOnParent/1, + centerOnParent/2,centre/1,centre/2,centreOnParent/1,centreOnParent/2, + clearBackground/1,clientToScreen/2,clientToScreen/3,close/1,close/2, + connect/2,connect/3,convertDialogToPixels/2,convertPixelsToDialog/2, + destroyChildren/1,disable/1,disconnect/1,disconnect/2,disconnect/3, + enable/1,enable/2,findWindow/2,fit/1,fitInside/1,freeze/1,getAcceleratorTable/1, + getBackgroundColour/1,getBackgroundStyle/1,getBestSize/1,getCaret/1, + getCharHeight/1,getCharWidth/1,getChildren/1,getClientSize/1,getContainingSizer/1, + getCursor/1,getDropTarget/1,getEventHandler/1,getExtraStyle/1,getFont/1, + getForegroundColour/1,getGrandParent/1,getHandle/1,getHelpText/1, + getId/1,getLabel/1,getMaxSize/1,getMinSize/1,getName/1,getParent/1, + getPosition/1,getRect/1,getScreenPosition/1,getScreenRect/1,getScrollPos/2, + getScrollRange/2,getScrollThumb/2,getSize/1,getSizer/1,getTextExtent/2, + getTextExtent/3,getToolTip/1,getUpdateRegion/1,getVirtualSize/1,getWindowStyleFlag/1, + getWindowVariant/1,hasCapture/1,hasScrollbar/2,hasTransparentBackground/1, + hide/1,inheritAttributes/1,initDialog/1,invalidateBestSize/1,isEnabled/1, + isExposed/2,isExposed/3,isExposed/5,isRetained/1,isShown/1,isTopLevel/1, + layout/1,lineDown/1,lineUp/1,lower/1,makeModal/1,makeModal/2,move/2, + move/3,move/4,moveAfterInTabOrder/2,moveBeforeInTabOrder/2,navigate/1, + navigate/2,pageDown/1,pageUp/1,parent_class/1,popEventHandler/1,popEventHandler/2, + popupMenu/2,popupMenu/3,popupMenu/4,position/3,raise/1,refresh/1,refresh/2, + refreshRect/2,refreshRect/3,releaseMouse/1,removeChild/2,reparent/2, + screenToClient/1,screenToClient/2,scrollLines/2,scrollPages/2,scrollWindow/3, + scrollWindow/4,setAcceleratorTable/2,setAutoLayout/2,setBackgroundColour/2, + setBackgroundStyle/2,setCaret/2,setClientSize/2,setClientSize/3,setContainingSizer/2, + setCursor/2,setDropTarget/2,setExtraStyle/2,setFocus/1,setFocusFromKbd/1, + setFont/2,setForegroundColour/2,setHelpText/2,setId/2,setLabel/2,setMaxSize/2, + setMinSize/2,setName/2,setOwnBackgroundColour/2,setOwnFont/2,setOwnForegroundColour/2, + setPalette/2,setScrollPos/3,setScrollPos/4,setScrollbar/5,setScrollbar/6, + setSize/2,setSize/3,setSize/5,setSize/6,setSizeHints/2,setSizeHints/3, + setSizeHints/4,setSizer/2,setSizer/3,setSizerAndFit/2,setSizerAndFit/3, + setThemeEnabled/2,setToolTip/2,setVirtualSize/2,setVirtualSize/3, + setVirtualSizeHints/2,setVirtualSizeHints/3,setVirtualSizeHints/4, + setWindowStyle/2,setWindowStyleFlag/2,setWindowVariant/2,shouldInheritColours/1, + show/1,show/2,thaw/1,transferDataFromWindow/1,transferDataToWindow/1, + update/1,updateWindowUI/1,updateWindowUI/2,validate/1,warpPointer/3]). + +-export_type([wxPopupTransientWindow/0]). +%% @hidden +parent_class(wxPopupWindow) -> true; +parent_class(wxWindow) -> true; +parent_class(wxEvtHandler) -> true; +parent_class(_Class) -> erlang:error({badtype, ?MODULE}). + +-type wxPopupTransientWindow() :: wx:wx_object(). +%% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxpopuptransientwindow.html#wxpopuptransientwindowwxpopuptransientwindow">external documentation</a>. +-spec new() -> wxPopupTransientWindow(). +new() -> + wxe_util:construct(?wxPopupTransientWindow_new_0, + <<>>). + +%% @equiv new(Parent, []) +-spec new(Parent) -> wxPopupTransientWindow() when + Parent::wxWindow:wxWindow(). + +new(Parent) + when is_record(Parent, wx_ref) -> + new(Parent, []). + +%% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxpopuptransientwindow.html#wxpopuptransientwindowwxpopuptransientwindow">external documentation</a>. +-spec new(Parent, [Option]) -> wxPopupTransientWindow() when + Parent::wxWindow:wxWindow(), + Option :: {style, integer()}. +new(#wx_ref{type=ParentT,ref=ParentRef}, Options) + when is_list(Options) -> + ?CLASS(ParentT,wxWindow), + MOpts = fun({style, Style}, Acc) -> [<<1:32/?UI,Style:32/?UI>>|Acc]; + (BadOpt, _) -> erlang:error({badoption, BadOpt}) end, + BinOpt = list_to_binary(lists:foldl(MOpts, [<<0:32>>], Options)), + wxe_util:construct(?wxPopupTransientWindow_new_2, + <<ParentRef:32/?UI, 0:32,BinOpt/binary>>). + +%% @equiv popup(This, []) +-spec popup(This) -> ok when + This::wxPopupTransientWindow(). + +popup(This) + when is_record(This, wx_ref) -> + popup(This, []). + +%% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxpopuptransientwindow.html#wxpopuptransientwindowpopup">external documentation</a>. +-spec popup(This, [Option]) -> ok when + This::wxPopupTransientWindow(), + Option :: {focus, wxWindow:wxWindow()}. +popup(#wx_ref{type=ThisT,ref=ThisRef}, Options) + when is_list(Options) -> + ?CLASS(ThisT,wxPopupTransientWindow), + MOpts = fun({focus, #wx_ref{type=FocusT,ref=FocusRef}}, Acc) -> ?CLASS(FocusT,wxWindow),[<<1:32/?UI,FocusRef:32/?UI>>|Acc]; + (BadOpt, _) -> erlang:error({badoption, BadOpt}) end, + BinOpt = list_to_binary(lists:foldl(MOpts, [<<0:32>>], Options)), + wxe_util:cast(?wxPopupTransientWindow_Popup, + <<ThisRef:32/?UI, 0:32,BinOpt/binary>>). + +%% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxpopuptransientwindow.html#wxpopuptransientwindowdismiss">external documentation</a>. +-spec dismiss(This) -> ok when + This::wxPopupTransientWindow(). +dismiss(#wx_ref{type=ThisT,ref=ThisRef}) -> + ?CLASS(ThisT,wxPopupTransientWindow), + wxe_util:cast(?wxPopupTransientWindow_Dismiss, + <<ThisRef:32/?UI>>). + +%% @doc Destroys this object, do not use object again +-spec destroy(This::wxPopupTransientWindow()) -> ok. +destroy(Obj=#wx_ref{type=Type}) -> + ?CLASS(Type,wxPopupTransientWindow), + wxe_util:destroy(?DESTROY_OBJECT,Obj), + ok. + %% From wxPopupWindow +%% @hidden +position(This,PtOrigin,Size) -> wxPopupWindow:position(This,PtOrigin,Size). + %% From wxWindow +%% @hidden +warpPointer(This,X,Y) -> wxWindow:warpPointer(This,X,Y). +%% @hidden +validate(This) -> wxWindow:validate(This). +%% @hidden +updateWindowUI(This, Options) -> wxWindow:updateWindowUI(This, Options). +%% @hidden +updateWindowUI(This) -> wxWindow:updateWindowUI(This). +%% @hidden +update(This) -> wxWindow:update(This). +%% @hidden +transferDataToWindow(This) -> wxWindow:transferDataToWindow(This). +%% @hidden +transferDataFromWindow(This) -> wxWindow:transferDataFromWindow(This). +%% @hidden +thaw(This) -> wxWindow:thaw(This). +%% @hidden +show(This, Options) -> wxWindow:show(This, Options). +%% @hidden +show(This) -> wxWindow:show(This). +%% @hidden +shouldInheritColours(This) -> wxWindow:shouldInheritColours(This). +%% @hidden +setWindowVariant(This,Variant) -> wxWindow:setWindowVariant(This,Variant). +%% @hidden +setWindowStyleFlag(This,Style) -> wxWindow:setWindowStyleFlag(This,Style). +%% @hidden +setWindowStyle(This,Style) -> wxWindow:setWindowStyle(This,Style). +%% @hidden +setVirtualSizeHints(This,MinW,MinH, Options) -> wxWindow:setVirtualSizeHints(This,MinW,MinH, Options). +%% @hidden +setVirtualSizeHints(This,MinW,MinH) -> wxWindow:setVirtualSizeHints(This,MinW,MinH). +%% @hidden +setVirtualSizeHints(This,MinSize) -> wxWindow:setVirtualSizeHints(This,MinSize). +%% @hidden +setVirtualSize(This,X,Y) -> wxWindow:setVirtualSize(This,X,Y). +%% @hidden +setVirtualSize(This,Size) -> wxWindow:setVirtualSize(This,Size). +%% @hidden +setToolTip(This,Tip) -> wxWindow:setToolTip(This,Tip). +%% @hidden +setThemeEnabled(This,EnableTheme) -> wxWindow:setThemeEnabled(This,EnableTheme). +%% @hidden +setSizerAndFit(This,Sizer, Options) -> wxWindow:setSizerAndFit(This,Sizer, Options). +%% @hidden +setSizerAndFit(This,Sizer) -> wxWindow:setSizerAndFit(This,Sizer). +%% @hidden +setSizer(This,Sizer, Options) -> wxWindow:setSizer(This,Sizer, Options). +%% @hidden +setSizer(This,Sizer) -> wxWindow:setSizer(This,Sizer). +%% @hidden +setSizeHints(This,MinW,MinH, Options) -> wxWindow:setSizeHints(This,MinW,MinH, Options). +%% @hidden +setSizeHints(This,MinW,MinH) -> wxWindow:setSizeHints(This,MinW,MinH). +%% @hidden +setSizeHints(This,MinSize) -> wxWindow:setSizeHints(This,MinSize). +%% @hidden +setSize(This,X,Y,Width,Height, Options) -> wxWindow:setSize(This,X,Y,Width,Height, Options). +%% @hidden +setSize(This,X,Y,Width,Height) -> wxWindow:setSize(This,X,Y,Width,Height). +%% @hidden +setSize(This,Width,Height) -> wxWindow:setSize(This,Width,Height). +%% @hidden +setSize(This,Rect) -> wxWindow:setSize(This,Rect). +%% @hidden +setScrollPos(This,Orient,Pos, Options) -> wxWindow:setScrollPos(This,Orient,Pos, Options). +%% @hidden +setScrollPos(This,Orient,Pos) -> wxWindow:setScrollPos(This,Orient,Pos). +%% @hidden +setScrollbar(This,Orient,Pos,ThumbVisible,Range, Options) -> wxWindow:setScrollbar(This,Orient,Pos,ThumbVisible,Range, Options). +%% @hidden +setScrollbar(This,Orient,Pos,ThumbVisible,Range) -> wxWindow:setScrollbar(This,Orient,Pos,ThumbVisible,Range). +%% @hidden +setPalette(This,Pal) -> wxWindow:setPalette(This,Pal). +%% @hidden +setName(This,Name) -> wxWindow:setName(This,Name). +%% @hidden +setLabel(This,Label) -> wxWindow:setLabel(This,Label). +%% @hidden +setId(This,Winid) -> wxWindow:setId(This,Winid). +%% @hidden +setHelpText(This,Text) -> wxWindow:setHelpText(This,Text). +%% @hidden +setForegroundColour(This,Colour) -> wxWindow:setForegroundColour(This,Colour). +%% @hidden +setFont(This,Font) -> wxWindow:setFont(This,Font). +%% @hidden +setFocusFromKbd(This) -> wxWindow:setFocusFromKbd(This). +%% @hidden +setFocus(This) -> wxWindow:setFocus(This). +%% @hidden +setExtraStyle(This,ExStyle) -> wxWindow:setExtraStyle(This,ExStyle). +%% @hidden +setDropTarget(This,DropTarget) -> wxWindow:setDropTarget(This,DropTarget). +%% @hidden +setOwnForegroundColour(This,Colour) -> wxWindow:setOwnForegroundColour(This,Colour). +%% @hidden +setOwnFont(This,Font) -> wxWindow:setOwnFont(This,Font). +%% @hidden +setOwnBackgroundColour(This,Colour) -> wxWindow:setOwnBackgroundColour(This,Colour). +%% @hidden +setMinSize(This,MinSize) -> wxWindow:setMinSize(This,MinSize). +%% @hidden +setMaxSize(This,MaxSize) -> wxWindow:setMaxSize(This,MaxSize). +%% @hidden +setCursor(This,Cursor) -> wxWindow:setCursor(This,Cursor). +%% @hidden +setContainingSizer(This,Sizer) -> wxWindow:setContainingSizer(This,Sizer). +%% @hidden +setClientSize(This,Width,Height) -> wxWindow:setClientSize(This,Width,Height). +%% @hidden +setClientSize(This,Size) -> wxWindow:setClientSize(This,Size). +%% @hidden +setCaret(This,Caret) -> wxWindow:setCaret(This,Caret). +%% @hidden +setBackgroundStyle(This,Style) -> wxWindow:setBackgroundStyle(This,Style). +%% @hidden +setBackgroundColour(This,Colour) -> wxWindow:setBackgroundColour(This,Colour). +%% @hidden +setAutoLayout(This,AutoLayout) -> wxWindow:setAutoLayout(This,AutoLayout). +%% @hidden +setAcceleratorTable(This,Accel) -> wxWindow:setAcceleratorTable(This,Accel). +%% @hidden +scrollWindow(This,Dx,Dy, Options) -> wxWindow:scrollWindow(This,Dx,Dy, Options). +%% @hidden +scrollWindow(This,Dx,Dy) -> wxWindow:scrollWindow(This,Dx,Dy). +%% @hidden +scrollPages(This,Pages) -> wxWindow:scrollPages(This,Pages). +%% @hidden +scrollLines(This,Lines) -> wxWindow:scrollLines(This,Lines). +%% @hidden +screenToClient(This,Pt) -> wxWindow:screenToClient(This,Pt). +%% @hidden +screenToClient(This) -> wxWindow:screenToClient(This). +%% @hidden +reparent(This,NewParent) -> wxWindow:reparent(This,NewParent). +%% @hidden +removeChild(This,Child) -> wxWindow:removeChild(This,Child). +%% @hidden +releaseMouse(This) -> wxWindow:releaseMouse(This). +%% @hidden +refreshRect(This,Rect, Options) -> wxWindow:refreshRect(This,Rect, Options). +%% @hidden +refreshRect(This,Rect) -> wxWindow:refreshRect(This,Rect). +%% @hidden +refresh(This, Options) -> wxWindow:refresh(This, Options). +%% @hidden +refresh(This) -> wxWindow:refresh(This). +%% @hidden +raise(This) -> wxWindow:raise(This). +%% @hidden +popupMenu(This,Menu,X,Y) -> wxWindow:popupMenu(This,Menu,X,Y). +%% @hidden +popupMenu(This,Menu, Options) -> wxWindow:popupMenu(This,Menu, Options). +%% @hidden +popupMenu(This,Menu) -> wxWindow:popupMenu(This,Menu). +%% @hidden +popEventHandler(This, Options) -> wxWindow:popEventHandler(This, Options). +%% @hidden +popEventHandler(This) -> wxWindow:popEventHandler(This). +%% @hidden +pageUp(This) -> wxWindow:pageUp(This). +%% @hidden +pageDown(This) -> wxWindow:pageDown(This). +%% @hidden +navigate(This, Options) -> wxWindow:navigate(This, Options). +%% @hidden +navigate(This) -> wxWindow:navigate(This). +%% @hidden +moveBeforeInTabOrder(This,Win) -> wxWindow:moveBeforeInTabOrder(This,Win). +%% @hidden +moveAfterInTabOrder(This,Win) -> wxWindow:moveAfterInTabOrder(This,Win). +%% @hidden +move(This,X,Y, Options) -> wxWindow:move(This,X,Y, Options). +%% @hidden +move(This,X,Y) -> wxWindow:move(This,X,Y). +%% @hidden +move(This,Pt) -> wxWindow:move(This,Pt). +%% @hidden +makeModal(This, Options) -> wxWindow:makeModal(This, Options). +%% @hidden +makeModal(This) -> wxWindow:makeModal(This). +%% @hidden +lower(This) -> wxWindow:lower(This). +%% @hidden +lineUp(This) -> wxWindow:lineUp(This). +%% @hidden +lineDown(This) -> wxWindow:lineDown(This). +%% @hidden +layout(This) -> wxWindow:layout(This). +%% @hidden +isTopLevel(This) -> wxWindow:isTopLevel(This). +%% @hidden +isShown(This) -> wxWindow:isShown(This). +%% @hidden +isRetained(This) -> wxWindow:isRetained(This). +%% @hidden +isExposed(This,X,Y,W,H) -> wxWindow:isExposed(This,X,Y,W,H). +%% @hidden +isExposed(This,X,Y) -> wxWindow:isExposed(This,X,Y). +%% @hidden +isExposed(This,Pt) -> wxWindow:isExposed(This,Pt). +%% @hidden +isEnabled(This) -> wxWindow:isEnabled(This). +%% @hidden +invalidateBestSize(This) -> wxWindow:invalidateBestSize(This). +%% @hidden +initDialog(This) -> wxWindow:initDialog(This). +%% @hidden +inheritAttributes(This) -> wxWindow:inheritAttributes(This). +%% @hidden +hide(This) -> wxWindow:hide(This). +%% @hidden +hasTransparentBackground(This) -> wxWindow:hasTransparentBackground(This). +%% @hidden +hasScrollbar(This,Orient) -> wxWindow:hasScrollbar(This,Orient). +%% @hidden +hasCapture(This) -> wxWindow:hasCapture(This). +%% @hidden +getWindowVariant(This) -> wxWindow:getWindowVariant(This). +%% @hidden +getWindowStyleFlag(This) -> wxWindow:getWindowStyleFlag(This). +%% @hidden +getVirtualSize(This) -> wxWindow:getVirtualSize(This). +%% @hidden +getUpdateRegion(This) -> wxWindow:getUpdateRegion(This). +%% @hidden +getToolTip(This) -> wxWindow:getToolTip(This). +%% @hidden +getTextExtent(This,String, Options) -> wxWindow:getTextExtent(This,String, Options). +%% @hidden +getTextExtent(This,String) -> wxWindow:getTextExtent(This,String). +%% @hidden +getSizer(This) -> wxWindow:getSizer(This). +%% @hidden +getSize(This) -> wxWindow:getSize(This). +%% @hidden +getScrollThumb(This,Orient) -> wxWindow:getScrollThumb(This,Orient). +%% @hidden +getScrollRange(This,Orient) -> wxWindow:getScrollRange(This,Orient). +%% @hidden +getScrollPos(This,Orient) -> wxWindow:getScrollPos(This,Orient). +%% @hidden +getScreenRect(This) -> wxWindow:getScreenRect(This). +%% @hidden +getScreenPosition(This) -> wxWindow:getScreenPosition(This). +%% @hidden +getRect(This) -> wxWindow:getRect(This). +%% @hidden +getPosition(This) -> wxWindow:getPosition(This). +%% @hidden +getParent(This) -> wxWindow:getParent(This). +%% @hidden +getName(This) -> wxWindow:getName(This). +%% @hidden +getMinSize(This) -> wxWindow:getMinSize(This). +%% @hidden +getMaxSize(This) -> wxWindow:getMaxSize(This). +%% @hidden +getLabel(This) -> wxWindow:getLabel(This). +%% @hidden +getId(This) -> wxWindow:getId(This). +%% @hidden +getHelpText(This) -> wxWindow:getHelpText(This). +%% @hidden +getHandle(This) -> wxWindow:getHandle(This). +%% @hidden +getGrandParent(This) -> wxWindow:getGrandParent(This). +%% @hidden +getForegroundColour(This) -> wxWindow:getForegroundColour(This). +%% @hidden +getFont(This) -> wxWindow:getFont(This). +%% @hidden +getExtraStyle(This) -> wxWindow:getExtraStyle(This). +%% @hidden +getEventHandler(This) -> wxWindow:getEventHandler(This). +%% @hidden +getDropTarget(This) -> wxWindow:getDropTarget(This). +%% @hidden +getCursor(This) -> wxWindow:getCursor(This). +%% @hidden +getContainingSizer(This) -> wxWindow:getContainingSizer(This). +%% @hidden +getClientSize(This) -> wxWindow:getClientSize(This). +%% @hidden +getChildren(This) -> wxWindow:getChildren(This). +%% @hidden +getCharWidth(This) -> wxWindow:getCharWidth(This). +%% @hidden +getCharHeight(This) -> wxWindow:getCharHeight(This). +%% @hidden +getCaret(This) -> wxWindow:getCaret(This). +%% @hidden +getBestSize(This) -> wxWindow:getBestSize(This). +%% @hidden +getBackgroundStyle(This) -> wxWindow:getBackgroundStyle(This). +%% @hidden +getBackgroundColour(This) -> wxWindow:getBackgroundColour(This). +%% @hidden +getAcceleratorTable(This) -> wxWindow:getAcceleratorTable(This). +%% @hidden +freeze(This) -> wxWindow:freeze(This). +%% @hidden +fitInside(This) -> wxWindow:fitInside(This). +%% @hidden +fit(This) -> wxWindow:fit(This). +%% @hidden +findWindow(This,Winid) -> wxWindow:findWindow(This,Winid). +%% @hidden +enable(This, Options) -> wxWindow:enable(This, Options). +%% @hidden +enable(This) -> wxWindow:enable(This). +%% @hidden +disable(This) -> wxWindow:disable(This). +%% @hidden +destroyChildren(This) -> wxWindow:destroyChildren(This). +%% @hidden +convertPixelsToDialog(This,Sz) -> wxWindow:convertPixelsToDialog(This,Sz). +%% @hidden +convertDialogToPixels(This,Sz) -> wxWindow:convertDialogToPixels(This,Sz). +%% @hidden +close(This, Options) -> wxWindow:close(This, Options). +%% @hidden +close(This) -> wxWindow:close(This). +%% @hidden +clientToScreen(This,X,Y) -> wxWindow:clientToScreen(This,X,Y). +%% @hidden +clientToScreen(This,Pt) -> wxWindow:clientToScreen(This,Pt). +%% @hidden +clearBackground(This) -> wxWindow:clearBackground(This). +%% @hidden +centreOnParent(This, Options) -> wxWindow:centreOnParent(This, Options). +%% @hidden +centreOnParent(This) -> wxWindow:centreOnParent(This). +%% @hidden +centre(This, Options) -> wxWindow:centre(This, Options). +%% @hidden +centre(This) -> wxWindow:centre(This). +%% @hidden +centerOnParent(This, Options) -> wxWindow:centerOnParent(This, Options). +%% @hidden +centerOnParent(This) -> wxWindow:centerOnParent(This). +%% @hidden +center(This, Options) -> wxWindow:center(This, Options). +%% @hidden +center(This) -> wxWindow:center(This). +%% @hidden +captureMouse(This) -> wxWindow:captureMouse(This). +%% @hidden +cacheBestSize(This,Size) -> wxWindow:cacheBestSize(This,Size). + %% From wxEvtHandler +%% @hidden +disconnect(This,EventType, Options) -> wxEvtHandler:disconnect(This,EventType, Options). +%% @hidden +disconnect(This,EventType) -> wxEvtHandler:disconnect(This,EventType). +%% @hidden +disconnect(This) -> wxEvtHandler:disconnect(This). +%% @hidden +connect(This,EventType, Options) -> wxEvtHandler:connect(This,EventType, Options). +%% @hidden +connect(This,EventType) -> wxEvtHandler:connect(This,EventType). diff --git a/lib/wx/src/gen/wxPopupWindow.erl b/lib/wx/src/gen/wxPopupWindow.erl new file mode 100644 index 0000000000..415185d574 --- /dev/null +++ b/lib/wx/src/gen/wxPopupWindow.erl @@ -0,0 +1,503 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2008-2014. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% This file is generated DO NOT EDIT + +%% @doc See external documentation: <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxpopupwindow.html">wxPopupWindow</a>. +%% <p>This class is derived (and can use functions) from: +%% <br />{@link wxWindow} +%% <br />{@link wxEvtHandler} +%% </p> +%% @type wxPopupWindow(). An object reference, The representation is internal +%% and can be changed without notice. It can't be used for comparsion +%% stored on disc or distributed for use on other nodes. + +-module(wxPopupWindow). +-include("wxe.hrl"). +-export([create/2,create/3,destroy/1,new/0,new/1,new/2,position/3]). + +%% inherited exports +-export([cacheBestSize/2,captureMouse/1,center/1,center/2,centerOnParent/1, + centerOnParent/2,centre/1,centre/2,centreOnParent/1,centreOnParent/2, + clearBackground/1,clientToScreen/2,clientToScreen/3,close/1,close/2, + connect/2,connect/3,convertDialogToPixels/2,convertPixelsToDialog/2, + destroyChildren/1,disable/1,disconnect/1,disconnect/2,disconnect/3, + enable/1,enable/2,findWindow/2,fit/1,fitInside/1,freeze/1,getAcceleratorTable/1, + getBackgroundColour/1,getBackgroundStyle/1,getBestSize/1,getCaret/1, + getCharHeight/1,getCharWidth/1,getChildren/1,getClientSize/1,getContainingSizer/1, + getCursor/1,getDropTarget/1,getEventHandler/1,getExtraStyle/1,getFont/1, + getForegroundColour/1,getGrandParent/1,getHandle/1,getHelpText/1, + getId/1,getLabel/1,getMaxSize/1,getMinSize/1,getName/1,getParent/1, + getPosition/1,getRect/1,getScreenPosition/1,getScreenRect/1,getScrollPos/2, + getScrollRange/2,getScrollThumb/2,getSize/1,getSizer/1,getTextExtent/2, + getTextExtent/3,getToolTip/1,getUpdateRegion/1,getVirtualSize/1,getWindowStyleFlag/1, + getWindowVariant/1,hasCapture/1,hasScrollbar/2,hasTransparentBackground/1, + hide/1,inheritAttributes/1,initDialog/1,invalidateBestSize/1,isEnabled/1, + isExposed/2,isExposed/3,isExposed/5,isRetained/1,isShown/1,isTopLevel/1, + layout/1,lineDown/1,lineUp/1,lower/1,makeModal/1,makeModal/2,move/2, + move/3,move/4,moveAfterInTabOrder/2,moveBeforeInTabOrder/2,navigate/1, + navigate/2,pageDown/1,pageUp/1,parent_class/1,popEventHandler/1,popEventHandler/2, + popupMenu/2,popupMenu/3,popupMenu/4,raise/1,refresh/1,refresh/2,refreshRect/2, + refreshRect/3,releaseMouse/1,removeChild/2,reparent/2,screenToClient/1, + screenToClient/2,scrollLines/2,scrollPages/2,scrollWindow/3,scrollWindow/4, + setAcceleratorTable/2,setAutoLayout/2,setBackgroundColour/2,setBackgroundStyle/2, + setCaret/2,setClientSize/2,setClientSize/3,setContainingSizer/2,setCursor/2, + setDropTarget/2,setExtraStyle/2,setFocus/1,setFocusFromKbd/1,setFont/2, + setForegroundColour/2,setHelpText/2,setId/2,setLabel/2,setMaxSize/2, + setMinSize/2,setName/2,setOwnBackgroundColour/2,setOwnFont/2,setOwnForegroundColour/2, + setPalette/2,setScrollPos/3,setScrollPos/4,setScrollbar/5,setScrollbar/6, + setSize/2,setSize/3,setSize/5,setSize/6,setSizeHints/2,setSizeHints/3, + setSizeHints/4,setSizer/2,setSizer/3,setSizerAndFit/2,setSizerAndFit/3, + setThemeEnabled/2,setToolTip/2,setVirtualSize/2,setVirtualSize/3, + setVirtualSizeHints/2,setVirtualSizeHints/3,setVirtualSizeHints/4, + setWindowStyle/2,setWindowStyleFlag/2,setWindowVariant/2,shouldInheritColours/1, + show/1,show/2,thaw/1,transferDataFromWindow/1,transferDataToWindow/1, + update/1,updateWindowUI/1,updateWindowUI/2,validate/1,warpPointer/3]). + +-export_type([wxPopupWindow/0]). +%% @hidden +parent_class(wxWindow) -> true; +parent_class(wxEvtHandler) -> true; +parent_class(_Class) -> erlang:error({badtype, ?MODULE}). + +-type wxPopupWindow() :: wx:wx_object(). +%% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxpopupwindow.html#wxpopupwindowwxpopupwindow">external documentation</a>. +-spec new() -> wxPopupWindow(). +new() -> + wxe_util:construct(?wxPopupWindow_new_0, + <<>>). + +%% @equiv new(Parent, []) +-spec new(Parent) -> wxPopupWindow() when + Parent::wxWindow:wxWindow(). + +new(Parent) + when is_record(Parent, wx_ref) -> + new(Parent, []). + +%% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxpopupwindow.html#wxpopupwindowwxpopupwindow">external documentation</a>. +-spec new(Parent, [Option]) -> wxPopupWindow() when + Parent::wxWindow:wxWindow(), + Option :: {flags, integer()}. +new(#wx_ref{type=ParentT,ref=ParentRef}, Options) + when is_list(Options) -> + ?CLASS(ParentT,wxWindow), + MOpts = fun({flags, Flags}, Acc) -> [<<1:32/?UI,Flags:32/?UI>>|Acc]; + (BadOpt, _) -> erlang:error({badoption, BadOpt}) end, + BinOpt = list_to_binary(lists:foldl(MOpts, [<<0:32>>], Options)), + wxe_util:construct(?wxPopupWindow_new_2, + <<ParentRef:32/?UI, 0:32,BinOpt/binary>>). + +%% @equiv create(This,Parent, []) +-spec create(This, Parent) -> boolean() when + This::wxPopupWindow(), Parent::wxWindow:wxWindow(). + +create(This,Parent) + when is_record(This, wx_ref),is_record(Parent, wx_ref) -> + create(This,Parent, []). + +%% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxpopupwindow.html#wxpopupwindowcreate">external documentation</a>. +-spec create(This, Parent, [Option]) -> boolean() when + This::wxPopupWindow(), Parent::wxWindow:wxWindow(), + Option :: {flags, integer()}. +create(#wx_ref{type=ThisT,ref=ThisRef},#wx_ref{type=ParentT,ref=ParentRef}, Options) + when is_list(Options) -> + ?CLASS(ThisT,wxPopupWindow), + ?CLASS(ParentT,wxWindow), + MOpts = fun({flags, Flags}, Acc) -> [<<1:32/?UI,Flags:32/?UI>>|Acc]; + (BadOpt, _) -> erlang:error({badoption, BadOpt}) end, + BinOpt = list_to_binary(lists:foldl(MOpts, [<<0:32>>], Options)), + wxe_util:call(?wxPopupWindow_Create, + <<ThisRef:32/?UI,ParentRef:32/?UI, BinOpt/binary>>). + +%% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxpopupwindow.html#wxpopupwindowposition">external documentation</a>. +-spec position(This, PtOrigin, Size) -> ok when + This::wxPopupWindow(), PtOrigin::{X::integer(), Y::integer()}, Size::{W::integer(), H::integer()}. +position(#wx_ref{type=ThisT,ref=ThisRef},{PtOriginX,PtOriginY},{SizeW,SizeH}) + when is_integer(PtOriginX),is_integer(PtOriginY),is_integer(SizeW),is_integer(SizeH) -> + ?CLASS(ThisT,wxPopupWindow), + wxe_util:cast(?wxPopupWindow_Position, + <<ThisRef:32/?UI,PtOriginX:32/?UI,PtOriginY:32/?UI,SizeW:32/?UI,SizeH:32/?UI>>). + +%% @doc Destroys this object, do not use object again +-spec destroy(This::wxPopupWindow()) -> ok. +destroy(Obj=#wx_ref{type=Type}) -> + ?CLASS(Type,wxPopupWindow), + wxe_util:destroy(?DESTROY_OBJECT,Obj), + ok. + %% From wxWindow +%% @hidden +warpPointer(This,X,Y) -> wxWindow:warpPointer(This,X,Y). +%% @hidden +validate(This) -> wxWindow:validate(This). +%% @hidden +updateWindowUI(This, Options) -> wxWindow:updateWindowUI(This, Options). +%% @hidden +updateWindowUI(This) -> wxWindow:updateWindowUI(This). +%% @hidden +update(This) -> wxWindow:update(This). +%% @hidden +transferDataToWindow(This) -> wxWindow:transferDataToWindow(This). +%% @hidden +transferDataFromWindow(This) -> wxWindow:transferDataFromWindow(This). +%% @hidden +thaw(This) -> wxWindow:thaw(This). +%% @hidden +show(This, Options) -> wxWindow:show(This, Options). +%% @hidden +show(This) -> wxWindow:show(This). +%% @hidden +shouldInheritColours(This) -> wxWindow:shouldInheritColours(This). +%% @hidden +setWindowVariant(This,Variant) -> wxWindow:setWindowVariant(This,Variant). +%% @hidden +setWindowStyleFlag(This,Style) -> wxWindow:setWindowStyleFlag(This,Style). +%% @hidden +setWindowStyle(This,Style) -> wxWindow:setWindowStyle(This,Style). +%% @hidden +setVirtualSizeHints(This,MinW,MinH, Options) -> wxWindow:setVirtualSizeHints(This,MinW,MinH, Options). +%% @hidden +setVirtualSizeHints(This,MinW,MinH) -> wxWindow:setVirtualSizeHints(This,MinW,MinH). +%% @hidden +setVirtualSizeHints(This,MinSize) -> wxWindow:setVirtualSizeHints(This,MinSize). +%% @hidden +setVirtualSize(This,X,Y) -> wxWindow:setVirtualSize(This,X,Y). +%% @hidden +setVirtualSize(This,Size) -> wxWindow:setVirtualSize(This,Size). +%% @hidden +setToolTip(This,Tip) -> wxWindow:setToolTip(This,Tip). +%% @hidden +setThemeEnabled(This,EnableTheme) -> wxWindow:setThemeEnabled(This,EnableTheme). +%% @hidden +setSizerAndFit(This,Sizer, Options) -> wxWindow:setSizerAndFit(This,Sizer, Options). +%% @hidden +setSizerAndFit(This,Sizer) -> wxWindow:setSizerAndFit(This,Sizer). +%% @hidden +setSizer(This,Sizer, Options) -> wxWindow:setSizer(This,Sizer, Options). +%% @hidden +setSizer(This,Sizer) -> wxWindow:setSizer(This,Sizer). +%% @hidden +setSizeHints(This,MinW,MinH, Options) -> wxWindow:setSizeHints(This,MinW,MinH, Options). +%% @hidden +setSizeHints(This,MinW,MinH) -> wxWindow:setSizeHints(This,MinW,MinH). +%% @hidden +setSizeHints(This,MinSize) -> wxWindow:setSizeHints(This,MinSize). +%% @hidden +setSize(This,X,Y,Width,Height, Options) -> wxWindow:setSize(This,X,Y,Width,Height, Options). +%% @hidden +setSize(This,X,Y,Width,Height) -> wxWindow:setSize(This,X,Y,Width,Height). +%% @hidden +setSize(This,Width,Height) -> wxWindow:setSize(This,Width,Height). +%% @hidden +setSize(This,Rect) -> wxWindow:setSize(This,Rect). +%% @hidden +setScrollPos(This,Orient,Pos, Options) -> wxWindow:setScrollPos(This,Orient,Pos, Options). +%% @hidden +setScrollPos(This,Orient,Pos) -> wxWindow:setScrollPos(This,Orient,Pos). +%% @hidden +setScrollbar(This,Orient,Pos,ThumbVisible,Range, Options) -> wxWindow:setScrollbar(This,Orient,Pos,ThumbVisible,Range, Options). +%% @hidden +setScrollbar(This,Orient,Pos,ThumbVisible,Range) -> wxWindow:setScrollbar(This,Orient,Pos,ThumbVisible,Range). +%% @hidden +setPalette(This,Pal) -> wxWindow:setPalette(This,Pal). +%% @hidden +setName(This,Name) -> wxWindow:setName(This,Name). +%% @hidden +setLabel(This,Label) -> wxWindow:setLabel(This,Label). +%% @hidden +setId(This,Winid) -> wxWindow:setId(This,Winid). +%% @hidden +setHelpText(This,Text) -> wxWindow:setHelpText(This,Text). +%% @hidden +setForegroundColour(This,Colour) -> wxWindow:setForegroundColour(This,Colour). +%% @hidden +setFont(This,Font) -> wxWindow:setFont(This,Font). +%% @hidden +setFocusFromKbd(This) -> wxWindow:setFocusFromKbd(This). +%% @hidden +setFocus(This) -> wxWindow:setFocus(This). +%% @hidden +setExtraStyle(This,ExStyle) -> wxWindow:setExtraStyle(This,ExStyle). +%% @hidden +setDropTarget(This,DropTarget) -> wxWindow:setDropTarget(This,DropTarget). +%% @hidden +setOwnForegroundColour(This,Colour) -> wxWindow:setOwnForegroundColour(This,Colour). +%% @hidden +setOwnFont(This,Font) -> wxWindow:setOwnFont(This,Font). +%% @hidden +setOwnBackgroundColour(This,Colour) -> wxWindow:setOwnBackgroundColour(This,Colour). +%% @hidden +setMinSize(This,MinSize) -> wxWindow:setMinSize(This,MinSize). +%% @hidden +setMaxSize(This,MaxSize) -> wxWindow:setMaxSize(This,MaxSize). +%% @hidden +setCursor(This,Cursor) -> wxWindow:setCursor(This,Cursor). +%% @hidden +setContainingSizer(This,Sizer) -> wxWindow:setContainingSizer(This,Sizer). +%% @hidden +setClientSize(This,Width,Height) -> wxWindow:setClientSize(This,Width,Height). +%% @hidden +setClientSize(This,Size) -> wxWindow:setClientSize(This,Size). +%% @hidden +setCaret(This,Caret) -> wxWindow:setCaret(This,Caret). +%% @hidden +setBackgroundStyle(This,Style) -> wxWindow:setBackgroundStyle(This,Style). +%% @hidden +setBackgroundColour(This,Colour) -> wxWindow:setBackgroundColour(This,Colour). +%% @hidden +setAutoLayout(This,AutoLayout) -> wxWindow:setAutoLayout(This,AutoLayout). +%% @hidden +setAcceleratorTable(This,Accel) -> wxWindow:setAcceleratorTable(This,Accel). +%% @hidden +scrollWindow(This,Dx,Dy, Options) -> wxWindow:scrollWindow(This,Dx,Dy, Options). +%% @hidden +scrollWindow(This,Dx,Dy) -> wxWindow:scrollWindow(This,Dx,Dy). +%% @hidden +scrollPages(This,Pages) -> wxWindow:scrollPages(This,Pages). +%% @hidden +scrollLines(This,Lines) -> wxWindow:scrollLines(This,Lines). +%% @hidden +screenToClient(This,Pt) -> wxWindow:screenToClient(This,Pt). +%% @hidden +screenToClient(This) -> wxWindow:screenToClient(This). +%% @hidden +reparent(This,NewParent) -> wxWindow:reparent(This,NewParent). +%% @hidden +removeChild(This,Child) -> wxWindow:removeChild(This,Child). +%% @hidden +releaseMouse(This) -> wxWindow:releaseMouse(This). +%% @hidden +refreshRect(This,Rect, Options) -> wxWindow:refreshRect(This,Rect, Options). +%% @hidden +refreshRect(This,Rect) -> wxWindow:refreshRect(This,Rect). +%% @hidden +refresh(This, Options) -> wxWindow:refresh(This, Options). +%% @hidden +refresh(This) -> wxWindow:refresh(This). +%% @hidden +raise(This) -> wxWindow:raise(This). +%% @hidden +popupMenu(This,Menu,X,Y) -> wxWindow:popupMenu(This,Menu,X,Y). +%% @hidden +popupMenu(This,Menu, Options) -> wxWindow:popupMenu(This,Menu, Options). +%% @hidden +popupMenu(This,Menu) -> wxWindow:popupMenu(This,Menu). +%% @hidden +popEventHandler(This, Options) -> wxWindow:popEventHandler(This, Options). +%% @hidden +popEventHandler(This) -> wxWindow:popEventHandler(This). +%% @hidden +pageUp(This) -> wxWindow:pageUp(This). +%% @hidden +pageDown(This) -> wxWindow:pageDown(This). +%% @hidden +navigate(This, Options) -> wxWindow:navigate(This, Options). +%% @hidden +navigate(This) -> wxWindow:navigate(This). +%% @hidden +moveBeforeInTabOrder(This,Win) -> wxWindow:moveBeforeInTabOrder(This,Win). +%% @hidden +moveAfterInTabOrder(This,Win) -> wxWindow:moveAfterInTabOrder(This,Win). +%% @hidden +move(This,X,Y, Options) -> wxWindow:move(This,X,Y, Options). +%% @hidden +move(This,X,Y) -> wxWindow:move(This,X,Y). +%% @hidden +move(This,Pt) -> wxWindow:move(This,Pt). +%% @hidden +makeModal(This, Options) -> wxWindow:makeModal(This, Options). +%% @hidden +makeModal(This) -> wxWindow:makeModal(This). +%% @hidden +lower(This) -> wxWindow:lower(This). +%% @hidden +lineUp(This) -> wxWindow:lineUp(This). +%% @hidden +lineDown(This) -> wxWindow:lineDown(This). +%% @hidden +layout(This) -> wxWindow:layout(This). +%% @hidden +isTopLevel(This) -> wxWindow:isTopLevel(This). +%% @hidden +isShown(This) -> wxWindow:isShown(This). +%% @hidden +isRetained(This) -> wxWindow:isRetained(This). +%% @hidden +isExposed(This,X,Y,W,H) -> wxWindow:isExposed(This,X,Y,W,H). +%% @hidden +isExposed(This,X,Y) -> wxWindow:isExposed(This,X,Y). +%% @hidden +isExposed(This,Pt) -> wxWindow:isExposed(This,Pt). +%% @hidden +isEnabled(This) -> wxWindow:isEnabled(This). +%% @hidden +invalidateBestSize(This) -> wxWindow:invalidateBestSize(This). +%% @hidden +initDialog(This) -> wxWindow:initDialog(This). +%% @hidden +inheritAttributes(This) -> wxWindow:inheritAttributes(This). +%% @hidden +hide(This) -> wxWindow:hide(This). +%% @hidden +hasTransparentBackground(This) -> wxWindow:hasTransparentBackground(This). +%% @hidden +hasScrollbar(This,Orient) -> wxWindow:hasScrollbar(This,Orient). +%% @hidden +hasCapture(This) -> wxWindow:hasCapture(This). +%% @hidden +getWindowVariant(This) -> wxWindow:getWindowVariant(This). +%% @hidden +getWindowStyleFlag(This) -> wxWindow:getWindowStyleFlag(This). +%% @hidden +getVirtualSize(This) -> wxWindow:getVirtualSize(This). +%% @hidden +getUpdateRegion(This) -> wxWindow:getUpdateRegion(This). +%% @hidden +getToolTip(This) -> wxWindow:getToolTip(This). +%% @hidden +getTextExtent(This,String, Options) -> wxWindow:getTextExtent(This,String, Options). +%% @hidden +getTextExtent(This,String) -> wxWindow:getTextExtent(This,String). +%% @hidden +getSizer(This) -> wxWindow:getSizer(This). +%% @hidden +getSize(This) -> wxWindow:getSize(This). +%% @hidden +getScrollThumb(This,Orient) -> wxWindow:getScrollThumb(This,Orient). +%% @hidden +getScrollRange(This,Orient) -> wxWindow:getScrollRange(This,Orient). +%% @hidden +getScrollPos(This,Orient) -> wxWindow:getScrollPos(This,Orient). +%% @hidden +getScreenRect(This) -> wxWindow:getScreenRect(This). +%% @hidden +getScreenPosition(This) -> wxWindow:getScreenPosition(This). +%% @hidden +getRect(This) -> wxWindow:getRect(This). +%% @hidden +getPosition(This) -> wxWindow:getPosition(This). +%% @hidden +getParent(This) -> wxWindow:getParent(This). +%% @hidden +getName(This) -> wxWindow:getName(This). +%% @hidden +getMinSize(This) -> wxWindow:getMinSize(This). +%% @hidden +getMaxSize(This) -> wxWindow:getMaxSize(This). +%% @hidden +getLabel(This) -> wxWindow:getLabel(This). +%% @hidden +getId(This) -> wxWindow:getId(This). +%% @hidden +getHelpText(This) -> wxWindow:getHelpText(This). +%% @hidden +getHandle(This) -> wxWindow:getHandle(This). +%% @hidden +getGrandParent(This) -> wxWindow:getGrandParent(This). +%% @hidden +getForegroundColour(This) -> wxWindow:getForegroundColour(This). +%% @hidden +getFont(This) -> wxWindow:getFont(This). +%% @hidden +getExtraStyle(This) -> wxWindow:getExtraStyle(This). +%% @hidden +getEventHandler(This) -> wxWindow:getEventHandler(This). +%% @hidden +getDropTarget(This) -> wxWindow:getDropTarget(This). +%% @hidden +getCursor(This) -> wxWindow:getCursor(This). +%% @hidden +getContainingSizer(This) -> wxWindow:getContainingSizer(This). +%% @hidden +getClientSize(This) -> wxWindow:getClientSize(This). +%% @hidden +getChildren(This) -> wxWindow:getChildren(This). +%% @hidden +getCharWidth(This) -> wxWindow:getCharWidth(This). +%% @hidden +getCharHeight(This) -> wxWindow:getCharHeight(This). +%% @hidden +getCaret(This) -> wxWindow:getCaret(This). +%% @hidden +getBestSize(This) -> wxWindow:getBestSize(This). +%% @hidden +getBackgroundStyle(This) -> wxWindow:getBackgroundStyle(This). +%% @hidden +getBackgroundColour(This) -> wxWindow:getBackgroundColour(This). +%% @hidden +getAcceleratorTable(This) -> wxWindow:getAcceleratorTable(This). +%% @hidden +freeze(This) -> wxWindow:freeze(This). +%% @hidden +fitInside(This) -> wxWindow:fitInside(This). +%% @hidden +fit(This) -> wxWindow:fit(This). +%% @hidden +findWindow(This,Winid) -> wxWindow:findWindow(This,Winid). +%% @hidden +enable(This, Options) -> wxWindow:enable(This, Options). +%% @hidden +enable(This) -> wxWindow:enable(This). +%% @hidden +disable(This) -> wxWindow:disable(This). +%% @hidden +destroyChildren(This) -> wxWindow:destroyChildren(This). +%% @hidden +convertPixelsToDialog(This,Sz) -> wxWindow:convertPixelsToDialog(This,Sz). +%% @hidden +convertDialogToPixels(This,Sz) -> wxWindow:convertDialogToPixels(This,Sz). +%% @hidden +close(This, Options) -> wxWindow:close(This, Options). +%% @hidden +close(This) -> wxWindow:close(This). +%% @hidden +clientToScreen(This,X,Y) -> wxWindow:clientToScreen(This,X,Y). +%% @hidden +clientToScreen(This,Pt) -> wxWindow:clientToScreen(This,Pt). +%% @hidden +clearBackground(This) -> wxWindow:clearBackground(This). +%% @hidden +centreOnParent(This, Options) -> wxWindow:centreOnParent(This, Options). +%% @hidden +centreOnParent(This) -> wxWindow:centreOnParent(This). +%% @hidden +centre(This, Options) -> wxWindow:centre(This, Options). +%% @hidden +centre(This) -> wxWindow:centre(This). +%% @hidden +centerOnParent(This, Options) -> wxWindow:centerOnParent(This, Options). +%% @hidden +centerOnParent(This) -> wxWindow:centerOnParent(This). +%% @hidden +center(This, Options) -> wxWindow:center(This, Options). +%% @hidden +center(This) -> wxWindow:center(This). +%% @hidden +captureMouse(This) -> wxWindow:captureMouse(This). +%% @hidden +cacheBestSize(This,Size) -> wxWindow:cacheBestSize(This,Size). + %% From wxEvtHandler +%% @hidden +disconnect(This,EventType, Options) -> wxEvtHandler:disconnect(This,EventType, Options). +%% @hidden +disconnect(This,EventType) -> wxEvtHandler:disconnect(This,EventType). +%% @hidden +disconnect(This) -> wxEvtHandler:disconnect(This). +%% @hidden +connect(This,EventType, Options) -> wxEvtHandler:connect(This,EventType, Options). +%% @hidden +connect(This,EventType) -> wxEvtHandler:connect(This,EventType). diff --git a/lib/wx/src/gen/wxTextCtrl.erl b/lib/wx/src/gen/wxTextCtrl.erl index cb85652ceb..7e6852b9c8 100644 --- a/lib/wx/src/gen/wxTextCtrl.erl +++ b/lib/wx/src/gen/wxTextCtrl.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2013. All Rights Reserved. +%% Copyright Ericsson AB 2008-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -29,8 +29,8 @@ -module(wxTextCtrl). -include("wxe.hrl"). --export([appendText/2,canCopy/1,canCut/1,canPaste/1,canRedo/1,canUndo/1,clear/1, - copy/1,create/3,create/4,cut/1,destroy/1,discardEdits/1,emulateKeyPress/2, +-export([appendText/2,canCopy/1,canCut/1,canPaste/1,canRedo/1,canUndo/1,changeValue/2, + clear/1,copy/1,create/3,create/4,cut/1,destroy/1,discardEdits/1,emulateKeyPress/2, getDefaultStyle/1,getInsertionPoint/1,getLastPosition/1,getLineLength/2, getLineText/2,getNumberOfLines/1,getRange/3,getSelection/1,getStringSelection/1, getStyle/3,getValue/1,isEditable/1,isModified/1,isMultiLine/1,isSingleLine/1, @@ -232,6 +232,16 @@ discardEdits(#wx_ref{type=ThisT,ref=ThisRef}) -> wxe_util:cast(?wxTextCtrl_DiscardEdits, <<ThisRef:32/?UI>>). +%% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxtextctrl.html#wxtextctrlchangevalue">external documentation</a>. +-spec changeValue(This, Value) -> ok when + This::wxTextCtrl(), Value::unicode:chardata(). +changeValue(#wx_ref{type=ThisT,ref=ThisRef},Value) + when is_list(Value) -> + ?CLASS(ThisT,wxTextCtrl), + Value_UC = unicode:characters_to_binary([Value,0]), + wxe_util:cast(?wxTextCtrl_ChangeValue, + <<ThisRef:32/?UI,(byte_size(Value_UC)):32/?UI,(Value_UC)/binary, 0:(((8- ((0+byte_size(Value_UC)) band 16#7)) band 16#7))/unit:8>>). + %% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_wxtextctrl.html#wxtextctrlemulatekeypress">external documentation</a>. -spec emulateKeyPress(This, Event) -> boolean() when This::wxTextCtrl(), Event::wxKeyEvent:wxKeyEvent(). diff --git a/lib/wx/src/gen/wx_misc.erl b/lib/wx/src/gen/wx_misc.erl index 96912ce651..397d48b727 100644 --- a/lib/wx/src/gen/wx_misc.erl +++ b/lib/wx/src/gen/wx_misc.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2013. All Rights Reserved. +%% Copyright Ericsson AB 2008-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -30,6 +30,8 @@ launchDefaultBrowser/2,newId/0,registerId/1,setDetectableAutoRepeat/1, shell/0,shell/1,shutdown/1]). +-export([displaySize/0,setCursor/1]). + %% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_miscellany.html#wxgetkeystate">external documentation</a>. %%<br /> Key = ?WXK_BACK | ?WXK_TAB | ?WXK_RETURN | ?WXK_ESCAPE | ?WXK_SPACE | ?WXK_DELETE | ?WXK_START | ?WXK_LBUTTON | ?WXK_RBUTTON | ?WXK_CANCEL | ?WXK_MBUTTON | ?WXK_CLEAR | ?WXK_SHIFT | ?WXK_ALT | ?WXK_CONTROL | ?WXK_MENU | ?WXK_PAUSE | ?WXK_CAPITAL | ?WXK_END | ?WXK_HOME | ?WXK_LEFT | ?WXK_UP | ?WXK_RIGHT | ?WXK_DOWN | ?WXK_SELECT | ?WXK_PRINT | ?WXK_EXECUTE | ?WXK_SNAPSHOT | ?WXK_INSERT | ?WXK_HELP | ?WXK_NUMPAD0 | ?WXK_NUMPAD1 | ?WXK_NUMPAD2 | ?WXK_NUMPAD3 | ?WXK_NUMPAD4 | ?WXK_NUMPAD5 | ?WXK_NUMPAD6 | ?WXK_NUMPAD7 | ?WXK_NUMPAD8 | ?WXK_NUMPAD9 | ?WXK_MULTIPLY | ?WXK_ADD | ?WXK_SEPARATOR | ?WXK_SUBTRACT | ?WXK_DECIMAL | ?WXK_DIVIDE | ?WXK_F1 | ?WXK_F2 | ?WXK_F3 | ?WXK_F4 | ?WXK_F5 | ?WXK_F6 | ?WXK_F7 | ?WXK_F8 | ?WXK_F9 | ?WXK_F10 | ?WXK_F11 | ?WXK_F12 | ?WXK_F13 | ?WXK_F14 | ?WXK_F15 | ?WXK_F16 | ?WXK_F17 | ?WXK_F18 | ?WXK_F19 | ?WXK_F20 | ?WXK_F21 | ?WXK_F22 | ?WXK_F23 | ?WXK_F24 | ?WXK_NUMLOCK | ?WXK_SCROLL | ?WXK_PAGEUP | ?WXK_PAGEDOWN | ?WXK_NUMPAD_SPACE | ?WXK_NUMPAD_TAB | ?WXK_NUMPAD_ENTER | ?WXK_NUMPAD_F1 | ?WXK_NUMPAD_F2 | ?WXK_NUMPAD_F3 | ?WXK_NUMPAD_F4 | ?WXK_NUMPAD_HOME | ?WXK_NUMPAD_LEFT | ?WXK_NUMPAD_UP | ?WXK_NUMPAD_RIGHT | ?WXK_NUMPAD_DOWN | ?WXK_NUMPAD_PAGEUP | ?WXK_NUMPAD_PAGEDOWN | ?WXK_NUMPAD_END | ?WXK_NUMPAD_BEGIN | ?WXK_NUMPAD_INSERT | ?WXK_NUMPAD_DELETE | ?WXK_NUMPAD_EQUAL | ?WXK_NUMPAD_MULTIPLY | ?WXK_NUMPAD_ADD | ?WXK_NUMPAD_SEPARATOR | ?WXK_NUMPAD_SUBTRACT | ?WXK_NUMPAD_DECIMAL | ?WXK_NUMPAD_DIVIDE | ?WXK_WINDOWS_LEFT | ?WXK_WINDOWS_RIGHT | ?WXK_WINDOWS_MENU | ?WXK_COMMAND | ?WXK_SPECIAL1 | ?WXK_SPECIAL2 | ?WXK_SPECIAL3 | ?WXK_SPECIAL4 | ?WXK_SPECIAL5 | ?WXK_SPECIAL6 | ?WXK_SPECIAL7 | ?WXK_SPECIAL8 | ?WXK_SPECIAL9 | ?WXK_SPECIAL10 | ?WXK_SPECIAL11 | ?WXK_SPECIAL12 | ?WXK_SPECIAL13 | ?WXK_SPECIAL14 | ?WXK_SPECIAL15 | ?WXK_SPECIAL16 | ?WXK_SPECIAL17 | ?WXK_SPECIAL18 | ?WXK_SPECIAL19 | ?WXK_SPECIAL20 -spec getKeyState(Key) -> boolean() when @@ -224,3 +226,17 @@ isPlatform64Bit() -> wxe_util:call(?utils_wxIsPlatform64Bit, <<>>). +%% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_gdicmn.html#gdicmnwxdisplaysize">external documentation</a>. +-spec displaySize() -> {Width::integer(), Height::integer()}. +displaySize() -> + wxe_util:call(?gdicmn_wxDisplaySize, + <<>>). + +%% @doc See <a href="http://www.wxwidgets.org/manuals/2.8.12/wx_gdicmn.html#gdicmnwxsetcursor">external documentation</a>. +-spec setCursor(Cursor) -> ok when + Cursor::wxCursor:wxCursor(). +setCursor(#wx_ref{type=CursorT,ref=CursorRef}) -> + ?CLASS(CursorT,wxCursor), + wxe_util:cast(?gdicmn_wxSetCursor, + <<CursorRef:32/?UI>>). + diff --git a/lib/wx/src/gen/wxe_debug.hrl b/lib/wx/src/gen/wxe_debug.hrl index 1f4f22f704..7d4db9ba97 100644 --- a/lib/wx/src/gen/wxe_debug.hrl +++ b/lib/wx/src/gen/wxe_debug.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2013. All Rights Reserved. +%% Copyright Ericsson AB 2008-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -1669,1642 +1669,1677 @@ wxdebug_table() -> {1824, {wxTextCtrl, create, 3}}, {1825, {wxTextCtrl, cut, 0}}, {1826, {wxTextCtrl, discardEdits, 0}}, - {1827, {wxTextCtrl, emulateKeyPress, 1}}, - {1828, {wxTextCtrl, getDefaultStyle, 0}}, - {1829, {wxTextCtrl, getInsertionPoint, 0}}, - {1830, {wxTextCtrl, getLastPosition, 0}}, - {1831, {wxTextCtrl, getLineLength, 1}}, - {1832, {wxTextCtrl, getLineText, 1}}, - {1833, {wxTextCtrl, getNumberOfLines, 0}}, - {1834, {wxTextCtrl, getRange, 2}}, - {1835, {wxTextCtrl, getSelection, 2}}, - {1836, {wxTextCtrl, getStringSelection, 0}}, - {1837, {wxTextCtrl, getStyle, 2}}, - {1838, {wxTextCtrl, getValue, 0}}, - {1839, {wxTextCtrl, isEditable, 0}}, - {1840, {wxTextCtrl, isModified, 0}}, - {1841, {wxTextCtrl, isMultiLine, 0}}, - {1842, {wxTextCtrl, isSingleLine, 0}}, - {1843, {wxTextCtrl, loadFile, 2}}, - {1844, {wxTextCtrl, markDirty, 0}}, - {1845, {wxTextCtrl, paste, 0}}, - {1846, {wxTextCtrl, positionToXY, 3}}, - {1847, {wxTextCtrl, redo, 0}}, - {1848, {wxTextCtrl, remove, 2}}, - {1849, {wxTextCtrl, replace, 3}}, - {1850, {wxTextCtrl, saveFile, 1}}, - {1851, {wxTextCtrl, setDefaultStyle, 1}}, - {1852, {wxTextCtrl, setEditable, 1}}, - {1853, {wxTextCtrl, setInsertionPoint, 1}}, - {1854, {wxTextCtrl, setInsertionPointEnd, 0}}, - {1856, {wxTextCtrl, setMaxLength, 1}}, - {1857, {wxTextCtrl, setSelection, 2}}, - {1858, {wxTextCtrl, setStyle, 3}}, - {1859, {wxTextCtrl, setValue, 1}}, - {1860, {wxTextCtrl, showPosition, 1}}, - {1861, {wxTextCtrl, undo, 0}}, - {1862, {wxTextCtrl, writeText, 1}}, - {1863, {wxTextCtrl, xYToPosition, 2}}, - {1866, {wxNotebook, new_0, 0}}, - {1867, {wxNotebook, new_3, 3}}, - {1868, {wxNotebook, destruct, 0}}, - {1869, {wxNotebook, addPage, 3}}, - {1870, {wxNotebook, advanceSelection, 1}}, - {1871, {wxNotebook, assignImageList, 1}}, - {1872, {wxNotebook, create, 3}}, - {1873, {wxNotebook, deleteAllPages, 0}}, - {1874, {wxNotebook, deletePage, 1}}, - {1875, {wxNotebook, removePage, 1}}, - {1876, {wxNotebook, getCurrentPage, 0}}, - {1877, {wxNotebook, getImageList, 0}}, - {1879, {wxNotebook, getPage, 1}}, - {1880, {wxNotebook, getPageCount, 0}}, - {1881, {wxNotebook, getPageImage, 1}}, - {1882, {wxNotebook, getPageText, 1}}, - {1883, {wxNotebook, getRowCount, 0}}, - {1884, {wxNotebook, getSelection, 0}}, - {1885, {wxNotebook, getThemeBackgroundColour, 0}}, - {1887, {wxNotebook, hitTest, 2}}, - {1889, {wxNotebook, insertPage, 4}}, - {1890, {wxNotebook, setImageList, 1}}, - {1891, {wxNotebook, setPadding, 1}}, - {1892, {wxNotebook, setPageSize, 1}}, - {1893, {wxNotebook, setPageImage, 2}}, - {1894, {wxNotebook, setPageText, 2}}, - {1895, {wxNotebook, setSelection, 1}}, - {1896, {wxNotebook, changeSelection, 1}}, - {1897, {wxChoicebook, new_0, 0}}, - {1898, {wxChoicebook, new_3, 3}}, - {1899, {wxChoicebook, addPage, 3}}, - {1900, {wxChoicebook, advanceSelection, 1}}, - {1901, {wxChoicebook, assignImageList, 1}}, - {1902, {wxChoicebook, create, 3}}, - {1903, {wxChoicebook, deleteAllPages, 0}}, - {1904, {wxChoicebook, deletePage, 1}}, - {1905, {wxChoicebook, removePage, 1}}, - {1906, {wxChoicebook, getCurrentPage, 0}}, - {1907, {wxChoicebook, getImageList, 0}}, - {1909, {wxChoicebook, getPage, 1}}, - {1910, {wxChoicebook, getPageCount, 0}}, - {1911, {wxChoicebook, getPageImage, 1}}, - {1912, {wxChoicebook, getPageText, 1}}, - {1913, {wxChoicebook, getSelection, 0}}, - {1914, {wxChoicebook, hitTest, 2}}, - {1915, {wxChoicebook, insertPage, 4}}, - {1916, {wxChoicebook, setImageList, 1}}, - {1917, {wxChoicebook, setPageSize, 1}}, - {1918, {wxChoicebook, setPageImage, 2}}, - {1919, {wxChoicebook, setPageText, 2}}, - {1920, {wxChoicebook, setSelection, 1}}, - {1921, {wxChoicebook, changeSelection, 1}}, - {1922, {wxChoicebook, 'Destroy', undefined}}, - {1923, {wxToolbook, new_0, 0}}, - {1924, {wxToolbook, new_3, 3}}, - {1925, {wxToolbook, addPage, 3}}, - {1926, {wxToolbook, advanceSelection, 1}}, - {1927, {wxToolbook, assignImageList, 1}}, - {1928, {wxToolbook, create, 3}}, - {1929, {wxToolbook, deleteAllPages, 0}}, - {1930, {wxToolbook, deletePage, 1}}, - {1931, {wxToolbook, removePage, 1}}, - {1932, {wxToolbook, getCurrentPage, 0}}, - {1933, {wxToolbook, getImageList, 0}}, - {1935, {wxToolbook, getPage, 1}}, - {1936, {wxToolbook, getPageCount, 0}}, - {1937, {wxToolbook, getPageImage, 1}}, - {1938, {wxToolbook, getPageText, 1}}, - {1939, {wxToolbook, getSelection, 0}}, - {1941, {wxToolbook, hitTest, 2}}, - {1942, {wxToolbook, insertPage, 4}}, - {1943, {wxToolbook, setImageList, 1}}, - {1944, {wxToolbook, setPageSize, 1}}, - {1945, {wxToolbook, setPageImage, 2}}, - {1946, {wxToolbook, setPageText, 2}}, - {1947, {wxToolbook, setSelection, 1}}, - {1948, {wxToolbook, changeSelection, 1}}, - {1949, {wxToolbook, 'Destroy', undefined}}, - {1950, {wxListbook, new_0, 0}}, - {1951, {wxListbook, new_3, 3}}, - {1952, {wxListbook, addPage, 3}}, - {1953, {wxListbook, advanceSelection, 1}}, - {1954, {wxListbook, assignImageList, 1}}, - {1955, {wxListbook, create, 3}}, - {1956, {wxListbook, deleteAllPages, 0}}, - {1957, {wxListbook, deletePage, 1}}, - {1958, {wxListbook, removePage, 1}}, - {1959, {wxListbook, getCurrentPage, 0}}, - {1960, {wxListbook, getImageList, 0}}, - {1962, {wxListbook, getPage, 1}}, - {1963, {wxListbook, getPageCount, 0}}, - {1964, {wxListbook, getPageImage, 1}}, - {1965, {wxListbook, getPageText, 1}}, - {1966, {wxListbook, getSelection, 0}}, - {1968, {wxListbook, hitTest, 2}}, - {1969, {wxListbook, insertPage, 4}}, - {1970, {wxListbook, setImageList, 1}}, - {1971, {wxListbook, setPageSize, 1}}, - {1972, {wxListbook, setPageImage, 2}}, - {1973, {wxListbook, setPageText, 2}}, - {1974, {wxListbook, setSelection, 1}}, - {1975, {wxListbook, changeSelection, 1}}, - {1976, {wxListbook, 'Destroy', undefined}}, - {1977, {wxTreebook, new_0, 0}}, - {1978, {wxTreebook, new_3, 3}}, - {1979, {wxTreebook, addPage, 3}}, - {1980, {wxTreebook, advanceSelection, 1}}, - {1981, {wxTreebook, assignImageList, 1}}, - {1982, {wxTreebook, create, 3}}, - {1983, {wxTreebook, deleteAllPages, 0}}, - {1984, {wxTreebook, deletePage, 1}}, - {1985, {wxTreebook, removePage, 1}}, - {1986, {wxTreebook, getCurrentPage, 0}}, - {1987, {wxTreebook, getImageList, 0}}, - {1989, {wxTreebook, getPage, 1}}, - {1990, {wxTreebook, getPageCount, 0}}, - {1991, {wxTreebook, getPageImage, 1}}, - {1992, {wxTreebook, getPageText, 1}}, - {1993, {wxTreebook, getSelection, 0}}, - {1994, {wxTreebook, expandNode, 2}}, - {1995, {wxTreebook, isNodeExpanded, 1}}, - {1997, {wxTreebook, hitTest, 2}}, - {1998, {wxTreebook, insertPage, 4}}, - {1999, {wxTreebook, insertSubPage, 4}}, - {2000, {wxTreebook, setImageList, 1}}, - {2001, {wxTreebook, setPageSize, 1}}, - {2002, {wxTreebook, setPageImage, 2}}, - {2003, {wxTreebook, setPageText, 2}}, - {2004, {wxTreebook, setSelection, 1}}, - {2005, {wxTreebook, changeSelection, 1}}, - {2006, {wxTreebook, 'Destroy', undefined}}, - {2009, {wxTreeCtrl, new_2, 2}}, - {2010, {wxTreeCtrl, new_0, 0}}, - {2012, {wxTreeCtrl, destruct, 0}}, - {2013, {wxTreeCtrl, addRoot, 2}}, - {2014, {wxTreeCtrl, appendItem, 3}}, - {2015, {wxTreeCtrl, assignImageList, 1}}, - {2016, {wxTreeCtrl, assignStateImageList, 1}}, - {2017, {wxTreeCtrl, collapse, 1}}, - {2018, {wxTreeCtrl, collapseAndReset, 1}}, - {2019, {wxTreeCtrl, create, 2}}, - {2020, {wxTreeCtrl, delete, 1}}, - {2021, {wxTreeCtrl, deleteAllItems, 0}}, - {2022, {wxTreeCtrl, deleteChildren, 1}}, - {2023, {wxTreeCtrl, editLabel, 1}}, - {2024, {wxTreeCtrl, ensureVisible, 1}}, - {2025, {wxTreeCtrl, expand, 1}}, - {2026, {wxTreeCtrl, getBoundingRect, 3}}, - {2028, {wxTreeCtrl, getChildrenCount, 2}}, - {2029, {wxTreeCtrl, getCount, 0}}, - {2030, {wxTreeCtrl, getEditControl, 0}}, - {2031, {wxTreeCtrl, getFirstChild, 2}}, - {2032, {wxTreeCtrl, getNextChild, 2}}, - {2033, {wxTreeCtrl, getFirstVisibleItem, 0}}, - {2034, {wxTreeCtrl, getImageList, 0}}, - {2035, {wxTreeCtrl, getIndent, 0}}, - {2036, {wxTreeCtrl, getItemBackgroundColour, 1}}, - {2037, {wxTreeCtrl, getItemData, 1}}, - {2038, {wxTreeCtrl, getItemFont, 1}}, - {2039, {wxTreeCtrl, getItemImage_1, 1}}, - {2040, {wxTreeCtrl, getItemImage_2, 2}}, - {2041, {wxTreeCtrl, getItemText, 1}}, - {2042, {wxTreeCtrl, getItemTextColour, 1}}, - {2043, {wxTreeCtrl, getLastChild, 1}}, - {2044, {wxTreeCtrl, getNextSibling, 1}}, - {2045, {wxTreeCtrl, getNextVisible, 1}}, - {2046, {wxTreeCtrl, getItemParent, 1}}, - {2047, {wxTreeCtrl, getPrevSibling, 1}}, - {2048, {wxTreeCtrl, getPrevVisible, 1}}, - {2049, {wxTreeCtrl, getRootItem, 0}}, - {2050, {wxTreeCtrl, getSelection, 0}}, - {2051, {wxTreeCtrl, getSelections, 1}}, - {2052, {wxTreeCtrl, getStateImageList, 0}}, - {2053, {wxTreeCtrl, hitTest, 2}}, - {2055, {wxTreeCtrl, insertItem, 4}}, - {2056, {wxTreeCtrl, isBold, 1}}, - {2057, {wxTreeCtrl, isExpanded, 1}}, - {2058, {wxTreeCtrl, isSelected, 1}}, - {2059, {wxTreeCtrl, isVisible, 1}}, - {2060, {wxTreeCtrl, itemHasChildren, 1}}, - {2061, {wxTreeCtrl, isTreeItemIdOk, 1}}, - {2062, {wxTreeCtrl, prependItem, 3}}, - {2063, {wxTreeCtrl, scrollTo, 1}}, - {2064, {wxTreeCtrl, selectItem_1, 1}}, - {2065, {wxTreeCtrl, selectItem_2, 2}}, - {2066, {wxTreeCtrl, setIndent, 1}}, - {2067, {wxTreeCtrl, setImageList, 1}}, - {2068, {wxTreeCtrl, setItemBackgroundColour, 2}}, - {2069, {wxTreeCtrl, setItemBold, 2}}, - {2070, {wxTreeCtrl, setItemData, 2}}, - {2071, {wxTreeCtrl, setItemDropHighlight, 2}}, - {2072, {wxTreeCtrl, setItemFont, 2}}, - {2073, {wxTreeCtrl, setItemHasChildren, 2}}, - {2074, {wxTreeCtrl, setItemImage_2, 2}}, - {2075, {wxTreeCtrl, setItemImage_3, 3}}, - {2076, {wxTreeCtrl, setItemText, 2}}, - {2077, {wxTreeCtrl, setItemTextColour, 2}}, - {2078, {wxTreeCtrl, setStateImageList, 1}}, - {2079, {wxTreeCtrl, setWindowStyle, 1}}, - {2080, {wxTreeCtrl, sortChildren, 1}}, - {2081, {wxTreeCtrl, toggle, 1}}, - {2082, {wxTreeCtrl, toggleItemSelection, 1}}, - {2083, {wxTreeCtrl, unselect, 0}}, - {2084, {wxTreeCtrl, unselectAll, 0}}, - {2085, {wxTreeCtrl, unselectItem, 1}}, - {2086, {wxScrollBar, new_0, 0}}, - {2087, {wxScrollBar, new_3, 3}}, - {2088, {wxScrollBar, destruct, 0}}, - {2089, {wxScrollBar, create, 3}}, - {2090, {wxScrollBar, getRange, 0}}, - {2091, {wxScrollBar, getPageSize, 0}}, - {2092, {wxScrollBar, getThumbPosition, 0}}, - {2093, {wxScrollBar, getThumbSize, 0}}, - {2094, {wxScrollBar, setThumbPosition, 1}}, - {2095, {wxScrollBar, setScrollbar, 5}}, - {2097, {wxSpinButton, new_2, 2}}, - {2098, {wxSpinButton, new_0, 0}}, - {2099, {wxSpinButton, create, 2}}, - {2100, {wxSpinButton, getMax, 0}}, - {2101, {wxSpinButton, getMin, 0}}, - {2102, {wxSpinButton, getValue, 0}}, - {2103, {wxSpinButton, setRange, 2}}, - {2104, {wxSpinButton, setValue, 1}}, - {2105, {wxSpinButton, 'Destroy', undefined}}, - {2106, {wxSpinCtrl, new_0, 0}}, - {2107, {wxSpinCtrl, new_2, 2}}, - {2109, {wxSpinCtrl, create, 2}}, - {2112, {wxSpinCtrl, setValue_1_1, 1}}, - {2113, {wxSpinCtrl, setValue_1_0, 1}}, - {2115, {wxSpinCtrl, getValue, 0}}, - {2117, {wxSpinCtrl, setRange, 2}}, - {2118, {wxSpinCtrl, setSelection, 2}}, - {2120, {wxSpinCtrl, getMin, 0}}, - {2122, {wxSpinCtrl, getMax, 0}}, - {2123, {wxSpinCtrl, 'Destroy', undefined}}, - {2124, {wxStaticText, new_0, 0}}, - {2125, {wxStaticText, new_4, 4}}, - {2126, {wxStaticText, create, 4}}, - {2127, {wxStaticText, getLabel, 0}}, - {2128, {wxStaticText, setLabel, 1}}, - {2129, {wxStaticText, wrap, 1}}, - {2130, {wxStaticText, 'Destroy', undefined}}, - {2131, {wxStaticBitmap, new_0, 0}}, - {2132, {wxStaticBitmap, new_4, 4}}, - {2133, {wxStaticBitmap, create, 4}}, - {2134, {wxStaticBitmap, getBitmap, 0}}, - {2135, {wxStaticBitmap, setBitmap, 1}}, - {2136, {wxStaticBitmap, 'Destroy', undefined}}, - {2137, {wxRadioBox, new, 7}}, - {2139, {wxRadioBox, destruct, 0}}, - {2140, {wxRadioBox, create, 7}}, - {2141, {wxRadioBox, enable_2, 2}}, - {2142, {wxRadioBox, enable_1, 1}}, - {2143, {wxRadioBox, getSelection, 0}}, - {2144, {wxRadioBox, getString, 1}}, - {2145, {wxRadioBox, setSelection, 1}}, - {2146, {wxRadioBox, show_2, 2}}, - {2147, {wxRadioBox, show_1, 1}}, - {2148, {wxRadioBox, getColumnCount, 0}}, - {2149, {wxRadioBox, getItemHelpText, 1}}, - {2150, {wxRadioBox, getItemToolTip, 1}}, - {2152, {wxRadioBox, getItemFromPoint, 1}}, - {2153, {wxRadioBox, getRowCount, 0}}, - {2154, {wxRadioBox, isItemEnabled, 1}}, - {2155, {wxRadioBox, isItemShown, 1}}, - {2156, {wxRadioBox, setItemHelpText, 2}}, - {2157, {wxRadioBox, setItemToolTip, 2}}, - {2158, {wxRadioButton, new_0, 0}}, - {2159, {wxRadioButton, new_4, 4}}, - {2160, {wxRadioButton, create, 4}}, - {2161, {wxRadioButton, getValue, 0}}, - {2162, {wxRadioButton, setValue, 1}}, - {2163, {wxRadioButton, 'Destroy', undefined}}, - {2165, {wxSlider, new_6, 6}}, - {2166, {wxSlider, new_0, 0}}, - {2167, {wxSlider, create, 6}}, - {2168, {wxSlider, getLineSize, 0}}, - {2169, {wxSlider, getMax, 0}}, - {2170, {wxSlider, getMin, 0}}, - {2171, {wxSlider, getPageSize, 0}}, - {2172, {wxSlider, getThumbLength, 0}}, - {2173, {wxSlider, getValue, 0}}, - {2174, {wxSlider, setLineSize, 1}}, - {2175, {wxSlider, setPageSize, 1}}, - {2176, {wxSlider, setRange, 2}}, - {2177, {wxSlider, setThumbLength, 1}}, - {2178, {wxSlider, setValue, 1}}, - {2179, {wxSlider, 'Destroy', undefined}}, - {2181, {wxDialog, new_4, 4}}, - {2182, {wxDialog, new_0, 0}}, - {2184, {wxDialog, destruct, 0}}, - {2185, {wxDialog, create, 4}}, - {2186, {wxDialog, createButtonSizer, 1}}, - {2187, {wxDialog, createStdDialogButtonSizer, 1}}, - {2188, {wxDialog, endModal, 1}}, - {2189, {wxDialog, getAffirmativeId, 0}}, - {2190, {wxDialog, getReturnCode, 0}}, - {2191, {wxDialog, isModal, 0}}, - {2192, {wxDialog, setAffirmativeId, 1}}, - {2193, {wxDialog, setReturnCode, 1}}, - {2194, {wxDialog, show, 1}}, - {2195, {wxDialog, showModal, 0}}, - {2196, {wxColourDialog, new_0, 0}}, - {2197, {wxColourDialog, new_2, 2}}, - {2198, {wxColourDialog, destruct, 0}}, - {2199, {wxColourDialog, create, 2}}, - {2200, {wxColourDialog, getColourData, 0}}, - {2201, {wxColourData, new_0, 0}}, - {2202, {wxColourData, new_1, 1}}, - {2203, {wxColourData, destruct, 0}}, - {2204, {wxColourData, getChooseFull, 0}}, - {2205, {wxColourData, getColour, 0}}, - {2207, {wxColourData, getCustomColour, 1}}, - {2208, {wxColourData, setChooseFull, 1}}, - {2209, {wxColourData, setColour, 1}}, - {2210, {wxColourData, setCustomColour, 2}}, - {2211, {wxPalette, new_0, 0}}, - {2212, {wxPalette, new_4, 4}}, - {2214, {wxPalette, destruct, 0}}, - {2215, {wxPalette, create, 4}}, - {2216, {wxPalette, getColoursCount, 0}}, - {2217, {wxPalette, getPixel, 3}}, - {2218, {wxPalette, getRGB, 4}}, - {2219, {wxPalette, isOk, 0}}, - {2223, {wxDirDialog, new, 2}}, - {2224, {wxDirDialog, destruct, 0}}, - {2225, {wxDirDialog, getPath, 0}}, - {2226, {wxDirDialog, getMessage, 0}}, - {2227, {wxDirDialog, setMessage, 1}}, - {2228, {wxDirDialog, setPath, 1}}, - {2232, {wxFileDialog, new, 2}}, - {2233, {wxFileDialog, destruct, 0}}, - {2234, {wxFileDialog, getDirectory, 0}}, - {2235, {wxFileDialog, getFilename, 0}}, - {2236, {wxFileDialog, getFilenames, 1}}, - {2237, {wxFileDialog, getFilterIndex, 0}}, - {2238, {wxFileDialog, getMessage, 0}}, - {2239, {wxFileDialog, getPath, 0}}, - {2240, {wxFileDialog, getPaths, 1}}, - {2241, {wxFileDialog, getWildcard, 0}}, - {2242, {wxFileDialog, setDirectory, 1}}, - {2243, {wxFileDialog, setFilename, 1}}, - {2244, {wxFileDialog, setFilterIndex, 1}}, - {2245, {wxFileDialog, setMessage, 1}}, - {2246, {wxFileDialog, setPath, 1}}, - {2247, {wxFileDialog, setWildcard, 1}}, - {2248, {wxPickerBase, setInternalMargin, 1}}, - {2249, {wxPickerBase, getInternalMargin, 0}}, - {2250, {wxPickerBase, setTextCtrlProportion, 1}}, - {2251, {wxPickerBase, setPickerCtrlProportion, 1}}, - {2252, {wxPickerBase, getTextCtrlProportion, 0}}, - {2253, {wxPickerBase, getPickerCtrlProportion, 0}}, - {2254, {wxPickerBase, hasTextCtrl, 0}}, - {2255, {wxPickerBase, getTextCtrl, 0}}, - {2256, {wxPickerBase, isTextCtrlGrowable, 0}}, - {2257, {wxPickerBase, setPickerCtrlGrowable, 1}}, - {2258, {wxPickerBase, setTextCtrlGrowable, 1}}, - {2259, {wxPickerBase, isPickerCtrlGrowable, 0}}, - {2260, {wxFilePickerCtrl, new_0, 0}}, - {2261, {wxFilePickerCtrl, new_3, 3}}, - {2262, {wxFilePickerCtrl, create, 3}}, - {2263, {wxFilePickerCtrl, getPath, 0}}, - {2264, {wxFilePickerCtrl, setPath, 1}}, - {2265, {wxFilePickerCtrl, 'Destroy', undefined}}, - {2266, {wxDirPickerCtrl, new_0, 0}}, - {2267, {wxDirPickerCtrl, new_3, 3}}, - {2268, {wxDirPickerCtrl, create, 3}}, - {2269, {wxDirPickerCtrl, getPath, 0}}, - {2270, {wxDirPickerCtrl, setPath, 1}}, - {2271, {wxDirPickerCtrl, 'Destroy', undefined}}, - {2272, {wxColourPickerCtrl, new_0, 0}}, - {2273, {wxColourPickerCtrl, new_3, 3}}, - {2274, {wxColourPickerCtrl, create, 3}}, - {2275, {wxColourPickerCtrl, getColour, 0}}, - {2276, {wxColourPickerCtrl, setColour_1_1, 1}}, - {2277, {wxColourPickerCtrl, setColour_1_0, 1}}, - {2278, {wxColourPickerCtrl, 'Destroy', undefined}}, - {2279, {wxDatePickerCtrl, new_0, 0}}, - {2280, {wxDatePickerCtrl, new_3, 3}}, - {2281, {wxDatePickerCtrl, getRange, 2}}, - {2282, {wxDatePickerCtrl, getValue, 0}}, - {2283, {wxDatePickerCtrl, setRange, 2}}, - {2284, {wxDatePickerCtrl, setValue, 1}}, - {2285, {wxDatePickerCtrl, 'Destroy', undefined}}, - {2286, {wxFontPickerCtrl, new_0, 0}}, - {2287, {wxFontPickerCtrl, new_3, 3}}, - {2288, {wxFontPickerCtrl, create, 3}}, - {2289, {wxFontPickerCtrl, getSelectedFont, 0}}, - {2290, {wxFontPickerCtrl, setSelectedFont, 1}}, - {2291, {wxFontPickerCtrl, getMaxPointSize, 0}}, - {2292, {wxFontPickerCtrl, setMaxPointSize, 1}}, - {2293, {wxFontPickerCtrl, 'Destroy', undefined}}, - {2296, {wxFindReplaceDialog, new_0, 0}}, - {2297, {wxFindReplaceDialog, new_4, 4}}, - {2298, {wxFindReplaceDialog, destruct, 0}}, - {2299, {wxFindReplaceDialog, create, 4}}, - {2300, {wxFindReplaceDialog, getData, 0}}, - {2301, {wxFindReplaceData, new_0, 0}}, - {2302, {wxFindReplaceData, new_1, 1}}, - {2303, {wxFindReplaceData, getFindString, 0}}, - {2304, {wxFindReplaceData, getReplaceString, 0}}, - {2305, {wxFindReplaceData, getFlags, 0}}, - {2306, {wxFindReplaceData, setFlags, 1}}, - {2307, {wxFindReplaceData, setFindString, 1}}, - {2308, {wxFindReplaceData, setReplaceString, 1}}, - {2309, {wxFindReplaceData, 'Destroy', undefined}}, - {2310, {wxMultiChoiceDialog, new_0, 0}}, - {2312, {wxMultiChoiceDialog, new_5, 5}}, - {2313, {wxMultiChoiceDialog, getSelections, 0}}, - {2314, {wxMultiChoiceDialog, setSelections, 1}}, - {2315, {wxMultiChoiceDialog, 'Destroy', undefined}}, - {2316, {wxSingleChoiceDialog, new_0, 0}}, - {2318, {wxSingleChoiceDialog, new_5, 5}}, - {2319, {wxSingleChoiceDialog, getSelection, 0}}, - {2320, {wxSingleChoiceDialog, getStringSelection, 0}}, - {2321, {wxSingleChoiceDialog, setSelection, 1}}, - {2322, {wxSingleChoiceDialog, 'Destroy', undefined}}, - {2323, {wxTextEntryDialog, new, 3}}, - {2324, {wxTextEntryDialog, getValue, 0}}, - {2325, {wxTextEntryDialog, setValue, 1}}, - {2326, {wxTextEntryDialog, 'Destroy', undefined}}, - {2327, {wxPasswordEntryDialog, new, 3}}, - {2328, {wxPasswordEntryDialog, 'Destroy', undefined}}, - {2329, {wxFontData, new_0, 0}}, - {2330, {wxFontData, new_1, 1}}, - {2331, {wxFontData, destruct, 0}}, - {2332, {wxFontData, enableEffects, 1}}, - {2333, {wxFontData, getAllowSymbols, 0}}, - {2334, {wxFontData, getColour, 0}}, - {2335, {wxFontData, getChosenFont, 0}}, - {2336, {wxFontData, getEnableEffects, 0}}, - {2337, {wxFontData, getInitialFont, 0}}, - {2338, {wxFontData, getShowHelp, 0}}, - {2339, {wxFontData, setAllowSymbols, 1}}, - {2340, {wxFontData, setChosenFont, 1}}, - {2341, {wxFontData, setColour, 1}}, - {2342, {wxFontData, setInitialFont, 1}}, - {2343, {wxFontData, setRange, 2}}, - {2344, {wxFontData, setShowHelp, 1}}, - {2348, {wxFontDialog, new_0, 0}}, - {2350, {wxFontDialog, new_2, 2}}, - {2352, {wxFontDialog, create, 2}}, - {2353, {wxFontDialog, getFontData, 0}}, - {2355, {wxFontDialog, 'Destroy', undefined}}, - {2356, {wxProgressDialog, new, 3}}, - {2357, {wxProgressDialog, destruct, 0}}, - {2358, {wxProgressDialog, resume, 0}}, - {2359, {wxProgressDialog, update_2, 2}}, - {2360, {wxProgressDialog, update_0, 0}}, - {2361, {wxMessageDialog, new, 3}}, - {2362, {wxMessageDialog, destruct, 0}}, - {2363, {wxPageSetupDialog, new, 2}}, - {2364, {wxPageSetupDialog, destruct, 0}}, - {2365, {wxPageSetupDialog, getPageSetupData, 0}}, - {2366, {wxPageSetupDialog, showModal, 0}}, - {2367, {wxPageSetupDialogData, new_0, 0}}, - {2368, {wxPageSetupDialogData, new_1_0, 1}}, - {2369, {wxPageSetupDialogData, new_1_1, 1}}, - {2370, {wxPageSetupDialogData, destruct, 0}}, - {2371, {wxPageSetupDialogData, enableHelp, 1}}, - {2372, {wxPageSetupDialogData, enableMargins, 1}}, - {2373, {wxPageSetupDialogData, enableOrientation, 1}}, - {2374, {wxPageSetupDialogData, enablePaper, 1}}, - {2375, {wxPageSetupDialogData, enablePrinter, 1}}, - {2376, {wxPageSetupDialogData, getDefaultMinMargins, 0}}, - {2377, {wxPageSetupDialogData, getEnableMargins, 0}}, - {2378, {wxPageSetupDialogData, getEnableOrientation, 0}}, - {2379, {wxPageSetupDialogData, getEnablePaper, 0}}, - {2380, {wxPageSetupDialogData, getEnablePrinter, 0}}, - {2381, {wxPageSetupDialogData, getEnableHelp, 0}}, - {2382, {wxPageSetupDialogData, getDefaultInfo, 0}}, - {2383, {wxPageSetupDialogData, getMarginTopLeft, 0}}, - {2384, {wxPageSetupDialogData, getMarginBottomRight, 0}}, - {2385, {wxPageSetupDialogData, getMinMarginTopLeft, 0}}, - {2386, {wxPageSetupDialogData, getMinMarginBottomRight, 0}}, - {2387, {wxPageSetupDialogData, getPaperId, 0}}, - {2388, {wxPageSetupDialogData, getPaperSize, 0}}, - {2390, {wxPageSetupDialogData, getPrintData, 0}}, - {2391, {wxPageSetupDialogData, isOk, 0}}, - {2392, {wxPageSetupDialogData, setDefaultInfo, 1}}, - {2393, {wxPageSetupDialogData, setDefaultMinMargins, 1}}, - {2394, {wxPageSetupDialogData, setMarginTopLeft, 1}}, - {2395, {wxPageSetupDialogData, setMarginBottomRight, 1}}, - {2396, {wxPageSetupDialogData, setMinMarginTopLeft, 1}}, - {2397, {wxPageSetupDialogData, setMinMarginBottomRight, 1}}, - {2398, {wxPageSetupDialogData, setPaperId, 1}}, - {2399, {wxPageSetupDialogData, setPaperSize_1_1, 1}}, - {2400, {wxPageSetupDialogData, setPaperSize_1_0, 1}}, - {2401, {wxPageSetupDialogData, setPrintData, 1}}, - {2402, {wxPrintDialog, new_2_0, 2}}, - {2403, {wxPrintDialog, new_2_1, 2}}, - {2404, {wxPrintDialog, destruct, 0}}, - {2405, {wxPrintDialog, getPrintDialogData, 0}}, - {2406, {wxPrintDialog, getPrintDC, 0}}, - {2407, {wxPrintDialogData, new_0, 0}}, - {2408, {wxPrintDialogData, new_1_1, 1}}, - {2409, {wxPrintDialogData, new_1_0, 1}}, - {2410, {wxPrintDialogData, destruct, 0}}, - {2411, {wxPrintDialogData, enableHelp, 1}}, - {2412, {wxPrintDialogData, enablePageNumbers, 1}}, - {2413, {wxPrintDialogData, enablePrintToFile, 1}}, - {2414, {wxPrintDialogData, enableSelection, 1}}, - {2415, {wxPrintDialogData, getAllPages, 0}}, - {2416, {wxPrintDialogData, getCollate, 0}}, - {2417, {wxPrintDialogData, getFromPage, 0}}, - {2418, {wxPrintDialogData, getMaxPage, 0}}, - {2419, {wxPrintDialogData, getMinPage, 0}}, - {2420, {wxPrintDialogData, getNoCopies, 0}}, - {2421, {wxPrintDialogData, getPrintData, 0}}, - {2422, {wxPrintDialogData, getPrintToFile, 0}}, - {2423, {wxPrintDialogData, getSelection, 0}}, - {2424, {wxPrintDialogData, getToPage, 0}}, - {2425, {wxPrintDialogData, isOk, 0}}, - {2426, {wxPrintDialogData, setCollate, 1}}, - {2427, {wxPrintDialogData, setFromPage, 1}}, - {2428, {wxPrintDialogData, setMaxPage, 1}}, - {2429, {wxPrintDialogData, setMinPage, 1}}, - {2430, {wxPrintDialogData, setNoCopies, 1}}, - {2431, {wxPrintDialogData, setPrintData, 1}}, - {2432, {wxPrintDialogData, setPrintToFile, 1}}, - {2433, {wxPrintDialogData, setSelection, 1}}, - {2434, {wxPrintDialogData, setToPage, 1}}, - {2435, {wxPrintData, new_0, 0}}, - {2436, {wxPrintData, new_1, 1}}, - {2437, {wxPrintData, destruct, 0}}, - {2438, {wxPrintData, getCollate, 0}}, - {2439, {wxPrintData, getBin, 0}}, - {2440, {wxPrintData, getColour, 0}}, - {2441, {wxPrintData, getDuplex, 0}}, - {2442, {wxPrintData, getNoCopies, 0}}, - {2443, {wxPrintData, getOrientation, 0}}, - {2444, {wxPrintData, getPaperId, 0}}, - {2445, {wxPrintData, getPrinterName, 0}}, - {2446, {wxPrintData, getQuality, 0}}, - {2447, {wxPrintData, isOk, 0}}, - {2448, {wxPrintData, setBin, 1}}, - {2449, {wxPrintData, setCollate, 1}}, - {2450, {wxPrintData, setColour, 1}}, - {2451, {wxPrintData, setDuplex, 1}}, - {2452, {wxPrintData, setNoCopies, 1}}, - {2453, {wxPrintData, setOrientation, 1}}, - {2454, {wxPrintData, setPaperId, 1}}, - {2455, {wxPrintData, setPrinterName, 1}}, - {2456, {wxPrintData, setQuality, 1}}, - {2459, {wxPrintPreview, new_2, 2}}, - {2460, {wxPrintPreview, new_3, 3}}, - {2462, {wxPrintPreview, destruct, 0}}, - {2463, {wxPrintPreview, getCanvas, 0}}, - {2464, {wxPrintPreview, getCurrentPage, 0}}, - {2465, {wxPrintPreview, getFrame, 0}}, - {2466, {wxPrintPreview, getMaxPage, 0}}, - {2467, {wxPrintPreview, getMinPage, 0}}, - {2468, {wxPrintPreview, getPrintout, 0}}, - {2469, {wxPrintPreview, getPrintoutForPrinting, 0}}, - {2470, {wxPrintPreview, isOk, 0}}, - {2471, {wxPrintPreview, paintPage, 2}}, - {2472, {wxPrintPreview, print, 1}}, - {2473, {wxPrintPreview, renderPage, 1}}, - {2474, {wxPrintPreview, setCanvas, 1}}, - {2475, {wxPrintPreview, setCurrentPage, 1}}, - {2476, {wxPrintPreview, setFrame, 1}}, - {2477, {wxPrintPreview, setPrintout, 1}}, - {2478, {wxPrintPreview, setZoom, 1}}, - {2479, {wxPreviewFrame, new, 3}}, - {2480, {wxPreviewFrame, destruct, 0}}, - {2481, {wxPreviewFrame, createControlBar, 0}}, - {2482, {wxPreviewFrame, createCanvas, 0}}, - {2483, {wxPreviewFrame, initialize, 0}}, - {2484, {wxPreviewFrame, onCloseWindow, 1}}, - {2485, {wxPreviewControlBar, new, 4}}, - {2486, {wxPreviewControlBar, destruct, 0}}, - {2487, {wxPreviewControlBar, createButtons, 0}}, - {2488, {wxPreviewControlBar, getPrintPreview, 0}}, - {2489, {wxPreviewControlBar, getZoomControl, 0}}, - {2490, {wxPreviewControlBar, setZoomControl, 1}}, - {2492, {wxPrinter, new, 1}}, - {2493, {wxPrinter, createAbortWindow, 2}}, - {2494, {wxPrinter, getAbort, 0}}, - {2495, {wxPrinter, getLastError, 0}}, - {2496, {wxPrinter, getPrintDialogData, 0}}, - {2497, {wxPrinter, print, 3}}, - {2498, {wxPrinter, printDialog, 1}}, - {2499, {wxPrinter, reportError, 3}}, - {2500, {wxPrinter, setup, 1}}, - {2501, {wxPrinter, 'Destroy', undefined}}, - {2502, {wxXmlResource, new_1, 1}}, - {2503, {wxXmlResource, new_2, 2}}, - {2504, {wxXmlResource, destruct, 0}}, - {2505, {wxXmlResource, attachUnknownControl, 3}}, - {2506, {wxXmlResource, clearHandlers, 0}}, - {2507, {wxXmlResource, compareVersion, 4}}, - {2508, {wxXmlResource, get, 0}}, - {2509, {wxXmlResource, getFlags, 0}}, - {2510, {wxXmlResource, getVersion, 0}}, - {2511, {wxXmlResource, getXRCID, 2}}, - {2512, {wxXmlResource, initAllHandlers, 0}}, - {2513, {wxXmlResource, load, 1}}, - {2514, {wxXmlResource, loadBitmap, 1}}, - {2515, {wxXmlResource, loadDialog_2, 2}}, - {2516, {wxXmlResource, loadDialog_3, 3}}, - {2517, {wxXmlResource, loadFrame_2, 2}}, - {2518, {wxXmlResource, loadFrame_3, 3}}, - {2519, {wxXmlResource, loadIcon, 1}}, - {2520, {wxXmlResource, loadMenu, 1}}, - {2521, {wxXmlResource, loadMenuBar_2, 2}}, - {2522, {wxXmlResource, loadMenuBar_1, 1}}, - {2523, {wxXmlResource, loadPanel_2, 2}}, - {2524, {wxXmlResource, loadPanel_3, 3}}, - {2525, {wxXmlResource, loadToolBar, 2}}, - {2526, {wxXmlResource, set, 1}}, - {2527, {wxXmlResource, setFlags, 1}}, - {2528, {wxXmlResource, unload, 1}}, - {2529, {wxXmlResource, xrcctrl, 3}}, - {2530, {wxHtmlEasyPrinting, new, 1}}, - {2531, {wxHtmlEasyPrinting, destruct, 0}}, - {2532, {wxHtmlEasyPrinting, getPrintData, 0}}, - {2533, {wxHtmlEasyPrinting, getPageSetupData, 0}}, - {2534, {wxHtmlEasyPrinting, previewFile, 1}}, - {2535, {wxHtmlEasyPrinting, previewText, 2}}, - {2536, {wxHtmlEasyPrinting, printFile, 1}}, - {2537, {wxHtmlEasyPrinting, printText, 2}}, - {2538, {wxHtmlEasyPrinting, pageSetup, 0}}, - {2539, {wxHtmlEasyPrinting, setFonts, 3}}, - {2540, {wxHtmlEasyPrinting, setHeader, 2}}, - {2541, {wxHtmlEasyPrinting, setFooter, 2}}, - {2543, {wxGLCanvas, new_2, 2}}, - {2544, {wxGLCanvas, new_3_1, 3}}, - {2545, {wxGLCanvas, new_3_0, 3}}, - {2546, {wxGLCanvas, getContext, 0}}, - {2548, {wxGLCanvas, setCurrent, 0}}, - {2549, {wxGLCanvas, swapBuffers, 0}}, - {2550, {wxGLCanvas, 'Destroy', undefined}}, - {2551, {wxAuiManager, new, 1}}, - {2552, {wxAuiManager, destruct, 0}}, - {2553, {wxAuiManager, addPane_2_1, 2}}, - {2554, {wxAuiManager, addPane_3, 3}}, - {2555, {wxAuiManager, addPane_2_0, 2}}, - {2556, {wxAuiManager, detachPane, 1}}, - {2557, {wxAuiManager, getAllPanes, 0}}, - {2558, {wxAuiManager, getArtProvider, 0}}, - {2559, {wxAuiManager, getDockSizeConstraint, 2}}, - {2560, {wxAuiManager, getFlags, 0}}, - {2561, {wxAuiManager, getManagedWindow, 0}}, - {2562, {wxAuiManager, getManager, 1}}, - {2563, {wxAuiManager, getPane_1_1, 1}}, - {2564, {wxAuiManager, getPane_1_0, 1}}, - {2565, {wxAuiManager, hideHint, 0}}, - {2566, {wxAuiManager, insertPane, 3}}, - {2567, {wxAuiManager, loadPaneInfo, 2}}, - {2568, {wxAuiManager, loadPerspective, 2}}, - {2569, {wxAuiManager, savePaneInfo, 1}}, - {2570, {wxAuiManager, savePerspective, 0}}, - {2571, {wxAuiManager, setArtProvider, 1}}, - {2572, {wxAuiManager, setDockSizeConstraint, 2}}, - {2573, {wxAuiManager, setFlags, 1}}, - {2574, {wxAuiManager, setManagedWindow, 1}}, - {2575, {wxAuiManager, showHint, 1}}, - {2576, {wxAuiManager, unInit, 0}}, - {2577, {wxAuiManager, update, 0}}, - {2578, {wxAuiPaneInfo, new_0, 0}}, - {2579, {wxAuiPaneInfo, new_1, 1}}, - {2580, {wxAuiPaneInfo, destruct, 0}}, - {2581, {wxAuiPaneInfo, bestSize_1, 1}}, - {2582, {wxAuiPaneInfo, bestSize_2, 2}}, - {2583, {wxAuiPaneInfo, bottom, 0}}, - {2584, {wxAuiPaneInfo, bottomDockable, 1}}, - {2585, {wxAuiPaneInfo, caption, 1}}, - {2586, {wxAuiPaneInfo, captionVisible, 1}}, - {2587, {wxAuiPaneInfo, centre, 0}}, - {2588, {wxAuiPaneInfo, centrePane, 0}}, - {2589, {wxAuiPaneInfo, closeButton, 1}}, - {2590, {wxAuiPaneInfo, defaultPane, 0}}, - {2591, {wxAuiPaneInfo, destroyOnClose, 1}}, - {2592, {wxAuiPaneInfo, direction, 1}}, - {2593, {wxAuiPaneInfo, dock, 0}}, - {2594, {wxAuiPaneInfo, dockable, 1}}, - {2595, {wxAuiPaneInfo, fixed, 0}}, - {2596, {wxAuiPaneInfo, float, 0}}, - {2597, {wxAuiPaneInfo, floatable, 1}}, - {2598, {wxAuiPaneInfo, floatingPosition_1, 1}}, - {2599, {wxAuiPaneInfo, floatingPosition_2, 2}}, - {2600, {wxAuiPaneInfo, floatingSize_1, 1}}, - {2601, {wxAuiPaneInfo, floatingSize_2, 2}}, - {2602, {wxAuiPaneInfo, gripper, 1}}, - {2603, {wxAuiPaneInfo, gripperTop, 1}}, - {2604, {wxAuiPaneInfo, hasBorder, 0}}, - {2605, {wxAuiPaneInfo, hasCaption, 0}}, - {2606, {wxAuiPaneInfo, hasCloseButton, 0}}, - {2607, {wxAuiPaneInfo, hasFlag, 1}}, - {2608, {wxAuiPaneInfo, hasGripper, 0}}, - {2609, {wxAuiPaneInfo, hasGripperTop, 0}}, - {2610, {wxAuiPaneInfo, hasMaximizeButton, 0}}, - {2611, {wxAuiPaneInfo, hasMinimizeButton, 0}}, - {2612, {wxAuiPaneInfo, hasPinButton, 0}}, - {2613, {wxAuiPaneInfo, hide, 0}}, - {2614, {wxAuiPaneInfo, isBottomDockable, 0}}, - {2615, {wxAuiPaneInfo, isDocked, 0}}, - {2616, {wxAuiPaneInfo, isFixed, 0}}, - {2617, {wxAuiPaneInfo, isFloatable, 0}}, - {2618, {wxAuiPaneInfo, isFloating, 0}}, - {2619, {wxAuiPaneInfo, isLeftDockable, 0}}, - {2620, {wxAuiPaneInfo, isMovable, 0}}, - {2621, {wxAuiPaneInfo, isOk, 0}}, - {2622, {wxAuiPaneInfo, isResizable, 0}}, - {2623, {wxAuiPaneInfo, isRightDockable, 0}}, - {2624, {wxAuiPaneInfo, isShown, 0}}, - {2625, {wxAuiPaneInfo, isToolbar, 0}}, - {2626, {wxAuiPaneInfo, isTopDockable, 0}}, - {2627, {wxAuiPaneInfo, layer, 1}}, - {2628, {wxAuiPaneInfo, left, 0}}, - {2629, {wxAuiPaneInfo, leftDockable, 1}}, - {2630, {wxAuiPaneInfo, maxSize_1, 1}}, - {2631, {wxAuiPaneInfo, maxSize_2, 2}}, - {2632, {wxAuiPaneInfo, maximizeButton, 1}}, - {2633, {wxAuiPaneInfo, minSize_1, 1}}, - {2634, {wxAuiPaneInfo, minSize_2, 2}}, - {2635, {wxAuiPaneInfo, minimizeButton, 1}}, - {2636, {wxAuiPaneInfo, movable, 1}}, - {2637, {wxAuiPaneInfo, name, 1}}, - {2638, {wxAuiPaneInfo, paneBorder, 1}}, - {2639, {wxAuiPaneInfo, pinButton, 1}}, - {2640, {wxAuiPaneInfo, position, 1}}, - {2641, {wxAuiPaneInfo, resizable, 1}}, - {2642, {wxAuiPaneInfo, right, 0}}, - {2643, {wxAuiPaneInfo, rightDockable, 1}}, - {2644, {wxAuiPaneInfo, row, 1}}, - {2645, {wxAuiPaneInfo, safeSet, 1}}, - {2646, {wxAuiPaneInfo, setFlag, 2}}, - {2647, {wxAuiPaneInfo, show, 1}}, - {2648, {wxAuiPaneInfo, toolbarPane, 0}}, - {2649, {wxAuiPaneInfo, top, 0}}, - {2650, {wxAuiPaneInfo, topDockable, 1}}, - {2651, {wxAuiPaneInfo, window, 1}}, - {2652, {wxAuiNotebook, new_0, 0}}, - {2653, {wxAuiNotebook, new_2, 2}}, - {2654, {wxAuiNotebook, addPage, 3}}, - {2655, {wxAuiNotebook, create, 2}}, - {2656, {wxAuiNotebook, deletePage, 1}}, - {2657, {wxAuiNotebook, getArtProvider, 0}}, - {2658, {wxAuiNotebook, getPage, 1}}, - {2659, {wxAuiNotebook, getPageBitmap, 1}}, - {2660, {wxAuiNotebook, getPageCount, 0}}, - {2661, {wxAuiNotebook, getPageIndex, 1}}, - {2662, {wxAuiNotebook, getPageText, 1}}, - {2663, {wxAuiNotebook, getSelection, 0}}, - {2664, {wxAuiNotebook, insertPage, 4}}, - {2665, {wxAuiNotebook, removePage, 1}}, - {2666, {wxAuiNotebook, setArtProvider, 1}}, - {2667, {wxAuiNotebook, setFont, 1}}, - {2668, {wxAuiNotebook, setPageBitmap, 2}}, - {2669, {wxAuiNotebook, setPageText, 2}}, - {2670, {wxAuiNotebook, setSelection, 1}}, - {2671, {wxAuiNotebook, setTabCtrlHeight, 1}}, - {2672, {wxAuiNotebook, setUniformBitmapSize, 1}}, - {2673, {wxAuiNotebook, 'Destroy', undefined}}, - {2674, {wxMDIParentFrame, new_0, 0}}, - {2675, {wxMDIParentFrame, new_4, 4}}, - {2676, {wxMDIParentFrame, destruct, 0}}, - {2677, {wxMDIParentFrame, activateNext, 0}}, - {2678, {wxMDIParentFrame, activatePrevious, 0}}, - {2679, {wxMDIParentFrame, arrangeIcons, 0}}, - {2680, {wxMDIParentFrame, cascade, 0}}, - {2681, {wxMDIParentFrame, create, 4}}, - {2682, {wxMDIParentFrame, getActiveChild, 0}}, - {2683, {wxMDIParentFrame, getClientWindow, 0}}, - {2684, {wxMDIParentFrame, tile, 1}}, - {2685, {wxMDIChildFrame, new_0, 0}}, - {2686, {wxMDIChildFrame, new_4, 4}}, - {2687, {wxMDIChildFrame, destruct, 0}}, - {2688, {wxMDIChildFrame, activate, 0}}, - {2689, {wxMDIChildFrame, create, 4}}, - {2690, {wxMDIChildFrame, maximize, 1}}, - {2691, {wxMDIChildFrame, restore, 0}}, - {2692, {wxMDIClientWindow, new_0, 0}}, - {2693, {wxMDIClientWindow, new_2, 2}}, - {2694, {wxMDIClientWindow, destruct, 0}}, - {2695, {wxMDIClientWindow, createClient, 2}}, - {2696, {wxLayoutAlgorithm, new, 0}}, - {2697, {wxLayoutAlgorithm, layoutFrame, 2}}, - {2698, {wxLayoutAlgorithm, layoutMDIFrame, 2}}, - {2699, {wxLayoutAlgorithm, layoutWindow, 2}}, - {2700, {wxLayoutAlgorithm, 'Destroy', undefined}}, - {2701, {wxEvent, getId, 0}}, - {2702, {wxEvent, getSkipped, 0}}, - {2703, {wxEvent, getTimestamp, 0}}, - {2704, {wxEvent, isCommandEvent, 0}}, - {2705, {wxEvent, resumePropagation, 1}}, - {2706, {wxEvent, shouldPropagate, 0}}, - {2707, {wxEvent, skip, 1}}, - {2708, {wxEvent, stopPropagation, 0}}, - {2709, {wxCommandEvent, getClientData, 0}}, - {2710, {wxCommandEvent, getExtraLong, 0}}, - {2711, {wxCommandEvent, getInt, 0}}, - {2712, {wxCommandEvent, getSelection, 0}}, - {2713, {wxCommandEvent, getString, 0}}, - {2714, {wxCommandEvent, isChecked, 0}}, - {2715, {wxCommandEvent, isSelection, 0}}, - {2716, {wxCommandEvent, setInt, 1}}, - {2717, {wxCommandEvent, setString, 1}}, - {2718, {wxScrollEvent, getOrientation, 0}}, - {2719, {wxScrollEvent, getPosition, 0}}, - {2720, {wxScrollWinEvent, getOrientation, 0}}, - {2721, {wxScrollWinEvent, getPosition, 0}}, - {2722, {wxMouseEvent, altDown, 0}}, - {2723, {wxMouseEvent, button, 1}}, - {2724, {wxMouseEvent, buttonDClick, 1}}, - {2725, {wxMouseEvent, buttonDown, 1}}, - {2726, {wxMouseEvent, buttonUp, 1}}, - {2727, {wxMouseEvent, cmdDown, 0}}, - {2728, {wxMouseEvent, controlDown, 0}}, - {2729, {wxMouseEvent, dragging, 0}}, - {2730, {wxMouseEvent, entering, 0}}, - {2731, {wxMouseEvent, getButton, 0}}, - {2734, {wxMouseEvent, getPosition, 0}}, - {2735, {wxMouseEvent, getLogicalPosition, 1}}, - {2736, {wxMouseEvent, getLinesPerAction, 0}}, - {2737, {wxMouseEvent, getWheelRotation, 0}}, - {2738, {wxMouseEvent, getWheelDelta, 0}}, - {2739, {wxMouseEvent, getX, 0}}, - {2740, {wxMouseEvent, getY, 0}}, - {2741, {wxMouseEvent, isButton, 0}}, - {2742, {wxMouseEvent, isPageScroll, 0}}, - {2743, {wxMouseEvent, leaving, 0}}, - {2744, {wxMouseEvent, leftDClick, 0}}, - {2745, {wxMouseEvent, leftDown, 0}}, - {2746, {wxMouseEvent, leftIsDown, 0}}, - {2747, {wxMouseEvent, leftUp, 0}}, - {2748, {wxMouseEvent, metaDown, 0}}, - {2749, {wxMouseEvent, middleDClick, 0}}, - {2750, {wxMouseEvent, middleDown, 0}}, - {2751, {wxMouseEvent, middleIsDown, 0}}, - {2752, {wxMouseEvent, middleUp, 0}}, - {2753, {wxMouseEvent, moving, 0}}, - {2754, {wxMouseEvent, rightDClick, 0}}, - {2755, {wxMouseEvent, rightDown, 0}}, - {2756, {wxMouseEvent, rightIsDown, 0}}, - {2757, {wxMouseEvent, rightUp, 0}}, - {2758, {wxMouseEvent, shiftDown, 0}}, - {2759, {wxSetCursorEvent, getCursor, 0}}, - {2760, {wxSetCursorEvent, getX, 0}}, - {2761, {wxSetCursorEvent, getY, 0}}, - {2762, {wxSetCursorEvent, hasCursor, 0}}, - {2763, {wxSetCursorEvent, setCursor, 1}}, - {2764, {wxKeyEvent, altDown, 0}}, - {2765, {wxKeyEvent, cmdDown, 0}}, - {2766, {wxKeyEvent, controlDown, 0}}, - {2767, {wxKeyEvent, getKeyCode, 0}}, - {2768, {wxKeyEvent, getModifiers, 0}}, - {2771, {wxKeyEvent, getPosition, 0}}, - {2772, {wxKeyEvent, getRawKeyCode, 0}}, - {2773, {wxKeyEvent, getRawKeyFlags, 0}}, - {2774, {wxKeyEvent, getUnicodeKey, 0}}, - {2775, {wxKeyEvent, getX, 0}}, - {2776, {wxKeyEvent, getY, 0}}, - {2777, {wxKeyEvent, hasModifiers, 0}}, - {2778, {wxKeyEvent, metaDown, 0}}, - {2779, {wxKeyEvent, shiftDown, 0}}, - {2780, {wxSizeEvent, getSize, 0}}, - {2781, {wxMoveEvent, getPosition, 0}}, - {2782, {wxEraseEvent, getDC, 0}}, - {2783, {wxFocusEvent, getWindow, 0}}, - {2784, {wxChildFocusEvent, getWindow, 0}}, - {2785, {wxMenuEvent, getMenu, 0}}, - {2786, {wxMenuEvent, getMenuId, 0}}, - {2787, {wxMenuEvent, isPopup, 0}}, - {2788, {wxCloseEvent, canVeto, 0}}, - {2789, {wxCloseEvent, getLoggingOff, 0}}, - {2790, {wxCloseEvent, setCanVeto, 1}}, - {2791, {wxCloseEvent, setLoggingOff, 1}}, - {2792, {wxCloseEvent, veto, 1}}, - {2793, {wxShowEvent, setShow, 1}}, - {2794, {wxShowEvent, getShow, 0}}, - {2795, {wxIconizeEvent, iconized, 0}}, - {2796, {wxJoystickEvent, buttonDown, 1}}, - {2797, {wxJoystickEvent, buttonIsDown, 1}}, - {2798, {wxJoystickEvent, buttonUp, 1}}, - {2799, {wxJoystickEvent, getButtonChange, 0}}, - {2800, {wxJoystickEvent, getButtonState, 0}}, - {2801, {wxJoystickEvent, getJoystick, 0}}, - {2802, {wxJoystickEvent, getPosition, 0}}, - {2803, {wxJoystickEvent, getZPosition, 0}}, - {2804, {wxJoystickEvent, isButton, 0}}, - {2805, {wxJoystickEvent, isMove, 0}}, - {2806, {wxJoystickEvent, isZMove, 0}}, - {2807, {wxUpdateUIEvent, canUpdate, 1}}, - {2808, {wxUpdateUIEvent, check, 1}}, - {2809, {wxUpdateUIEvent, enable, 1}}, - {2810, {wxUpdateUIEvent, show, 1}}, - {2811, {wxUpdateUIEvent, getChecked, 0}}, - {2812, {wxUpdateUIEvent, getEnabled, 0}}, - {2813, {wxUpdateUIEvent, getShown, 0}}, - {2814, {wxUpdateUIEvent, getSetChecked, 0}}, - {2815, {wxUpdateUIEvent, getSetEnabled, 0}}, - {2816, {wxUpdateUIEvent, getSetShown, 0}}, - {2817, {wxUpdateUIEvent, getSetText, 0}}, - {2818, {wxUpdateUIEvent, getText, 0}}, - {2819, {wxUpdateUIEvent, getMode, 0}}, - {2820, {wxUpdateUIEvent, getUpdateInterval, 0}}, - {2821, {wxUpdateUIEvent, resetUpdateTime, 0}}, - {2822, {wxUpdateUIEvent, setMode, 1}}, - {2823, {wxUpdateUIEvent, setText, 1}}, - {2824, {wxUpdateUIEvent, setUpdateInterval, 1}}, - {2825, {wxMouseCaptureChangedEvent, getCapturedWindow, 0}}, - {2826, {wxPaletteChangedEvent, setChangedWindow, 1}}, - {2827, {wxPaletteChangedEvent, getChangedWindow, 0}}, - {2828, {wxQueryNewPaletteEvent, setPaletteRealized, 1}}, - {2829, {wxQueryNewPaletteEvent, getPaletteRealized, 0}}, - {2830, {wxNavigationKeyEvent, getDirection, 0}}, - {2831, {wxNavigationKeyEvent, setDirection, 1}}, - {2832, {wxNavigationKeyEvent, isWindowChange, 0}}, - {2833, {wxNavigationKeyEvent, setWindowChange, 1}}, - {2834, {wxNavigationKeyEvent, isFromTab, 0}}, - {2835, {wxNavigationKeyEvent, setFromTab, 1}}, - {2836, {wxNavigationKeyEvent, getCurrentFocus, 0}}, - {2837, {wxNavigationKeyEvent, setCurrentFocus, 1}}, - {2838, {wxHelpEvent, getOrigin, 0}}, - {2839, {wxHelpEvent, getPosition, 0}}, - {2840, {wxHelpEvent, setOrigin, 1}}, - {2841, {wxHelpEvent, setPosition, 1}}, - {2842, {wxContextMenuEvent, getPosition, 0}}, - {2843, {wxContextMenuEvent, setPosition, 1}}, - {2844, {wxIdleEvent, canSend, 1}}, - {2845, {wxIdleEvent, getMode, 0}}, - {2846, {wxIdleEvent, requestMore, 1}}, - {2847, {wxIdleEvent, moreRequested, 0}}, - {2848, {wxIdleEvent, setMode, 1}}, - {2849, {wxGridEvent, altDown, 0}}, - {2850, {wxGridEvent, controlDown, 0}}, - {2851, {wxGridEvent, getCol, 0}}, - {2852, {wxGridEvent, getPosition, 0}}, - {2853, {wxGridEvent, getRow, 0}}, - {2854, {wxGridEvent, metaDown, 0}}, - {2855, {wxGridEvent, selecting, 0}}, - {2856, {wxGridEvent, shiftDown, 0}}, - {2857, {wxNotifyEvent, allow, 0}}, - {2858, {wxNotifyEvent, isAllowed, 0}}, - {2859, {wxNotifyEvent, veto, 0}}, - {2860, {wxSashEvent, getEdge, 0}}, - {2861, {wxSashEvent, getDragRect, 0}}, - {2862, {wxSashEvent, getDragStatus, 0}}, - {2863, {wxListEvent, getCacheFrom, 0}}, - {2864, {wxListEvent, getCacheTo, 0}}, - {2865, {wxListEvent, getKeyCode, 0}}, - {2866, {wxListEvent, getIndex, 0}}, - {2867, {wxListEvent, getColumn, 0}}, - {2868, {wxListEvent, getPoint, 0}}, - {2869, {wxListEvent, getLabel, 0}}, - {2870, {wxListEvent, getText, 0}}, - {2871, {wxListEvent, getImage, 0}}, - {2872, {wxListEvent, getData, 0}}, - {2873, {wxListEvent, getMask, 0}}, - {2874, {wxListEvent, getItem, 0}}, - {2875, {wxListEvent, isEditCancelled, 0}}, - {2876, {wxDateEvent, getDate, 0}}, - {2877, {wxCalendarEvent, getWeekDay, 0}}, - {2878, {wxFileDirPickerEvent, getPath, 0}}, - {2879, {wxColourPickerEvent, getColour, 0}}, - {2880, {wxFontPickerEvent, getFont, 0}}, - {2881, {wxStyledTextEvent, getPosition, 0}}, - {2882, {wxStyledTextEvent, getKey, 0}}, - {2883, {wxStyledTextEvent, getModifiers, 0}}, - {2884, {wxStyledTextEvent, getModificationType, 0}}, - {2885, {wxStyledTextEvent, getText, 0}}, - {2886, {wxStyledTextEvent, getLength, 0}}, - {2887, {wxStyledTextEvent, getLinesAdded, 0}}, - {2888, {wxStyledTextEvent, getLine, 0}}, - {2889, {wxStyledTextEvent, getFoldLevelNow, 0}}, - {2890, {wxStyledTextEvent, getFoldLevelPrev, 0}}, - {2891, {wxStyledTextEvent, getMargin, 0}}, - {2892, {wxStyledTextEvent, getMessage, 0}}, - {2893, {wxStyledTextEvent, getWParam, 0}}, - {2894, {wxStyledTextEvent, getLParam, 0}}, - {2895, {wxStyledTextEvent, getListType, 0}}, - {2896, {wxStyledTextEvent, getX, 0}}, - {2897, {wxStyledTextEvent, getY, 0}}, - {2898, {wxStyledTextEvent, getDragText, 0}}, - {2899, {wxStyledTextEvent, getDragAllowMove, 0}}, - {2900, {wxStyledTextEvent, getDragResult, 0}}, - {2901, {wxStyledTextEvent, getShift, 0}}, - {2902, {wxStyledTextEvent, getControl, 0}}, - {2903, {wxStyledTextEvent, getAlt, 0}}, - {2904, {utils, getKeyState, 1}}, - {2905, {utils, getMousePosition, 2}}, - {2906, {utils, getMouseState, 0}}, - {2907, {utils, setDetectableAutoRepeat, 1}}, - {2908, {utils, bell, 0}}, - {2909, {utils, findMenuItemId, 3}}, - {2910, {utils, genericFindWindowAtPoint, 1}}, - {2911, {utils, findWindowAtPoint, 1}}, - {2912, {utils, beginBusyCursor, 1}}, - {2913, {utils, endBusyCursor, 0}}, - {2914, {utils, isBusy, 0}}, - {2915, {utils, shutdown, 1}}, - {2916, {utils, shell, 1}}, - {2917, {utils, launchDefaultBrowser, 2}}, - {2918, {utils, getEmailAddress, 0}}, - {2919, {utils, getUserId, 0}}, - {2920, {utils, getHomeDir, 0}}, - {2921, {utils, newId, 0}}, - {2922, {utils, registerId, 1}}, - {2923, {utils, getCurrentId, 0}}, - {2924, {utils, getOsDescription, 0}}, - {2925, {utils, isPlatformLittleEndian, 0}}, - {2926, {utils, isPlatform64Bit, 0}}, - {2927, {wxPrintout, new, 1}}, - {2928, {wxPrintout, destruct, 0}}, - {2929, {wxPrintout, getDC, 0}}, - {2930, {wxPrintout, getPageSizeMM, 2}}, - {2931, {wxPrintout, getPageSizePixels, 2}}, - {2932, {wxPrintout, getPaperRectPixels, 0}}, - {2933, {wxPrintout, getPPIPrinter, 2}}, - {2934, {wxPrintout, getPPIScreen, 2}}, - {2935, {wxPrintout, getTitle, 0}}, - {2936, {wxPrintout, isPreview, 0}}, - {2937, {wxPrintout, fitThisSizeToPaper, 1}}, - {2938, {wxPrintout, fitThisSizeToPage, 1}}, - {2939, {wxPrintout, fitThisSizeToPageMargins, 2}}, - {2940, {wxPrintout, mapScreenSizeToPaper, 0}}, - {2941, {wxPrintout, mapScreenSizeToPage, 0}}, - {2942, {wxPrintout, mapScreenSizeToPageMargins, 1}}, - {2943, {wxPrintout, mapScreenSizeToDevice, 0}}, - {2944, {wxPrintout, getLogicalPaperRect, 0}}, - {2945, {wxPrintout, getLogicalPageRect, 0}}, - {2946, {wxPrintout, getLogicalPageMarginsRect, 1}}, - {2947, {wxPrintout, setLogicalOrigin, 2}}, - {2948, {wxPrintout, offsetLogicalOrigin, 2}}, - {2949, {wxStyledTextCtrl, new_2, 2}}, - {2950, {wxStyledTextCtrl, new_0, 0}}, - {2951, {wxStyledTextCtrl, destruct, 0}}, - {2952, {wxStyledTextCtrl, create, 2}}, - {2953, {wxStyledTextCtrl, addText, 1}}, - {2954, {wxStyledTextCtrl, addStyledText, 1}}, - {2955, {wxStyledTextCtrl, insertText, 2}}, - {2956, {wxStyledTextCtrl, clearAll, 0}}, - {2957, {wxStyledTextCtrl, clearDocumentStyle, 0}}, - {2958, {wxStyledTextCtrl, getLength, 0}}, - {2959, {wxStyledTextCtrl, getCharAt, 1}}, - {2960, {wxStyledTextCtrl, getCurrentPos, 0}}, - {2961, {wxStyledTextCtrl, getAnchor, 0}}, - {2962, {wxStyledTextCtrl, getStyleAt, 1}}, - {2963, {wxStyledTextCtrl, redo, 0}}, - {2964, {wxStyledTextCtrl, setUndoCollection, 1}}, - {2965, {wxStyledTextCtrl, selectAll, 0}}, - {2966, {wxStyledTextCtrl, setSavePoint, 0}}, - {2967, {wxStyledTextCtrl, getStyledText, 2}}, - {2968, {wxStyledTextCtrl, canRedo, 0}}, - {2969, {wxStyledTextCtrl, markerLineFromHandle, 1}}, - {2970, {wxStyledTextCtrl, markerDeleteHandle, 1}}, - {2971, {wxStyledTextCtrl, getUndoCollection, 0}}, - {2972, {wxStyledTextCtrl, getViewWhiteSpace, 0}}, - {2973, {wxStyledTextCtrl, setViewWhiteSpace, 1}}, - {2974, {wxStyledTextCtrl, positionFromPoint, 1}}, - {2975, {wxStyledTextCtrl, positionFromPointClose, 2}}, - {2976, {wxStyledTextCtrl, gotoLine, 1}}, - {2977, {wxStyledTextCtrl, gotoPos, 1}}, - {2978, {wxStyledTextCtrl, setAnchor, 1}}, - {2979, {wxStyledTextCtrl, getCurLine, 1}}, - {2980, {wxStyledTextCtrl, getEndStyled, 0}}, - {2981, {wxStyledTextCtrl, convertEOLs, 1}}, - {2982, {wxStyledTextCtrl, getEOLMode, 0}}, - {2983, {wxStyledTextCtrl, setEOLMode, 1}}, - {2984, {wxStyledTextCtrl, startStyling, 2}}, - {2985, {wxStyledTextCtrl, setStyling, 2}}, - {2986, {wxStyledTextCtrl, getBufferedDraw, 0}}, - {2987, {wxStyledTextCtrl, setBufferedDraw, 1}}, - {2988, {wxStyledTextCtrl, setTabWidth, 1}}, - {2989, {wxStyledTextCtrl, getTabWidth, 0}}, - {2990, {wxStyledTextCtrl, setCodePage, 1}}, - {2991, {wxStyledTextCtrl, markerDefine, 3}}, - {2992, {wxStyledTextCtrl, markerSetForeground, 2}}, - {2993, {wxStyledTextCtrl, markerSetBackground, 2}}, - {2994, {wxStyledTextCtrl, markerAdd, 2}}, - {2995, {wxStyledTextCtrl, markerDelete, 2}}, - {2996, {wxStyledTextCtrl, markerDeleteAll, 1}}, - {2997, {wxStyledTextCtrl, markerGet, 1}}, - {2998, {wxStyledTextCtrl, markerNext, 2}}, - {2999, {wxStyledTextCtrl, markerPrevious, 2}}, - {3000, {wxStyledTextCtrl, markerDefineBitmap, 2}}, - {3001, {wxStyledTextCtrl, markerAddSet, 2}}, - {3002, {wxStyledTextCtrl, markerSetAlpha, 2}}, - {3003, {wxStyledTextCtrl, setMarginType, 2}}, - {3004, {wxStyledTextCtrl, getMarginType, 1}}, - {3005, {wxStyledTextCtrl, setMarginWidth, 2}}, - {3006, {wxStyledTextCtrl, getMarginWidth, 1}}, - {3007, {wxStyledTextCtrl, setMarginMask, 2}}, - {3008, {wxStyledTextCtrl, getMarginMask, 1}}, - {3009, {wxStyledTextCtrl, setMarginSensitive, 2}}, - {3010, {wxStyledTextCtrl, getMarginSensitive, 1}}, - {3011, {wxStyledTextCtrl, styleClearAll, 0}}, - {3012, {wxStyledTextCtrl, styleSetForeground, 2}}, - {3013, {wxStyledTextCtrl, styleSetBackground, 2}}, - {3014, {wxStyledTextCtrl, styleSetBold, 2}}, - {3015, {wxStyledTextCtrl, styleSetItalic, 2}}, - {3016, {wxStyledTextCtrl, styleSetSize, 2}}, - {3017, {wxStyledTextCtrl, styleSetFaceName, 2}}, - {3018, {wxStyledTextCtrl, styleSetEOLFilled, 2}}, - {3019, {wxStyledTextCtrl, styleResetDefault, 0}}, - {3020, {wxStyledTextCtrl, styleSetUnderline, 2}}, - {3021, {wxStyledTextCtrl, styleSetCase, 2}}, - {3022, {wxStyledTextCtrl, styleSetHotSpot, 2}}, - {3023, {wxStyledTextCtrl, setSelForeground, 2}}, - {3024, {wxStyledTextCtrl, setSelBackground, 2}}, - {3025, {wxStyledTextCtrl, getSelAlpha, 0}}, - {3026, {wxStyledTextCtrl, setSelAlpha, 1}}, - {3027, {wxStyledTextCtrl, setCaretForeground, 1}}, - {3028, {wxStyledTextCtrl, cmdKeyAssign, 3}}, - {3029, {wxStyledTextCtrl, cmdKeyClear, 2}}, - {3030, {wxStyledTextCtrl, cmdKeyClearAll, 0}}, - {3031, {wxStyledTextCtrl, setStyleBytes, 2}}, - {3032, {wxStyledTextCtrl, styleSetVisible, 2}}, - {3033, {wxStyledTextCtrl, getCaretPeriod, 0}}, - {3034, {wxStyledTextCtrl, setCaretPeriod, 1}}, - {3035, {wxStyledTextCtrl, setWordChars, 1}}, - {3036, {wxStyledTextCtrl, beginUndoAction, 0}}, - {3037, {wxStyledTextCtrl, endUndoAction, 0}}, - {3038, {wxStyledTextCtrl, indicatorSetStyle, 2}}, - {3039, {wxStyledTextCtrl, indicatorGetStyle, 1}}, - {3040, {wxStyledTextCtrl, indicatorSetForeground, 2}}, - {3041, {wxStyledTextCtrl, indicatorGetForeground, 1}}, - {3042, {wxStyledTextCtrl, setWhitespaceForeground, 2}}, - {3043, {wxStyledTextCtrl, setWhitespaceBackground, 2}}, - {3044, {wxStyledTextCtrl, getStyleBits, 0}}, - {3045, {wxStyledTextCtrl, setLineState, 2}}, - {3046, {wxStyledTextCtrl, getLineState, 1}}, - {3047, {wxStyledTextCtrl, getMaxLineState, 0}}, - {3048, {wxStyledTextCtrl, getCaretLineVisible, 0}}, - {3049, {wxStyledTextCtrl, setCaretLineVisible, 1}}, - {3050, {wxStyledTextCtrl, getCaretLineBackground, 0}}, - {3051, {wxStyledTextCtrl, setCaretLineBackground, 1}}, - {3052, {wxStyledTextCtrl, autoCompShow, 2}}, - {3053, {wxStyledTextCtrl, autoCompCancel, 0}}, - {3054, {wxStyledTextCtrl, autoCompActive, 0}}, - {3055, {wxStyledTextCtrl, autoCompPosStart, 0}}, - {3056, {wxStyledTextCtrl, autoCompComplete, 0}}, - {3057, {wxStyledTextCtrl, autoCompStops, 1}}, - {3058, {wxStyledTextCtrl, autoCompSetSeparator, 1}}, - {3059, {wxStyledTextCtrl, autoCompGetSeparator, 0}}, - {3060, {wxStyledTextCtrl, autoCompSelect, 1}}, - {3061, {wxStyledTextCtrl, autoCompSetCancelAtStart, 1}}, - {3062, {wxStyledTextCtrl, autoCompGetCancelAtStart, 0}}, - {3063, {wxStyledTextCtrl, autoCompSetFillUps, 1}}, - {3064, {wxStyledTextCtrl, autoCompSetChooseSingle, 1}}, - {3065, {wxStyledTextCtrl, autoCompGetChooseSingle, 0}}, - {3066, {wxStyledTextCtrl, autoCompSetIgnoreCase, 1}}, - {3067, {wxStyledTextCtrl, autoCompGetIgnoreCase, 0}}, - {3068, {wxStyledTextCtrl, userListShow, 2}}, - {3069, {wxStyledTextCtrl, autoCompSetAutoHide, 1}}, - {3070, {wxStyledTextCtrl, autoCompGetAutoHide, 0}}, - {3071, {wxStyledTextCtrl, autoCompSetDropRestOfWord, 1}}, - {3072, {wxStyledTextCtrl, autoCompGetDropRestOfWord, 0}}, - {3073, {wxStyledTextCtrl, registerImage, 2}}, - {3074, {wxStyledTextCtrl, clearRegisteredImages, 0}}, - {3075, {wxStyledTextCtrl, autoCompGetTypeSeparator, 0}}, - {3076, {wxStyledTextCtrl, autoCompSetTypeSeparator, 1}}, - {3077, {wxStyledTextCtrl, autoCompSetMaxWidth, 1}}, - {3078, {wxStyledTextCtrl, autoCompGetMaxWidth, 0}}, - {3079, {wxStyledTextCtrl, autoCompSetMaxHeight, 1}}, - {3080, {wxStyledTextCtrl, autoCompGetMaxHeight, 0}}, - {3081, {wxStyledTextCtrl, setIndent, 1}}, - {3082, {wxStyledTextCtrl, getIndent, 0}}, - {3083, {wxStyledTextCtrl, setUseTabs, 1}}, - {3084, {wxStyledTextCtrl, getUseTabs, 0}}, - {3085, {wxStyledTextCtrl, setLineIndentation, 2}}, - {3086, {wxStyledTextCtrl, getLineIndentation, 1}}, - {3087, {wxStyledTextCtrl, getLineIndentPosition, 1}}, - {3088, {wxStyledTextCtrl, getColumn, 1}}, - {3089, {wxStyledTextCtrl, setUseHorizontalScrollBar, 1}}, - {3090, {wxStyledTextCtrl, getUseHorizontalScrollBar, 0}}, - {3091, {wxStyledTextCtrl, setIndentationGuides, 1}}, - {3092, {wxStyledTextCtrl, getIndentationGuides, 0}}, - {3093, {wxStyledTextCtrl, setHighlightGuide, 1}}, - {3094, {wxStyledTextCtrl, getHighlightGuide, 0}}, - {3095, {wxStyledTextCtrl, getLineEndPosition, 1}}, - {3096, {wxStyledTextCtrl, getCodePage, 0}}, - {3097, {wxStyledTextCtrl, getCaretForeground, 0}}, - {3098, {wxStyledTextCtrl, getReadOnly, 0}}, - {3099, {wxStyledTextCtrl, setCurrentPos, 1}}, - {3100, {wxStyledTextCtrl, setSelectionStart, 1}}, - {3101, {wxStyledTextCtrl, getSelectionStart, 0}}, - {3102, {wxStyledTextCtrl, setSelectionEnd, 1}}, - {3103, {wxStyledTextCtrl, getSelectionEnd, 0}}, - {3104, {wxStyledTextCtrl, setPrintMagnification, 1}}, - {3105, {wxStyledTextCtrl, getPrintMagnification, 0}}, - {3106, {wxStyledTextCtrl, setPrintColourMode, 1}}, - {3107, {wxStyledTextCtrl, getPrintColourMode, 0}}, - {3108, {wxStyledTextCtrl, findText, 4}}, - {3109, {wxStyledTextCtrl, formatRange, 7}}, - {3110, {wxStyledTextCtrl, getFirstVisibleLine, 0}}, - {3111, {wxStyledTextCtrl, getLine, 1}}, - {3112, {wxStyledTextCtrl, getLineCount, 0}}, - {3113, {wxStyledTextCtrl, setMarginLeft, 1}}, - {3114, {wxStyledTextCtrl, getMarginLeft, 0}}, - {3115, {wxStyledTextCtrl, setMarginRight, 1}}, - {3116, {wxStyledTextCtrl, getMarginRight, 0}}, - {3117, {wxStyledTextCtrl, getModify, 0}}, - {3118, {wxStyledTextCtrl, setSelection, 2}}, - {3119, {wxStyledTextCtrl, getSelectedText, 0}}, - {3120, {wxStyledTextCtrl, getTextRange, 2}}, - {3121, {wxStyledTextCtrl, hideSelection, 1}}, - {3122, {wxStyledTextCtrl, lineFromPosition, 1}}, - {3123, {wxStyledTextCtrl, positionFromLine, 1}}, - {3124, {wxStyledTextCtrl, lineScroll, 2}}, - {3125, {wxStyledTextCtrl, ensureCaretVisible, 0}}, - {3126, {wxStyledTextCtrl, replaceSelection, 1}}, - {3127, {wxStyledTextCtrl, setReadOnly, 1}}, - {3128, {wxStyledTextCtrl, canPaste, 0}}, - {3129, {wxStyledTextCtrl, canUndo, 0}}, - {3130, {wxStyledTextCtrl, emptyUndoBuffer, 0}}, - {3131, {wxStyledTextCtrl, undo, 0}}, - {3132, {wxStyledTextCtrl, cut, 0}}, - {3133, {wxStyledTextCtrl, copy, 0}}, - {3134, {wxStyledTextCtrl, paste, 0}}, - {3135, {wxStyledTextCtrl, clear, 0}}, - {3136, {wxStyledTextCtrl, setText, 1}}, - {3137, {wxStyledTextCtrl, getText, 0}}, - {3138, {wxStyledTextCtrl, getTextLength, 0}}, - {3139, {wxStyledTextCtrl, getOvertype, 0}}, - {3140, {wxStyledTextCtrl, setCaretWidth, 1}}, - {3141, {wxStyledTextCtrl, getCaretWidth, 0}}, - {3142, {wxStyledTextCtrl, setTargetStart, 1}}, - {3143, {wxStyledTextCtrl, getTargetStart, 0}}, - {3144, {wxStyledTextCtrl, setTargetEnd, 1}}, - {3145, {wxStyledTextCtrl, getTargetEnd, 0}}, - {3146, {wxStyledTextCtrl, replaceTarget, 1}}, - {3147, {wxStyledTextCtrl, searchInTarget, 1}}, - {3148, {wxStyledTextCtrl, setSearchFlags, 1}}, - {3149, {wxStyledTextCtrl, getSearchFlags, 0}}, - {3150, {wxStyledTextCtrl, callTipShow, 2}}, - {3151, {wxStyledTextCtrl, callTipCancel, 0}}, - {3152, {wxStyledTextCtrl, callTipActive, 0}}, - {3153, {wxStyledTextCtrl, callTipPosAtStart, 0}}, - {3154, {wxStyledTextCtrl, callTipSetHighlight, 2}}, - {3155, {wxStyledTextCtrl, callTipSetBackground, 1}}, - {3156, {wxStyledTextCtrl, callTipSetForeground, 1}}, - {3157, {wxStyledTextCtrl, callTipSetForegroundHighlight, 1}}, - {3158, {wxStyledTextCtrl, callTipUseStyle, 1}}, - {3159, {wxStyledTextCtrl, visibleFromDocLine, 1}}, - {3160, {wxStyledTextCtrl, docLineFromVisible, 1}}, - {3161, {wxStyledTextCtrl, wrapCount, 1}}, - {3162, {wxStyledTextCtrl, setFoldLevel, 2}}, - {3163, {wxStyledTextCtrl, getFoldLevel, 1}}, - {3164, {wxStyledTextCtrl, getLastChild, 2}}, - {3165, {wxStyledTextCtrl, getFoldParent, 1}}, - {3166, {wxStyledTextCtrl, showLines, 2}}, - {3167, {wxStyledTextCtrl, hideLines, 2}}, - {3168, {wxStyledTextCtrl, getLineVisible, 1}}, - {3169, {wxStyledTextCtrl, setFoldExpanded, 2}}, - {3170, {wxStyledTextCtrl, getFoldExpanded, 1}}, - {3171, {wxStyledTextCtrl, toggleFold, 1}}, - {3172, {wxStyledTextCtrl, ensureVisible, 1}}, - {3173, {wxStyledTextCtrl, setFoldFlags, 1}}, - {3174, {wxStyledTextCtrl, ensureVisibleEnforcePolicy, 1}}, - {3175, {wxStyledTextCtrl, setTabIndents, 1}}, - {3176, {wxStyledTextCtrl, getTabIndents, 0}}, - {3177, {wxStyledTextCtrl, setBackSpaceUnIndents, 1}}, - {3178, {wxStyledTextCtrl, getBackSpaceUnIndents, 0}}, - {3179, {wxStyledTextCtrl, setMouseDwellTime, 1}}, - {3180, {wxStyledTextCtrl, getMouseDwellTime, 0}}, - {3181, {wxStyledTextCtrl, wordStartPosition, 2}}, - {3182, {wxStyledTextCtrl, wordEndPosition, 2}}, - {3183, {wxStyledTextCtrl, setWrapMode, 1}}, - {3184, {wxStyledTextCtrl, getWrapMode, 0}}, - {3185, {wxStyledTextCtrl, setWrapVisualFlags, 1}}, - {3186, {wxStyledTextCtrl, getWrapVisualFlags, 0}}, - {3187, {wxStyledTextCtrl, setWrapVisualFlagsLocation, 1}}, - {3188, {wxStyledTextCtrl, getWrapVisualFlagsLocation, 0}}, - {3189, {wxStyledTextCtrl, setWrapStartIndent, 1}}, - {3190, {wxStyledTextCtrl, getWrapStartIndent, 0}}, - {3191, {wxStyledTextCtrl, setLayoutCache, 1}}, - {3192, {wxStyledTextCtrl, getLayoutCache, 0}}, - {3193, {wxStyledTextCtrl, setScrollWidth, 1}}, - {3194, {wxStyledTextCtrl, getScrollWidth, 0}}, - {3195, {wxStyledTextCtrl, textWidth, 2}}, - {3196, {wxStyledTextCtrl, getEndAtLastLine, 0}}, - {3197, {wxStyledTextCtrl, textHeight, 1}}, - {3198, {wxStyledTextCtrl, setUseVerticalScrollBar, 1}}, - {3199, {wxStyledTextCtrl, getUseVerticalScrollBar, 0}}, - {3200, {wxStyledTextCtrl, appendText, 1}}, - {3201, {wxStyledTextCtrl, getTwoPhaseDraw, 0}}, - {3202, {wxStyledTextCtrl, setTwoPhaseDraw, 1}}, - {3203, {wxStyledTextCtrl, targetFromSelection, 0}}, - {3204, {wxStyledTextCtrl, linesJoin, 0}}, - {3205, {wxStyledTextCtrl, linesSplit, 1}}, - {3206, {wxStyledTextCtrl, setFoldMarginColour, 2}}, - {3207, {wxStyledTextCtrl, setFoldMarginHiColour, 2}}, - {3208, {wxStyledTextCtrl, lineDown, 0}}, - {3209, {wxStyledTextCtrl, lineDownExtend, 0}}, - {3210, {wxStyledTextCtrl, lineUp, 0}}, - {3211, {wxStyledTextCtrl, lineUpExtend, 0}}, - {3212, {wxStyledTextCtrl, charLeft, 0}}, - {3213, {wxStyledTextCtrl, charLeftExtend, 0}}, - {3214, {wxStyledTextCtrl, charRight, 0}}, - {3215, {wxStyledTextCtrl, charRightExtend, 0}}, - {3216, {wxStyledTextCtrl, wordLeft, 0}}, - {3217, {wxStyledTextCtrl, wordLeftExtend, 0}}, - {3218, {wxStyledTextCtrl, wordRight, 0}}, - {3219, {wxStyledTextCtrl, wordRightExtend, 0}}, - {3220, {wxStyledTextCtrl, home, 0}}, - {3221, {wxStyledTextCtrl, homeExtend, 0}}, - {3222, {wxStyledTextCtrl, lineEnd, 0}}, - {3223, {wxStyledTextCtrl, lineEndExtend, 0}}, - {3224, {wxStyledTextCtrl, documentStart, 0}}, - {3225, {wxStyledTextCtrl, documentStartExtend, 0}}, - {3226, {wxStyledTextCtrl, documentEnd, 0}}, - {3227, {wxStyledTextCtrl, documentEndExtend, 0}}, - {3228, {wxStyledTextCtrl, pageUp, 0}}, - {3229, {wxStyledTextCtrl, pageUpExtend, 0}}, - {3230, {wxStyledTextCtrl, pageDown, 0}}, - {3231, {wxStyledTextCtrl, pageDownExtend, 0}}, - {3232, {wxStyledTextCtrl, editToggleOvertype, 0}}, - {3233, {wxStyledTextCtrl, cancel, 0}}, - {3234, {wxStyledTextCtrl, deleteBack, 0}}, - {3235, {wxStyledTextCtrl, tab, 0}}, - {3236, {wxStyledTextCtrl, backTab, 0}}, - {3237, {wxStyledTextCtrl, newLine, 0}}, - {3238, {wxStyledTextCtrl, formFeed, 0}}, - {3239, {wxStyledTextCtrl, vCHome, 0}}, - {3240, {wxStyledTextCtrl, vCHomeExtend, 0}}, - {3241, {wxStyledTextCtrl, zoomIn, 0}}, - {3242, {wxStyledTextCtrl, zoomOut, 0}}, - {3243, {wxStyledTextCtrl, delWordLeft, 0}}, - {3244, {wxStyledTextCtrl, delWordRight, 0}}, - {3245, {wxStyledTextCtrl, lineCut, 0}}, - {3246, {wxStyledTextCtrl, lineDelete, 0}}, - {3247, {wxStyledTextCtrl, lineTranspose, 0}}, - {3248, {wxStyledTextCtrl, lineDuplicate, 0}}, - {3249, {wxStyledTextCtrl, lowerCase, 0}}, - {3250, {wxStyledTextCtrl, upperCase, 0}}, - {3251, {wxStyledTextCtrl, lineScrollDown, 0}}, - {3252, {wxStyledTextCtrl, lineScrollUp, 0}}, - {3253, {wxStyledTextCtrl, deleteBackNotLine, 0}}, - {3254, {wxStyledTextCtrl, homeDisplay, 0}}, - {3255, {wxStyledTextCtrl, homeDisplayExtend, 0}}, - {3256, {wxStyledTextCtrl, lineEndDisplay, 0}}, - {3257, {wxStyledTextCtrl, lineEndDisplayExtend, 0}}, - {3258, {wxStyledTextCtrl, homeWrapExtend, 0}}, - {3259, {wxStyledTextCtrl, lineEndWrap, 0}}, - {3260, {wxStyledTextCtrl, lineEndWrapExtend, 0}}, - {3261, {wxStyledTextCtrl, vCHomeWrap, 0}}, - {3262, {wxStyledTextCtrl, vCHomeWrapExtend, 0}}, - {3263, {wxStyledTextCtrl, lineCopy, 0}}, - {3264, {wxStyledTextCtrl, moveCaretInsideView, 0}}, - {3265, {wxStyledTextCtrl, lineLength, 1}}, - {3266, {wxStyledTextCtrl, braceHighlight, 2}}, - {3267, {wxStyledTextCtrl, braceBadLight, 1}}, - {3268, {wxStyledTextCtrl, braceMatch, 1}}, - {3269, {wxStyledTextCtrl, getViewEOL, 0}}, - {3270, {wxStyledTextCtrl, setViewEOL, 1}}, - {3271, {wxStyledTextCtrl, setModEventMask, 1}}, - {3272, {wxStyledTextCtrl, getEdgeColumn, 0}}, - {3273, {wxStyledTextCtrl, setEdgeColumn, 1}}, - {3274, {wxStyledTextCtrl, setEdgeMode, 1}}, - {3275, {wxStyledTextCtrl, getEdgeMode, 0}}, - {3276, {wxStyledTextCtrl, getEdgeColour, 0}}, - {3277, {wxStyledTextCtrl, setEdgeColour, 1}}, - {3278, {wxStyledTextCtrl, searchAnchor, 0}}, - {3279, {wxStyledTextCtrl, searchNext, 2}}, - {3280, {wxStyledTextCtrl, searchPrev, 2}}, - {3281, {wxStyledTextCtrl, linesOnScreen, 0}}, - {3282, {wxStyledTextCtrl, usePopUp, 1}}, - {3283, {wxStyledTextCtrl, selectionIsRectangle, 0}}, - {3284, {wxStyledTextCtrl, setZoom, 1}}, - {3285, {wxStyledTextCtrl, getZoom, 0}}, - {3286, {wxStyledTextCtrl, getModEventMask, 0}}, - {3287, {wxStyledTextCtrl, setSTCFocus, 1}}, - {3288, {wxStyledTextCtrl, getSTCFocus, 0}}, - {3289, {wxStyledTextCtrl, setStatus, 1}}, - {3290, {wxStyledTextCtrl, getStatus, 0}}, - {3291, {wxStyledTextCtrl, setMouseDownCaptures, 1}}, - {3292, {wxStyledTextCtrl, getMouseDownCaptures, 0}}, - {3293, {wxStyledTextCtrl, setSTCCursor, 1}}, - {3294, {wxStyledTextCtrl, getSTCCursor, 0}}, - {3295, {wxStyledTextCtrl, setControlCharSymbol, 1}}, - {3296, {wxStyledTextCtrl, getControlCharSymbol, 0}}, - {3297, {wxStyledTextCtrl, wordPartLeft, 0}}, - {3298, {wxStyledTextCtrl, wordPartLeftExtend, 0}}, - {3299, {wxStyledTextCtrl, wordPartRight, 0}}, - {3300, {wxStyledTextCtrl, wordPartRightExtend, 0}}, - {3301, {wxStyledTextCtrl, setVisiblePolicy, 2}}, - {3302, {wxStyledTextCtrl, delLineLeft, 0}}, - {3303, {wxStyledTextCtrl, delLineRight, 0}}, - {3304, {wxStyledTextCtrl, getXOffset, 0}}, - {3305, {wxStyledTextCtrl, chooseCaretX, 0}}, - {3306, {wxStyledTextCtrl, setXCaretPolicy, 2}}, - {3307, {wxStyledTextCtrl, setYCaretPolicy, 2}}, - {3308, {wxStyledTextCtrl, getPrintWrapMode, 0}}, - {3309, {wxStyledTextCtrl, setHotspotActiveForeground, 2}}, - {3310, {wxStyledTextCtrl, setHotspotActiveBackground, 2}}, - {3311, {wxStyledTextCtrl, setHotspotActiveUnderline, 1}}, - {3312, {wxStyledTextCtrl, setHotspotSingleLine, 1}}, - {3313, {wxStyledTextCtrl, paraDownExtend, 0}}, - {3314, {wxStyledTextCtrl, paraUp, 0}}, - {3315, {wxStyledTextCtrl, paraUpExtend, 0}}, - {3316, {wxStyledTextCtrl, positionBefore, 1}}, - {3317, {wxStyledTextCtrl, positionAfter, 1}}, - {3318, {wxStyledTextCtrl, copyRange, 2}}, - {3319, {wxStyledTextCtrl, copyText, 2}}, - {3320, {wxStyledTextCtrl, setSelectionMode, 1}}, - {3321, {wxStyledTextCtrl, getSelectionMode, 0}}, - {3322, {wxStyledTextCtrl, lineDownRectExtend, 0}}, - {3323, {wxStyledTextCtrl, lineUpRectExtend, 0}}, - {3324, {wxStyledTextCtrl, charLeftRectExtend, 0}}, - {3325, {wxStyledTextCtrl, charRightRectExtend, 0}}, - {3326, {wxStyledTextCtrl, homeRectExtend, 0}}, - {3327, {wxStyledTextCtrl, vCHomeRectExtend, 0}}, - {3328, {wxStyledTextCtrl, lineEndRectExtend, 0}}, - {3329, {wxStyledTextCtrl, pageUpRectExtend, 0}}, - {3330, {wxStyledTextCtrl, pageDownRectExtend, 0}}, - {3331, {wxStyledTextCtrl, stutteredPageUp, 0}}, - {3332, {wxStyledTextCtrl, stutteredPageUpExtend, 0}}, - {3333, {wxStyledTextCtrl, stutteredPageDown, 0}}, - {3334, {wxStyledTextCtrl, stutteredPageDownExtend, 0}}, - {3335, {wxStyledTextCtrl, wordLeftEnd, 0}}, - {3336, {wxStyledTextCtrl, wordLeftEndExtend, 0}}, - {3337, {wxStyledTextCtrl, wordRightEnd, 0}}, - {3338, {wxStyledTextCtrl, wordRightEndExtend, 0}}, - {3339, {wxStyledTextCtrl, setWhitespaceChars, 1}}, - {3340, {wxStyledTextCtrl, setCharsDefault, 0}}, - {3341, {wxStyledTextCtrl, autoCompGetCurrent, 0}}, - {3342, {wxStyledTextCtrl, allocate, 1}}, - {3343, {wxStyledTextCtrl, findColumn, 2}}, - {3344, {wxStyledTextCtrl, getCaretSticky, 0}}, - {3345, {wxStyledTextCtrl, setCaretSticky, 1}}, - {3346, {wxStyledTextCtrl, toggleCaretSticky, 0}}, - {3347, {wxStyledTextCtrl, setPasteConvertEndings, 1}}, - {3348, {wxStyledTextCtrl, getPasteConvertEndings, 0}}, - {3349, {wxStyledTextCtrl, selectionDuplicate, 0}}, - {3350, {wxStyledTextCtrl, setCaretLineBackAlpha, 1}}, - {3351, {wxStyledTextCtrl, getCaretLineBackAlpha, 0}}, - {3352, {wxStyledTextCtrl, startRecord, 0}}, - {3353, {wxStyledTextCtrl, stopRecord, 0}}, - {3354, {wxStyledTextCtrl, setLexer, 1}}, - {3355, {wxStyledTextCtrl, getLexer, 0}}, - {3356, {wxStyledTextCtrl, colourise, 2}}, - {3357, {wxStyledTextCtrl, setProperty, 2}}, - {3358, {wxStyledTextCtrl, setKeyWords, 2}}, - {3359, {wxStyledTextCtrl, setLexerLanguage, 1}}, - {3360, {wxStyledTextCtrl, getProperty, 1}}, - {3361, {wxStyledTextCtrl, getStyleBitsNeeded, 0}}, - {3362, {wxStyledTextCtrl, getCurrentLine, 0}}, - {3363, {wxStyledTextCtrl, styleSetSpec, 2}}, - {3364, {wxStyledTextCtrl, styleSetFont, 2}}, - {3365, {wxStyledTextCtrl, styleSetFontAttr, 7}}, - {3366, {wxStyledTextCtrl, styleSetCharacterSet, 2}}, - {3367, {wxStyledTextCtrl, styleSetFontEncoding, 2}}, - {3368, {wxStyledTextCtrl, cmdKeyExecute, 1}}, - {3369, {wxStyledTextCtrl, setMargins, 2}}, - {3370, {wxStyledTextCtrl, getSelection, 2}}, - {3371, {wxStyledTextCtrl, pointFromPosition, 1}}, - {3372, {wxStyledTextCtrl, scrollToLine, 1}}, - {3373, {wxStyledTextCtrl, scrollToColumn, 1}}, - {3374, {wxStyledTextCtrl, setVScrollBar, 1}}, - {3375, {wxStyledTextCtrl, setHScrollBar, 1}}, - {3376, {wxStyledTextCtrl, getLastKeydownProcessed, 0}}, - {3377, {wxStyledTextCtrl, setLastKeydownProcessed, 1}}, - {3378, {wxStyledTextCtrl, saveFile, 1}}, - {3379, {wxStyledTextCtrl, loadFile, 1}}, - {3380, {wxStyledTextCtrl, doDragOver, 3}}, - {3381, {wxStyledTextCtrl, doDropText, 3}}, - {3382, {wxStyledTextCtrl, getUseAntiAliasing, 0}}, - {3383, {wxStyledTextCtrl, addTextRaw, 1}}, - {3384, {wxStyledTextCtrl, insertTextRaw, 2}}, - {3385, {wxStyledTextCtrl, getCurLineRaw, 1}}, - {3386, {wxStyledTextCtrl, getLineRaw, 1}}, - {3387, {wxStyledTextCtrl, getSelectedTextRaw, 0}}, - {3388, {wxStyledTextCtrl, getTextRangeRaw, 2}}, - {3389, {wxStyledTextCtrl, setTextRaw, 1}}, - {3390, {wxStyledTextCtrl, getTextRaw, 0}}, - {3391, {wxStyledTextCtrl, appendTextRaw, 1}}, - {3392, {wxArtProvider, getBitmap, 2}}, - {3393, {wxArtProvider, getIcon, 2}}, - {3394, {wxTreeEvent, getKeyCode, 0}}, - {3395, {wxTreeEvent, getItem, 0}}, - {3396, {wxTreeEvent, getKeyEvent, 0}}, - {3397, {wxTreeEvent, getLabel, 0}}, - {3398, {wxTreeEvent, getOldItem, 0}}, - {3399, {wxTreeEvent, getPoint, 0}}, - {3400, {wxTreeEvent, isEditCancelled, 0}}, - {3401, {wxTreeEvent, setToolTip, 1}}, - {3402, {wxNotebookEvent, getOldSelection, 0}}, - {3403, {wxNotebookEvent, getSelection, 0}}, - {3404, {wxNotebookEvent, setOldSelection, 1}}, - {3405, {wxNotebookEvent, setSelection, 1}}, - {3406, {wxFileDataObject, new, 0}}, - {3407, {wxFileDataObject, addFile, 1}}, - {3408, {wxFileDataObject, getFilenames, 0}}, - {3409, {wxFileDataObject, 'Destroy', undefined}}, - {3410, {wxTextDataObject, new, 1}}, - {3411, {wxTextDataObject, getTextLength, 0}}, - {3412, {wxTextDataObject, getText, 0}}, - {3413, {wxTextDataObject, setText, 1}}, - {3414, {wxTextDataObject, 'Destroy', undefined}}, - {3415, {wxBitmapDataObject, new_1_1, 1}}, - {3416, {wxBitmapDataObject, new_1_0, 1}}, - {3417, {wxBitmapDataObject, getBitmap, 0}}, - {3418, {wxBitmapDataObject, setBitmap, 1}}, - {3419, {wxBitmapDataObject, 'Destroy', undefined}}, - {3421, {wxClipboard, new, 0}}, - {3422, {wxClipboard, destruct, 0}}, - {3423, {wxClipboard, addData, 1}}, - {3424, {wxClipboard, clear, 0}}, - {3425, {wxClipboard, close, 0}}, - {3426, {wxClipboard, flush, 0}}, - {3427, {wxClipboard, getData, 1}}, - {3428, {wxClipboard, isOpened, 0}}, - {3429, {wxClipboard, open, 0}}, - {3430, {wxClipboard, setData, 1}}, - {3432, {wxClipboard, usePrimarySelection, 1}}, - {3433, {wxClipboard, isSupported, 1}}, - {3434, {wxClipboard, get, 0}}, - {3435, {wxSpinEvent, getPosition, 0}}, - {3436, {wxSpinEvent, setPosition, 1}}, - {3437, {wxSplitterWindow, new_0, 0}}, - {3438, {wxSplitterWindow, new_2, 2}}, - {3439, {wxSplitterWindow, destruct, 0}}, - {3440, {wxSplitterWindow, create, 2}}, - {3441, {wxSplitterWindow, getMinimumPaneSize, 0}}, - {3442, {wxSplitterWindow, getSashGravity, 0}}, - {3443, {wxSplitterWindow, getSashPosition, 0}}, - {3444, {wxSplitterWindow, getSplitMode, 0}}, - {3445, {wxSplitterWindow, getWindow1, 0}}, - {3446, {wxSplitterWindow, getWindow2, 0}}, - {3447, {wxSplitterWindow, initialize, 1}}, - {3448, {wxSplitterWindow, isSplit, 0}}, - {3449, {wxSplitterWindow, replaceWindow, 2}}, - {3450, {wxSplitterWindow, setSashGravity, 1}}, - {3451, {wxSplitterWindow, setSashPosition, 2}}, - {3452, {wxSplitterWindow, setSashSize, 1}}, - {3453, {wxSplitterWindow, setMinimumPaneSize, 1}}, - {3454, {wxSplitterWindow, setSplitMode, 1}}, - {3455, {wxSplitterWindow, splitHorizontally, 3}}, - {3456, {wxSplitterWindow, splitVertically, 3}}, - {3457, {wxSplitterWindow, unsplit, 1}}, - {3458, {wxSplitterWindow, updateSize, 0}}, - {3459, {wxSplitterEvent, getSashPosition, 0}}, - {3460, {wxSplitterEvent, getX, 0}}, - {3461, {wxSplitterEvent, getY, 0}}, - {3462, {wxSplitterEvent, getWindowBeingRemoved, 0}}, - {3463, {wxSplitterEvent, setSashPosition, 1}}, - {3464, {wxHtmlWindow, new_0, 0}}, - {3465, {wxHtmlWindow, new_2, 2}}, - {3466, {wxHtmlWindow, appendToPage, 1}}, - {3467, {wxHtmlWindow, getOpenedAnchor, 0}}, - {3468, {wxHtmlWindow, getOpenedPage, 0}}, - {3469, {wxHtmlWindow, getOpenedPageTitle, 0}}, - {3470, {wxHtmlWindow, getRelatedFrame, 0}}, - {3471, {wxHtmlWindow, historyBack, 0}}, - {3472, {wxHtmlWindow, historyCanBack, 0}}, - {3473, {wxHtmlWindow, historyCanForward, 0}}, - {3474, {wxHtmlWindow, historyClear, 0}}, - {3475, {wxHtmlWindow, historyForward, 0}}, - {3476, {wxHtmlWindow, loadFile, 1}}, - {3477, {wxHtmlWindow, loadPage, 1}}, - {3478, {wxHtmlWindow, selectAll, 0}}, - {3479, {wxHtmlWindow, selectionToText, 0}}, - {3480, {wxHtmlWindow, selectLine, 1}}, - {3481, {wxHtmlWindow, selectWord, 1}}, - {3482, {wxHtmlWindow, setBorders, 1}}, - {3483, {wxHtmlWindow, setFonts, 3}}, - {3484, {wxHtmlWindow, setPage, 1}}, - {3485, {wxHtmlWindow, setRelatedFrame, 2}}, - {3486, {wxHtmlWindow, setRelatedStatusBar, 1}}, - {3487, {wxHtmlWindow, toText, 0}}, - {3488, {wxHtmlWindow, 'Destroy', undefined}}, - {3489, {wxHtmlLinkEvent, getLinkInfo, 0}}, - {3490, {wxSystemSettings, getColour, 1}}, - {3491, {wxSystemSettings, getFont, 1}}, - {3492, {wxSystemSettings, getMetric, 2}}, - {3493, {wxSystemSettings, getScreenType, 0}}, - {3494, {wxSystemOptions, getOption, 1}}, - {3495, {wxSystemOptions, getOptionInt, 1}}, - {3496, {wxSystemOptions, hasOption, 1}}, - {3497, {wxSystemOptions, isFalse, 1}}, - {3498, {wxSystemOptions, setOption_2_1, 2}}, - {3499, {wxSystemOptions, setOption_2_0, 2}}, - {3500, {wxAuiNotebookEvent, setSelection, 1}}, - {3501, {wxAuiNotebookEvent, getSelection, 0}}, - {3502, {wxAuiNotebookEvent, setOldSelection, 1}}, - {3503, {wxAuiNotebookEvent, getOldSelection, 0}}, - {3504, {wxAuiNotebookEvent, setDragSource, 1}}, - {3505, {wxAuiNotebookEvent, getDragSource, 0}}, - {3506, {wxAuiManagerEvent, setManager, 1}}, - {3507, {wxAuiManagerEvent, getManager, 0}}, - {3508, {wxAuiManagerEvent, setPane, 1}}, - {3509, {wxAuiManagerEvent, getPane, 0}}, - {3510, {wxAuiManagerEvent, setButton, 1}}, - {3511, {wxAuiManagerEvent, getButton, 0}}, - {3512, {wxAuiManagerEvent, setDC, 1}}, - {3513, {wxAuiManagerEvent, getDC, 0}}, - {3514, {wxAuiManagerEvent, veto, 1}}, - {3515, {wxAuiManagerEvent, getVeto, 0}}, - {3516, {wxAuiManagerEvent, setCanVeto, 1}}, - {3517, {wxAuiManagerEvent, canVeto, 0}}, - {3518, {wxLogNull, new, 0}}, - {3519, {wxLogNull, 'Destroy', undefined}}, - {3520, {wxTaskBarIcon, new, 0}}, - {3521, {wxTaskBarIcon, destruct, 0}}, - {3522, {wxTaskBarIcon, popupMenu, 1}}, - {3523, {wxTaskBarIcon, removeIcon, 0}}, - {3524, {wxTaskBarIcon, setIcon, 2}}, + {1827, {wxTextCtrl, changeValue, 1}}, + {1828, {wxTextCtrl, emulateKeyPress, 1}}, + {1829, {wxTextCtrl, getDefaultStyle, 0}}, + {1830, {wxTextCtrl, getInsertionPoint, 0}}, + {1831, {wxTextCtrl, getLastPosition, 0}}, + {1832, {wxTextCtrl, getLineLength, 1}}, + {1833, {wxTextCtrl, getLineText, 1}}, + {1834, {wxTextCtrl, getNumberOfLines, 0}}, + {1835, {wxTextCtrl, getRange, 2}}, + {1836, {wxTextCtrl, getSelection, 2}}, + {1837, {wxTextCtrl, getStringSelection, 0}}, + {1838, {wxTextCtrl, getStyle, 2}}, + {1839, {wxTextCtrl, getValue, 0}}, + {1840, {wxTextCtrl, isEditable, 0}}, + {1841, {wxTextCtrl, isModified, 0}}, + {1842, {wxTextCtrl, isMultiLine, 0}}, + {1843, {wxTextCtrl, isSingleLine, 0}}, + {1844, {wxTextCtrl, loadFile, 2}}, + {1845, {wxTextCtrl, markDirty, 0}}, + {1846, {wxTextCtrl, paste, 0}}, + {1847, {wxTextCtrl, positionToXY, 3}}, + {1848, {wxTextCtrl, redo, 0}}, + {1849, {wxTextCtrl, remove, 2}}, + {1850, {wxTextCtrl, replace, 3}}, + {1851, {wxTextCtrl, saveFile, 1}}, + {1852, {wxTextCtrl, setDefaultStyle, 1}}, + {1853, {wxTextCtrl, setEditable, 1}}, + {1854, {wxTextCtrl, setInsertionPoint, 1}}, + {1855, {wxTextCtrl, setInsertionPointEnd, 0}}, + {1857, {wxTextCtrl, setMaxLength, 1}}, + {1858, {wxTextCtrl, setSelection, 2}}, + {1859, {wxTextCtrl, setStyle, 3}}, + {1860, {wxTextCtrl, setValue, 1}}, + {1861, {wxTextCtrl, showPosition, 1}}, + {1862, {wxTextCtrl, undo, 0}}, + {1863, {wxTextCtrl, writeText, 1}}, + {1864, {wxTextCtrl, xYToPosition, 2}}, + {1867, {wxNotebook, new_0, 0}}, + {1868, {wxNotebook, new_3, 3}}, + {1869, {wxNotebook, destruct, 0}}, + {1870, {wxNotebook, addPage, 3}}, + {1871, {wxNotebook, advanceSelection, 1}}, + {1872, {wxNotebook, assignImageList, 1}}, + {1873, {wxNotebook, create, 3}}, + {1874, {wxNotebook, deleteAllPages, 0}}, + {1875, {wxNotebook, deletePage, 1}}, + {1876, {wxNotebook, removePage, 1}}, + {1877, {wxNotebook, getCurrentPage, 0}}, + {1878, {wxNotebook, getImageList, 0}}, + {1880, {wxNotebook, getPage, 1}}, + {1881, {wxNotebook, getPageCount, 0}}, + {1882, {wxNotebook, getPageImage, 1}}, + {1883, {wxNotebook, getPageText, 1}}, + {1884, {wxNotebook, getRowCount, 0}}, + {1885, {wxNotebook, getSelection, 0}}, + {1886, {wxNotebook, getThemeBackgroundColour, 0}}, + {1888, {wxNotebook, hitTest, 2}}, + {1890, {wxNotebook, insertPage, 4}}, + {1891, {wxNotebook, setImageList, 1}}, + {1892, {wxNotebook, setPadding, 1}}, + {1893, {wxNotebook, setPageSize, 1}}, + {1894, {wxNotebook, setPageImage, 2}}, + {1895, {wxNotebook, setPageText, 2}}, + {1896, {wxNotebook, setSelection, 1}}, + {1897, {wxNotebook, changeSelection, 1}}, + {1898, {wxChoicebook, new_0, 0}}, + {1899, {wxChoicebook, new_3, 3}}, + {1900, {wxChoicebook, addPage, 3}}, + {1901, {wxChoicebook, advanceSelection, 1}}, + {1902, {wxChoicebook, assignImageList, 1}}, + {1903, {wxChoicebook, create, 3}}, + {1904, {wxChoicebook, deleteAllPages, 0}}, + {1905, {wxChoicebook, deletePage, 1}}, + {1906, {wxChoicebook, removePage, 1}}, + {1907, {wxChoicebook, getCurrentPage, 0}}, + {1908, {wxChoicebook, getImageList, 0}}, + {1910, {wxChoicebook, getPage, 1}}, + {1911, {wxChoicebook, getPageCount, 0}}, + {1912, {wxChoicebook, getPageImage, 1}}, + {1913, {wxChoicebook, getPageText, 1}}, + {1914, {wxChoicebook, getSelection, 0}}, + {1915, {wxChoicebook, hitTest, 2}}, + {1916, {wxChoicebook, insertPage, 4}}, + {1917, {wxChoicebook, setImageList, 1}}, + {1918, {wxChoicebook, setPageSize, 1}}, + {1919, {wxChoicebook, setPageImage, 2}}, + {1920, {wxChoicebook, setPageText, 2}}, + {1921, {wxChoicebook, setSelection, 1}}, + {1922, {wxChoicebook, changeSelection, 1}}, + {1923, {wxChoicebook, 'Destroy', undefined}}, + {1924, {wxToolbook, new_0, 0}}, + {1925, {wxToolbook, new_3, 3}}, + {1926, {wxToolbook, addPage, 3}}, + {1927, {wxToolbook, advanceSelection, 1}}, + {1928, {wxToolbook, assignImageList, 1}}, + {1929, {wxToolbook, create, 3}}, + {1930, {wxToolbook, deleteAllPages, 0}}, + {1931, {wxToolbook, deletePage, 1}}, + {1932, {wxToolbook, removePage, 1}}, + {1933, {wxToolbook, getCurrentPage, 0}}, + {1934, {wxToolbook, getImageList, 0}}, + {1936, {wxToolbook, getPage, 1}}, + {1937, {wxToolbook, getPageCount, 0}}, + {1938, {wxToolbook, getPageImage, 1}}, + {1939, {wxToolbook, getPageText, 1}}, + {1940, {wxToolbook, getSelection, 0}}, + {1942, {wxToolbook, hitTest, 2}}, + {1943, {wxToolbook, insertPage, 4}}, + {1944, {wxToolbook, setImageList, 1}}, + {1945, {wxToolbook, setPageSize, 1}}, + {1946, {wxToolbook, setPageImage, 2}}, + {1947, {wxToolbook, setPageText, 2}}, + {1948, {wxToolbook, setSelection, 1}}, + {1949, {wxToolbook, changeSelection, 1}}, + {1950, {wxToolbook, 'Destroy', undefined}}, + {1951, {wxListbook, new_0, 0}}, + {1952, {wxListbook, new_3, 3}}, + {1953, {wxListbook, addPage, 3}}, + {1954, {wxListbook, advanceSelection, 1}}, + {1955, {wxListbook, assignImageList, 1}}, + {1956, {wxListbook, create, 3}}, + {1957, {wxListbook, deleteAllPages, 0}}, + {1958, {wxListbook, deletePage, 1}}, + {1959, {wxListbook, removePage, 1}}, + {1960, {wxListbook, getCurrentPage, 0}}, + {1961, {wxListbook, getImageList, 0}}, + {1963, {wxListbook, getPage, 1}}, + {1964, {wxListbook, getPageCount, 0}}, + {1965, {wxListbook, getPageImage, 1}}, + {1966, {wxListbook, getPageText, 1}}, + {1967, {wxListbook, getSelection, 0}}, + {1969, {wxListbook, hitTest, 2}}, + {1970, {wxListbook, insertPage, 4}}, + {1971, {wxListbook, setImageList, 1}}, + {1972, {wxListbook, setPageSize, 1}}, + {1973, {wxListbook, setPageImage, 2}}, + {1974, {wxListbook, setPageText, 2}}, + {1975, {wxListbook, setSelection, 1}}, + {1976, {wxListbook, changeSelection, 1}}, + {1977, {wxListbook, 'Destroy', undefined}}, + {1978, {wxTreebook, new_0, 0}}, + {1979, {wxTreebook, new_3, 3}}, + {1980, {wxTreebook, addPage, 3}}, + {1981, {wxTreebook, advanceSelection, 1}}, + {1982, {wxTreebook, assignImageList, 1}}, + {1983, {wxTreebook, create, 3}}, + {1984, {wxTreebook, deleteAllPages, 0}}, + {1985, {wxTreebook, deletePage, 1}}, + {1986, {wxTreebook, removePage, 1}}, + {1987, {wxTreebook, getCurrentPage, 0}}, + {1988, {wxTreebook, getImageList, 0}}, + {1990, {wxTreebook, getPage, 1}}, + {1991, {wxTreebook, getPageCount, 0}}, + {1992, {wxTreebook, getPageImage, 1}}, + {1993, {wxTreebook, getPageText, 1}}, + {1994, {wxTreebook, getSelection, 0}}, + {1995, {wxTreebook, expandNode, 2}}, + {1996, {wxTreebook, isNodeExpanded, 1}}, + {1998, {wxTreebook, hitTest, 2}}, + {1999, {wxTreebook, insertPage, 4}}, + {2000, {wxTreebook, insertSubPage, 4}}, + {2001, {wxTreebook, setImageList, 1}}, + {2002, {wxTreebook, setPageSize, 1}}, + {2003, {wxTreebook, setPageImage, 2}}, + {2004, {wxTreebook, setPageText, 2}}, + {2005, {wxTreebook, setSelection, 1}}, + {2006, {wxTreebook, changeSelection, 1}}, + {2007, {wxTreebook, 'Destroy', undefined}}, + {2010, {wxTreeCtrl, new_2, 2}}, + {2011, {wxTreeCtrl, new_0, 0}}, + {2013, {wxTreeCtrl, destruct, 0}}, + {2014, {wxTreeCtrl, addRoot, 2}}, + {2015, {wxTreeCtrl, appendItem, 3}}, + {2016, {wxTreeCtrl, assignImageList, 1}}, + {2017, {wxTreeCtrl, assignStateImageList, 1}}, + {2018, {wxTreeCtrl, collapse, 1}}, + {2019, {wxTreeCtrl, collapseAndReset, 1}}, + {2020, {wxTreeCtrl, create, 2}}, + {2021, {wxTreeCtrl, delete, 1}}, + {2022, {wxTreeCtrl, deleteAllItems, 0}}, + {2023, {wxTreeCtrl, deleteChildren, 1}}, + {2024, {wxTreeCtrl, editLabel, 1}}, + {2025, {wxTreeCtrl, ensureVisible, 1}}, + {2026, {wxTreeCtrl, expand, 1}}, + {2027, {wxTreeCtrl, getBoundingRect, 3}}, + {2029, {wxTreeCtrl, getChildrenCount, 2}}, + {2030, {wxTreeCtrl, getCount, 0}}, + {2031, {wxTreeCtrl, getEditControl, 0}}, + {2032, {wxTreeCtrl, getFirstChild, 2}}, + {2033, {wxTreeCtrl, getNextChild, 2}}, + {2034, {wxTreeCtrl, getFirstVisibleItem, 0}}, + {2035, {wxTreeCtrl, getImageList, 0}}, + {2036, {wxTreeCtrl, getIndent, 0}}, + {2037, {wxTreeCtrl, getItemBackgroundColour, 1}}, + {2038, {wxTreeCtrl, getItemData, 1}}, + {2039, {wxTreeCtrl, getItemFont, 1}}, + {2040, {wxTreeCtrl, getItemImage_1, 1}}, + {2041, {wxTreeCtrl, getItemImage_2, 2}}, + {2042, {wxTreeCtrl, getItemText, 1}}, + {2043, {wxTreeCtrl, getItemTextColour, 1}}, + {2044, {wxTreeCtrl, getLastChild, 1}}, + {2045, {wxTreeCtrl, getNextSibling, 1}}, + {2046, {wxTreeCtrl, getNextVisible, 1}}, + {2047, {wxTreeCtrl, getItemParent, 1}}, + {2048, {wxTreeCtrl, getPrevSibling, 1}}, + {2049, {wxTreeCtrl, getPrevVisible, 1}}, + {2050, {wxTreeCtrl, getRootItem, 0}}, + {2051, {wxTreeCtrl, getSelection, 0}}, + {2052, {wxTreeCtrl, getSelections, 1}}, + {2053, {wxTreeCtrl, getStateImageList, 0}}, + {2054, {wxTreeCtrl, hitTest, 2}}, + {2056, {wxTreeCtrl, insertItem, 4}}, + {2057, {wxTreeCtrl, isBold, 1}}, + {2058, {wxTreeCtrl, isExpanded, 1}}, + {2059, {wxTreeCtrl, isSelected, 1}}, + {2060, {wxTreeCtrl, isVisible, 1}}, + {2061, {wxTreeCtrl, itemHasChildren, 1}}, + {2062, {wxTreeCtrl, isTreeItemIdOk, 1}}, + {2063, {wxTreeCtrl, prependItem, 3}}, + {2064, {wxTreeCtrl, scrollTo, 1}}, + {2065, {wxTreeCtrl, selectItem_1, 1}}, + {2066, {wxTreeCtrl, selectItem_2, 2}}, + {2067, {wxTreeCtrl, setIndent, 1}}, + {2068, {wxTreeCtrl, setImageList, 1}}, + {2069, {wxTreeCtrl, setItemBackgroundColour, 2}}, + {2070, {wxTreeCtrl, setItemBold, 2}}, + {2071, {wxTreeCtrl, setItemData, 2}}, + {2072, {wxTreeCtrl, setItemDropHighlight, 2}}, + {2073, {wxTreeCtrl, setItemFont, 2}}, + {2074, {wxTreeCtrl, setItemHasChildren, 2}}, + {2075, {wxTreeCtrl, setItemImage_2, 2}}, + {2076, {wxTreeCtrl, setItemImage_3, 3}}, + {2077, {wxTreeCtrl, setItemText, 2}}, + {2078, {wxTreeCtrl, setItemTextColour, 2}}, + {2079, {wxTreeCtrl, setStateImageList, 1}}, + {2080, {wxTreeCtrl, setWindowStyle, 1}}, + {2081, {wxTreeCtrl, sortChildren, 1}}, + {2082, {wxTreeCtrl, toggle, 1}}, + {2083, {wxTreeCtrl, toggleItemSelection, 1}}, + {2084, {wxTreeCtrl, unselect, 0}}, + {2085, {wxTreeCtrl, unselectAll, 0}}, + {2086, {wxTreeCtrl, unselectItem, 1}}, + {2087, {wxScrollBar, new_0, 0}}, + {2088, {wxScrollBar, new_3, 3}}, + {2089, {wxScrollBar, destruct, 0}}, + {2090, {wxScrollBar, create, 3}}, + {2091, {wxScrollBar, getRange, 0}}, + {2092, {wxScrollBar, getPageSize, 0}}, + {2093, {wxScrollBar, getThumbPosition, 0}}, + {2094, {wxScrollBar, getThumbSize, 0}}, + {2095, {wxScrollBar, setThumbPosition, 1}}, + {2096, {wxScrollBar, setScrollbar, 5}}, + {2098, {wxSpinButton, new_2, 2}}, + {2099, {wxSpinButton, new_0, 0}}, + {2100, {wxSpinButton, create, 2}}, + {2101, {wxSpinButton, getMax, 0}}, + {2102, {wxSpinButton, getMin, 0}}, + {2103, {wxSpinButton, getValue, 0}}, + {2104, {wxSpinButton, setRange, 2}}, + {2105, {wxSpinButton, setValue, 1}}, + {2106, {wxSpinButton, 'Destroy', undefined}}, + {2107, {wxSpinCtrl, new_0, 0}}, + {2108, {wxSpinCtrl, new_2, 2}}, + {2110, {wxSpinCtrl, create, 2}}, + {2113, {wxSpinCtrl, setValue_1_1, 1}}, + {2114, {wxSpinCtrl, setValue_1_0, 1}}, + {2116, {wxSpinCtrl, getValue, 0}}, + {2118, {wxSpinCtrl, setRange, 2}}, + {2119, {wxSpinCtrl, setSelection, 2}}, + {2121, {wxSpinCtrl, getMin, 0}}, + {2123, {wxSpinCtrl, getMax, 0}}, + {2124, {wxSpinCtrl, 'Destroy', undefined}}, + {2125, {wxStaticText, new_0, 0}}, + {2126, {wxStaticText, new_4, 4}}, + {2127, {wxStaticText, create, 4}}, + {2128, {wxStaticText, getLabel, 0}}, + {2129, {wxStaticText, setLabel, 1}}, + {2130, {wxStaticText, wrap, 1}}, + {2131, {wxStaticText, 'Destroy', undefined}}, + {2132, {wxStaticBitmap, new_0, 0}}, + {2133, {wxStaticBitmap, new_4, 4}}, + {2134, {wxStaticBitmap, create, 4}}, + {2135, {wxStaticBitmap, getBitmap, 0}}, + {2136, {wxStaticBitmap, setBitmap, 1}}, + {2137, {wxStaticBitmap, 'Destroy', undefined}}, + {2138, {wxRadioBox, new, 7}}, + {2140, {wxRadioBox, destruct, 0}}, + {2141, {wxRadioBox, create, 7}}, + {2142, {wxRadioBox, enable_2, 2}}, + {2143, {wxRadioBox, enable_1, 1}}, + {2144, {wxRadioBox, getSelection, 0}}, + {2145, {wxRadioBox, getString, 1}}, + {2146, {wxRadioBox, setSelection, 1}}, + {2147, {wxRadioBox, show_2, 2}}, + {2148, {wxRadioBox, show_1, 1}}, + {2149, {wxRadioBox, getColumnCount, 0}}, + {2150, {wxRadioBox, getItemHelpText, 1}}, + {2151, {wxRadioBox, getItemToolTip, 1}}, + {2153, {wxRadioBox, getItemFromPoint, 1}}, + {2154, {wxRadioBox, getRowCount, 0}}, + {2155, {wxRadioBox, isItemEnabled, 1}}, + {2156, {wxRadioBox, isItemShown, 1}}, + {2157, {wxRadioBox, setItemHelpText, 2}}, + {2158, {wxRadioBox, setItemToolTip, 2}}, + {2159, {wxRadioButton, new_0, 0}}, + {2160, {wxRadioButton, new_4, 4}}, + {2161, {wxRadioButton, create, 4}}, + {2162, {wxRadioButton, getValue, 0}}, + {2163, {wxRadioButton, setValue, 1}}, + {2164, {wxRadioButton, 'Destroy', undefined}}, + {2166, {wxSlider, new_6, 6}}, + {2167, {wxSlider, new_0, 0}}, + {2168, {wxSlider, create, 6}}, + {2169, {wxSlider, getLineSize, 0}}, + {2170, {wxSlider, getMax, 0}}, + {2171, {wxSlider, getMin, 0}}, + {2172, {wxSlider, getPageSize, 0}}, + {2173, {wxSlider, getThumbLength, 0}}, + {2174, {wxSlider, getValue, 0}}, + {2175, {wxSlider, setLineSize, 1}}, + {2176, {wxSlider, setPageSize, 1}}, + {2177, {wxSlider, setRange, 2}}, + {2178, {wxSlider, setThumbLength, 1}}, + {2179, {wxSlider, setValue, 1}}, + {2180, {wxSlider, 'Destroy', undefined}}, + {2182, {wxDialog, new_4, 4}}, + {2183, {wxDialog, new_0, 0}}, + {2185, {wxDialog, destruct, 0}}, + {2186, {wxDialog, create, 4}}, + {2187, {wxDialog, createButtonSizer, 1}}, + {2188, {wxDialog, createStdDialogButtonSizer, 1}}, + {2189, {wxDialog, endModal, 1}}, + {2190, {wxDialog, getAffirmativeId, 0}}, + {2191, {wxDialog, getReturnCode, 0}}, + {2192, {wxDialog, isModal, 0}}, + {2193, {wxDialog, setAffirmativeId, 1}}, + {2194, {wxDialog, setReturnCode, 1}}, + {2195, {wxDialog, show, 1}}, + {2196, {wxDialog, showModal, 0}}, + {2197, {wxColourDialog, new_0, 0}}, + {2198, {wxColourDialog, new_2, 2}}, + {2199, {wxColourDialog, destruct, 0}}, + {2200, {wxColourDialog, create, 2}}, + {2201, {wxColourDialog, getColourData, 0}}, + {2202, {wxColourData, new_0, 0}}, + {2203, {wxColourData, new_1, 1}}, + {2204, {wxColourData, destruct, 0}}, + {2205, {wxColourData, getChooseFull, 0}}, + {2206, {wxColourData, getColour, 0}}, + {2208, {wxColourData, getCustomColour, 1}}, + {2209, {wxColourData, setChooseFull, 1}}, + {2210, {wxColourData, setColour, 1}}, + {2211, {wxColourData, setCustomColour, 2}}, + {2212, {wxPalette, new_0, 0}}, + {2213, {wxPalette, new_4, 4}}, + {2215, {wxPalette, destruct, 0}}, + {2216, {wxPalette, create, 4}}, + {2217, {wxPalette, getColoursCount, 0}}, + {2218, {wxPalette, getPixel, 3}}, + {2219, {wxPalette, getRGB, 4}}, + {2220, {wxPalette, isOk, 0}}, + {2224, {wxDirDialog, new, 2}}, + {2225, {wxDirDialog, destruct, 0}}, + {2226, {wxDirDialog, getPath, 0}}, + {2227, {wxDirDialog, getMessage, 0}}, + {2228, {wxDirDialog, setMessage, 1}}, + {2229, {wxDirDialog, setPath, 1}}, + {2233, {wxFileDialog, new, 2}}, + {2234, {wxFileDialog, destruct, 0}}, + {2235, {wxFileDialog, getDirectory, 0}}, + {2236, {wxFileDialog, getFilename, 0}}, + {2237, {wxFileDialog, getFilenames, 1}}, + {2238, {wxFileDialog, getFilterIndex, 0}}, + {2239, {wxFileDialog, getMessage, 0}}, + {2240, {wxFileDialog, getPath, 0}}, + {2241, {wxFileDialog, getPaths, 1}}, + {2242, {wxFileDialog, getWildcard, 0}}, + {2243, {wxFileDialog, setDirectory, 1}}, + {2244, {wxFileDialog, setFilename, 1}}, + {2245, {wxFileDialog, setFilterIndex, 1}}, + {2246, {wxFileDialog, setMessage, 1}}, + {2247, {wxFileDialog, setPath, 1}}, + {2248, {wxFileDialog, setWildcard, 1}}, + {2249, {wxPickerBase, setInternalMargin, 1}}, + {2250, {wxPickerBase, getInternalMargin, 0}}, + {2251, {wxPickerBase, setTextCtrlProportion, 1}}, + {2252, {wxPickerBase, setPickerCtrlProportion, 1}}, + {2253, {wxPickerBase, getTextCtrlProportion, 0}}, + {2254, {wxPickerBase, getPickerCtrlProportion, 0}}, + {2255, {wxPickerBase, hasTextCtrl, 0}}, + {2256, {wxPickerBase, getTextCtrl, 0}}, + {2257, {wxPickerBase, isTextCtrlGrowable, 0}}, + {2258, {wxPickerBase, setPickerCtrlGrowable, 1}}, + {2259, {wxPickerBase, setTextCtrlGrowable, 1}}, + {2260, {wxPickerBase, isPickerCtrlGrowable, 0}}, + {2261, {wxFilePickerCtrl, new_0, 0}}, + {2262, {wxFilePickerCtrl, new_3, 3}}, + {2263, {wxFilePickerCtrl, create, 3}}, + {2264, {wxFilePickerCtrl, getPath, 0}}, + {2265, {wxFilePickerCtrl, setPath, 1}}, + {2266, {wxFilePickerCtrl, 'Destroy', undefined}}, + {2267, {wxDirPickerCtrl, new_0, 0}}, + {2268, {wxDirPickerCtrl, new_3, 3}}, + {2269, {wxDirPickerCtrl, create, 3}}, + {2270, {wxDirPickerCtrl, getPath, 0}}, + {2271, {wxDirPickerCtrl, setPath, 1}}, + {2272, {wxDirPickerCtrl, 'Destroy', undefined}}, + {2273, {wxColourPickerCtrl, new_0, 0}}, + {2274, {wxColourPickerCtrl, new_3, 3}}, + {2275, {wxColourPickerCtrl, create, 3}}, + {2276, {wxColourPickerCtrl, getColour, 0}}, + {2277, {wxColourPickerCtrl, setColour_1_1, 1}}, + {2278, {wxColourPickerCtrl, setColour_1_0, 1}}, + {2279, {wxColourPickerCtrl, 'Destroy', undefined}}, + {2280, {wxDatePickerCtrl, new_0, 0}}, + {2281, {wxDatePickerCtrl, new_3, 3}}, + {2282, {wxDatePickerCtrl, getRange, 2}}, + {2283, {wxDatePickerCtrl, getValue, 0}}, + {2284, {wxDatePickerCtrl, setRange, 2}}, + {2285, {wxDatePickerCtrl, setValue, 1}}, + {2286, {wxDatePickerCtrl, 'Destroy', undefined}}, + {2287, {wxFontPickerCtrl, new_0, 0}}, + {2288, {wxFontPickerCtrl, new_3, 3}}, + {2289, {wxFontPickerCtrl, create, 3}}, + {2290, {wxFontPickerCtrl, getSelectedFont, 0}}, + {2291, {wxFontPickerCtrl, setSelectedFont, 1}}, + {2292, {wxFontPickerCtrl, getMaxPointSize, 0}}, + {2293, {wxFontPickerCtrl, setMaxPointSize, 1}}, + {2294, {wxFontPickerCtrl, 'Destroy', undefined}}, + {2297, {wxFindReplaceDialog, new_0, 0}}, + {2298, {wxFindReplaceDialog, new_4, 4}}, + {2299, {wxFindReplaceDialog, destruct, 0}}, + {2300, {wxFindReplaceDialog, create, 4}}, + {2301, {wxFindReplaceDialog, getData, 0}}, + {2302, {wxFindReplaceData, new_0, 0}}, + {2303, {wxFindReplaceData, new_1, 1}}, + {2304, {wxFindReplaceData, getFindString, 0}}, + {2305, {wxFindReplaceData, getReplaceString, 0}}, + {2306, {wxFindReplaceData, getFlags, 0}}, + {2307, {wxFindReplaceData, setFlags, 1}}, + {2308, {wxFindReplaceData, setFindString, 1}}, + {2309, {wxFindReplaceData, setReplaceString, 1}}, + {2310, {wxFindReplaceData, 'Destroy', undefined}}, + {2311, {wxMultiChoiceDialog, new_0, 0}}, + {2313, {wxMultiChoiceDialog, new_5, 5}}, + {2314, {wxMultiChoiceDialog, getSelections, 0}}, + {2315, {wxMultiChoiceDialog, setSelections, 1}}, + {2316, {wxMultiChoiceDialog, 'Destroy', undefined}}, + {2317, {wxSingleChoiceDialog, new_0, 0}}, + {2319, {wxSingleChoiceDialog, new_5, 5}}, + {2320, {wxSingleChoiceDialog, getSelection, 0}}, + {2321, {wxSingleChoiceDialog, getStringSelection, 0}}, + {2322, {wxSingleChoiceDialog, setSelection, 1}}, + {2323, {wxSingleChoiceDialog, 'Destroy', undefined}}, + {2324, {wxTextEntryDialog, new, 3}}, + {2325, {wxTextEntryDialog, getValue, 0}}, + {2326, {wxTextEntryDialog, setValue, 1}}, + {2327, {wxTextEntryDialog, 'Destroy', undefined}}, + {2328, {wxPasswordEntryDialog, new, 3}}, + {2329, {wxPasswordEntryDialog, 'Destroy', undefined}}, + {2330, {wxFontData, new_0, 0}}, + {2331, {wxFontData, new_1, 1}}, + {2332, {wxFontData, destruct, 0}}, + {2333, {wxFontData, enableEffects, 1}}, + {2334, {wxFontData, getAllowSymbols, 0}}, + {2335, {wxFontData, getColour, 0}}, + {2336, {wxFontData, getChosenFont, 0}}, + {2337, {wxFontData, getEnableEffects, 0}}, + {2338, {wxFontData, getInitialFont, 0}}, + {2339, {wxFontData, getShowHelp, 0}}, + {2340, {wxFontData, setAllowSymbols, 1}}, + {2341, {wxFontData, setChosenFont, 1}}, + {2342, {wxFontData, setColour, 1}}, + {2343, {wxFontData, setInitialFont, 1}}, + {2344, {wxFontData, setRange, 2}}, + {2345, {wxFontData, setShowHelp, 1}}, + {2349, {wxFontDialog, new_0, 0}}, + {2351, {wxFontDialog, new_2, 2}}, + {2353, {wxFontDialog, create, 2}}, + {2354, {wxFontDialog, getFontData, 0}}, + {2356, {wxFontDialog, 'Destroy', undefined}}, + {2357, {wxProgressDialog, new, 3}}, + {2358, {wxProgressDialog, destruct, 0}}, + {2359, {wxProgressDialog, resume, 0}}, + {2360, {wxProgressDialog, update_2, 2}}, + {2361, {wxProgressDialog, update_0, 0}}, + {2362, {wxMessageDialog, new, 3}}, + {2363, {wxMessageDialog, destruct, 0}}, + {2364, {wxPageSetupDialog, new, 2}}, + {2365, {wxPageSetupDialog, destruct, 0}}, + {2366, {wxPageSetupDialog, getPageSetupData, 0}}, + {2367, {wxPageSetupDialog, showModal, 0}}, + {2368, {wxPageSetupDialogData, new_0, 0}}, + {2369, {wxPageSetupDialogData, new_1_0, 1}}, + {2370, {wxPageSetupDialogData, new_1_1, 1}}, + {2371, {wxPageSetupDialogData, destruct, 0}}, + {2372, {wxPageSetupDialogData, enableHelp, 1}}, + {2373, {wxPageSetupDialogData, enableMargins, 1}}, + {2374, {wxPageSetupDialogData, enableOrientation, 1}}, + {2375, {wxPageSetupDialogData, enablePaper, 1}}, + {2376, {wxPageSetupDialogData, enablePrinter, 1}}, + {2377, {wxPageSetupDialogData, getDefaultMinMargins, 0}}, + {2378, {wxPageSetupDialogData, getEnableMargins, 0}}, + {2379, {wxPageSetupDialogData, getEnableOrientation, 0}}, + {2380, {wxPageSetupDialogData, getEnablePaper, 0}}, + {2381, {wxPageSetupDialogData, getEnablePrinter, 0}}, + {2382, {wxPageSetupDialogData, getEnableHelp, 0}}, + {2383, {wxPageSetupDialogData, getDefaultInfo, 0}}, + {2384, {wxPageSetupDialogData, getMarginTopLeft, 0}}, + {2385, {wxPageSetupDialogData, getMarginBottomRight, 0}}, + {2386, {wxPageSetupDialogData, getMinMarginTopLeft, 0}}, + {2387, {wxPageSetupDialogData, getMinMarginBottomRight, 0}}, + {2388, {wxPageSetupDialogData, getPaperId, 0}}, + {2389, {wxPageSetupDialogData, getPaperSize, 0}}, + {2391, {wxPageSetupDialogData, getPrintData, 0}}, + {2392, {wxPageSetupDialogData, isOk, 0}}, + {2393, {wxPageSetupDialogData, setDefaultInfo, 1}}, + {2394, {wxPageSetupDialogData, setDefaultMinMargins, 1}}, + {2395, {wxPageSetupDialogData, setMarginTopLeft, 1}}, + {2396, {wxPageSetupDialogData, setMarginBottomRight, 1}}, + {2397, {wxPageSetupDialogData, setMinMarginTopLeft, 1}}, + {2398, {wxPageSetupDialogData, setMinMarginBottomRight, 1}}, + {2399, {wxPageSetupDialogData, setPaperId, 1}}, + {2400, {wxPageSetupDialogData, setPaperSize_1_1, 1}}, + {2401, {wxPageSetupDialogData, setPaperSize_1_0, 1}}, + {2402, {wxPageSetupDialogData, setPrintData, 1}}, + {2403, {wxPrintDialog, new_2_0, 2}}, + {2404, {wxPrintDialog, new_2_1, 2}}, + {2405, {wxPrintDialog, destruct, 0}}, + {2406, {wxPrintDialog, getPrintDialogData, 0}}, + {2407, {wxPrintDialog, getPrintDC, 0}}, + {2408, {wxPrintDialogData, new_0, 0}}, + {2409, {wxPrintDialogData, new_1_1, 1}}, + {2410, {wxPrintDialogData, new_1_0, 1}}, + {2411, {wxPrintDialogData, destruct, 0}}, + {2412, {wxPrintDialogData, enableHelp, 1}}, + {2413, {wxPrintDialogData, enablePageNumbers, 1}}, + {2414, {wxPrintDialogData, enablePrintToFile, 1}}, + {2415, {wxPrintDialogData, enableSelection, 1}}, + {2416, {wxPrintDialogData, getAllPages, 0}}, + {2417, {wxPrintDialogData, getCollate, 0}}, + {2418, {wxPrintDialogData, getFromPage, 0}}, + {2419, {wxPrintDialogData, getMaxPage, 0}}, + {2420, {wxPrintDialogData, getMinPage, 0}}, + {2421, {wxPrintDialogData, getNoCopies, 0}}, + {2422, {wxPrintDialogData, getPrintData, 0}}, + {2423, {wxPrintDialogData, getPrintToFile, 0}}, + {2424, {wxPrintDialogData, getSelection, 0}}, + {2425, {wxPrintDialogData, getToPage, 0}}, + {2426, {wxPrintDialogData, isOk, 0}}, + {2427, {wxPrintDialogData, setCollate, 1}}, + {2428, {wxPrintDialogData, setFromPage, 1}}, + {2429, {wxPrintDialogData, setMaxPage, 1}}, + {2430, {wxPrintDialogData, setMinPage, 1}}, + {2431, {wxPrintDialogData, setNoCopies, 1}}, + {2432, {wxPrintDialogData, setPrintData, 1}}, + {2433, {wxPrintDialogData, setPrintToFile, 1}}, + {2434, {wxPrintDialogData, setSelection, 1}}, + {2435, {wxPrintDialogData, setToPage, 1}}, + {2436, {wxPrintData, new_0, 0}}, + {2437, {wxPrintData, new_1, 1}}, + {2438, {wxPrintData, destruct, 0}}, + {2439, {wxPrintData, getCollate, 0}}, + {2440, {wxPrintData, getBin, 0}}, + {2441, {wxPrintData, getColour, 0}}, + {2442, {wxPrintData, getDuplex, 0}}, + {2443, {wxPrintData, getNoCopies, 0}}, + {2444, {wxPrintData, getOrientation, 0}}, + {2445, {wxPrintData, getPaperId, 0}}, + {2446, {wxPrintData, getPrinterName, 0}}, + {2447, {wxPrintData, getQuality, 0}}, + {2448, {wxPrintData, isOk, 0}}, + {2449, {wxPrintData, setBin, 1}}, + {2450, {wxPrintData, setCollate, 1}}, + {2451, {wxPrintData, setColour, 1}}, + {2452, {wxPrintData, setDuplex, 1}}, + {2453, {wxPrintData, setNoCopies, 1}}, + {2454, {wxPrintData, setOrientation, 1}}, + {2455, {wxPrintData, setPaperId, 1}}, + {2456, {wxPrintData, setPrinterName, 1}}, + {2457, {wxPrintData, setQuality, 1}}, + {2460, {wxPrintPreview, new_2, 2}}, + {2461, {wxPrintPreview, new_3, 3}}, + {2463, {wxPrintPreview, destruct, 0}}, + {2464, {wxPrintPreview, getCanvas, 0}}, + {2465, {wxPrintPreview, getCurrentPage, 0}}, + {2466, {wxPrintPreview, getFrame, 0}}, + {2467, {wxPrintPreview, getMaxPage, 0}}, + {2468, {wxPrintPreview, getMinPage, 0}}, + {2469, {wxPrintPreview, getPrintout, 0}}, + {2470, {wxPrintPreview, getPrintoutForPrinting, 0}}, + {2471, {wxPrintPreview, isOk, 0}}, + {2472, {wxPrintPreview, paintPage, 2}}, + {2473, {wxPrintPreview, print, 1}}, + {2474, {wxPrintPreview, renderPage, 1}}, + {2475, {wxPrintPreview, setCanvas, 1}}, + {2476, {wxPrintPreview, setCurrentPage, 1}}, + {2477, {wxPrintPreview, setFrame, 1}}, + {2478, {wxPrintPreview, setPrintout, 1}}, + {2479, {wxPrintPreview, setZoom, 1}}, + {2480, {wxPreviewFrame, new, 3}}, + {2481, {wxPreviewFrame, destruct, 0}}, + {2482, {wxPreviewFrame, createControlBar, 0}}, + {2483, {wxPreviewFrame, createCanvas, 0}}, + {2484, {wxPreviewFrame, initialize, 0}}, + {2485, {wxPreviewFrame, onCloseWindow, 1}}, + {2486, {wxPreviewControlBar, new, 4}}, + {2487, {wxPreviewControlBar, destruct, 0}}, + {2488, {wxPreviewControlBar, createButtons, 0}}, + {2489, {wxPreviewControlBar, getPrintPreview, 0}}, + {2490, {wxPreviewControlBar, getZoomControl, 0}}, + {2491, {wxPreviewControlBar, setZoomControl, 1}}, + {2493, {wxPrinter, new, 1}}, + {2494, {wxPrinter, createAbortWindow, 2}}, + {2495, {wxPrinter, getAbort, 0}}, + {2496, {wxPrinter, getLastError, 0}}, + {2497, {wxPrinter, getPrintDialogData, 0}}, + {2498, {wxPrinter, print, 3}}, + {2499, {wxPrinter, printDialog, 1}}, + {2500, {wxPrinter, reportError, 3}}, + {2501, {wxPrinter, setup, 1}}, + {2502, {wxPrinter, 'Destroy', undefined}}, + {2503, {wxXmlResource, new_1, 1}}, + {2504, {wxXmlResource, new_2, 2}}, + {2505, {wxXmlResource, destruct, 0}}, + {2506, {wxXmlResource, attachUnknownControl, 3}}, + {2507, {wxXmlResource, clearHandlers, 0}}, + {2508, {wxXmlResource, compareVersion, 4}}, + {2509, {wxXmlResource, get, 0}}, + {2510, {wxXmlResource, getFlags, 0}}, + {2511, {wxXmlResource, getVersion, 0}}, + {2512, {wxXmlResource, getXRCID, 2}}, + {2513, {wxXmlResource, initAllHandlers, 0}}, + {2514, {wxXmlResource, load, 1}}, + {2515, {wxXmlResource, loadBitmap, 1}}, + {2516, {wxXmlResource, loadDialog_2, 2}}, + {2517, {wxXmlResource, loadDialog_3, 3}}, + {2518, {wxXmlResource, loadFrame_2, 2}}, + {2519, {wxXmlResource, loadFrame_3, 3}}, + {2520, {wxXmlResource, loadIcon, 1}}, + {2521, {wxXmlResource, loadMenu, 1}}, + {2522, {wxXmlResource, loadMenuBar_2, 2}}, + {2523, {wxXmlResource, loadMenuBar_1, 1}}, + {2524, {wxXmlResource, loadPanel_2, 2}}, + {2525, {wxXmlResource, loadPanel_3, 3}}, + {2526, {wxXmlResource, loadToolBar, 2}}, + {2527, {wxXmlResource, set, 1}}, + {2528, {wxXmlResource, setFlags, 1}}, + {2529, {wxXmlResource, unload, 1}}, + {2530, {wxXmlResource, xrcctrl, 3}}, + {2531, {wxHtmlEasyPrinting, new, 1}}, + {2532, {wxHtmlEasyPrinting, destruct, 0}}, + {2533, {wxHtmlEasyPrinting, getPrintData, 0}}, + {2534, {wxHtmlEasyPrinting, getPageSetupData, 0}}, + {2535, {wxHtmlEasyPrinting, previewFile, 1}}, + {2536, {wxHtmlEasyPrinting, previewText, 2}}, + {2537, {wxHtmlEasyPrinting, printFile, 1}}, + {2538, {wxHtmlEasyPrinting, printText, 2}}, + {2539, {wxHtmlEasyPrinting, pageSetup, 0}}, + {2540, {wxHtmlEasyPrinting, setFonts, 3}}, + {2541, {wxHtmlEasyPrinting, setHeader, 2}}, + {2542, {wxHtmlEasyPrinting, setFooter, 2}}, + {2544, {wxGLCanvas, new_2, 2}}, + {2545, {wxGLCanvas, new_3_1, 3}}, + {2546, {wxGLCanvas, new_3_0, 3}}, + {2547, {wxGLCanvas, getContext, 0}}, + {2549, {wxGLCanvas, setCurrent, 0}}, + {2550, {wxGLCanvas, swapBuffers, 0}}, + {2551, {wxGLCanvas, 'Destroy', undefined}}, + {2552, {wxAuiManager, new, 1}}, + {2553, {wxAuiManager, destruct, 0}}, + {2554, {wxAuiManager, addPane_2_1, 2}}, + {2555, {wxAuiManager, addPane_3, 3}}, + {2556, {wxAuiManager, addPane_2_0, 2}}, + {2557, {wxAuiManager, detachPane, 1}}, + {2558, {wxAuiManager, getAllPanes, 0}}, + {2559, {wxAuiManager, getArtProvider, 0}}, + {2560, {wxAuiManager, getDockSizeConstraint, 2}}, + {2561, {wxAuiManager, getFlags, 0}}, + {2562, {wxAuiManager, getManagedWindow, 0}}, + {2563, {wxAuiManager, getManager, 1}}, + {2564, {wxAuiManager, getPane_1_1, 1}}, + {2565, {wxAuiManager, getPane_1_0, 1}}, + {2566, {wxAuiManager, hideHint, 0}}, + {2567, {wxAuiManager, insertPane, 3}}, + {2568, {wxAuiManager, loadPaneInfo, 2}}, + {2569, {wxAuiManager, loadPerspective, 2}}, + {2570, {wxAuiManager, savePaneInfo, 1}}, + {2571, {wxAuiManager, savePerspective, 0}}, + {2572, {wxAuiManager, setArtProvider, 1}}, + {2573, {wxAuiManager, setDockSizeConstraint, 2}}, + {2574, {wxAuiManager, setFlags, 1}}, + {2575, {wxAuiManager, setManagedWindow, 1}}, + {2576, {wxAuiManager, showHint, 1}}, + {2577, {wxAuiManager, unInit, 0}}, + {2578, {wxAuiManager, update, 0}}, + {2579, {wxAuiPaneInfo, new_0, 0}}, + {2580, {wxAuiPaneInfo, new_1, 1}}, + {2581, {wxAuiPaneInfo, destruct, 0}}, + {2582, {wxAuiPaneInfo, bestSize_1, 1}}, + {2583, {wxAuiPaneInfo, bestSize_2, 2}}, + {2584, {wxAuiPaneInfo, bottom, 0}}, + {2585, {wxAuiPaneInfo, bottomDockable, 1}}, + {2586, {wxAuiPaneInfo, caption, 1}}, + {2587, {wxAuiPaneInfo, captionVisible, 1}}, + {2588, {wxAuiPaneInfo, centre, 0}}, + {2589, {wxAuiPaneInfo, centrePane, 0}}, + {2590, {wxAuiPaneInfo, closeButton, 1}}, + {2591, {wxAuiPaneInfo, defaultPane, 0}}, + {2592, {wxAuiPaneInfo, destroyOnClose, 1}}, + {2593, {wxAuiPaneInfo, direction, 1}}, + {2594, {wxAuiPaneInfo, dock, 0}}, + {2595, {wxAuiPaneInfo, dockable, 1}}, + {2596, {wxAuiPaneInfo, fixed, 0}}, + {2597, {wxAuiPaneInfo, float, 0}}, + {2598, {wxAuiPaneInfo, floatable, 1}}, + {2599, {wxAuiPaneInfo, floatingPosition_1, 1}}, + {2600, {wxAuiPaneInfo, floatingPosition_2, 2}}, + {2601, {wxAuiPaneInfo, floatingSize_1, 1}}, + {2602, {wxAuiPaneInfo, floatingSize_2, 2}}, + {2603, {wxAuiPaneInfo, gripper, 1}}, + {2604, {wxAuiPaneInfo, gripperTop, 1}}, + {2605, {wxAuiPaneInfo, hasBorder, 0}}, + {2606, {wxAuiPaneInfo, hasCaption, 0}}, + {2607, {wxAuiPaneInfo, hasCloseButton, 0}}, + {2608, {wxAuiPaneInfo, hasFlag, 1}}, + {2609, {wxAuiPaneInfo, hasGripper, 0}}, + {2610, {wxAuiPaneInfo, hasGripperTop, 0}}, + {2611, {wxAuiPaneInfo, hasMaximizeButton, 0}}, + {2612, {wxAuiPaneInfo, hasMinimizeButton, 0}}, + {2613, {wxAuiPaneInfo, hasPinButton, 0}}, + {2614, {wxAuiPaneInfo, hide, 0}}, + {2615, {wxAuiPaneInfo, isBottomDockable, 0}}, + {2616, {wxAuiPaneInfo, isDocked, 0}}, + {2617, {wxAuiPaneInfo, isFixed, 0}}, + {2618, {wxAuiPaneInfo, isFloatable, 0}}, + {2619, {wxAuiPaneInfo, isFloating, 0}}, + {2620, {wxAuiPaneInfo, isLeftDockable, 0}}, + {2621, {wxAuiPaneInfo, isMovable, 0}}, + {2622, {wxAuiPaneInfo, isOk, 0}}, + {2623, {wxAuiPaneInfo, isResizable, 0}}, + {2624, {wxAuiPaneInfo, isRightDockable, 0}}, + {2625, {wxAuiPaneInfo, isShown, 0}}, + {2626, {wxAuiPaneInfo, isToolbar, 0}}, + {2627, {wxAuiPaneInfo, isTopDockable, 0}}, + {2628, {wxAuiPaneInfo, layer, 1}}, + {2629, {wxAuiPaneInfo, left, 0}}, + {2630, {wxAuiPaneInfo, leftDockable, 1}}, + {2631, {wxAuiPaneInfo, maxSize_1, 1}}, + {2632, {wxAuiPaneInfo, maxSize_2, 2}}, + {2633, {wxAuiPaneInfo, maximizeButton, 1}}, + {2634, {wxAuiPaneInfo, minSize_1, 1}}, + {2635, {wxAuiPaneInfo, minSize_2, 2}}, + {2636, {wxAuiPaneInfo, minimizeButton, 1}}, + {2637, {wxAuiPaneInfo, movable, 1}}, + {2638, {wxAuiPaneInfo, name, 1}}, + {2639, {wxAuiPaneInfo, paneBorder, 1}}, + {2640, {wxAuiPaneInfo, pinButton, 1}}, + {2641, {wxAuiPaneInfo, position, 1}}, + {2642, {wxAuiPaneInfo, resizable, 1}}, + {2643, {wxAuiPaneInfo, right, 0}}, + {2644, {wxAuiPaneInfo, rightDockable, 1}}, + {2645, {wxAuiPaneInfo, row, 1}}, + {2646, {wxAuiPaneInfo, safeSet, 1}}, + {2647, {wxAuiPaneInfo, setFlag, 2}}, + {2648, {wxAuiPaneInfo, show, 1}}, + {2649, {wxAuiPaneInfo, toolbarPane, 0}}, + {2650, {wxAuiPaneInfo, top, 0}}, + {2651, {wxAuiPaneInfo, topDockable, 1}}, + {2652, {wxAuiPaneInfo, window, 1}}, + {2653, {wxAuiNotebook, new_0, 0}}, + {2654, {wxAuiNotebook, new_2, 2}}, + {2655, {wxAuiNotebook, addPage, 3}}, + {2656, {wxAuiNotebook, create, 2}}, + {2657, {wxAuiNotebook, deletePage, 1}}, + {2658, {wxAuiNotebook, getArtProvider, 0}}, + {2659, {wxAuiNotebook, getPage, 1}}, + {2660, {wxAuiNotebook, getPageBitmap, 1}}, + {2661, {wxAuiNotebook, getPageCount, 0}}, + {2662, {wxAuiNotebook, getPageIndex, 1}}, + {2663, {wxAuiNotebook, getPageText, 1}}, + {2664, {wxAuiNotebook, getSelection, 0}}, + {2665, {wxAuiNotebook, insertPage, 4}}, + {2666, {wxAuiNotebook, removePage, 1}}, + {2667, {wxAuiNotebook, setArtProvider, 1}}, + {2668, {wxAuiNotebook, setFont, 1}}, + {2669, {wxAuiNotebook, setPageBitmap, 2}}, + {2670, {wxAuiNotebook, setPageText, 2}}, + {2671, {wxAuiNotebook, setSelection, 1}}, + {2672, {wxAuiNotebook, setTabCtrlHeight, 1}}, + {2673, {wxAuiNotebook, setUniformBitmapSize, 1}}, + {2674, {wxAuiNotebook, 'Destroy', undefined}}, + {2675, {wxMDIParentFrame, new_0, 0}}, + {2676, {wxMDIParentFrame, new_4, 4}}, + {2677, {wxMDIParentFrame, destruct, 0}}, + {2678, {wxMDIParentFrame, activateNext, 0}}, + {2679, {wxMDIParentFrame, activatePrevious, 0}}, + {2680, {wxMDIParentFrame, arrangeIcons, 0}}, + {2681, {wxMDIParentFrame, cascade, 0}}, + {2682, {wxMDIParentFrame, create, 4}}, + {2683, {wxMDIParentFrame, getActiveChild, 0}}, + {2684, {wxMDIParentFrame, getClientWindow, 0}}, + {2685, {wxMDIParentFrame, tile, 1}}, + {2686, {wxMDIChildFrame, new_0, 0}}, + {2687, {wxMDIChildFrame, new_4, 4}}, + {2688, {wxMDIChildFrame, destruct, 0}}, + {2689, {wxMDIChildFrame, activate, 0}}, + {2690, {wxMDIChildFrame, create, 4}}, + {2691, {wxMDIChildFrame, maximize, 1}}, + {2692, {wxMDIChildFrame, restore, 0}}, + {2693, {wxMDIClientWindow, new_0, 0}}, + {2694, {wxMDIClientWindow, new_2, 2}}, + {2695, {wxMDIClientWindow, destruct, 0}}, + {2696, {wxMDIClientWindow, createClient, 2}}, + {2697, {wxLayoutAlgorithm, new, 0}}, + {2698, {wxLayoutAlgorithm, layoutFrame, 2}}, + {2699, {wxLayoutAlgorithm, layoutMDIFrame, 2}}, + {2700, {wxLayoutAlgorithm, layoutWindow, 2}}, + {2701, {wxLayoutAlgorithm, 'Destroy', undefined}}, + {2702, {wxEvent, getId, 0}}, + {2703, {wxEvent, getSkipped, 0}}, + {2704, {wxEvent, getTimestamp, 0}}, + {2705, {wxEvent, isCommandEvent, 0}}, + {2706, {wxEvent, resumePropagation, 1}}, + {2707, {wxEvent, shouldPropagate, 0}}, + {2708, {wxEvent, skip, 1}}, + {2709, {wxEvent, stopPropagation, 0}}, + {2710, {wxCommandEvent, getClientData, 0}}, + {2711, {wxCommandEvent, getExtraLong, 0}}, + {2712, {wxCommandEvent, getInt, 0}}, + {2713, {wxCommandEvent, getSelection, 0}}, + {2714, {wxCommandEvent, getString, 0}}, + {2715, {wxCommandEvent, isChecked, 0}}, + {2716, {wxCommandEvent, isSelection, 0}}, + {2717, {wxCommandEvent, setInt, 1}}, + {2718, {wxCommandEvent, setString, 1}}, + {2719, {wxScrollEvent, getOrientation, 0}}, + {2720, {wxScrollEvent, getPosition, 0}}, + {2721, {wxScrollWinEvent, getOrientation, 0}}, + {2722, {wxScrollWinEvent, getPosition, 0}}, + {2723, {wxMouseEvent, altDown, 0}}, + {2724, {wxMouseEvent, button, 1}}, + {2725, {wxMouseEvent, buttonDClick, 1}}, + {2726, {wxMouseEvent, buttonDown, 1}}, + {2727, {wxMouseEvent, buttonUp, 1}}, + {2728, {wxMouseEvent, cmdDown, 0}}, + {2729, {wxMouseEvent, controlDown, 0}}, + {2730, {wxMouseEvent, dragging, 0}}, + {2731, {wxMouseEvent, entering, 0}}, + {2732, {wxMouseEvent, getButton, 0}}, + {2735, {wxMouseEvent, getPosition, 0}}, + {2736, {wxMouseEvent, getLogicalPosition, 1}}, + {2737, {wxMouseEvent, getLinesPerAction, 0}}, + {2738, {wxMouseEvent, getWheelRotation, 0}}, + {2739, {wxMouseEvent, getWheelDelta, 0}}, + {2740, {wxMouseEvent, getX, 0}}, + {2741, {wxMouseEvent, getY, 0}}, + {2742, {wxMouseEvent, isButton, 0}}, + {2743, {wxMouseEvent, isPageScroll, 0}}, + {2744, {wxMouseEvent, leaving, 0}}, + {2745, {wxMouseEvent, leftDClick, 0}}, + {2746, {wxMouseEvent, leftDown, 0}}, + {2747, {wxMouseEvent, leftIsDown, 0}}, + {2748, {wxMouseEvent, leftUp, 0}}, + {2749, {wxMouseEvent, metaDown, 0}}, + {2750, {wxMouseEvent, middleDClick, 0}}, + {2751, {wxMouseEvent, middleDown, 0}}, + {2752, {wxMouseEvent, middleIsDown, 0}}, + {2753, {wxMouseEvent, middleUp, 0}}, + {2754, {wxMouseEvent, moving, 0}}, + {2755, {wxMouseEvent, rightDClick, 0}}, + {2756, {wxMouseEvent, rightDown, 0}}, + {2757, {wxMouseEvent, rightIsDown, 0}}, + {2758, {wxMouseEvent, rightUp, 0}}, + {2759, {wxMouseEvent, shiftDown, 0}}, + {2760, {wxSetCursorEvent, getCursor, 0}}, + {2761, {wxSetCursorEvent, getX, 0}}, + {2762, {wxSetCursorEvent, getY, 0}}, + {2763, {wxSetCursorEvent, hasCursor, 0}}, + {2764, {wxSetCursorEvent, setCursor, 1}}, + {2765, {wxKeyEvent, altDown, 0}}, + {2766, {wxKeyEvent, cmdDown, 0}}, + {2767, {wxKeyEvent, controlDown, 0}}, + {2768, {wxKeyEvent, getKeyCode, 0}}, + {2769, {wxKeyEvent, getModifiers, 0}}, + {2772, {wxKeyEvent, getPosition, 0}}, + {2773, {wxKeyEvent, getRawKeyCode, 0}}, + {2774, {wxKeyEvent, getRawKeyFlags, 0}}, + {2775, {wxKeyEvent, getUnicodeKey, 0}}, + {2776, {wxKeyEvent, getX, 0}}, + {2777, {wxKeyEvent, getY, 0}}, + {2778, {wxKeyEvent, hasModifiers, 0}}, + {2779, {wxKeyEvent, metaDown, 0}}, + {2780, {wxKeyEvent, shiftDown, 0}}, + {2781, {wxSizeEvent, getSize, 0}}, + {2782, {wxMoveEvent, getPosition, 0}}, + {2783, {wxEraseEvent, getDC, 0}}, + {2784, {wxFocusEvent, getWindow, 0}}, + {2785, {wxChildFocusEvent, getWindow, 0}}, + {2786, {wxMenuEvent, getMenu, 0}}, + {2787, {wxMenuEvent, getMenuId, 0}}, + {2788, {wxMenuEvent, isPopup, 0}}, + {2789, {wxCloseEvent, canVeto, 0}}, + {2790, {wxCloseEvent, getLoggingOff, 0}}, + {2791, {wxCloseEvent, setCanVeto, 1}}, + {2792, {wxCloseEvent, setLoggingOff, 1}}, + {2793, {wxCloseEvent, veto, 1}}, + {2794, {wxShowEvent, setShow, 1}}, + {2795, {wxShowEvent, getShow, 0}}, + {2796, {wxIconizeEvent, iconized, 0}}, + {2797, {wxJoystickEvent, buttonDown, 1}}, + {2798, {wxJoystickEvent, buttonIsDown, 1}}, + {2799, {wxJoystickEvent, buttonUp, 1}}, + {2800, {wxJoystickEvent, getButtonChange, 0}}, + {2801, {wxJoystickEvent, getButtonState, 0}}, + {2802, {wxJoystickEvent, getJoystick, 0}}, + {2803, {wxJoystickEvent, getPosition, 0}}, + {2804, {wxJoystickEvent, getZPosition, 0}}, + {2805, {wxJoystickEvent, isButton, 0}}, + {2806, {wxJoystickEvent, isMove, 0}}, + {2807, {wxJoystickEvent, isZMove, 0}}, + {2808, {wxUpdateUIEvent, canUpdate, 1}}, + {2809, {wxUpdateUIEvent, check, 1}}, + {2810, {wxUpdateUIEvent, enable, 1}}, + {2811, {wxUpdateUIEvent, show, 1}}, + {2812, {wxUpdateUIEvent, getChecked, 0}}, + {2813, {wxUpdateUIEvent, getEnabled, 0}}, + {2814, {wxUpdateUIEvent, getShown, 0}}, + {2815, {wxUpdateUIEvent, getSetChecked, 0}}, + {2816, {wxUpdateUIEvent, getSetEnabled, 0}}, + {2817, {wxUpdateUIEvent, getSetShown, 0}}, + {2818, {wxUpdateUIEvent, getSetText, 0}}, + {2819, {wxUpdateUIEvent, getText, 0}}, + {2820, {wxUpdateUIEvent, getMode, 0}}, + {2821, {wxUpdateUIEvent, getUpdateInterval, 0}}, + {2822, {wxUpdateUIEvent, resetUpdateTime, 0}}, + {2823, {wxUpdateUIEvent, setMode, 1}}, + {2824, {wxUpdateUIEvent, setText, 1}}, + {2825, {wxUpdateUIEvent, setUpdateInterval, 1}}, + {2826, {wxMouseCaptureChangedEvent, getCapturedWindow, 0}}, + {2827, {wxPaletteChangedEvent, setChangedWindow, 1}}, + {2828, {wxPaletteChangedEvent, getChangedWindow, 0}}, + {2829, {wxQueryNewPaletteEvent, setPaletteRealized, 1}}, + {2830, {wxQueryNewPaletteEvent, getPaletteRealized, 0}}, + {2831, {wxNavigationKeyEvent, getDirection, 0}}, + {2832, {wxNavigationKeyEvent, setDirection, 1}}, + {2833, {wxNavigationKeyEvent, isWindowChange, 0}}, + {2834, {wxNavigationKeyEvent, setWindowChange, 1}}, + {2835, {wxNavigationKeyEvent, isFromTab, 0}}, + {2836, {wxNavigationKeyEvent, setFromTab, 1}}, + {2837, {wxNavigationKeyEvent, getCurrentFocus, 0}}, + {2838, {wxNavigationKeyEvent, setCurrentFocus, 1}}, + {2839, {wxHelpEvent, getOrigin, 0}}, + {2840, {wxHelpEvent, getPosition, 0}}, + {2841, {wxHelpEvent, setOrigin, 1}}, + {2842, {wxHelpEvent, setPosition, 1}}, + {2843, {wxContextMenuEvent, getPosition, 0}}, + {2844, {wxContextMenuEvent, setPosition, 1}}, + {2845, {wxIdleEvent, canSend, 1}}, + {2846, {wxIdleEvent, getMode, 0}}, + {2847, {wxIdleEvent, requestMore, 1}}, + {2848, {wxIdleEvent, moreRequested, 0}}, + {2849, {wxIdleEvent, setMode, 1}}, + {2850, {wxGridEvent, altDown, 0}}, + {2851, {wxGridEvent, controlDown, 0}}, + {2852, {wxGridEvent, getCol, 0}}, + {2853, {wxGridEvent, getPosition, 0}}, + {2854, {wxGridEvent, getRow, 0}}, + {2855, {wxGridEvent, metaDown, 0}}, + {2856, {wxGridEvent, selecting, 0}}, + {2857, {wxGridEvent, shiftDown, 0}}, + {2858, {wxNotifyEvent, allow, 0}}, + {2859, {wxNotifyEvent, isAllowed, 0}}, + {2860, {wxNotifyEvent, veto, 0}}, + {2861, {wxSashEvent, getEdge, 0}}, + {2862, {wxSashEvent, getDragRect, 0}}, + {2863, {wxSashEvent, getDragStatus, 0}}, + {2864, {wxListEvent, getCacheFrom, 0}}, + {2865, {wxListEvent, getCacheTo, 0}}, + {2866, {wxListEvent, getKeyCode, 0}}, + {2867, {wxListEvent, getIndex, 0}}, + {2868, {wxListEvent, getColumn, 0}}, + {2869, {wxListEvent, getPoint, 0}}, + {2870, {wxListEvent, getLabel, 0}}, + {2871, {wxListEvent, getText, 0}}, + {2872, {wxListEvent, getImage, 0}}, + {2873, {wxListEvent, getData, 0}}, + {2874, {wxListEvent, getMask, 0}}, + {2875, {wxListEvent, getItem, 0}}, + {2876, {wxListEvent, isEditCancelled, 0}}, + {2877, {wxDateEvent, getDate, 0}}, + {2878, {wxCalendarEvent, getWeekDay, 0}}, + {2879, {wxFileDirPickerEvent, getPath, 0}}, + {2880, {wxColourPickerEvent, getColour, 0}}, + {2881, {wxFontPickerEvent, getFont, 0}}, + {2882, {wxStyledTextEvent, getPosition, 0}}, + {2883, {wxStyledTextEvent, getKey, 0}}, + {2884, {wxStyledTextEvent, getModifiers, 0}}, + {2885, {wxStyledTextEvent, getModificationType, 0}}, + {2886, {wxStyledTextEvent, getText, 0}}, + {2887, {wxStyledTextEvent, getLength, 0}}, + {2888, {wxStyledTextEvent, getLinesAdded, 0}}, + {2889, {wxStyledTextEvent, getLine, 0}}, + {2890, {wxStyledTextEvent, getFoldLevelNow, 0}}, + {2891, {wxStyledTextEvent, getFoldLevelPrev, 0}}, + {2892, {wxStyledTextEvent, getMargin, 0}}, + {2893, {wxStyledTextEvent, getMessage, 0}}, + {2894, {wxStyledTextEvent, getWParam, 0}}, + {2895, {wxStyledTextEvent, getLParam, 0}}, + {2896, {wxStyledTextEvent, getListType, 0}}, + {2897, {wxStyledTextEvent, getX, 0}}, + {2898, {wxStyledTextEvent, getY, 0}}, + {2899, {wxStyledTextEvent, getDragText, 0}}, + {2900, {wxStyledTextEvent, getDragAllowMove, 0}}, + {2901, {wxStyledTextEvent, getDragResult, 0}}, + {2902, {wxStyledTextEvent, getShift, 0}}, + {2903, {wxStyledTextEvent, getControl, 0}}, + {2904, {wxStyledTextEvent, getAlt, 0}}, + {2905, {utils, getKeyState, 1}}, + {2906, {utils, getMousePosition, 2}}, + {2907, {utils, getMouseState, 0}}, + {2908, {utils, setDetectableAutoRepeat, 1}}, + {2909, {utils, bell, 0}}, + {2910, {utils, findMenuItemId, 3}}, + {2911, {utils, genericFindWindowAtPoint, 1}}, + {2912, {utils, findWindowAtPoint, 1}}, + {2913, {utils, beginBusyCursor, 1}}, + {2914, {utils, endBusyCursor, 0}}, + {2915, {utils, isBusy, 0}}, + {2916, {utils, shutdown, 1}}, + {2917, {utils, shell, 1}}, + {2918, {utils, launchDefaultBrowser, 2}}, + {2919, {utils, getEmailAddress, 0}}, + {2920, {utils, getUserId, 0}}, + {2921, {utils, getHomeDir, 0}}, + {2922, {utils, newId, 0}}, + {2923, {utils, registerId, 1}}, + {2924, {utils, getCurrentId, 0}}, + {2925, {utils, getOsDescription, 0}}, + {2926, {utils, isPlatformLittleEndian, 0}}, + {2927, {utils, isPlatform64Bit, 0}}, + {2928, {gdicmn, displaySize, 2}}, + {2929, {gdicmn, setCursor, 1}}, + {2930, {wxPrintout, new, 1}}, + {2931, {wxPrintout, destruct, 0}}, + {2932, {wxPrintout, getDC, 0}}, + {2933, {wxPrintout, getPageSizeMM, 2}}, + {2934, {wxPrintout, getPageSizePixels, 2}}, + {2935, {wxPrintout, getPaperRectPixels, 0}}, + {2936, {wxPrintout, getPPIPrinter, 2}}, + {2937, {wxPrintout, getPPIScreen, 2}}, + {2938, {wxPrintout, getTitle, 0}}, + {2939, {wxPrintout, isPreview, 0}}, + {2940, {wxPrintout, fitThisSizeToPaper, 1}}, + {2941, {wxPrintout, fitThisSizeToPage, 1}}, + {2942, {wxPrintout, fitThisSizeToPageMargins, 2}}, + {2943, {wxPrintout, mapScreenSizeToPaper, 0}}, + {2944, {wxPrintout, mapScreenSizeToPage, 0}}, + {2945, {wxPrintout, mapScreenSizeToPageMargins, 1}}, + {2946, {wxPrintout, mapScreenSizeToDevice, 0}}, + {2947, {wxPrintout, getLogicalPaperRect, 0}}, + {2948, {wxPrintout, getLogicalPageRect, 0}}, + {2949, {wxPrintout, getLogicalPageMarginsRect, 1}}, + {2950, {wxPrintout, setLogicalOrigin, 2}}, + {2951, {wxPrintout, offsetLogicalOrigin, 2}}, + {2952, {wxStyledTextCtrl, new_2, 2}}, + {2953, {wxStyledTextCtrl, new_0, 0}}, + {2954, {wxStyledTextCtrl, destruct, 0}}, + {2955, {wxStyledTextCtrl, create, 2}}, + {2956, {wxStyledTextCtrl, addText, 1}}, + {2957, {wxStyledTextCtrl, addStyledText, 1}}, + {2958, {wxStyledTextCtrl, insertText, 2}}, + {2959, {wxStyledTextCtrl, clearAll, 0}}, + {2960, {wxStyledTextCtrl, clearDocumentStyle, 0}}, + {2961, {wxStyledTextCtrl, getLength, 0}}, + {2962, {wxStyledTextCtrl, getCharAt, 1}}, + {2963, {wxStyledTextCtrl, getCurrentPos, 0}}, + {2964, {wxStyledTextCtrl, getAnchor, 0}}, + {2965, {wxStyledTextCtrl, getStyleAt, 1}}, + {2966, {wxStyledTextCtrl, redo, 0}}, + {2967, {wxStyledTextCtrl, setUndoCollection, 1}}, + {2968, {wxStyledTextCtrl, selectAll, 0}}, + {2969, {wxStyledTextCtrl, setSavePoint, 0}}, + {2970, {wxStyledTextCtrl, getStyledText, 2}}, + {2971, {wxStyledTextCtrl, canRedo, 0}}, + {2972, {wxStyledTextCtrl, markerLineFromHandle, 1}}, + {2973, {wxStyledTextCtrl, markerDeleteHandle, 1}}, + {2974, {wxStyledTextCtrl, getUndoCollection, 0}}, + {2975, {wxStyledTextCtrl, getViewWhiteSpace, 0}}, + {2976, {wxStyledTextCtrl, setViewWhiteSpace, 1}}, + {2977, {wxStyledTextCtrl, positionFromPoint, 1}}, + {2978, {wxStyledTextCtrl, positionFromPointClose, 2}}, + {2979, {wxStyledTextCtrl, gotoLine, 1}}, + {2980, {wxStyledTextCtrl, gotoPos, 1}}, + {2981, {wxStyledTextCtrl, setAnchor, 1}}, + {2982, {wxStyledTextCtrl, getCurLine, 1}}, + {2983, {wxStyledTextCtrl, getEndStyled, 0}}, + {2984, {wxStyledTextCtrl, convertEOLs, 1}}, + {2985, {wxStyledTextCtrl, getEOLMode, 0}}, + {2986, {wxStyledTextCtrl, setEOLMode, 1}}, + {2987, {wxStyledTextCtrl, startStyling, 2}}, + {2988, {wxStyledTextCtrl, setStyling, 2}}, + {2989, {wxStyledTextCtrl, getBufferedDraw, 0}}, + {2990, {wxStyledTextCtrl, setBufferedDraw, 1}}, + {2991, {wxStyledTextCtrl, setTabWidth, 1}}, + {2992, {wxStyledTextCtrl, getTabWidth, 0}}, + {2993, {wxStyledTextCtrl, setCodePage, 1}}, + {2994, {wxStyledTextCtrl, markerDefine, 3}}, + {2995, {wxStyledTextCtrl, markerSetForeground, 2}}, + {2996, {wxStyledTextCtrl, markerSetBackground, 2}}, + {2997, {wxStyledTextCtrl, markerAdd, 2}}, + {2998, {wxStyledTextCtrl, markerDelete, 2}}, + {2999, {wxStyledTextCtrl, markerDeleteAll, 1}}, + {3000, {wxStyledTextCtrl, markerGet, 1}}, + {3001, {wxStyledTextCtrl, markerNext, 2}}, + {3002, {wxStyledTextCtrl, markerPrevious, 2}}, + {3003, {wxStyledTextCtrl, markerDefineBitmap, 2}}, + {3004, {wxStyledTextCtrl, markerAddSet, 2}}, + {3005, {wxStyledTextCtrl, markerSetAlpha, 2}}, + {3006, {wxStyledTextCtrl, setMarginType, 2}}, + {3007, {wxStyledTextCtrl, getMarginType, 1}}, + {3008, {wxStyledTextCtrl, setMarginWidth, 2}}, + {3009, {wxStyledTextCtrl, getMarginWidth, 1}}, + {3010, {wxStyledTextCtrl, setMarginMask, 2}}, + {3011, {wxStyledTextCtrl, getMarginMask, 1}}, + {3012, {wxStyledTextCtrl, setMarginSensitive, 2}}, + {3013, {wxStyledTextCtrl, getMarginSensitive, 1}}, + {3014, {wxStyledTextCtrl, styleClearAll, 0}}, + {3015, {wxStyledTextCtrl, styleSetForeground, 2}}, + {3016, {wxStyledTextCtrl, styleSetBackground, 2}}, + {3017, {wxStyledTextCtrl, styleSetBold, 2}}, + {3018, {wxStyledTextCtrl, styleSetItalic, 2}}, + {3019, {wxStyledTextCtrl, styleSetSize, 2}}, + {3020, {wxStyledTextCtrl, styleSetFaceName, 2}}, + {3021, {wxStyledTextCtrl, styleSetEOLFilled, 2}}, + {3022, {wxStyledTextCtrl, styleResetDefault, 0}}, + {3023, {wxStyledTextCtrl, styleSetUnderline, 2}}, + {3024, {wxStyledTextCtrl, styleSetCase, 2}}, + {3025, {wxStyledTextCtrl, styleSetHotSpot, 2}}, + {3026, {wxStyledTextCtrl, setSelForeground, 2}}, + {3027, {wxStyledTextCtrl, setSelBackground, 2}}, + {3028, {wxStyledTextCtrl, getSelAlpha, 0}}, + {3029, {wxStyledTextCtrl, setSelAlpha, 1}}, + {3030, {wxStyledTextCtrl, setCaretForeground, 1}}, + {3031, {wxStyledTextCtrl, cmdKeyAssign, 3}}, + {3032, {wxStyledTextCtrl, cmdKeyClear, 2}}, + {3033, {wxStyledTextCtrl, cmdKeyClearAll, 0}}, + {3034, {wxStyledTextCtrl, setStyleBytes, 2}}, + {3035, {wxStyledTextCtrl, styleSetVisible, 2}}, + {3036, {wxStyledTextCtrl, getCaretPeriod, 0}}, + {3037, {wxStyledTextCtrl, setCaretPeriod, 1}}, + {3038, {wxStyledTextCtrl, setWordChars, 1}}, + {3039, {wxStyledTextCtrl, beginUndoAction, 0}}, + {3040, {wxStyledTextCtrl, endUndoAction, 0}}, + {3041, {wxStyledTextCtrl, indicatorSetStyle, 2}}, + {3042, {wxStyledTextCtrl, indicatorGetStyle, 1}}, + {3043, {wxStyledTextCtrl, indicatorSetForeground, 2}}, + {3044, {wxStyledTextCtrl, indicatorGetForeground, 1}}, + {3045, {wxStyledTextCtrl, setWhitespaceForeground, 2}}, + {3046, {wxStyledTextCtrl, setWhitespaceBackground, 2}}, + {3047, {wxStyledTextCtrl, getStyleBits, 0}}, + {3048, {wxStyledTextCtrl, setLineState, 2}}, + {3049, {wxStyledTextCtrl, getLineState, 1}}, + {3050, {wxStyledTextCtrl, getMaxLineState, 0}}, + {3051, {wxStyledTextCtrl, getCaretLineVisible, 0}}, + {3052, {wxStyledTextCtrl, setCaretLineVisible, 1}}, + {3053, {wxStyledTextCtrl, getCaretLineBackground, 0}}, + {3054, {wxStyledTextCtrl, setCaretLineBackground, 1}}, + {3055, {wxStyledTextCtrl, autoCompShow, 2}}, + {3056, {wxStyledTextCtrl, autoCompCancel, 0}}, + {3057, {wxStyledTextCtrl, autoCompActive, 0}}, + {3058, {wxStyledTextCtrl, autoCompPosStart, 0}}, + {3059, {wxStyledTextCtrl, autoCompComplete, 0}}, + {3060, {wxStyledTextCtrl, autoCompStops, 1}}, + {3061, {wxStyledTextCtrl, autoCompSetSeparator, 1}}, + {3062, {wxStyledTextCtrl, autoCompGetSeparator, 0}}, + {3063, {wxStyledTextCtrl, autoCompSelect, 1}}, + {3064, {wxStyledTextCtrl, autoCompSetCancelAtStart, 1}}, + {3065, {wxStyledTextCtrl, autoCompGetCancelAtStart, 0}}, + {3066, {wxStyledTextCtrl, autoCompSetFillUps, 1}}, + {3067, {wxStyledTextCtrl, autoCompSetChooseSingle, 1}}, + {3068, {wxStyledTextCtrl, autoCompGetChooseSingle, 0}}, + {3069, {wxStyledTextCtrl, autoCompSetIgnoreCase, 1}}, + {3070, {wxStyledTextCtrl, autoCompGetIgnoreCase, 0}}, + {3071, {wxStyledTextCtrl, userListShow, 2}}, + {3072, {wxStyledTextCtrl, autoCompSetAutoHide, 1}}, + {3073, {wxStyledTextCtrl, autoCompGetAutoHide, 0}}, + {3074, {wxStyledTextCtrl, autoCompSetDropRestOfWord, 1}}, + {3075, {wxStyledTextCtrl, autoCompGetDropRestOfWord, 0}}, + {3076, {wxStyledTextCtrl, registerImage, 2}}, + {3077, {wxStyledTextCtrl, clearRegisteredImages, 0}}, + {3078, {wxStyledTextCtrl, autoCompGetTypeSeparator, 0}}, + {3079, {wxStyledTextCtrl, autoCompSetTypeSeparator, 1}}, + {3080, {wxStyledTextCtrl, autoCompSetMaxWidth, 1}}, + {3081, {wxStyledTextCtrl, autoCompGetMaxWidth, 0}}, + {3082, {wxStyledTextCtrl, autoCompSetMaxHeight, 1}}, + {3083, {wxStyledTextCtrl, autoCompGetMaxHeight, 0}}, + {3084, {wxStyledTextCtrl, setIndent, 1}}, + {3085, {wxStyledTextCtrl, getIndent, 0}}, + {3086, {wxStyledTextCtrl, setUseTabs, 1}}, + {3087, {wxStyledTextCtrl, getUseTabs, 0}}, + {3088, {wxStyledTextCtrl, setLineIndentation, 2}}, + {3089, {wxStyledTextCtrl, getLineIndentation, 1}}, + {3090, {wxStyledTextCtrl, getLineIndentPosition, 1}}, + {3091, {wxStyledTextCtrl, getColumn, 1}}, + {3092, {wxStyledTextCtrl, setUseHorizontalScrollBar, 1}}, + {3093, {wxStyledTextCtrl, getUseHorizontalScrollBar, 0}}, + {3094, {wxStyledTextCtrl, setIndentationGuides, 1}}, + {3095, {wxStyledTextCtrl, getIndentationGuides, 0}}, + {3096, {wxStyledTextCtrl, setHighlightGuide, 1}}, + {3097, {wxStyledTextCtrl, getHighlightGuide, 0}}, + {3098, {wxStyledTextCtrl, getLineEndPosition, 1}}, + {3099, {wxStyledTextCtrl, getCodePage, 0}}, + {3100, {wxStyledTextCtrl, getCaretForeground, 0}}, + {3101, {wxStyledTextCtrl, getReadOnly, 0}}, + {3102, {wxStyledTextCtrl, setCurrentPos, 1}}, + {3103, {wxStyledTextCtrl, setSelectionStart, 1}}, + {3104, {wxStyledTextCtrl, getSelectionStart, 0}}, + {3105, {wxStyledTextCtrl, setSelectionEnd, 1}}, + {3106, {wxStyledTextCtrl, getSelectionEnd, 0}}, + {3107, {wxStyledTextCtrl, setPrintMagnification, 1}}, + {3108, {wxStyledTextCtrl, getPrintMagnification, 0}}, + {3109, {wxStyledTextCtrl, setPrintColourMode, 1}}, + {3110, {wxStyledTextCtrl, getPrintColourMode, 0}}, + {3111, {wxStyledTextCtrl, findText, 4}}, + {3112, {wxStyledTextCtrl, formatRange, 7}}, + {3113, {wxStyledTextCtrl, getFirstVisibleLine, 0}}, + {3114, {wxStyledTextCtrl, getLine, 1}}, + {3115, {wxStyledTextCtrl, getLineCount, 0}}, + {3116, {wxStyledTextCtrl, setMarginLeft, 1}}, + {3117, {wxStyledTextCtrl, getMarginLeft, 0}}, + {3118, {wxStyledTextCtrl, setMarginRight, 1}}, + {3119, {wxStyledTextCtrl, getMarginRight, 0}}, + {3120, {wxStyledTextCtrl, getModify, 0}}, + {3121, {wxStyledTextCtrl, setSelection, 2}}, + {3122, {wxStyledTextCtrl, getSelectedText, 0}}, + {3123, {wxStyledTextCtrl, getTextRange, 2}}, + {3124, {wxStyledTextCtrl, hideSelection, 1}}, + {3125, {wxStyledTextCtrl, lineFromPosition, 1}}, + {3126, {wxStyledTextCtrl, positionFromLine, 1}}, + {3127, {wxStyledTextCtrl, lineScroll, 2}}, + {3128, {wxStyledTextCtrl, ensureCaretVisible, 0}}, + {3129, {wxStyledTextCtrl, replaceSelection, 1}}, + {3130, {wxStyledTextCtrl, setReadOnly, 1}}, + {3131, {wxStyledTextCtrl, canPaste, 0}}, + {3132, {wxStyledTextCtrl, canUndo, 0}}, + {3133, {wxStyledTextCtrl, emptyUndoBuffer, 0}}, + {3134, {wxStyledTextCtrl, undo, 0}}, + {3135, {wxStyledTextCtrl, cut, 0}}, + {3136, {wxStyledTextCtrl, copy, 0}}, + {3137, {wxStyledTextCtrl, paste, 0}}, + {3138, {wxStyledTextCtrl, clear, 0}}, + {3139, {wxStyledTextCtrl, setText, 1}}, + {3140, {wxStyledTextCtrl, getText, 0}}, + {3141, {wxStyledTextCtrl, getTextLength, 0}}, + {3142, {wxStyledTextCtrl, getOvertype, 0}}, + {3143, {wxStyledTextCtrl, setCaretWidth, 1}}, + {3144, {wxStyledTextCtrl, getCaretWidth, 0}}, + {3145, {wxStyledTextCtrl, setTargetStart, 1}}, + {3146, {wxStyledTextCtrl, getTargetStart, 0}}, + {3147, {wxStyledTextCtrl, setTargetEnd, 1}}, + {3148, {wxStyledTextCtrl, getTargetEnd, 0}}, + {3149, {wxStyledTextCtrl, replaceTarget, 1}}, + {3150, {wxStyledTextCtrl, searchInTarget, 1}}, + {3151, {wxStyledTextCtrl, setSearchFlags, 1}}, + {3152, {wxStyledTextCtrl, getSearchFlags, 0}}, + {3153, {wxStyledTextCtrl, callTipShow, 2}}, + {3154, {wxStyledTextCtrl, callTipCancel, 0}}, + {3155, {wxStyledTextCtrl, callTipActive, 0}}, + {3156, {wxStyledTextCtrl, callTipPosAtStart, 0}}, + {3157, {wxStyledTextCtrl, callTipSetHighlight, 2}}, + {3158, {wxStyledTextCtrl, callTipSetBackground, 1}}, + {3159, {wxStyledTextCtrl, callTipSetForeground, 1}}, + {3160, {wxStyledTextCtrl, callTipSetForegroundHighlight, 1}}, + {3161, {wxStyledTextCtrl, callTipUseStyle, 1}}, + {3162, {wxStyledTextCtrl, visibleFromDocLine, 1}}, + {3163, {wxStyledTextCtrl, docLineFromVisible, 1}}, + {3164, {wxStyledTextCtrl, wrapCount, 1}}, + {3165, {wxStyledTextCtrl, setFoldLevel, 2}}, + {3166, {wxStyledTextCtrl, getFoldLevel, 1}}, + {3167, {wxStyledTextCtrl, getLastChild, 2}}, + {3168, {wxStyledTextCtrl, getFoldParent, 1}}, + {3169, {wxStyledTextCtrl, showLines, 2}}, + {3170, {wxStyledTextCtrl, hideLines, 2}}, + {3171, {wxStyledTextCtrl, getLineVisible, 1}}, + {3172, {wxStyledTextCtrl, setFoldExpanded, 2}}, + {3173, {wxStyledTextCtrl, getFoldExpanded, 1}}, + {3174, {wxStyledTextCtrl, toggleFold, 1}}, + {3175, {wxStyledTextCtrl, ensureVisible, 1}}, + {3176, {wxStyledTextCtrl, setFoldFlags, 1}}, + {3177, {wxStyledTextCtrl, ensureVisibleEnforcePolicy, 1}}, + {3178, {wxStyledTextCtrl, setTabIndents, 1}}, + {3179, {wxStyledTextCtrl, getTabIndents, 0}}, + {3180, {wxStyledTextCtrl, setBackSpaceUnIndents, 1}}, + {3181, {wxStyledTextCtrl, getBackSpaceUnIndents, 0}}, + {3182, {wxStyledTextCtrl, setMouseDwellTime, 1}}, + {3183, {wxStyledTextCtrl, getMouseDwellTime, 0}}, + {3184, {wxStyledTextCtrl, wordStartPosition, 2}}, + {3185, {wxStyledTextCtrl, wordEndPosition, 2}}, + {3186, {wxStyledTextCtrl, setWrapMode, 1}}, + {3187, {wxStyledTextCtrl, getWrapMode, 0}}, + {3188, {wxStyledTextCtrl, setWrapVisualFlags, 1}}, + {3189, {wxStyledTextCtrl, getWrapVisualFlags, 0}}, + {3190, {wxStyledTextCtrl, setWrapVisualFlagsLocation, 1}}, + {3191, {wxStyledTextCtrl, getWrapVisualFlagsLocation, 0}}, + {3192, {wxStyledTextCtrl, setWrapStartIndent, 1}}, + {3193, {wxStyledTextCtrl, getWrapStartIndent, 0}}, + {3194, {wxStyledTextCtrl, setLayoutCache, 1}}, + {3195, {wxStyledTextCtrl, getLayoutCache, 0}}, + {3196, {wxStyledTextCtrl, setScrollWidth, 1}}, + {3197, {wxStyledTextCtrl, getScrollWidth, 0}}, + {3198, {wxStyledTextCtrl, textWidth, 2}}, + {3199, {wxStyledTextCtrl, getEndAtLastLine, 0}}, + {3200, {wxStyledTextCtrl, textHeight, 1}}, + {3201, {wxStyledTextCtrl, setUseVerticalScrollBar, 1}}, + {3202, {wxStyledTextCtrl, getUseVerticalScrollBar, 0}}, + {3203, {wxStyledTextCtrl, appendText, 1}}, + {3204, {wxStyledTextCtrl, getTwoPhaseDraw, 0}}, + {3205, {wxStyledTextCtrl, setTwoPhaseDraw, 1}}, + {3206, {wxStyledTextCtrl, targetFromSelection, 0}}, + {3207, {wxStyledTextCtrl, linesJoin, 0}}, + {3208, {wxStyledTextCtrl, linesSplit, 1}}, + {3209, {wxStyledTextCtrl, setFoldMarginColour, 2}}, + {3210, {wxStyledTextCtrl, setFoldMarginHiColour, 2}}, + {3211, {wxStyledTextCtrl, lineDown, 0}}, + {3212, {wxStyledTextCtrl, lineDownExtend, 0}}, + {3213, {wxStyledTextCtrl, lineUp, 0}}, + {3214, {wxStyledTextCtrl, lineUpExtend, 0}}, + {3215, {wxStyledTextCtrl, charLeft, 0}}, + {3216, {wxStyledTextCtrl, charLeftExtend, 0}}, + {3217, {wxStyledTextCtrl, charRight, 0}}, + {3218, {wxStyledTextCtrl, charRightExtend, 0}}, + {3219, {wxStyledTextCtrl, wordLeft, 0}}, + {3220, {wxStyledTextCtrl, wordLeftExtend, 0}}, + {3221, {wxStyledTextCtrl, wordRight, 0}}, + {3222, {wxStyledTextCtrl, wordRightExtend, 0}}, + {3223, {wxStyledTextCtrl, home, 0}}, + {3224, {wxStyledTextCtrl, homeExtend, 0}}, + {3225, {wxStyledTextCtrl, lineEnd, 0}}, + {3226, {wxStyledTextCtrl, lineEndExtend, 0}}, + {3227, {wxStyledTextCtrl, documentStart, 0}}, + {3228, {wxStyledTextCtrl, documentStartExtend, 0}}, + {3229, {wxStyledTextCtrl, documentEnd, 0}}, + {3230, {wxStyledTextCtrl, documentEndExtend, 0}}, + {3231, {wxStyledTextCtrl, pageUp, 0}}, + {3232, {wxStyledTextCtrl, pageUpExtend, 0}}, + {3233, {wxStyledTextCtrl, pageDown, 0}}, + {3234, {wxStyledTextCtrl, pageDownExtend, 0}}, + {3235, {wxStyledTextCtrl, editToggleOvertype, 0}}, + {3236, {wxStyledTextCtrl, cancel, 0}}, + {3237, {wxStyledTextCtrl, deleteBack, 0}}, + {3238, {wxStyledTextCtrl, tab, 0}}, + {3239, {wxStyledTextCtrl, backTab, 0}}, + {3240, {wxStyledTextCtrl, newLine, 0}}, + {3241, {wxStyledTextCtrl, formFeed, 0}}, + {3242, {wxStyledTextCtrl, vCHome, 0}}, + {3243, {wxStyledTextCtrl, vCHomeExtend, 0}}, + {3244, {wxStyledTextCtrl, zoomIn, 0}}, + {3245, {wxStyledTextCtrl, zoomOut, 0}}, + {3246, {wxStyledTextCtrl, delWordLeft, 0}}, + {3247, {wxStyledTextCtrl, delWordRight, 0}}, + {3248, {wxStyledTextCtrl, lineCut, 0}}, + {3249, {wxStyledTextCtrl, lineDelete, 0}}, + {3250, {wxStyledTextCtrl, lineTranspose, 0}}, + {3251, {wxStyledTextCtrl, lineDuplicate, 0}}, + {3252, {wxStyledTextCtrl, lowerCase, 0}}, + {3253, {wxStyledTextCtrl, upperCase, 0}}, + {3254, {wxStyledTextCtrl, lineScrollDown, 0}}, + {3255, {wxStyledTextCtrl, lineScrollUp, 0}}, + {3256, {wxStyledTextCtrl, deleteBackNotLine, 0}}, + {3257, {wxStyledTextCtrl, homeDisplay, 0}}, + {3258, {wxStyledTextCtrl, homeDisplayExtend, 0}}, + {3259, {wxStyledTextCtrl, lineEndDisplay, 0}}, + {3260, {wxStyledTextCtrl, lineEndDisplayExtend, 0}}, + {3261, {wxStyledTextCtrl, homeWrapExtend, 0}}, + {3262, {wxStyledTextCtrl, lineEndWrap, 0}}, + {3263, {wxStyledTextCtrl, lineEndWrapExtend, 0}}, + {3264, {wxStyledTextCtrl, vCHomeWrap, 0}}, + {3265, {wxStyledTextCtrl, vCHomeWrapExtend, 0}}, + {3266, {wxStyledTextCtrl, lineCopy, 0}}, + {3267, {wxStyledTextCtrl, moveCaretInsideView, 0}}, + {3268, {wxStyledTextCtrl, lineLength, 1}}, + {3269, {wxStyledTextCtrl, braceHighlight, 2}}, + {3270, {wxStyledTextCtrl, braceBadLight, 1}}, + {3271, {wxStyledTextCtrl, braceMatch, 1}}, + {3272, {wxStyledTextCtrl, getViewEOL, 0}}, + {3273, {wxStyledTextCtrl, setViewEOL, 1}}, + {3274, {wxStyledTextCtrl, setModEventMask, 1}}, + {3275, {wxStyledTextCtrl, getEdgeColumn, 0}}, + {3276, {wxStyledTextCtrl, setEdgeColumn, 1}}, + {3277, {wxStyledTextCtrl, setEdgeMode, 1}}, + {3278, {wxStyledTextCtrl, getEdgeMode, 0}}, + {3279, {wxStyledTextCtrl, getEdgeColour, 0}}, + {3280, {wxStyledTextCtrl, setEdgeColour, 1}}, + {3281, {wxStyledTextCtrl, searchAnchor, 0}}, + {3282, {wxStyledTextCtrl, searchNext, 2}}, + {3283, {wxStyledTextCtrl, searchPrev, 2}}, + {3284, {wxStyledTextCtrl, linesOnScreen, 0}}, + {3285, {wxStyledTextCtrl, usePopUp, 1}}, + {3286, {wxStyledTextCtrl, selectionIsRectangle, 0}}, + {3287, {wxStyledTextCtrl, setZoom, 1}}, + {3288, {wxStyledTextCtrl, getZoom, 0}}, + {3289, {wxStyledTextCtrl, getModEventMask, 0}}, + {3290, {wxStyledTextCtrl, setSTCFocus, 1}}, + {3291, {wxStyledTextCtrl, getSTCFocus, 0}}, + {3292, {wxStyledTextCtrl, setStatus, 1}}, + {3293, {wxStyledTextCtrl, getStatus, 0}}, + {3294, {wxStyledTextCtrl, setMouseDownCaptures, 1}}, + {3295, {wxStyledTextCtrl, getMouseDownCaptures, 0}}, + {3296, {wxStyledTextCtrl, setSTCCursor, 1}}, + {3297, {wxStyledTextCtrl, getSTCCursor, 0}}, + {3298, {wxStyledTextCtrl, setControlCharSymbol, 1}}, + {3299, {wxStyledTextCtrl, getControlCharSymbol, 0}}, + {3300, {wxStyledTextCtrl, wordPartLeft, 0}}, + {3301, {wxStyledTextCtrl, wordPartLeftExtend, 0}}, + {3302, {wxStyledTextCtrl, wordPartRight, 0}}, + {3303, {wxStyledTextCtrl, wordPartRightExtend, 0}}, + {3304, {wxStyledTextCtrl, setVisiblePolicy, 2}}, + {3305, {wxStyledTextCtrl, delLineLeft, 0}}, + {3306, {wxStyledTextCtrl, delLineRight, 0}}, + {3307, {wxStyledTextCtrl, getXOffset, 0}}, + {3308, {wxStyledTextCtrl, chooseCaretX, 0}}, + {3309, {wxStyledTextCtrl, setXCaretPolicy, 2}}, + {3310, {wxStyledTextCtrl, setYCaretPolicy, 2}}, + {3311, {wxStyledTextCtrl, getPrintWrapMode, 0}}, + {3312, {wxStyledTextCtrl, setHotspotActiveForeground, 2}}, + {3313, {wxStyledTextCtrl, setHotspotActiveBackground, 2}}, + {3314, {wxStyledTextCtrl, setHotspotActiveUnderline, 1}}, + {3315, {wxStyledTextCtrl, setHotspotSingleLine, 1}}, + {3316, {wxStyledTextCtrl, paraDownExtend, 0}}, + {3317, {wxStyledTextCtrl, paraUp, 0}}, + {3318, {wxStyledTextCtrl, paraUpExtend, 0}}, + {3319, {wxStyledTextCtrl, positionBefore, 1}}, + {3320, {wxStyledTextCtrl, positionAfter, 1}}, + {3321, {wxStyledTextCtrl, copyRange, 2}}, + {3322, {wxStyledTextCtrl, copyText, 2}}, + {3323, {wxStyledTextCtrl, setSelectionMode, 1}}, + {3324, {wxStyledTextCtrl, getSelectionMode, 0}}, + {3325, {wxStyledTextCtrl, lineDownRectExtend, 0}}, + {3326, {wxStyledTextCtrl, lineUpRectExtend, 0}}, + {3327, {wxStyledTextCtrl, charLeftRectExtend, 0}}, + {3328, {wxStyledTextCtrl, charRightRectExtend, 0}}, + {3329, {wxStyledTextCtrl, homeRectExtend, 0}}, + {3330, {wxStyledTextCtrl, vCHomeRectExtend, 0}}, + {3331, {wxStyledTextCtrl, lineEndRectExtend, 0}}, + {3332, {wxStyledTextCtrl, pageUpRectExtend, 0}}, + {3333, {wxStyledTextCtrl, pageDownRectExtend, 0}}, + {3334, {wxStyledTextCtrl, stutteredPageUp, 0}}, + {3335, {wxStyledTextCtrl, stutteredPageUpExtend, 0}}, + {3336, {wxStyledTextCtrl, stutteredPageDown, 0}}, + {3337, {wxStyledTextCtrl, stutteredPageDownExtend, 0}}, + {3338, {wxStyledTextCtrl, wordLeftEnd, 0}}, + {3339, {wxStyledTextCtrl, wordLeftEndExtend, 0}}, + {3340, {wxStyledTextCtrl, wordRightEnd, 0}}, + {3341, {wxStyledTextCtrl, wordRightEndExtend, 0}}, + {3342, {wxStyledTextCtrl, setWhitespaceChars, 1}}, + {3343, {wxStyledTextCtrl, setCharsDefault, 0}}, + {3344, {wxStyledTextCtrl, autoCompGetCurrent, 0}}, + {3345, {wxStyledTextCtrl, allocate, 1}}, + {3346, {wxStyledTextCtrl, findColumn, 2}}, + {3347, {wxStyledTextCtrl, getCaretSticky, 0}}, + {3348, {wxStyledTextCtrl, setCaretSticky, 1}}, + {3349, {wxStyledTextCtrl, toggleCaretSticky, 0}}, + {3350, {wxStyledTextCtrl, setPasteConvertEndings, 1}}, + {3351, {wxStyledTextCtrl, getPasteConvertEndings, 0}}, + {3352, {wxStyledTextCtrl, selectionDuplicate, 0}}, + {3353, {wxStyledTextCtrl, setCaretLineBackAlpha, 1}}, + {3354, {wxStyledTextCtrl, getCaretLineBackAlpha, 0}}, + {3355, {wxStyledTextCtrl, startRecord, 0}}, + {3356, {wxStyledTextCtrl, stopRecord, 0}}, + {3357, {wxStyledTextCtrl, setLexer, 1}}, + {3358, {wxStyledTextCtrl, getLexer, 0}}, + {3359, {wxStyledTextCtrl, colourise, 2}}, + {3360, {wxStyledTextCtrl, setProperty, 2}}, + {3361, {wxStyledTextCtrl, setKeyWords, 2}}, + {3362, {wxStyledTextCtrl, setLexerLanguage, 1}}, + {3363, {wxStyledTextCtrl, getProperty, 1}}, + {3364, {wxStyledTextCtrl, getStyleBitsNeeded, 0}}, + {3365, {wxStyledTextCtrl, getCurrentLine, 0}}, + {3366, {wxStyledTextCtrl, styleSetSpec, 2}}, + {3367, {wxStyledTextCtrl, styleSetFont, 2}}, + {3368, {wxStyledTextCtrl, styleSetFontAttr, 7}}, + {3369, {wxStyledTextCtrl, styleSetCharacterSet, 2}}, + {3370, {wxStyledTextCtrl, styleSetFontEncoding, 2}}, + {3371, {wxStyledTextCtrl, cmdKeyExecute, 1}}, + {3372, {wxStyledTextCtrl, setMargins, 2}}, + {3373, {wxStyledTextCtrl, getSelection, 2}}, + {3374, {wxStyledTextCtrl, pointFromPosition, 1}}, + {3375, {wxStyledTextCtrl, scrollToLine, 1}}, + {3376, {wxStyledTextCtrl, scrollToColumn, 1}}, + {3377, {wxStyledTextCtrl, setVScrollBar, 1}}, + {3378, {wxStyledTextCtrl, setHScrollBar, 1}}, + {3379, {wxStyledTextCtrl, getLastKeydownProcessed, 0}}, + {3380, {wxStyledTextCtrl, setLastKeydownProcessed, 1}}, + {3381, {wxStyledTextCtrl, saveFile, 1}}, + {3382, {wxStyledTextCtrl, loadFile, 1}}, + {3383, {wxStyledTextCtrl, doDragOver, 3}}, + {3384, {wxStyledTextCtrl, doDropText, 3}}, + {3385, {wxStyledTextCtrl, getUseAntiAliasing, 0}}, + {3386, {wxStyledTextCtrl, addTextRaw, 1}}, + {3387, {wxStyledTextCtrl, insertTextRaw, 2}}, + {3388, {wxStyledTextCtrl, getCurLineRaw, 1}}, + {3389, {wxStyledTextCtrl, getLineRaw, 1}}, + {3390, {wxStyledTextCtrl, getSelectedTextRaw, 0}}, + {3391, {wxStyledTextCtrl, getTextRangeRaw, 2}}, + {3392, {wxStyledTextCtrl, setTextRaw, 1}}, + {3393, {wxStyledTextCtrl, getTextRaw, 0}}, + {3394, {wxStyledTextCtrl, appendTextRaw, 1}}, + {3395, {wxArtProvider, getBitmap, 2}}, + {3396, {wxArtProvider, getIcon, 2}}, + {3397, {wxTreeEvent, getKeyCode, 0}}, + {3398, {wxTreeEvent, getItem, 0}}, + {3399, {wxTreeEvent, getKeyEvent, 0}}, + {3400, {wxTreeEvent, getLabel, 0}}, + {3401, {wxTreeEvent, getOldItem, 0}}, + {3402, {wxTreeEvent, getPoint, 0}}, + {3403, {wxTreeEvent, isEditCancelled, 0}}, + {3404, {wxTreeEvent, setToolTip, 1}}, + {3405, {wxNotebookEvent, getOldSelection, 0}}, + {3406, {wxNotebookEvent, getSelection, 0}}, + {3407, {wxNotebookEvent, setOldSelection, 1}}, + {3408, {wxNotebookEvent, setSelection, 1}}, + {3409, {wxFileDataObject, new, 0}}, + {3410, {wxFileDataObject, addFile, 1}}, + {3411, {wxFileDataObject, getFilenames, 0}}, + {3412, {wxFileDataObject, 'Destroy', undefined}}, + {3413, {wxTextDataObject, new, 1}}, + {3414, {wxTextDataObject, getTextLength, 0}}, + {3415, {wxTextDataObject, getText, 0}}, + {3416, {wxTextDataObject, setText, 1}}, + {3417, {wxTextDataObject, 'Destroy', undefined}}, + {3418, {wxBitmapDataObject, new_1_1, 1}}, + {3419, {wxBitmapDataObject, new_1_0, 1}}, + {3420, {wxBitmapDataObject, getBitmap, 0}}, + {3421, {wxBitmapDataObject, setBitmap, 1}}, + {3422, {wxBitmapDataObject, 'Destroy', undefined}}, + {3424, {wxClipboard, new, 0}}, + {3425, {wxClipboard, destruct, 0}}, + {3426, {wxClipboard, addData, 1}}, + {3427, {wxClipboard, clear, 0}}, + {3428, {wxClipboard, close, 0}}, + {3429, {wxClipboard, flush, 0}}, + {3430, {wxClipboard, getData, 1}}, + {3431, {wxClipboard, isOpened, 0}}, + {3432, {wxClipboard, open, 0}}, + {3433, {wxClipboard, setData, 1}}, + {3435, {wxClipboard, usePrimarySelection, 1}}, + {3436, {wxClipboard, isSupported, 1}}, + {3437, {wxClipboard, get, 0}}, + {3438, {wxSpinEvent, getPosition, 0}}, + {3439, {wxSpinEvent, setPosition, 1}}, + {3440, {wxSplitterWindow, new_0, 0}}, + {3441, {wxSplitterWindow, new_2, 2}}, + {3442, {wxSplitterWindow, destruct, 0}}, + {3443, {wxSplitterWindow, create, 2}}, + {3444, {wxSplitterWindow, getMinimumPaneSize, 0}}, + {3445, {wxSplitterWindow, getSashGravity, 0}}, + {3446, {wxSplitterWindow, getSashPosition, 0}}, + {3447, {wxSplitterWindow, getSplitMode, 0}}, + {3448, {wxSplitterWindow, getWindow1, 0}}, + {3449, {wxSplitterWindow, getWindow2, 0}}, + {3450, {wxSplitterWindow, initialize, 1}}, + {3451, {wxSplitterWindow, isSplit, 0}}, + {3452, {wxSplitterWindow, replaceWindow, 2}}, + {3453, {wxSplitterWindow, setSashGravity, 1}}, + {3454, {wxSplitterWindow, setSashPosition, 2}}, + {3455, {wxSplitterWindow, setSashSize, 1}}, + {3456, {wxSplitterWindow, setMinimumPaneSize, 1}}, + {3457, {wxSplitterWindow, setSplitMode, 1}}, + {3458, {wxSplitterWindow, splitHorizontally, 3}}, + {3459, {wxSplitterWindow, splitVertically, 3}}, + {3460, {wxSplitterWindow, unsplit, 1}}, + {3461, {wxSplitterWindow, updateSize, 0}}, + {3462, {wxSplitterEvent, getSashPosition, 0}}, + {3463, {wxSplitterEvent, getX, 0}}, + {3464, {wxSplitterEvent, getY, 0}}, + {3465, {wxSplitterEvent, getWindowBeingRemoved, 0}}, + {3466, {wxSplitterEvent, setSashPosition, 1}}, + {3467, {wxHtmlWindow, new_0, 0}}, + {3468, {wxHtmlWindow, new_2, 2}}, + {3469, {wxHtmlWindow, appendToPage, 1}}, + {3470, {wxHtmlWindow, getOpenedAnchor, 0}}, + {3471, {wxHtmlWindow, getOpenedPage, 0}}, + {3472, {wxHtmlWindow, getOpenedPageTitle, 0}}, + {3473, {wxHtmlWindow, getRelatedFrame, 0}}, + {3474, {wxHtmlWindow, historyBack, 0}}, + {3475, {wxHtmlWindow, historyCanBack, 0}}, + {3476, {wxHtmlWindow, historyCanForward, 0}}, + {3477, {wxHtmlWindow, historyClear, 0}}, + {3478, {wxHtmlWindow, historyForward, 0}}, + {3479, {wxHtmlWindow, loadFile, 1}}, + {3480, {wxHtmlWindow, loadPage, 1}}, + {3481, {wxHtmlWindow, selectAll, 0}}, + {3482, {wxHtmlWindow, selectionToText, 0}}, + {3483, {wxHtmlWindow, selectLine, 1}}, + {3484, {wxHtmlWindow, selectWord, 1}}, + {3485, {wxHtmlWindow, setBorders, 1}}, + {3486, {wxHtmlWindow, setFonts, 3}}, + {3487, {wxHtmlWindow, setPage, 1}}, + {3488, {wxHtmlWindow, setRelatedFrame, 2}}, + {3489, {wxHtmlWindow, setRelatedStatusBar, 1}}, + {3490, {wxHtmlWindow, toText, 0}}, + {3491, {wxHtmlWindow, 'Destroy', undefined}}, + {3492, {wxHtmlLinkEvent, getLinkInfo, 0}}, + {3493, {wxSystemSettings, getColour, 1}}, + {3494, {wxSystemSettings, getFont, 1}}, + {3495, {wxSystemSettings, getMetric, 2}}, + {3496, {wxSystemSettings, getScreenType, 0}}, + {3497, {wxSystemOptions, getOption, 1}}, + {3498, {wxSystemOptions, getOptionInt, 1}}, + {3499, {wxSystemOptions, hasOption, 1}}, + {3500, {wxSystemOptions, isFalse, 1}}, + {3501, {wxSystemOptions, setOption_2_1, 2}}, + {3502, {wxSystemOptions, setOption_2_0, 2}}, + {3503, {wxAuiNotebookEvent, setSelection, 1}}, + {3504, {wxAuiNotebookEvent, getSelection, 0}}, + {3505, {wxAuiNotebookEvent, setOldSelection, 1}}, + {3506, {wxAuiNotebookEvent, getOldSelection, 0}}, + {3507, {wxAuiNotebookEvent, setDragSource, 1}}, + {3508, {wxAuiNotebookEvent, getDragSource, 0}}, + {3509, {wxAuiManagerEvent, setManager, 1}}, + {3510, {wxAuiManagerEvent, getManager, 0}}, + {3511, {wxAuiManagerEvent, setPane, 1}}, + {3512, {wxAuiManagerEvent, getPane, 0}}, + {3513, {wxAuiManagerEvent, setButton, 1}}, + {3514, {wxAuiManagerEvent, getButton, 0}}, + {3515, {wxAuiManagerEvent, setDC, 1}}, + {3516, {wxAuiManagerEvent, getDC, 0}}, + {3517, {wxAuiManagerEvent, veto, 1}}, + {3518, {wxAuiManagerEvent, getVeto, 0}}, + {3519, {wxAuiManagerEvent, setCanVeto, 1}}, + {3520, {wxAuiManagerEvent, canVeto, 0}}, + {3521, {wxLogNull, new, 0}}, + {3522, {wxLogNull, 'Destroy', undefined}}, + {3523, {wxTaskBarIcon, new, 0}}, + {3524, {wxTaskBarIcon, destruct, 0}}, + {3525, {wxTaskBarIcon, popupMenu, 1}}, + {3526, {wxTaskBarIcon, removeIcon, 0}}, + {3527, {wxTaskBarIcon, setIcon, 2}}, + {3528, {wxLocale, new_0, 0}}, + {3530, {wxLocale, new_2, 2}}, + {3531, {wxLocale, destruct, 0}}, + {3533, {wxLocale, init, 1}}, + {3534, {wxLocale, addCatalog_1, 1}}, + {3535, {wxLocale, addCatalog_3, 3}}, + {3536, {wxLocale, addCatalogLookupPathPrefix, 1}}, + {3537, {wxLocale, getCanonicalName, 0}}, + {3538, {wxLocale, getLanguage, 0}}, + {3539, {wxLocale, getLanguageName, 1}}, + {3540, {wxLocale, getLocale, 0}}, + {3541, {wxLocale, getName, 0}}, + {3542, {wxLocale, getString_2, 2}}, + {3543, {wxLocale, getString_4, 4}}, + {3544, {wxLocale, getHeaderValue, 2}}, + {3545, {wxLocale, getSysName, 0}}, + {3546, {wxLocale, getSystemEncoding, 0}}, + {3547, {wxLocale, getSystemEncodingName, 0}}, + {3548, {wxLocale, getSystemLanguage, 0}}, + {3549, {wxLocale, isLoaded, 1}}, + {3550, {wxLocale, isOk, 0}}, + {3551, {wxActivateEvent, getActive, 0}}, + {3553, {wxPopupWindow, new_2, 2}}, + {3554, {wxPopupWindow, new_0, 0}}, + {3556, {wxPopupWindow, destruct, 0}}, + {3557, {wxPopupWindow, create, 2}}, + {3558, {wxPopupWindow, position, 2}}, + {3559, {wxPopupTransientWindow, new_0, 0}}, + {3560, {wxPopupTransientWindow, new_2, 2}}, + {3561, {wxPopupTransientWindow, destruct, 0}}, + {3562, {wxPopupTransientWindow, popup, 1}}, + {3563, {wxPopupTransientWindow, dismiss, 0}}, {-1, {mod, func, -1}} ]. diff --git a/lib/wx/src/gen/wxe_funcs.hrl b/lib/wx/src/gen/wxe_funcs.hrl index 109368adb3..82d0d265d7 100644 --- a/lib/wx/src/gen/wxe_funcs.hrl +++ b/lib/wx/src/gen/wxe_funcs.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2013. All Rights Reserved. +%% Copyright Ericsson AB 2008-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -1666,1639 +1666,1674 @@ -define(wxTextCtrl_Create, 1824). -define(wxTextCtrl_Cut, 1825). -define(wxTextCtrl_DiscardEdits, 1826). --define(wxTextCtrl_EmulateKeyPress, 1827). --define(wxTextCtrl_GetDefaultStyle, 1828). --define(wxTextCtrl_GetInsertionPoint, 1829). --define(wxTextCtrl_GetLastPosition, 1830). --define(wxTextCtrl_GetLineLength, 1831). --define(wxTextCtrl_GetLineText, 1832). --define(wxTextCtrl_GetNumberOfLines, 1833). --define(wxTextCtrl_GetRange, 1834). --define(wxTextCtrl_GetSelection, 1835). --define(wxTextCtrl_GetStringSelection, 1836). --define(wxTextCtrl_GetStyle, 1837). --define(wxTextCtrl_GetValue, 1838). --define(wxTextCtrl_IsEditable, 1839). --define(wxTextCtrl_IsModified, 1840). --define(wxTextCtrl_IsMultiLine, 1841). --define(wxTextCtrl_IsSingleLine, 1842). --define(wxTextCtrl_LoadFile, 1843). --define(wxTextCtrl_MarkDirty, 1844). --define(wxTextCtrl_Paste, 1845). --define(wxTextCtrl_PositionToXY, 1846). --define(wxTextCtrl_Redo, 1847). --define(wxTextCtrl_Remove, 1848). --define(wxTextCtrl_Replace, 1849). --define(wxTextCtrl_SaveFile, 1850). --define(wxTextCtrl_SetDefaultStyle, 1851). --define(wxTextCtrl_SetEditable, 1852). --define(wxTextCtrl_SetInsertionPoint, 1853). --define(wxTextCtrl_SetInsertionPointEnd, 1854). --define(wxTextCtrl_SetMaxLength, 1856). --define(wxTextCtrl_SetSelection, 1857). --define(wxTextCtrl_SetStyle, 1858). --define(wxTextCtrl_SetValue, 1859). --define(wxTextCtrl_ShowPosition, 1860). --define(wxTextCtrl_Undo, 1861). --define(wxTextCtrl_WriteText, 1862). --define(wxTextCtrl_XYToPosition, 1863). --define(wxNotebook_new_0, 1866). --define(wxNotebook_new_3, 1867). --define(wxNotebook_destruct, 1868). --define(wxNotebook_AddPage, 1869). --define(wxNotebook_AdvanceSelection, 1870). --define(wxNotebook_AssignImageList, 1871). --define(wxNotebook_Create, 1872). --define(wxNotebook_DeleteAllPages, 1873). --define(wxNotebook_DeletePage, 1874). --define(wxNotebook_RemovePage, 1875). --define(wxNotebook_GetCurrentPage, 1876). --define(wxNotebook_GetImageList, 1877). --define(wxNotebook_GetPage, 1879). --define(wxNotebook_GetPageCount, 1880). --define(wxNotebook_GetPageImage, 1881). --define(wxNotebook_GetPageText, 1882). --define(wxNotebook_GetRowCount, 1883). --define(wxNotebook_GetSelection, 1884). --define(wxNotebook_GetThemeBackgroundColour, 1885). --define(wxNotebook_HitTest, 1887). --define(wxNotebook_InsertPage, 1889). --define(wxNotebook_SetImageList, 1890). --define(wxNotebook_SetPadding, 1891). --define(wxNotebook_SetPageSize, 1892). --define(wxNotebook_SetPageImage, 1893). --define(wxNotebook_SetPageText, 1894). --define(wxNotebook_SetSelection, 1895). --define(wxNotebook_ChangeSelection, 1896). --define(wxChoicebook_new_0, 1897). --define(wxChoicebook_new_3, 1898). --define(wxChoicebook_AddPage, 1899). --define(wxChoicebook_AdvanceSelection, 1900). --define(wxChoicebook_AssignImageList, 1901). --define(wxChoicebook_Create, 1902). --define(wxChoicebook_DeleteAllPages, 1903). --define(wxChoicebook_DeletePage, 1904). --define(wxChoicebook_RemovePage, 1905). --define(wxChoicebook_GetCurrentPage, 1906). --define(wxChoicebook_GetImageList, 1907). --define(wxChoicebook_GetPage, 1909). --define(wxChoicebook_GetPageCount, 1910). --define(wxChoicebook_GetPageImage, 1911). --define(wxChoicebook_GetPageText, 1912). --define(wxChoicebook_GetSelection, 1913). --define(wxChoicebook_HitTest, 1914). --define(wxChoicebook_InsertPage, 1915). --define(wxChoicebook_SetImageList, 1916). --define(wxChoicebook_SetPageSize, 1917). --define(wxChoicebook_SetPageImage, 1918). --define(wxChoicebook_SetPageText, 1919). --define(wxChoicebook_SetSelection, 1920). --define(wxChoicebook_ChangeSelection, 1921). --define(wxChoicebook_destroy, 1922). --define(wxToolbook_new_0, 1923). --define(wxToolbook_new_3, 1924). --define(wxToolbook_AddPage, 1925). --define(wxToolbook_AdvanceSelection, 1926). --define(wxToolbook_AssignImageList, 1927). --define(wxToolbook_Create, 1928). --define(wxToolbook_DeleteAllPages, 1929). --define(wxToolbook_DeletePage, 1930). --define(wxToolbook_RemovePage, 1931). --define(wxToolbook_GetCurrentPage, 1932). --define(wxToolbook_GetImageList, 1933). --define(wxToolbook_GetPage, 1935). --define(wxToolbook_GetPageCount, 1936). --define(wxToolbook_GetPageImage, 1937). --define(wxToolbook_GetPageText, 1938). --define(wxToolbook_GetSelection, 1939). --define(wxToolbook_HitTest, 1941). --define(wxToolbook_InsertPage, 1942). --define(wxToolbook_SetImageList, 1943). --define(wxToolbook_SetPageSize, 1944). --define(wxToolbook_SetPageImage, 1945). --define(wxToolbook_SetPageText, 1946). --define(wxToolbook_SetSelection, 1947). --define(wxToolbook_ChangeSelection, 1948). --define(wxToolbook_destroy, 1949). --define(wxListbook_new_0, 1950). --define(wxListbook_new_3, 1951). --define(wxListbook_AddPage, 1952). --define(wxListbook_AdvanceSelection, 1953). --define(wxListbook_AssignImageList, 1954). --define(wxListbook_Create, 1955). --define(wxListbook_DeleteAllPages, 1956). --define(wxListbook_DeletePage, 1957). --define(wxListbook_RemovePage, 1958). --define(wxListbook_GetCurrentPage, 1959). --define(wxListbook_GetImageList, 1960). --define(wxListbook_GetPage, 1962). --define(wxListbook_GetPageCount, 1963). --define(wxListbook_GetPageImage, 1964). --define(wxListbook_GetPageText, 1965). --define(wxListbook_GetSelection, 1966). --define(wxListbook_HitTest, 1968). --define(wxListbook_InsertPage, 1969). --define(wxListbook_SetImageList, 1970). --define(wxListbook_SetPageSize, 1971). --define(wxListbook_SetPageImage, 1972). --define(wxListbook_SetPageText, 1973). --define(wxListbook_SetSelection, 1974). --define(wxListbook_ChangeSelection, 1975). --define(wxListbook_destroy, 1976). --define(wxTreebook_new_0, 1977). --define(wxTreebook_new_3, 1978). --define(wxTreebook_AddPage, 1979). --define(wxTreebook_AdvanceSelection, 1980). --define(wxTreebook_AssignImageList, 1981). --define(wxTreebook_Create, 1982). --define(wxTreebook_DeleteAllPages, 1983). --define(wxTreebook_DeletePage, 1984). --define(wxTreebook_RemovePage, 1985). --define(wxTreebook_GetCurrentPage, 1986). --define(wxTreebook_GetImageList, 1987). --define(wxTreebook_GetPage, 1989). --define(wxTreebook_GetPageCount, 1990). --define(wxTreebook_GetPageImage, 1991). --define(wxTreebook_GetPageText, 1992). --define(wxTreebook_GetSelection, 1993). --define(wxTreebook_ExpandNode, 1994). --define(wxTreebook_IsNodeExpanded, 1995). --define(wxTreebook_HitTest, 1997). --define(wxTreebook_InsertPage, 1998). --define(wxTreebook_InsertSubPage, 1999). --define(wxTreebook_SetImageList, 2000). --define(wxTreebook_SetPageSize, 2001). --define(wxTreebook_SetPageImage, 2002). --define(wxTreebook_SetPageText, 2003). --define(wxTreebook_SetSelection, 2004). --define(wxTreebook_ChangeSelection, 2005). --define(wxTreebook_destroy, 2006). --define(wxTreeCtrl_new_2, 2009). --define(wxTreeCtrl_new_0, 2010). --define(wxTreeCtrl_destruct, 2012). --define(wxTreeCtrl_AddRoot, 2013). --define(wxTreeCtrl_AppendItem, 2014). --define(wxTreeCtrl_AssignImageList, 2015). --define(wxTreeCtrl_AssignStateImageList, 2016). --define(wxTreeCtrl_Collapse, 2017). --define(wxTreeCtrl_CollapseAndReset, 2018). --define(wxTreeCtrl_Create, 2019). --define(wxTreeCtrl_Delete, 2020). --define(wxTreeCtrl_DeleteAllItems, 2021). --define(wxTreeCtrl_DeleteChildren, 2022). --define(wxTreeCtrl_EditLabel, 2023). --define(wxTreeCtrl_EnsureVisible, 2024). --define(wxTreeCtrl_Expand, 2025). --define(wxTreeCtrl_GetBoundingRect, 2026). --define(wxTreeCtrl_GetChildrenCount, 2028). --define(wxTreeCtrl_GetCount, 2029). --define(wxTreeCtrl_GetEditControl, 2030). --define(wxTreeCtrl_GetFirstChild, 2031). --define(wxTreeCtrl_GetNextChild, 2032). --define(wxTreeCtrl_GetFirstVisibleItem, 2033). --define(wxTreeCtrl_GetImageList, 2034). --define(wxTreeCtrl_GetIndent, 2035). --define(wxTreeCtrl_GetItemBackgroundColour, 2036). --define(wxTreeCtrl_GetItemData, 2037). --define(wxTreeCtrl_GetItemFont, 2038). --define(wxTreeCtrl_GetItemImage_1, 2039). --define(wxTreeCtrl_GetItemImage_2, 2040). --define(wxTreeCtrl_GetItemText, 2041). --define(wxTreeCtrl_GetItemTextColour, 2042). --define(wxTreeCtrl_GetLastChild, 2043). --define(wxTreeCtrl_GetNextSibling, 2044). --define(wxTreeCtrl_GetNextVisible, 2045). --define(wxTreeCtrl_GetItemParent, 2046). --define(wxTreeCtrl_GetPrevSibling, 2047). --define(wxTreeCtrl_GetPrevVisible, 2048). --define(wxTreeCtrl_GetRootItem, 2049). --define(wxTreeCtrl_GetSelection, 2050). --define(wxTreeCtrl_GetSelections, 2051). --define(wxTreeCtrl_GetStateImageList, 2052). --define(wxTreeCtrl_HitTest, 2053). --define(wxTreeCtrl_InsertItem, 2055). --define(wxTreeCtrl_IsBold, 2056). --define(wxTreeCtrl_IsExpanded, 2057). --define(wxTreeCtrl_IsSelected, 2058). --define(wxTreeCtrl_IsVisible, 2059). --define(wxTreeCtrl_ItemHasChildren, 2060). --define(wxTreeCtrl_IsTreeItemIdOk, 2061). --define(wxTreeCtrl_PrependItem, 2062). --define(wxTreeCtrl_ScrollTo, 2063). --define(wxTreeCtrl_SelectItem_1, 2064). --define(wxTreeCtrl_SelectItem_2, 2065). --define(wxTreeCtrl_SetIndent, 2066). --define(wxTreeCtrl_SetImageList, 2067). --define(wxTreeCtrl_SetItemBackgroundColour, 2068). --define(wxTreeCtrl_SetItemBold, 2069). --define(wxTreeCtrl_SetItemData, 2070). --define(wxTreeCtrl_SetItemDropHighlight, 2071). --define(wxTreeCtrl_SetItemFont, 2072). --define(wxTreeCtrl_SetItemHasChildren, 2073). --define(wxTreeCtrl_SetItemImage_2, 2074). --define(wxTreeCtrl_SetItemImage_3, 2075). --define(wxTreeCtrl_SetItemText, 2076). --define(wxTreeCtrl_SetItemTextColour, 2077). --define(wxTreeCtrl_SetStateImageList, 2078). --define(wxTreeCtrl_SetWindowStyle, 2079). --define(wxTreeCtrl_SortChildren, 2080). --define(wxTreeCtrl_Toggle, 2081). --define(wxTreeCtrl_ToggleItemSelection, 2082). --define(wxTreeCtrl_Unselect, 2083). --define(wxTreeCtrl_UnselectAll, 2084). --define(wxTreeCtrl_UnselectItem, 2085). --define(wxScrollBar_new_0, 2086). --define(wxScrollBar_new_3, 2087). --define(wxScrollBar_destruct, 2088). --define(wxScrollBar_Create, 2089). --define(wxScrollBar_GetRange, 2090). --define(wxScrollBar_GetPageSize, 2091). --define(wxScrollBar_GetThumbPosition, 2092). --define(wxScrollBar_GetThumbSize, 2093). --define(wxScrollBar_SetThumbPosition, 2094). --define(wxScrollBar_SetScrollbar, 2095). --define(wxSpinButton_new_2, 2097). --define(wxSpinButton_new_0, 2098). --define(wxSpinButton_Create, 2099). --define(wxSpinButton_GetMax, 2100). --define(wxSpinButton_GetMin, 2101). --define(wxSpinButton_GetValue, 2102). --define(wxSpinButton_SetRange, 2103). --define(wxSpinButton_SetValue, 2104). --define(wxSpinButton_destroy, 2105). --define(wxSpinCtrl_new_0, 2106). --define(wxSpinCtrl_new_2, 2107). --define(wxSpinCtrl_Create, 2109). --define(wxSpinCtrl_SetValue_1_1, 2112). --define(wxSpinCtrl_SetValue_1_0, 2113). --define(wxSpinCtrl_GetValue, 2115). --define(wxSpinCtrl_SetRange, 2117). --define(wxSpinCtrl_SetSelection, 2118). --define(wxSpinCtrl_GetMin, 2120). --define(wxSpinCtrl_GetMax, 2122). --define(wxSpinCtrl_destroy, 2123). --define(wxStaticText_new_0, 2124). --define(wxStaticText_new_4, 2125). --define(wxStaticText_Create, 2126). --define(wxStaticText_GetLabel, 2127). --define(wxStaticText_SetLabel, 2128). --define(wxStaticText_Wrap, 2129). --define(wxStaticText_destroy, 2130). --define(wxStaticBitmap_new_0, 2131). --define(wxStaticBitmap_new_4, 2132). --define(wxStaticBitmap_Create, 2133). --define(wxStaticBitmap_GetBitmap, 2134). --define(wxStaticBitmap_SetBitmap, 2135). --define(wxStaticBitmap_destroy, 2136). --define(wxRadioBox_new, 2137). --define(wxRadioBox_destruct, 2139). --define(wxRadioBox_Create, 2140). --define(wxRadioBox_Enable_2, 2141). --define(wxRadioBox_Enable_1, 2142). --define(wxRadioBox_GetSelection, 2143). --define(wxRadioBox_GetString, 2144). --define(wxRadioBox_SetSelection, 2145). --define(wxRadioBox_Show_2, 2146). --define(wxRadioBox_Show_1, 2147). --define(wxRadioBox_GetColumnCount, 2148). --define(wxRadioBox_GetItemHelpText, 2149). --define(wxRadioBox_GetItemToolTip, 2150). --define(wxRadioBox_GetItemFromPoint, 2152). --define(wxRadioBox_GetRowCount, 2153). --define(wxRadioBox_IsItemEnabled, 2154). --define(wxRadioBox_IsItemShown, 2155). --define(wxRadioBox_SetItemHelpText, 2156). --define(wxRadioBox_SetItemToolTip, 2157). --define(wxRadioButton_new_0, 2158). --define(wxRadioButton_new_4, 2159). --define(wxRadioButton_Create, 2160). --define(wxRadioButton_GetValue, 2161). --define(wxRadioButton_SetValue, 2162). --define(wxRadioButton_destroy, 2163). --define(wxSlider_new_6, 2165). --define(wxSlider_new_0, 2166). --define(wxSlider_Create, 2167). --define(wxSlider_GetLineSize, 2168). --define(wxSlider_GetMax, 2169). --define(wxSlider_GetMin, 2170). --define(wxSlider_GetPageSize, 2171). --define(wxSlider_GetThumbLength, 2172). --define(wxSlider_GetValue, 2173). --define(wxSlider_SetLineSize, 2174). --define(wxSlider_SetPageSize, 2175). --define(wxSlider_SetRange, 2176). --define(wxSlider_SetThumbLength, 2177). --define(wxSlider_SetValue, 2178). --define(wxSlider_destroy, 2179). --define(wxDialog_new_4, 2181). --define(wxDialog_new_0, 2182). --define(wxDialog_destruct, 2184). --define(wxDialog_Create, 2185). --define(wxDialog_CreateButtonSizer, 2186). --define(wxDialog_CreateStdDialogButtonSizer, 2187). --define(wxDialog_EndModal, 2188). --define(wxDialog_GetAffirmativeId, 2189). --define(wxDialog_GetReturnCode, 2190). --define(wxDialog_IsModal, 2191). --define(wxDialog_SetAffirmativeId, 2192). --define(wxDialog_SetReturnCode, 2193). --define(wxDialog_Show, 2194). --define(wxDialog_ShowModal, 2195). --define(wxColourDialog_new_0, 2196). --define(wxColourDialog_new_2, 2197). --define(wxColourDialog_destruct, 2198). --define(wxColourDialog_Create, 2199). --define(wxColourDialog_GetColourData, 2200). --define(wxColourData_new_0, 2201). --define(wxColourData_new_1, 2202). --define(wxColourData_destruct, 2203). --define(wxColourData_GetChooseFull, 2204). --define(wxColourData_GetColour, 2205). --define(wxColourData_GetCustomColour, 2207). --define(wxColourData_SetChooseFull, 2208). --define(wxColourData_SetColour, 2209). --define(wxColourData_SetCustomColour, 2210). --define(wxPalette_new_0, 2211). --define(wxPalette_new_4, 2212). --define(wxPalette_destruct, 2214). --define(wxPalette_Create, 2215). --define(wxPalette_GetColoursCount, 2216). --define(wxPalette_GetPixel, 2217). --define(wxPalette_GetRGB, 2218). --define(wxPalette_IsOk, 2219). --define(wxDirDialog_new, 2223). --define(wxDirDialog_destruct, 2224). --define(wxDirDialog_GetPath, 2225). --define(wxDirDialog_GetMessage, 2226). --define(wxDirDialog_SetMessage, 2227). --define(wxDirDialog_SetPath, 2228). --define(wxFileDialog_new, 2232). --define(wxFileDialog_destruct, 2233). --define(wxFileDialog_GetDirectory, 2234). --define(wxFileDialog_GetFilename, 2235). --define(wxFileDialog_GetFilenames, 2236). --define(wxFileDialog_GetFilterIndex, 2237). --define(wxFileDialog_GetMessage, 2238). --define(wxFileDialog_GetPath, 2239). --define(wxFileDialog_GetPaths, 2240). --define(wxFileDialog_GetWildcard, 2241). --define(wxFileDialog_SetDirectory, 2242). --define(wxFileDialog_SetFilename, 2243). --define(wxFileDialog_SetFilterIndex, 2244). --define(wxFileDialog_SetMessage, 2245). --define(wxFileDialog_SetPath, 2246). --define(wxFileDialog_SetWildcard, 2247). --define(wxPickerBase_SetInternalMargin, 2248). --define(wxPickerBase_GetInternalMargin, 2249). --define(wxPickerBase_SetTextCtrlProportion, 2250). --define(wxPickerBase_SetPickerCtrlProportion, 2251). --define(wxPickerBase_GetTextCtrlProportion, 2252). --define(wxPickerBase_GetPickerCtrlProportion, 2253). --define(wxPickerBase_HasTextCtrl, 2254). --define(wxPickerBase_GetTextCtrl, 2255). --define(wxPickerBase_IsTextCtrlGrowable, 2256). --define(wxPickerBase_SetPickerCtrlGrowable, 2257). --define(wxPickerBase_SetTextCtrlGrowable, 2258). --define(wxPickerBase_IsPickerCtrlGrowable, 2259). --define(wxFilePickerCtrl_new_0, 2260). --define(wxFilePickerCtrl_new_3, 2261). --define(wxFilePickerCtrl_Create, 2262). --define(wxFilePickerCtrl_GetPath, 2263). --define(wxFilePickerCtrl_SetPath, 2264). --define(wxFilePickerCtrl_destroy, 2265). --define(wxDirPickerCtrl_new_0, 2266). --define(wxDirPickerCtrl_new_3, 2267). --define(wxDirPickerCtrl_Create, 2268). --define(wxDirPickerCtrl_GetPath, 2269). --define(wxDirPickerCtrl_SetPath, 2270). --define(wxDirPickerCtrl_destroy, 2271). --define(wxColourPickerCtrl_new_0, 2272). --define(wxColourPickerCtrl_new_3, 2273). --define(wxColourPickerCtrl_Create, 2274). --define(wxColourPickerCtrl_GetColour, 2275). --define(wxColourPickerCtrl_SetColour_1_1, 2276). --define(wxColourPickerCtrl_SetColour_1_0, 2277). --define(wxColourPickerCtrl_destroy, 2278). --define(wxDatePickerCtrl_new_0, 2279). --define(wxDatePickerCtrl_new_3, 2280). --define(wxDatePickerCtrl_GetRange, 2281). --define(wxDatePickerCtrl_GetValue, 2282). --define(wxDatePickerCtrl_SetRange, 2283). --define(wxDatePickerCtrl_SetValue, 2284). --define(wxDatePickerCtrl_destroy, 2285). --define(wxFontPickerCtrl_new_0, 2286). --define(wxFontPickerCtrl_new_3, 2287). --define(wxFontPickerCtrl_Create, 2288). --define(wxFontPickerCtrl_GetSelectedFont, 2289). --define(wxFontPickerCtrl_SetSelectedFont, 2290). --define(wxFontPickerCtrl_GetMaxPointSize, 2291). --define(wxFontPickerCtrl_SetMaxPointSize, 2292). --define(wxFontPickerCtrl_destroy, 2293). --define(wxFindReplaceDialog_new_0, 2296). --define(wxFindReplaceDialog_new_4, 2297). --define(wxFindReplaceDialog_destruct, 2298). --define(wxFindReplaceDialog_Create, 2299). --define(wxFindReplaceDialog_GetData, 2300). --define(wxFindReplaceData_new_0, 2301). --define(wxFindReplaceData_new_1, 2302). --define(wxFindReplaceData_GetFindString, 2303). --define(wxFindReplaceData_GetReplaceString, 2304). --define(wxFindReplaceData_GetFlags, 2305). --define(wxFindReplaceData_SetFlags, 2306). --define(wxFindReplaceData_SetFindString, 2307). --define(wxFindReplaceData_SetReplaceString, 2308). --define(wxFindReplaceData_destroy, 2309). --define(wxMultiChoiceDialog_new_0, 2310). --define(wxMultiChoiceDialog_new_5, 2312). --define(wxMultiChoiceDialog_GetSelections, 2313). --define(wxMultiChoiceDialog_SetSelections, 2314). --define(wxMultiChoiceDialog_destroy, 2315). --define(wxSingleChoiceDialog_new_0, 2316). --define(wxSingleChoiceDialog_new_5, 2318). --define(wxSingleChoiceDialog_GetSelection, 2319). --define(wxSingleChoiceDialog_GetStringSelection, 2320). --define(wxSingleChoiceDialog_SetSelection, 2321). --define(wxSingleChoiceDialog_destroy, 2322). --define(wxTextEntryDialog_new, 2323). --define(wxTextEntryDialog_GetValue, 2324). --define(wxTextEntryDialog_SetValue, 2325). --define(wxTextEntryDialog_destroy, 2326). --define(wxPasswordEntryDialog_new, 2327). --define(wxPasswordEntryDialog_destroy, 2328). --define(wxFontData_new_0, 2329). --define(wxFontData_new_1, 2330). --define(wxFontData_destruct, 2331). --define(wxFontData_EnableEffects, 2332). --define(wxFontData_GetAllowSymbols, 2333). --define(wxFontData_GetColour, 2334). --define(wxFontData_GetChosenFont, 2335). --define(wxFontData_GetEnableEffects, 2336). --define(wxFontData_GetInitialFont, 2337). --define(wxFontData_GetShowHelp, 2338). --define(wxFontData_SetAllowSymbols, 2339). --define(wxFontData_SetChosenFont, 2340). --define(wxFontData_SetColour, 2341). --define(wxFontData_SetInitialFont, 2342). --define(wxFontData_SetRange, 2343). --define(wxFontData_SetShowHelp, 2344). --define(wxFontDialog_new_0, 2348). --define(wxFontDialog_new_2, 2350). --define(wxFontDialog_Create, 2352). --define(wxFontDialog_GetFontData, 2353). --define(wxFontDialog_destroy, 2355). --define(wxProgressDialog_new, 2356). --define(wxProgressDialog_destruct, 2357). --define(wxProgressDialog_Resume, 2358). --define(wxProgressDialog_Update_2, 2359). --define(wxProgressDialog_Update_0, 2360). --define(wxMessageDialog_new, 2361). --define(wxMessageDialog_destruct, 2362). --define(wxPageSetupDialog_new, 2363). --define(wxPageSetupDialog_destruct, 2364). --define(wxPageSetupDialog_GetPageSetupData, 2365). --define(wxPageSetupDialog_ShowModal, 2366). --define(wxPageSetupDialogData_new_0, 2367). --define(wxPageSetupDialogData_new_1_0, 2368). --define(wxPageSetupDialogData_new_1_1, 2369). --define(wxPageSetupDialogData_destruct, 2370). --define(wxPageSetupDialogData_EnableHelp, 2371). --define(wxPageSetupDialogData_EnableMargins, 2372). --define(wxPageSetupDialogData_EnableOrientation, 2373). --define(wxPageSetupDialogData_EnablePaper, 2374). --define(wxPageSetupDialogData_EnablePrinter, 2375). --define(wxPageSetupDialogData_GetDefaultMinMargins, 2376). --define(wxPageSetupDialogData_GetEnableMargins, 2377). --define(wxPageSetupDialogData_GetEnableOrientation, 2378). --define(wxPageSetupDialogData_GetEnablePaper, 2379). --define(wxPageSetupDialogData_GetEnablePrinter, 2380). --define(wxPageSetupDialogData_GetEnableHelp, 2381). --define(wxPageSetupDialogData_GetDefaultInfo, 2382). --define(wxPageSetupDialogData_GetMarginTopLeft, 2383). --define(wxPageSetupDialogData_GetMarginBottomRight, 2384). --define(wxPageSetupDialogData_GetMinMarginTopLeft, 2385). --define(wxPageSetupDialogData_GetMinMarginBottomRight, 2386). --define(wxPageSetupDialogData_GetPaperId, 2387). --define(wxPageSetupDialogData_GetPaperSize, 2388). --define(wxPageSetupDialogData_GetPrintData, 2390). --define(wxPageSetupDialogData_IsOk, 2391). --define(wxPageSetupDialogData_SetDefaultInfo, 2392). --define(wxPageSetupDialogData_SetDefaultMinMargins, 2393). --define(wxPageSetupDialogData_SetMarginTopLeft, 2394). --define(wxPageSetupDialogData_SetMarginBottomRight, 2395). --define(wxPageSetupDialogData_SetMinMarginTopLeft, 2396). --define(wxPageSetupDialogData_SetMinMarginBottomRight, 2397). --define(wxPageSetupDialogData_SetPaperId, 2398). --define(wxPageSetupDialogData_SetPaperSize_1_1, 2399). --define(wxPageSetupDialogData_SetPaperSize_1_0, 2400). --define(wxPageSetupDialogData_SetPrintData, 2401). --define(wxPrintDialog_new_2_0, 2402). --define(wxPrintDialog_new_2_1, 2403). --define(wxPrintDialog_destruct, 2404). --define(wxPrintDialog_GetPrintDialogData, 2405). --define(wxPrintDialog_GetPrintDC, 2406). --define(wxPrintDialogData_new_0, 2407). --define(wxPrintDialogData_new_1_1, 2408). --define(wxPrintDialogData_new_1_0, 2409). --define(wxPrintDialogData_destruct, 2410). --define(wxPrintDialogData_EnableHelp, 2411). --define(wxPrintDialogData_EnablePageNumbers, 2412). --define(wxPrintDialogData_EnablePrintToFile, 2413). --define(wxPrintDialogData_EnableSelection, 2414). --define(wxPrintDialogData_GetAllPages, 2415). --define(wxPrintDialogData_GetCollate, 2416). --define(wxPrintDialogData_GetFromPage, 2417). --define(wxPrintDialogData_GetMaxPage, 2418). --define(wxPrintDialogData_GetMinPage, 2419). --define(wxPrintDialogData_GetNoCopies, 2420). --define(wxPrintDialogData_GetPrintData, 2421). --define(wxPrintDialogData_GetPrintToFile, 2422). --define(wxPrintDialogData_GetSelection, 2423). --define(wxPrintDialogData_GetToPage, 2424). --define(wxPrintDialogData_IsOk, 2425). --define(wxPrintDialogData_SetCollate, 2426). --define(wxPrintDialogData_SetFromPage, 2427). --define(wxPrintDialogData_SetMaxPage, 2428). --define(wxPrintDialogData_SetMinPage, 2429). --define(wxPrintDialogData_SetNoCopies, 2430). --define(wxPrintDialogData_SetPrintData, 2431). --define(wxPrintDialogData_SetPrintToFile, 2432). --define(wxPrintDialogData_SetSelection, 2433). --define(wxPrintDialogData_SetToPage, 2434). --define(wxPrintData_new_0, 2435). --define(wxPrintData_new_1, 2436). --define(wxPrintData_destruct, 2437). --define(wxPrintData_GetCollate, 2438). --define(wxPrintData_GetBin, 2439). --define(wxPrintData_GetColour, 2440). --define(wxPrintData_GetDuplex, 2441). --define(wxPrintData_GetNoCopies, 2442). --define(wxPrintData_GetOrientation, 2443). --define(wxPrintData_GetPaperId, 2444). --define(wxPrintData_GetPrinterName, 2445). --define(wxPrintData_GetQuality, 2446). --define(wxPrintData_IsOk, 2447). --define(wxPrintData_SetBin, 2448). --define(wxPrintData_SetCollate, 2449). --define(wxPrintData_SetColour, 2450). --define(wxPrintData_SetDuplex, 2451). --define(wxPrintData_SetNoCopies, 2452). --define(wxPrintData_SetOrientation, 2453). --define(wxPrintData_SetPaperId, 2454). --define(wxPrintData_SetPrinterName, 2455). --define(wxPrintData_SetQuality, 2456). --define(wxPrintPreview_new_2, 2459). --define(wxPrintPreview_new_3, 2460). --define(wxPrintPreview_destruct, 2462). --define(wxPrintPreview_GetCanvas, 2463). --define(wxPrintPreview_GetCurrentPage, 2464). --define(wxPrintPreview_GetFrame, 2465). --define(wxPrintPreview_GetMaxPage, 2466). --define(wxPrintPreview_GetMinPage, 2467). --define(wxPrintPreview_GetPrintout, 2468). --define(wxPrintPreview_GetPrintoutForPrinting, 2469). --define(wxPrintPreview_IsOk, 2470). --define(wxPrintPreview_PaintPage, 2471). --define(wxPrintPreview_Print, 2472). --define(wxPrintPreview_RenderPage, 2473). --define(wxPrintPreview_SetCanvas, 2474). --define(wxPrintPreview_SetCurrentPage, 2475). --define(wxPrintPreview_SetFrame, 2476). --define(wxPrintPreview_SetPrintout, 2477). --define(wxPrintPreview_SetZoom, 2478). --define(wxPreviewFrame_new, 2479). --define(wxPreviewFrame_destruct, 2480). --define(wxPreviewFrame_CreateControlBar, 2481). --define(wxPreviewFrame_CreateCanvas, 2482). --define(wxPreviewFrame_Initialize, 2483). --define(wxPreviewFrame_OnCloseWindow, 2484). --define(wxPreviewControlBar_new, 2485). --define(wxPreviewControlBar_destruct, 2486). --define(wxPreviewControlBar_CreateButtons, 2487). --define(wxPreviewControlBar_GetPrintPreview, 2488). --define(wxPreviewControlBar_GetZoomControl, 2489). --define(wxPreviewControlBar_SetZoomControl, 2490). --define(wxPrinter_new, 2492). --define(wxPrinter_CreateAbortWindow, 2493). --define(wxPrinter_GetAbort, 2494). --define(wxPrinter_GetLastError, 2495). --define(wxPrinter_GetPrintDialogData, 2496). --define(wxPrinter_Print, 2497). --define(wxPrinter_PrintDialog, 2498). --define(wxPrinter_ReportError, 2499). --define(wxPrinter_Setup, 2500). --define(wxPrinter_destroy, 2501). --define(wxXmlResource_new_1, 2502). --define(wxXmlResource_new_2, 2503). --define(wxXmlResource_destruct, 2504). --define(wxXmlResource_AttachUnknownControl, 2505). --define(wxXmlResource_ClearHandlers, 2506). --define(wxXmlResource_CompareVersion, 2507). --define(wxXmlResource_Get, 2508). --define(wxXmlResource_GetFlags, 2509). --define(wxXmlResource_GetVersion, 2510). --define(wxXmlResource_GetXRCID, 2511). --define(wxXmlResource_InitAllHandlers, 2512). --define(wxXmlResource_Load, 2513). --define(wxXmlResource_LoadBitmap, 2514). --define(wxXmlResource_LoadDialog_2, 2515). --define(wxXmlResource_LoadDialog_3, 2516). --define(wxXmlResource_LoadFrame_2, 2517). --define(wxXmlResource_LoadFrame_3, 2518). --define(wxXmlResource_LoadIcon, 2519). --define(wxXmlResource_LoadMenu, 2520). --define(wxXmlResource_LoadMenuBar_2, 2521). --define(wxXmlResource_LoadMenuBar_1, 2522). --define(wxXmlResource_LoadPanel_2, 2523). --define(wxXmlResource_LoadPanel_3, 2524). --define(wxXmlResource_LoadToolBar, 2525). --define(wxXmlResource_Set, 2526). --define(wxXmlResource_SetFlags, 2527). --define(wxXmlResource_Unload, 2528). --define(wxXmlResource_xrcctrl, 2529). --define(wxHtmlEasyPrinting_new, 2530). --define(wxHtmlEasyPrinting_destruct, 2531). --define(wxHtmlEasyPrinting_GetPrintData, 2532). --define(wxHtmlEasyPrinting_GetPageSetupData, 2533). --define(wxHtmlEasyPrinting_PreviewFile, 2534). --define(wxHtmlEasyPrinting_PreviewText, 2535). --define(wxHtmlEasyPrinting_PrintFile, 2536). --define(wxHtmlEasyPrinting_PrintText, 2537). --define(wxHtmlEasyPrinting_PageSetup, 2538). --define(wxHtmlEasyPrinting_SetFonts, 2539). --define(wxHtmlEasyPrinting_SetHeader, 2540). --define(wxHtmlEasyPrinting_SetFooter, 2541). --define(wxGLCanvas_new_2, 2543). --define(wxGLCanvas_new_3_1, 2544). --define(wxGLCanvas_new_3_0, 2545). --define(wxGLCanvas_GetContext, 2546). --define(wxGLCanvas_SetCurrent, 2548). --define(wxGLCanvas_SwapBuffers, 2549). --define(wxGLCanvas_destroy, 2550). --define(wxAuiManager_new, 2551). --define(wxAuiManager_destruct, 2552). --define(wxAuiManager_AddPane_2_1, 2553). --define(wxAuiManager_AddPane_3, 2554). --define(wxAuiManager_AddPane_2_0, 2555). --define(wxAuiManager_DetachPane, 2556). --define(wxAuiManager_GetAllPanes, 2557). --define(wxAuiManager_GetArtProvider, 2558). --define(wxAuiManager_GetDockSizeConstraint, 2559). --define(wxAuiManager_GetFlags, 2560). --define(wxAuiManager_GetManagedWindow, 2561). --define(wxAuiManager_GetManager, 2562). --define(wxAuiManager_GetPane_1_1, 2563). --define(wxAuiManager_GetPane_1_0, 2564). --define(wxAuiManager_HideHint, 2565). --define(wxAuiManager_InsertPane, 2566). --define(wxAuiManager_LoadPaneInfo, 2567). --define(wxAuiManager_LoadPerspective, 2568). --define(wxAuiManager_SavePaneInfo, 2569). --define(wxAuiManager_SavePerspective, 2570). --define(wxAuiManager_SetArtProvider, 2571). --define(wxAuiManager_SetDockSizeConstraint, 2572). --define(wxAuiManager_SetFlags, 2573). --define(wxAuiManager_SetManagedWindow, 2574). --define(wxAuiManager_ShowHint, 2575). --define(wxAuiManager_UnInit, 2576). --define(wxAuiManager_Update, 2577). --define(wxAuiPaneInfo_new_0, 2578). --define(wxAuiPaneInfo_new_1, 2579). --define(wxAuiPaneInfo_destruct, 2580). --define(wxAuiPaneInfo_BestSize_1, 2581). --define(wxAuiPaneInfo_BestSize_2, 2582). --define(wxAuiPaneInfo_Bottom, 2583). --define(wxAuiPaneInfo_BottomDockable, 2584). --define(wxAuiPaneInfo_Caption, 2585). --define(wxAuiPaneInfo_CaptionVisible, 2586). --define(wxAuiPaneInfo_Centre, 2587). --define(wxAuiPaneInfo_CentrePane, 2588). --define(wxAuiPaneInfo_CloseButton, 2589). --define(wxAuiPaneInfo_DefaultPane, 2590). --define(wxAuiPaneInfo_DestroyOnClose, 2591). --define(wxAuiPaneInfo_Direction, 2592). --define(wxAuiPaneInfo_Dock, 2593). --define(wxAuiPaneInfo_Dockable, 2594). --define(wxAuiPaneInfo_Fixed, 2595). --define(wxAuiPaneInfo_Float, 2596). --define(wxAuiPaneInfo_Floatable, 2597). --define(wxAuiPaneInfo_FloatingPosition_1, 2598). --define(wxAuiPaneInfo_FloatingPosition_2, 2599). --define(wxAuiPaneInfo_FloatingSize_1, 2600). --define(wxAuiPaneInfo_FloatingSize_2, 2601). --define(wxAuiPaneInfo_Gripper, 2602). --define(wxAuiPaneInfo_GripperTop, 2603). --define(wxAuiPaneInfo_HasBorder, 2604). --define(wxAuiPaneInfo_HasCaption, 2605). --define(wxAuiPaneInfo_HasCloseButton, 2606). --define(wxAuiPaneInfo_HasFlag, 2607). --define(wxAuiPaneInfo_HasGripper, 2608). --define(wxAuiPaneInfo_HasGripperTop, 2609). --define(wxAuiPaneInfo_HasMaximizeButton, 2610). --define(wxAuiPaneInfo_HasMinimizeButton, 2611). --define(wxAuiPaneInfo_HasPinButton, 2612). --define(wxAuiPaneInfo_Hide, 2613). --define(wxAuiPaneInfo_IsBottomDockable, 2614). --define(wxAuiPaneInfo_IsDocked, 2615). --define(wxAuiPaneInfo_IsFixed, 2616). --define(wxAuiPaneInfo_IsFloatable, 2617). --define(wxAuiPaneInfo_IsFloating, 2618). --define(wxAuiPaneInfo_IsLeftDockable, 2619). --define(wxAuiPaneInfo_IsMovable, 2620). --define(wxAuiPaneInfo_IsOk, 2621). --define(wxAuiPaneInfo_IsResizable, 2622). --define(wxAuiPaneInfo_IsRightDockable, 2623). --define(wxAuiPaneInfo_IsShown, 2624). --define(wxAuiPaneInfo_IsToolbar, 2625). --define(wxAuiPaneInfo_IsTopDockable, 2626). --define(wxAuiPaneInfo_Layer, 2627). --define(wxAuiPaneInfo_Left, 2628). --define(wxAuiPaneInfo_LeftDockable, 2629). --define(wxAuiPaneInfo_MaxSize_1, 2630). --define(wxAuiPaneInfo_MaxSize_2, 2631). --define(wxAuiPaneInfo_MaximizeButton, 2632). --define(wxAuiPaneInfo_MinSize_1, 2633). --define(wxAuiPaneInfo_MinSize_2, 2634). --define(wxAuiPaneInfo_MinimizeButton, 2635). --define(wxAuiPaneInfo_Movable, 2636). --define(wxAuiPaneInfo_Name, 2637). --define(wxAuiPaneInfo_PaneBorder, 2638). --define(wxAuiPaneInfo_PinButton, 2639). --define(wxAuiPaneInfo_Position, 2640). --define(wxAuiPaneInfo_Resizable, 2641). --define(wxAuiPaneInfo_Right, 2642). --define(wxAuiPaneInfo_RightDockable, 2643). --define(wxAuiPaneInfo_Row, 2644). --define(wxAuiPaneInfo_SafeSet, 2645). --define(wxAuiPaneInfo_SetFlag, 2646). --define(wxAuiPaneInfo_Show, 2647). --define(wxAuiPaneInfo_ToolbarPane, 2648). --define(wxAuiPaneInfo_Top, 2649). --define(wxAuiPaneInfo_TopDockable, 2650). --define(wxAuiPaneInfo_Window, 2651). --define(wxAuiNotebook_new_0, 2652). --define(wxAuiNotebook_new_2, 2653). --define(wxAuiNotebook_AddPage, 2654). --define(wxAuiNotebook_Create, 2655). --define(wxAuiNotebook_DeletePage, 2656). --define(wxAuiNotebook_GetArtProvider, 2657). --define(wxAuiNotebook_GetPage, 2658). --define(wxAuiNotebook_GetPageBitmap, 2659). --define(wxAuiNotebook_GetPageCount, 2660). --define(wxAuiNotebook_GetPageIndex, 2661). --define(wxAuiNotebook_GetPageText, 2662). --define(wxAuiNotebook_GetSelection, 2663). --define(wxAuiNotebook_InsertPage, 2664). --define(wxAuiNotebook_RemovePage, 2665). --define(wxAuiNotebook_SetArtProvider, 2666). --define(wxAuiNotebook_SetFont, 2667). --define(wxAuiNotebook_SetPageBitmap, 2668). --define(wxAuiNotebook_SetPageText, 2669). --define(wxAuiNotebook_SetSelection, 2670). --define(wxAuiNotebook_SetTabCtrlHeight, 2671). --define(wxAuiNotebook_SetUniformBitmapSize, 2672). --define(wxAuiNotebook_destroy, 2673). --define(wxMDIParentFrame_new_0, 2674). --define(wxMDIParentFrame_new_4, 2675). --define(wxMDIParentFrame_destruct, 2676). --define(wxMDIParentFrame_ActivateNext, 2677). --define(wxMDIParentFrame_ActivatePrevious, 2678). --define(wxMDIParentFrame_ArrangeIcons, 2679). --define(wxMDIParentFrame_Cascade, 2680). --define(wxMDIParentFrame_Create, 2681). --define(wxMDIParentFrame_GetActiveChild, 2682). --define(wxMDIParentFrame_GetClientWindow, 2683). --define(wxMDIParentFrame_Tile, 2684). --define(wxMDIChildFrame_new_0, 2685). --define(wxMDIChildFrame_new_4, 2686). --define(wxMDIChildFrame_destruct, 2687). --define(wxMDIChildFrame_Activate, 2688). --define(wxMDIChildFrame_Create, 2689). --define(wxMDIChildFrame_Maximize, 2690). --define(wxMDIChildFrame_Restore, 2691). --define(wxMDIClientWindow_new_0, 2692). --define(wxMDIClientWindow_new_2, 2693). --define(wxMDIClientWindow_destruct, 2694). --define(wxMDIClientWindow_CreateClient, 2695). --define(wxLayoutAlgorithm_new, 2696). --define(wxLayoutAlgorithm_LayoutFrame, 2697). --define(wxLayoutAlgorithm_LayoutMDIFrame, 2698). --define(wxLayoutAlgorithm_LayoutWindow, 2699). --define(wxLayoutAlgorithm_destroy, 2700). --define(wxEvent_GetId, 2701). --define(wxEvent_GetSkipped, 2702). --define(wxEvent_GetTimestamp, 2703). --define(wxEvent_IsCommandEvent, 2704). --define(wxEvent_ResumePropagation, 2705). --define(wxEvent_ShouldPropagate, 2706). --define(wxEvent_Skip, 2707). --define(wxEvent_StopPropagation, 2708). --define(wxCommandEvent_getClientData, 2709). --define(wxCommandEvent_GetExtraLong, 2710). --define(wxCommandEvent_GetInt, 2711). --define(wxCommandEvent_GetSelection, 2712). --define(wxCommandEvent_GetString, 2713). --define(wxCommandEvent_IsChecked, 2714). --define(wxCommandEvent_IsSelection, 2715). --define(wxCommandEvent_SetInt, 2716). --define(wxCommandEvent_SetString, 2717). --define(wxScrollEvent_GetOrientation, 2718). --define(wxScrollEvent_GetPosition, 2719). --define(wxScrollWinEvent_GetOrientation, 2720). --define(wxScrollWinEvent_GetPosition, 2721). --define(wxMouseEvent_AltDown, 2722). --define(wxMouseEvent_Button, 2723). --define(wxMouseEvent_ButtonDClick, 2724). --define(wxMouseEvent_ButtonDown, 2725). --define(wxMouseEvent_ButtonUp, 2726). --define(wxMouseEvent_CmdDown, 2727). --define(wxMouseEvent_ControlDown, 2728). --define(wxMouseEvent_Dragging, 2729). --define(wxMouseEvent_Entering, 2730). --define(wxMouseEvent_GetButton, 2731). --define(wxMouseEvent_GetPosition, 2734). --define(wxMouseEvent_GetLogicalPosition, 2735). --define(wxMouseEvent_GetLinesPerAction, 2736). --define(wxMouseEvent_GetWheelRotation, 2737). --define(wxMouseEvent_GetWheelDelta, 2738). --define(wxMouseEvent_GetX, 2739). --define(wxMouseEvent_GetY, 2740). --define(wxMouseEvent_IsButton, 2741). --define(wxMouseEvent_IsPageScroll, 2742). --define(wxMouseEvent_Leaving, 2743). --define(wxMouseEvent_LeftDClick, 2744). --define(wxMouseEvent_LeftDown, 2745). --define(wxMouseEvent_LeftIsDown, 2746). --define(wxMouseEvent_LeftUp, 2747). --define(wxMouseEvent_MetaDown, 2748). --define(wxMouseEvent_MiddleDClick, 2749). --define(wxMouseEvent_MiddleDown, 2750). --define(wxMouseEvent_MiddleIsDown, 2751). --define(wxMouseEvent_MiddleUp, 2752). --define(wxMouseEvent_Moving, 2753). --define(wxMouseEvent_RightDClick, 2754). --define(wxMouseEvent_RightDown, 2755). --define(wxMouseEvent_RightIsDown, 2756). --define(wxMouseEvent_RightUp, 2757). --define(wxMouseEvent_ShiftDown, 2758). --define(wxSetCursorEvent_GetCursor, 2759). --define(wxSetCursorEvent_GetX, 2760). --define(wxSetCursorEvent_GetY, 2761). --define(wxSetCursorEvent_HasCursor, 2762). --define(wxSetCursorEvent_SetCursor, 2763). --define(wxKeyEvent_AltDown, 2764). --define(wxKeyEvent_CmdDown, 2765). --define(wxKeyEvent_ControlDown, 2766). --define(wxKeyEvent_GetKeyCode, 2767). --define(wxKeyEvent_GetModifiers, 2768). --define(wxKeyEvent_GetPosition, 2771). --define(wxKeyEvent_GetRawKeyCode, 2772). --define(wxKeyEvent_GetRawKeyFlags, 2773). --define(wxKeyEvent_GetUnicodeKey, 2774). --define(wxKeyEvent_GetX, 2775). --define(wxKeyEvent_GetY, 2776). --define(wxKeyEvent_HasModifiers, 2777). --define(wxKeyEvent_MetaDown, 2778). --define(wxKeyEvent_ShiftDown, 2779). --define(wxSizeEvent_GetSize, 2780). --define(wxMoveEvent_GetPosition, 2781). --define(wxEraseEvent_GetDC, 2782). --define(wxFocusEvent_GetWindow, 2783). --define(wxChildFocusEvent_GetWindow, 2784). --define(wxMenuEvent_GetMenu, 2785). --define(wxMenuEvent_GetMenuId, 2786). --define(wxMenuEvent_IsPopup, 2787). --define(wxCloseEvent_CanVeto, 2788). --define(wxCloseEvent_GetLoggingOff, 2789). --define(wxCloseEvent_SetCanVeto, 2790). --define(wxCloseEvent_SetLoggingOff, 2791). --define(wxCloseEvent_Veto, 2792). --define(wxShowEvent_SetShow, 2793). --define(wxShowEvent_GetShow, 2794). --define(wxIconizeEvent_Iconized, 2795). --define(wxJoystickEvent_ButtonDown, 2796). --define(wxJoystickEvent_ButtonIsDown, 2797). --define(wxJoystickEvent_ButtonUp, 2798). --define(wxJoystickEvent_GetButtonChange, 2799). --define(wxJoystickEvent_GetButtonState, 2800). --define(wxJoystickEvent_GetJoystick, 2801). --define(wxJoystickEvent_GetPosition, 2802). --define(wxJoystickEvent_GetZPosition, 2803). --define(wxJoystickEvent_IsButton, 2804). --define(wxJoystickEvent_IsMove, 2805). --define(wxJoystickEvent_IsZMove, 2806). --define(wxUpdateUIEvent_CanUpdate, 2807). --define(wxUpdateUIEvent_Check, 2808). --define(wxUpdateUIEvent_Enable, 2809). --define(wxUpdateUIEvent_Show, 2810). --define(wxUpdateUIEvent_GetChecked, 2811). --define(wxUpdateUIEvent_GetEnabled, 2812). --define(wxUpdateUIEvent_GetShown, 2813). --define(wxUpdateUIEvent_GetSetChecked, 2814). --define(wxUpdateUIEvent_GetSetEnabled, 2815). --define(wxUpdateUIEvent_GetSetShown, 2816). --define(wxUpdateUIEvent_GetSetText, 2817). --define(wxUpdateUIEvent_GetText, 2818). --define(wxUpdateUIEvent_GetMode, 2819). --define(wxUpdateUIEvent_GetUpdateInterval, 2820). --define(wxUpdateUIEvent_ResetUpdateTime, 2821). --define(wxUpdateUIEvent_SetMode, 2822). --define(wxUpdateUIEvent_SetText, 2823). --define(wxUpdateUIEvent_SetUpdateInterval, 2824). --define(wxMouseCaptureChangedEvent_GetCapturedWindow, 2825). --define(wxPaletteChangedEvent_SetChangedWindow, 2826). --define(wxPaletteChangedEvent_GetChangedWindow, 2827). --define(wxQueryNewPaletteEvent_SetPaletteRealized, 2828). --define(wxQueryNewPaletteEvent_GetPaletteRealized, 2829). --define(wxNavigationKeyEvent_GetDirection, 2830). --define(wxNavigationKeyEvent_SetDirection, 2831). --define(wxNavigationKeyEvent_IsWindowChange, 2832). --define(wxNavigationKeyEvent_SetWindowChange, 2833). --define(wxNavigationKeyEvent_IsFromTab, 2834). --define(wxNavigationKeyEvent_SetFromTab, 2835). --define(wxNavigationKeyEvent_GetCurrentFocus, 2836). --define(wxNavigationKeyEvent_SetCurrentFocus, 2837). --define(wxHelpEvent_GetOrigin, 2838). --define(wxHelpEvent_GetPosition, 2839). --define(wxHelpEvent_SetOrigin, 2840). --define(wxHelpEvent_SetPosition, 2841). --define(wxContextMenuEvent_GetPosition, 2842). --define(wxContextMenuEvent_SetPosition, 2843). --define(wxIdleEvent_CanSend, 2844). --define(wxIdleEvent_GetMode, 2845). --define(wxIdleEvent_RequestMore, 2846). --define(wxIdleEvent_MoreRequested, 2847). --define(wxIdleEvent_SetMode, 2848). --define(wxGridEvent_AltDown, 2849). --define(wxGridEvent_ControlDown, 2850). --define(wxGridEvent_GetCol, 2851). --define(wxGridEvent_GetPosition, 2852). --define(wxGridEvent_GetRow, 2853). --define(wxGridEvent_MetaDown, 2854). --define(wxGridEvent_Selecting, 2855). --define(wxGridEvent_ShiftDown, 2856). --define(wxNotifyEvent_Allow, 2857). --define(wxNotifyEvent_IsAllowed, 2858). --define(wxNotifyEvent_Veto, 2859). --define(wxSashEvent_GetEdge, 2860). --define(wxSashEvent_GetDragRect, 2861). --define(wxSashEvent_GetDragStatus, 2862). --define(wxListEvent_GetCacheFrom, 2863). --define(wxListEvent_GetCacheTo, 2864). --define(wxListEvent_GetKeyCode, 2865). --define(wxListEvent_GetIndex, 2866). --define(wxListEvent_GetColumn, 2867). --define(wxListEvent_GetPoint, 2868). --define(wxListEvent_GetLabel, 2869). --define(wxListEvent_GetText, 2870). --define(wxListEvent_GetImage, 2871). --define(wxListEvent_GetData, 2872). --define(wxListEvent_GetMask, 2873). --define(wxListEvent_GetItem, 2874). --define(wxListEvent_IsEditCancelled, 2875). --define(wxDateEvent_GetDate, 2876). --define(wxCalendarEvent_GetWeekDay, 2877). --define(wxFileDirPickerEvent_GetPath, 2878). --define(wxColourPickerEvent_GetColour, 2879). --define(wxFontPickerEvent_GetFont, 2880). --define(wxStyledTextEvent_GetPosition, 2881). --define(wxStyledTextEvent_GetKey, 2882). --define(wxStyledTextEvent_GetModifiers, 2883). --define(wxStyledTextEvent_GetModificationType, 2884). --define(wxStyledTextEvent_GetText, 2885). --define(wxStyledTextEvent_GetLength, 2886). --define(wxStyledTextEvent_GetLinesAdded, 2887). --define(wxStyledTextEvent_GetLine, 2888). --define(wxStyledTextEvent_GetFoldLevelNow, 2889). --define(wxStyledTextEvent_GetFoldLevelPrev, 2890). --define(wxStyledTextEvent_GetMargin, 2891). --define(wxStyledTextEvent_GetMessage, 2892). --define(wxStyledTextEvent_GetWParam, 2893). --define(wxStyledTextEvent_GetLParam, 2894). --define(wxStyledTextEvent_GetListType, 2895). --define(wxStyledTextEvent_GetX, 2896). --define(wxStyledTextEvent_GetY, 2897). --define(wxStyledTextEvent_GetDragText, 2898). --define(wxStyledTextEvent_GetDragAllowMove, 2899). --define(wxStyledTextEvent_GetDragResult, 2900). --define(wxStyledTextEvent_GetShift, 2901). --define(wxStyledTextEvent_GetControl, 2902). --define(wxStyledTextEvent_GetAlt, 2903). --define(utils_wxGetKeyState, 2904). --define(utils_wxGetMousePosition, 2905). --define(utils_wxGetMouseState, 2906). --define(utils_wxSetDetectableAutoRepeat, 2907). --define(utils_wxBell, 2908). --define(utils_wxFindMenuItemId, 2909). --define(utils_wxGenericFindWindowAtPoint, 2910). --define(utils_wxFindWindowAtPoint, 2911). --define(utils_wxBeginBusyCursor, 2912). --define(utils_wxEndBusyCursor, 2913). --define(utils_wxIsBusy, 2914). --define(utils_wxShutdown, 2915). --define(utils_wxShell, 2916). --define(utils_wxLaunchDefaultBrowser, 2917). --define(utils_wxGetEmailAddress, 2918). --define(utils_wxGetUserId, 2919). --define(utils_wxGetHomeDir, 2920). --define(utils_wxNewId, 2921). --define(utils_wxRegisterId, 2922). --define(utils_wxGetCurrentId, 2923). --define(utils_wxGetOsDescription, 2924). --define(utils_wxIsPlatformLittleEndian, 2925). --define(utils_wxIsPlatform64Bit, 2926). --define(wxPrintout_new, 2927). --define(wxPrintout_destruct, 2928). --define(wxPrintout_GetDC, 2929). --define(wxPrintout_GetPageSizeMM, 2930). --define(wxPrintout_GetPageSizePixels, 2931). --define(wxPrintout_GetPaperRectPixels, 2932). --define(wxPrintout_GetPPIPrinter, 2933). --define(wxPrintout_GetPPIScreen, 2934). --define(wxPrintout_GetTitle, 2935). --define(wxPrintout_IsPreview, 2936). --define(wxPrintout_FitThisSizeToPaper, 2937). --define(wxPrintout_FitThisSizeToPage, 2938). --define(wxPrintout_FitThisSizeToPageMargins, 2939). --define(wxPrintout_MapScreenSizeToPaper, 2940). --define(wxPrintout_MapScreenSizeToPage, 2941). --define(wxPrintout_MapScreenSizeToPageMargins, 2942). --define(wxPrintout_MapScreenSizeToDevice, 2943). --define(wxPrintout_GetLogicalPaperRect, 2944). --define(wxPrintout_GetLogicalPageRect, 2945). --define(wxPrintout_GetLogicalPageMarginsRect, 2946). --define(wxPrintout_SetLogicalOrigin, 2947). --define(wxPrintout_OffsetLogicalOrigin, 2948). --define(wxStyledTextCtrl_new_2, 2949). --define(wxStyledTextCtrl_new_0, 2950). --define(wxStyledTextCtrl_destruct, 2951). --define(wxStyledTextCtrl_Create, 2952). --define(wxStyledTextCtrl_AddText, 2953). --define(wxStyledTextCtrl_AddStyledText, 2954). --define(wxStyledTextCtrl_InsertText, 2955). --define(wxStyledTextCtrl_ClearAll, 2956). --define(wxStyledTextCtrl_ClearDocumentStyle, 2957). --define(wxStyledTextCtrl_GetLength, 2958). --define(wxStyledTextCtrl_GetCharAt, 2959). --define(wxStyledTextCtrl_GetCurrentPos, 2960). --define(wxStyledTextCtrl_GetAnchor, 2961). --define(wxStyledTextCtrl_GetStyleAt, 2962). --define(wxStyledTextCtrl_Redo, 2963). --define(wxStyledTextCtrl_SetUndoCollection, 2964). --define(wxStyledTextCtrl_SelectAll, 2965). --define(wxStyledTextCtrl_SetSavePoint, 2966). --define(wxStyledTextCtrl_GetStyledText, 2967). --define(wxStyledTextCtrl_CanRedo, 2968). --define(wxStyledTextCtrl_MarkerLineFromHandle, 2969). --define(wxStyledTextCtrl_MarkerDeleteHandle, 2970). --define(wxStyledTextCtrl_GetUndoCollection, 2971). --define(wxStyledTextCtrl_GetViewWhiteSpace, 2972). --define(wxStyledTextCtrl_SetViewWhiteSpace, 2973). --define(wxStyledTextCtrl_PositionFromPoint, 2974). --define(wxStyledTextCtrl_PositionFromPointClose, 2975). --define(wxStyledTextCtrl_GotoLine, 2976). --define(wxStyledTextCtrl_GotoPos, 2977). --define(wxStyledTextCtrl_SetAnchor, 2978). --define(wxStyledTextCtrl_GetCurLine, 2979). --define(wxStyledTextCtrl_GetEndStyled, 2980). --define(wxStyledTextCtrl_ConvertEOLs, 2981). --define(wxStyledTextCtrl_GetEOLMode, 2982). --define(wxStyledTextCtrl_SetEOLMode, 2983). --define(wxStyledTextCtrl_StartStyling, 2984). --define(wxStyledTextCtrl_SetStyling, 2985). --define(wxStyledTextCtrl_GetBufferedDraw, 2986). --define(wxStyledTextCtrl_SetBufferedDraw, 2987). --define(wxStyledTextCtrl_SetTabWidth, 2988). --define(wxStyledTextCtrl_GetTabWidth, 2989). --define(wxStyledTextCtrl_SetCodePage, 2990). --define(wxStyledTextCtrl_MarkerDefine, 2991). --define(wxStyledTextCtrl_MarkerSetForeground, 2992). --define(wxStyledTextCtrl_MarkerSetBackground, 2993). --define(wxStyledTextCtrl_MarkerAdd, 2994). --define(wxStyledTextCtrl_MarkerDelete, 2995). --define(wxStyledTextCtrl_MarkerDeleteAll, 2996). --define(wxStyledTextCtrl_MarkerGet, 2997). --define(wxStyledTextCtrl_MarkerNext, 2998). --define(wxStyledTextCtrl_MarkerPrevious, 2999). --define(wxStyledTextCtrl_MarkerDefineBitmap, 3000). --define(wxStyledTextCtrl_MarkerAddSet, 3001). --define(wxStyledTextCtrl_MarkerSetAlpha, 3002). --define(wxStyledTextCtrl_SetMarginType, 3003). --define(wxStyledTextCtrl_GetMarginType, 3004). --define(wxStyledTextCtrl_SetMarginWidth, 3005). --define(wxStyledTextCtrl_GetMarginWidth, 3006). --define(wxStyledTextCtrl_SetMarginMask, 3007). --define(wxStyledTextCtrl_GetMarginMask, 3008). --define(wxStyledTextCtrl_SetMarginSensitive, 3009). --define(wxStyledTextCtrl_GetMarginSensitive, 3010). --define(wxStyledTextCtrl_StyleClearAll, 3011). --define(wxStyledTextCtrl_StyleSetForeground, 3012). --define(wxStyledTextCtrl_StyleSetBackground, 3013). --define(wxStyledTextCtrl_StyleSetBold, 3014). --define(wxStyledTextCtrl_StyleSetItalic, 3015). --define(wxStyledTextCtrl_StyleSetSize, 3016). --define(wxStyledTextCtrl_StyleSetFaceName, 3017). --define(wxStyledTextCtrl_StyleSetEOLFilled, 3018). --define(wxStyledTextCtrl_StyleResetDefault, 3019). --define(wxStyledTextCtrl_StyleSetUnderline, 3020). --define(wxStyledTextCtrl_StyleSetCase, 3021). --define(wxStyledTextCtrl_StyleSetHotSpot, 3022). --define(wxStyledTextCtrl_SetSelForeground, 3023). --define(wxStyledTextCtrl_SetSelBackground, 3024). --define(wxStyledTextCtrl_GetSelAlpha, 3025). --define(wxStyledTextCtrl_SetSelAlpha, 3026). --define(wxStyledTextCtrl_SetCaretForeground, 3027). --define(wxStyledTextCtrl_CmdKeyAssign, 3028). --define(wxStyledTextCtrl_CmdKeyClear, 3029). --define(wxStyledTextCtrl_CmdKeyClearAll, 3030). --define(wxStyledTextCtrl_SetStyleBytes, 3031). --define(wxStyledTextCtrl_StyleSetVisible, 3032). --define(wxStyledTextCtrl_GetCaretPeriod, 3033). --define(wxStyledTextCtrl_SetCaretPeriod, 3034). --define(wxStyledTextCtrl_SetWordChars, 3035). --define(wxStyledTextCtrl_BeginUndoAction, 3036). --define(wxStyledTextCtrl_EndUndoAction, 3037). --define(wxStyledTextCtrl_IndicatorSetStyle, 3038). --define(wxStyledTextCtrl_IndicatorGetStyle, 3039). --define(wxStyledTextCtrl_IndicatorSetForeground, 3040). --define(wxStyledTextCtrl_IndicatorGetForeground, 3041). --define(wxStyledTextCtrl_SetWhitespaceForeground, 3042). --define(wxStyledTextCtrl_SetWhitespaceBackground, 3043). --define(wxStyledTextCtrl_GetStyleBits, 3044). --define(wxStyledTextCtrl_SetLineState, 3045). --define(wxStyledTextCtrl_GetLineState, 3046). --define(wxStyledTextCtrl_GetMaxLineState, 3047). --define(wxStyledTextCtrl_GetCaretLineVisible, 3048). --define(wxStyledTextCtrl_SetCaretLineVisible, 3049). --define(wxStyledTextCtrl_GetCaretLineBackground, 3050). --define(wxStyledTextCtrl_SetCaretLineBackground, 3051). --define(wxStyledTextCtrl_AutoCompShow, 3052). --define(wxStyledTextCtrl_AutoCompCancel, 3053). --define(wxStyledTextCtrl_AutoCompActive, 3054). --define(wxStyledTextCtrl_AutoCompPosStart, 3055). --define(wxStyledTextCtrl_AutoCompComplete, 3056). --define(wxStyledTextCtrl_AutoCompStops, 3057). --define(wxStyledTextCtrl_AutoCompSetSeparator, 3058). --define(wxStyledTextCtrl_AutoCompGetSeparator, 3059). --define(wxStyledTextCtrl_AutoCompSelect, 3060). --define(wxStyledTextCtrl_AutoCompSetCancelAtStart, 3061). --define(wxStyledTextCtrl_AutoCompGetCancelAtStart, 3062). --define(wxStyledTextCtrl_AutoCompSetFillUps, 3063). --define(wxStyledTextCtrl_AutoCompSetChooseSingle, 3064). --define(wxStyledTextCtrl_AutoCompGetChooseSingle, 3065). --define(wxStyledTextCtrl_AutoCompSetIgnoreCase, 3066). --define(wxStyledTextCtrl_AutoCompGetIgnoreCase, 3067). --define(wxStyledTextCtrl_UserListShow, 3068). --define(wxStyledTextCtrl_AutoCompSetAutoHide, 3069). --define(wxStyledTextCtrl_AutoCompGetAutoHide, 3070). --define(wxStyledTextCtrl_AutoCompSetDropRestOfWord, 3071). --define(wxStyledTextCtrl_AutoCompGetDropRestOfWord, 3072). --define(wxStyledTextCtrl_RegisterImage, 3073). --define(wxStyledTextCtrl_ClearRegisteredImages, 3074). --define(wxStyledTextCtrl_AutoCompGetTypeSeparator, 3075). --define(wxStyledTextCtrl_AutoCompSetTypeSeparator, 3076). --define(wxStyledTextCtrl_AutoCompSetMaxWidth, 3077). --define(wxStyledTextCtrl_AutoCompGetMaxWidth, 3078). --define(wxStyledTextCtrl_AutoCompSetMaxHeight, 3079). --define(wxStyledTextCtrl_AutoCompGetMaxHeight, 3080). --define(wxStyledTextCtrl_SetIndent, 3081). --define(wxStyledTextCtrl_GetIndent, 3082). --define(wxStyledTextCtrl_SetUseTabs, 3083). --define(wxStyledTextCtrl_GetUseTabs, 3084). --define(wxStyledTextCtrl_SetLineIndentation, 3085). --define(wxStyledTextCtrl_GetLineIndentation, 3086). --define(wxStyledTextCtrl_GetLineIndentPosition, 3087). --define(wxStyledTextCtrl_GetColumn, 3088). --define(wxStyledTextCtrl_SetUseHorizontalScrollBar, 3089). --define(wxStyledTextCtrl_GetUseHorizontalScrollBar, 3090). --define(wxStyledTextCtrl_SetIndentationGuides, 3091). --define(wxStyledTextCtrl_GetIndentationGuides, 3092). --define(wxStyledTextCtrl_SetHighlightGuide, 3093). --define(wxStyledTextCtrl_GetHighlightGuide, 3094). --define(wxStyledTextCtrl_GetLineEndPosition, 3095). --define(wxStyledTextCtrl_GetCodePage, 3096). --define(wxStyledTextCtrl_GetCaretForeground, 3097). --define(wxStyledTextCtrl_GetReadOnly, 3098). --define(wxStyledTextCtrl_SetCurrentPos, 3099). --define(wxStyledTextCtrl_SetSelectionStart, 3100). --define(wxStyledTextCtrl_GetSelectionStart, 3101). --define(wxStyledTextCtrl_SetSelectionEnd, 3102). --define(wxStyledTextCtrl_GetSelectionEnd, 3103). --define(wxStyledTextCtrl_SetPrintMagnification, 3104). --define(wxStyledTextCtrl_GetPrintMagnification, 3105). --define(wxStyledTextCtrl_SetPrintColourMode, 3106). --define(wxStyledTextCtrl_GetPrintColourMode, 3107). --define(wxStyledTextCtrl_FindText, 3108). --define(wxStyledTextCtrl_FormatRange, 3109). --define(wxStyledTextCtrl_GetFirstVisibleLine, 3110). --define(wxStyledTextCtrl_GetLine, 3111). --define(wxStyledTextCtrl_GetLineCount, 3112). --define(wxStyledTextCtrl_SetMarginLeft, 3113). --define(wxStyledTextCtrl_GetMarginLeft, 3114). --define(wxStyledTextCtrl_SetMarginRight, 3115). --define(wxStyledTextCtrl_GetMarginRight, 3116). --define(wxStyledTextCtrl_GetModify, 3117). --define(wxStyledTextCtrl_SetSelection, 3118). --define(wxStyledTextCtrl_GetSelectedText, 3119). --define(wxStyledTextCtrl_GetTextRange, 3120). --define(wxStyledTextCtrl_HideSelection, 3121). --define(wxStyledTextCtrl_LineFromPosition, 3122). --define(wxStyledTextCtrl_PositionFromLine, 3123). --define(wxStyledTextCtrl_LineScroll, 3124). --define(wxStyledTextCtrl_EnsureCaretVisible, 3125). --define(wxStyledTextCtrl_ReplaceSelection, 3126). --define(wxStyledTextCtrl_SetReadOnly, 3127). --define(wxStyledTextCtrl_CanPaste, 3128). --define(wxStyledTextCtrl_CanUndo, 3129). --define(wxStyledTextCtrl_EmptyUndoBuffer, 3130). --define(wxStyledTextCtrl_Undo, 3131). --define(wxStyledTextCtrl_Cut, 3132). --define(wxStyledTextCtrl_Copy, 3133). --define(wxStyledTextCtrl_Paste, 3134). --define(wxStyledTextCtrl_Clear, 3135). --define(wxStyledTextCtrl_SetText, 3136). --define(wxStyledTextCtrl_GetText, 3137). --define(wxStyledTextCtrl_GetTextLength, 3138). --define(wxStyledTextCtrl_GetOvertype, 3139). --define(wxStyledTextCtrl_SetCaretWidth, 3140). --define(wxStyledTextCtrl_GetCaretWidth, 3141). --define(wxStyledTextCtrl_SetTargetStart, 3142). --define(wxStyledTextCtrl_GetTargetStart, 3143). --define(wxStyledTextCtrl_SetTargetEnd, 3144). --define(wxStyledTextCtrl_GetTargetEnd, 3145). --define(wxStyledTextCtrl_ReplaceTarget, 3146). --define(wxStyledTextCtrl_SearchInTarget, 3147). --define(wxStyledTextCtrl_SetSearchFlags, 3148). --define(wxStyledTextCtrl_GetSearchFlags, 3149). --define(wxStyledTextCtrl_CallTipShow, 3150). --define(wxStyledTextCtrl_CallTipCancel, 3151). --define(wxStyledTextCtrl_CallTipActive, 3152). --define(wxStyledTextCtrl_CallTipPosAtStart, 3153). --define(wxStyledTextCtrl_CallTipSetHighlight, 3154). --define(wxStyledTextCtrl_CallTipSetBackground, 3155). --define(wxStyledTextCtrl_CallTipSetForeground, 3156). --define(wxStyledTextCtrl_CallTipSetForegroundHighlight, 3157). --define(wxStyledTextCtrl_CallTipUseStyle, 3158). --define(wxStyledTextCtrl_VisibleFromDocLine, 3159). --define(wxStyledTextCtrl_DocLineFromVisible, 3160). --define(wxStyledTextCtrl_WrapCount, 3161). --define(wxStyledTextCtrl_SetFoldLevel, 3162). --define(wxStyledTextCtrl_GetFoldLevel, 3163). --define(wxStyledTextCtrl_GetLastChild, 3164). --define(wxStyledTextCtrl_GetFoldParent, 3165). --define(wxStyledTextCtrl_ShowLines, 3166). --define(wxStyledTextCtrl_HideLines, 3167). --define(wxStyledTextCtrl_GetLineVisible, 3168). --define(wxStyledTextCtrl_SetFoldExpanded, 3169). --define(wxStyledTextCtrl_GetFoldExpanded, 3170). --define(wxStyledTextCtrl_ToggleFold, 3171). --define(wxStyledTextCtrl_EnsureVisible, 3172). --define(wxStyledTextCtrl_SetFoldFlags, 3173). --define(wxStyledTextCtrl_EnsureVisibleEnforcePolicy, 3174). --define(wxStyledTextCtrl_SetTabIndents, 3175). --define(wxStyledTextCtrl_GetTabIndents, 3176). --define(wxStyledTextCtrl_SetBackSpaceUnIndents, 3177). --define(wxStyledTextCtrl_GetBackSpaceUnIndents, 3178). --define(wxStyledTextCtrl_SetMouseDwellTime, 3179). --define(wxStyledTextCtrl_GetMouseDwellTime, 3180). --define(wxStyledTextCtrl_WordStartPosition, 3181). --define(wxStyledTextCtrl_WordEndPosition, 3182). --define(wxStyledTextCtrl_SetWrapMode, 3183). --define(wxStyledTextCtrl_GetWrapMode, 3184). --define(wxStyledTextCtrl_SetWrapVisualFlags, 3185). --define(wxStyledTextCtrl_GetWrapVisualFlags, 3186). --define(wxStyledTextCtrl_SetWrapVisualFlagsLocation, 3187). --define(wxStyledTextCtrl_GetWrapVisualFlagsLocation, 3188). --define(wxStyledTextCtrl_SetWrapStartIndent, 3189). --define(wxStyledTextCtrl_GetWrapStartIndent, 3190). --define(wxStyledTextCtrl_SetLayoutCache, 3191). --define(wxStyledTextCtrl_GetLayoutCache, 3192). --define(wxStyledTextCtrl_SetScrollWidth, 3193). --define(wxStyledTextCtrl_GetScrollWidth, 3194). --define(wxStyledTextCtrl_TextWidth, 3195). --define(wxStyledTextCtrl_GetEndAtLastLine, 3196). --define(wxStyledTextCtrl_TextHeight, 3197). --define(wxStyledTextCtrl_SetUseVerticalScrollBar, 3198). --define(wxStyledTextCtrl_GetUseVerticalScrollBar, 3199). --define(wxStyledTextCtrl_AppendText, 3200). --define(wxStyledTextCtrl_GetTwoPhaseDraw, 3201). --define(wxStyledTextCtrl_SetTwoPhaseDraw, 3202). --define(wxStyledTextCtrl_TargetFromSelection, 3203). --define(wxStyledTextCtrl_LinesJoin, 3204). --define(wxStyledTextCtrl_LinesSplit, 3205). --define(wxStyledTextCtrl_SetFoldMarginColour, 3206). --define(wxStyledTextCtrl_SetFoldMarginHiColour, 3207). --define(wxStyledTextCtrl_LineDown, 3208). --define(wxStyledTextCtrl_LineDownExtend, 3209). --define(wxStyledTextCtrl_LineUp, 3210). --define(wxStyledTextCtrl_LineUpExtend, 3211). --define(wxStyledTextCtrl_CharLeft, 3212). --define(wxStyledTextCtrl_CharLeftExtend, 3213). --define(wxStyledTextCtrl_CharRight, 3214). --define(wxStyledTextCtrl_CharRightExtend, 3215). --define(wxStyledTextCtrl_WordLeft, 3216). --define(wxStyledTextCtrl_WordLeftExtend, 3217). --define(wxStyledTextCtrl_WordRight, 3218). --define(wxStyledTextCtrl_WordRightExtend, 3219). --define(wxStyledTextCtrl_Home, 3220). --define(wxStyledTextCtrl_HomeExtend, 3221). --define(wxStyledTextCtrl_LineEnd, 3222). --define(wxStyledTextCtrl_LineEndExtend, 3223). --define(wxStyledTextCtrl_DocumentStart, 3224). --define(wxStyledTextCtrl_DocumentStartExtend, 3225). --define(wxStyledTextCtrl_DocumentEnd, 3226). --define(wxStyledTextCtrl_DocumentEndExtend, 3227). --define(wxStyledTextCtrl_PageUp, 3228). --define(wxStyledTextCtrl_PageUpExtend, 3229). --define(wxStyledTextCtrl_PageDown, 3230). --define(wxStyledTextCtrl_PageDownExtend, 3231). --define(wxStyledTextCtrl_EditToggleOvertype, 3232). --define(wxStyledTextCtrl_Cancel, 3233). --define(wxStyledTextCtrl_DeleteBack, 3234). --define(wxStyledTextCtrl_Tab, 3235). --define(wxStyledTextCtrl_BackTab, 3236). --define(wxStyledTextCtrl_NewLine, 3237). --define(wxStyledTextCtrl_FormFeed, 3238). --define(wxStyledTextCtrl_VCHome, 3239). --define(wxStyledTextCtrl_VCHomeExtend, 3240). --define(wxStyledTextCtrl_ZoomIn, 3241). --define(wxStyledTextCtrl_ZoomOut, 3242). --define(wxStyledTextCtrl_DelWordLeft, 3243). --define(wxStyledTextCtrl_DelWordRight, 3244). --define(wxStyledTextCtrl_LineCut, 3245). --define(wxStyledTextCtrl_LineDelete, 3246). --define(wxStyledTextCtrl_LineTranspose, 3247). --define(wxStyledTextCtrl_LineDuplicate, 3248). --define(wxStyledTextCtrl_LowerCase, 3249). --define(wxStyledTextCtrl_UpperCase, 3250). --define(wxStyledTextCtrl_LineScrollDown, 3251). --define(wxStyledTextCtrl_LineScrollUp, 3252). --define(wxStyledTextCtrl_DeleteBackNotLine, 3253). --define(wxStyledTextCtrl_HomeDisplay, 3254). --define(wxStyledTextCtrl_HomeDisplayExtend, 3255). --define(wxStyledTextCtrl_LineEndDisplay, 3256). --define(wxStyledTextCtrl_LineEndDisplayExtend, 3257). --define(wxStyledTextCtrl_HomeWrapExtend, 3258). --define(wxStyledTextCtrl_LineEndWrap, 3259). --define(wxStyledTextCtrl_LineEndWrapExtend, 3260). --define(wxStyledTextCtrl_VCHomeWrap, 3261). --define(wxStyledTextCtrl_VCHomeWrapExtend, 3262). --define(wxStyledTextCtrl_LineCopy, 3263). --define(wxStyledTextCtrl_MoveCaretInsideView, 3264). --define(wxStyledTextCtrl_LineLength, 3265). --define(wxStyledTextCtrl_BraceHighlight, 3266). --define(wxStyledTextCtrl_BraceBadLight, 3267). --define(wxStyledTextCtrl_BraceMatch, 3268). --define(wxStyledTextCtrl_GetViewEOL, 3269). --define(wxStyledTextCtrl_SetViewEOL, 3270). --define(wxStyledTextCtrl_SetModEventMask, 3271). --define(wxStyledTextCtrl_GetEdgeColumn, 3272). --define(wxStyledTextCtrl_SetEdgeColumn, 3273). --define(wxStyledTextCtrl_SetEdgeMode, 3274). --define(wxStyledTextCtrl_GetEdgeMode, 3275). --define(wxStyledTextCtrl_GetEdgeColour, 3276). --define(wxStyledTextCtrl_SetEdgeColour, 3277). --define(wxStyledTextCtrl_SearchAnchor, 3278). --define(wxStyledTextCtrl_SearchNext, 3279). --define(wxStyledTextCtrl_SearchPrev, 3280). --define(wxStyledTextCtrl_LinesOnScreen, 3281). --define(wxStyledTextCtrl_UsePopUp, 3282). --define(wxStyledTextCtrl_SelectionIsRectangle, 3283). --define(wxStyledTextCtrl_SetZoom, 3284). --define(wxStyledTextCtrl_GetZoom, 3285). --define(wxStyledTextCtrl_GetModEventMask, 3286). --define(wxStyledTextCtrl_SetSTCFocus, 3287). --define(wxStyledTextCtrl_GetSTCFocus, 3288). --define(wxStyledTextCtrl_SetStatus, 3289). --define(wxStyledTextCtrl_GetStatus, 3290). --define(wxStyledTextCtrl_SetMouseDownCaptures, 3291). --define(wxStyledTextCtrl_GetMouseDownCaptures, 3292). --define(wxStyledTextCtrl_SetSTCCursor, 3293). --define(wxStyledTextCtrl_GetSTCCursor, 3294). --define(wxStyledTextCtrl_SetControlCharSymbol, 3295). --define(wxStyledTextCtrl_GetControlCharSymbol, 3296). --define(wxStyledTextCtrl_WordPartLeft, 3297). --define(wxStyledTextCtrl_WordPartLeftExtend, 3298). --define(wxStyledTextCtrl_WordPartRight, 3299). --define(wxStyledTextCtrl_WordPartRightExtend, 3300). --define(wxStyledTextCtrl_SetVisiblePolicy, 3301). --define(wxStyledTextCtrl_DelLineLeft, 3302). --define(wxStyledTextCtrl_DelLineRight, 3303). --define(wxStyledTextCtrl_GetXOffset, 3304). --define(wxStyledTextCtrl_ChooseCaretX, 3305). --define(wxStyledTextCtrl_SetXCaretPolicy, 3306). --define(wxStyledTextCtrl_SetYCaretPolicy, 3307). --define(wxStyledTextCtrl_GetPrintWrapMode, 3308). --define(wxStyledTextCtrl_SetHotspotActiveForeground, 3309). --define(wxStyledTextCtrl_SetHotspotActiveBackground, 3310). --define(wxStyledTextCtrl_SetHotspotActiveUnderline, 3311). --define(wxStyledTextCtrl_SetHotspotSingleLine, 3312). --define(wxStyledTextCtrl_ParaDownExtend, 3313). --define(wxStyledTextCtrl_ParaUp, 3314). --define(wxStyledTextCtrl_ParaUpExtend, 3315). --define(wxStyledTextCtrl_PositionBefore, 3316). --define(wxStyledTextCtrl_PositionAfter, 3317). --define(wxStyledTextCtrl_CopyRange, 3318). --define(wxStyledTextCtrl_CopyText, 3319). --define(wxStyledTextCtrl_SetSelectionMode, 3320). --define(wxStyledTextCtrl_GetSelectionMode, 3321). --define(wxStyledTextCtrl_LineDownRectExtend, 3322). --define(wxStyledTextCtrl_LineUpRectExtend, 3323). --define(wxStyledTextCtrl_CharLeftRectExtend, 3324). --define(wxStyledTextCtrl_CharRightRectExtend, 3325). --define(wxStyledTextCtrl_HomeRectExtend, 3326). --define(wxStyledTextCtrl_VCHomeRectExtend, 3327). --define(wxStyledTextCtrl_LineEndRectExtend, 3328). --define(wxStyledTextCtrl_PageUpRectExtend, 3329). --define(wxStyledTextCtrl_PageDownRectExtend, 3330). --define(wxStyledTextCtrl_StutteredPageUp, 3331). --define(wxStyledTextCtrl_StutteredPageUpExtend, 3332). --define(wxStyledTextCtrl_StutteredPageDown, 3333). --define(wxStyledTextCtrl_StutteredPageDownExtend, 3334). --define(wxStyledTextCtrl_WordLeftEnd, 3335). --define(wxStyledTextCtrl_WordLeftEndExtend, 3336). --define(wxStyledTextCtrl_WordRightEnd, 3337). --define(wxStyledTextCtrl_WordRightEndExtend, 3338). --define(wxStyledTextCtrl_SetWhitespaceChars, 3339). --define(wxStyledTextCtrl_SetCharsDefault, 3340). --define(wxStyledTextCtrl_AutoCompGetCurrent, 3341). --define(wxStyledTextCtrl_Allocate, 3342). --define(wxStyledTextCtrl_FindColumn, 3343). --define(wxStyledTextCtrl_GetCaretSticky, 3344). --define(wxStyledTextCtrl_SetCaretSticky, 3345). --define(wxStyledTextCtrl_ToggleCaretSticky, 3346). --define(wxStyledTextCtrl_SetPasteConvertEndings, 3347). --define(wxStyledTextCtrl_GetPasteConvertEndings, 3348). --define(wxStyledTextCtrl_SelectionDuplicate, 3349). --define(wxStyledTextCtrl_SetCaretLineBackAlpha, 3350). --define(wxStyledTextCtrl_GetCaretLineBackAlpha, 3351). --define(wxStyledTextCtrl_StartRecord, 3352). --define(wxStyledTextCtrl_StopRecord, 3353). --define(wxStyledTextCtrl_SetLexer, 3354). --define(wxStyledTextCtrl_GetLexer, 3355). --define(wxStyledTextCtrl_Colourise, 3356). --define(wxStyledTextCtrl_SetProperty, 3357). --define(wxStyledTextCtrl_SetKeyWords, 3358). --define(wxStyledTextCtrl_SetLexerLanguage, 3359). --define(wxStyledTextCtrl_GetProperty, 3360). --define(wxStyledTextCtrl_GetStyleBitsNeeded, 3361). --define(wxStyledTextCtrl_GetCurrentLine, 3362). --define(wxStyledTextCtrl_StyleSetSpec, 3363). --define(wxStyledTextCtrl_StyleSetFont, 3364). --define(wxStyledTextCtrl_StyleSetFontAttr, 3365). --define(wxStyledTextCtrl_StyleSetCharacterSet, 3366). --define(wxStyledTextCtrl_StyleSetFontEncoding, 3367). --define(wxStyledTextCtrl_CmdKeyExecute, 3368). --define(wxStyledTextCtrl_SetMargins, 3369). --define(wxStyledTextCtrl_GetSelection, 3370). --define(wxStyledTextCtrl_PointFromPosition, 3371). --define(wxStyledTextCtrl_ScrollToLine, 3372). --define(wxStyledTextCtrl_ScrollToColumn, 3373). --define(wxStyledTextCtrl_SetVScrollBar, 3374). --define(wxStyledTextCtrl_SetHScrollBar, 3375). --define(wxStyledTextCtrl_GetLastKeydownProcessed, 3376). --define(wxStyledTextCtrl_SetLastKeydownProcessed, 3377). --define(wxStyledTextCtrl_SaveFile, 3378). --define(wxStyledTextCtrl_LoadFile, 3379). --define(wxStyledTextCtrl_DoDragOver, 3380). --define(wxStyledTextCtrl_DoDropText, 3381). --define(wxStyledTextCtrl_GetUseAntiAliasing, 3382). --define(wxStyledTextCtrl_AddTextRaw, 3383). --define(wxStyledTextCtrl_InsertTextRaw, 3384). --define(wxStyledTextCtrl_GetCurLineRaw, 3385). --define(wxStyledTextCtrl_GetLineRaw, 3386). --define(wxStyledTextCtrl_GetSelectedTextRaw, 3387). --define(wxStyledTextCtrl_GetTextRangeRaw, 3388). --define(wxStyledTextCtrl_SetTextRaw, 3389). --define(wxStyledTextCtrl_GetTextRaw, 3390). --define(wxStyledTextCtrl_AppendTextRaw, 3391). --define(wxArtProvider_GetBitmap, 3392). --define(wxArtProvider_GetIcon, 3393). --define(wxTreeEvent_GetKeyCode, 3394). --define(wxTreeEvent_GetItem, 3395). --define(wxTreeEvent_GetKeyEvent, 3396). --define(wxTreeEvent_GetLabel, 3397). --define(wxTreeEvent_GetOldItem, 3398). --define(wxTreeEvent_GetPoint, 3399). --define(wxTreeEvent_IsEditCancelled, 3400). --define(wxTreeEvent_SetToolTip, 3401). --define(wxNotebookEvent_GetOldSelection, 3402). --define(wxNotebookEvent_GetSelection, 3403). --define(wxNotebookEvent_SetOldSelection, 3404). --define(wxNotebookEvent_SetSelection, 3405). --define(wxFileDataObject_new, 3406). --define(wxFileDataObject_AddFile, 3407). --define(wxFileDataObject_GetFilenames, 3408). --define(wxFileDataObject_destroy, 3409). --define(wxTextDataObject_new, 3410). --define(wxTextDataObject_GetTextLength, 3411). --define(wxTextDataObject_GetText, 3412). --define(wxTextDataObject_SetText, 3413). --define(wxTextDataObject_destroy, 3414). --define(wxBitmapDataObject_new_1_1, 3415). --define(wxBitmapDataObject_new_1_0, 3416). --define(wxBitmapDataObject_GetBitmap, 3417). --define(wxBitmapDataObject_SetBitmap, 3418). --define(wxBitmapDataObject_destroy, 3419). --define(wxClipboard_new, 3421). --define(wxClipboard_destruct, 3422). --define(wxClipboard_AddData, 3423). --define(wxClipboard_Clear, 3424). --define(wxClipboard_Close, 3425). --define(wxClipboard_Flush, 3426). --define(wxClipboard_GetData, 3427). --define(wxClipboard_IsOpened, 3428). --define(wxClipboard_Open, 3429). --define(wxClipboard_SetData, 3430). --define(wxClipboard_UsePrimarySelection, 3432). --define(wxClipboard_IsSupported, 3433). --define(wxClipboard_Get, 3434). --define(wxSpinEvent_GetPosition, 3435). --define(wxSpinEvent_SetPosition, 3436). --define(wxSplitterWindow_new_0, 3437). --define(wxSplitterWindow_new_2, 3438). --define(wxSplitterWindow_destruct, 3439). --define(wxSplitterWindow_Create, 3440). --define(wxSplitterWindow_GetMinimumPaneSize, 3441). --define(wxSplitterWindow_GetSashGravity, 3442). --define(wxSplitterWindow_GetSashPosition, 3443). --define(wxSplitterWindow_GetSplitMode, 3444). --define(wxSplitterWindow_GetWindow1, 3445). --define(wxSplitterWindow_GetWindow2, 3446). --define(wxSplitterWindow_Initialize, 3447). --define(wxSplitterWindow_IsSplit, 3448). --define(wxSplitterWindow_ReplaceWindow, 3449). --define(wxSplitterWindow_SetSashGravity, 3450). --define(wxSplitterWindow_SetSashPosition, 3451). --define(wxSplitterWindow_SetSashSize, 3452). --define(wxSplitterWindow_SetMinimumPaneSize, 3453). --define(wxSplitterWindow_SetSplitMode, 3454). --define(wxSplitterWindow_SplitHorizontally, 3455). --define(wxSplitterWindow_SplitVertically, 3456). --define(wxSplitterWindow_Unsplit, 3457). --define(wxSplitterWindow_UpdateSize, 3458). --define(wxSplitterEvent_GetSashPosition, 3459). --define(wxSplitterEvent_GetX, 3460). --define(wxSplitterEvent_GetY, 3461). --define(wxSplitterEvent_GetWindowBeingRemoved, 3462). --define(wxSplitterEvent_SetSashPosition, 3463). --define(wxHtmlWindow_new_0, 3464). --define(wxHtmlWindow_new_2, 3465). --define(wxHtmlWindow_AppendToPage, 3466). --define(wxHtmlWindow_GetOpenedAnchor, 3467). --define(wxHtmlWindow_GetOpenedPage, 3468). --define(wxHtmlWindow_GetOpenedPageTitle, 3469). --define(wxHtmlWindow_GetRelatedFrame, 3470). --define(wxHtmlWindow_HistoryBack, 3471). --define(wxHtmlWindow_HistoryCanBack, 3472). --define(wxHtmlWindow_HistoryCanForward, 3473). --define(wxHtmlWindow_HistoryClear, 3474). --define(wxHtmlWindow_HistoryForward, 3475). --define(wxHtmlWindow_LoadFile, 3476). --define(wxHtmlWindow_LoadPage, 3477). --define(wxHtmlWindow_SelectAll, 3478). --define(wxHtmlWindow_SelectionToText, 3479). --define(wxHtmlWindow_SelectLine, 3480). --define(wxHtmlWindow_SelectWord, 3481). --define(wxHtmlWindow_SetBorders, 3482). --define(wxHtmlWindow_SetFonts, 3483). --define(wxHtmlWindow_SetPage, 3484). --define(wxHtmlWindow_SetRelatedFrame, 3485). --define(wxHtmlWindow_SetRelatedStatusBar, 3486). --define(wxHtmlWindow_ToText, 3487). --define(wxHtmlWindow_destroy, 3488). --define(wxHtmlLinkEvent_GetLinkInfo, 3489). --define(wxSystemSettings_GetColour, 3490). --define(wxSystemSettings_GetFont, 3491). --define(wxSystemSettings_GetMetric, 3492). --define(wxSystemSettings_GetScreenType, 3493). --define(wxSystemOptions_GetOption, 3494). --define(wxSystemOptions_GetOptionInt, 3495). --define(wxSystemOptions_HasOption, 3496). --define(wxSystemOptions_IsFalse, 3497). --define(wxSystemOptions_SetOption_2_1, 3498). --define(wxSystemOptions_SetOption_2_0, 3499). --define(wxAuiNotebookEvent_SetSelection, 3500). --define(wxAuiNotebookEvent_GetSelection, 3501). --define(wxAuiNotebookEvent_SetOldSelection, 3502). --define(wxAuiNotebookEvent_GetOldSelection, 3503). --define(wxAuiNotebookEvent_SetDragSource, 3504). --define(wxAuiNotebookEvent_GetDragSource, 3505). --define(wxAuiManagerEvent_SetManager, 3506). --define(wxAuiManagerEvent_GetManager, 3507). --define(wxAuiManagerEvent_SetPane, 3508). --define(wxAuiManagerEvent_GetPane, 3509). --define(wxAuiManagerEvent_SetButton, 3510). --define(wxAuiManagerEvent_GetButton, 3511). --define(wxAuiManagerEvent_SetDC, 3512). --define(wxAuiManagerEvent_GetDC, 3513). --define(wxAuiManagerEvent_Veto, 3514). --define(wxAuiManagerEvent_GetVeto, 3515). --define(wxAuiManagerEvent_SetCanVeto, 3516). --define(wxAuiManagerEvent_CanVeto, 3517). --define(wxLogNull_new, 3518). --define(wxLogNull_destroy, 3519). --define(wxTaskBarIcon_new, 3520). --define(wxTaskBarIcon_destruct, 3521). --define(wxTaskBarIcon_PopupMenu, 3522). --define(wxTaskBarIcon_RemoveIcon, 3523). --define(wxTaskBarIcon_SetIcon, 3524). +-define(wxTextCtrl_ChangeValue, 1827). +-define(wxTextCtrl_EmulateKeyPress, 1828). +-define(wxTextCtrl_GetDefaultStyle, 1829). +-define(wxTextCtrl_GetInsertionPoint, 1830). +-define(wxTextCtrl_GetLastPosition, 1831). +-define(wxTextCtrl_GetLineLength, 1832). +-define(wxTextCtrl_GetLineText, 1833). +-define(wxTextCtrl_GetNumberOfLines, 1834). +-define(wxTextCtrl_GetRange, 1835). +-define(wxTextCtrl_GetSelection, 1836). +-define(wxTextCtrl_GetStringSelection, 1837). +-define(wxTextCtrl_GetStyle, 1838). +-define(wxTextCtrl_GetValue, 1839). +-define(wxTextCtrl_IsEditable, 1840). +-define(wxTextCtrl_IsModified, 1841). +-define(wxTextCtrl_IsMultiLine, 1842). +-define(wxTextCtrl_IsSingleLine, 1843). +-define(wxTextCtrl_LoadFile, 1844). +-define(wxTextCtrl_MarkDirty, 1845). +-define(wxTextCtrl_Paste, 1846). +-define(wxTextCtrl_PositionToXY, 1847). +-define(wxTextCtrl_Redo, 1848). +-define(wxTextCtrl_Remove, 1849). +-define(wxTextCtrl_Replace, 1850). +-define(wxTextCtrl_SaveFile, 1851). +-define(wxTextCtrl_SetDefaultStyle, 1852). +-define(wxTextCtrl_SetEditable, 1853). +-define(wxTextCtrl_SetInsertionPoint, 1854). +-define(wxTextCtrl_SetInsertionPointEnd, 1855). +-define(wxTextCtrl_SetMaxLength, 1857). +-define(wxTextCtrl_SetSelection, 1858). +-define(wxTextCtrl_SetStyle, 1859). +-define(wxTextCtrl_SetValue, 1860). +-define(wxTextCtrl_ShowPosition, 1861). +-define(wxTextCtrl_Undo, 1862). +-define(wxTextCtrl_WriteText, 1863). +-define(wxTextCtrl_XYToPosition, 1864). +-define(wxNotebook_new_0, 1867). +-define(wxNotebook_new_3, 1868). +-define(wxNotebook_destruct, 1869). +-define(wxNotebook_AddPage, 1870). +-define(wxNotebook_AdvanceSelection, 1871). +-define(wxNotebook_AssignImageList, 1872). +-define(wxNotebook_Create, 1873). +-define(wxNotebook_DeleteAllPages, 1874). +-define(wxNotebook_DeletePage, 1875). +-define(wxNotebook_RemovePage, 1876). +-define(wxNotebook_GetCurrentPage, 1877). +-define(wxNotebook_GetImageList, 1878). +-define(wxNotebook_GetPage, 1880). +-define(wxNotebook_GetPageCount, 1881). +-define(wxNotebook_GetPageImage, 1882). +-define(wxNotebook_GetPageText, 1883). +-define(wxNotebook_GetRowCount, 1884). +-define(wxNotebook_GetSelection, 1885). +-define(wxNotebook_GetThemeBackgroundColour, 1886). +-define(wxNotebook_HitTest, 1888). +-define(wxNotebook_InsertPage, 1890). +-define(wxNotebook_SetImageList, 1891). +-define(wxNotebook_SetPadding, 1892). +-define(wxNotebook_SetPageSize, 1893). +-define(wxNotebook_SetPageImage, 1894). +-define(wxNotebook_SetPageText, 1895). +-define(wxNotebook_SetSelection, 1896). +-define(wxNotebook_ChangeSelection, 1897). +-define(wxChoicebook_new_0, 1898). +-define(wxChoicebook_new_3, 1899). +-define(wxChoicebook_AddPage, 1900). +-define(wxChoicebook_AdvanceSelection, 1901). +-define(wxChoicebook_AssignImageList, 1902). +-define(wxChoicebook_Create, 1903). +-define(wxChoicebook_DeleteAllPages, 1904). +-define(wxChoicebook_DeletePage, 1905). +-define(wxChoicebook_RemovePage, 1906). +-define(wxChoicebook_GetCurrentPage, 1907). +-define(wxChoicebook_GetImageList, 1908). +-define(wxChoicebook_GetPage, 1910). +-define(wxChoicebook_GetPageCount, 1911). +-define(wxChoicebook_GetPageImage, 1912). +-define(wxChoicebook_GetPageText, 1913). +-define(wxChoicebook_GetSelection, 1914). +-define(wxChoicebook_HitTest, 1915). +-define(wxChoicebook_InsertPage, 1916). +-define(wxChoicebook_SetImageList, 1917). +-define(wxChoicebook_SetPageSize, 1918). +-define(wxChoicebook_SetPageImage, 1919). +-define(wxChoicebook_SetPageText, 1920). +-define(wxChoicebook_SetSelection, 1921). +-define(wxChoicebook_ChangeSelection, 1922). +-define(wxChoicebook_destroy, 1923). +-define(wxToolbook_new_0, 1924). +-define(wxToolbook_new_3, 1925). +-define(wxToolbook_AddPage, 1926). +-define(wxToolbook_AdvanceSelection, 1927). +-define(wxToolbook_AssignImageList, 1928). +-define(wxToolbook_Create, 1929). +-define(wxToolbook_DeleteAllPages, 1930). +-define(wxToolbook_DeletePage, 1931). +-define(wxToolbook_RemovePage, 1932). +-define(wxToolbook_GetCurrentPage, 1933). +-define(wxToolbook_GetImageList, 1934). +-define(wxToolbook_GetPage, 1936). +-define(wxToolbook_GetPageCount, 1937). +-define(wxToolbook_GetPageImage, 1938). +-define(wxToolbook_GetPageText, 1939). +-define(wxToolbook_GetSelection, 1940). +-define(wxToolbook_HitTest, 1942). +-define(wxToolbook_InsertPage, 1943). +-define(wxToolbook_SetImageList, 1944). +-define(wxToolbook_SetPageSize, 1945). +-define(wxToolbook_SetPageImage, 1946). +-define(wxToolbook_SetPageText, 1947). +-define(wxToolbook_SetSelection, 1948). +-define(wxToolbook_ChangeSelection, 1949). +-define(wxToolbook_destroy, 1950). +-define(wxListbook_new_0, 1951). +-define(wxListbook_new_3, 1952). +-define(wxListbook_AddPage, 1953). +-define(wxListbook_AdvanceSelection, 1954). +-define(wxListbook_AssignImageList, 1955). +-define(wxListbook_Create, 1956). +-define(wxListbook_DeleteAllPages, 1957). +-define(wxListbook_DeletePage, 1958). +-define(wxListbook_RemovePage, 1959). +-define(wxListbook_GetCurrentPage, 1960). +-define(wxListbook_GetImageList, 1961). +-define(wxListbook_GetPage, 1963). +-define(wxListbook_GetPageCount, 1964). +-define(wxListbook_GetPageImage, 1965). +-define(wxListbook_GetPageText, 1966). +-define(wxListbook_GetSelection, 1967). +-define(wxListbook_HitTest, 1969). +-define(wxListbook_InsertPage, 1970). +-define(wxListbook_SetImageList, 1971). +-define(wxListbook_SetPageSize, 1972). +-define(wxListbook_SetPageImage, 1973). +-define(wxListbook_SetPageText, 1974). +-define(wxListbook_SetSelection, 1975). +-define(wxListbook_ChangeSelection, 1976). +-define(wxListbook_destroy, 1977). +-define(wxTreebook_new_0, 1978). +-define(wxTreebook_new_3, 1979). +-define(wxTreebook_AddPage, 1980). +-define(wxTreebook_AdvanceSelection, 1981). +-define(wxTreebook_AssignImageList, 1982). +-define(wxTreebook_Create, 1983). +-define(wxTreebook_DeleteAllPages, 1984). +-define(wxTreebook_DeletePage, 1985). +-define(wxTreebook_RemovePage, 1986). +-define(wxTreebook_GetCurrentPage, 1987). +-define(wxTreebook_GetImageList, 1988). +-define(wxTreebook_GetPage, 1990). +-define(wxTreebook_GetPageCount, 1991). +-define(wxTreebook_GetPageImage, 1992). +-define(wxTreebook_GetPageText, 1993). +-define(wxTreebook_GetSelection, 1994). +-define(wxTreebook_ExpandNode, 1995). +-define(wxTreebook_IsNodeExpanded, 1996). +-define(wxTreebook_HitTest, 1998). +-define(wxTreebook_InsertPage, 1999). +-define(wxTreebook_InsertSubPage, 2000). +-define(wxTreebook_SetImageList, 2001). +-define(wxTreebook_SetPageSize, 2002). +-define(wxTreebook_SetPageImage, 2003). +-define(wxTreebook_SetPageText, 2004). +-define(wxTreebook_SetSelection, 2005). +-define(wxTreebook_ChangeSelection, 2006). +-define(wxTreebook_destroy, 2007). +-define(wxTreeCtrl_new_2, 2010). +-define(wxTreeCtrl_new_0, 2011). +-define(wxTreeCtrl_destruct, 2013). +-define(wxTreeCtrl_AddRoot, 2014). +-define(wxTreeCtrl_AppendItem, 2015). +-define(wxTreeCtrl_AssignImageList, 2016). +-define(wxTreeCtrl_AssignStateImageList, 2017). +-define(wxTreeCtrl_Collapse, 2018). +-define(wxTreeCtrl_CollapseAndReset, 2019). +-define(wxTreeCtrl_Create, 2020). +-define(wxTreeCtrl_Delete, 2021). +-define(wxTreeCtrl_DeleteAllItems, 2022). +-define(wxTreeCtrl_DeleteChildren, 2023). +-define(wxTreeCtrl_EditLabel, 2024). +-define(wxTreeCtrl_EnsureVisible, 2025). +-define(wxTreeCtrl_Expand, 2026). +-define(wxTreeCtrl_GetBoundingRect, 2027). +-define(wxTreeCtrl_GetChildrenCount, 2029). +-define(wxTreeCtrl_GetCount, 2030). +-define(wxTreeCtrl_GetEditControl, 2031). +-define(wxTreeCtrl_GetFirstChild, 2032). +-define(wxTreeCtrl_GetNextChild, 2033). +-define(wxTreeCtrl_GetFirstVisibleItem, 2034). +-define(wxTreeCtrl_GetImageList, 2035). +-define(wxTreeCtrl_GetIndent, 2036). +-define(wxTreeCtrl_GetItemBackgroundColour, 2037). +-define(wxTreeCtrl_GetItemData, 2038). +-define(wxTreeCtrl_GetItemFont, 2039). +-define(wxTreeCtrl_GetItemImage_1, 2040). +-define(wxTreeCtrl_GetItemImage_2, 2041). +-define(wxTreeCtrl_GetItemText, 2042). +-define(wxTreeCtrl_GetItemTextColour, 2043). +-define(wxTreeCtrl_GetLastChild, 2044). +-define(wxTreeCtrl_GetNextSibling, 2045). +-define(wxTreeCtrl_GetNextVisible, 2046). +-define(wxTreeCtrl_GetItemParent, 2047). +-define(wxTreeCtrl_GetPrevSibling, 2048). +-define(wxTreeCtrl_GetPrevVisible, 2049). +-define(wxTreeCtrl_GetRootItem, 2050). +-define(wxTreeCtrl_GetSelection, 2051). +-define(wxTreeCtrl_GetSelections, 2052). +-define(wxTreeCtrl_GetStateImageList, 2053). +-define(wxTreeCtrl_HitTest, 2054). +-define(wxTreeCtrl_InsertItem, 2056). +-define(wxTreeCtrl_IsBold, 2057). +-define(wxTreeCtrl_IsExpanded, 2058). +-define(wxTreeCtrl_IsSelected, 2059). +-define(wxTreeCtrl_IsVisible, 2060). +-define(wxTreeCtrl_ItemHasChildren, 2061). +-define(wxTreeCtrl_IsTreeItemIdOk, 2062). +-define(wxTreeCtrl_PrependItem, 2063). +-define(wxTreeCtrl_ScrollTo, 2064). +-define(wxTreeCtrl_SelectItem_1, 2065). +-define(wxTreeCtrl_SelectItem_2, 2066). +-define(wxTreeCtrl_SetIndent, 2067). +-define(wxTreeCtrl_SetImageList, 2068). +-define(wxTreeCtrl_SetItemBackgroundColour, 2069). +-define(wxTreeCtrl_SetItemBold, 2070). +-define(wxTreeCtrl_SetItemData, 2071). +-define(wxTreeCtrl_SetItemDropHighlight, 2072). +-define(wxTreeCtrl_SetItemFont, 2073). +-define(wxTreeCtrl_SetItemHasChildren, 2074). +-define(wxTreeCtrl_SetItemImage_2, 2075). +-define(wxTreeCtrl_SetItemImage_3, 2076). +-define(wxTreeCtrl_SetItemText, 2077). +-define(wxTreeCtrl_SetItemTextColour, 2078). +-define(wxTreeCtrl_SetStateImageList, 2079). +-define(wxTreeCtrl_SetWindowStyle, 2080). +-define(wxTreeCtrl_SortChildren, 2081). +-define(wxTreeCtrl_Toggle, 2082). +-define(wxTreeCtrl_ToggleItemSelection, 2083). +-define(wxTreeCtrl_Unselect, 2084). +-define(wxTreeCtrl_UnselectAll, 2085). +-define(wxTreeCtrl_UnselectItem, 2086). +-define(wxScrollBar_new_0, 2087). +-define(wxScrollBar_new_3, 2088). +-define(wxScrollBar_destruct, 2089). +-define(wxScrollBar_Create, 2090). +-define(wxScrollBar_GetRange, 2091). +-define(wxScrollBar_GetPageSize, 2092). +-define(wxScrollBar_GetThumbPosition, 2093). +-define(wxScrollBar_GetThumbSize, 2094). +-define(wxScrollBar_SetThumbPosition, 2095). +-define(wxScrollBar_SetScrollbar, 2096). +-define(wxSpinButton_new_2, 2098). +-define(wxSpinButton_new_0, 2099). +-define(wxSpinButton_Create, 2100). +-define(wxSpinButton_GetMax, 2101). +-define(wxSpinButton_GetMin, 2102). +-define(wxSpinButton_GetValue, 2103). +-define(wxSpinButton_SetRange, 2104). +-define(wxSpinButton_SetValue, 2105). +-define(wxSpinButton_destroy, 2106). +-define(wxSpinCtrl_new_0, 2107). +-define(wxSpinCtrl_new_2, 2108). +-define(wxSpinCtrl_Create, 2110). +-define(wxSpinCtrl_SetValue_1_1, 2113). +-define(wxSpinCtrl_SetValue_1_0, 2114). +-define(wxSpinCtrl_GetValue, 2116). +-define(wxSpinCtrl_SetRange, 2118). +-define(wxSpinCtrl_SetSelection, 2119). +-define(wxSpinCtrl_GetMin, 2121). +-define(wxSpinCtrl_GetMax, 2123). +-define(wxSpinCtrl_destroy, 2124). +-define(wxStaticText_new_0, 2125). +-define(wxStaticText_new_4, 2126). +-define(wxStaticText_Create, 2127). +-define(wxStaticText_GetLabel, 2128). +-define(wxStaticText_SetLabel, 2129). +-define(wxStaticText_Wrap, 2130). +-define(wxStaticText_destroy, 2131). +-define(wxStaticBitmap_new_0, 2132). +-define(wxStaticBitmap_new_4, 2133). +-define(wxStaticBitmap_Create, 2134). +-define(wxStaticBitmap_GetBitmap, 2135). +-define(wxStaticBitmap_SetBitmap, 2136). +-define(wxStaticBitmap_destroy, 2137). +-define(wxRadioBox_new, 2138). +-define(wxRadioBox_destruct, 2140). +-define(wxRadioBox_Create, 2141). +-define(wxRadioBox_Enable_2, 2142). +-define(wxRadioBox_Enable_1, 2143). +-define(wxRadioBox_GetSelection, 2144). +-define(wxRadioBox_GetString, 2145). +-define(wxRadioBox_SetSelection, 2146). +-define(wxRadioBox_Show_2, 2147). +-define(wxRadioBox_Show_1, 2148). +-define(wxRadioBox_GetColumnCount, 2149). +-define(wxRadioBox_GetItemHelpText, 2150). +-define(wxRadioBox_GetItemToolTip, 2151). +-define(wxRadioBox_GetItemFromPoint, 2153). +-define(wxRadioBox_GetRowCount, 2154). +-define(wxRadioBox_IsItemEnabled, 2155). +-define(wxRadioBox_IsItemShown, 2156). +-define(wxRadioBox_SetItemHelpText, 2157). +-define(wxRadioBox_SetItemToolTip, 2158). +-define(wxRadioButton_new_0, 2159). +-define(wxRadioButton_new_4, 2160). +-define(wxRadioButton_Create, 2161). +-define(wxRadioButton_GetValue, 2162). +-define(wxRadioButton_SetValue, 2163). +-define(wxRadioButton_destroy, 2164). +-define(wxSlider_new_6, 2166). +-define(wxSlider_new_0, 2167). +-define(wxSlider_Create, 2168). +-define(wxSlider_GetLineSize, 2169). +-define(wxSlider_GetMax, 2170). +-define(wxSlider_GetMin, 2171). +-define(wxSlider_GetPageSize, 2172). +-define(wxSlider_GetThumbLength, 2173). +-define(wxSlider_GetValue, 2174). +-define(wxSlider_SetLineSize, 2175). +-define(wxSlider_SetPageSize, 2176). +-define(wxSlider_SetRange, 2177). +-define(wxSlider_SetThumbLength, 2178). +-define(wxSlider_SetValue, 2179). +-define(wxSlider_destroy, 2180). +-define(wxDialog_new_4, 2182). +-define(wxDialog_new_0, 2183). +-define(wxDialog_destruct, 2185). +-define(wxDialog_Create, 2186). +-define(wxDialog_CreateButtonSizer, 2187). +-define(wxDialog_CreateStdDialogButtonSizer, 2188). +-define(wxDialog_EndModal, 2189). +-define(wxDialog_GetAffirmativeId, 2190). +-define(wxDialog_GetReturnCode, 2191). +-define(wxDialog_IsModal, 2192). +-define(wxDialog_SetAffirmativeId, 2193). +-define(wxDialog_SetReturnCode, 2194). +-define(wxDialog_Show, 2195). +-define(wxDialog_ShowModal, 2196). +-define(wxColourDialog_new_0, 2197). +-define(wxColourDialog_new_2, 2198). +-define(wxColourDialog_destruct, 2199). +-define(wxColourDialog_Create, 2200). +-define(wxColourDialog_GetColourData, 2201). +-define(wxColourData_new_0, 2202). +-define(wxColourData_new_1, 2203). +-define(wxColourData_destruct, 2204). +-define(wxColourData_GetChooseFull, 2205). +-define(wxColourData_GetColour, 2206). +-define(wxColourData_GetCustomColour, 2208). +-define(wxColourData_SetChooseFull, 2209). +-define(wxColourData_SetColour, 2210). +-define(wxColourData_SetCustomColour, 2211). +-define(wxPalette_new_0, 2212). +-define(wxPalette_new_4, 2213). +-define(wxPalette_destruct, 2215). +-define(wxPalette_Create, 2216). +-define(wxPalette_GetColoursCount, 2217). +-define(wxPalette_GetPixel, 2218). +-define(wxPalette_GetRGB, 2219). +-define(wxPalette_IsOk, 2220). +-define(wxDirDialog_new, 2224). +-define(wxDirDialog_destruct, 2225). +-define(wxDirDialog_GetPath, 2226). +-define(wxDirDialog_GetMessage, 2227). +-define(wxDirDialog_SetMessage, 2228). +-define(wxDirDialog_SetPath, 2229). +-define(wxFileDialog_new, 2233). +-define(wxFileDialog_destruct, 2234). +-define(wxFileDialog_GetDirectory, 2235). +-define(wxFileDialog_GetFilename, 2236). +-define(wxFileDialog_GetFilenames, 2237). +-define(wxFileDialog_GetFilterIndex, 2238). +-define(wxFileDialog_GetMessage, 2239). +-define(wxFileDialog_GetPath, 2240). +-define(wxFileDialog_GetPaths, 2241). +-define(wxFileDialog_GetWildcard, 2242). +-define(wxFileDialog_SetDirectory, 2243). +-define(wxFileDialog_SetFilename, 2244). +-define(wxFileDialog_SetFilterIndex, 2245). +-define(wxFileDialog_SetMessage, 2246). +-define(wxFileDialog_SetPath, 2247). +-define(wxFileDialog_SetWildcard, 2248). +-define(wxPickerBase_SetInternalMargin, 2249). +-define(wxPickerBase_GetInternalMargin, 2250). +-define(wxPickerBase_SetTextCtrlProportion, 2251). +-define(wxPickerBase_SetPickerCtrlProportion, 2252). +-define(wxPickerBase_GetTextCtrlProportion, 2253). +-define(wxPickerBase_GetPickerCtrlProportion, 2254). +-define(wxPickerBase_HasTextCtrl, 2255). +-define(wxPickerBase_GetTextCtrl, 2256). +-define(wxPickerBase_IsTextCtrlGrowable, 2257). +-define(wxPickerBase_SetPickerCtrlGrowable, 2258). +-define(wxPickerBase_SetTextCtrlGrowable, 2259). +-define(wxPickerBase_IsPickerCtrlGrowable, 2260). +-define(wxFilePickerCtrl_new_0, 2261). +-define(wxFilePickerCtrl_new_3, 2262). +-define(wxFilePickerCtrl_Create, 2263). +-define(wxFilePickerCtrl_GetPath, 2264). +-define(wxFilePickerCtrl_SetPath, 2265). +-define(wxFilePickerCtrl_destroy, 2266). +-define(wxDirPickerCtrl_new_0, 2267). +-define(wxDirPickerCtrl_new_3, 2268). +-define(wxDirPickerCtrl_Create, 2269). +-define(wxDirPickerCtrl_GetPath, 2270). +-define(wxDirPickerCtrl_SetPath, 2271). +-define(wxDirPickerCtrl_destroy, 2272). +-define(wxColourPickerCtrl_new_0, 2273). +-define(wxColourPickerCtrl_new_3, 2274). +-define(wxColourPickerCtrl_Create, 2275). +-define(wxColourPickerCtrl_GetColour, 2276). +-define(wxColourPickerCtrl_SetColour_1_1, 2277). +-define(wxColourPickerCtrl_SetColour_1_0, 2278). +-define(wxColourPickerCtrl_destroy, 2279). +-define(wxDatePickerCtrl_new_0, 2280). +-define(wxDatePickerCtrl_new_3, 2281). +-define(wxDatePickerCtrl_GetRange, 2282). +-define(wxDatePickerCtrl_GetValue, 2283). +-define(wxDatePickerCtrl_SetRange, 2284). +-define(wxDatePickerCtrl_SetValue, 2285). +-define(wxDatePickerCtrl_destroy, 2286). +-define(wxFontPickerCtrl_new_0, 2287). +-define(wxFontPickerCtrl_new_3, 2288). +-define(wxFontPickerCtrl_Create, 2289). +-define(wxFontPickerCtrl_GetSelectedFont, 2290). +-define(wxFontPickerCtrl_SetSelectedFont, 2291). +-define(wxFontPickerCtrl_GetMaxPointSize, 2292). +-define(wxFontPickerCtrl_SetMaxPointSize, 2293). +-define(wxFontPickerCtrl_destroy, 2294). +-define(wxFindReplaceDialog_new_0, 2297). +-define(wxFindReplaceDialog_new_4, 2298). +-define(wxFindReplaceDialog_destruct, 2299). +-define(wxFindReplaceDialog_Create, 2300). +-define(wxFindReplaceDialog_GetData, 2301). +-define(wxFindReplaceData_new_0, 2302). +-define(wxFindReplaceData_new_1, 2303). +-define(wxFindReplaceData_GetFindString, 2304). +-define(wxFindReplaceData_GetReplaceString, 2305). +-define(wxFindReplaceData_GetFlags, 2306). +-define(wxFindReplaceData_SetFlags, 2307). +-define(wxFindReplaceData_SetFindString, 2308). +-define(wxFindReplaceData_SetReplaceString, 2309). +-define(wxFindReplaceData_destroy, 2310). +-define(wxMultiChoiceDialog_new_0, 2311). +-define(wxMultiChoiceDialog_new_5, 2313). +-define(wxMultiChoiceDialog_GetSelections, 2314). +-define(wxMultiChoiceDialog_SetSelections, 2315). +-define(wxMultiChoiceDialog_destroy, 2316). +-define(wxSingleChoiceDialog_new_0, 2317). +-define(wxSingleChoiceDialog_new_5, 2319). +-define(wxSingleChoiceDialog_GetSelection, 2320). +-define(wxSingleChoiceDialog_GetStringSelection, 2321). +-define(wxSingleChoiceDialog_SetSelection, 2322). +-define(wxSingleChoiceDialog_destroy, 2323). +-define(wxTextEntryDialog_new, 2324). +-define(wxTextEntryDialog_GetValue, 2325). +-define(wxTextEntryDialog_SetValue, 2326). +-define(wxTextEntryDialog_destroy, 2327). +-define(wxPasswordEntryDialog_new, 2328). +-define(wxPasswordEntryDialog_destroy, 2329). +-define(wxFontData_new_0, 2330). +-define(wxFontData_new_1, 2331). +-define(wxFontData_destruct, 2332). +-define(wxFontData_EnableEffects, 2333). +-define(wxFontData_GetAllowSymbols, 2334). +-define(wxFontData_GetColour, 2335). +-define(wxFontData_GetChosenFont, 2336). +-define(wxFontData_GetEnableEffects, 2337). +-define(wxFontData_GetInitialFont, 2338). +-define(wxFontData_GetShowHelp, 2339). +-define(wxFontData_SetAllowSymbols, 2340). +-define(wxFontData_SetChosenFont, 2341). +-define(wxFontData_SetColour, 2342). +-define(wxFontData_SetInitialFont, 2343). +-define(wxFontData_SetRange, 2344). +-define(wxFontData_SetShowHelp, 2345). +-define(wxFontDialog_new_0, 2349). +-define(wxFontDialog_new_2, 2351). +-define(wxFontDialog_Create, 2353). +-define(wxFontDialog_GetFontData, 2354). +-define(wxFontDialog_destroy, 2356). +-define(wxProgressDialog_new, 2357). +-define(wxProgressDialog_destruct, 2358). +-define(wxProgressDialog_Resume, 2359). +-define(wxProgressDialog_Update_2, 2360). +-define(wxProgressDialog_Update_0, 2361). +-define(wxMessageDialog_new, 2362). +-define(wxMessageDialog_destruct, 2363). +-define(wxPageSetupDialog_new, 2364). +-define(wxPageSetupDialog_destruct, 2365). +-define(wxPageSetupDialog_GetPageSetupData, 2366). +-define(wxPageSetupDialog_ShowModal, 2367). +-define(wxPageSetupDialogData_new_0, 2368). +-define(wxPageSetupDialogData_new_1_0, 2369). +-define(wxPageSetupDialogData_new_1_1, 2370). +-define(wxPageSetupDialogData_destruct, 2371). +-define(wxPageSetupDialogData_EnableHelp, 2372). +-define(wxPageSetupDialogData_EnableMargins, 2373). +-define(wxPageSetupDialogData_EnableOrientation, 2374). +-define(wxPageSetupDialogData_EnablePaper, 2375). +-define(wxPageSetupDialogData_EnablePrinter, 2376). +-define(wxPageSetupDialogData_GetDefaultMinMargins, 2377). +-define(wxPageSetupDialogData_GetEnableMargins, 2378). +-define(wxPageSetupDialogData_GetEnableOrientation, 2379). +-define(wxPageSetupDialogData_GetEnablePaper, 2380). +-define(wxPageSetupDialogData_GetEnablePrinter, 2381). +-define(wxPageSetupDialogData_GetEnableHelp, 2382). +-define(wxPageSetupDialogData_GetDefaultInfo, 2383). +-define(wxPageSetupDialogData_GetMarginTopLeft, 2384). +-define(wxPageSetupDialogData_GetMarginBottomRight, 2385). +-define(wxPageSetupDialogData_GetMinMarginTopLeft, 2386). +-define(wxPageSetupDialogData_GetMinMarginBottomRight, 2387). +-define(wxPageSetupDialogData_GetPaperId, 2388). +-define(wxPageSetupDialogData_GetPaperSize, 2389). +-define(wxPageSetupDialogData_GetPrintData, 2391). +-define(wxPageSetupDialogData_IsOk, 2392). +-define(wxPageSetupDialogData_SetDefaultInfo, 2393). +-define(wxPageSetupDialogData_SetDefaultMinMargins, 2394). +-define(wxPageSetupDialogData_SetMarginTopLeft, 2395). +-define(wxPageSetupDialogData_SetMarginBottomRight, 2396). +-define(wxPageSetupDialogData_SetMinMarginTopLeft, 2397). +-define(wxPageSetupDialogData_SetMinMarginBottomRight, 2398). +-define(wxPageSetupDialogData_SetPaperId, 2399). +-define(wxPageSetupDialogData_SetPaperSize_1_1, 2400). +-define(wxPageSetupDialogData_SetPaperSize_1_0, 2401). +-define(wxPageSetupDialogData_SetPrintData, 2402). +-define(wxPrintDialog_new_2_0, 2403). +-define(wxPrintDialog_new_2_1, 2404). +-define(wxPrintDialog_destruct, 2405). +-define(wxPrintDialog_GetPrintDialogData, 2406). +-define(wxPrintDialog_GetPrintDC, 2407). +-define(wxPrintDialogData_new_0, 2408). +-define(wxPrintDialogData_new_1_1, 2409). +-define(wxPrintDialogData_new_1_0, 2410). +-define(wxPrintDialogData_destruct, 2411). +-define(wxPrintDialogData_EnableHelp, 2412). +-define(wxPrintDialogData_EnablePageNumbers, 2413). +-define(wxPrintDialogData_EnablePrintToFile, 2414). +-define(wxPrintDialogData_EnableSelection, 2415). +-define(wxPrintDialogData_GetAllPages, 2416). +-define(wxPrintDialogData_GetCollate, 2417). +-define(wxPrintDialogData_GetFromPage, 2418). +-define(wxPrintDialogData_GetMaxPage, 2419). +-define(wxPrintDialogData_GetMinPage, 2420). +-define(wxPrintDialogData_GetNoCopies, 2421). +-define(wxPrintDialogData_GetPrintData, 2422). +-define(wxPrintDialogData_GetPrintToFile, 2423). +-define(wxPrintDialogData_GetSelection, 2424). +-define(wxPrintDialogData_GetToPage, 2425). +-define(wxPrintDialogData_IsOk, 2426). +-define(wxPrintDialogData_SetCollate, 2427). +-define(wxPrintDialogData_SetFromPage, 2428). +-define(wxPrintDialogData_SetMaxPage, 2429). +-define(wxPrintDialogData_SetMinPage, 2430). +-define(wxPrintDialogData_SetNoCopies, 2431). +-define(wxPrintDialogData_SetPrintData, 2432). +-define(wxPrintDialogData_SetPrintToFile, 2433). +-define(wxPrintDialogData_SetSelection, 2434). +-define(wxPrintDialogData_SetToPage, 2435). +-define(wxPrintData_new_0, 2436). +-define(wxPrintData_new_1, 2437). +-define(wxPrintData_destruct, 2438). +-define(wxPrintData_GetCollate, 2439). +-define(wxPrintData_GetBin, 2440). +-define(wxPrintData_GetColour, 2441). +-define(wxPrintData_GetDuplex, 2442). +-define(wxPrintData_GetNoCopies, 2443). +-define(wxPrintData_GetOrientation, 2444). +-define(wxPrintData_GetPaperId, 2445). +-define(wxPrintData_GetPrinterName, 2446). +-define(wxPrintData_GetQuality, 2447). +-define(wxPrintData_IsOk, 2448). +-define(wxPrintData_SetBin, 2449). +-define(wxPrintData_SetCollate, 2450). +-define(wxPrintData_SetColour, 2451). +-define(wxPrintData_SetDuplex, 2452). +-define(wxPrintData_SetNoCopies, 2453). +-define(wxPrintData_SetOrientation, 2454). +-define(wxPrintData_SetPaperId, 2455). +-define(wxPrintData_SetPrinterName, 2456). +-define(wxPrintData_SetQuality, 2457). +-define(wxPrintPreview_new_2, 2460). +-define(wxPrintPreview_new_3, 2461). +-define(wxPrintPreview_destruct, 2463). +-define(wxPrintPreview_GetCanvas, 2464). +-define(wxPrintPreview_GetCurrentPage, 2465). +-define(wxPrintPreview_GetFrame, 2466). +-define(wxPrintPreview_GetMaxPage, 2467). +-define(wxPrintPreview_GetMinPage, 2468). +-define(wxPrintPreview_GetPrintout, 2469). +-define(wxPrintPreview_GetPrintoutForPrinting, 2470). +-define(wxPrintPreview_IsOk, 2471). +-define(wxPrintPreview_PaintPage, 2472). +-define(wxPrintPreview_Print, 2473). +-define(wxPrintPreview_RenderPage, 2474). +-define(wxPrintPreview_SetCanvas, 2475). +-define(wxPrintPreview_SetCurrentPage, 2476). +-define(wxPrintPreview_SetFrame, 2477). +-define(wxPrintPreview_SetPrintout, 2478). +-define(wxPrintPreview_SetZoom, 2479). +-define(wxPreviewFrame_new, 2480). +-define(wxPreviewFrame_destruct, 2481). +-define(wxPreviewFrame_CreateControlBar, 2482). +-define(wxPreviewFrame_CreateCanvas, 2483). +-define(wxPreviewFrame_Initialize, 2484). +-define(wxPreviewFrame_OnCloseWindow, 2485). +-define(wxPreviewControlBar_new, 2486). +-define(wxPreviewControlBar_destruct, 2487). +-define(wxPreviewControlBar_CreateButtons, 2488). +-define(wxPreviewControlBar_GetPrintPreview, 2489). +-define(wxPreviewControlBar_GetZoomControl, 2490). +-define(wxPreviewControlBar_SetZoomControl, 2491). +-define(wxPrinter_new, 2493). +-define(wxPrinter_CreateAbortWindow, 2494). +-define(wxPrinter_GetAbort, 2495). +-define(wxPrinter_GetLastError, 2496). +-define(wxPrinter_GetPrintDialogData, 2497). +-define(wxPrinter_Print, 2498). +-define(wxPrinter_PrintDialog, 2499). +-define(wxPrinter_ReportError, 2500). +-define(wxPrinter_Setup, 2501). +-define(wxPrinter_destroy, 2502). +-define(wxXmlResource_new_1, 2503). +-define(wxXmlResource_new_2, 2504). +-define(wxXmlResource_destruct, 2505). +-define(wxXmlResource_AttachUnknownControl, 2506). +-define(wxXmlResource_ClearHandlers, 2507). +-define(wxXmlResource_CompareVersion, 2508). +-define(wxXmlResource_Get, 2509). +-define(wxXmlResource_GetFlags, 2510). +-define(wxXmlResource_GetVersion, 2511). +-define(wxXmlResource_GetXRCID, 2512). +-define(wxXmlResource_InitAllHandlers, 2513). +-define(wxXmlResource_Load, 2514). +-define(wxXmlResource_LoadBitmap, 2515). +-define(wxXmlResource_LoadDialog_2, 2516). +-define(wxXmlResource_LoadDialog_3, 2517). +-define(wxXmlResource_LoadFrame_2, 2518). +-define(wxXmlResource_LoadFrame_3, 2519). +-define(wxXmlResource_LoadIcon, 2520). +-define(wxXmlResource_LoadMenu, 2521). +-define(wxXmlResource_LoadMenuBar_2, 2522). +-define(wxXmlResource_LoadMenuBar_1, 2523). +-define(wxXmlResource_LoadPanel_2, 2524). +-define(wxXmlResource_LoadPanel_3, 2525). +-define(wxXmlResource_LoadToolBar, 2526). +-define(wxXmlResource_Set, 2527). +-define(wxXmlResource_SetFlags, 2528). +-define(wxXmlResource_Unload, 2529). +-define(wxXmlResource_xrcctrl, 2530). +-define(wxHtmlEasyPrinting_new, 2531). +-define(wxHtmlEasyPrinting_destruct, 2532). +-define(wxHtmlEasyPrinting_GetPrintData, 2533). +-define(wxHtmlEasyPrinting_GetPageSetupData, 2534). +-define(wxHtmlEasyPrinting_PreviewFile, 2535). +-define(wxHtmlEasyPrinting_PreviewText, 2536). +-define(wxHtmlEasyPrinting_PrintFile, 2537). +-define(wxHtmlEasyPrinting_PrintText, 2538). +-define(wxHtmlEasyPrinting_PageSetup, 2539). +-define(wxHtmlEasyPrinting_SetFonts, 2540). +-define(wxHtmlEasyPrinting_SetHeader, 2541). +-define(wxHtmlEasyPrinting_SetFooter, 2542). +-define(wxGLCanvas_new_2, 2544). +-define(wxGLCanvas_new_3_1, 2545). +-define(wxGLCanvas_new_3_0, 2546). +-define(wxGLCanvas_GetContext, 2547). +-define(wxGLCanvas_SetCurrent, 2549). +-define(wxGLCanvas_SwapBuffers, 2550). +-define(wxGLCanvas_destroy, 2551). +-define(wxAuiManager_new, 2552). +-define(wxAuiManager_destruct, 2553). +-define(wxAuiManager_AddPane_2_1, 2554). +-define(wxAuiManager_AddPane_3, 2555). +-define(wxAuiManager_AddPane_2_0, 2556). +-define(wxAuiManager_DetachPane, 2557). +-define(wxAuiManager_GetAllPanes, 2558). +-define(wxAuiManager_GetArtProvider, 2559). +-define(wxAuiManager_GetDockSizeConstraint, 2560). +-define(wxAuiManager_GetFlags, 2561). +-define(wxAuiManager_GetManagedWindow, 2562). +-define(wxAuiManager_GetManager, 2563). +-define(wxAuiManager_GetPane_1_1, 2564). +-define(wxAuiManager_GetPane_1_0, 2565). +-define(wxAuiManager_HideHint, 2566). +-define(wxAuiManager_InsertPane, 2567). +-define(wxAuiManager_LoadPaneInfo, 2568). +-define(wxAuiManager_LoadPerspective, 2569). +-define(wxAuiManager_SavePaneInfo, 2570). +-define(wxAuiManager_SavePerspective, 2571). +-define(wxAuiManager_SetArtProvider, 2572). +-define(wxAuiManager_SetDockSizeConstraint, 2573). +-define(wxAuiManager_SetFlags, 2574). +-define(wxAuiManager_SetManagedWindow, 2575). +-define(wxAuiManager_ShowHint, 2576). +-define(wxAuiManager_UnInit, 2577). +-define(wxAuiManager_Update, 2578). +-define(wxAuiPaneInfo_new_0, 2579). +-define(wxAuiPaneInfo_new_1, 2580). +-define(wxAuiPaneInfo_destruct, 2581). +-define(wxAuiPaneInfo_BestSize_1, 2582). +-define(wxAuiPaneInfo_BestSize_2, 2583). +-define(wxAuiPaneInfo_Bottom, 2584). +-define(wxAuiPaneInfo_BottomDockable, 2585). +-define(wxAuiPaneInfo_Caption, 2586). +-define(wxAuiPaneInfo_CaptionVisible, 2587). +-define(wxAuiPaneInfo_Centre, 2588). +-define(wxAuiPaneInfo_CentrePane, 2589). +-define(wxAuiPaneInfo_CloseButton, 2590). +-define(wxAuiPaneInfo_DefaultPane, 2591). +-define(wxAuiPaneInfo_DestroyOnClose, 2592). +-define(wxAuiPaneInfo_Direction, 2593). +-define(wxAuiPaneInfo_Dock, 2594). +-define(wxAuiPaneInfo_Dockable, 2595). +-define(wxAuiPaneInfo_Fixed, 2596). +-define(wxAuiPaneInfo_Float, 2597). +-define(wxAuiPaneInfo_Floatable, 2598). +-define(wxAuiPaneInfo_FloatingPosition_1, 2599). +-define(wxAuiPaneInfo_FloatingPosition_2, 2600). +-define(wxAuiPaneInfo_FloatingSize_1, 2601). +-define(wxAuiPaneInfo_FloatingSize_2, 2602). +-define(wxAuiPaneInfo_Gripper, 2603). +-define(wxAuiPaneInfo_GripperTop, 2604). +-define(wxAuiPaneInfo_HasBorder, 2605). +-define(wxAuiPaneInfo_HasCaption, 2606). +-define(wxAuiPaneInfo_HasCloseButton, 2607). +-define(wxAuiPaneInfo_HasFlag, 2608). +-define(wxAuiPaneInfo_HasGripper, 2609). +-define(wxAuiPaneInfo_HasGripperTop, 2610). +-define(wxAuiPaneInfo_HasMaximizeButton, 2611). +-define(wxAuiPaneInfo_HasMinimizeButton, 2612). +-define(wxAuiPaneInfo_HasPinButton, 2613). +-define(wxAuiPaneInfo_Hide, 2614). +-define(wxAuiPaneInfo_IsBottomDockable, 2615). +-define(wxAuiPaneInfo_IsDocked, 2616). +-define(wxAuiPaneInfo_IsFixed, 2617). +-define(wxAuiPaneInfo_IsFloatable, 2618). +-define(wxAuiPaneInfo_IsFloating, 2619). +-define(wxAuiPaneInfo_IsLeftDockable, 2620). +-define(wxAuiPaneInfo_IsMovable, 2621). +-define(wxAuiPaneInfo_IsOk, 2622). +-define(wxAuiPaneInfo_IsResizable, 2623). +-define(wxAuiPaneInfo_IsRightDockable, 2624). +-define(wxAuiPaneInfo_IsShown, 2625). +-define(wxAuiPaneInfo_IsToolbar, 2626). +-define(wxAuiPaneInfo_IsTopDockable, 2627). +-define(wxAuiPaneInfo_Layer, 2628). +-define(wxAuiPaneInfo_Left, 2629). +-define(wxAuiPaneInfo_LeftDockable, 2630). +-define(wxAuiPaneInfo_MaxSize_1, 2631). +-define(wxAuiPaneInfo_MaxSize_2, 2632). +-define(wxAuiPaneInfo_MaximizeButton, 2633). +-define(wxAuiPaneInfo_MinSize_1, 2634). +-define(wxAuiPaneInfo_MinSize_2, 2635). +-define(wxAuiPaneInfo_MinimizeButton, 2636). +-define(wxAuiPaneInfo_Movable, 2637). +-define(wxAuiPaneInfo_Name, 2638). +-define(wxAuiPaneInfo_PaneBorder, 2639). +-define(wxAuiPaneInfo_PinButton, 2640). +-define(wxAuiPaneInfo_Position, 2641). +-define(wxAuiPaneInfo_Resizable, 2642). +-define(wxAuiPaneInfo_Right, 2643). +-define(wxAuiPaneInfo_RightDockable, 2644). +-define(wxAuiPaneInfo_Row, 2645). +-define(wxAuiPaneInfo_SafeSet, 2646). +-define(wxAuiPaneInfo_SetFlag, 2647). +-define(wxAuiPaneInfo_Show, 2648). +-define(wxAuiPaneInfo_ToolbarPane, 2649). +-define(wxAuiPaneInfo_Top, 2650). +-define(wxAuiPaneInfo_TopDockable, 2651). +-define(wxAuiPaneInfo_Window, 2652). +-define(wxAuiNotebook_new_0, 2653). +-define(wxAuiNotebook_new_2, 2654). +-define(wxAuiNotebook_AddPage, 2655). +-define(wxAuiNotebook_Create, 2656). +-define(wxAuiNotebook_DeletePage, 2657). +-define(wxAuiNotebook_GetArtProvider, 2658). +-define(wxAuiNotebook_GetPage, 2659). +-define(wxAuiNotebook_GetPageBitmap, 2660). +-define(wxAuiNotebook_GetPageCount, 2661). +-define(wxAuiNotebook_GetPageIndex, 2662). +-define(wxAuiNotebook_GetPageText, 2663). +-define(wxAuiNotebook_GetSelection, 2664). +-define(wxAuiNotebook_InsertPage, 2665). +-define(wxAuiNotebook_RemovePage, 2666). +-define(wxAuiNotebook_SetArtProvider, 2667). +-define(wxAuiNotebook_SetFont, 2668). +-define(wxAuiNotebook_SetPageBitmap, 2669). +-define(wxAuiNotebook_SetPageText, 2670). +-define(wxAuiNotebook_SetSelection, 2671). +-define(wxAuiNotebook_SetTabCtrlHeight, 2672). +-define(wxAuiNotebook_SetUniformBitmapSize, 2673). +-define(wxAuiNotebook_destroy, 2674). +-define(wxMDIParentFrame_new_0, 2675). +-define(wxMDIParentFrame_new_4, 2676). +-define(wxMDIParentFrame_destruct, 2677). +-define(wxMDIParentFrame_ActivateNext, 2678). +-define(wxMDIParentFrame_ActivatePrevious, 2679). +-define(wxMDIParentFrame_ArrangeIcons, 2680). +-define(wxMDIParentFrame_Cascade, 2681). +-define(wxMDIParentFrame_Create, 2682). +-define(wxMDIParentFrame_GetActiveChild, 2683). +-define(wxMDIParentFrame_GetClientWindow, 2684). +-define(wxMDIParentFrame_Tile, 2685). +-define(wxMDIChildFrame_new_0, 2686). +-define(wxMDIChildFrame_new_4, 2687). +-define(wxMDIChildFrame_destruct, 2688). +-define(wxMDIChildFrame_Activate, 2689). +-define(wxMDIChildFrame_Create, 2690). +-define(wxMDIChildFrame_Maximize, 2691). +-define(wxMDIChildFrame_Restore, 2692). +-define(wxMDIClientWindow_new_0, 2693). +-define(wxMDIClientWindow_new_2, 2694). +-define(wxMDIClientWindow_destruct, 2695). +-define(wxMDIClientWindow_CreateClient, 2696). +-define(wxLayoutAlgorithm_new, 2697). +-define(wxLayoutAlgorithm_LayoutFrame, 2698). +-define(wxLayoutAlgorithm_LayoutMDIFrame, 2699). +-define(wxLayoutAlgorithm_LayoutWindow, 2700). +-define(wxLayoutAlgorithm_destroy, 2701). +-define(wxEvent_GetId, 2702). +-define(wxEvent_GetSkipped, 2703). +-define(wxEvent_GetTimestamp, 2704). +-define(wxEvent_IsCommandEvent, 2705). +-define(wxEvent_ResumePropagation, 2706). +-define(wxEvent_ShouldPropagate, 2707). +-define(wxEvent_Skip, 2708). +-define(wxEvent_StopPropagation, 2709). +-define(wxCommandEvent_getClientData, 2710). +-define(wxCommandEvent_GetExtraLong, 2711). +-define(wxCommandEvent_GetInt, 2712). +-define(wxCommandEvent_GetSelection, 2713). +-define(wxCommandEvent_GetString, 2714). +-define(wxCommandEvent_IsChecked, 2715). +-define(wxCommandEvent_IsSelection, 2716). +-define(wxCommandEvent_SetInt, 2717). +-define(wxCommandEvent_SetString, 2718). +-define(wxScrollEvent_GetOrientation, 2719). +-define(wxScrollEvent_GetPosition, 2720). +-define(wxScrollWinEvent_GetOrientation, 2721). +-define(wxScrollWinEvent_GetPosition, 2722). +-define(wxMouseEvent_AltDown, 2723). +-define(wxMouseEvent_Button, 2724). +-define(wxMouseEvent_ButtonDClick, 2725). +-define(wxMouseEvent_ButtonDown, 2726). +-define(wxMouseEvent_ButtonUp, 2727). +-define(wxMouseEvent_CmdDown, 2728). +-define(wxMouseEvent_ControlDown, 2729). +-define(wxMouseEvent_Dragging, 2730). +-define(wxMouseEvent_Entering, 2731). +-define(wxMouseEvent_GetButton, 2732). +-define(wxMouseEvent_GetPosition, 2735). +-define(wxMouseEvent_GetLogicalPosition, 2736). +-define(wxMouseEvent_GetLinesPerAction, 2737). +-define(wxMouseEvent_GetWheelRotation, 2738). +-define(wxMouseEvent_GetWheelDelta, 2739). +-define(wxMouseEvent_GetX, 2740). +-define(wxMouseEvent_GetY, 2741). +-define(wxMouseEvent_IsButton, 2742). +-define(wxMouseEvent_IsPageScroll, 2743). +-define(wxMouseEvent_Leaving, 2744). +-define(wxMouseEvent_LeftDClick, 2745). +-define(wxMouseEvent_LeftDown, 2746). +-define(wxMouseEvent_LeftIsDown, 2747). +-define(wxMouseEvent_LeftUp, 2748). +-define(wxMouseEvent_MetaDown, 2749). +-define(wxMouseEvent_MiddleDClick, 2750). +-define(wxMouseEvent_MiddleDown, 2751). +-define(wxMouseEvent_MiddleIsDown, 2752). +-define(wxMouseEvent_MiddleUp, 2753). +-define(wxMouseEvent_Moving, 2754). +-define(wxMouseEvent_RightDClick, 2755). +-define(wxMouseEvent_RightDown, 2756). +-define(wxMouseEvent_RightIsDown, 2757). +-define(wxMouseEvent_RightUp, 2758). +-define(wxMouseEvent_ShiftDown, 2759). +-define(wxSetCursorEvent_GetCursor, 2760). +-define(wxSetCursorEvent_GetX, 2761). +-define(wxSetCursorEvent_GetY, 2762). +-define(wxSetCursorEvent_HasCursor, 2763). +-define(wxSetCursorEvent_SetCursor, 2764). +-define(wxKeyEvent_AltDown, 2765). +-define(wxKeyEvent_CmdDown, 2766). +-define(wxKeyEvent_ControlDown, 2767). +-define(wxKeyEvent_GetKeyCode, 2768). +-define(wxKeyEvent_GetModifiers, 2769). +-define(wxKeyEvent_GetPosition, 2772). +-define(wxKeyEvent_GetRawKeyCode, 2773). +-define(wxKeyEvent_GetRawKeyFlags, 2774). +-define(wxKeyEvent_GetUnicodeKey, 2775). +-define(wxKeyEvent_GetX, 2776). +-define(wxKeyEvent_GetY, 2777). +-define(wxKeyEvent_HasModifiers, 2778). +-define(wxKeyEvent_MetaDown, 2779). +-define(wxKeyEvent_ShiftDown, 2780). +-define(wxSizeEvent_GetSize, 2781). +-define(wxMoveEvent_GetPosition, 2782). +-define(wxEraseEvent_GetDC, 2783). +-define(wxFocusEvent_GetWindow, 2784). +-define(wxChildFocusEvent_GetWindow, 2785). +-define(wxMenuEvent_GetMenu, 2786). +-define(wxMenuEvent_GetMenuId, 2787). +-define(wxMenuEvent_IsPopup, 2788). +-define(wxCloseEvent_CanVeto, 2789). +-define(wxCloseEvent_GetLoggingOff, 2790). +-define(wxCloseEvent_SetCanVeto, 2791). +-define(wxCloseEvent_SetLoggingOff, 2792). +-define(wxCloseEvent_Veto, 2793). +-define(wxShowEvent_SetShow, 2794). +-define(wxShowEvent_GetShow, 2795). +-define(wxIconizeEvent_Iconized, 2796). +-define(wxJoystickEvent_ButtonDown, 2797). +-define(wxJoystickEvent_ButtonIsDown, 2798). +-define(wxJoystickEvent_ButtonUp, 2799). +-define(wxJoystickEvent_GetButtonChange, 2800). +-define(wxJoystickEvent_GetButtonState, 2801). +-define(wxJoystickEvent_GetJoystick, 2802). +-define(wxJoystickEvent_GetPosition, 2803). +-define(wxJoystickEvent_GetZPosition, 2804). +-define(wxJoystickEvent_IsButton, 2805). +-define(wxJoystickEvent_IsMove, 2806). +-define(wxJoystickEvent_IsZMove, 2807). +-define(wxUpdateUIEvent_CanUpdate, 2808). +-define(wxUpdateUIEvent_Check, 2809). +-define(wxUpdateUIEvent_Enable, 2810). +-define(wxUpdateUIEvent_Show, 2811). +-define(wxUpdateUIEvent_GetChecked, 2812). +-define(wxUpdateUIEvent_GetEnabled, 2813). +-define(wxUpdateUIEvent_GetShown, 2814). +-define(wxUpdateUIEvent_GetSetChecked, 2815). +-define(wxUpdateUIEvent_GetSetEnabled, 2816). +-define(wxUpdateUIEvent_GetSetShown, 2817). +-define(wxUpdateUIEvent_GetSetText, 2818). +-define(wxUpdateUIEvent_GetText, 2819). +-define(wxUpdateUIEvent_GetMode, 2820). +-define(wxUpdateUIEvent_GetUpdateInterval, 2821). +-define(wxUpdateUIEvent_ResetUpdateTime, 2822). +-define(wxUpdateUIEvent_SetMode, 2823). +-define(wxUpdateUIEvent_SetText, 2824). +-define(wxUpdateUIEvent_SetUpdateInterval, 2825). +-define(wxMouseCaptureChangedEvent_GetCapturedWindow, 2826). +-define(wxPaletteChangedEvent_SetChangedWindow, 2827). +-define(wxPaletteChangedEvent_GetChangedWindow, 2828). +-define(wxQueryNewPaletteEvent_SetPaletteRealized, 2829). +-define(wxQueryNewPaletteEvent_GetPaletteRealized, 2830). +-define(wxNavigationKeyEvent_GetDirection, 2831). +-define(wxNavigationKeyEvent_SetDirection, 2832). +-define(wxNavigationKeyEvent_IsWindowChange, 2833). +-define(wxNavigationKeyEvent_SetWindowChange, 2834). +-define(wxNavigationKeyEvent_IsFromTab, 2835). +-define(wxNavigationKeyEvent_SetFromTab, 2836). +-define(wxNavigationKeyEvent_GetCurrentFocus, 2837). +-define(wxNavigationKeyEvent_SetCurrentFocus, 2838). +-define(wxHelpEvent_GetOrigin, 2839). +-define(wxHelpEvent_GetPosition, 2840). +-define(wxHelpEvent_SetOrigin, 2841). +-define(wxHelpEvent_SetPosition, 2842). +-define(wxContextMenuEvent_GetPosition, 2843). +-define(wxContextMenuEvent_SetPosition, 2844). +-define(wxIdleEvent_CanSend, 2845). +-define(wxIdleEvent_GetMode, 2846). +-define(wxIdleEvent_RequestMore, 2847). +-define(wxIdleEvent_MoreRequested, 2848). +-define(wxIdleEvent_SetMode, 2849). +-define(wxGridEvent_AltDown, 2850). +-define(wxGridEvent_ControlDown, 2851). +-define(wxGridEvent_GetCol, 2852). +-define(wxGridEvent_GetPosition, 2853). +-define(wxGridEvent_GetRow, 2854). +-define(wxGridEvent_MetaDown, 2855). +-define(wxGridEvent_Selecting, 2856). +-define(wxGridEvent_ShiftDown, 2857). +-define(wxNotifyEvent_Allow, 2858). +-define(wxNotifyEvent_IsAllowed, 2859). +-define(wxNotifyEvent_Veto, 2860). +-define(wxSashEvent_GetEdge, 2861). +-define(wxSashEvent_GetDragRect, 2862). +-define(wxSashEvent_GetDragStatus, 2863). +-define(wxListEvent_GetCacheFrom, 2864). +-define(wxListEvent_GetCacheTo, 2865). +-define(wxListEvent_GetKeyCode, 2866). +-define(wxListEvent_GetIndex, 2867). +-define(wxListEvent_GetColumn, 2868). +-define(wxListEvent_GetPoint, 2869). +-define(wxListEvent_GetLabel, 2870). +-define(wxListEvent_GetText, 2871). +-define(wxListEvent_GetImage, 2872). +-define(wxListEvent_GetData, 2873). +-define(wxListEvent_GetMask, 2874). +-define(wxListEvent_GetItem, 2875). +-define(wxListEvent_IsEditCancelled, 2876). +-define(wxDateEvent_GetDate, 2877). +-define(wxCalendarEvent_GetWeekDay, 2878). +-define(wxFileDirPickerEvent_GetPath, 2879). +-define(wxColourPickerEvent_GetColour, 2880). +-define(wxFontPickerEvent_GetFont, 2881). +-define(wxStyledTextEvent_GetPosition, 2882). +-define(wxStyledTextEvent_GetKey, 2883). +-define(wxStyledTextEvent_GetModifiers, 2884). +-define(wxStyledTextEvent_GetModificationType, 2885). +-define(wxStyledTextEvent_GetText, 2886). +-define(wxStyledTextEvent_GetLength, 2887). +-define(wxStyledTextEvent_GetLinesAdded, 2888). +-define(wxStyledTextEvent_GetLine, 2889). +-define(wxStyledTextEvent_GetFoldLevelNow, 2890). +-define(wxStyledTextEvent_GetFoldLevelPrev, 2891). +-define(wxStyledTextEvent_GetMargin, 2892). +-define(wxStyledTextEvent_GetMessage, 2893). +-define(wxStyledTextEvent_GetWParam, 2894). +-define(wxStyledTextEvent_GetLParam, 2895). +-define(wxStyledTextEvent_GetListType, 2896). +-define(wxStyledTextEvent_GetX, 2897). +-define(wxStyledTextEvent_GetY, 2898). +-define(wxStyledTextEvent_GetDragText, 2899). +-define(wxStyledTextEvent_GetDragAllowMove, 2900). +-define(wxStyledTextEvent_GetDragResult, 2901). +-define(wxStyledTextEvent_GetShift, 2902). +-define(wxStyledTextEvent_GetControl, 2903). +-define(wxStyledTextEvent_GetAlt, 2904). +-define(utils_wxGetKeyState, 2905). +-define(utils_wxGetMousePosition, 2906). +-define(utils_wxGetMouseState, 2907). +-define(utils_wxSetDetectableAutoRepeat, 2908). +-define(utils_wxBell, 2909). +-define(utils_wxFindMenuItemId, 2910). +-define(utils_wxGenericFindWindowAtPoint, 2911). +-define(utils_wxFindWindowAtPoint, 2912). +-define(utils_wxBeginBusyCursor, 2913). +-define(utils_wxEndBusyCursor, 2914). +-define(utils_wxIsBusy, 2915). +-define(utils_wxShutdown, 2916). +-define(utils_wxShell, 2917). +-define(utils_wxLaunchDefaultBrowser, 2918). +-define(utils_wxGetEmailAddress, 2919). +-define(utils_wxGetUserId, 2920). +-define(utils_wxGetHomeDir, 2921). +-define(utils_wxNewId, 2922). +-define(utils_wxRegisterId, 2923). +-define(utils_wxGetCurrentId, 2924). +-define(utils_wxGetOsDescription, 2925). +-define(utils_wxIsPlatformLittleEndian, 2926). +-define(utils_wxIsPlatform64Bit, 2927). +-define(gdicmn_wxDisplaySize, 2928). +-define(gdicmn_wxSetCursor, 2929). +-define(wxPrintout_new, 2930). +-define(wxPrintout_destruct, 2931). +-define(wxPrintout_GetDC, 2932). +-define(wxPrintout_GetPageSizeMM, 2933). +-define(wxPrintout_GetPageSizePixels, 2934). +-define(wxPrintout_GetPaperRectPixels, 2935). +-define(wxPrintout_GetPPIPrinter, 2936). +-define(wxPrintout_GetPPIScreen, 2937). +-define(wxPrintout_GetTitle, 2938). +-define(wxPrintout_IsPreview, 2939). +-define(wxPrintout_FitThisSizeToPaper, 2940). +-define(wxPrintout_FitThisSizeToPage, 2941). +-define(wxPrintout_FitThisSizeToPageMargins, 2942). +-define(wxPrintout_MapScreenSizeToPaper, 2943). +-define(wxPrintout_MapScreenSizeToPage, 2944). +-define(wxPrintout_MapScreenSizeToPageMargins, 2945). +-define(wxPrintout_MapScreenSizeToDevice, 2946). +-define(wxPrintout_GetLogicalPaperRect, 2947). +-define(wxPrintout_GetLogicalPageRect, 2948). +-define(wxPrintout_GetLogicalPageMarginsRect, 2949). +-define(wxPrintout_SetLogicalOrigin, 2950). +-define(wxPrintout_OffsetLogicalOrigin, 2951). +-define(wxStyledTextCtrl_new_2, 2952). +-define(wxStyledTextCtrl_new_0, 2953). +-define(wxStyledTextCtrl_destruct, 2954). +-define(wxStyledTextCtrl_Create, 2955). +-define(wxStyledTextCtrl_AddText, 2956). +-define(wxStyledTextCtrl_AddStyledText, 2957). +-define(wxStyledTextCtrl_InsertText, 2958). +-define(wxStyledTextCtrl_ClearAll, 2959). +-define(wxStyledTextCtrl_ClearDocumentStyle, 2960). +-define(wxStyledTextCtrl_GetLength, 2961). +-define(wxStyledTextCtrl_GetCharAt, 2962). +-define(wxStyledTextCtrl_GetCurrentPos, 2963). +-define(wxStyledTextCtrl_GetAnchor, 2964). +-define(wxStyledTextCtrl_GetStyleAt, 2965). +-define(wxStyledTextCtrl_Redo, 2966). +-define(wxStyledTextCtrl_SetUndoCollection, 2967). +-define(wxStyledTextCtrl_SelectAll, 2968). +-define(wxStyledTextCtrl_SetSavePoint, 2969). +-define(wxStyledTextCtrl_GetStyledText, 2970). +-define(wxStyledTextCtrl_CanRedo, 2971). +-define(wxStyledTextCtrl_MarkerLineFromHandle, 2972). +-define(wxStyledTextCtrl_MarkerDeleteHandle, 2973). +-define(wxStyledTextCtrl_GetUndoCollection, 2974). +-define(wxStyledTextCtrl_GetViewWhiteSpace, 2975). +-define(wxStyledTextCtrl_SetViewWhiteSpace, 2976). +-define(wxStyledTextCtrl_PositionFromPoint, 2977). +-define(wxStyledTextCtrl_PositionFromPointClose, 2978). +-define(wxStyledTextCtrl_GotoLine, 2979). +-define(wxStyledTextCtrl_GotoPos, 2980). +-define(wxStyledTextCtrl_SetAnchor, 2981). +-define(wxStyledTextCtrl_GetCurLine, 2982). +-define(wxStyledTextCtrl_GetEndStyled, 2983). +-define(wxStyledTextCtrl_ConvertEOLs, 2984). +-define(wxStyledTextCtrl_GetEOLMode, 2985). +-define(wxStyledTextCtrl_SetEOLMode, 2986). +-define(wxStyledTextCtrl_StartStyling, 2987). +-define(wxStyledTextCtrl_SetStyling, 2988). +-define(wxStyledTextCtrl_GetBufferedDraw, 2989). +-define(wxStyledTextCtrl_SetBufferedDraw, 2990). +-define(wxStyledTextCtrl_SetTabWidth, 2991). +-define(wxStyledTextCtrl_GetTabWidth, 2992). +-define(wxStyledTextCtrl_SetCodePage, 2993). +-define(wxStyledTextCtrl_MarkerDefine, 2994). +-define(wxStyledTextCtrl_MarkerSetForeground, 2995). +-define(wxStyledTextCtrl_MarkerSetBackground, 2996). +-define(wxStyledTextCtrl_MarkerAdd, 2997). +-define(wxStyledTextCtrl_MarkerDelete, 2998). +-define(wxStyledTextCtrl_MarkerDeleteAll, 2999). +-define(wxStyledTextCtrl_MarkerGet, 3000). +-define(wxStyledTextCtrl_MarkerNext, 3001). +-define(wxStyledTextCtrl_MarkerPrevious, 3002). +-define(wxStyledTextCtrl_MarkerDefineBitmap, 3003). +-define(wxStyledTextCtrl_MarkerAddSet, 3004). +-define(wxStyledTextCtrl_MarkerSetAlpha, 3005). +-define(wxStyledTextCtrl_SetMarginType, 3006). +-define(wxStyledTextCtrl_GetMarginType, 3007). +-define(wxStyledTextCtrl_SetMarginWidth, 3008). +-define(wxStyledTextCtrl_GetMarginWidth, 3009). +-define(wxStyledTextCtrl_SetMarginMask, 3010). +-define(wxStyledTextCtrl_GetMarginMask, 3011). +-define(wxStyledTextCtrl_SetMarginSensitive, 3012). +-define(wxStyledTextCtrl_GetMarginSensitive, 3013). +-define(wxStyledTextCtrl_StyleClearAll, 3014). +-define(wxStyledTextCtrl_StyleSetForeground, 3015). +-define(wxStyledTextCtrl_StyleSetBackground, 3016). +-define(wxStyledTextCtrl_StyleSetBold, 3017). +-define(wxStyledTextCtrl_StyleSetItalic, 3018). +-define(wxStyledTextCtrl_StyleSetSize, 3019). +-define(wxStyledTextCtrl_StyleSetFaceName, 3020). +-define(wxStyledTextCtrl_StyleSetEOLFilled, 3021). +-define(wxStyledTextCtrl_StyleResetDefault, 3022). +-define(wxStyledTextCtrl_StyleSetUnderline, 3023). +-define(wxStyledTextCtrl_StyleSetCase, 3024). +-define(wxStyledTextCtrl_StyleSetHotSpot, 3025). +-define(wxStyledTextCtrl_SetSelForeground, 3026). +-define(wxStyledTextCtrl_SetSelBackground, 3027). +-define(wxStyledTextCtrl_GetSelAlpha, 3028). +-define(wxStyledTextCtrl_SetSelAlpha, 3029). +-define(wxStyledTextCtrl_SetCaretForeground, 3030). +-define(wxStyledTextCtrl_CmdKeyAssign, 3031). +-define(wxStyledTextCtrl_CmdKeyClear, 3032). +-define(wxStyledTextCtrl_CmdKeyClearAll, 3033). +-define(wxStyledTextCtrl_SetStyleBytes, 3034). +-define(wxStyledTextCtrl_StyleSetVisible, 3035). +-define(wxStyledTextCtrl_GetCaretPeriod, 3036). +-define(wxStyledTextCtrl_SetCaretPeriod, 3037). +-define(wxStyledTextCtrl_SetWordChars, 3038). +-define(wxStyledTextCtrl_BeginUndoAction, 3039). +-define(wxStyledTextCtrl_EndUndoAction, 3040). +-define(wxStyledTextCtrl_IndicatorSetStyle, 3041). +-define(wxStyledTextCtrl_IndicatorGetStyle, 3042). +-define(wxStyledTextCtrl_IndicatorSetForeground, 3043). +-define(wxStyledTextCtrl_IndicatorGetForeground, 3044). +-define(wxStyledTextCtrl_SetWhitespaceForeground, 3045). +-define(wxStyledTextCtrl_SetWhitespaceBackground, 3046). +-define(wxStyledTextCtrl_GetStyleBits, 3047). +-define(wxStyledTextCtrl_SetLineState, 3048). +-define(wxStyledTextCtrl_GetLineState, 3049). +-define(wxStyledTextCtrl_GetMaxLineState, 3050). +-define(wxStyledTextCtrl_GetCaretLineVisible, 3051). +-define(wxStyledTextCtrl_SetCaretLineVisible, 3052). +-define(wxStyledTextCtrl_GetCaretLineBackground, 3053). +-define(wxStyledTextCtrl_SetCaretLineBackground, 3054). +-define(wxStyledTextCtrl_AutoCompShow, 3055). +-define(wxStyledTextCtrl_AutoCompCancel, 3056). +-define(wxStyledTextCtrl_AutoCompActive, 3057). +-define(wxStyledTextCtrl_AutoCompPosStart, 3058). +-define(wxStyledTextCtrl_AutoCompComplete, 3059). +-define(wxStyledTextCtrl_AutoCompStops, 3060). +-define(wxStyledTextCtrl_AutoCompSetSeparator, 3061). +-define(wxStyledTextCtrl_AutoCompGetSeparator, 3062). +-define(wxStyledTextCtrl_AutoCompSelect, 3063). +-define(wxStyledTextCtrl_AutoCompSetCancelAtStart, 3064). +-define(wxStyledTextCtrl_AutoCompGetCancelAtStart, 3065). +-define(wxStyledTextCtrl_AutoCompSetFillUps, 3066). +-define(wxStyledTextCtrl_AutoCompSetChooseSingle, 3067). +-define(wxStyledTextCtrl_AutoCompGetChooseSingle, 3068). +-define(wxStyledTextCtrl_AutoCompSetIgnoreCase, 3069). +-define(wxStyledTextCtrl_AutoCompGetIgnoreCase, 3070). +-define(wxStyledTextCtrl_UserListShow, 3071). +-define(wxStyledTextCtrl_AutoCompSetAutoHide, 3072). +-define(wxStyledTextCtrl_AutoCompGetAutoHide, 3073). +-define(wxStyledTextCtrl_AutoCompSetDropRestOfWord, 3074). +-define(wxStyledTextCtrl_AutoCompGetDropRestOfWord, 3075). +-define(wxStyledTextCtrl_RegisterImage, 3076). +-define(wxStyledTextCtrl_ClearRegisteredImages, 3077). +-define(wxStyledTextCtrl_AutoCompGetTypeSeparator, 3078). +-define(wxStyledTextCtrl_AutoCompSetTypeSeparator, 3079). +-define(wxStyledTextCtrl_AutoCompSetMaxWidth, 3080). +-define(wxStyledTextCtrl_AutoCompGetMaxWidth, 3081). +-define(wxStyledTextCtrl_AutoCompSetMaxHeight, 3082). +-define(wxStyledTextCtrl_AutoCompGetMaxHeight, 3083). +-define(wxStyledTextCtrl_SetIndent, 3084). +-define(wxStyledTextCtrl_GetIndent, 3085). +-define(wxStyledTextCtrl_SetUseTabs, 3086). +-define(wxStyledTextCtrl_GetUseTabs, 3087). +-define(wxStyledTextCtrl_SetLineIndentation, 3088). +-define(wxStyledTextCtrl_GetLineIndentation, 3089). +-define(wxStyledTextCtrl_GetLineIndentPosition, 3090). +-define(wxStyledTextCtrl_GetColumn, 3091). +-define(wxStyledTextCtrl_SetUseHorizontalScrollBar, 3092). +-define(wxStyledTextCtrl_GetUseHorizontalScrollBar, 3093). +-define(wxStyledTextCtrl_SetIndentationGuides, 3094). +-define(wxStyledTextCtrl_GetIndentationGuides, 3095). +-define(wxStyledTextCtrl_SetHighlightGuide, 3096). +-define(wxStyledTextCtrl_GetHighlightGuide, 3097). +-define(wxStyledTextCtrl_GetLineEndPosition, 3098). +-define(wxStyledTextCtrl_GetCodePage, 3099). +-define(wxStyledTextCtrl_GetCaretForeground, 3100). +-define(wxStyledTextCtrl_GetReadOnly, 3101). +-define(wxStyledTextCtrl_SetCurrentPos, 3102). +-define(wxStyledTextCtrl_SetSelectionStart, 3103). +-define(wxStyledTextCtrl_GetSelectionStart, 3104). +-define(wxStyledTextCtrl_SetSelectionEnd, 3105). +-define(wxStyledTextCtrl_GetSelectionEnd, 3106). +-define(wxStyledTextCtrl_SetPrintMagnification, 3107). +-define(wxStyledTextCtrl_GetPrintMagnification, 3108). +-define(wxStyledTextCtrl_SetPrintColourMode, 3109). +-define(wxStyledTextCtrl_GetPrintColourMode, 3110). +-define(wxStyledTextCtrl_FindText, 3111). +-define(wxStyledTextCtrl_FormatRange, 3112). +-define(wxStyledTextCtrl_GetFirstVisibleLine, 3113). +-define(wxStyledTextCtrl_GetLine, 3114). +-define(wxStyledTextCtrl_GetLineCount, 3115). +-define(wxStyledTextCtrl_SetMarginLeft, 3116). +-define(wxStyledTextCtrl_GetMarginLeft, 3117). +-define(wxStyledTextCtrl_SetMarginRight, 3118). +-define(wxStyledTextCtrl_GetMarginRight, 3119). +-define(wxStyledTextCtrl_GetModify, 3120). +-define(wxStyledTextCtrl_SetSelection, 3121). +-define(wxStyledTextCtrl_GetSelectedText, 3122). +-define(wxStyledTextCtrl_GetTextRange, 3123). +-define(wxStyledTextCtrl_HideSelection, 3124). +-define(wxStyledTextCtrl_LineFromPosition, 3125). +-define(wxStyledTextCtrl_PositionFromLine, 3126). +-define(wxStyledTextCtrl_LineScroll, 3127). +-define(wxStyledTextCtrl_EnsureCaretVisible, 3128). +-define(wxStyledTextCtrl_ReplaceSelection, 3129). +-define(wxStyledTextCtrl_SetReadOnly, 3130). +-define(wxStyledTextCtrl_CanPaste, 3131). +-define(wxStyledTextCtrl_CanUndo, 3132). +-define(wxStyledTextCtrl_EmptyUndoBuffer, 3133). +-define(wxStyledTextCtrl_Undo, 3134). +-define(wxStyledTextCtrl_Cut, 3135). +-define(wxStyledTextCtrl_Copy, 3136). +-define(wxStyledTextCtrl_Paste, 3137). +-define(wxStyledTextCtrl_Clear, 3138). +-define(wxStyledTextCtrl_SetText, 3139). +-define(wxStyledTextCtrl_GetText, 3140). +-define(wxStyledTextCtrl_GetTextLength, 3141). +-define(wxStyledTextCtrl_GetOvertype, 3142). +-define(wxStyledTextCtrl_SetCaretWidth, 3143). +-define(wxStyledTextCtrl_GetCaretWidth, 3144). +-define(wxStyledTextCtrl_SetTargetStart, 3145). +-define(wxStyledTextCtrl_GetTargetStart, 3146). +-define(wxStyledTextCtrl_SetTargetEnd, 3147). +-define(wxStyledTextCtrl_GetTargetEnd, 3148). +-define(wxStyledTextCtrl_ReplaceTarget, 3149). +-define(wxStyledTextCtrl_SearchInTarget, 3150). +-define(wxStyledTextCtrl_SetSearchFlags, 3151). +-define(wxStyledTextCtrl_GetSearchFlags, 3152). +-define(wxStyledTextCtrl_CallTipShow, 3153). +-define(wxStyledTextCtrl_CallTipCancel, 3154). +-define(wxStyledTextCtrl_CallTipActive, 3155). +-define(wxStyledTextCtrl_CallTipPosAtStart, 3156). +-define(wxStyledTextCtrl_CallTipSetHighlight, 3157). +-define(wxStyledTextCtrl_CallTipSetBackground, 3158). +-define(wxStyledTextCtrl_CallTipSetForeground, 3159). +-define(wxStyledTextCtrl_CallTipSetForegroundHighlight, 3160). +-define(wxStyledTextCtrl_CallTipUseStyle, 3161). +-define(wxStyledTextCtrl_VisibleFromDocLine, 3162). +-define(wxStyledTextCtrl_DocLineFromVisible, 3163). +-define(wxStyledTextCtrl_WrapCount, 3164). +-define(wxStyledTextCtrl_SetFoldLevel, 3165). +-define(wxStyledTextCtrl_GetFoldLevel, 3166). +-define(wxStyledTextCtrl_GetLastChild, 3167). +-define(wxStyledTextCtrl_GetFoldParent, 3168). +-define(wxStyledTextCtrl_ShowLines, 3169). +-define(wxStyledTextCtrl_HideLines, 3170). +-define(wxStyledTextCtrl_GetLineVisible, 3171). +-define(wxStyledTextCtrl_SetFoldExpanded, 3172). +-define(wxStyledTextCtrl_GetFoldExpanded, 3173). +-define(wxStyledTextCtrl_ToggleFold, 3174). +-define(wxStyledTextCtrl_EnsureVisible, 3175). +-define(wxStyledTextCtrl_SetFoldFlags, 3176). +-define(wxStyledTextCtrl_EnsureVisibleEnforcePolicy, 3177). +-define(wxStyledTextCtrl_SetTabIndents, 3178). +-define(wxStyledTextCtrl_GetTabIndents, 3179). +-define(wxStyledTextCtrl_SetBackSpaceUnIndents, 3180). +-define(wxStyledTextCtrl_GetBackSpaceUnIndents, 3181). +-define(wxStyledTextCtrl_SetMouseDwellTime, 3182). +-define(wxStyledTextCtrl_GetMouseDwellTime, 3183). +-define(wxStyledTextCtrl_WordStartPosition, 3184). +-define(wxStyledTextCtrl_WordEndPosition, 3185). +-define(wxStyledTextCtrl_SetWrapMode, 3186). +-define(wxStyledTextCtrl_GetWrapMode, 3187). +-define(wxStyledTextCtrl_SetWrapVisualFlags, 3188). +-define(wxStyledTextCtrl_GetWrapVisualFlags, 3189). +-define(wxStyledTextCtrl_SetWrapVisualFlagsLocation, 3190). +-define(wxStyledTextCtrl_GetWrapVisualFlagsLocation, 3191). +-define(wxStyledTextCtrl_SetWrapStartIndent, 3192). +-define(wxStyledTextCtrl_GetWrapStartIndent, 3193). +-define(wxStyledTextCtrl_SetLayoutCache, 3194). +-define(wxStyledTextCtrl_GetLayoutCache, 3195). +-define(wxStyledTextCtrl_SetScrollWidth, 3196). +-define(wxStyledTextCtrl_GetScrollWidth, 3197). +-define(wxStyledTextCtrl_TextWidth, 3198). +-define(wxStyledTextCtrl_GetEndAtLastLine, 3199). +-define(wxStyledTextCtrl_TextHeight, 3200). +-define(wxStyledTextCtrl_SetUseVerticalScrollBar, 3201). +-define(wxStyledTextCtrl_GetUseVerticalScrollBar, 3202). +-define(wxStyledTextCtrl_AppendText, 3203). +-define(wxStyledTextCtrl_GetTwoPhaseDraw, 3204). +-define(wxStyledTextCtrl_SetTwoPhaseDraw, 3205). +-define(wxStyledTextCtrl_TargetFromSelection, 3206). +-define(wxStyledTextCtrl_LinesJoin, 3207). +-define(wxStyledTextCtrl_LinesSplit, 3208). +-define(wxStyledTextCtrl_SetFoldMarginColour, 3209). +-define(wxStyledTextCtrl_SetFoldMarginHiColour, 3210). +-define(wxStyledTextCtrl_LineDown, 3211). +-define(wxStyledTextCtrl_LineDownExtend, 3212). +-define(wxStyledTextCtrl_LineUp, 3213). +-define(wxStyledTextCtrl_LineUpExtend, 3214). +-define(wxStyledTextCtrl_CharLeft, 3215). +-define(wxStyledTextCtrl_CharLeftExtend, 3216). +-define(wxStyledTextCtrl_CharRight, 3217). +-define(wxStyledTextCtrl_CharRightExtend, 3218). +-define(wxStyledTextCtrl_WordLeft, 3219). +-define(wxStyledTextCtrl_WordLeftExtend, 3220). +-define(wxStyledTextCtrl_WordRight, 3221). +-define(wxStyledTextCtrl_WordRightExtend, 3222). +-define(wxStyledTextCtrl_Home, 3223). +-define(wxStyledTextCtrl_HomeExtend, 3224). +-define(wxStyledTextCtrl_LineEnd, 3225). +-define(wxStyledTextCtrl_LineEndExtend, 3226). +-define(wxStyledTextCtrl_DocumentStart, 3227). +-define(wxStyledTextCtrl_DocumentStartExtend, 3228). +-define(wxStyledTextCtrl_DocumentEnd, 3229). +-define(wxStyledTextCtrl_DocumentEndExtend, 3230). +-define(wxStyledTextCtrl_PageUp, 3231). +-define(wxStyledTextCtrl_PageUpExtend, 3232). +-define(wxStyledTextCtrl_PageDown, 3233). +-define(wxStyledTextCtrl_PageDownExtend, 3234). +-define(wxStyledTextCtrl_EditToggleOvertype, 3235). +-define(wxStyledTextCtrl_Cancel, 3236). +-define(wxStyledTextCtrl_DeleteBack, 3237). +-define(wxStyledTextCtrl_Tab, 3238). +-define(wxStyledTextCtrl_BackTab, 3239). +-define(wxStyledTextCtrl_NewLine, 3240). +-define(wxStyledTextCtrl_FormFeed, 3241). +-define(wxStyledTextCtrl_VCHome, 3242). +-define(wxStyledTextCtrl_VCHomeExtend, 3243). +-define(wxStyledTextCtrl_ZoomIn, 3244). +-define(wxStyledTextCtrl_ZoomOut, 3245). +-define(wxStyledTextCtrl_DelWordLeft, 3246). +-define(wxStyledTextCtrl_DelWordRight, 3247). +-define(wxStyledTextCtrl_LineCut, 3248). +-define(wxStyledTextCtrl_LineDelete, 3249). +-define(wxStyledTextCtrl_LineTranspose, 3250). +-define(wxStyledTextCtrl_LineDuplicate, 3251). +-define(wxStyledTextCtrl_LowerCase, 3252). +-define(wxStyledTextCtrl_UpperCase, 3253). +-define(wxStyledTextCtrl_LineScrollDown, 3254). +-define(wxStyledTextCtrl_LineScrollUp, 3255). +-define(wxStyledTextCtrl_DeleteBackNotLine, 3256). +-define(wxStyledTextCtrl_HomeDisplay, 3257). +-define(wxStyledTextCtrl_HomeDisplayExtend, 3258). +-define(wxStyledTextCtrl_LineEndDisplay, 3259). +-define(wxStyledTextCtrl_LineEndDisplayExtend, 3260). +-define(wxStyledTextCtrl_HomeWrapExtend, 3261). +-define(wxStyledTextCtrl_LineEndWrap, 3262). +-define(wxStyledTextCtrl_LineEndWrapExtend, 3263). +-define(wxStyledTextCtrl_VCHomeWrap, 3264). +-define(wxStyledTextCtrl_VCHomeWrapExtend, 3265). +-define(wxStyledTextCtrl_LineCopy, 3266). +-define(wxStyledTextCtrl_MoveCaretInsideView, 3267). +-define(wxStyledTextCtrl_LineLength, 3268). +-define(wxStyledTextCtrl_BraceHighlight, 3269). +-define(wxStyledTextCtrl_BraceBadLight, 3270). +-define(wxStyledTextCtrl_BraceMatch, 3271). +-define(wxStyledTextCtrl_GetViewEOL, 3272). +-define(wxStyledTextCtrl_SetViewEOL, 3273). +-define(wxStyledTextCtrl_SetModEventMask, 3274). +-define(wxStyledTextCtrl_GetEdgeColumn, 3275). +-define(wxStyledTextCtrl_SetEdgeColumn, 3276). +-define(wxStyledTextCtrl_SetEdgeMode, 3277). +-define(wxStyledTextCtrl_GetEdgeMode, 3278). +-define(wxStyledTextCtrl_GetEdgeColour, 3279). +-define(wxStyledTextCtrl_SetEdgeColour, 3280). +-define(wxStyledTextCtrl_SearchAnchor, 3281). +-define(wxStyledTextCtrl_SearchNext, 3282). +-define(wxStyledTextCtrl_SearchPrev, 3283). +-define(wxStyledTextCtrl_LinesOnScreen, 3284). +-define(wxStyledTextCtrl_UsePopUp, 3285). +-define(wxStyledTextCtrl_SelectionIsRectangle, 3286). +-define(wxStyledTextCtrl_SetZoom, 3287). +-define(wxStyledTextCtrl_GetZoom, 3288). +-define(wxStyledTextCtrl_GetModEventMask, 3289). +-define(wxStyledTextCtrl_SetSTCFocus, 3290). +-define(wxStyledTextCtrl_GetSTCFocus, 3291). +-define(wxStyledTextCtrl_SetStatus, 3292). +-define(wxStyledTextCtrl_GetStatus, 3293). +-define(wxStyledTextCtrl_SetMouseDownCaptures, 3294). +-define(wxStyledTextCtrl_GetMouseDownCaptures, 3295). +-define(wxStyledTextCtrl_SetSTCCursor, 3296). +-define(wxStyledTextCtrl_GetSTCCursor, 3297). +-define(wxStyledTextCtrl_SetControlCharSymbol, 3298). +-define(wxStyledTextCtrl_GetControlCharSymbol, 3299). +-define(wxStyledTextCtrl_WordPartLeft, 3300). +-define(wxStyledTextCtrl_WordPartLeftExtend, 3301). +-define(wxStyledTextCtrl_WordPartRight, 3302). +-define(wxStyledTextCtrl_WordPartRightExtend, 3303). +-define(wxStyledTextCtrl_SetVisiblePolicy, 3304). +-define(wxStyledTextCtrl_DelLineLeft, 3305). +-define(wxStyledTextCtrl_DelLineRight, 3306). +-define(wxStyledTextCtrl_GetXOffset, 3307). +-define(wxStyledTextCtrl_ChooseCaretX, 3308). +-define(wxStyledTextCtrl_SetXCaretPolicy, 3309). +-define(wxStyledTextCtrl_SetYCaretPolicy, 3310). +-define(wxStyledTextCtrl_GetPrintWrapMode, 3311). +-define(wxStyledTextCtrl_SetHotspotActiveForeground, 3312). +-define(wxStyledTextCtrl_SetHotspotActiveBackground, 3313). +-define(wxStyledTextCtrl_SetHotspotActiveUnderline, 3314). +-define(wxStyledTextCtrl_SetHotspotSingleLine, 3315). +-define(wxStyledTextCtrl_ParaDownExtend, 3316). +-define(wxStyledTextCtrl_ParaUp, 3317). +-define(wxStyledTextCtrl_ParaUpExtend, 3318). +-define(wxStyledTextCtrl_PositionBefore, 3319). +-define(wxStyledTextCtrl_PositionAfter, 3320). +-define(wxStyledTextCtrl_CopyRange, 3321). +-define(wxStyledTextCtrl_CopyText, 3322). +-define(wxStyledTextCtrl_SetSelectionMode, 3323). +-define(wxStyledTextCtrl_GetSelectionMode, 3324). +-define(wxStyledTextCtrl_LineDownRectExtend, 3325). +-define(wxStyledTextCtrl_LineUpRectExtend, 3326). +-define(wxStyledTextCtrl_CharLeftRectExtend, 3327). +-define(wxStyledTextCtrl_CharRightRectExtend, 3328). +-define(wxStyledTextCtrl_HomeRectExtend, 3329). +-define(wxStyledTextCtrl_VCHomeRectExtend, 3330). +-define(wxStyledTextCtrl_LineEndRectExtend, 3331). +-define(wxStyledTextCtrl_PageUpRectExtend, 3332). +-define(wxStyledTextCtrl_PageDownRectExtend, 3333). +-define(wxStyledTextCtrl_StutteredPageUp, 3334). +-define(wxStyledTextCtrl_StutteredPageUpExtend, 3335). +-define(wxStyledTextCtrl_StutteredPageDown, 3336). +-define(wxStyledTextCtrl_StutteredPageDownExtend, 3337). +-define(wxStyledTextCtrl_WordLeftEnd, 3338). +-define(wxStyledTextCtrl_WordLeftEndExtend, 3339). +-define(wxStyledTextCtrl_WordRightEnd, 3340). +-define(wxStyledTextCtrl_WordRightEndExtend, 3341). +-define(wxStyledTextCtrl_SetWhitespaceChars, 3342). +-define(wxStyledTextCtrl_SetCharsDefault, 3343). +-define(wxStyledTextCtrl_AutoCompGetCurrent, 3344). +-define(wxStyledTextCtrl_Allocate, 3345). +-define(wxStyledTextCtrl_FindColumn, 3346). +-define(wxStyledTextCtrl_GetCaretSticky, 3347). +-define(wxStyledTextCtrl_SetCaretSticky, 3348). +-define(wxStyledTextCtrl_ToggleCaretSticky, 3349). +-define(wxStyledTextCtrl_SetPasteConvertEndings, 3350). +-define(wxStyledTextCtrl_GetPasteConvertEndings, 3351). +-define(wxStyledTextCtrl_SelectionDuplicate, 3352). +-define(wxStyledTextCtrl_SetCaretLineBackAlpha, 3353). +-define(wxStyledTextCtrl_GetCaretLineBackAlpha, 3354). +-define(wxStyledTextCtrl_StartRecord, 3355). +-define(wxStyledTextCtrl_StopRecord, 3356). +-define(wxStyledTextCtrl_SetLexer, 3357). +-define(wxStyledTextCtrl_GetLexer, 3358). +-define(wxStyledTextCtrl_Colourise, 3359). +-define(wxStyledTextCtrl_SetProperty, 3360). +-define(wxStyledTextCtrl_SetKeyWords, 3361). +-define(wxStyledTextCtrl_SetLexerLanguage, 3362). +-define(wxStyledTextCtrl_GetProperty, 3363). +-define(wxStyledTextCtrl_GetStyleBitsNeeded, 3364). +-define(wxStyledTextCtrl_GetCurrentLine, 3365). +-define(wxStyledTextCtrl_StyleSetSpec, 3366). +-define(wxStyledTextCtrl_StyleSetFont, 3367). +-define(wxStyledTextCtrl_StyleSetFontAttr, 3368). +-define(wxStyledTextCtrl_StyleSetCharacterSet, 3369). +-define(wxStyledTextCtrl_StyleSetFontEncoding, 3370). +-define(wxStyledTextCtrl_CmdKeyExecute, 3371). +-define(wxStyledTextCtrl_SetMargins, 3372). +-define(wxStyledTextCtrl_GetSelection, 3373). +-define(wxStyledTextCtrl_PointFromPosition, 3374). +-define(wxStyledTextCtrl_ScrollToLine, 3375). +-define(wxStyledTextCtrl_ScrollToColumn, 3376). +-define(wxStyledTextCtrl_SetVScrollBar, 3377). +-define(wxStyledTextCtrl_SetHScrollBar, 3378). +-define(wxStyledTextCtrl_GetLastKeydownProcessed, 3379). +-define(wxStyledTextCtrl_SetLastKeydownProcessed, 3380). +-define(wxStyledTextCtrl_SaveFile, 3381). +-define(wxStyledTextCtrl_LoadFile, 3382). +-define(wxStyledTextCtrl_DoDragOver, 3383). +-define(wxStyledTextCtrl_DoDropText, 3384). +-define(wxStyledTextCtrl_GetUseAntiAliasing, 3385). +-define(wxStyledTextCtrl_AddTextRaw, 3386). +-define(wxStyledTextCtrl_InsertTextRaw, 3387). +-define(wxStyledTextCtrl_GetCurLineRaw, 3388). +-define(wxStyledTextCtrl_GetLineRaw, 3389). +-define(wxStyledTextCtrl_GetSelectedTextRaw, 3390). +-define(wxStyledTextCtrl_GetTextRangeRaw, 3391). +-define(wxStyledTextCtrl_SetTextRaw, 3392). +-define(wxStyledTextCtrl_GetTextRaw, 3393). +-define(wxStyledTextCtrl_AppendTextRaw, 3394). +-define(wxArtProvider_GetBitmap, 3395). +-define(wxArtProvider_GetIcon, 3396). +-define(wxTreeEvent_GetKeyCode, 3397). +-define(wxTreeEvent_GetItem, 3398). +-define(wxTreeEvent_GetKeyEvent, 3399). +-define(wxTreeEvent_GetLabel, 3400). +-define(wxTreeEvent_GetOldItem, 3401). +-define(wxTreeEvent_GetPoint, 3402). +-define(wxTreeEvent_IsEditCancelled, 3403). +-define(wxTreeEvent_SetToolTip, 3404). +-define(wxNotebookEvent_GetOldSelection, 3405). +-define(wxNotebookEvent_GetSelection, 3406). +-define(wxNotebookEvent_SetOldSelection, 3407). +-define(wxNotebookEvent_SetSelection, 3408). +-define(wxFileDataObject_new, 3409). +-define(wxFileDataObject_AddFile, 3410). +-define(wxFileDataObject_GetFilenames, 3411). +-define(wxFileDataObject_destroy, 3412). +-define(wxTextDataObject_new, 3413). +-define(wxTextDataObject_GetTextLength, 3414). +-define(wxTextDataObject_GetText, 3415). +-define(wxTextDataObject_SetText, 3416). +-define(wxTextDataObject_destroy, 3417). +-define(wxBitmapDataObject_new_1_1, 3418). +-define(wxBitmapDataObject_new_1_0, 3419). +-define(wxBitmapDataObject_GetBitmap, 3420). +-define(wxBitmapDataObject_SetBitmap, 3421). +-define(wxBitmapDataObject_destroy, 3422). +-define(wxClipboard_new, 3424). +-define(wxClipboard_destruct, 3425). +-define(wxClipboard_AddData, 3426). +-define(wxClipboard_Clear, 3427). +-define(wxClipboard_Close, 3428). +-define(wxClipboard_Flush, 3429). +-define(wxClipboard_GetData, 3430). +-define(wxClipboard_IsOpened, 3431). +-define(wxClipboard_Open, 3432). +-define(wxClipboard_SetData, 3433). +-define(wxClipboard_UsePrimarySelection, 3435). +-define(wxClipboard_IsSupported, 3436). +-define(wxClipboard_Get, 3437). +-define(wxSpinEvent_GetPosition, 3438). +-define(wxSpinEvent_SetPosition, 3439). +-define(wxSplitterWindow_new_0, 3440). +-define(wxSplitterWindow_new_2, 3441). +-define(wxSplitterWindow_destruct, 3442). +-define(wxSplitterWindow_Create, 3443). +-define(wxSplitterWindow_GetMinimumPaneSize, 3444). +-define(wxSplitterWindow_GetSashGravity, 3445). +-define(wxSplitterWindow_GetSashPosition, 3446). +-define(wxSplitterWindow_GetSplitMode, 3447). +-define(wxSplitterWindow_GetWindow1, 3448). +-define(wxSplitterWindow_GetWindow2, 3449). +-define(wxSplitterWindow_Initialize, 3450). +-define(wxSplitterWindow_IsSplit, 3451). +-define(wxSplitterWindow_ReplaceWindow, 3452). +-define(wxSplitterWindow_SetSashGravity, 3453). +-define(wxSplitterWindow_SetSashPosition, 3454). +-define(wxSplitterWindow_SetSashSize, 3455). +-define(wxSplitterWindow_SetMinimumPaneSize, 3456). +-define(wxSplitterWindow_SetSplitMode, 3457). +-define(wxSplitterWindow_SplitHorizontally, 3458). +-define(wxSplitterWindow_SplitVertically, 3459). +-define(wxSplitterWindow_Unsplit, 3460). +-define(wxSplitterWindow_UpdateSize, 3461). +-define(wxSplitterEvent_GetSashPosition, 3462). +-define(wxSplitterEvent_GetX, 3463). +-define(wxSplitterEvent_GetY, 3464). +-define(wxSplitterEvent_GetWindowBeingRemoved, 3465). +-define(wxSplitterEvent_SetSashPosition, 3466). +-define(wxHtmlWindow_new_0, 3467). +-define(wxHtmlWindow_new_2, 3468). +-define(wxHtmlWindow_AppendToPage, 3469). +-define(wxHtmlWindow_GetOpenedAnchor, 3470). +-define(wxHtmlWindow_GetOpenedPage, 3471). +-define(wxHtmlWindow_GetOpenedPageTitle, 3472). +-define(wxHtmlWindow_GetRelatedFrame, 3473). +-define(wxHtmlWindow_HistoryBack, 3474). +-define(wxHtmlWindow_HistoryCanBack, 3475). +-define(wxHtmlWindow_HistoryCanForward, 3476). +-define(wxHtmlWindow_HistoryClear, 3477). +-define(wxHtmlWindow_HistoryForward, 3478). +-define(wxHtmlWindow_LoadFile, 3479). +-define(wxHtmlWindow_LoadPage, 3480). +-define(wxHtmlWindow_SelectAll, 3481). +-define(wxHtmlWindow_SelectionToText, 3482). +-define(wxHtmlWindow_SelectLine, 3483). +-define(wxHtmlWindow_SelectWord, 3484). +-define(wxHtmlWindow_SetBorders, 3485). +-define(wxHtmlWindow_SetFonts, 3486). +-define(wxHtmlWindow_SetPage, 3487). +-define(wxHtmlWindow_SetRelatedFrame, 3488). +-define(wxHtmlWindow_SetRelatedStatusBar, 3489). +-define(wxHtmlWindow_ToText, 3490). +-define(wxHtmlWindow_destroy, 3491). +-define(wxHtmlLinkEvent_GetLinkInfo, 3492). +-define(wxSystemSettings_GetColour, 3493). +-define(wxSystemSettings_GetFont, 3494). +-define(wxSystemSettings_GetMetric, 3495). +-define(wxSystemSettings_GetScreenType, 3496). +-define(wxSystemOptions_GetOption, 3497). +-define(wxSystemOptions_GetOptionInt, 3498). +-define(wxSystemOptions_HasOption, 3499). +-define(wxSystemOptions_IsFalse, 3500). +-define(wxSystemOptions_SetOption_2_1, 3501). +-define(wxSystemOptions_SetOption_2_0, 3502). +-define(wxAuiNotebookEvent_SetSelection, 3503). +-define(wxAuiNotebookEvent_GetSelection, 3504). +-define(wxAuiNotebookEvent_SetOldSelection, 3505). +-define(wxAuiNotebookEvent_GetOldSelection, 3506). +-define(wxAuiNotebookEvent_SetDragSource, 3507). +-define(wxAuiNotebookEvent_GetDragSource, 3508). +-define(wxAuiManagerEvent_SetManager, 3509). +-define(wxAuiManagerEvent_GetManager, 3510). +-define(wxAuiManagerEvent_SetPane, 3511). +-define(wxAuiManagerEvent_GetPane, 3512). +-define(wxAuiManagerEvent_SetButton, 3513). +-define(wxAuiManagerEvent_GetButton, 3514). +-define(wxAuiManagerEvent_SetDC, 3515). +-define(wxAuiManagerEvent_GetDC, 3516). +-define(wxAuiManagerEvent_Veto, 3517). +-define(wxAuiManagerEvent_GetVeto, 3518). +-define(wxAuiManagerEvent_SetCanVeto, 3519). +-define(wxAuiManagerEvent_CanVeto, 3520). +-define(wxLogNull_new, 3521). +-define(wxLogNull_destroy, 3522). +-define(wxTaskBarIcon_new, 3523). +-define(wxTaskBarIcon_destruct, 3524). +-define(wxTaskBarIcon_PopupMenu, 3525). +-define(wxTaskBarIcon_RemoveIcon, 3526). +-define(wxTaskBarIcon_SetIcon, 3527). +-define(wxLocale_new_0, 3528). +-define(wxLocale_new_2, 3530). +-define(wxLocale_destruct, 3531). +-define(wxLocale_Init, 3533). +-define(wxLocale_AddCatalog_1, 3534). +-define(wxLocale_AddCatalog_3, 3535). +-define(wxLocale_AddCatalogLookupPathPrefix, 3536). +-define(wxLocale_GetCanonicalName, 3537). +-define(wxLocale_GetLanguage, 3538). +-define(wxLocale_GetLanguageName, 3539). +-define(wxLocale_GetLocale, 3540). +-define(wxLocale_GetName, 3541). +-define(wxLocale_GetString_2, 3542). +-define(wxLocale_GetString_4, 3543). +-define(wxLocale_GetHeaderValue, 3544). +-define(wxLocale_GetSysName, 3545). +-define(wxLocale_GetSystemEncoding, 3546). +-define(wxLocale_GetSystemEncodingName, 3547). +-define(wxLocale_GetSystemLanguage, 3548). +-define(wxLocale_IsLoaded, 3549). +-define(wxLocale_IsOk, 3550). +-define(wxActivateEvent_GetActive, 3551). +-define(wxPopupWindow_new_2, 3553). +-define(wxPopupWindow_new_0, 3554). +-define(wxPopupWindow_destruct, 3556). +-define(wxPopupWindow_Create, 3557). +-define(wxPopupWindow_Position, 3558). +-define(wxPopupTransientWindow_new_0, 3559). +-define(wxPopupTransientWindow_new_2, 3560). +-define(wxPopupTransientWindow_destruct, 3561). +-define(wxPopupTransientWindow_Popup, 3562). +-define(wxPopupTransientWindow_Dismiss, 3563). diff --git a/lib/wx/src/wxe_master.erl b/lib/wx/src/wxe_master.erl index b98a7c793e..4b8a8f5720 100644 --- a/lib/wx/src/wxe_master.erl +++ b/lib/wx/src/wxe_master.erl @@ -28,7 +28,7 @@ -behaviour(gen_server). %% API --export([start/1, init_port/1, init_opengl/0]). +-export([start/1, init_port/1, init_opengl/0, fetch_msgs/0]). %% gen_server callbacks -export([init/1, handle_call/3, handle_cast/2, handle_info/2, @@ -36,7 +36,9 @@ -record(state, {cb_port, %% Callback port and to erlang messages goes via it. users, %% List of wx servers, needed ?? - driver}). %% Driver name so wx_server can create it's own port + driver, %% Driver name so wx_server can create it's own port + msgs=[] %% Early messages (such as openfiles on OSX) + }). -include("wxe.hrl"). -include("gen/wxe_debug.hrl"). @@ -76,12 +78,18 @@ init_port(SilentStart) -> %%-------------------------------------------------------------------- -%% Initlizes the opengl library +%% Initalizes the opengl library %%-------------------------------------------------------------------- init_opengl() -> GLLib = wxe_util:wxgl_dl(), wxe_util:call(?WXE_INIT_OPENGL, <<(list_to_binary(GLLib))/binary, 0:8>>). +%%-------------------------------------------------------------------- +%% Fetch early messages, hack to get start up args on mac +%%-------------------------------------------------------------------- +fetch_msgs() -> + gen_server:call(?MODULE, fetch_msgs, infinity). + %%==================================================================== %% gen_server callbacks %%==================================================================== @@ -152,6 +160,8 @@ init([SilentStart]) -> %%-------------------------------------------------------------------- handle_call(init_port, From, State=#state{driver=Driver,cb_port=CBPort, users=Users}) -> {reply, {Driver,CBPort}, State#state{users=gb_sets:add(From,Users)}}; +handle_call(fetch_msgs, _From, State=#state{msgs=Msgs}) -> + {reply, lists:reverse(Msgs), State#state{msgs=[]}}; handle_call(_Request, _From, State) -> %%io:format("Unknown request ~p sent to ~p from ~p ~n",[_Request, ?MODULE, _From]), Reply = ok, @@ -182,6 +192,8 @@ handle_info({wxe_driver, internal_error, Msg}, State) -> handle_info({wxe_driver, debug, Msg}, State) -> io:format("WX DBG: ~s~n", [Msg]), {noreply, State}; +handle_info({wxe_driver, open_file, File}, State=#state{msgs=Msgs}) -> + {noreply, State#state{msgs=[File|Msgs]}}; handle_info(_Info, State) -> io:format("Unknown message ~p sent to ~p~n",[_Info, ?MODULE]), {noreply, State}. diff --git a/lib/wx/src/wxe_server.erl b/lib/wx/src/wxe_server.erl index aed9dca7ce..465b9da2e0 100644 --- a/lib/wx/src/wxe_server.erl +++ b/lib/wx/src/wxe_server.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2013. All Rights Reserved. +%% Copyright Ericsson AB 2008-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -192,17 +192,6 @@ handle_info({'DOWN',_,process,Pid,_}, State=#state{users=Users0,cleaners=Cs}) -> {noreply, State} end; -handle_info(Msg = {'_wxe_destroy_', Pid}, State) - when is_pid(Pid) -> - case erlang:is_process_alive(Pid) of - true -> - Pid ! Msg, - ok; - false -> - ok - end, - {noreply, State}; - handle_info(_Info, State) -> ?log("Unknown message ~p sent to ~p~n",[_Info, ?MODULE]), {noreply, State}. diff --git a/lib/wx/test/wx_basic_SUITE.erl b/lib/wx/test/wx_basic_SUITE.erl index 7bdbd4594c..e3bbb21a23 100644 --- a/lib/wx/test/wx_basic_SUITE.erl +++ b/lib/wx/test/wx_basic_SUITE.erl @@ -341,23 +341,22 @@ wx_object(Config) -> Me = self(), ?m({call, foobar, {Me, _}}, wx_object:call(Frame, foobar)), ?m(ok, wx_object:cast(Frame, foobar2)), - ?m([{cast, foobar2}], flush()), + ?m([{cast, foobar2}|_], flush()), FramePid = wx_object:get_pid(Frame), io:format("wx_object pid ~p~n",[FramePid]), FramePid ! foo3, - ?m([{info, foo3}], flush()), + ?m([{info, foo3}|_], flush()), ?m(ok, wx_object:cast(Frame, fun(_) -> hehe end)), - ?m([{cast, hehe}], flush()), + ?m([{cast, hehe}|_], flush()), wxWindow:refresh(Frame), - ?m([{sync_event, #wx{event=#wxPaint{}}, _}], flush()), + ?m([{sync_event, #wx{event=#wxPaint{}}, _}|_], flush()), ?m(ok, wx_object:cast(Frame, fun(_) -> timer:sleep(200), slept end)), %% The sleep above should not hinder the Paint event below %% Which it did in my buggy handling of the sync_callback wxWindow:refresh(Frame), - ?m([{sync_event, #wx{event=#wxPaint{}}, _}], flush()), timer:sleep(500), - ?m([{cast, slept}], flush()), + ?m([{sync_event, #wx{event=#wxPaint{}}, _}, {cast, slept}|_], flush()), Monitor = erlang:monitor(process, FramePid), case proplists:get_value(user, Config, false) of @@ -397,7 +396,7 @@ check_events([], Async, Sync) -> end. flush() -> - flush([], 500). + flush([], 1500). flush(Acc, Wait) -> receive diff --git a/lib/wx/test/wx_class_SUITE.erl b/lib/wx/test/wx_class_SUITE.erl index 0e151ccc9b..b127e6b71d 100644 --- a/lib/wx/test/wx_class_SUITE.erl +++ b/lib/wx/test/wx_class_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2013. All Rights Reserved. +%% Copyright Ericsson AB 2008-2014. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -50,7 +50,7 @@ suite() -> [{ct_hooks,[ts_install_cth]}]. all() -> [calendarCtrl, treeCtrl, notebook, staticBoxSizer, clipboard, helpFrame, htmlWindow, listCtrlSort, listCtrlVirtual, - radioBox, systemSettings, taskBarIcon, toolbar]. + radioBox, systemSettings, taskBarIcon, toolbar, popup]. groups() -> []. @@ -344,21 +344,21 @@ listCtrlSort(Config) -> Wx = wx:new(), Frame = wxFrame:new(Wx, ?wxID_ANY, "Frame"), - LC = wxListCtrl:new(Frame, [{style, ?wxLC_REPORT bor ?wxLC_SORT_ASCENDING}]), + LC = wxListCtrl:new(Frame, [{style, ?wxLC_REPORT}]), %% must be done crashes in wxwidgets otherwise. wxListCtrl:insertColumn(LC, 0, "Column"), Add = fun(Int) -> - wxListCtrl:insertItem(LC, Int, integer_to_list(Int)), + wxListCtrl:insertItem(LC, Int, "ABC " ++ integer_to_list(Int)), %% ItemData Can only be integers currently - wxListCtrl:setItemData(LC, Int, abs(2500-Int)) + wxListCtrl:setItemData(LC, Int, abs(50-Int)) end, - wx:foreach(Add, lists:seq(0,5000)), + wx:foreach(Add, lists:seq(0,50)), wxWindow:show(Frame), - timer:sleep(200), + timer:sleep(2000), Sort = fun() -> wxListCtrl:sortItems(LC, fun(A, B) -> @@ -374,11 +374,12 @@ listCtrlSort(Config) -> io:format("Sorted ~p ~n",[Time]), Item = wxListItem:new(), + wxListItem:setMask(Item, ?wxLIST_MASK_TEXT), _List = wx:map(fun(Int) -> wxListItem:setId(Item, Int), ?m(true, wxListCtrl:getItem(LC, Item)), - io:format("~s~n",[wxListItem:getText(Item)]) - end, lists:seq(0,100)), + io:format("~p: ~s~n",[Int, wxListItem:getText(Item)]) + end, lists:seq(0,10)), wxListItem:destroy(Item), wx_test_lib:wx_destroy(Frame,Config). @@ -510,3 +511,50 @@ toolbar(Config) -> wxFrame:connect(Frame, command_menu_selected, [{callback, Add}, {id, 747}]), wxFrame:show(Frame), wx_test_lib:wx_destroy(Frame,Config). + + +popup(TestInfo) when is_atom(TestInfo) -> wx_test_lib:tc_info(TestInfo); +popup(Config) -> + Wx = wx:new(), + Frame = wxFrame:new(Wx, ?wxID_ANY, "Frame"), + TB = wxFrame:createToolBar(Frame), + wxToolBar:addTool(TB, 747, "PressMe", wxArtProvider:getBitmap("wxART_COPY", [{size, {16,16}}]), + [{shortHelp, "Press Me"}]), + + Log = fun(#wx{id=Id, event=Ev}, Obj) -> + io:format("Got ~p from ~p~n", [Id, Ev]), + wxEvent:skip(Obj) + end, + CreatePopup = fun() -> + Pop = wxPopupTransientWindow:new(Frame), + Panel = wxPanel:new(Pop), + Sz = wxBoxSizer:new(?wxVERTICAL), + wxSizer:add(Sz, wxButton:new(Panel, 42, [{label, "A button"}])), + wxSizer:add(Sz, Txt = wxStaticText:new(Panel, 43, "Some static text")), + wxSizer:add(Sz, wxButton:new(Panel, 44, [{label, "B button"}])), + wxPanel:setSizerAndFit(Panel, Sz), + wxSizer:setSizeHints(Sz, Pop), + wxWindow:connect(Pop, command_button_clicked, [{callback, Log}]), + wxWindow:connect(Txt, left_up, [{callback, Log}]), + wxWindow:connect(Txt, middle_up, [{callback, Log}]), + wxWindow:connect(Txt, right_up, [{callback, Log}]), + wxWindow:connect(Pop, show, [{callback, Log}]), + Pos = wx_misc:getMousePosition(), + wxPopupTransientWindow:position(Pop, Pos, {-1, -1}), + wxPopupTransientWindow:popup(Pop), + Pop + end, + wxFrame:connect(Frame, command_menu_selected, [{id, 747}]), + wxFrame:show(Frame), + + Pop = CreatePopup(), + Scale = case wx_test_lib:user_available(Config) of + true -> 25; + false -> 1 + end, + receive + #wx{} -> CreatePopup() + after 200*Scale -> + wxPopupTransientWindow:dismiss(Pop) + end, + wx_test_lib:wx_destroy(Frame,Config). diff --git a/lib/wx/vsn.mk b/lib/wx/vsn.mk index 5523c20440..24e8c2ed11 100644 --- a/lib/wx/vsn.mk +++ b/lib/wx/vsn.mk @@ -1 +1 @@ -WX_VSN = 1.2 +WX_VSN = 1.3.1 diff --git a/lib/xmerl/doc/src/motorcycles2html.erl b/lib/xmerl/doc/src/motorcycles2html.erl index dfbd19e359..45c713e1ac 100644 --- a/lib/xmerl/doc/src/motorcycles2html.erl +++ b/lib/xmerl/doc/src/motorcycles2html.erl @@ -7,7 +7,7 @@ %%%------------------------------------------------------------------- -module(motorcycles2html). --include("xmerl.hrl"). +-include_lib("xmerl/include/xmerl.hrl"). -import(xmerl_xs, [ xslapply/2, value_of/1, select/2, built_in_rules/2 ]). @@ -57,12 +57,12 @@ template(E) -> built_in_rules(fun template/1, E). %% sorts on the bike name element, unwraps the bike information and %% inserts a line feed and indentation on each bike element. sort_by_manufacturer(L) -> - Tuples=[X1||X1={H,T} <- L], + Tuples=[X1||X1={_,_} <- L], SortedTS = lists:keysort(1,Tuples), InsertRefName_UnWrap= fun([{[Name],V}|Rest],Name,F)-> [V|F(Rest,Name,F)]; - ([{[Name],V}|Rest],PreviousName,F) -> + ([{[Name],V}|Rest],_PreviousName,F) -> [["<a name=\"",Name,"\"></>"],V|F(Rest,Name,F)]; ([],_,_) -> [] end, @@ -71,7 +71,7 @@ sort_by_manufacturer(L) -> WS = "\n ", Fun=fun([H|T],Acc,F)-> F(T,[H,WS|Acc],F); - ([],Acc,F)-> + ([],Acc,_F)-> lists:reverse([WS|Acc]) end, if length(SortedRefed) > 0 -> @@ -96,13 +96,12 @@ remove_duplicates([A|L],Acc) -> end. make_ref([]) -> []; -make_ref([H]) when atom(H) -> +make_ref([H]) when is_atom(H) -> "<ul><a href=\"#"++atom_to_list(H)++"\">"++atom_to_list(H)++"</a></ul>"; -make_ref([H]) when list(H) -> +make_ref([H]) when is_list(H) -> "<ul><a href=\"#"++H++"\">\s"++H++"</a></ul>"; -make_ref([H|T]) when atom(H) -> +make_ref([H|T]) when is_atom(H) -> ["<ul><a href=\"#"++atom_to_list(H)++"\">\s"++atom_to_list(H)++",\n</a></ul>" |make_ref(T)]; -make_ref([H|T]) when list(H) -> +make_ref([H|T]) when is_list(H) -> ["<ul><a href=\"#"++H++"\">\s"++H++",\n</a></ul>"|make_ref(T)]. - diff --git a/lib/xmerl/doc/src/notes.xml b/lib/xmerl/doc/src/notes.xml index b020b9bfa3..3fa1f01a79 100644 --- a/lib/xmerl/doc/src/notes.xml +++ b/lib/xmerl/doc/src/notes.xml @@ -31,6 +31,35 @@ <p>This document describes the changes made to the Xmerl application.</p> +<section><title>Xmerl 1.3.7</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Application upgrade (appup) files are corrected for the + following applications: </p> + <p> + <c>asn1, common_test, compiler, crypto, debugger, + dialyzer, edoc, eldap, erl_docgen, et, eunit, gs, hipe, + inets, observer, odbc, os_mon, otp_mibs, parsetools, + percept, public_key, reltool, runtime_tools, ssh, + syntax_tools, test_server, tools, typer, webtool, wx, + xmerl</c></p> + <p> + A new test utility for testing appup files is added to + test_server. This is now used by most applications in + OTP.</p> + <p> + (Thanks to Tobias Schlager)</p> + <p> + Own Id: OTP-11744</p> + </item> + </list> + </section> + +</section> + <section><title>Xmerl 1.3.6</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/make/ose_lm.mk.in b/make/ose_lm.mk.in index 2e659ada1a..5455ad94c6 100644 --- a/make/ose_lm.mk.in +++ b/make/ose_lm.mk.in @@ -37,7 +37,9 @@ LDR1FLAGS = @erl_xcomp_ose_ldflags_pass1@ LDR2FLAGS = @erl_xcomp_ose_ldflags_pass2@ OSEROOT = @erl_xcomp_ose_OSEROOT@ LCF = @erl_xcomp_ose_LM_LCF@ -LMCONF = @erl_xcomp_ose_LM_CONF@ +BEAM_LMCONF = @erl_xcomp_ose_BEAM_LM_CONF@ +EPMD_LMCONF = @erl_xcomp_ose_EPMD_LM_CONF@ +RUN_ERL_LMCONF = @erl_xcomp_ose_RUN_ERL_LM_CONF@ STRIP = @erl_xcomp_ose_STRIP@ LM_POST_LINK = @erl_xcomp_ose_LM_POST_LINK@ LM_SET_CONF = @erl_xcomp_ose_LM_SET_CONF@ @@ -52,9 +52,6 @@ usage () echo " release [-a] <target_dir> - creates full release to <target_dir>" echo " smp [-a] - build an Erlang system, smp flavor only" echo " tests <dir> - Build testsuites to <dir>" - echo " patch_app [-f] <target_dir> <app1>... - build given apps to <target_dir>" - echo " If core apps are patched, new start scripts will be created" - echo " and 'Install' must be run again." echo "" echo "These are for cleaning up an open source distribution" echo "with prebuilt files, so that it resembles the clean developers" @@ -1192,141 +1189,6 @@ do_release () release || exit 1 } -do_patch_app () -{ - # If target dir exists and has an installation of same major release, then - # build given apps. - # If patch includes erts, kernel, stdlib, sasl, then find latest - # erts, kernel, stdlib, sasl and create .rel files. - # Create .script/.boot - - if [ $# -lt 2 ]; then - usage - exit 1 - fi - - setup_make - if [ X`$MAKE is_cross_configured` = Xyes ]; then - TARGET=`$MAKE target_configured` - fi - if [ "x$1" = "x-f" ]; then - force="-force" - shift - else - force= - fi - target_dir=$1 - if [ ! -d "$target_dir/releases/$otp_major_vsn" ]; then - echo "No OTP $otp_major_vsn installation in $target_dir" 1>&2 - exit 1 - fi - - shift - - otp_version=`cat "$target_dir/releases/$otp_major_vsn/OTP_VERSION"` || { echo "Not able to read $target_dir/releases/$otp_major_vsn/OTP_VERSION" 1>&2; exit 1; } - { echo "$otp_version" | sed "s|^\([^\*]*\)\**|\1\*\*|g" > "$target_dir/releases/$otp_major_vsn/OTP_VERSION"; } 2>/dev/null || { echo "Not able to update $target_dir/OTP_VERSION" 1>&2; exit 1; } - - PATH="$ERL_TOP/bootstrap/bin:$PATH" $ERL_TOP/make/verify_runtime_dependencies -release "$otp_major_vsn" -source "$ERL_TOP" -target "$target_dir" $force "$@" - - if [ $? -ne 0 ]; then - exit $? - fi - - # Build all applications to target - for app in "$@"; do - if [ "$app" = "erts" ] && [ -d $ERL_TOP/$app ]; then - (cd $ERL_TOP/$app && $MAKE MAKE="$MAKE" TARGET=$TARGET \ - TESTROOT="$target_dir" release) || exit 1 - elif [ "$app" != "erts" ] && [ -d $ERL_TOP/lib/$app ]; then - (cd $ERL_TOP/lib/$app && $MAKE MAKE="$MAKE" TARGET=$TARGET \ - TESTROOT="$target_dir" release) || exit 1 - else - echo "Invalid application $app" 1>&2 - exit 1 - fi - done - - # If erts, kernel, stdlib or sasl is included, find versions - for app in "$@"; do - if [ "$app" = "erts" ]; then - erts_vsn=`grep '^VSN' erts/vsn.mk | sed "s|^VSN.*=[^0-9]*\([0-9].*\)$|\1|g"` - update_rel=true - elif [ "$app" = "kernel" ]; then - kernel_vsn=`sed "s|^KERNEL_VSN[^=]*=[^0-9]*\([0-9].*\)$|\1|g" lib/kernel/vsn.mk` - update_rel=true - elif [ "$app" = "stdlib" ]; then - stdlib_vsn=`sed "s|^STDLIB_VSN[^=]*=[^0-9]*\([0-9].*\)$|\1|g" lib/stdlib/vsn.mk` - update_rel=true - elif [ "$app" = "sasl" ]; then - sasl_vsn=`sed "s|^SASL_VSN[^=]*=[^0-9]*\([0-9].*\)$|\1|g" lib/sasl/vsn.mk` - update_rel=true - fi - done - - # and find the old versions for those not included - if [ "X$update_rel" != "X" ]; then - if [ "X$erts_vsn" = "X" ]; then - erts_vsns=`ls -d "$target_dir"/erts-* | sed "s|$target_dir/erts-\([0-9\.].*\)|\1|g"` - erts_vsn=`echo "$erts_vsns" | sort -t '.' -g | tail -n 1` - fi - if [ "X$kernel_vsn" = "X" ]; then - kernel_vsns=`ls -d "$target_dir"/lib/kernel-* | sed "s|$target_dir/lib/kernel-\([0-9\.].*\)|\1|g"` - kernel_vsn=`echo "$kernel_vsns" | sort -t '.' -g | tail -n 1` - fi - if [ "X$stdlib_vsn" = "X" ]; then - stdlib_vsns=`ls -d "$target_dir"/lib/stdlib-* | sed "s|$target_dir/lib/stdlib-\([0-9\.].*\)|\1|g"` - stdlib_vsn=`echo "$stdlib_vsns" | sort -t '.' -g | tail -n 1` - fi - if [ "X$sasl_vsn" = "X" ]; then - sasl_vsns=`ls -d "$target_dir"/lib/sasl-* | sed "s|$target_dir/lib/sasl-\([0-9\.].*\)|\1|g"` - sasl_vsn=`echo "$sasl_vsns" | sort -t '.' -g | tail -n 1` - fi - - # Generate .rel, .script and .boot - to tmp dir - start_clean="{release, {\"Erlang/OTP\",\"$otp_major_vsn\"}, {erts, \"$erts_vsn\"},\n [{kernel,\"$kernel_vsn\"},\n {stdlib,\"$stdlib_vsn\"}]}.\n" - start_sasl="{release, {\"Erlang/OTP\",\"$otp_major_vsn\"}, {erts, \"$erts_vsn\"},\n [{kernel,\"$kernel_vsn\"},\n {stdlib,\"$stdlib_vsn\"},\n {sasl,\"$sasl_vsn\"}]}.\n" - - tmp_dir="$target_dir/tmp"; - if [ ! -d "$tmp_dir" ]; then - mkdir "$tmp_dir" - fi - echo "$start_sasl" > "$tmp_dir/start_sasl.rel" - echo "$start_clean" > "$tmp_dir/start_clean.rel" - echo "$start_clean" > "$tmp_dir/no_dot_erlang.rel" - - erlc="$ERL_TOP/bootstrap/bin/erlc" - if [ ! -x "$erlc" ]; then - echo "erlc not found, can not create .script and .boot files" 1>&2 - exit 1 - fi - - $erlc -I"$target_dir"/lib/*/ebin -o$tmp_dir $tmp_dir/start_sasl.rel || exit 1 - $erlc -I"$target_dir"/lib/*/ebin -o$tmp_dir +no_warn_sasl $tmp_dir/start_clean.rel || exit 1 - $erlc -I"$target_dir"/lib/*/ebin -o$tmp_dir +no_warn_sasl +no_dot_erlang $tmp_dir/no_dot_erlang.rel || exit 1 - - # Generate RELEASES file - erl="$ERL_TOP/bootstrap/bin/erl" - if [ ! -x "$erl" ]; then - echo "erl not found, can not create RELEASES file" 1>&2 - exit 1 - fi - "$erl" -noinput +B -eval "release_handler:create_RELEASES(\"%ERL_ROOT%\", \"$tmp_dir\", \"$tmp_dir/start_sasl.rel\", []), halt()" || exit 1 - - # If all good so far, move generated files into target area - mv "$tmp_dir/RELEASES" "$target_dir/releases/RELEASES.src" - mv "$tmp_dir"/* "$target_dir/releases/$otp_major_vsn" - rmdir "$tmp_dir" - - # Remove old start scripts (forces a new run of Install) - rm -f "$target_dir"/releases/RELEASES - rm -f "$target_dir"/bin/*.script - rm -f "$target_dir"/bin/*.boot - rm -f "$target_dir"/bin/erl - fi - -} - - do_tests () { setup_make @@ -1587,9 +1449,6 @@ case "$1" in shift fi; do_release "$2";; - patch_app) - shift; - do_patch_app "$@";; tests) if [ $minus_a_flag = true ]; then shift diff --git a/otp_versions.table b/otp_versions.table index e69de29bb2..32fe5ae733 100644 --- a/otp_versions.table +++ b/otp_versions.table @@ -0,0 +1,10 @@ +OTP-17.3 : asn1-3.0.2 common_test-1.8.2 compiler-5.0.2 crypto-3.4.1 dialyzer-2.7.2 diameter-1.7.1 edoc-0.7.15 erl_docgen-0.3.6 erl_interface-3.7.18 erts-6.2 eunit-2.2.8 hipe-3.11.1 ic-4.3.6 inets-5.10.3 jinterface-1.5.10 kernel-3.0.3 megaco-3.17.2 mnesia-4.12.3 observer-2.0.2 odbc-2.10.21 os_mon-2.3 ose-1.0.1 public_key-0.22.1 sasl-2.4.1 snmp-5.1 ssh-3.0.5 ssl-5.3.6 stdlib-2.2 tools-2.7 wx-1.3.1 # cosEvent-2.1.15 cosEventDomain-1.1.14 cosFileTransfer-1.1.16 cosNotification-1.1.21 cosProperty-1.1.17 cosTime-1.1.14 cosTransactions-1.2.14 debugger-4.0.1 eldap-1.0.3 et-1.5 gs-1.5.16 orber-3.7 otp_mibs-1.0.9 parsetools-2.0.11 percept-0.8.9 reltool-0.6.6 runtime_tools-1.8.14 syntax_tools-1.6.16 test_server-3.7.1 typer-0.9.8 webtool-0.8.10 xmerl-1.3.7 : +OTP-17.2.2 : mnesia-4.12.2 # asn1-3.0.1 common_test-1.8.1 compiler-5.0.1 cosEvent-2.1.15 cosEventDomain-1.1.14 cosFileTransfer-1.1.16 cosNotification-1.1.21 cosProperty-1.1.17 cosTime-1.1.14 cosTransactions-1.2.14 crypto-3.4 debugger-4.0.1 dialyzer-2.7.1 diameter-1.7 edoc-0.7.14 eldap-1.0.3 erl_docgen-0.3.5 erl_interface-3.7.17 erts-6.1.2 et-1.5 eunit-2.2.7 gs-1.5.16 hipe-3.11 ic-4.3.5 inets-5.10.2 jinterface-1.5.9 kernel-3.0.2 megaco-3.17.1 observer-2.0.1 odbc-2.10.20 orber-3.7 os_mon-2.2.15 ose-1.0 otp_mibs-1.0.9 parsetools-2.0.11 percept-0.8.9 public_key-0.22 reltool-0.6.6 runtime_tools-1.8.14 sasl-2.4 snmp-5.0 ssh-3.0.4 ssl-5.3.5 stdlib-2.1.1 syntax_tools-1.6.16 test_server-3.7.1 tools-2.6.15 typer-0.9.8 webtool-0.8.10 wx-1.3 xmerl-1.3.7 : +OTP-17.2.1 : ssh-3.0.4 # asn1-3.0.1 common_test-1.8.1 compiler-5.0.1 cosEvent-2.1.15 cosEventDomain-1.1.14 cosFileTransfer-1.1.16 cosNotification-1.1.21 cosProperty-1.1.17 cosTime-1.1.14 cosTransactions-1.2.14 crypto-3.4 debugger-4.0.1 dialyzer-2.7.1 diameter-1.7 edoc-0.7.14 eldap-1.0.3 erl_docgen-0.3.5 erl_interface-3.7.17 erts-6.1.2 et-1.5 eunit-2.2.7 gs-1.5.16 hipe-3.11 ic-4.3.5 inets-5.10.2 jinterface-1.5.9 kernel-3.0.2 megaco-3.17.1 mnesia-4.12.1 observer-2.0.1 odbc-2.10.20 orber-3.7 os_mon-2.2.15 ose-1.0 otp_mibs-1.0.9 parsetools-2.0.11 percept-0.8.9 public_key-0.22 reltool-0.6.6 runtime_tools-1.8.14 sasl-2.4 snmp-5.0 ssl-5.3.5 stdlib-2.1.1 syntax_tools-1.6.16 test_server-3.7.1 tools-2.6.15 typer-0.9.8 webtool-0.8.10 wx-1.3 xmerl-1.3.7 : +OTP-17.2 : orber-3.7 snmp-5.0 # asn1-3.0.1 common_test-1.8.1 compiler-5.0.1 cosEvent-2.1.15 cosEventDomain-1.1.14 cosFileTransfer-1.1.16 cosNotification-1.1.21 cosProperty-1.1.17 cosTime-1.1.14 cosTransactions-1.2.14 crypto-3.4 debugger-4.0.1 dialyzer-2.7.1 diameter-1.7 edoc-0.7.14 eldap-1.0.3 erl_docgen-0.3.5 erl_interface-3.7.17 erts-6.1.2 et-1.5 eunit-2.2.7 gs-1.5.16 hipe-3.11 ic-4.3.5 inets-5.10.2 jinterface-1.5.9 kernel-3.0.2 megaco-3.17.1 mnesia-4.12.1 observer-2.0.1 odbc-2.10.20 os_mon-2.2.15 ose-1.0 otp_mibs-1.0.9 parsetools-2.0.11 percept-0.8.9 public_key-0.22 reltool-0.6.6 runtime_tools-1.8.14 sasl-2.4 ssh-3.0.3 ssl-5.3.5 stdlib-2.1.1 syntax_tools-1.6.16 test_server-3.7.1 tools-2.6.15 typer-0.9.8 webtool-0.8.10 wx-1.3 xmerl-1.3.7 : +OTP-17.1.2 : erts-6.1.2 kernel-3.0.2 stdlib-2.1.1 # asn1-3.0.1 common_test-1.8.1 compiler-5.0.1 cosEvent-2.1.15 cosEventDomain-1.1.14 cosFileTransfer-1.1.16 cosNotification-1.1.21 cosProperty-1.1.17 cosTime-1.1.14 cosTransactions-1.2.14 crypto-3.4 debugger-4.0.1 dialyzer-2.7.1 diameter-1.7 edoc-0.7.14 eldap-1.0.3 erl_docgen-0.3.5 erl_interface-3.7.17 et-1.5 eunit-2.2.7 gs-1.5.16 hipe-3.11 ic-4.3.5 inets-5.10.2 jinterface-1.5.9 megaco-3.17.1 mnesia-4.12.1 observer-2.0.1 odbc-2.10.20 orber-3.6.27 os_mon-2.2.15 ose-1.0 otp_mibs-1.0.9 parsetools-2.0.11 percept-0.8.9 public_key-0.22 reltool-0.6.6 runtime_tools-1.8.14 sasl-2.4 snmp-4.25.1 ssh-3.0.3 ssl-5.3.5 syntax_tools-1.6.16 test_server-3.7.1 tools-2.6.15 typer-0.9.8 webtool-0.8.10 wx-1.3 xmerl-1.3.7 : +OTP-17.1.1 : edoc-0.7.14 erts-6.1.1 syntax_tools-1.6.16 # asn1-3.0.1 common_test-1.8.1 compiler-5.0.1 cosEvent-2.1.15 cosEventDomain-1.1.14 cosFileTransfer-1.1.16 cosNotification-1.1.21 cosProperty-1.1.17 cosTime-1.1.14 cosTransactions-1.2.14 crypto-3.4 debugger-4.0.1 dialyzer-2.7.1 diameter-1.7 eldap-1.0.3 erl_docgen-0.3.5 erl_interface-3.7.17 et-1.5 eunit-2.2.7 gs-1.5.16 hipe-3.11 ic-4.3.5 inets-5.10.2 jinterface-1.5.9 kernel-3.0.1 megaco-3.17.1 mnesia-4.12.1 observer-2.0.1 odbc-2.10.20 orber-3.6.27 os_mon-2.2.15 ose-1.0 otp_mibs-1.0.9 parsetools-2.0.11 percept-0.8.9 public_key-0.22 reltool-0.6.6 runtime_tools-1.8.14 sasl-2.4 snmp-4.25.1 ssh-3.0.3 ssl-5.3.5 stdlib-2.1 test_server-3.7.1 tools-2.6.15 typer-0.9.8 webtool-0.8.10 wx-1.3 xmerl-1.3.7 : +OTP-17.1 : asn1-3.0.1 common_test-1.8.1 compiler-5.0.1 crypto-3.4 debugger-4.0.1 dialyzer-2.7.1 diameter-1.7 erl_interface-3.7.17 erts-6.1 hipe-3.11 inets-5.10.2 kernel-3.0.1 mnesia-4.12.1 observer-2.0.1 reltool-0.6.6 ssh-3.0.3 ssl-5.3.5 stdlib-2.1 syntax_tools-1.6.15 test_server-3.7.1 tools-2.6.15 typer-0.9.8 wx-1.3 # cosEvent-2.1.15 cosEventDomain-1.1.14 cosFileTransfer-1.1.16 cosNotification-1.1.21 cosProperty-1.1.17 cosTime-1.1.14 cosTransactions-1.2.14 edoc-0.7.13 eldap-1.0.3 erl_docgen-0.3.5 et-1.5 eunit-2.2.7 gs-1.5.16 ic-4.3.5 jinterface-1.5.9 megaco-3.17.1 odbc-2.10.20 orber-3.6.27 os_mon-2.2.15 ose-1.0 otp_mibs-1.0.9 parsetools-2.0.11 percept-0.8.9 public_key-0.22 runtime_tools-1.8.14 sasl-2.4 snmp-4.25.1 webtool-0.8.10 xmerl-1.3.7 : +OTP-17.0.2 : inets-5.10.1 ssh-3.0.2 # asn1-3.0 common_test-1.8 compiler-5.0 cosEvent-2.1.15 cosEventDomain-1.1.14 cosFileTransfer-1.1.16 cosNotification-1.1.21 cosProperty-1.1.17 cosTime-1.1.14 cosTransactions-1.2.14 crypto-3.3 debugger-4.0 dialyzer-2.7 diameter-1.6 edoc-0.7.13 eldap-1.0.3 erl_docgen-0.3.5 erl_interface-3.7.16 erts-6.0.1 et-1.5 eunit-2.2.7 gs-1.5.16 hipe-3.10.3 ic-4.3.5 jinterface-1.5.9 kernel-3.0 megaco-3.17.1 mnesia-4.12 observer-2.0 odbc-2.10.20 orber-3.6.27 os_mon-2.2.15 ose-1.0 otp_mibs-1.0.9 parsetools-2.0.11 percept-0.8.9 public_key-0.22 reltool-0.6.5 runtime_tools-1.8.14 sasl-2.4 snmp-4.25.1 ssl-5.3.4 stdlib-2.0 syntax_tools-1.6.14 test_server-3.7 tools-2.6.14 typer-0.9.7 webtool-0.8.10 wx-1.2 xmerl-1.3.7 : +OTP-17.0.1 : erts-6.0.1 typer-0.9.7 # asn1-3.0 common_test-1.8 compiler-5.0 cosEvent-2.1.15 cosEventDomain-1.1.14 cosFileTransfer-1.1.16 cosNotification-1.1.21 cosProperty-1.1.17 cosTime-1.1.14 cosTransactions-1.2.14 crypto-3.3 debugger-4.0 dialyzer-2.7 diameter-1.6 edoc-0.7.13 eldap-1.0.3 erl_docgen-0.3.5 erl_interface-3.7.16 et-1.5 eunit-2.2.7 gs-1.5.16 hipe-3.10.3 ic-4.3.5 inets-5.10 jinterface-1.5.9 kernel-3.0 megaco-3.17.1 mnesia-4.12 observer-2.0 odbc-2.10.20 orber-3.6.27 os_mon-2.2.15 ose-1.0 otp_mibs-1.0.9 parsetools-2.0.11 percept-0.8.9 public_key-0.22 reltool-0.6.5 runtime_tools-1.8.14 sasl-2.4 snmp-4.25.1 ssh-3.0.1 ssl-5.3.4 stdlib-2.0 syntax_tools-1.6.14 test_server-3.7 tools-2.6.14 webtool-0.8.10 wx-1.2 xmerl-1.3.7 : +OTP-17.0 : asn1-3.0 common_test-1.8 compiler-5.0 cosEvent-2.1.15 cosEventDomain-1.1.14 cosFileTransfer-1.1.16 cosNotification-1.1.21 cosProperty-1.1.17 cosTime-1.1.14 cosTransactions-1.2.14 crypto-3.3 debugger-4.0 dialyzer-2.7 diameter-1.6 edoc-0.7.13 eldap-1.0.3 erl_docgen-0.3.5 erl_interface-3.7.16 erts-6.0 et-1.5 eunit-2.2.7 gs-1.5.16 hipe-3.10.3 ic-4.3.5 inets-5.10 jinterface-1.5.9 kernel-3.0 megaco-3.17.1 mnesia-4.12 observer-2.0 odbc-2.10.20 orber-3.6.27 os_mon-2.2.15 ose-1.0 otp_mibs-1.0.9 parsetools-2.0.11 percept-0.8.9 public_key-0.22 reltool-0.6.5 runtime_tools-1.8.14 sasl-2.4 snmp-4.25.1 ssh-3.0.1 ssl-5.3.4 stdlib-2.0 syntax_tools-1.6.14 test_server-3.7 tools-2.6.14 typer-0.9.6 webtool-0.8.10 wx-1.2 xmerl-1.3.7 # : diff --git a/system/doc/design_principles/applications.xml b/system/doc/design_principles/applications.xml index 228ca1f2bf..7b030115df 100644 --- a/system/doc/design_principles/applications.xml +++ b/system/doc/design_principles/applications.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>1997</year><year>2013</year> + <year>1997</year><year>2014</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -157,8 +157,9 @@ ch_app:stop([])</code> all applications have dependencies to at least <c>kernel</c> and <c>stdlib</c>.</item> </taglist> - <p>The syntax and contents of of the application resource file - are described in detail in <c>app(4)</c>.</p> + <note><p>The syntax and contents of of the application resource file + are described in detail in the<seealso marker="kernel:app"> + Application resource file reference</seealso>.</p></note> </section> <section> diff --git a/system/doc/design_principles/release_handling.xml b/system/doc/design_principles/release_handling.xml index 2a5831b89f..ba8a88d1c2 100644 --- a/system/doc/design_principles/release_handling.xml +++ b/system/doc/design_principles/release_handling.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2003</year><year>2013</year> + <year>2003</year><year>2014</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -329,13 +329,26 @@ automatically ensured.</p> <p>When the release handler encounters the instruction, it first generates a temporary boot file, which starts the new versions - of the emulator and the core applications. Then it shuts down + of the emulator and the core applications, and the old version + of all other applications. Then it shuts down the current emulator by calling <c>init:reboot()</c>, see <c>init(3)</c>. All processes are terminated gracefully and the system is rebooted by the heart program, using the temporary boot file. After the reboot, the rest of the relup instructions are executed. This is done as a part of the temporary boot script.</p> + <warning> + <p>Since this mechanism causes the new versions of the + emulator and core applications to run with the old version of + other applications during startup, extra care must be taken to + avoid incompatibility. Incompatible changes in the core + applications may in some situations be necessary. If possible, + such changes are preceded by deprecation over two major + releases before the actual change. To make sure your + application is not crashed by an incompatible change, always + remove any call to deprecated functions as soon as + possible.</p> + </warning> <p>An info report is written when the upgrade is completed. To programatically find out if the upgrade is complete, call <c>release_handler:which_releases(current)</c> and check diff --git a/system/doc/design_principles/spec_proc.xml b/system/doc/design_principles/spec_proc.xml index 69bf2e0448..e4fb5fdca7 100644 --- a/system/doc/design_principles/spec_proc.xml +++ b/system/doc/design_principles/spec_proc.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>1997</year><year>2013</year> + <year>1997</year><year>2014</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -431,11 +431,11 @@ loop(...) -> <section> <title>User-Defined Behaviours</title> - <p>To implement a user-defined behaviour, write code similar to + <p><marker id="behaviours"/>To implement a user-defined behaviour, write code similar to code for a special process but calling functions in a callback module for handling specific tasks.</p> <p>If it is desired that the compiler should warn for missing callback - functions, as it does for the OTP behaviours, add callback attributes in the + functions, as it does for the OTP behaviours, add <c>-callback</c> attributes in the behaviour module to describe the expected callbacks:</p> <code type="none"> -callback Name1(Arg1_1, Arg1_2, ..., Arg1_N1) -> Res1. @@ -445,15 +445,15 @@ loop(...) -> <p>where <c>NameX</c> are the names of the expected callbacks and <c>ArgX_Y</c>, <c>ResX</c> are types as they are described in Specifications for functions in <seealso marker="../reference_manual/typespec">Types and - Function Specifications</seealso>. The whole syntax of spec attributes is - supported by callback attributes.</p> + Function Specifications</seealso>. The whole syntax of <c>-spec</c> attribute is + supported by <c>-callback</c> attribute.</p> <p>Alternatively you may directly implement and export the function:</p> <code type="none"> behaviour_info(callbacks) -> - [{Name1,Arity1},...,{NameN,ArityN}].</code> - <p>where each <c>{Name,Arity}</c> specifies the name and arity of a callback + [{Name1, Arity1},...,{NameN, ArityN}].</code> + <p>where each <c>{Name, Arity}</c> specifies the name and arity of a callback function. This function is otherwise automatically generated by the compiler - using the callback attributes.</p> + using the <c>-callback</c> attributes.</p> <p>When the compiler encounters the module attribute <c>-behaviour(Behaviour).</c> in a module <c>Mod</c>, it will call <c>Behaviour:behaviour_info(callbacks)</c> and compare the result with the diff --git a/system/doc/efficiency_guide/advanced.xml b/system/doc/efficiency_guide/advanced.xml index b5771a5929..51f1b2612c 100644 --- a/system/doc/efficiency_guide/advanced.xml +++ b/system/doc/efficiency_guide/advanced.xml @@ -183,7 +183,7 @@ On 64-bit architectures: 4 words for a reference from the current local node, an <tag><em>Open ports</em></tag> <item> <marker id="ports"></marker> - <p>The maximum number of simultaneously oper Erlang ports is + <p>The maximum number of simultaneously open Erlang ports is often by default 16384. This limit can be configured at startup, for more information see the <seealso marker="erts:erl#max_ports"><c>+Q</c></seealso> diff --git a/system/doc/efficiency_guide/binaryhandling.xml b/system/doc/efficiency_guide/binaryhandling.xml index 6b0df49011..4ba1378059 100644 --- a/system/doc/efficiency_guide/binaryhandling.xml +++ b/system/doc/efficiency_guide/binaryhandling.xml @@ -5,7 +5,7 @@ <header> <copyright> <year>2007</year> - <year>2013</year> + <year>2014</year> <holder>Ericsson AB, All Rights Reserved</holder> </copyright> <legalnotice> @@ -237,8 +237,9 @@ Bin = <<Bin1,...>> %% Bin1 will be COPIED <p><c>Bin1</c> will be copied in the third line.</p> <p>The same thing happens if you insert a binary into an <em>ets</em> - table or send it to a port using <c>erlang:port_command/2</c>.</p> - + table or send it to a port using <c>erlang:port_command/2</c> or pass it to + <seealso marker="erts:erl_nif#enif_inspect_binary">enif_inspect_binary</seealso> + in a NIF.</p> <p>Matching a binary will also cause it to shrink and the next append operation will copy the binary data:</p> diff --git a/system/doc/getting_started/seq_prog.xml b/system/doc/getting_started/seq_prog.xml index 3830a34e5a..fd49102263 100644 --- a/system/doc/getting_started/seq_prog.xml +++ b/system/doc/getting_started/seq_prog.xml @@ -419,6 +419,129 @@ list_length([First | Rest]) -> </section> <section> + <title>Maps</title> + <p>Maps are a set of key to value associations. These associations + are encapsulated with "#{" and "}". To create an association from + "key" to value 42, we write:</p> +<code type="none"> +> #{ "key" => 42 }. +#{"key" => 42}</code> + <p>We will jump straight into the deep end with an example using some + interesting features.</p> + <p>The following example shows how we calculate alpha blending using + maps to reference color and alpha channels:</p> + <code type="none"> +-module(color). + +-export([new/4, blend/2]). + +-define(is_channel(V), (is_float(V) andalso V >= 0.0 andalso V =< 1.0)). + +new(R,G,B,A) when ?is_channel(R), ?is_channel(G), + ?is_channel(B), ?is_channel(A) -> + #{red => R, green => G, blue => B, alpha => A}. + +blend(Src,Dst) -> + blend(Src,Dst,alpha(Src,Dst)). + +blend(Src,Dst,Alpha) when Alpha > 0.0 -> + Dst#{ + red := red(Src,Dst) / Alpha, + green := green(Src,Dst) / Alpha, + blue := blue(Src,Dst) / Alpha, + alpha := Alpha + }; +blend(_,Dst,_) -> + Dst#{ + red := 0.0, + green := 0.0, + blue := 0.0, + alpha := 0.0 + }. + +alpha(#{alpha := SA}, #{alpha := DA}) -> + SA + DA*(1.0 - SA). + +red(#{red := SV, alpha := SA}, #{red := DV, alpha := DA}) -> + SV*SA + DV*DA*(1.0 - SA). +green(#{green := SV, alpha := SA}, #{green := DV, alpha := DA}) -> + SV*SA + DV*DA*(1.0 - SA). +blue(#{blue := SV, alpha := SA}, #{blue := DV, alpha := DA}) -> + SV*SA + DV*DA*(1.0 - SA).</code> + <p>Compile (file <c>color.erl</c>) and test:</p> + <pre> +> <input>c(color).</input> +{ok,color} +> <input>C1 = color:new(0.3,0.4,0.5,1.0).</input> +#{alpha => 1.0,blue => 0.5,green => 0.4,red => 0.3} +> <input>C2 = color:new(1.0,0.8,0.1,0.3).</input> +#{alpha => 0.3,blue => 0.1,green => 0.8,red => 1.0} +> <input>color:blend(C1,C2).</input> +#{alpha => 1.0,blue => 0.5,green => 0.4,red => 0.3} +> <input>color:blend(C2,C1).</input> +#{alpha => 1.0,blue => 0.38,green => 0.52,red => 0.51} +</pre> + <p>This example warrant some explanation:</p> + <code type="none"> +-define(is_channel(V), (is_float(V) andalso V >= 0.0 andalso V =< 1.0)).</code> + <p> + First we define a macro <c>is_channel</c> to help with our guard tests. + This is only here for convenience and to reduce syntax cluttering. + + You can read more about <seealso marker="doc/reference_manual:macros">Macros</seealso> + in the Erlang Reference Manual. + </p> + <code type="none"> +new(R,G,B,A) when ?is_channel(R), ?is_channel(G), + ?is_channel(B), ?is_channel(A) -> + #{red => R, green => G, blue => B, alpha => A}.</code> + <p> + The function <c>new/4</c> creates a new map term with and lets the keys + <c>red</c>, <c>green</c>, <c>blue</c> and <c>alpha</c> be associated + with an initial value. In this case we only allow for float + values between and including 0.0 and 1.0 as ensured by the <c>?is_channel/1</c> macro + for each argument. Only the <c>=></c> operator is allowed when creating a new map. + </p> + <p> + By calling <c>blend/2</c> on any color term created by <c>new/4</c> we can calculate + the resulting color as determined by the two maps terms. + </p> + <p> + The first thing <c>blend/2</c> does is to calculate the resulting alpha channel. + </p> + <code type="none"> +alpha(#{alpha := SA}, #{alpha := DA}) -> + SA + DA*(1.0 - SA).</code> + <p> + We fetch the value associated with key <c>alpha</c> for both arguments using + the <c>:=</c> operator. Any other keys + in the map are ignored, only the key <c>alpha</c> is required and checked for. + </p> + <p>This is also the case for functions <c>red/2</c>, <c>blue/2</c> and <c>green/2</c>.</p> + <code type="none"> +red(#{red := SV, alpha := SA}, #{red := DV, alpha := DA}) -> + SV*SA + DV*DA*(1.0 - SA).</code> + <p> + The difference here is that we check for two keys in each map argument. The other keys + are ignored. + </p> + <p> + Finally we return the resulting color in <c>blend/3</c>. + </p> + <code type="none"> +blend(Src,Dst,Alpha) when Alpha > 0.0 -> + Dst#{ + red := red(Src,Dst) / Alpha, + green := green(Src,Dst) / Alpha, + blue := blue(Src,Dst) / Alpha, + alpha := Alpha + };</code> + <p> + We update the <c>Dst</c> map with new channel values. The syntax for updating an existing key with a new value is done with <c>:=</c> operator. + </p> + </section> + + <section> <title>Standard Modules and Manual Pages</title> <p>Erlang has a lot of standard modules to help you do things. For example, the module <c>io</c> contains a lot of functions to help diff --git a/system/doc/installation_guide/Makefile b/system/doc/installation_guide/Makefile index 6923f52d8a..83210bd21f 100644 --- a/system/doc/installation_guide/Makefile +++ b/system/doc/installation_guide/Makefile @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 1996-2012. All Rights Reserved. +# Copyright Ericsson AB 1996-2014. All Rights Reserved. # # The contents of this file are subject to the Erlang Public License, # Version 1.1, (the "License"); you may not use this file except in @@ -58,8 +58,7 @@ XML_FILES = \ GENERATED_XML_FILES = \ INSTALL.xml \ INSTALL-CROSS.xml \ - INSTALL-WIN32.xml \ - MARKDOWN.xml + INSTALL-WIN32.xml # ---------------------------------------------------- @@ -74,8 +73,7 @@ REDIRECT_HTML_DIR = $(HTMLDIR)/source REDIRECT_HTML_FILES = \ $(REDIRECT_HTML_DIR)/INSTALL.html \ $(REDIRECT_HTML_DIR)/INSTALL-CROSS.html \ - $(REDIRECT_HTML_DIR)/INSTALL-WIN32.html \ - $(REDIRECT_HTML_DIR)/MARKDOWN.html + $(REDIRECT_HTML_DIR)/INSTALL-WIN32.html # ---------------------------------------------------- # FLAGS diff --git a/system/doc/installation_guide/install-binary.xml b/system/doc/installation_guide/install-binary.xml index 07027a39f6..af7dab6e44 100644 --- a/system/doc/installation_guide/install-binary.xml +++ b/system/doc/installation_guide/install-binary.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2000</year><year>2013</year> + <year>2000</year><year>2014</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -31,117 +31,40 @@ <rev>C</rev> <file>install-binary.xml</file> </header> - - <section> - <title>UNIX</title> - - <section> - <title>Introduction</title> - <p>The system is delivered as a single compressed tar file.</p> - <p>To browse the on-line HTML documentation, Netscape or an equivalent - browser supporting frames is needed.</p> - </section> - - <section> - <title>Installation Procedure</title> - <p>When installed, the entire system, except for a small start-up - script, resides in a single directory tree. The location of this - directory tree can be chosen arbitrarily by the installer, and it - does not need to be in the user's <c>$PATH</c>. The only requirements - are that the file system where it is placed has enough free space, - and that the users who run Erlang/OTP have read access to it. In the - example below, the directory tree is assumed to be located at - <c>/usr/local/erlang</c>, which is here called the <em>top-level directory</em>.</p> - <p>It is assumed that you have the compressed tar file, the name of - which is <c><![CDATA[<PREFIX>.tar.gz]]></c>, where <c><![CDATA[<PREFIX>]]></c> is a string - denoting the particular Erlang/OTP release, e.g. - <c>otp_LXA_11930_sunos5_R9B</c>.</p> - <p>Wherever the string <c><![CDATA[<PREFIX>]]></c> is used below, it should - be replaced by the actual name prefix of the compressed tar file.</p> - <p>The tape archive file does not have one single directory in which - all other files are rooted. Therefore the tape archive file must be - extracted into an empty (newly created) directory.</p> - <list type="ordered"> - <item> - <p>If the <em>top-level directory</em> does not already exist, - create it:</p> - <pre> -mkdir /usr/local/erlang</pre> - </item> - <item> - <p>Change the current directory to the <em>top level directory</em>:</p> - <pre> -cd /usr/local/erlang</pre> - </item> - <item> - <p>Create the <em>installation directory</em> with an appropriate - name. For example:</p> - <pre> -mkdir otp_r7b</pre> - </item> - <item> - <p>Change to the <em>installation directory</em>, e.g.</p> - <pre> -cd otp_r7b</pre> - </item> - <item> - <p>Assuming the compressed tar file resides in the directory - <c><![CDATA[<SOME-DIR>]]></c>,. extract the compressed tar file into the - current directory:</p> - <pre> -gunzip -c <SOME-DIR>/<PREFIX>.tar.gz | tar xfp -</pre> - </item> - <item> - <p>Read the <c>README</c> file in the installation directory for - last minute updates, before proceeding.</p> - </item> - <item> - <p>Run the <c>Install</c> script in the installation directory, - with the absolute path of the installation directory as argument,</p> - <pre> -./Install /usr/local/erlang/otp_r7b</pre> - <p>and supply answers to the prompts.</p> - <p>In most cases, there is a default answer in square brackets - (<c>[]</c>). If the default is satisfactory, just press - <c><![CDATA[<Return>]]></c>. In general you are only prompted for one thing:</p> - <list type="bulleted"> - <item> - <p>"Do you want to use a minimal system startup instead of the - SASL startup?" <br></br> - - In a minimal system, only the Kernel and STDLIB applications - are loaded and started. If the SASL startup is used, the SASL - application is included as well. Normally, the minimal system - is enough.</p> - </item> - </list> - </item> - <item> - <p>Make Erlang/OTP available for users, either by putting the path - <c>/usr/local/erlang/otp_r7b/bin</c> in users <c>$PATH</c> - variable, or link the executable - <c>/usr/local/erlang/otp_r7b/bin/erl</c> accordingly, e.g.:</p> - <pre> -ln -s /usr/local/erlang/otp_r7b/bin/erl /usr/local/bin/erl </pre> - </item> - </list> - </section> - </section> - <section> <title>Windows</title> <section> <title>Introduction</title> - <p>The system is delivered as a single <c>.exe</c> file.</p> - <p>To browse the on-line HTML documentation, Netscape or an equivalent - browser supporting frames is needed.</p> + <p>The system is delivered as a Windows Installer executable. + Get it from our <url href="http://www.erlang.org/download.html">download page</url>.</p> </section> <section> - <title>Installation Procedure</title> + <title>Installation</title> <p>The installation procedure is is automated. Double-click the <c>.exe</c> file icon and follow the instructions.</p> </section> + <section> + <title>Verification</title> + <list type="bulleted"> + <item> + <p>Start Erlang/OTP by double-clicking on the Erlang shortcut icon on the + desktop.</p> + <p>Expect a command line window to pop up with an output looking something like this:</p> + <pre> + Erlang/OTP 17 [erts-6.0] [64-bit] [smp:2:2] + + Eshell V6.0 (abort with ^G) + 1></pre> + </item> + <item> + <p>Exit by entering the command <c>halt()</c>,</p> + <pre> + 2> <input>halt().</input></pre> + <p>which will close the Erlang/OTP shell. </p> + </item> + </list> + </section> </section> </chapter> diff --git a/system/doc/installation_guide/part.xml b/system/doc/installation_guide/part.xml index 19808fd165..02bf98db7c 100644 --- a/system/doc/installation_guide/part.xml +++ b/system/doc/installation_guide/part.xml @@ -4,7 +4,7 @@ <part xmlns:xi="http://www.w3.org/2001/XInclude"> <header> <copyright> - <year>2000</year><year>2013</year> + <year>2000</year><year>2014</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -32,9 +32,7 @@ <p>How to install Erlang/OTP on UNIX or Windows.</p> </description> <xi:include href="install-binary.xml"/> - <xi:include href="verification.xml"/> <xi:include href="INSTALL.xml"/> <xi:include href="INSTALL-CROSS.xml"/> <xi:include href="INSTALL-WIN32.xml"/> - <xi:include href="MARKDOWN.xml"/> </part>
\ No newline at end of file diff --git a/system/doc/installation_guide/verification.xml b/system/doc/installation_guide/verification.xml deleted file mode 100644 index 391ddfb7b8..0000000000 --- a/system/doc/installation_guide/verification.xml +++ /dev/null @@ -1,102 +0,0 @@ -<?xml version="1.0" encoding="utf-8" ?> -<!DOCTYPE chapter SYSTEM "chapter.dtd"> - -<chapter> - <header> - <copyright> - <year>2000</year><year>2013</year> - <holder>Ericsson AB. All Rights Reserved.</holder> - </copyright> - <legalnotice> - The contents of this file are subject to the Erlang Public License, - Version 1.1, (the "License"); you may not use this file except in - compliance with the License. You should have received a copy of the - Erlang Public License along with this software. If not, it can be - retrieved online at http://www.erlang.org/. - - Software distributed under the License is distributed on an "AS IS" - basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See - the License for the specific language governing rights and limitations - under the License. - - </legalnotice> - - <title>Installation Verification</title> - <prepared>Peter Högfeldt</prepared> - <responsible>Peter Högfeldt</responsible> - <docno></docno> - <approved>(Peter Högfeldt</approved> - <checked></checked> - <date>1997-05-26</date> - <rev>C</rev> - <file>verification.xml</file> - </header> - <p>This chapter is about verifying your installation by performing - a few simple tests to see that your system is properly installed.</p> - - <section> - <title>UNIX</title> - <list type="bulleted"> - <item> - <p>Start Erlang/OTP from the command line,</p> - <pre> - unix> erl</pre> - <p>Expect the following output:</p> - <pre> - Erlang (BEAM) emulator version 5.0.1 [threads] - - Eshell V5.0.1 (abort with ^G) - 1></pre> - </item> - <item> - <p>Start the GS-based toolbar from the Erlang shell, </p> - <pre> - 1> <input>toolbar:start().</input></pre> - <p>and check that the toolbar window pops up. - </p> - <p><em>Note:</em> The trailing full stop (<c>"."</c>) is an end marker - for all commands in the Erlang shell, and must be entered for a - command to begin execution.</p> - </item> - <item> - <p>Exit by entering the command <c>halt()</c>,</p> - <pre> - 2> <input>halt().</input></pre> - <p>which should end both the toolbar window and the command line window. </p> - </item> - </list> - </section> - - <section> - <title>Windows</title> - <list type="bulleted"> - <item> - <p>Start Erlang/OTP by double-clicking on the Erlang shortcut icon on the - desktop.</p> - <p>Expect a command line window to pop up with the following output,</p> - <pre> - Erlang (BEAM) emulator version 5.0.1 [threads] - - Eshell V5.0.1 (abort with ^G) - 1></pre> - </item> - <item> - <p>Start the GS-based toolbar from the Erlang shell, </p> - <pre> - 1> <input>toolbar:start().</input></pre> - <p>and check that the toolbar window pops up. - </p> - <p><em>Note:</em> The trailing full stop (<c>"."</c>) is an end marker - for all commands in the Erlang shell, and must be entered for a - command to begin execution.</p> - </item> - <item> - <p>Exit by entering the command <c>halt()</c>,</p> - <pre> - 2> <input>halt().</input></pre> - <p>which should end both the toolbar window and the command line window. </p> - </item> - </list> - </section> -</chapter> - diff --git a/system/doc/installation_guide/xmlfiles.mk b/system/doc/installation_guide/xmlfiles.mk index 3995c607af..c443334cd7 100644 --- a/system/doc/installation_guide/xmlfiles.mk +++ b/system/doc/installation_guide/xmlfiles.mk @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 2009-2013. All Rights Reserved. +# Copyright Ericsson AB 2009-2014. All Rights Reserved. # # The contents of this file are subject to the Erlang Public License, # Version 1.1, (the "License"); you may not use this file except in @@ -18,8 +18,6 @@ # INST_GUIDE_CHAPTER_FILES = \ install-binary.xml \ - verification.xml \ INSTALL.xml \ INSTALL-CROSS.xml \ - INSTALL-WIN32.xml \ - MARKDOWN.xml + INSTALL-WIN32.xml diff --git a/system/doc/reference_manual/data_types.xml b/system/doc/reference_manual/data_types.xml index 8c690d6b86..37c0db5ff7 100644 --- a/system/doc/reference_manual/data_types.xml +++ b/system/doc/reference_manual/data_types.xml @@ -190,6 +190,38 @@ adam </section> <section> + <title>Map</title> + <p>Compound data type with a variable number of key-value associations:</p> + <pre> +#{Key1=>Value1,...,KeyN=>ValueN}</pre> + <p>Each key-value association in the map is called an + <em>association pair</em>. The key and value parts of the pair are + called <em>elements</em>. The number of association pairs is said to be + the <em>size</em> of the map.</p> + <p>There exists a number of BIFs to manipulate maps.</p> + <p>Examples:</p> + <pre> +1> <input>M1 = #{name=>adam,age=>24,date=>{july,29}}.</input> +#{age => 24,date => {july,29},name => adam} +2> <input>maps:get(name,M1).</input> +adam +3> <input>maps:get(date,M1).</input> +{july,29} +4> <input>M2 = maps:update(age,25,M1).</input> +#{age => 25,date => {july,29},name => adam} +5> <input>map_size(M).</input> +3 +6> <input>map_size(#{}).</input> +0</pre> + <p>A collection of maps processing functions can be found in + the STDLIB module <seealso marker="stdlib:maps"><c>maps</c></seealso>.</p> + <p>Read more about <seealso marker="expressions#map_expressions">Maps</seealso>.</p> + <note> + <p>Maps are considered experimental during OTP 17.</p> + </note> + </section> + + <section> <title>List</title> <p>Compound data type with a variable number of terms.</p> <pre> diff --git a/system/doc/reference_manual/expressions.xml b/system/doc/reference_manual/expressions.xml index e9de3e006e..fa8f9b2e8f 100644 --- a/system/doc/reference_manual/expressions.xml +++ b/system/doc/reference_manual/expressions.xml @@ -56,7 +56,7 @@ Expr1 + Expr2</code> <marker id="term"></marker> <title>Terms</title> <p>The simplest form of expression is a term, that is an integer, - float, atom, string, list or tuple. + float, atom, string, list, map or tuple. The return value is the term itself.</p> </section> @@ -792,6 +792,245 @@ Expr1 -- Expr2</pre> </section> <section> + <marker id="map_expressions"></marker> + <title>Map Expressions</title> + <section> + <title>Creating Maps</title> + <p> + Constructing a new map is done by letting an expression <c>K</c> be associated with + another expression <c>V</c>: + </p> + <code>#{ K => V }</code> + <p> + New maps may include multiple associations at construction by listing every + association: + </p> + <code>#{ K1 => V1, .., Kn => Vn }</code> + <p> + An empty map is constructed by not associating any terms with each other: + </p> + <code>#{}</code> + <p> + All keys and values in the map are terms. Any expression is first evaluated and + then the resulting terms are used as <em>key</em> and <em>value</em> respectively. + </p> + <p> + Keys and values are separated by the <c>=></c> arrow and associations are + separated by <c>,</c>. + </p> + + <p> + Examples: + </p> + <code> +M0 = #{}, % empty map +M1 = #{a => <<"hello">>}, % single association with literals +M2 = #{1 => 2, b => b}, % multiple associations with literals +M3 = #{k => {A,B}}, % single association with variables +M4 = #{{"w", 1} => f()}. % compound key associated with an evaluated expression</code> + <p> + where, <c>A</c> and <c>B</c> are any expressions and <c>M0</c> through <c>M4</c> + are the resulting map terms. + </p> + <p> + If two matching keys are declared, the latter key will take precedence. + </p> + <p> + Example: + </p> + +<pre> +1> <input>#{1 => a, 1 => b}.</input> +#{1 => b } +2> <input>#{1.0 => a, 1 => b}.</input> +#{1 => b, 1.0 => a} +</pre> + <p> + The order in which the expressions constructing the keys and their + associated values are evaluated is not defined. The syntactic order of + the key-value pairs in the construction is of no relevance, except in + the above mentioned case of two matching keys. + </p> + </section> + + <section> + <title>Updating Maps</title> + <p> + Updating a map has similar syntax as constructing it. + </p> + <p> + An expression defining the map to be updated is put in front of the expression + defining the keys to be updated and their respective values. + </p> + <code>M#{ K => V }</code> + <p> + where <c>M</c> is a term of type map and <c>K</c> and <c>V</c> are any expression. + </p> + <p> + If key <c>K</c> does not match any existing key in the map, a new association + will be created from key <c>K</c> to value <c>V</c>. If key <c>K</c> matches + an existing key in map <c>M</c> its associated value will be replaced by the + new value <c>V</c>. In both cases the evaluated map expression will return a new map. + </p> + <p> + If <c>M</c> is not of type map an exception of type <c>badmap</c> is thrown. + </p> + <p> + To only update an existing value, the following syntax is used, + </p> + <code>M#{ K := V } </code> + <p> + where <c>M</c> is an term of type map, <c>V</c> is an expression and <c>K</c> + is an expression which evaluates to an existing key in <c>M</c>. + </p> + <p> + If key <c>K</c> does not match any existing keys in map <c>M</c> an exception + of type <c>badarg</c> will be triggered at runtime. If a matching key <c>K</c> + is present in map <c>M</c> its associated value will be replaced by the new + value <c>V</c> and the evaluated map expression returns a new map. + </p> + <p> + If <c>M</c> is not of type map an exception of type <c>badmap</c> is thrown. + </p> + <p> + Examples: + </p> + <code> +M0 = #{}, +M1 = M0#{a => 0}, +M2 = M1#{a => 1, b => 2}, +M3 = M2#{"function" => fun() -> f() end}, +M4 = M3#{a := 2, b := 3}. % 'a' and 'b' was added in `M1` and `M2`.</code> + <p> + where <c>M0</c> is any map. It follows that <c>M1 .. M4</c> are maps as well. + </p> + <p> + More Examples: + </p> +<pre> +1> <input>M = #{1 => a}.</input> +#{1 => a } +2> <input>M#{1.0 => b}.</input> +#{1 => a, 1.0 => b}. +3> <input>M#{1 := b}.</input> +#{1 => b} +4> <input>M#{1.0 := b}.</input> +** exception error: bad argument +</pre> + <p> + As in construction, the order in which the key and value expressions + are evaluated is not defined. The + syntactic order of the key-value pairs in the update is of no + relevance, except in the case where two keys match, in which + case the latter value is used. + </p> + </section> + + <section> + <title>Maps in Patterns</title> + <p> + Matching of key-value associations from maps is done in the following way: + </p> + + <code>#{ K := V } = M</code> + <p> + where <c>M</c> is any map. The key <c>K</c> has to be an expression with bound + variables or a literals, and <c>V</c> can be any pattern with either bound or + unbound variables. + </p> + <p> + If the variable <c>V</c> is unbound, it will be bound to the value associated + with the key <c>K</c>, which has to exist in the map <c>M</c>. If the variable + <c>V</c> is bound, it has to match the value associated with <c>K</c> in <c>M</c>. + </p> + <p> Example: </p> +<code> +1> <input>M = #{"tuple" => {1,2}}.</input> +#{"tuple" => {1,2}} +2> <input>#{"tuple" := {1,B}} = M.</input> +#{"tuple" => {1,2}} +3> <input>B.</input> +2.</code> + <p> + This will bind variable <c>B</c> to integer <c>2</c>. + </p> + <p> + Similarly, multiple values from the map may be matched: + </p> + <code>#{ K1 := V1, .., Kn := Vn } = M</code> + <p> + where keys <c>K1 .. Kn</c> are any expressions with literals or bound variables. If all + keys exist in map <c>M</c> all variables in <c>V1 .. Vn</c> will be matched to the + associated values of their respective keys. + </p> + <p> + If the matching conditions are not met, the match will fail, either with + </p> + <list> + <item> + a <c>badmatch</c> exception, if used in the context of the matching operator + as in the example, + </item> + <item> + or resulting in the next clause being tested in function heads and + case expressions. + </item> + </list> + <p> + Matching in maps only allows for <c>:=</c> as delimiters of associations. + The order in which keys are declared in matching has no relevance. + </p> + <p> + Duplicate keys are allowed in matching and will match each pattern associated + to the keys. + </p> + <code>#{ K := V1, K := V2 } = M</code> + <p> + Matching an expression against an empty map literal will match its type but + no variables will be bound: + </p> + <code>#{} = Expr</code> + <p> + This expression will match if the expression <c>Expr</c> is of type map, otherwise + it will fail with an exception <c>badmatch</c>. + </p> + <section> + <title>Matching syntax: Example with literals in function heads</title> + <p> + Matching of literals as keys are allowed in function heads. + </p> + <code> +%% only start if not_started +handle_call(start, From, #{ state := not_started } = S) -> +... + {reply, ok, S#{ state := start }}; + +%% only change if started +handle_call(change, From, #{ state := start } = S) -> +... + {reply, ok, S#{ state := changed }};</code> + </section> + </section> + <section> + <title>Maps in Guards</title> + <p> + Maps are allowed in guards as long as all sub-expressions are valid guard expressions. + </p> + <p> + Two guard BIFs handles maps: + </p> + <list> + <item> + <seealso marker="erts:erlang#is_map/1">is_map/1</seealso> + </item> + <item> + <seealso marker="erts:erlang#map_size/1">map_size/1</seealso> + </item> + </list> + </section> + </section> + + <section> <marker id="bit_syntax"></marker> <title>Bit Syntax Expressions</title> <code type="none"><![CDATA[<<>> @@ -1348,6 +1587,9 @@ end</pre> <cell align="left" valign="middle"><c>is_list/1</c></cell> </row> <row> + <cell align="left" valign="middle"><c>is_map/1</c></cell> + </row> + <row> <cell align="left" valign="middle"><c>is_number/1</c></cell> </row> <row> @@ -1398,6 +1640,9 @@ end</pre> <cell align="left" valign="middle"><c>length(List)</c></cell> </row> <row> + <cell align="left" valign="middle"><c>map_size(Map)</c></cell> + </row> + <row> <cell align="left" valign="middle"><c>node()</c></cell> </row> <row> diff --git a/system/doc/reference_manual/modules.xml b/system/doc/reference_manual/modules.xml index 9e5f4de385..f0ec7ef165 100644 --- a/system/doc/reference_manual/modules.xml +++ b/system/doc/reference_manual/modules.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2003</year><year>2013</year> + <year>2003</year><year>2014</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -53,10 +53,10 @@ fact(0) -> % | <pre> -Tag(Value).</pre> <p><c>Tag</c> must be an atom, while <c>Value</c> must be a literal - term. As a convenience in user-defined attributes, the literal term - <c>Value</c> the syntax <c>Name/Arity</c> - (where <c>Name</c> is an atom and <c>Arity</c> a positive integer) - will be translated to <c>{Name,Arity}</c>.</p> + term. As a convenience in user-defined attributes, if the literal term + <c>Value</c> has the syntax <c>Name/Arity</c> + (where <c>Name</c> is an atom and <c>Arity</c> a positive integer), + the term <c>Name/Arity</c> will be translated to <c>{Name,Arity}</c>.</p> <p>Any module attribute can be specified. The attributes are stored in the compiled code and can be retrieved by calling @@ -134,8 +134,21 @@ fact(0) -> % | standard behaviours <c>gen_server</c>, <c>gen_fsm</c>, <c>gen_event</c> or <c>supervisor</c>.</p> <p>The spelling <c>behavior</c> is also accepted.</p> - <p>Read more about behaviours and callback modules in OTP Design - Principles.</p> + <p>The callback functions of the module can be specified either + directly by the exported function <c>behaviour_info/1</c>:</p> + <pre> +behaviour_info(callbacks) -> Callbacks.</pre> + <p>or by a <c>-callback</c> attribute for each callback + function:</p> + <pre> +-callback Name(Arguments) -> Result.</pre> + <p>where <c>Arguments</c> is a list of zero or more arguments. + The <c>-callback</c> attribute is to be preferred since the + extra type information can be used by tools to produce + documentation or find discrepancies.</p> + <p>Read more about behaviours and callback modules in + <seealso marker="doc/design_principles:spec_proc#behaviours"> + OTP Design Principles</seealso>.</p> </section> <section> diff --git a/system/doc/reference_manual/processes.xml b/system/doc/reference_manual/processes.xml index 20bab1eb48..95ae0672ec 100644 --- a/system/doc/reference_manual/processes.xml +++ b/system/doc/reference_manual/processes.xml @@ -114,8 +114,8 @@ spawn(Module, Name, Args) -> pid() <p>Two processes can be <em>linked</em> to each other. A link between two processes <c>Pid1</c> and <c>Pid2</c> is created by <c>Pid1</c> calling the BIF <c>link(Pid2)</c> (or vice versa). - There also exists a number a <c>spawn_link</c> BIFs, which spawns - and links to a process in one operation.</p> + There also exist a number of <c>spawn_link</c> BIFs, which spawn + and link to a process in one operation.</p> <p>Links are bidirectional and there can only be one link between two processes. Repeated calls to <c>link(Pid)</c> have no effect.</p> <p>A link can be removed by calling the BIF <c>unlink(Pid)</c>.</p> diff --git a/system/doc/reference_manual/typespec.xml b/system/doc/reference_manual/typespec.xml index 71aec732cf..cc35c6eb21 100644 --- a/system/doc/reference_manual/typespec.xml +++ b/system/doc/reference_manual/typespec.xml @@ -100,6 +100,7 @@ | Fun | Integer | List + | Map | Tuple | Union | UserDefined %% described in Section 6.3 @@ -126,10 +127,17 @@ | nonempty_improper_list(Type1, Type2) %% Type1 and Type2 as above | nonempty_list(Type) %% Proper non-empty list + Map :: map() %% stands for a map of any size + | #{} %% stands for a map of any size + | #{PairList} + Tuple :: tuple() %% stands for a tuple of any size | {} | {TList} + PairList :: Type => Type + | Type => Type, PairList + TList :: Type | Type, TList @@ -275,6 +283,10 @@ Records have been extended to possibly contain type information. This is described in the sub-section <seealso marker="#typeinrecords">"Type information in record declarations"</seealso> below. </p> + <note> + <p>Map types, both <c>map()</c> and <c>#{ ... }</c>, are considered experimental during OTP 17.</p> + <p>No type information of maps pairs, only the containing map types, are used by Dialyzer in OTP 17.</p> + </note> </section> <section> diff --git a/system/doc/system_architecture_intro/sys_arch_intro.xml b/system/doc/system_architecture_intro/sys_arch_intro.xml index 62add510ca..3e88548861 100644 --- a/system/doc/system_architecture_intro/sys_arch_intro.xml +++ b/system/doc/system_architecture_intro/sys_arch_intro.xml @@ -150,7 +150,7 @@ <item>Chapter 8: "Operation and Management Principles" describes the model for operation and maintenance of sub-systems.</item> <item>Chapter 9: "Tutorial" gives an orientation of the different interoperability mechanism, which can be used when integrating an - Erlang program with a program written in an other programming language.</item> + Erlang program with a program written in another programming language.</item> </list> </section> diff --git a/system/doc/system_principles/create_target.xmlsrc b/system/doc/system_principles/create_target.xmlsrc index fbc935d708..b5f8d8ac4d 100644 --- a/system/doc/system_principles/create_target.xmlsrc +++ b/system/doc/system_principles/create_target.xmlsrc @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2002</year><year>2013</year> + <year>2002</year><year>2014</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -21,7 +21,7 @@ </legalnotice> - <title>Creating a First Target System</title> + <title>Creating and Upgrading a Target System</title> <prepared>Peter Högfeldt</prepared> <responsible></responsible> <docno></docno> @@ -71,23 +71,24 @@ </section> <section> + <marker id="create"/> <title>Creating a Target System</title> <p>It is assumed that you have a working Erlang/OTP system structured according to the OTP Design Principles.</p> - <p><em>Step 1.</em> First create a <c>.rel</c> file (see - <c>rel(4)</c>) that specifies the <c>erts</c> version - and lists all applications that should be included in the new - basic target system. An example is the following + <p><em>Step 1.</em> First create a <c>.rel</c> file (see <seealso + marker="sasl:rel">rel(4)</seealso>) that specifies the <c>erts</c> + version and lists all applications that should be included in the + new basic target system. An example is the following <c>mysystem.rel</c> file:</p> <code type="none"> %% mysystem.rel {release, {"MYSYSTEM", "FIRST"}, - {erts, "5.1"}, - [{kernel, "2.7"}, - {stdlib, "1.10"}, - {sasl, "1.9.3"}, - {pea, "1.0"}]}. </code> + {erts, "5.10.4"}, + [{kernel, "2.16.4"}, + {stdlib, "1.19.4"}, + {sasl, "2.3.4"}, + {pea, "1.0"}]}.</code> <p>The listed applications are not only original Erlang/OTP applications but possibly also new applications that you have written yourself (here examplified by the application @@ -116,13 +117,13 @@ os> <input>erl -pa /home/user/target_system/myapps/pea-1.0/ebin</input></pre> <c>systools:make_tar/2</c>. That file has the following contents:</p> <code type="none"> -erts-5.1/bin/ +erts-5.10.4/bin/ releases/FIRST/start.boot releases/FIRST/mysystem.rel releases/mysystem.rel -lib/kernel-2.7/ -lib/stdlib-1.10/ -lib/sasl-1.9.3/ +lib/kernel-2.16.4/ +lib/stdlib-1.19.4/ +lib/sasl-2.3.4/ lib/pea-1.0/ </code> <p>The file <c>releases/FIRST/start.boot</c> is a copy of our <c>mysystem.boot</c></p> @@ -142,16 +143,19 @@ lib/pea-1.0/ </code> <item>Creates the temporary directory <c>tmp</c> and extracts the tar file <c>mysystem.tar.gz</c> into that directory. </item> <item>Deletes the <c>erl</c> and <c>start</c> files from - <c>tmp/erts-5.1/bin</c>. These files will be created again from + <c>tmp/erts-5.10.4/bin</c>. These files will be created again from source when installing the release.</item> <item>Creates the directory <c>tmp/bin</c>.</item> <item>Copies the previously created file <c>plain.boot</c> to <c>tmp/bin/start.boot</c>.</item> <item>Copies the files <c>epmd</c>, <c>run_erl</c>, and - <c>to_erl</c> from the directory <c>tmp/erts-5.1/bin</c> to + <c>to_erl</c> from the directory <c>tmp/erts-5.10.4/bin</c> to the directory <c>tmp/bin</c>.</item> + <item>Creates the directory <c>tmp/log</c>, which will be used + if the system is started as embedded with the <c>bin/start</c> + script.</item> <item>Creates the file <c>tmp/releases/start_erl.data</c> with - the contents "5.1 FIRST". This file is to be passed as data + the contents "5.10.4 FIRST". This file is to be passed as data file to the <c>start_erl</c> script. </item> <item>Recreates the file <c>mysystem.tar.gz</c> from the directories @@ -171,11 +175,11 @@ lib/pea-1.0/ </code> <item>Extracts the tar file <c>mysystem.tar.gz</c> into the target directory <c>/usr/local/erl-target</c>.</item> <item>In the target directory reads the file <c>releases/start_erl.data</c> - in order to find the Erlang runtime system version ("5.1").</item> + in order to find the Erlang runtime system version ("5.10.4").</item> <item>Substitutes <c>%FINAL_ROOTDIR%</c> and <c>%EMU%</c> for <c>/usr/local/erl-target</c> and <c>beam</c>, respectively, in the files <c>erl.src</c>, <c>start.src</c>, and - <c>start_erl.src</c> of the target <c>erts-5.1/bin</c> + <c>start_erl.src</c> of the target <c>erts-5.10.4/bin</c> directory, and puts the resulting files <c>erl</c>, <c>start</c>, and <c>run_erl</c> in the target <c>bin</c> directory.</item> @@ -185,6 +189,7 @@ lib/pea-1.0/ </code> </section> <section> + <marker id="start"/> <title>Starting a Target System</title> <p>Now we have a target system that can be started in various ways.</p> <p>We start it as a <em>basic target system</em> by invoking</p> @@ -193,7 +198,7 @@ os> <input>/usr/local/erl-target/bin/erl</input></pre> <p>where only the <c>kernel</c> and <c>stdlib</c> applications are started, i.e. the system is started as an ordinary development system. There are only two files needed for all this to work: - <c>bin/erl</c> file (obtained from <c>erts-5.1/bin/erl.src</c>) + <c>bin/erl</c> file (obtained from <c>erts-5.10.4/bin/erl.src</c>) and the <c>bin/start.boot</c> file (a copy of <c>plain.boot</c>).</p> <p>We can also start a distributed system (requires <c>bin/epmd</c>).</p> <p>To start all applications specified in the original @@ -208,9 +213,10 @@ os> <input>/usr/local/erl-target/bin/erl -boot /usr/local/erl-target/releases/FI <c>bin/run_erl</c>, which in turn calls <c>bin/start_erl</c> (roughly, <c>start_erl</c> is an embedded variant of <c>erl</c>). </p> - <p>The shell script <c>start</c> is only an example. You should - edit it to suite your needs. Typically it is executed when the - UNIX system boots.</p> + <p>The shell script <c>start</c>, which is generated from + erts-5.10.4/bin/start.src during installation, is only an + example. You should edit it to suite your needs. Typically it is + executed when the UNIX system boots.</p> <p><c>run_erl</c> is a wrapper that provides logging of output from the run-time system to file. It also provides a simple mechanism for attaching to the Erlang shell (<c>to_erl</c>).</p> @@ -218,7 +224,7 @@ os> <input>/usr/local/erl-target/bin/erl -boot /usr/local/erl-target/releases/FI (<c>"/usr/local/erl-target"</c>), the releases directory (<c>"/usr/local/erl-target/releases"</c>), and the location of the <c>start_erl.data</c> file. It reads the run-time system - version (<c>"5.1"</c>) and release version (<c>"FIRST"</c>) from + version (<c>"5.10.4"</c>) and release version (<c>"FIRST"</c>) from the <c>start_erl.data</c> file, starts the run-time system of the version found, and provides <c>-boot</c> flag specifying the boot file of the release version found @@ -258,6 +264,198 @@ os> <input>/usr/local/erl-target/bin/erl -boot /usr/local/erl-target/releases/FI </section> <section> + <title>Creating the Next Version</title> + + <p> + In this example the <c>pea</c> application has been changed, and + so are <c>erts</c>, <c>kernel</c>, <c>stdlib</c> and + <c>sasl</c>. + </p> + + <p> + <em>Step 1.</em> Create the <c>.rel</c> file: + </p> + <code type="none"> +%% mysystem2.rel +{release, + {"MYSYSTEM", "SECOND"}, + {erts, "6.0"}, + [{kernel, "3.0"}, + {stdlib, "2.0"}, + {sasl, "2.4"}, + {pea, "2.0"}]}.</code> + <p> + <em>Step 2.</em> Create the application upgrade file (see + <seealso marker="sasl:appup">appup(4)</seealso>) for <c>pea</c>, + for example: + </p> + <code type="none"> +%% pea.appup +{"2.0", + [{"1.0",[{load_module,pea_lib}]}], + [{"1.0",[{load_module,pea_lib}]}]}.</code> + <p> + <em>Step 3.</em> From the directory where the + <c>mysystem2.rel</c> file reside, start the Erlang/OTP system: + </p> + <pre> +os> <input>erl -pa /home/user/target_system/myapps/pea-2.0/ebin</input></pre> + <p>giving the path to the new version of <c>pea</c>. </p> + + <p> + <em>Step 4.</em> Create the release upgrade file (see <seealso + marker="sasl:relup">relup(4)</seealso>): + </p> + <pre> +1> <input>systools:make_relup("mysystem2",["mysystem"],["mysystem"],[{path,["/home/user/target_system/myapps/pea-1.0/ebin","/my/old/erlang/lib/*/ebin"]}]).</input></pre> + <p> + where <c>"mysystem"</c> is the base release and + <c>"mysystem2"</c> is the release to upgrade to. + </p> + <p> + Note that the <c>path</c> option is used for pointing out the + old version of all applications. (The new versions are already + in the code path - assuming of course that the erlang node on + which this is executed is running the correct version of + Erlang/OTP.) + </p> + <p> + <em>Step 5.</em> Create the new release: + </p> + <pre> +2> <input>target_system:create("mysystem2").</input></pre> + <p> + Given that the <c>relup</c> file generated in step 4 above is + now located in the current directory, it will automatically be + included in the release package. + </p> + </section> + + <section> + <title>Upgrading the Target System</title> + <p> + This part is done on the target node, and for this example we + want the node to be running as an embedded system with the + <c>-heart</c> option, allowing automatic restart of the + node. See <seealso marker="#start">Starting a Target + System</seealso> above for more information. + </p> + <p> + We add <c>-heart</c> to <c>bin/start</c>: + </p> + <code type="none"> +#!/bin/sh +ROOTDIR=/usr/local/erl-target/ + +if [ -z "$RELDIR" ] +then + RELDIR=$ROOTDIR/releases +fi + +START_ERL_DATA=${1:-$RELDIR/start_erl.data} + +$ROOTDIR/bin/run_erl -daemon /tmp/ $ROOTDIR/log "exec $ROOTDIR/bin/start_erl $ROOTDIR $RELDIR $START_ERL_DATA -heart</code> + <p> + And we use the simplest possible <c>sys.config</c>, which we + store in <c>releases/FIRST</c>: + </p> + <code type="none"> +%% sys.config +[].</code> + <p> + Finally, in order to prepare the upgrade, we need to put the new + release package in the <c>releases</c> directory of the first + target system: + </p> + <pre> +os> <input>cp mysystem2.tar.gz /usr/local/erl-target/releases</input></pre> + <p> + And assuming that the node has been started like this: + </p> + <pre> +os> <input>/usr/local/erl-target/bin/start</input></pre> + <p> + it can be accessed like this: + </p> + <pre> +os> <input>/usr/local/erl-target/bin/to_erl /tmp/erlang.pipe.1</input></pre> + <p> + Also note that logs can be found in + <c>/usr/local/erl-target/log</c>. This directory is specified as + an argument to <c>run_erl</c>in the start script listed above. + </p> + <p> + <em>Step 1.</em> Unpack the release: + </p> + <pre> +1> <input>{ok,Vsn} = release_handler:unpack_release("mysystem2").</input></pre> + <p> + <em>Step 2.</em> Install the release: + </p> + <pre> +2> <input>release_handler:install_release(Vsn).</input> +<output>{continue_after_restart,"FIRST",[]} +heart: Tue Apr 1 12:15:10 2014: Erlang has closed. +heart: Tue Apr 1 12:15:11 2014: Executed "/usr/local/erl-target/bin/start /usr/local/erl-target/releases/new_start_erl.data" -> 0. Terminating. +[End]</output></pre> + <p> + The above return value and output after the call to + <c>release_handler:install_release/1</c> means that the + <c>release_handler</c> has restarted the node by using + <c>heart</c>. This will always be done when the upgrade involves + a change of <c>erts</c>, <c>kernel</c>, <c>stdlib</c> or + <c>sasl</c>. See <seealso marker="upgrade">Upgrade when + Erlang/OTP has Changed</seealso> for more infomation about this. + </p> + <p> + The node will be accessable via a new pipe: + </p> + <pre> +os> <input>/usr/local/erl-target/bin/to_erl /tmp/erlang.pipe.2</input></pre> + <p> + Let's see which releases we have in our system: + </p> + <pre> +1> <input>release_handler:which_releases().</input> +<output>[{"MYSYSTEM","SECOND", + ["kernel-3.0","stdlib-2.0","sasl-2.4","pea-2.0"], + current}, + {"MYSYSTEM","FIRST", + ["kernel-2.16.4","stdlib-1.19.4","sasl-2.3.4","pea-1.0"], + permanent}]</output></pre> + <p> + Our new release, "SECOND", is now the current release, but we + can also see that our "FIRST" release is still permanent. This + means that if the node would be restarted at this point, it + would come up running the "FIRST" release again. + </p> + <p> + <em>Step 3.</em> Make the new release permanent: + </p> + <pre> +2> <input>release_handler:make_permanent("SECOND").</input></pre> + + <p> + Now look at the releases again: + </p> + + <pre> +3> <input>release_handler:which_releases().</input> +<output>[{"MYSYSTEM","SECOND", + ["kernel-3.0","stdlib-2.0","sasl-2.4","pea-2.0"], + permanent}, + {"MYSYSTEM","FIRST", + ["kernel-2.16.4","stdlib-1.19.4","sasl-2.3.4","pea-1.0"], + old}]</output></pre> + + <p> + Here we see that the new release version is <c>permanent</c>, so + it would be safe to restart the node. + </p> + + </section> + + <section> <title>Listing of target_system.erl</title> <p>This module can also be found in the <c>examples</c> directory of the <c>sasl</c> application.</p> diff --git a/system/doc/system_principles/part.xml b/system/doc/system_principles/part.xml index 811428baae..d05c89fde8 100644 --- a/system/doc/system_principles/part.xml +++ b/system/doc/system_principles/part.xml @@ -4,7 +4,7 @@ <part xmlns:xi="http://www.w3.org/2001/XInclude"> <header> <copyright> - <year>1997</year><year>2013</year> + <year>1997</year><year>2014</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -31,6 +31,6 @@ <xi:include href="system_principles.xml"/> <xi:include href="error_logging.xml"/> <xi:include href="create_target.xml"/> + <xi:include href="upgrade.xml"/> <xi:include href="versions.xml"/> </part> - diff --git a/system/doc/system_principles/system_principles.xml b/system/doc/system_principles/system_principles.xml index 4f2202fdd1..70c69b1dab 100644 --- a/system/doc/system_principles/system_principles.xml +++ b/system/doc/system_principles/system_principles.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>1996</year><year>2013</year> + <year>1996</year><year>2014</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -34,13 +34,12 @@ <p>An Erlang runtime system is started with the command <c>erl</c>:</p> <pre> % <input>erl</input> -Erlang (BEAM) emulator version 5.2.3.5 [hipe] [threads:0] +Erlang/OTP 17 [erts-6.0] [hipe] [smp:8:8] -Eshell V5.2.3.5 (abort with ^G) +Eshell V6.0 (abort with ^G) 1> </pre> <p><c>erl</c> understands a number of command line arguments, see - <c>erl(1)</c>. A number of them are also described in this - chapter.</p> + <c>erl(1)</c>. A number of them are also described in this chapter.</p> <p>Application programs can access the values of the command line arguments by calling one of the functions <c>init:get_argument(Key)</c>, or <c>init:get_arguments()</c>. diff --git a/system/doc/system_principles/upgrade.xml b/system/doc/system_principles/upgrade.xml new file mode 100644 index 0000000000..68e48da0b8 --- /dev/null +++ b/system/doc/system_principles/upgrade.xml @@ -0,0 +1,118 @@ +<?xml version="1.0" encoding="utf-8" ?> +<!DOCTYPE chapter SYSTEM "chapter.dtd"> + +<chapter> + <header> + <copyright> + <year>2014</year> + <holder>Ericsson AB. All Rights Reserved.</holder> + </copyright> + <legalnotice> + The contents of this file are subject to the Erlang Public License, + Version 1.1, (the "License"); you may not use this file except in + compliance with the License. You should have received a copy of the + Erlang Public License along with this software. If not, it can be + retrieved online at http://www.erlang.org/. + + Software distributed under the License is distributed on an "AS IS" + basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + the License for the specific language governing rights and limitations + under the License. + + </legalnotice> + + <title>Upgrade when Erlang/OTP has Changed</title> + <prepared></prepared> + <responsible></responsible> + <docno></docno> + <approved></approved> + <checked></checked> + <date>2014-02-19</date> + <rev></rev> + <file>upgrade.xml</file> + </header> + + <section> + <title>Introduction</title> + <p> + As of Erlang/OTP 17, most applications deliver a valid + application upgrade (<c>appup</c>) file. In earlier releases, a + majority of the applications in Erlang/OTP did not support + upgrade at all. Many of the applications use the + <c>restart_application</c> instruction. These are applications + for which it is not crucial to support real soft upgrade, for + instance tools and library applications. The + <c>restart_application</c> instruction + ensures that all modules in the application are reloaded and + thereby running the new code. + </p> + </section> + + <section> + <title>Upgrade of core applications</title> + <p> + The core applications ERTS, Kernel, STDLIB + and SASL never allow real soft upgrade, but require the + Erlang emulator to be restarted. This is indicated to the + <c>release_handler</c> by the upgrade instruction + <c>restart_new_emulator</c>. This instruction will always be the + very first instruction executed, and it will restart the + emulator with the new versions of the above mentioned core + applications and the old versions of all other + applications. When the node is back up all other upgrade instructions are + executed, making sure each application is finally running its + new version. + </p> + + <p> + It might seem strange to do a two-step upgrade instead of + just restarting the emulator with the new version of all + applications. The reason for this design decision is to allow + <c>code_change</c> functions to have side effects, for example changing + data on disk. It also makes sure that the upgrade mechanism for + non-core applications does not differ depending on whether or not + core applications are changed at the same time. + </p> + + <p> + If, however, the more brutal variant is preferred, it is + possible to handwrite the release upgrade file using only the + single upgrade instruction <c>restart_emulator</c>. This + instruction, in contrast to <c>restart_new_emulator</c>, will + cause the emulator to restart with the new versions of + <em>all</em> applications. + </p> + + <p> + <em>Note</em> that if other instructions are included before + <c>restart_emulator</c> in the handwritten <c>relup</c> file, + they will be executed in the old emulator. This is a big risk + since there is no guarantee that new beam code can be loaded + into the old emulator. Adding instructions after + <c>restart_emulator</c> has no effect as the + <c>release_handler</c> will not do any attempt at executing + them. + </p> + + <p> + See <seealso marker="sasl:relup">relup(4)</seealso> for + information about the release upgrade file, and <seealso + marker="sasl:appup">appup(4)</seealso> for further information + about upgrade instructions. + </p> + </section> + + <section> + <title>Applications that still do not allow code upgrade</title> + <p> + A few applications, for instance HiPE do not support + upgrade at all. This is indicated by an application upgrade file + containing only <c>{Vsn,[],[]}</c>. Any attempt at creating a release + upgrade file with such input will fail. + The only way to force an upgrade involving applications like this is to + handwrite the <c>relup</c> file, preferably as described above + with only the <c>restart_emulator</c> instruction. + </p> + + </section> +</chapter> diff --git a/system/doc/system_principles/versions.xml b/system/doc/system_principles/versions.xml index 2bf0d18010..ff042f4a3b 100644 --- a/system/doc/system_principles/versions.xml +++ b/system/doc/system_principles/versions.xml @@ -61,14 +61,13 @@ <c>filename:join([<seealso marker="kernel:code#root_dir/0">code:root_dir()</seealso>, "releases", <seealso marker="erts:erlang#system_info_otp_release">erlang:system_info(otp_release)</seealso>, "OTP_VERSION"]).</c></p> <p>If the version read from the <c>OTP_VERSION</c> file in a development system has a <c>**</c> suffix, the system has been - patched using the - <seealso marker="doc/installation_guide:PATCH-APP"><c>$ERL_TOP/otp_build patch_app</c></seealso> - tool. In this case, the system consists of application versions from - multiple OTP versions. The version preceding the <c>**</c> + patched using the <c>otp_patch_apply</c> tool available to + licensed customers. In this case, the system consists of application + versions from multiple OTP versions. The version preceding the <c>**</c> suffix corresponds to the OTP version of the base system that has been patched. Note that if a development system is updated by - other means than <c>$ERL_TOP/otp_build patch_app</c>, the - <c>OTP_VERSION</c> file may identify wrong OTP version.</p> + other means than <c>otp_patch_apply</c>, the <c>OTP_VERSION</c> file + may identify an incorrect OTP version.</p> <p>No <c>OTP_VERSION</c> file will be placed in a <seealso marker="create_target">target system</seealso> created diff --git a/system/doc/system_principles/xmlfiles.mk b/system/doc/system_principles/xmlfiles.mk index 9743949798..c60ffbad28 100644 --- a/system/doc/system_principles/xmlfiles.mk +++ b/system/doc/system_principles/xmlfiles.mk @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 2009. All Rights Reserved. +# Copyright Ericsson AB 2009-2014. All Rights Reserved. # # The contents of this file are subject to the Erlang Public License, # Version 1.1, (the "License"); you may not use this file except in @@ -20,4 +20,5 @@ SYSTEM_PRINCIPLES_CHAPTER_FILES = \ system_principles.xml \ error_logging.xml \ create_target.xml \ + upgrade.xml \ versions.xml diff --git a/xcomp/erl-xcomp-powerpc-ose5.conf b/xcomp/erl-xcomp-powerpc-ose5.conf index 60b75b36c1..bf49b32827 100644 --- a/xcomp/erl-xcomp-powerpc-ose5.conf +++ b/xcomp/erl-xcomp-powerpc-ose5.conf @@ -338,8 +338,14 @@ erl_xcomp_ose_LM_ELF_SIZE="${OSEROOT}/bin/${HOST}/lm_elf_size" # * `erl_xcomp_ose_LM_LCF` - OSE load module linker configuration file erl_xcomp_ose_LM_LCF="${ERL_TOP}/erts/emulator/sys/ose/gcc_${GCCVERSION}_lm_ppc.lcf" -# * `erl_xcomp_ose_LM_CONF` - OSE load module default configuration file -erl_xcomp_ose_LM_CONF="${ERL_TOP}/erts/emulator/sys/ose/default.lmconf" +# * `erl_xcomp_ose_BEAM_LM_CONF` - beam OSE load module configuration file +erl_xcomp_ose_BEAM_LM_CONF="${ERL_TOP}/erts/emulator/sys/ose/beam.lmconf" + +# * `erl_xcomp_ose_RUN_ERL_LM_CONF` - run_erl_lm OSE load module configuration file +erl_xcomp_ose_RUN_ERL_LM_CONF="${ERL_TOP}/erts/etc/ose/etc.lmconf" + +# * `erl_xcomp_ose_EPMD_LM_CONF` - epmd OSE load module configuration file +erl_xcomp_ose_EPMD_LM_CONF="${ERL_TOP}/erts/etc/ose/etc.lmconf" # * `erl_xcomp_ose_CONFD` - OSE confd source file erl_xcomp_ose_CONFD="${OSEROOT}/src/ose_confd.c" diff --git a/xcomp/erl-xcomp-vars.sh b/xcomp/erl-xcomp-vars.sh index c5e245b4b6..5e55c3af1b 100644 --- a/xcomp/erl-xcomp-vars.sh +++ b/xcomp/erl-xcomp-vars.sh @@ -26,4 +26,4 @@ # and precious variables in $ERL_TOP/erts/aclocal.m4. # -erl_xcomp_vars="erl_xcomp_sysroot erl_xcomp_isysroot erl_xcomp_bigendian erl_xcomp_double_middle_endian erl_xcomp_linux_clock_gettime_correction erl_xcomp_linux_nptl erl_xcomp_linux_usable_sigusrx erl_xcomp_linux_usable_sigaltstack erl_xcomp_poll erl_xcomp_kqueue erl_xcomp_putenv_copy erl_xcomp_reliable_fpe erl_xcomp_getaddrinfo erl_xcomp_gethrvtime_procfs_ioctl erl_xcomp_clock_gettime_cpu_time erl_xcomp_after_morecore_hook erl_xcomp_dlsym_brk_wrappers erl_xcomp_posix_memalign erl_xcomp_ose_ldflags_pass1 erl_xcomp_ose_ldflags_pass2 erl_xcomp_ose_OSEROOT erl_xcomp_ose_STRIP erl_xcomp_ose_LM_POST_LINK erl_xcomp_ose_LM_SET_CONF erl_xcomp_ose_LM_GET_CONF erl_xcomp_ose_LM_ELF_SIZE erl_xcomp_ose_LM_LCF erl_xcomp_ose_LM_CONF erl_xcomp_ose_CONFD erl_xcomp_ose_CRT0_LM" +erl_xcomp_vars="erl_xcomp_sysroot erl_xcomp_isysroot erl_xcomp_bigendian erl_xcomp_double_middle_endian erl_xcomp_linux_clock_gettime_correction erl_xcomp_linux_nptl erl_xcomp_linux_usable_sigusrx erl_xcomp_linux_usable_sigaltstack erl_xcomp_poll erl_xcomp_kqueue erl_xcomp_putenv_copy erl_xcomp_reliable_fpe erl_xcomp_getaddrinfo erl_xcomp_gethrvtime_procfs_ioctl erl_xcomp_clock_gettime_cpu_time erl_xcomp_after_morecore_hook erl_xcomp_dlsym_brk_wrappers erl_xcomp_posix_memalign erl_xcomp_ose_ldflags_pass1 erl_xcomp_ose_ldflags_pass2 erl_xcomp_ose_OSEROOT erl_xcomp_ose_STRIP erl_xcomp_ose_LM_POST_LINK erl_xcomp_ose_LM_SET_CONF erl_xcomp_ose_LM_GET_CONF erl_xcomp_ose_LM_ELF_SIZE erl_xcomp_ose_LM_LCF erl_xcomp_ose_BEAM_LM_CONF erl_xcomp_ose_EPMD_LM_CONF erl_xcomp_ose_RUN_ERL_LM_CONF erl_xcomp_ose_CONFD erl_xcomp_ose_CRT0_LM" |