diff options
1258 files changed, 17103 insertions, 8085 deletions
diff --git a/.gitignore b/.gitignore index 79b1d5dfad..0e9d07757f 100644 --- a/.gitignore +++ b/.gitignore @@ -119,7 +119,9 @@ JAVADOC-GENERATED /bootstrap/lib/common_test /bootstrap/lib/edoc /bootstrap/lib/erl_docgen +/bootstrap/lib/erl_interface /bootstrap/lib/hipe +/bootstrap/lib/jinterface /bootstrap/lib/parsetools /bootstrap/lib/runtime_tools /bootstrap/lib/sasl diff --git a/.travis.yml b/.travis.yml index 1438ea865a..ee724f8947 100644 --- a/.travis.yml +++ b/.travis.yml @@ -23,25 +23,67 @@ addons: matrix: include: + # Dialyzer is first as it takes the longest to run + - env: Linux64Dialyzer + addons: + apt: + packages: + # Don't build with wx, java or xslt to get faster build + - autoconf + - libncurses-dev + - build-essential + - libssl-dev + script: + - ./scripts/build-otp + - ./scripts/run-dialyzer - env: Linux32 - os: linux services: - docker script: - ./scripts/build-docker-otp 32 sh -c "scripts/build-otp release && ./otp_build tests && scripts/run-smoke-tests && bin/dialyzer --build_plt --apps erts kernel stdlib" - - env: Linux64Dialyzer - os: linux - script: - - ./scripts/build-otp - - ./scripts/run-dialyzer - env: Linux64SmokeTest - os: linux script: - ./scripts/build-otp - ./otp_build tests - - make release_docs - - make xmllint - ./scripts/run-smoke-tests + - env: Linux64Docbuild + script: + - ./scripts/build-otp docs + deploy: + provider: pages + repo: erlang/cd + target-branch: master + skip-cleanup: true + keep-history: true + verbose: true + github-token: $ERLANG_CD_GITHUB_TOKEN + on: + # We only deploy on pushes to branches + all_branches: true + tags: false + condition: $TRAVIS_PULL_REQUEST = "false" + repo: erlang/otp + # This stage publishes a otp bundle that contains multiple + # Erlang/OTP source repositories + - stage: deploy + env: Deploy + if: tag =~ ^OTP-[0-9]+\.[0-9]+$ + script: + - ./scripts/bundle-otp + deploy: + provider: releases + skip_cleanup: true + api_key: + secure: vW5PN6zng5H5+TCvwfwpGZsABrdCWYcFwDm3KXq+plsecBmTayu/0jgNso5Z97FbzDGVTLHWchvywEYQWnmrEByyOrqH73v1LN6JEfN99VpSrdFr15IzhblcyU1R9ugYc3WEoYjX0Q1uGelDSWRuuQOPbzy8mZf3D4rSGonyraP7jPTdHhs5P3ZWk6OMFz+tCdF4XohXqbhXIBOeH/EKg0svX2u5IcV01/YOL8LHWz6G7+gqBryEXx1+ngjQXQmMQwd7Yg3WOKE4XV9gX8ixZsbpUPZXAQKF+VOYdEgeiIr1hI0tBQUYX7FYEzYH5MCxqng5RdaPTOAm1oQroyGkIcWSXzDwN4AhJ7xqa/0NRdEaBPdQzPBCc+pVUDkxBR1ytXjBQqdQMnI6184TDiU5XBnj3kmieLkkKPKQNoPve/Y8Q8zutw4GNc7gixGcQCjtAFUbrT73QVRrezQH0qIdt23rivvf2R7CCOWSmgzowrswmtHdgeEVbodUIBPTNp7qzlUk9gDp6vW0XrOC4qEFI+VaY5PsEOXrrxZmI3gGGJgsbfzRvzvvupQcLNERniJ67r/uumbForpL0x1c65scKuMWwcn1wqt2OLbDoIIuM31Ph2HX/09TTqECU7CTvqLT5MnbZHXGjY9c3ch+sY3tSfaEX6aazl/Dqx28c7boCEw= + file: + - ${TRAVIS_TAG}-bundle.txt + - ${TRAVIS_TAG}-bundle.tar.gz + on: + # We only deploy on pushes to tags that match the regexp + tags: true + condition: $TRAVIS_TAG =~ ^OTP-[0-9]+\.[0-9]+$ + repo: erlang/otp + before_script: - set -e diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index c89301d0eb..96db464b52 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -43,6 +43,7 @@ for more information. You can contribute to Erlang/OTP by opening a Pull Request. Make sure you create a new branch for your pull request with `git checkout -b new-branch-name`. +Give the branch a short but descriptive name, like `stdlib/lists-length-fix`. Never do your work directly on `maint` or `master`. ## Fixing a bug @@ -108,6 +109,7 @@ conflicts or include the latest changes. compiled and that it works. * Check for unnecessary whitespace before committing with `git diff --check`. +However, do not fix preexisting whitespace errors in otherwise untouched source lines. Check your coding style: diff --git a/HOWTO/INSTALL-RASPBERRYPI3.md b/HOWTO/INSTALL-RASPBERRYPI3.md new file mode 100644 index 0000000000..536d095cb4 --- /dev/null +++ b/HOWTO/INSTALL-RASPBERRYPI3.md @@ -0,0 +1,323 @@ +# Cross Compiling Erlang/OTP - Raspberry Pi 3 + + +## Introduction + +This document describes how to build a toolchain and cross compile Erlang/OTP +to Raspberry Pi 3 on macOS High Sierra. It is recommended to consult +[Building and Installing Erlang/OTP](https://github.com/erlang/otp/blob/master/HOWTO/INSTALL.md) and [Cross Compiling Erlang/OTP](https://github.com/erlang/otp/blob/master/HOWTO/INSTALL-CROSS.md) before attempting to follow the instructions in this guide. + +The whole process takes several hours and depending on the package versions different problems may arise that require additional +fixes not described in this document. In other words, it is not fun to build a toolchain. I assume that you have a Mac and would +like to develop Erlang/OTP applications based on the latest OTP release (or master) that not yet released as a binary for +Raspberry Pi 3. + +The first and most time consuming step is building the toolchain from scratch. Once your cross compiler is ready you cross compile +all library dependencies and create the sysroot file system. In the last step you cross compile Erlang/OTP using the new +toolchain and sysroot. + +#### Tested Configuration + +macOS High Sierra 10.13.2<br> +Raspberry Pi Model B Rev 1.2<br> +Crosstools-NG 1.23.0_1 + +> Note: /proc/device/tree/model contains model information of your +> Raspberry Pi. + + +#### Install Crosstool NG + + (1) + + $ brew install crosstool-ng + $ brew install grep --default-names # needed by crosstools-ng scripts + $ brew install md5sha1sum # needed by crosstools-ng populate script + + +#### Create case-sensitive disk images + + (2) + +Create two case-sensitive disk images using Disk Utility: + +`File -> New Image -> Blank Image...` + +Format: `Mac OS Extended (Case-sensitive, Journaled)` + +``` +/Volumes/xtools-build-env 15 GB +/Volumes/xtools 500 MB +``` + +The first image holds all source and object files while building the toolchain. The second image houses the compiled +toolchain. + +## Building the Toolchain + + +#### Configure crosstool-ng + + (4) + + $ ct-ng armv8-rpi3-linux-gnueabihf + $ ct-ng menuconfig + +#### Modify *path* section + +* Local tarballs directory: `/Volumes/xtools-build-env/src` +* Working directory: `/Volumes/xtools-build-env/.build` +* Prefix directory: `/Volumes/xtools/${CT_TARGET}` + +#### Modify *Extracting* section + +* Check option: _Stop after extracting tarballs_. + +> Note: The build shall stop after the tarballs have been extracted to give us time to fix source code problems. + +#### Enable STOP / RESTART + +Edit /Volumes/xtools-build-env/.config + `CT_DEBUG_CT_SAVE_STEPS=y` + +Should the build break at a particular build step, you can fix the problem and continue the build from where it broke. + +Short summary of the most common `ct-ng` commands: + +* Listing all build steps + +``` + $ ct-ng list-steps + + Available build steps, in order: + - companion_tools_for_build + - companion_libs_for_build + - binutils_for_build + - companion_tools_for_host + - companion_libs_for_host + - binutils_for_host + - cc_core_pass_1 + - kernel_headers + - libc_start_files + - cc_core_pass_2 + - libc + - cc_for_build + - cc_for_host + - libc_post_cc + - companion_libs_for_target + - binutils_for_target + - debug + - test_suite + - finish +``` + +* Re-run step +``` + $ ct-ng step +``` + +* Restart from step +``` + $ ct-ng step+ +``` + +* Run until step +``` + $ ct-ng +step +``` + +#### Fix file permissions on crosstool-NG.sh + + (5) + + $ chmod 744 /usr/local/Cellar/crosstool-ng/1.23.0_1/lib/crosstool-ng-1.23.0/scripts/crosstool-NG.sh + +#### Run build command + +Build process stops just after the tarballs have been extracted. + + (6) + + $ ct-ng build + + Retrieving needed toolchain components' tarballs + [EXTRA] Retrieving 'make-4.2.1' + [EXTRA] Retrieving 'm4-1.4.18' + [EXTRA] Retrieving 'linux-4.10.8' + [EXTRA] Retrieving 'gmp-6.1.2' + [EXTRA] Retrieving 'mpfr-3.1.5' + [EXTRA] Retrieving 'isl-0.16.1' + [EXTRA] Retrieving 'mpc-1.0.3' + [EXTRA] Retrieving 'expat-2.2.0' + [EXTRA] Retrieving 'ncurses-6.0' + [EXTRA] Retrieving 'libiconv-1.15' + [EXTRA] Retrieving 'gettext-0.19.8.1' + [EXTRA] Retrieving 'binutils-2.28' + [EXTRA] Retrieving 'gcc-6.3.0' + [EXTRA] Retrieving 'glibc-2.25' + [EXTRA] Retrieving 'gdb-7.12.1' + +#### Fix source files + + (7) + +Add macro to /Volumes/xtools-build-env/.build/src/gdb-7.12.1/gdb/doublest.c: +```C +#define min(a,b) \ + ({ typeof (a) _a = (a); \ + typeof (b) _b = (b); \ + _a < _b ? _a : _b; }) +``` + + (8) Update ulimit + + $ ulimit -n 1024 + +#### Modify *extract* section + + (8) + + $ ct-ng menuconfig + + Uncheck option: _Stop after extracting tarballs_ + +#### Re-run build command + +Restarts build process from where it previously stopped. + + (9) + + $ ct-ng build + +#### Fix gettext + +Build will fail at step `companion_tools_for_build` but it can be fixed by running autoreconf: + + (10) + + $ cd .build/src/gettext-0.19.8.1/ + $ ./autoreconf + $ ct-ng companion_tools_for_build+ + +#### Test the toolchain + + (11) + + $ cat > test.c + $ int main() { printf("Hello, world!\n"); return 0; } + $ /Volumes/xtools/arm-unknown-linux-gnueabi-gcc -o test test.c + + (12) OPTIONAL + + “Render the toolchain read-only” from crosstool-NG’s “Paths and misc options” configuration page. + + +## Cross compiling dependencies + + (13) + + $ export PATH=/Volumes/xtools/armv8-rpi3-linux-gnueabihf/bin:$PATH + +#### Cross compiling zlib + + (14) + + $ wget http://zlib.net/zlib-1.2.11.tar.gz + $ tar xf zlib-1.2.11.tar.gz + $ cd zlib-1.2.11 + $ CHOST=armv8-rpi3-linux-gnueabihf ./configure --prefix=/Users/<username>/git/raspberrypi/arm + $ make + $ make install + +#### Cross compiling openssl + + (15) + + $ wget http://openssl.org/source/openssl-1.1.0g.tar.gz + $ tar xf openssl-1.1.0g.tar.gz + $ cd openssl-1.1.0g + $ ./Configure linux-generic32 --prefix=/Users/<username>/git/raspberrypi/arm --openssldir=/Users/<username>/git/raspberrypi/arm/openssl --cross-compile-prefix=armv8-rpi3-linux-gnueabihf + $ make + $ make install + +#### Cross compiling ncurses + + (16) + + $ wget http://ftp.gnu.org/pub/gnu/ncurses/ncurses-5.9.tar.gz + + + (17) + +Apply patch: + +```patch +--- a/ncurses/base/MKlib_gen.sh ++++ b/ncurses/base/MKlib_gen.sh +@@ -474,11 +474,22 @@ sed -n -f $ED1 \ + -e 's/gen_$//' \ + -e 's/ / /g' >>$TMP + ++cat >$ED1 <<EOF ++s/ / /g ++s/^ // ++s/ $// ++s/P_NCURSES_BOOL/NCURSES_BOOL/g ++EOF ++ ++# A patch discussed here: ++# https://gcc.gnu.org/ml/gcc-patches/2014-06/msg02185.html ++# introduces spurious #line markers. Work around that by ignoring the system's ++# attempt to define "bool" and using our own symbol here. ++sed -e 's/bool/P_NCURSES_BOOL/g' $TMP > $ED2 ++cat $ED2 >$TMP ++ + $preprocessor $TMP 2>/dev/null \ +-| sed \ +- -e 's/ / /g' \ +- -e 's/^ //' \ +- -e 's/_Bool/NCURSES_BOOL/g' \ ++| sed -f $ED1 \ + | $AWK -f $AW2 \ + | sed -f $ED3 \ + | sed \ +``` + + (18) + + $ ./configure --build=x86_64-apple-darwin17.3.0 --host=armv8-rpi3-linux-gnueabihf --without-ada --without-cxx --without-cxx-binding --without-manpages --without-progs --without-tests --prefix=/usr --libdir=/lib --with-build-cc="gcc -D_GNU_SOURCE" --with-shared + $ make + $ make DESTDIR=/Users/<username>/git/raspberrypi/arm install + + (19) + +Compile ncurses test program: + + $ cd test + $ armv8-rpi3-linux-gnueabihf-gcc -o nctest ncurses.c -I${RPI_SYSROOT}/usr/include -L${RPI_SYSROOT}/lib -lncursesw + + +## Populating sysroot + + (19) + + Edit /Volumes/xtools/armv8-rpi3-linux-gnueabihf/bin/armv8-rpi3-linux-gnueabihf-populate: + + sed="gsed" + + (20) + + $ armv8-rpi3-linux-gnueabihf-populate -s /Users/<username>/git/raspberrypi/arm -d /Users/<username>/git/raspberrypi/sysroot + $ export RPI_SYSROOT=/Users/<username>/git/raspberrypi/sysroot + + +## Cross compiling Erlang/OTP + + (21) + + $ LC_CTYPE=C && LANG=C && ./otp_build autoconf + $ ./otp_build configure --disable-dynamic-ssl-lib --xcomp-conf=./xcomp/erl-xcomp-armv8-rpi3-linux-gnueabihf.conf + $ ./otp_build boot -a + $ ./otp_build release -a /Users/<username>/git/raspberrypi/erlang + $ tar czf erlang.tgz ./erlang + diff --git a/HOWTO/INSTALL.md b/HOWTO/INSTALL.md index 2597794496..456dafeba5 100644 --- a/HOWTO/INSTALL.md +++ b/HOWTO/INSTALL.md @@ -77,8 +77,6 @@ also find the utilities needed for building the documentation. Download from <http://www.oracle.com/technetwork/java/javase/downloads>. We have also tested with IBM's JDK 1.6.0. -* X Windows -- Development headers and libraries are needed - to build the Erlang/OTP application `gs` on Unix/Linux. * `flex` -- Headers and libraries are needed to build the flex scanner for the `megaco` application on Unix/Linux. * wxWidgets -- Toolkit for GUI applications. diff --git a/Makefile.in b/Makefile.in index e75bcf7f10..d880bfefa2 100644 --- a/Makefile.in +++ b/Makefile.in @@ -351,6 +351,9 @@ is_cross_configured: target_configured: @echo @TARGET@ +erlang_inst_libdir_configured: + @echo $(ERLANG_INST_LIBDIR) + bootstrap: depend all_bootstraps @@ -681,6 +684,14 @@ tertiary_bootstrap_copy: $(V_at)if test ! -d $(BOOTSTRAP_ROOT)/bootstrap/lib/common_test/include ; then mkdir $(BOOTSTRAP_ROOT)/bootstrap/lib/common_test/include ; fi $(V_at)if test ! -d $(BOOTSTRAP_ROOT)/bootstrap/lib/runtime_tools ; then mkdir $(BOOTSTRAP_ROOT)/bootstrap/lib/runtime_tools ; fi $(V_at)if test ! -d $(BOOTSTRAP_ROOT)/bootstrap/lib/runtime_tools/include ; then mkdir $(BOOTSTRAP_ROOT)/bootstrap/lib/runtime_tools/include ; fi + $(V_at)if test ! -d $(BOOTSTRAP_ROOT)/bootstrap/lib/erl_interface ; then mkdir $(BOOTSTRAP_ROOT)/bootstrap/lib/erl_interface ; fi + $(V_at)if test ! -d $(BOOTSTRAP_ROOT)/bootstrap/lib/erl_interface/include ; then mkdir $(BOOTSTRAP_ROOT)/bootstrap/lib/erl_interface/include ; fi + $(V_at)if test ! -d $(BOOTSTRAP_ROOT)/bootstrap/lib/jinterface/ ; then mkdir $(BOOTSTRAP_ROOT)/bootstrap/lib/jinterface/ ; fi + $(V_at)if test ! -d $(BOOTSTRAP_ROOT)/bootstrap/lib/jinterface/priv ; then mkdir $(BOOTSTRAP_ROOT)/bootstrap/lib/jinterface/priv ; fi + $(V_at)if test ! -d $(BOOTSTRAP_ROOT)/bootstrap/lib/jinterface/priv/com ; then mkdir $(BOOTSTRAP_ROOT)/bootstrap/lib/jinterface/priv/com ; fi + $(V_at)if test ! -d $(BOOTSTRAP_ROOT)/bootstrap/lib/jinterface/priv/com/ericsson ; then mkdir $(BOOTSTRAP_ROOT)/bootstrap/lib/jinterface/priv/com/ericsson ; fi + $(V_at)if test ! -d $(BOOTSTRAP_ROOT)/bootstrap/lib/jinterface/priv/com/ericsson/otp ; then mkdir $(BOOTSTRAP_ROOT)/bootstrap/lib/jinterface/priv/com/ericsson/otp ; fi + $(V_at)if test ! -d $(BOOTSTRAP_ROOT)/bootstrap/lib/jinterface/priv/com/ericsson/otp/erlang ; then mkdir $(BOOTSTRAP_ROOT)/bootstrap/lib/jinterface/priv/com/ericsson/otp/erlang ; fi $(V_at)for x in lib/sasl/ebin/*.beam; do \ BN=`basename $$x`; \ TF=$(BOOTSTRAP_ROOT)/bootstrap/lib/sasl/ebin/$$BN; \ @@ -749,6 +760,38 @@ tertiary_bootstrap_copy: cp $$x $$TF; \ true; \ done +# copy erl_interface includes + $(V_at)for x in lib/erl_interface/include/*; do \ + BN=`basename $$x`; \ + TF=$(BOOTSTRAP_ROOT)/bootstrap/lib/erl_interface/include/$$BN; \ + test -f $$TF && \ + test '!' -z "`find $$x -newer $$TF -print`" && \ + cp $$x $$TF; \ + test '!' -f $$TF && \ + cp $$x $$TF; \ + true; \ + done +# copy jinterface priv directory + $(V_at)for x in lib/jinterface/priv/OtpErlang.jar; do \ + BN=`basename $$x`; \ + TF=$(BOOTSTRAP_ROOT)/bootstrap/lib/jinterface/priv/$$BN; \ + test -f $$TF && \ + test '!' -z "`find $$x -newer $$TF -print`" && \ + cp $$x $$TF; \ + test '!' -f $$TF && \ + cp $$x $$TF; \ + true; \ + done + $(V_at)for x in lib/jinterface/priv/com/ericsson/otp/erlang/*; do \ + BN=`basename $$x`; \ + TF=$(BOOTSTRAP_ROOT)/bootstrap/lib/jinterface/priv/com/ericsson/otp/erlang/$$BN; \ + test -f $$TF && \ + test '!' -z "`find $$x -newer $$TF -print`" && \ + cp $$x $$TF; \ + test '!' -f $$TF && \ + cp $$x $$TF; \ + true; \ + done # $(V_at)cp lib/syntax_tools/ebin/*.beam $(BOOTSTRAP_ROOT)/bootstrap/lib/syntax_tools/ebin doc_bootstrap_build: diff --git a/OTP_VERSION b/OTP_VERSION index a28399bab8..a12e109d04 100644 --- a/OTP_VERSION +++ b/OTP_VERSION @@ -1 +1 @@ -21.0-rc2 +22.0-rc0 diff --git a/bootstrap/bin/no_dot_erlang.boot b/bootstrap/bin/no_dot_erlang.boot Binary files differindex 60f70c0f97..127072b3ed 100644 --- a/bootstrap/bin/no_dot_erlang.boot +++ b/bootstrap/bin/no_dot_erlang.boot diff --git a/bootstrap/bin/start.boot b/bootstrap/bin/start.boot Binary files differindex 60f70c0f97..127072b3ed 100644 --- a/bootstrap/bin/start.boot +++ b/bootstrap/bin/start.boot diff --git a/bootstrap/bin/start_clean.boot b/bootstrap/bin/start_clean.boot Binary files differindex 60f70c0f97..127072b3ed 100644 --- a/bootstrap/bin/start_clean.boot +++ b/bootstrap/bin/start_clean.boot diff --git a/bootstrap/lib/kernel/ebin/application_controller.beam b/bootstrap/lib/kernel/ebin/application_controller.beam Binary files differindex e57be279d5..4f52e506c2 100644 --- a/bootstrap/lib/kernel/ebin/application_controller.beam +++ b/bootstrap/lib/kernel/ebin/application_controller.beam diff --git a/bootstrap/lib/kernel/ebin/error_logger.beam b/bootstrap/lib/kernel/ebin/error_logger.beam Binary files differindex bb250d71f3..8dd3c07f71 100644 --- a/bootstrap/lib/kernel/ebin/error_logger.beam +++ b/bootstrap/lib/kernel/ebin/error_logger.beam diff --git a/bootstrap/lib/kernel/ebin/kernel.app b/bootstrap/lib/kernel/ebin/kernel.app index 0e1cf7cbb7..340ed8653f 100644 --- a/bootstrap/lib/kernel/ebin/kernel.app +++ b/bootstrap/lib/kernel/ebin/kernel.app @@ -140,9 +140,8 @@ inet_db, pg2]}, {applications, []}, - {env, [{logger_level, info}, - {logger_sasl_compatible, false}, - {logger_progress_reports, stop} + {env, [{logger_level, notice}, + {logger_sasl_compatible, false} ]}, {mod, {kernel, []}}, {runtime_dependencies, ["erts-10.0", "stdlib-3.5", "sasl-3.0"]} diff --git a/bootstrap/lib/kernel/ebin/logger.beam b/bootstrap/lib/kernel/ebin/logger.beam Binary files differindex 7cac680071..e6d8f5b2c6 100644 --- a/bootstrap/lib/kernel/ebin/logger.beam +++ b/bootstrap/lib/kernel/ebin/logger.beam diff --git a/bootstrap/lib/kernel/ebin/logger_backend.beam b/bootstrap/lib/kernel/ebin/logger_backend.beam Binary files differindex 5f9b0edc58..be448bd75c 100644 --- a/bootstrap/lib/kernel/ebin/logger_backend.beam +++ b/bootstrap/lib/kernel/ebin/logger_backend.beam diff --git a/bootstrap/lib/kernel/ebin/logger_config.beam b/bootstrap/lib/kernel/ebin/logger_config.beam Binary files differindex 1033fc7c3d..52f7847561 100644 --- a/bootstrap/lib/kernel/ebin/logger_config.beam +++ b/bootstrap/lib/kernel/ebin/logger_config.beam diff --git a/bootstrap/lib/kernel/ebin/logger_disk_log_h.beam b/bootstrap/lib/kernel/ebin/logger_disk_log_h.beam Binary files differindex 8ae617404c..cc1b649eb7 100644 --- a/bootstrap/lib/kernel/ebin/logger_disk_log_h.beam +++ b/bootstrap/lib/kernel/ebin/logger_disk_log_h.beam diff --git a/bootstrap/lib/kernel/ebin/logger_formatter.beam b/bootstrap/lib/kernel/ebin/logger_formatter.beam Binary files differindex 6216a8095e..7f7d77ab28 100644 --- a/bootstrap/lib/kernel/ebin/logger_formatter.beam +++ b/bootstrap/lib/kernel/ebin/logger_formatter.beam diff --git a/bootstrap/lib/kernel/ebin/logger_h_common.beam b/bootstrap/lib/kernel/ebin/logger_h_common.beam Binary files differindex fe2b7a45eb..7d96f2d69e 100644 --- a/bootstrap/lib/kernel/ebin/logger_h_common.beam +++ b/bootstrap/lib/kernel/ebin/logger_h_common.beam diff --git a/bootstrap/lib/kernel/ebin/logger_server.beam b/bootstrap/lib/kernel/ebin/logger_server.beam Binary files differindex 381c9292b4..4885c0992a 100644 --- a/bootstrap/lib/kernel/ebin/logger_server.beam +++ b/bootstrap/lib/kernel/ebin/logger_server.beam diff --git a/bootstrap/lib/kernel/ebin/logger_std_h.beam b/bootstrap/lib/kernel/ebin/logger_std_h.beam Binary files differindex e883c81366..e8c3bb83d0 100644 --- a/bootstrap/lib/kernel/ebin/logger_std_h.beam +++ b/bootstrap/lib/kernel/ebin/logger_std_h.beam diff --git a/bootstrap/lib/kernel/ebin/net_kernel.beam b/bootstrap/lib/kernel/ebin/net_kernel.beam Binary files differindex 03627a3d53..0a565d542d 100644 --- a/bootstrap/lib/kernel/ebin/net_kernel.beam +++ b/bootstrap/lib/kernel/ebin/net_kernel.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_eval.beam b/bootstrap/lib/stdlib/ebin/erl_eval.beam Binary files differindex 7da699085f..eea70718c8 100644 --- a/bootstrap/lib/stdlib/ebin/erl_eval.beam +++ b/bootstrap/lib/stdlib/ebin/erl_eval.beam diff --git a/bootstrap/lib/stdlib/ebin/gen_event.beam b/bootstrap/lib/stdlib/ebin/gen_event.beam Binary files differindex bb1d4f69ba..ef8d0cd929 100644 --- a/bootstrap/lib/stdlib/ebin/gen_event.beam +++ b/bootstrap/lib/stdlib/ebin/gen_event.beam diff --git a/bootstrap/lib/stdlib/ebin/gen_fsm.beam b/bootstrap/lib/stdlib/ebin/gen_fsm.beam Binary files differindex 670c6c6c5d..3a4df3499c 100644 --- a/bootstrap/lib/stdlib/ebin/gen_fsm.beam +++ b/bootstrap/lib/stdlib/ebin/gen_fsm.beam diff --git a/bootstrap/lib/stdlib/ebin/gen_server.beam b/bootstrap/lib/stdlib/ebin/gen_server.beam Binary files differindex 68e6390cbe..1752276cc8 100644 --- a/bootstrap/lib/stdlib/ebin/gen_server.beam +++ b/bootstrap/lib/stdlib/ebin/gen_server.beam diff --git a/bootstrap/lib/stdlib/ebin/gen_statem.beam b/bootstrap/lib/stdlib/ebin/gen_statem.beam Binary files differindex 54e6427c24..1c1b93a036 100644 --- a/bootstrap/lib/stdlib/ebin/gen_statem.beam +++ b/bootstrap/lib/stdlib/ebin/gen_statem.beam diff --git a/bootstrap/lib/stdlib/ebin/io_lib_pretty.beam b/bootstrap/lib/stdlib/ebin/io_lib_pretty.beam Binary files differindex de468aacca..dc91c318c5 100644 --- a/bootstrap/lib/stdlib/ebin/io_lib_pretty.beam +++ b/bootstrap/lib/stdlib/ebin/io_lib_pretty.beam diff --git a/bootstrap/lib/stdlib/ebin/maps.beam b/bootstrap/lib/stdlib/ebin/maps.beam Binary files differindex 85e997bb1c..e24b5955d5 100644 --- a/bootstrap/lib/stdlib/ebin/maps.beam +++ b/bootstrap/lib/stdlib/ebin/maps.beam diff --git a/bootstrap/lib/stdlib/ebin/otp_internal.beam b/bootstrap/lib/stdlib/ebin/otp_internal.beam Binary files differindex ab99005d7a..c42b18f6cf 100644 --- a/bootstrap/lib/stdlib/ebin/otp_internal.beam +++ b/bootstrap/lib/stdlib/ebin/otp_internal.beam diff --git a/bootstrap/lib/stdlib/ebin/proc_lib.beam b/bootstrap/lib/stdlib/ebin/proc_lib.beam Binary files differindex b8602a6490..ce36bcf6fc 100644 --- a/bootstrap/lib/stdlib/ebin/proc_lib.beam +++ b/bootstrap/lib/stdlib/ebin/proc_lib.beam diff --git a/bootstrap/lib/stdlib/ebin/supervisor.beam b/bootstrap/lib/stdlib/ebin/supervisor.beam Binary files differindex c00a557688..939f385c93 100644 --- a/bootstrap/lib/stdlib/ebin/supervisor.beam +++ b/bootstrap/lib/stdlib/ebin/supervisor.beam diff --git a/bootstrap/lib/stdlib/ebin/supervisor_bridge.beam b/bootstrap/lib/stdlib/ebin/supervisor_bridge.beam Binary files differindex cb61f6e6f9..6cfaa984df 100644 --- a/bootstrap/lib/stdlib/ebin/supervisor_bridge.beam +++ b/bootstrap/lib/stdlib/ebin/supervisor_bridge.beam diff --git a/bootstrap/lib/stdlib/ebin/uri_string.beam b/bootstrap/lib/stdlib/ebin/uri_string.beam Binary files differindex e14164a823..36aec511d8 100644 --- a/bootstrap/lib/stdlib/ebin/uri_string.beam +++ b/bootstrap/lib/stdlib/ebin/uri_string.beam diff --git a/bootstrap/lib/stdlib/include/assert.hrl b/bootstrap/lib/stdlib/include/assert.hrl index 2ec89e7d8a..28d25c6589 100644 --- a/bootstrap/lib/stdlib/include/assert.hrl +++ b/bootstrap/lib/stdlib/include/assert.hrl @@ -140,7 +140,7 @@ -endif. %% This is mostly a convenience which gives more detailed reports. -%% Note: Guard is a guarded pattern, and can not be used for value. +%% Note: Guard is a guarded pattern, and cannot be used for value. -ifdef(NOASSERT). -define(assertMatch(Guard, Expr), ok). -define(assertMatch(Guard, Expr, Comment), ok). @@ -289,7 +289,7 @@ end). -endif. -%% Note: Class and Term are patterns, and can not be used for value. +%% Note: Class and Term are patterns, and cannot be used for value. %% Term can be a guarded pattern, but Class cannot. -ifdef(NOASSERT). -define(assertException(Class, Term, Expr), ok). @@ -364,7 +364,7 @@ ?assertException(throw, Term, Expr, Comment)). %% This is the inverse case of assertException, for convenience. -%% Note: Class and Term are patterns, and can not be used for value. +%% Note: Class and Term are patterns, and cannot be used for value. %% Both Class and Term can be guarded patterns. -ifdef(NOASSERT). -define(assertNotException(Class, Term, Expr), ok). diff --git a/configure.in b/configure.in index 559049aca1..c3a7e5018d 100644 --- a/configure.in +++ b/configure.in @@ -479,7 +479,7 @@ if test -f "erts/doc/CONF_INFO"; then fi done if test $havexsltproc = "no"; then - echo ' The documentation can not be built.' + echo ' The documentation cannot be built.' else echo ' Using fakefop to generate placeholder PDF files.' fi diff --git a/erts/aclocal.m4 b/erts/aclocal.m4 index 99b96eb5bc..3d227e462c 100644 --- a/erts/aclocal.m4 +++ b/erts/aclocal.m4 @@ -1,7 +1,7 @@ dnl dnl %CopyrightBegin% dnl -dnl Copyright Ericsson AB 1998-2016. All Rights Reserved. +dnl Copyright Ericsson AB 1998-2018. All Rights Reserved. dnl dnl Licensed under the Apache License, Version 2.0 (the "License"); dnl you may not use this file except in compliance with the License. diff --git a/erts/autoconf/configure.vxworks b/erts/autoconf/configure.vxworks index 18ca1718d6..a253848403 100755 --- a/erts/autoconf/configure.vxworks +++ b/erts/autoconf/configure.vxworks @@ -2,7 +2,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 1997-2016. All Rights Reserved. +# Copyright Ericsson AB 1997-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -113,10 +113,11 @@ CONFIG_FILES="${ERL_TOP}/erts/emulator/$host/Makefile $erlint_dir/$host/eidefs.mk $epmd_dir/$host/Makefile $internal_tools_dir/make/$host/otp.mk + $internal_tools_dir/make/$host/otp_ded.mk $os_mon_dir/$host/Makefile $zlibdir/$host/Makefile $runtime_tools_dir/$host/Makefile - $tools_dir/$host/Makefile + $tools_dir/$host/Makefile" for file in $CONFIG_FILES; do new_name=`echo $file|sed "s%/$host/%/$target/%"` diff --git a/erts/autoconf/vxworks/sed.general b/erts/autoconf/vxworks/sed.general index 96a70e4148..a30eb51169 100644 --- a/erts/autoconf/vxworks/sed.general +++ b/erts/autoconf/vxworks/sed.general @@ -43,6 +43,11 @@ s|@LDFLAGS@|| # FIXME: A bit strange to clear out remaining DED_* s|@DED_LDFLAGS@|| s|@DED_CFLAGS@|| +s|@DED_EMU_THR_DEFS@|| +s|@DED_THR_DEFS@|| +s|@DED_SYS_INCLUDE@|| +s|@WERRORFLAGS@|| +s|@DED_STATIC_CFLAGS@|| s|@STATIC_CFLAGS@|| s|@GCCLIB@|libgcc.a| s|@DEFS@|| diff --git a/erts/autoconf/vxworks/sed.vxworks_cpu32 b/erts/autoconf/vxworks/sed.vxworks_cpu32 index 71663676e7..961adc4104 100644 --- a/erts/autoconf/vxworks/sed.vxworks_cpu32 +++ b/erts/autoconf/vxworks/sed.vxworks_cpu32 @@ -28,6 +28,7 @@ s|@host@|vxworks_cpu32| s|@system_type@|vxworks_cpu32| s|@CC@|@TTPREFIX@cc68k| s|@HCC@|gcc| +s|@GCC@|yes| s|@LD@|@TTPREFIX@ld68k| s|@LIBS@|| s|@DED_LD@|@TTPREFIX@ld68k| diff --git a/erts/autoconf/vxworks/sed.vxworks_ppc32 b/erts/autoconf/vxworks/sed.vxworks_ppc32 index 2146e862fd..1e2e760abc 100644 --- a/erts/autoconf/vxworks/sed.vxworks_ppc32 +++ b/erts/autoconf/vxworks/sed.vxworks_ppc32 @@ -33,6 +33,7 @@ s|@system_type@|vxworks_ppc32| s|@ARCH@|ppc32| s|@CC@|GCC_EXEC_PREFIX=@WIND_BASE@/gnu/3.4.4-vxworks-6.3/@HOST_TYPE@/lib/gcc-lib/ @WIND_BASE@/gnu/3.4.4-vxworks-6.3/@HOST_TYPE@/bin/ccppc -mlongcall| s|@HCC@|gcc| +s|@GCC@|yes| s|@LD@|GCC_EXEC_PREFIX=@WIND_BASE@/gnu/3.4.4-vxworks-6.3/@HOST_TYPE@/lib/gcc-lib/ @WIND_BASE@/gnu/3.4.4-vxworks-6.3/@HOST_TYPE@/bin/ldppc| s|@STRIP@|GCC_EXEC_PREFIX=@WIND_BASE@/gnu/3.4.4-vxworks-6.3/@HOST_TYPE@/lib/gcc-lib/ @WIND_BASE@/workbench-2.3/@HOST_TYPE@/bin/stripppc| s|@SYMPREFIX@|| diff --git a/erts/autoconf/vxworks/sed.vxworks_ppc603 b/erts/autoconf/vxworks/sed.vxworks_ppc603 index fca1ba76d9..5beaae4fc9 100644 --- a/erts/autoconf/vxworks/sed.vxworks_ppc603 +++ b/erts/autoconf/vxworks/sed.vxworks_ppc603 @@ -29,6 +29,7 @@ s|@system_type@|vxworks_ppc603| s|@ARCH@|ppc603| s|@CC@|@TTPREFIX@ccppc -mlongcall| s|@HCC@|gcc| +s|@GCC@|yes| s|@LD@|@TTPREFIX@ldppc| s|@STRIP@|@TTPREFIX@stripppc| s|@SYMPREFIX@|| diff --git a/erts/autoconf/vxworks/sed.vxworks_ppc603_nolongcall b/erts/autoconf/vxworks/sed.vxworks_ppc603_nolongcall index 51c589d79a..2c2f2fa372 100644 --- a/erts/autoconf/vxworks/sed.vxworks_ppc603_nolongcall +++ b/erts/autoconf/vxworks/sed.vxworks_ppc603_nolongcall @@ -29,6 +29,7 @@ s|@system_type@|vxworks_ppc603| s|@ARCH@|ppc603| s|@CC@|@TTPREFIX@ccppc| s|@HCC@|gcc| +s|@GCC@|yes| s|@LD@|@TTPREFIX@ldppc| s|@STRIP@|@TTPREFIX@stripppc| s|@SYMPREFIX@|| diff --git a/erts/autoconf/vxworks/sed.vxworks_ppc860 b/erts/autoconf/vxworks/sed.vxworks_ppc860 index 485504e706..71cf887476 100644 --- a/erts/autoconf/vxworks/sed.vxworks_ppc860 +++ b/erts/autoconf/vxworks/sed.vxworks_ppc860 @@ -29,6 +29,7 @@ s|@system_type@|vxworks_ppc860| s|@ARCH@|ppc860| s|@CC@|@TTPREFIX@ccppc -mlongcall| s|@HCC@|gcc| +s|@GCC@|yes| s|@LD@|@TTPREFIX@ldppc| s|@STRIP@|@TTPREFIX@stripppc| s|@SYMPREFIX@|| diff --git a/erts/autoconf/vxworks/sed.vxworks_simlinux b/erts/autoconf/vxworks/sed.vxworks_simlinux index 10cd7bbb82..06b1847602 100644 --- a/erts/autoconf/vxworks/sed.vxworks_simlinux +++ b/erts/autoconf/vxworks/sed.vxworks_simlinux @@ -36,6 +36,7 @@ s|@ARCH@|simlinux| s|@CC@|GCC_EXEC_PREFIX=@WIND_BASE@/gnu/3.4.4-vxworks-6.3/@HOST_TYPE@/lib/gcc-lib/ @WIND_BASE@/gnu/3.4.4-vxworks-6.3/@HOST_TYPE@/bin/ccpentium| s|@HCC@|gcc| +s|@GCC@|yes| s|@LD@|GCC_EXEC_PREFIX=@WIND_BASE@/gnu/3.4.4-vxworks-6.3/@HOST_TYPE@/lib/gcc-lib/ @WIND_BASE@/gnu/3.4.4-vxworks-6.3/@HOST_TYPE@/bin/ldpentium| diff --git a/erts/autoconf/vxworks/sed.vxworks_simso b/erts/autoconf/vxworks/sed.vxworks_simso index cd30f8c2b2..07606cad80 100644 --- a/erts/autoconf/vxworks/sed.vxworks_simso +++ b/erts/autoconf/vxworks/sed.vxworks_simso @@ -36,6 +36,7 @@ s|@ARCH@|simso| s|@CC@|GCC_EXEC_PREFIX=@WIND_BASE@/gnu/3.4.4-vxworks-6.3/@HOST_TYPE@/lib/gcc-lib/ @WIND_BASE@/gnu/3.4.4-vxworks-6.3/@HOST_TYPE@/bin/ccsparc| s|@HCC@|gcc| +s|@GCC@|yes| # Tornado2.2: s|@LD@|@TTPREFIX@ldsimso| s|@LD@|GCC_EXEC_PREFIX=@WIND_BASE@/gnu/3.4.4-vxworks-6.3/@HOST_TYPE@/lib/gcc-lib/ @WIND_BASE@/gnu/3.4.4-vxworks-6.3/@HOST_TYPE@/bin/ldsparc| diff --git a/erts/autoconf/vxworks/sed.vxworks_sparc b/erts/autoconf/vxworks/sed.vxworks_sparc index a3758423e8..59431fddf9 100644 --- a/erts/autoconf/vxworks/sed.vxworks_sparc +++ b/erts/autoconf/vxworks/sed.vxworks_sparc @@ -30,6 +30,7 @@ s/@host@/vxworks_sparc/ s/@system_type@/vxworks_sparc/ s/@CC@/\/home\/gandalf\/bsproj\/tools\/vw-gnu\/solaris.sparc\/bin\/ccsparc/ s/@HCC@/gcc/ +s/@GCC@/yes/ s/@LD@/\/home\/gandalf\/bsproj\/tools\/vw-gnu\/solaris.sparc\/bin\/ldsparc/ s/@DEBUG_FLAGS@/-g/ s/@GCCLIB_PATH@/\/home\/gandalf\/bsproj\/tools\/vw-gnu\/solaris.sparc\/lib\/gcc-lib\/sparc-wrs-vxworks\/cygnus-2.2.3.1\/libgcc.a/ diff --git a/erts/configure.in b/erts/configure.in index 10ea0b5e4b..2b3d97cfdd 100644 --- a/erts/configure.in +++ b/erts/configure.in @@ -2,7 +2,7 @@ dnl Process this file with autoconf to produce a configure script. -*-m4-*- dnl %CopyrightBegin% dnl -dnl Copyright Ericsson AB 1997-2017. All Rights Reserved. +dnl Copyright Ericsson AB 1997-2018. All Rights Reserved. dnl dnl Licensed under the Apache License, Version 2.0 (the "License"); dnl you may not use this file except in compliance with the License. @@ -2012,29 +2012,6 @@ esac AC_CHECK_DECLS([posix2time, time2posix],,,[#include <time.h>]) -disable_vfork=false -if test "x$EMU_THR_LIB_NAME" != "x"; then - AC_MSG_CHECKING([if vfork is known to hang multithreaded applications]) - case $host_os in - osf*) - AC_MSG_RESULT(yes) - disable_vfork=true;; - *) - AC_MSG_RESULT(no);; - esac -fi - -if test $disable_vfork = false; then - AC_FUNC_VFORK - if test $ac_cv_func_vfork_works = no; then - disable_vfork=true - fi -fi - -if test $disable_vfork = true; then - AC_DEFINE(DISABLE_VFORK, 1, [Define if you want to disable vfork.]) -fi - AC_FUNC_VPRINTF dnl The AC_DEFINEs are necessary for autoheader to work. :-( diff --git a/erts/doc/src/Makefile b/erts/doc/src/Makefile index 96cc4413a9..21aa3db864 100644 --- a/erts/doc/src/Makefile +++ b/erts/doc/src/Makefile @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 1997-2017. All Rights Reserved. +# Copyright Ericsson AB 1997-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -149,6 +149,7 @@ debug opt: clean: rm -rf $(HTMLDIR)/* + rm -rf $(XMLDIR) rm -f $(MAN1DIR)/* rm -f $(MAN3DIR)/* rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) diff --git a/erts/doc/src/absform.xml b/erts/doc/src/absform.xml index 158f4dc4e8..f29bb7b8f9 100644 --- a/erts/doc/src/absform.xml +++ b/erts/doc/src/absform.xml @@ -801,7 +801,8 @@ <c>{ann_type,LINE,[Rep(A),Rep(T_0)]}</c>.</p> </item> <item> - <p>If T is an atom or integer literal L, then Rep(T) = Rep(L).</p> + <p>If T is an atom, a character, or an integer literal L, + then Rep(T) = Rep(L).</p> </item> <item> <p>If T is a bitstring type <c><<_:M,_:_*N>></c>, diff --git a/erts/doc/src/alt_dist.xml b/erts/doc/src/alt_dist.xml index 92d40d8558..e6245130fc 100644 --- a/erts/doc/src/alt_dist.xml +++ b/erts/doc/src/alt_dist.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2000</year><year>2016</year> + <year>2000</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/erts/doc/src/driver_entry.xml b/erts/doc/src/driver_entry.xml index e8c7e26457..fd7d6223f6 100644 --- a/erts/doc/src/driver_entry.xml +++ b/erts/doc/src/driver_entry.xml @@ -4,7 +4,7 @@ <cref> <header> <copyright> - <year>2001</year><year>2016</year> + <year>2001</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/erts/doc/src/erl.xml b/erts/doc/src/erl.xml index 4cf0066999..05a9895687 100644 --- a/erts/doc/src/erl.xml +++ b/erts/doc/src/erl.xml @@ -4,7 +4,7 @@ <comref> <header> <copyright> - <year>1996</year><year>2017</year> + <year>1996</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/erts/doc/src/erl_dist_protocol.xml b/erts/doc/src/erl_dist_protocol.xml index 98a9a76b60..c90c8f9521 100644 --- a/erts/doc/src/erl_dist_protocol.xml +++ b/erts/doc/src/erl_dist_protocol.xml @@ -5,7 +5,7 @@ <header> <copyright> <year>2007</year> - <year>2017</year> + <year>2018</year> <holder>Ericsson AB, All Rights Reserved</holder> </copyright> <legalnotice> diff --git a/erts/doc/src/erl_driver.xml b/erts/doc/src/erl_driver.xml index e6c9905039..7055889e4a 100644 --- a/erts/doc/src/erl_driver.xml +++ b/erts/doc/src/erl_driver.xml @@ -4,7 +4,7 @@ <cref> <header> <copyright> - <year>2001</year><year>2017</year> + <year>2001</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/erts/doc/src/erl_nif.xml b/erts/doc/src/erl_nif.xml index 8a9ae58e99..a20b8ee884 100644 --- a/erts/doc/src/erl_nif.xml +++ b/erts/doc/src/erl_nif.xml @@ -4,7 +4,7 @@ <cref> <header> <copyright> - <year>2001</year><year>2017</year> + <year>2001</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -194,7 +194,7 @@ ok <p>Binaries are sequences of whole bytes. Bitstrings with an arbitrary bit length have no support yet.</p> </item> - <tag>Resource objects</tag> + <tag><marker id="resource_objects"/>Resource objects</tag> <item> <p>The use of resource objects is a safe way to return pointers to native data structures from a NIF. A resource object is @@ -1250,8 +1250,9 @@ typedef struct { <fsummary>Format strings and Erlang terms.</fsummary> <desc> <p>Similar to <c>fprintf</c> but this format string also accepts - <c>"%T"</c>, which formats Erlang terms.</p> - <p>This function was originally intenden for debugging purpose. It is not + <c>"%T"</c>, which formats Erlang terms of type + <seealso marker="#ERL_NIF_TERM"><c>ERL_NIF_TERM</c></seealso>.</p> + <p>This function is primarily intenden for debugging purpose. It is not recommended to print very large terms with <c>%T</c>. The function may change <c>errno</c>, even if successful.</p> </desc> @@ -3191,8 +3192,9 @@ if (retval & ERL_NIF_SELECT_STOP_CALLED) { <fsummary>Format strings and Erlang terms.</fsummary> <desc> <p>Similar to <c>snprintf</c> but this format string also accepts - <c>"%T"</c>, which formats Erlang terms.</p> - <p>This function was originally intenden for debugging purpose. It is not + <c>"%T"</c>, which formats Erlang terms of type + <seealso marker="#ERL_NIF_TERM"><c>ERL_NIF_TERM</c></seealso>.</p> + <p>This function is primarily intenden for debugging purpose. It is not recommended to print very large terms with <c>%T</c>. The function may change <c>errno</c>, even if successful.</p> </desc> diff --git a/erts/doc/src/erlang.xml b/erts/doc/src/erlang.xml index 15bd80e72f..bd33e35603 100644 --- a/erts/doc/src/erlang.xml +++ b/erts/doc/src/erlang.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1996</year><year>2017</year> + <year>1996</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -197,6 +197,15 @@ </desc> </datatype> + <datatype> + <name name="nif_resource"></name> + <desc> + <p>An opaque handle identifing a + <seealso marker="erl_nif#resource_objects">NIF resource object + </seealso>.</p> + </desc> + </datatype> + </datatypes> <funcs> @@ -1742,6 +1751,10 @@ true</pre> <item> <p><c>Pid</c> is the process identifier of the process that originally created the fun.</p> + <p>It might point to the <c>init</c> process if the + <c>Fun</c> was statically allocated when module was + loaded (this optimisation is performed for local + functions that do not capture the enviornment).</p> </item> <tag><c>{index, Index}</c></tag> <item> @@ -2068,8 +2081,15 @@ end</pre> Typically, this is used when a process started from a certain shell is to have another group leader than <c>init</c>.</p> + <p>The group leader should be rarely changed in + applications with a supervision tree, because OTP + assumes the group leader of their processes is + their application master.</p> <p>See also - <seealso marker="#group_leader/0"><c>group_leader/0</c></seealso>.</p> + <seealso marker="#group_leader/0"><c>group_leader/0</c></seealso> + and <seealso marker="doc/design_principles:applications#stopping">OTP + design principles</seealso> related to starting and stopping + applications.</p> </desc> </func> @@ -3765,13 +3785,6 @@ RealSystem = system + MissedSystem</code> If found, that driver is started. A driver runs in the Erlang work space, which means that it is linked with the Erlang runtime system.</p> - <p>When starting external programs on Solaris, the system - call <c>vfork</c> is used in preference to <c>fork</c> - for performance reasons, although it has a history of - being less robust. If there are problems using - <c>vfork</c>, setting environment variable - <c>ERL_NO_VFORK</c> to any value causes <c>fork</c> - to be used instead.</p> <p>For external programs, <c>PATH</c> is searched (or an equivalent method is used to find programs, depending on the OS). This is done by invoking @@ -4832,11 +4845,11 @@ RealSystem = system + MissedSystem</code> </item> <tag><c>error_logger</c></tag> <item> - <p>When set to <c>true</c>, the runtime system sends a - message to the current <seealso marker="kernel:error_logger"> - <c>error_logger</c></seealso> + <p>When set to <c>true</c>, the runtime system logs an + error event via <seealso marker="kernel:logger"> + <c>logger</c></seealso>, containing details about the process when the maximum - heap size is reached. One <c>error_logger</c> report is sent + heap size is reached. One log event is sent each time the limit is reached.</p> <p>If <c>error_logger</c> is not defined in the map, the system default is used. The default system default is <c>true</c>. @@ -4850,7 +4863,7 @@ RealSystem = system + MissedSystem</code> amount of memory that is used during the garbage collection. When contemplating using this option, it is recommended to first run it in production with <c>kill</c> set to <c>false</c> and inspect - the <c>error_logger</c> reports to see what the normal peak sizes + the log events to see what the normal peak sizes of the processes in the system is and then tune the value accordingly. </p> @@ -5288,10 +5301,10 @@ RealSystem = system + MissedSystem</code> <p><c><anno>MinBinVHeapSize</anno></c> is the minimum binary virtual heap size for the process.</p> </item> - <tag><c>{monitored_by, <anno>Pids</anno>}</c></tag> + <tag><c>{monitored_by, <anno>MonitoredBy</anno>}</c></tag> <item> - <p>A list of process identifiers monitoring the process (with - <c>monitor/2</c>).</p> + <p>A list of identifiers for all the processes, ports and NIF + resources, that are monitoring the process.</p> </item> <tag><c>{monitors, <anno>Monitors</anno>}</c></tag> <item> @@ -6163,7 +6176,7 @@ true</pre> <p>Monitors the new process (like <seealso marker="#monitor/2"><c>monitor/2</c></seealso> does).</p> </item> - <tag><c>{priority, <anno>Level</anno></c></tag> + <tag><c>{priority, <anno>Level</anno>}</c></tag> <item> <p>Sets the priority of the new process. Equivalent to executing <seealso marker="#process_flag_priority"> @@ -7382,7 +7395,7 @@ ok <note><p>If a scheduler fails to bind, this is often silently ignored, as it is not always possible to verify valid logical processor identifiers. If an error is reported, - it is reported to <c>error_logger</c>. To verify that the + an error event is logged. To verify that the schedulers have bound as requested, call <seealso marker="#system_info_scheduler_bindings"> <c>erlang:system_info(scheduler_bindings)</c></seealso>.</p> @@ -7574,7 +7587,7 @@ ok </func> <func> - <name name="system_info" arity="1" clause_i="75"/> + <name name="system_info" arity="1" clause_i="76"/> <fsummary>System info overview.</fsummary> <desc> <p>Returns information about the current system. @@ -7622,6 +7635,7 @@ ok <p> <seealso marker="#system_info_atom_count"><c>atom_count</c></seealso>, <seealso marker="#system_info_atom_limit"><c>atom_limit</c></seealso>, + <seealso marker="#system_info_ets_count"><c>ets_count</c></seealso>, <seealso marker="#system_info_ets_limit"><c>ets_limit</c></seealso>, <seealso marker="#system_info_port_count"><c>port_count</c></seealso>, <seealso marker="#system_info_port_limit"><c>port_limit</c></seealso>, @@ -7865,8 +7879,8 @@ ok <name name="system_info" arity="1" clause_i="12" anchor="system_info_cpu_topology"/> <!-- cpu_topology --> <name name="system_info" arity="1" clause_i="13"/> <!-- {cpu_topology, _} --> - <name name="system_info" arity="1" clause_i="37"/> <!-- logical_processors --> - <name name="system_info" arity="1" clause_i="72"/> <!-- update_cpu_info --> + <name name="system_info" arity="1" clause_i="38"/> <!-- logical_processors --> + <name name="system_info" arity="1" clause_i="73"/> <!-- update_cpu_info --> <fsummary>Information about the CPU topology of the system.</fsummary> <type name="cpu_topology"/> <type name="level_entry"/> @@ -8017,16 +8031,16 @@ ok </func> <func> - <name name="system_info" arity="1" clause_i="30" + <name name="system_info" arity="1" clause_i="31" anchor="system_info_process"/> <!-- fullsweep_after --> - <name name="system_info" arity="1" clause_i="31"/> <!-- garbage_collection --> - <name name="system_info" arity="1" clause_i="32"/> <!-- heap_sizes --> - <name name="system_info" arity="1" clause_i="33"/> <!-- heap_type --> - <name name="system_info" arity="1" clause_i="39"/> <!-- max_heap_size --> - <name name="system_info" arity="1" clause_i="40"/> <!-- message_queue_data --> - <name name="system_info" arity="1" clause_i="41"/> <!-- min_heap_size --> - <name name="system_info" arity="1" clause_i="42"/> <!-- min_bin_vheap_size --> - <name name="system_info" arity="1" clause_i="56"/> <!-- procs --> + <name name="system_info" arity="1" clause_i="32"/> <!-- garbage_collection --> + <name name="system_info" arity="1" clause_i="33"/> <!-- heap_sizes --> + <name name="system_info" arity="1" clause_i="34"/> <!-- heap_type --> + <name name="system_info" arity="1" clause_i="40"/> <!-- max_heap_size --> + <name name="system_info" arity="1" clause_i="41"/> <!-- message_queue_data --> + <name name="system_info" arity="1" clause_i="42"/> <!-- min_heap_size --> + <name name="system_info" arity="1" clause_i="43"/> <!-- min_bin_vheap_size --> + <name name="system_info" arity="1" clause_i="57"/> <!-- procs --> <fsummary>Information about the default process heap settings.</fsummary> <type name="message_queue_data"/> <type name="max_heap_size"/> @@ -8136,14 +8150,14 @@ ok </func> <func> - <name name="system_info" arity="1" clause_i="6" - anchor="system_info_limits"/> <!-- atom_count --> + <name name="system_info" arity="1" clause_i="6" anchor="system_info_limits"/> <!-- atom_count --> <name name="system_info" arity="1" clause_i="7"/> <!-- atom_limit --> - <name name="system_info" arity="1" clause_i="29"/> <!-- ets_limit --> - <name name="system_info" arity="1" clause_i="52"/> <!-- port_count --> - <name name="system_info" arity="1" clause_i="53"/> <!-- port_limit --> - <name name="system_info" arity="1" clause_i="54"/> <!-- process_count --> - <name name="system_info" arity="1" clause_i="55"/> <!-- process_limit --> + <name name="system_info" arity="1" clause_i="29"/> <!-- ets_count --> + <name name="system_info" arity="1" clause_i="30"/> <!-- ets_limit --> + <name name="system_info" arity="1" clause_i="53"/> <!-- port_count --> + <name name="system_info" arity="1" clause_i="54"/> <!-- port_limit --> + <name name="system_info" arity="1" clause_i="55"/> <!-- process_count --> + <name name="system_info" arity="1" clause_i="56"/> <!-- process_limit --> <fsummary>Information about various system limits.</fsummary> <desc> <marker id="system_info_limits"/> @@ -8166,7 +8180,7 @@ ok <c>erl(1)</c>. </p> </item> - <tag><marker id="system_info_ets_count"/> + <tag><marker id="system_info_ets_count"/> <c>ets_count</c></tag> <item> <p>Returns the number of ETS tables currently existing at the @@ -8218,13 +8232,13 @@ ok <func> <name name="system_info" arity="1" clause_i="26" anchor="system_info_time"/> <!-- end_time --> - <name name="system_info" arity="1" clause_i="49"/> <!-- os_monotonic_time_source --> - <name name="system_info" arity="1" clause_i="50"/> <!-- os_system_time_source --> - <name name="system_info" arity="1" clause_i="62"/> <!-- start_time --> - <name name="system_info" arity="1" clause_i="67"/> <!-- time_correction --> - <name name="system_info" arity="1" clause_i="68"/> <!-- time_offset --> - <name name="system_info" arity="1" clause_i="69"/> <!-- time_warp_mode --> - <name name="system_info" arity="1" clause_i="70"/> <!-- tolerant_timeofday --> + <name name="system_info" arity="1" clause_i="50"/> <!-- os_monotonic_time_source --> + <name name="system_info" arity="1" clause_i="51"/> <!-- os_system_time_source --> + <name name="system_info" arity="1" clause_i="63"/> <!-- start_time --> + <name name="system_info" arity="1" clause_i="68"/> <!-- time_correction --> + <name name="system_info" arity="1" clause_i="69"/> <!-- time_offset --> + <name name="system_info" arity="1" clause_i="70"/> <!-- time_warp_mode --> + <name name="system_info" arity="1" clause_i="71"/> <!-- tolerant_timeofday --> <fsummary>Information about system time.</fsummary> <desc> <marker id="system_info_time_tags"/> @@ -8448,16 +8462,16 @@ ok anchor="system_info_scheduler"/> <!-- dirty_cpu_schedulers --> <name name="system_info" arity="1" clause_i="18"/> <!-- dirty_cpu_schedulers_online --> <name name="system_info" arity="1" clause_i="19"/> <!-- dirty_io_schedulers --> - <name name="system_info" arity="1" clause_i="44"/> <!-- multi_scheduling --> - <name name="system_info" arity="1" clause_i="45"/> <!-- multi_scheduling_blockers --> - <name name="system_info" arity="1" clause_i="47"/> <!-- normal_multi_scheduling_blockers --> - <name name="system_info" arity="1" clause_i="57"/> <!-- scheduler_bind_type --> - <name name="system_info" arity="1" clause_i="58"/> <!-- scheduler_bindings --> - <name name="system_info" arity="1" clause_i="59"/> <!-- scheduler_id --> - <name name="system_info" arity="1" clause_i="60"/> <!-- schedulers --> - <name name="system_info" arity="1" clause_i="61"/> <!-- smp_support --> - <name name="system_info" arity="1" clause_i="65"/> <!-- threads --> - <name name="system_info" arity="1" clause_i="66"/> <!-- thread_pool_size --> + <name name="system_info" arity="1" clause_i="45"/> <!-- multi_scheduling --> + <name name="system_info" arity="1" clause_i="46"/> <!-- multi_scheduling_blockers --> + <name name="system_info" arity="1" clause_i="49"/> <!-- normal_multi_scheduling_blockers --> + <name name="system_info" arity="1" clause_i="58"/> <!-- scheduler_bind_type --> + <name name="system_info" arity="1" clause_i="59"/> <!-- scheduler_bindings --> + <name name="system_info" arity="1" clause_i="60"/> <!-- scheduler_id --> + <name name="system_info" arity="1" clause_i="61"/> <!-- schedulers --> + <name name="system_info" arity="1" clause_i="62"/> <!-- smp_support --> + <name name="system_info" arity="1" clause_i="66"/> <!-- threads --> + <name name="system_info" arity="1" clause_i="67"/> <!-- thread_pool_size --> <fsummary>Information about system schedulers.</fsummary> <desc> <marker id="system_info_scheduler_tags"/> @@ -8844,53 +8858,54 @@ ok <!-- <name name="system_info" arity="1" clause_i="26"/> end_time --> <!-- <name name="system_info" arity="1" clause_i="27"/> elib_malloc --> <!-- <name name="system_info" arity="1" clause_i="28"/> eager_check_io, removed --> - <!-- <name name="system_info" arity="1" clause_i="29"/> ets_limit --> - <!-- <name name="system_info" arity="1" clause_i="30"/> fullsweep_after --> - <!-- <name name="system_info" arity="1" clause_i="31"/> garbage_collection --> - <!-- <name name="system_info" arity="1" clause_i="32"/> heap_sizes --> - <!-- <name name="system_info" arity="1" clause_i="33"/> heap_type --> - <name name="system_info" arity="1" clause_i="34"/> <!-- info --> - <name name="system_info" arity="1" clause_i="35"/> <!-- kernel_poll --> - <name name="system_info" arity="1" clause_i="36"/> <!-- loaded --> - <!-- <name name="system_info" arity="1" clause_i="37"/> logical_processors --> - <name name="system_info" arity="1" clause_i="38"/> <!-- machine --> - <!-- <name name="system_info" arity="1" clause_i="39"/> max_heap_size --> - <!-- <name name="system_info" arity="1" clause_i="40"/> message_queue_data --> - <!-- <name name="system_info" arity="1" clause_i="41"/> min_heap_size --> - <!-- <name name="system_info" arity="1" clause_i="42"/> min_bin_vheap_size --> - <name name="system_info" arity="1" clause_i="43"/> <!-- modified_timing_level --> - <!-- <name name="system_info" arity="1" clause_i="44"/> multi_scheduling --> - <!-- <name name="system_info" arity="1" clause_i="45"/> multi_scheduling_blockers --> - <name name="system_info" arity="1" clause_i="46"/> <!-- nif_version --> - <!-- n<name name="system_info" arity="1" clause_i="47"/> ormal_multi_scheduling_blockers --> - <name name="system_info" arity="1" clause_i="48"/> <!-- otp_release --> - <!-- <name name="system_info" arity="1" clause_i="49"/> os_monotonic_time_source --> - <!-- <name name="system_info" arity="1" clause_i="50"/> os_system_time_source --> - <name name="system_info" arity="1" clause_i="51"/> <!-- port_parallelism --> - <!-- <name name="system_info" arity="1" clause_i="52"/> port_count --> - <!-- <name name="system_info" arity="1" clause_i="53"/> port_limit --> - <!-- <name name="system_info" arity="1" clause_i="54"/> process_count --> - <!-- <name name="system_info" arity="1" clause_i="55"/> process_limit --> - <!-- <name name="system_info" arity="1" clause_i="56"/> procs --> - <!-- <name name="system_info" arity="1" clause_i="57"/> scheduler_bind_type --> - <!-- <name name="system_info" arity="1" clause_i="58"/> scheduler_bindings --> - <!-- <name name="system_info" arity="1" clause_i="59"/> scheduler_id --> - <!-- <name name="system_info" arity="1" clause_i="60"/> schedulers --> - <!-- <name name="system_info" arity="1" clause_i="61"/> smp_support --> - <!-- <name name="system_info" arity="1" clause_i="62"/> start_time --> - <name name="system_info" arity="1" clause_i="63"/> <!-- system_version --> - <name name="system_info" arity="1" clause_i="64"/> <!-- system_architecture --> - <!-- <name name="system_info" arity="1" clause_i="65"/> threads --> - <!-- <name name="system_info" arity="1" clause_i="66"/> thread_pool_size --> - <!-- <name name="system_info" arity="1" clause_i="67"/> time_correction --> - <!-- <name name="system_info" arity="1" clause_i="68"/> time_offset --> - <!-- <name name="system_info" arity="1" clause_i="69"/> time_warp_mode --> - <!-- <name name="system_info" arity="1" clause_i="70"/> tolerant_timeofday --> - <name name="system_info" arity="1" clause_i="71"/> <!-- trace_control_word --> - <!-- <name name="system_info" arity="1" clause_i="72"/> update_cpu_info --> - <name name="system_info" arity="1" clause_i="73"/> <!-- version --> - <name name="system_info" arity="1" clause_i="74"/> <!-- wordsize --> - <!-- <name name="system_info" arity="1" clause_i="75"/> overview --> + <!-- <name name="system_info" arity="1" clause_i="29"/> ets_count --> + <!-- <name name="system_info" arity="1" clause_i="30"/> ets_limit --> + <!-- <name name="system_info" arity="1" clause_i="31"/> fullsweep_after --> + <!-- <name name="system_info" arity="1" clause_i="32"/> garbage_collection --> + <!-- <name name="system_info" arity="1" clause_i="33"/> heap_sizes --> + <!-- <name name="system_info" arity="1" clause_i="34"/> heap_type --> + <name name="system_info" arity="1" clause_i="35"/> <!-- info --> + <name name="system_info" arity="1" clause_i="36"/> <!-- kernel_poll --> + <name name="system_info" arity="1" clause_i="37"/> <!-- loaded --> + <!-- <name name="system_info" arity="1" clause_i="38"/> logical_processors --> + <name name="system_info" arity="1" clause_i="39"/> <!-- machine --> + <!-- <name name="system_info" arity="1" clause_i="40"/> max_heap_size --> + <!-- <name name="system_info" arity="1" clause_i="41"/> message_queue_data --> + <!-- <name name="system_info" arity="1" clause_i="42"/> min_heap_size --> + <!-- <name name="system_info" arity="1" clause_i="43"/> min_bin_vheap_size --> + <name name="system_info" arity="1" clause_i="44"/> <!-- modified_timing_level --> + <!-- <name name="system_info" arity="1" clause_i="45"/> multi_scheduling --> + <!-- <name name="system_info" arity="1" clause_i="46"/> multi_scheduling_blockers --> + <name name="system_info" arity="1" clause_i="47"/> <!-- nif_version --> + <!-- n<name name="system_info" arity="1" clause_i="48"/> ormal_multi_scheduling_blockers --> + <name name="system_info" arity="1" clause_i="49"/> <!-- otp_release --> + <!-- <name name="system_info" arity="1" clause_i="50"/> os_monotonic_time_source --> + <!-- <name name="system_info" arity="1" clause_i="51"/> os_system_time_source --> + <name name="system_info" arity="1" clause_i="52"/> <!-- port_parallelism --> + <!-- <name name="system_info" arity="1" clause_i="53"/> port_count --> + <!-- <name name="system_info" arity="1" clause_i="54"/> port_limit --> + <!-- <name name="system_info" arity="1" clause_i="55"/> process_count --> + <!-- <name name="system_info" arity="1" clause_i="56"/> process_limit --> + <!-- <name name="system_info" arity="1" clause_i="57"/> procs --> + <!-- <name name="system_info" arity="1" clause_i="58"/> scheduler_bind_type --> + <!-- <name name="system_info" arity="1" clause_i="59"/> scheduler_bindings --> + <!-- <name name="system_info" arity="1" clause_i="60"/> scheduler_id --> + <!-- <name name="system_info" arity="1" clause_i="61"/> schedulers --> + <!-- <name name="system_info" arity="1" clause_i="62"/> smp_support --> + <!-- <name name="system_info" arity="1" clause_i="63"/> start_time --> + <name name="system_info" arity="1" clause_i="64"/> <!-- system_version --> + <name name="system_info" arity="1" clause_i="65"/> <!-- system_architecture --> + <!-- <name name="system_info" arity="1" clause_i="66"/> threads --> + <!-- <name name="system_info" arity="1" clause_i="67"/> thread_pool_size --> + <!-- <name name="system_info" arity="1" clause_i="68"/> time_correction --> + <!-- <name name="system_info" arity="1" clause_i="69"/> time_offset --> + <!-- <name name="system_info" arity="1" clause_i="70"/> time_warp_mode --> + <!-- <name name="system_info" arity="1" clause_i="71"/> tolerant_timeofday --> + <name name="system_info" arity="1" clause_i="72"/> <!-- trace_control_word --> + <!-- <name name="system_info" arity="1" clause_i="73"/> update_cpu_info --> + <name name="system_info" arity="1" clause_i="74"/> <!-- version --> + <name name="system_info" arity="1" clause_i="75"/> <!-- wordsize --> + <!-- <name name="system_info" arity="1" clause_i="76"/> overview --> <fsummary>Information about the system.</fsummary> <desc> <marker id="system_info_misc_tags"/> diff --git a/erts/doc/src/erlc.xml b/erts/doc/src/erlc.xml index 2214b76a51..be9b4e8d97 100644 --- a/erts/doc/src/erlc.xml +++ b/erts/doc/src/erlc.xml @@ -4,7 +4,7 @@ <comref> <header> <copyright> - <year>1997</year><year>2016</year> + <year>1997</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/erts/doc/src/erts_alloc.xml b/erts/doc/src/erts_alloc.xml index 0893eb291c..962bc9a244 100644 --- a/erts/doc/src/erts_alloc.xml +++ b/erts/doc/src/erts_alloc.xml @@ -4,7 +4,7 @@ <cref> <header> <copyright> - <year>2002</year><year>2017</year> + <year>2002</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -487,11 +487,10 @@ utilization value used. Once a carrier is abandoned, no new allocations are made in it. When an allocator instance gets an increased multiblock carrier need, it first tries to fetch an - abandoned carrier from an allocator instance of the same - allocator type. If no abandoned carrier can be fetched, it - creates a new empty carrier. When an abandoned carrier has been - fetched, it will function as an ordinary carrier. This feature has - special requirements on the + abandoned carrier from another allocator instance. If no abandoned + carrier can be fetched, it creates a new empty carrier. When an + abandoned carrier has been fetched, it will function as an ordinary + carrier. This feature has special requirements on the <seealso marker="#M_as">allocation strategy</seealso> used. Only the strategies <c>aoff</c>, <c>aoffcbf</c>, <c>aoffcaobf</c>, <c>ageffcaoff</c>m, <c>ageffcbf</c> and <c>ageffcaobf</c> @@ -584,7 +583,7 @@ carriers are decided in section <seealso marker="#mseg_mbc_sizes"> The alloc_util Framework</seealso>. On - 32-bit Unix style OS this limit cannot be set > 128 MB.</p> + 32-bit Unix style OS this limit cannot be set > 64 MB.</p> </item> <tag><marker id="M_mbcgs"/><c><![CDATA[+M<S>mbcgs <ratio>]]></c></tag> <item> diff --git a/erts/doc/src/match_spec.xml b/erts/doc/src/match_spec.xml index 46a3daebe8..5cd6dc1750 100644 --- a/erts/doc/src/match_spec.xml +++ b/erts/doc/src/match_spec.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>1999</year><year>2016</year> + <year>1999</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/erts/doc/src/notes.xml b/erts/doc/src/notes.xml index f7f86084a9..fa02c9dd21 100644 --- a/erts/doc/src/notes.xml +++ b/erts/doc/src/notes.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2004</year><year>2017</year> + <year>2004</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -31,6 +31,1160 @@ </header> <p>This document describes the changes made to the ERTS application.</p> +<section><title>Erts 10.0.5</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fixed a bug which caused an emulator crash when + <c>enif_send()</c> was called by a NIF that executed on a + dirty scheduler. The bug was either triggered when the + NIF called <c>enif_send()</c> without a message + environment, or when the process executing the NIF was + <c>send</c> traced.</p> + <p> + Own Id: OTP-15223</p> + </item> + <item> + <p> + Fixed a bug causing some Erlang references to be + inconsistently ordered. This could for example cause + failure to look up certain elements with references as + keys in search data structures. This bug was introduced + in R13B02.</p> + <p> + Thanks to Simon Cornish for finding the bug and supplying + a fix.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-15225</p> + </item> + </list> + </section> + +</section> + +<section><title>Erts 10.0.4</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p>Fixed a bug that prevented the <c>noshell</c> option + from working correctly on Mac OS X and BSD.</p> + <p> + Own Id: OTP-15169</p> + </item> + <item> + <p>Fixed a crash when matching directly against a literal + map using a single key that had been saved on the + stack.</p> + <p> + Own Id: OTP-15184</p> + </item> + <item> + <p>Fix node crash when passing a bad time option to + <c>file:read_file_info/2</c>.</p> + <p> + Own Id: OTP-15196</p> + </item> + </list> + </section> + +</section> + +<section><title>Erts 10.0.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p>Fixed a scheduler bug that caused normal schedulers to + run dirty code.</p> + <p> + Own Id: OTP-15154</p> + </item> + <item> + <p> + Fixed a bug in <c>erlang:trace_info/2</c> which caused + the emulator to crash when a bad argument was passed. The + bug was introduced in ERTS version 10.0.</p> + <p> + Own Id: OTP-15183 Aux Id: ERL-670 </p> + </item> + </list> + </section> + +</section> + +<section><title>Erts 10.0.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p>Fixed a rare bug that could cause processes to be + scheduled after they had been freed.</p> + <p> + Own Id: OTP-15067 Aux Id: ERL-573 </p> + </item> + <item> + <p>Fixed a race condition in the inet driver that could + cause receive to hang when the emulator was compiled with + gcc 8.</p> + <p> + Own Id: OTP-15158 Aux Id: ERL-654 </p> + </item> + </list> + </section> + +</section> + +<section><title>Erts 10.0.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p>The keys used in <c>os:getenv</c> and <c>os:putenv</c> + are case-insensitive again on Windows.</p> + <p> + Own Id: OTP-15147 Aux Id: ERL-644 </p> + </item> + </list> + </section> + +</section> + +<section><title>Erts 10.0</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + The type specifications for <c>file:posix/0</c> and + <c>inet:posix/0</c> have been updated according to which + errors file and socket operations should be able to + return.</p> + <p> + Own Id: OTP-14019 Aux Id: ERL-550 </p> + </item> + <item> + <p> + Fix error printout from run_erl and a bug that could + cause unintended fds to be leaked into the started + program.</p> + <p> + Own Id: OTP-14537 Aux Id: PR1529 </p> + </item> + <item> + <p> File operations used to accept <seealso + marker="kernel:file#type-name_all">filenames</seealso> + containing null characters (integer value zero). This + caused the name to be truncated and in some cases + arguments to primitive operations to be mixed up. + Filenames containing null characters inside the filename + are now <em>rejected</em> and will cause primitive file + operations to fail. </p> <p> Also environment variable + operations used to accept <seealso + marker="kernel:os#type-env_var_name">names</seealso> and + <seealso + marker="kernel:os#type-env_var_value">values</seealso> of + environment variables containing null characters (integer + value zero). This caused operations to silently produce + erroneous results. Environment variable names and values + containing null characters inside the name or value are + now <em>rejected</em> and will cause environment variable + operations to fail. </p> <p>Primitive environment + variable operations also used to accept the <c>$=</c> + character in environment variable names causing various + problems. <c>$=</c> characters in environment variable + names are now also <em>rejected</em>. </p> <p>Also + <seealso + marker="kernel:os#cmd/1"><c>os:cmd/1</c></seealso> now + reject null characters inside its <seealso + marker="kernel:os#type-os_command">command</seealso>. + </p> <p><seealso + marker="erts:erlang#open_port/2"><c>erlang:open_port/2</c></seealso> + will also reject null characters inside the port name + from now on.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-14543 Aux Id: ERL-370 </p> + </item> + <item> + <p> + Fix bugs related to the bookkeeping of microstate + accounting states.</p> + <p> + Own Id: OTP-14652</p> + </item> + <item> + <p><c>os:putenv</c> and <c>os:getenv</c> no longer access + the process environment directly and instead work on a + thread-safe emulation. The only observable difference is + that it's <em>not</em> kept in sync with libc + <c>getenv(3)</c> / <c>putenv(3)</c>, so those who relied + on that behavior in drivers or NIFs will need to add + manual synchronization.</p> <p>On Windows this means that + you can no longer resolve DLL dependencies by modifying + the <c>PATH</c> just before loading the driver/NIF. To + make this less of a problem, the emulator now adds the + target DLL's folder to the DLL search path.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-14666</p> + </item> + <item> + <p>Corrected <c>erlang:is_builtin(erlang, M, F)</c> to + return <c>true</c> for <c>apply/2</c> and + <c>yield/0</c>.</p> + <p> + Own Id: OTP-14713 Aux Id: ERL-500 </p> + </item> + <item> + <p>Fixed a bug where the PATH environment variable wasn't + updated correctly on a release downgrade, effectively + keeping the PATH of the new release.</p> + <p> + Own Id: OTP-14719</p> + </item> + <item> + <p>A receive optimization that avoids scanning the entire + message queue when receiving a message containing a + freshly created reference could in rare circumstances + (involving recursive calls to the functions that does the + receive) cause the receive to hang. This has been + corrected.</p> + <p> + Own Id: OTP-14782 Aux Id: ERL-511 </p> + </item> + <item> + <p> + Fix building of Erlang/OTP on platforms which have small + data area with short addressing. For example the + PowerPC/RTEMS platform.</p> + <p> + Own Id: OTP-14909 Aux Id: PR-1692 </p> + </item> + <item> + <p>Fixed a crash when <c>enif_make_binary</c> is called + with a binary produced by <c>enif_inspect_binary</c> in a + different environment.</p> + <p> + Own Id: OTP-14931</p> + </item> + <item> + <p>Fixed a crash when <c>enif_make_binary</c> is called + more than once with a binary that had previously been + added to an <c>enif_ioq</c>.</p> + <p> + Own Id: OTP-14932</p> + </item> + <item> + <p> + The erl_child_setup program now ignores SIGTERM signals.</p> + <p> + Own Id: OTP-14943 Aux Id: ERL-576 </p> + </item> + <item> + <p> + Force 64-bit alignment on pre-allocators on architectures + which needs it.</p> + <p> + Own Id: OTP-14977</p> + </item> + <item> + <p> + Fixed a bug where dirty scheduler picked up non-dirty + work.</p> + <p> + Own Id: OTP-14978</p> + </item> + <item> + <p> + Calls to <c>gen_tcp:send/2</c> on closed sockets now + returns <c>{error, closed}</c> instead of + <c>{error,enotconn}</c>.</p> + <p> + Own Id: OTP-15001</p> + </item> + <item> + <p> + <c>erlang:monotonic_time/1</c> failed with <c>badarg</c> + when passing the <c>perf_counter</c> time unit as + argument.</p> + <p> + Own Id: OTP-15008</p> + </item> + <item> + <p> + Fix bug where rapid <c>init:restart()</c> calls would + sometimes crash because a code load request leaked in + between the restarts.</p> + <p> + Own Id: OTP-15013</p> + </item> + <item> + <p> + Improve <c>float_to_list(F, [{decimals,D}])</c> to closer + conform with <c>io_lib:format("~.*f", [D,F])</c>.</p> + <p> + There are however, still cases when <c>float_to_list</c> + does not produce the exact same result as + <c>io_lib:format</c>, especially for large values + <c>F</c> and/or many decimals <c>D</c>.</p> + <p> + Own Id: OTP-15015 Aux Id: OTP-14890 </p> + </item> + <item> + <p>Fixed a deadlock that would occur on certain + allocators when a reallocation failed with <c>+ramv</c> + enabled.</p> + <p> + Own Id: OTP-15024</p> + </item> + <item> + <p> + Fix bug that made it impossible to use an erl_tracer as + the seq_trace trace receiver.</p> + <p> + Own Id: OTP-15029</p> + </item> + <item> + <p> + Fix bug where a large (> 1 GB) emulator generated error + logger message would cause the emulator to crash.</p> + <p> + Own Id: OTP-15032</p> + </item> + <item> + <p>The emulator will no longer crash when reading the + file information of an ordinary file that has an NTFS + reparse point, such as files stored in a OneDrive-mapped + folder.</p> + <p> + Own Id: OTP-15062 Aux Id: ERL-615 </p> + </item> + <item> + <p> + Fixed bug in <c>enif_binary_to_term</c> which could cause + memory corruption for immediate terms (atoms, small + integers, pids, ports, empty lists).</p> + <p> + Own Id: OTP-15080</p> + </item> + <item> + <p> + Fixed bug in <c>erlang:system_profile/2</c> that could + cause superfluous <c>{profile,_,active,_,_}</c> messages + for terminating processes.</p> + <p> + Own Id: OTP-15085</p> + </item> + <item> + <p> + On OSs with per thread CPU time support, change + <c>cpu_timestamp</c> in <seealso + marker="erlang#trace/3">erlang:trace/3</seealso> to use + it instead of per process CPU time. This makes this + option useable on such OSs when running multiple + schedulers.</p> + <p> + Own Id: OTP-15090</p> + </item> + <item> + <p> + Fix segfault in abort_signal_task which could happen if a + port terminated while there were outstanding port tasks + that were not signals, for example a + ready_input/ready_output event.</p> + <p> + Own Id: OTP-15108 Aux Id: ERL-621 </p> + </item> + <item> + <p> + Fixed bug in <c>ets</c> that could cause VM crash if + process A terminates after fixating a table and process B + deletes the table at "the same time". The table fixation + could be done with <c>ets:safe_fixtable</c> or if process + A terminates in the middle of a long running + <c>select</c> or <c>match</c> call.</p> + <p> + Own Id: OTP-15109</p> + </item> + <item> + <p>Owner and group changes through + <c>file:write_file_info</c>, <c>file:change_owner</c>, + and <c>file:change_group</c> will no longer report + success on permission errors.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-15118</p> + </item> + <item> + <p> + Fix a bug error reporting from escripts on windows where + the error message would get garbled.</p> + <p> + Own Id: OTP-15119 Aux Id: PR-1826 </p> + </item> + <item> + <p> + Fix segfault when a process is interally re-scheduled + while being traced for in out events. This bug was + introduced in erts-8.0 (OTP-19.0).</p> + <p> + Own Id: OTP-15125</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p>It is now possible to open device files and FIFOs with + <c>file:open/2</c>.</p> + <p> + Own Id: OTP-11462</p> + </item> + <item> + <p> + The <c>erlang:system_flag(scheduler_wall_time,Bool)</c> + call is now reference counted and will be turned off if + the (last) process that started the performance + statistics dies. Thus it is no longer possible to start + the statistics with <c>rpc:call(Node, erlang, + system_flag, [scheduler_wall_time, true])</c> since it + will be turned off directly afterwards when the rpc + process dies.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-11694</p> + </item> + <item> + <p>A new logging API is added to Erlang/OTP, see the + <seealso + marker="kernel:logger"><c>logger(3)</c></seealso> manual + page, and section <seealso + marker="kernel:logger_chapter">Logging</seealso> in the + Kernel User's Guide.</p> + <p>Calls to <c>error_logger</c> are automatically + redirected to the new API, and legacy error logger event + handlers can still be used. It is, however, recommended + to use the Logger API directly when writing new code.</p> + <p>Notice the following potential incompatibilities:</p> + <list> <item><p>Kernel configuration parameters + <c>error_logger</c> still works, but is overruled if the + default handler's output destination is configured with + Kernel configuration parameter <c>logger</c>.</p> <p>In + general, parameters for configuring error logger are + overwritten by new parameters for configuring + Logger.</p></item> <item><p>The concept of SASL error + logging is deprecated, meaning that by default the SASL + application does not affect which log events are + logged.</p> <p>By default, supervisor reports and crash + reports are logged by the default Logger handler started + by Kernel, and end up at the same destination (terminal + or file) as other standard log event from Erlang/OTP.</p> + <p>Progress reports are not logged by default, but can be + enabled by setting the primary log level to info, for + example with the Kernel configuration parameter + <c>logger_level</c>.</p> <p>To obtain backwards + compatibility with the SASL error logging functionality + from earlier releases, set Kernel configuration parameter + <c>logger_sasl_compatible</c> to <c>true</c>. This + prevents the default Logger handler from logging any + supervisor-, crash-, or progress reports. Instead, SASL + adds a separate Logger handler during application start, + which takes care of these log events. The SASL + configuration parameters <c>sasl_error_logger</c> and + <c>sasl_errlog_type</c> specify the destination (terminal + or file) and severity level to log for these + events.</p></item></list> + <p> + Since Logger is new in Erlang/OTP 21.0, we do reserve the + right to introduce changes to the Logger API and + functionality in patches following this release. These + changes might or might not be backwards compatible with + the initial version.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-13295</p> + </item> + <item> + <p> + <c>gen_sctp:connect_init/4</c> or rather connect in + <c>inet_drv.c</c> for SCTP has been fixed to not check + the write file descriptor for writeability after a + connect, since for SCTP (SOCK_SEQPACKET) that property + does not seem to be any kind of indicator for when a + connect has finished. This fixes connects that the OS + returned as "in progress" that was misinterpreted by + <c>gen_sctp:connect_init</c> as failed.</p> + <p> + Own Id: OTP-13760 Aux Id: PR-1592 </p> + </item> + <item> + <p>The file driver has been rewritten as a NIF, + decreasing the latency of file operations. Notable + incompatibilities are:</p> <list> <item><p>The + <c>use_threads</c> option for <c>file:sendfile/5</c> no + longer has any effect; we either use non-blocking + <c>sendfile(2)</c> or fall back to <c>file:read</c> + + <c>gen_tcp:send</c>. </p></item> <item><p>The + file-specific DTrace probes have been removed. The same + effect can be achieved with normal tracing together with + the <c>nif__entry</c>/<c>nif__return</c> probes to track + scheduling.</p></item> </list> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-14256</p> + </item> + <item> + <p>The I/O polling functionality of erts has been + re-written to better make use of the OSs polling + mechanisms. This change means that erts will now always + prefer to use a kernel-polling mechanism if possible. + Also all of the I/O polling has been moved to dedicated + threads instead of being placed in the scheduler + loops.</p> <p>As a result of this, the <c>erl</c> options + <c>+K</c> and <c>+secio</c> have been removed. It is + still possible to disable kernel-poll, but it has to be + done at compile time through the configure option + <c>--disable-kernel-poll</c>.</p> <p>The new <c>erl</c> + options <seealso marker="erl#+IOt"><c>+IOt</c></seealso> + and <seealso marker="erl#+IOp"><c>+IOp</c></seealso> can + be used to change how many IO poll threads and poll sets + that erts should use. See their respective documentation + for more details.</p> + <p> + Own Id: OTP-14346</p> + </item> + <item> + <p>Truly asynchronous auto-connect. Earlier, when + <c>erlang:send</c> was aimed toward an unconnected node, + the function would not return until the connection setup + had completed (or failed). Now the function returns + directly after the message has been enqueued and the + connection setup started.</p> + <p>The same applies to all distributed operations that + may trigger auto-connect, i.e. <c>'!'</c>, <c>send</c>, + <c>link</c>, <c>monitor</c>, <c>monitor_node</c>, + <c>exit/2</c> and <c>group_leader</c>.</p> + <p>The interface for all these functions are unchanged as + they do not return connection failures. The only + exception is <c>erlang:monitor</c> where a <em>possible + incompatibility</em> is introduced: An attempt to monitor + a process on a primitive node (such as erl_interface or + jinterface), where remote process monitoring is not + implemented, will no longer fail with <c>badarg</c> + exception. Instead a monitor will be created, but it will + only supervise the connection to the node.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-14370</p> + </item> + <item> + <p>Changed the default behaviour of <c>.erlang</c> + loading: <c>.erlang</c> is no longer loaded from the + current directory. <c>c:erlangrc(PathList)</c> can be + used to search and load an <c>.erlang</c> file from user + specified directories.</p> <p><c>escript</c>, + <c>erlc</c>, <c>dialyzer</c> and <c>typer</c> no longer + load an <c>.erlang</c> at all.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-14439</p> + </item> + <item> + <p> + New functionality for implementation of alternative + carriers for the Erlang distribution has been introduced. + This mainly consists of support for usage of distribution + controller processes (previously only ports could be used + as distribution controllers). For more information see + <seealso marker="erts:alt_dist#distribution_module">ERTS + User's Guide ➜ How to implement an Alternative Carrier + for the Erlang Distribution ➜ Distribution + Module</seealso>.</p> + <p> + Own Id: OTP-14459</p> + </item> + <item> + <p> + Add support for the lcc compiler and in extension the + Elbrus 2000 platform.</p> + <p> + Own Id: OTP-14492</p> + </item> + <item> + <p>Support for "tuple calls" have been removed from the + run-time system. Tuple calls was an undocumented and + unsupported feature which allowed the module argument for + an apply operation to be a tuple: <c>Var = dict:new(), + Var:size()</c>. This "feature" frequently caused + confusion, especially when such call failed. The + stacktrace would point out functions that don't exist in + the source code.</p> + <p>For legacy code that need to use parameterized modules + or tuple calls for some other reason, there is a new + compiler option called <c>tuple_calls</c>. When this + option is given, the compiler will generate extra code + that emulates the old behavior for calls where the module + is a variable.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-14497</p> + </item> + <item> + <p>Creation of small maps with literal keys has been + optimized to be faster and potentially use less memory. + The keys are combined into a literal key tuple which is + put into the literal pool. The key tuple can be shared + between many instances of maps having the same keys.</p> + <p> + Own Id: OTP-14502</p> + </item> + <item> + <p> + When an exception is thrown, include the arguments of the + call in the stacktrace for BIFs <c>band</c>, <c>bor</c>, + <c>bsl</c>, <c>bsr</c>, <c>bxor</c>, <c>div</c>, + <c>rem</c> and the operators <c>+</c>, <c>-</c>, <c>*</c> + and <c>/</c>.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-14508</p> + </item> + <item> + <p> + The non-smp emulators have been removed. This means that + the configure options <c>--disable-threads</c> and + <c>--enable-plain-emulator</c> have been removed and + configure will now refuse to build Erlang/OTP on + platforms without thread support.</p> + <p> + In order to achieve a similar setup as the non-smp + emulator, it is possible to start Erlang/OTP with the + <c>+S 1</c> option.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-14518</p> + </item> + <item> + <p>Modules that use floating point constants compiled + with R15 or earlier will need to be re-compiled before + they can be loaded.</p> + <p> + Own Id: OTP-14575</p> + </item> + <item> + <p> + Implementation of true asynchronous signaling between + processes in order to improve scalability. Signals + affected include exit, monitor, demonitor, monitor + triggered, link, unlink, and group leader.</p> + <p> + Own Id: OTP-14589</p> + </item> + <item> + <p> + Added a PGO (profile guided optimization) pass to the + build step of erts. This can be disabled by passing + --disable-pgo to configure.</p> + <p> + Own Id: OTP-14604</p> + </item> + <item> + <p> + Improved the performance of <c>binary:split</c> and + <c>binary:match</c>.</p> + <p> + Own Id: OTP-14610 Aux Id: PR-1480 </p> + </item> + <item> + <p> + It is not longer possible to disable dirty schedulers + when building erlang.</p> + <p> + Own Id: OTP-14613</p> + </item> + <item> + <p>Loaded BEAM code in a 64-bit system requires less + memory because of better packing of operands for + instructions.</p> + <p>These memory savings were achieved by major + improvements to the <c>beam_makeops</c> scripts used when + building the run time system and BEAM compiler. There is + also new for documentation for <c>beam_makeops</c> that + describes how new BEAM instructions and loader + transformations can be implemented. The documentation is + found in here in a source directory or git repository: + erts/emulator/internal_doc/beam_makeops.md. An online + version can be found here: + https://github.com/erlang/otp/blob/master/erts/emulator/internal_doc/beam_makeops.md</p> + <p> + Own Id: OTP-14626</p> + </item> + <item> + <p><c>file:read_file</c> has been changed to read the + content of files that report a size of 0 even when data + can be read from them. An example of such a file is + <c>/proc/cpuinfo</c> on Linux.</p> + <p> + Own Id: OTP-14637 Aux Id: ERL-327 PR-1524 </p> + </item> + <item> + <p> + It is no longer possible to disable the <c>temp_alloc</c> + allocator. Disabling it caused serious performance + degradations and was never what was wanted.</p> + <p> + Own Id: OTP-14651</p> + </item> + <item> + <p>The reduction cost of sending messages is now + constant. It will no longer scale according to the length + of the receiving process' message queue.</p> + <p> + Own Id: OTP-14667</p> + </item> + <item> + <p> + Improved loading of modules with <c>-on_load</c> + directive, to no longer block all schedulers when the + load operation is completed.</p> + <p> + Own Id: OTP-14680</p> + </item> + <item> + <p> + On platforms with real-time signals available, SIGRTMIN+1 + is now used as the internal scheduler suspend signal + instead of SIGUSR2.</p> + <p> + Own Id: OTP-14682</p> + </item> + <item> + <p>When the value returned from a '<c>catch</c>' + expression is ignored, no stacktrace will be built if an + exception is caught. That will save time and produce less + garbage. There are also some minor optimizations of + '<c>try</c>/<c>catch</c>' both in the compiler and + run-time system.</p> + <p> + Own Id: OTP-14683</p> + </item> + <item> + <p>The guarantees and non-guarantees of + <c>erlang:get_stacktrace/0</c> are now documented.</p> + <p> + Own Id: OTP-14687</p> + </item> + <item> + <p>There is a new syntax in '<c>try/catch</c>' for + retrieving the stacktrace without calling + '<c>erlang:get_stacktrace/0</c>'. See the reference + manual for a description of the new syntax. The + '<c>erlang:get_stacktrace/0</c>' BIF is now + deprecated.</p> + <p> + Own Id: OTP-14692</p> + </item> + <item> + <p> + New 'used' option for <c>binary_to_term/2</c> that will + also return number of bytes actually read from the + binary. This enables easy access to any extra data in the + binary located directly after the returned term.</p> + <p> + Own Id: OTP-14780</p> + </item> + <item> + <p> + Added more statistics for + <c>erlang:system_info({allocator,A})</c> in the + <c>mbcs_pool</c> section.</p> + <p> + Own Id: OTP-14795 Aux Id: ERL-88 </p> + </item> + <item> + <p>Added <c>enif_ioq_peek_head</c> to retrieve Erlang + terms from NIF IO queues without having to resort to + copying.</p> + <p> + Own Id: OTP-14797</p> + </item> + <item> + <p>There is a new option '<c>makedep_side_effect</c>' for + the compiler and <c>-MMD</c> for '<c>erlc</c>' that + generates dependencies and continues to compile as + normal.</p> + <p> + Own Id: OTP-14830</p> + </item> + <item> + <p>Added <c>ets:whereis/1</c> for retrieving the table + identifier of a named table.</p> + <p> + Own Id: OTP-14884</p> + </item> + <item> + <p><c>seq_trace</c> labels may now be any erlang + term.</p> + <p> + Own Id: OTP-14899</p> + </item> + <item> + <p> + Optimized the common case of <c>monitor</c> followed by + <c>send</c> to the same local process. The monitor signal + is now delayed in order to be piggybacked with the sent + message and thereby only get one lock operation on the + message queue of the receiver. A delayed monitor signal + is flushed if no <c>send</c> has been done at the latest + when the process is scheduled out.</p> + <p> + Own Id: OTP-14901</p> + </item> + <item> + <p> + Make hipe compiled code work on x86_64 (amd64) with OS + security feature PIE, where executable code can be loaded + into a random location. Old behavior, if hipe was + enabled, was to disable PIE build options for the VM.</p> + <p> + Own Id: OTP-14903</p> + </item> + <item> + <p>The number of driver async threads will now default to + 1 as the standard drivers do not use them anymore. Users + that changed this value to tweak the file driver should + replace <c>+A</c> with <c>+SDio</c> since it now uses + dirty IO schedulers instead of async threads.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-14928</p> + </item> + <item> + <p> + Optimize <c>==</c> and <c>/=</c> for binaries with + different sizes to be constant in time instead of + proportional to the size of their common prefix.</p> + <p> + Own Id: OTP-14934 Aux Id: PR-1708 </p> + </item> + <item> + <p> + Refactorings making some internal process flags available + for other usage.</p> + <p> + Own Id: OTP-14948</p> + </item> + <item> + <p> + Removed need for HiPE to allocate native executable + memory in low 2GB address space on x86_64. Command line + option <c>+MXscs</c> is thereby obsolete and ignored.</p> + <p> + Own Id: OTP-14951</p> + </item> + <item> + <p>Added <c>enif_make_map_from_arrays</c> for creating a + populated map, analogous to + <c>enif_make_list_from_array</c>.</p> + <p> + Own Id: OTP-14954</p> + </item> + <item> + <p>Added configuration switches for busy-wait and wake up + thresholds for dirty schedulers, and changing these + settings for normal schedulers will no longer affect + dirty schedulers. </p> <p>Refer to the documentation for + details. The new switches are <seealso + marker="erl#+sbwtdcpu">+sbwtdcpu</seealso>, <seealso + marker="erl#+sbwtdio">+sbwtdio</seealso>, <seealso + marker="erl#+swtdcpu">+swtdcpu</seealso>, and <seealso + marker="erl#+swtdio">+swtdio</seealso>.</p> <p>The + default busy wait threshold for dirty scheduler threads + has also been lowered to <c>short</c>.</p> + <p> + Own Id: OTP-14959</p> + </item> + <item> + <p> + The list of "taints" now also includes dynamic loaded + drivers in addition to NIF libraries. Statically linked + drivers and NIF libraries that are part of erts are not + included. The "taints" are returned by + <c>system_info(taints)</c> and printed in the header of + <c>erl_crash.dump</c> files.</p> + <p> + Own Id: OTP-14960</p> + </item> + <item> + <p>Added <c>instrument:allocations</c> and + <c>instrument:carriers</c> for retrieving information + about memory utilization and fragmentation.</p> + <p>The old <c>instrument</c> interface has been removed, + as have the related options <c>+Mim</c> and + <c>+Mis</c>.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-14961</p> + </item> + <item> + <p>The process suspend functionality used by the <seealso + marker="erlang#suspend_process/2">erlang:suspend_process/2</seealso> + BIF has been reimplemented using the newly introduced + true asynchronous signaling between processes. This + mainly to reduce memory usage in the process control + block of all processes, but also in order to simplify the + implementation.</p> <warning> <p>You can easily create + deadlocks if processes suspends each other (directly or + in circles). In ERTS versions prior to ERTS version 10.0, + the runtime system prevented such deadlocks, but this + prevention has now been removed due to performance + reasons.</p> </warning> <p>Other ERTS internal + functionality that used the previous process suspend + functionality have also been reimplemented to use + asynchronous signaling instead.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-14964 Aux Id: OTP-14589 </p> + </item> + <item> + <p>Added the <c>nifs</c> option to + <c>?MODULE:module_info/1</c> for listing a module's + installed NIF functions.</p> + <p> + Own Id: OTP-14965</p> + </item> + <item> + <p> + New implementation of <c>erlang:process_info/[1,2]</c>.</p> + <p> + In the general case when inspecting another process, the + new implementation sends an asynchronous process-info + request signal to the other process and waits for the + result instead of locking the other process and reading + the result directly. In some special cases where no + conflicts occur, signal order wont be violated, and the + amount of data requested is guaranteed to be small, the + inspected process may be inspected directly.</p> + <p> + Appropriate amount of reductions are now also bumped when + inspecting a process.</p> + <p> + Own Id: OTP-14966</p> + </item> + <item> + <p> + Removed process start time from crash dump in order to + save memory in process control block.</p> + <p> + Own Id: OTP-14975 Aux Id: PR-1597 </p> + </item> + <item> + <p> + Optimize <c>erlang:put/2</c> when updating existing key + with a new immediate value (atom, small integer, pid, + port).</p> + <p> + Own Id: OTP-14976</p> + </item> + <item> + <p> + <c>erlang:process_info/1</c> has been changed to no + longer include <c>messages</c> by default. Instead + <c>erlang:process_info/2</c> should be used.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-14986 Aux Id: PR-1745 </p> + </item> + <item> + <p> + New <c>erlang:system_info(ets_count)</c> to get total + number of ets tables existing at the local node.</p> + <p> + Own Id: OTP-14987</p> + </item> + <item> + <p> + New NIF functions: <c>enif_mutex_name</c>, + <c>enif_cond_name</c>, <c>enif_rwlock_name</c>, + <c>enif_thread_name</c>, <c>enif_vfprintf</c>, + <c>enif_vsnprintf</c>.</p> + <p> + Own Id: OTP-14994</p> + </item> + <item> + <p>When <c>erlang:system_flag(backtrace_depth, 0)</c> has + been called, all exceptions will now contain the entry + for <em>one</em> function (despite the zero). It used to + be that a hand-made stack backtrace passed to + <c>erlang:raise/3</c> would be be truncated to an empty + list.</p> + <p> + Own Id: OTP-15026</p> + </item> + <item> + <p> + Fixed bug for named <c>ets</c> tables which could cause + unexpected results from matchspec iteration functions + (<c>ets:select*</c> and <c>ets:match*</c>) if the table + was deleted and recreated with the same name during the + iteration. The iteration could incorrectly continue + through the recreated table. The expected correct + behavior is now for the iteration call to fail with a + <c>badarg</c> exception if the table is deleted before + the iteration has completed.</p> + <p> + Own Id: OTP-15031</p> + </item> + <item> + <p>Two new guards BIFs operating on maps have been added: + <c>map_get/2</c> and <c>is_map_key/2</c>. They do the + same as <c>maps:get/2</c> and <c>maps:is_key/2</c>, + respectively, except that they are allowed to be used in + guards.</p> + <p> + Own Id: OTP-15037 Aux Id: PR-1784, PR-1802 </p> + </item> + <item> + <p> + Release run-queue lock while cleaning up terminated dirty + process.</p> + <p> + Own Id: OTP-15081</p> + </item> + <item> + <p>The callback module passed as <c>-epmd_module</c> to + erl has been expanded to be able to do name and port + resolving.</p> <p>Documentation has also been added in + the <seealso + marker="kernel:erl_epmd"><c>erl_epmd</c></seealso> + reference manual and ERTS User's Guide <seealso + marker="erts:alt_disco">How to Implement an Alternative + Service Discovery for Erlang Distribution</seealso>.</p> + <p> + Own Id: OTP-15086 Aux Id: PR-1694 </p> + </item> + </list> + </section> + +</section> + +<section><title>Erts 9.3.3.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fixed a bug which caused an emulator crash when + <c>enif_send()</c> was called by a NIF that executed on a + dirty scheduler. The bug was either triggered when the + NIF called <c>enif_send()</c> without a message + environment, or when the process executing the NIF was + <c>send</c> traced.</p> + <p> + Own Id: OTP-15223</p> + </item> + <item> + <p> + Fixed a bug causing some Erlang references to be + inconsistently ordered. This could for example cause + failure to look up certain elements with references as + keys in search data structures. This bug was introduced + in R13B02.</p> + <p> + Thanks to Simon Cornish for finding the bug and supplying + a fix.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-15225</p> + </item> + </list> + </section> + +</section> + +<section><title>Erts 9.3.3.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p>Fixed a race condition in the inet driver that could + cause receive to hang when the emulator was compiled with + gcc 8.</p> + <p> + Own Id: OTP-15158 Aux Id: ERL-654 </p> + </item> + <item> + <p> + Fix bug in generation of erl_crash.dump, which could + cause VM to crash.</p> + <p> + Bug exist since erts-9.2 (OTP-20.2).</p> + <p> + Own Id: OTP-15181</p> + </item> + </list> + </section> + +</section> + +<section><title>Erts 9.3.3.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p>Fixed a rare bug that could cause processes to be + scheduled after they had been freed.</p> + <p> + Own Id: OTP-15067 Aux Id: ERL-573 </p> + </item> + </list> + </section> + +</section> + +<section><title>Erts 9.3.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fixed bug in <c>ets</c> that could cause VM crash if + process A terminates after fixating a table and process B + deletes the table at "the same time". The table fixation + could be done with <c>ets:safe_fixtable</c> or if process + A terminates in the middle of a long running + <c>select</c> or <c>match</c> call.</p> + <p> + Own Id: OTP-15109</p> + </item> + </list> + </section> + +</section> + <section><title>Erts 9.3.2</title> <section><title>Fixed Bugs and Malfunctions</title> @@ -1438,6 +2592,63 @@ </section> +<section><title>Erts 8.3.5.5</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p>Fixed a race condition in the inet driver that could + cause receive to hang when the emulator was compiled with + gcc 8.</p> + <p> + Own Id: OTP-15158 Aux Id: ERL-654 </p> + </item> + <item> + <p> + Fixed a bug causing some Erlang references to be + inconsistently ordered. This could for example cause + failure to look up certain elements with references as + keys in search data structures. This bug was introduced + in R13B02.</p> + <p> + Thanks to Simon Cornish for finding the bug and supplying + a fix.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-15225</p> + </item> + </list> + </section> + +</section> + +<section><title>Erts 8.3.5.4</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p>Fixed a bug in file closure on Unix; close(2) was + retried on EINTR which could cause a different (recently + opened) file to be closed as well.</p> + <p> + Own Id: OTP-14775</p> + </item> + <item> + <p> + A race-condition when tearing down a connection with + active node monitors could cause the runtime system to + crash.</p> + <p> + This bug was introduced in ERTS version 8.0 (OTP 19.0).</p> + <p> + Own Id: OTP-14781 Aux Id: OTP-13047 </p> + </item> + </list> + </section> + +</section> + <section><title>Erts 8.3.5.3</title> <section><title>Fixed Bugs and Malfunctions</title> @@ -8848,7 +10059,7 @@ you use erlang:halt/2 with an integer first argument and an option list containing {flush,false} as the second argument. Note that now is flushing not dependant of the - exit code, and you can not only flush async threads + exit code, and you cannot only flush async threads operations which we deemed as a strange behaviour anyway. </p> <p>Also, erlang:halt/1,2 has gotten a new feature: If the diff --git a/erts/doc/src/part.xml b/erts/doc/src/part.xml index fc39cb30e6..05e9a24af8 100644 --- a/erts/doc/src/part.xml +++ b/erts/doc/src/part.xml @@ -4,7 +4,7 @@ <part xmlns:xi="http://www.w3.org/2001/XInclude"> <header> <copyright> - <year>1996</year><year>2016</year> + <year>1996</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/erts/doc/src/run_erl.xml b/erts/doc/src/run_erl.xml index e4c1b943c4..fa36457489 100644 --- a/erts/doc/src/run_erl.xml +++ b/erts/doc/src/run_erl.xml @@ -4,7 +4,7 @@ <comref> <header> <copyright> - <year>1999</year><year>2016</year> + <year>1999</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/erts/emulator/beam/atom.c b/erts/emulator/beam/atom.c index e5b7616a0d..5381611fab 100644 --- a/erts/emulator/beam/atom.c +++ b/erts/emulator/beam/atom.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1996-2016. All Rights Reserved. + * Copyright Ericsson AB 1996-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/atom.h b/erts/emulator/beam/atom.h index 385120a8d9..ca920679c6 100644 --- a/erts/emulator/beam/atom.h +++ b/erts/emulator/beam/atom.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1996-2016. All Rights Reserved. + * Copyright Ericsson AB 1996-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/beam_bif_load.c b/erts/emulator/beam/beam_bif_load.c index a0dbd9ec7b..d221e6aea6 100644 --- a/erts/emulator/beam/beam_bif_load.c +++ b/erts/emulator/beam/beam_bif_load.c @@ -1753,6 +1753,7 @@ BIF_RETTYPE erts_internal_purge_module_2(BIF_ALIST_2) if (literals) { ErtsLiteralAreaRef *ref; + ErtsMessage *mp; ref = erts_alloc(ERTS_ALC_T_LITERAL_REF, sizeof(ErtsLiteralAreaRef)); ref->literal_area = literals; @@ -1767,10 +1768,12 @@ BIF_RETTYPE erts_internal_purge_module_2(BIF_ALIST_2) release_literal_areas.last = ref; } erts_mtx_unlock(&release_literal_areas.mtx); + mp = erts_alloc_message(0, NULL); + ERL_MESSAGE_TOKEN(mp) = am_undefined; erts_queue_proc_message(BIF_P, erts_literal_area_collector, 0, - erts_alloc_message(0, NULL), + mp, am_copy_literals); } diff --git a/erts/emulator/beam/beam_debug.c b/erts/emulator/beam/beam_debug.c index b8a8d06315..9633de2021 100644 --- a/erts/emulator/beam/beam_debug.c +++ b/erts/emulator/beam/beam_debug.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1998-2017. All Rights Reserved. + * Copyright Ericsson AB 1998-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -667,7 +667,7 @@ print_op(fmtfn_t to, void *to_arg, int op, int size, BeamInstr* addr) ap++; break; case 'l': /* fr(N) */ - erts_print(to, to_arg, "fr(%d)", loader_reg_index(ap[0])); + erts_print(to, to_arg, "fr(%d)", ap[0] / sizeof(FloatDef)); ap++; break; default: @@ -1191,6 +1191,7 @@ dirty_send_message(Process *c_p, Eterm to, Eterm tag) mp = erts_alloc_message_heap(rp, &rp_locks, 3, &hp, &ohp); msg = TUPLE2(hp, tag, c_p->common.id); + ERL_MESSAGE_TOKEN(mp) = am_undefined; erts_queue_proc_message(c_p, rp, rp_locks, mp, msg); if (rp == real_c_p) diff --git a/erts/emulator/beam/beam_emu.c b/erts/emulator/beam/beam_emu.c index ab5920a67e..aa61a2d7f9 100644 --- a/erts/emulator/beam/beam_emu.c +++ b/erts/emulator/beam/beam_emu.c @@ -3061,12 +3061,14 @@ erts_gc_update_map_exact(Process* p, Eterm* reg, Uint live, Uint n, Eterm* new_p Uint need; flatmap_t *old_mp, *mp; Eterm res; + Eterm* old_hp; Eterm* hp; Eterm* E; Eterm* old_keys; Eterm* old_vals; Eterm new_key; Eterm map; + int changed = 0; n /= 2; /* Number of values to be updated */ ASSERT(n > 0); @@ -3133,6 +3135,7 @@ erts_gc_update_map_exact(Process* p, Eterm* reg, Uint live, Uint n, Eterm* new_p * Update map, keeping the old key tuple. */ + old_hp = p->htop; hp = p->htop; E = p->stop; @@ -3155,20 +3158,26 @@ erts_gc_update_map_exact(Process* p, Eterm* reg, Uint live, Uint n, Eterm* new_p /* Not same keys */ *hp++ = *old_vals; } else { - GET_TERM(new_p[1], *hp); - hp++; - n--; + GET_TERM(new_p[1], *hp); + if(*hp != *old_vals) changed = 1; + hp++; + n--; if (n == 0) { - /* - * All updates done. Copy remaining values - * and return the result. - */ - for (i++, old_vals++; i < num_old; i++) { - *hp++ = *old_vals++; - } - ASSERT(hp == p->htop + need); - p->htop = hp; - return res; + /* + * All updates done. Copy remaining values + * if any changed or return the original one. + */ + if(changed) { + for (i++, old_vals++; i < num_old; i++) { + *hp++ = *old_vals++; + } + ASSERT(hp == p->htop + need); + p->htop = hp; + return res; + } else { + p->htop = old_hp; + return map; + } } else { new_p += 2; GET_TERM(*new_p, new_key); diff --git a/erts/emulator/beam/beam_load.c b/erts/emulator/beam/beam_load.c index e61199a8fd..50cbb37f3e 100644 --- a/erts/emulator/beam/beam_load.c +++ b/erts/emulator/beam/beam_load.c @@ -4245,21 +4245,55 @@ gen_make_fun2(LoaderState* stp, GenOpArg idx) { ErlFunEntry* fe; GenOp* op; + Uint arity, num_free; if (idx.val >= stp->num_lambdas) { - stp->lambda_error = "missing or short chunk 'FunT'"; - fe = 0; + stp->lambda_error = "missing or short chunk 'FunT'"; + fe = 0; + num_free = 0; + arity = 0; } else { - fe = stp->lambdas[idx.val].fe; + fe = stp->lambdas[idx.val].fe; + num_free = stp->lambdas[idx.val].num_free; + arity = fe->arity; } NEW_GENOP(stp, op); - op->op = genop_i_make_fun_2; - op->arity = 2; - op->a[0].type = TAG_u; - op->a[0].val = (BeamInstr) fe; - op->a[1].type = TAG_u; - op->a[1].val = stp->lambdas[idx.val].num_free; + + /* + * It's possible this is called before init process is started, + * skip the optimisation in such case. + */ + if (num_free == 0 && erts_init_process_id != ERTS_INVALID_PID) { + Uint lit; + Eterm* hp; + ErlFunThing* funp; + + lit = new_literal(stp, &hp, ERL_FUN_SIZE); + funp = (ErlFunThing *) hp; + erts_refc_inc(&fe->refc, 2); + funp->thing_word = HEADER_FUN; + funp->next = NULL; + funp->fe = fe; + funp->num_free = 0; + funp->creator = erts_init_process_id; + funp->arity = arity; + + op->op = genop_move_2; + op->arity = 2; + op->a[0].type = TAG_q; + op->a[0].val = lit; + op->a[1].type = TAG_x; + op->a[1].val = 0; + } else { + op->op = genop_i_make_fun_2; + op->arity = 2; + op->a[0].type = TAG_u; + op->a[0].val = (BeamInstr) fe; + op->a[1].type = TAG_u; + op->a[1].val = num_free; + } + op->next = NULL; return op; } diff --git a/erts/emulator/beam/beam_ranges.c b/erts/emulator/beam/beam_ranges.c index f0c9496341..9f3153724a 100644 --- a/erts/emulator/beam/beam_ranges.c +++ b/erts/emulator/beam/beam_ranges.c @@ -35,10 +35,8 @@ typedef struct { /* * Used for crash dumping of literals. The size of erts_dump_lit_areas is - * always twice the number of active ranges (to allow for literals in both - * current and old code). + * always at least the number of active ranges. */ - ErtsLiteralArea** erts_dump_lit_areas; Uint erts_dump_num_lit_areas; @@ -180,8 +178,8 @@ erts_end_staging_ranges(int commit) (erts_aint_t) (r[dst].modules + r[dst].n / 2)); - if (r[dst].allocated * 2 > erts_dump_num_lit_areas) { - erts_dump_num_lit_areas *= 2; + if (r[dst].allocated > erts_dump_num_lit_areas) { + erts_dump_num_lit_areas = r[dst].allocated * 2; erts_dump_lit_areas = (ErtsLiteralArea **) erts_realloc(ERTS_ALC_T_CRASH_DUMP, (void *) erts_dump_lit_areas, diff --git a/erts/emulator/beam/bif.c b/erts/emulator/beam/bif.c index 97e1ee1286..f18af8bcd7 100644 --- a/erts/emulator/beam/bif.c +++ b/erts/emulator/beam/bif.c @@ -2063,7 +2063,7 @@ do_send(Process *p, Eterm to, Eterm msg, Eterm *refp, ErtsSendContext *ctx) if (p == rp) rp_locks |= ERTS_PROC_LOCK_MAIN; /* send to local process */ - erts_send_message(p, rp, &rp_locks, msg, 0); + erts_send_message(p, rp, &rp_locks, msg); erts_proc_unlock(rp, p == rp ? (rp_locks & ~ERTS_PROC_LOCK_MAIN) diff --git a/erts/emulator/beam/bif.tab b/erts/emulator/beam/bif.tab index 7548924178..a770524221 100644 --- a/erts/emulator/beam/bif.tab +++ b/erts/emulator/beam/bif.tab @@ -629,7 +629,6 @@ bif maps:from_list/1 bif maps:is_key/2 bif maps:keys/1 bif maps:merge/2 -bif maps:new/0 bif maps:put/3 bif maps:remove/2 bif maps:update/3 diff --git a/erts/emulator/beam/bif_instrs.tab b/erts/emulator/beam/bif_instrs.tab index 0f074280db..00854471a9 100644 --- a/erts/emulator/beam/bif_instrs.tab +++ b/erts/emulator/beam/bif_instrs.tab @@ -2,7 +2,7 @@ // // %CopyrightBegin% // -// Copyright Ericsson AB 2017. All Rights Reserved. +// Copyright Ericsson AB 2017-2018. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/big.c b/erts/emulator/beam/big.c index c5cb268f09..84338769e0 100644 --- a/erts/emulator/beam/big.c +++ b/erts/emulator/beam/big.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1996-2017. All Rights Reserved. + * Copyright Ericsson AB 1996-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/big.h b/erts/emulator/beam/big.h index 7556205063..a1ad75708c 100644 --- a/erts/emulator/beam/big.h +++ b/erts/emulator/beam/big.h @@ -42,7 +42,7 @@ typedef Uint16 ErtsHalfDigit; #undef BIG_HAVE_DOUBLE_DIGIT typedef Uint32 ErtsHalfDigit; #else -#error "can not determine machine size" +#error "cannot determine machine size" #endif typedef Uint dsize_t; /* Vector size type */ diff --git a/erts/emulator/beam/binary.c b/erts/emulator/beam/binary.c index d53f75c279..6a349764b2 100644 --- a/erts/emulator/beam/binary.c +++ b/erts/emulator/beam/binary.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1996-2017. All Rights Reserved. + * Copyright Ericsson AB 1996-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/break.c b/erts/emulator/beam/break.c index 9ff52c92b8..81531f6cc8 100644 --- a/erts/emulator/beam/break.c +++ b/erts/emulator/beam/break.c @@ -108,6 +108,7 @@ process_killer(void) erts_exit(0, ""); switch(j) { case 'k': + ASSERT(erts_init_process_id != ERTS_INVALID_PID); /* Send a 'kill' exit signal from init process */ erts_proc_sig_send_exit(NULL, erts_init_process_id, rp->common.id, am_kill, NIL, diff --git a/erts/emulator/beam/bs_instrs.tab b/erts/emulator/beam/bs_instrs.tab index 94e0000c8b..61eb02a7a2 100644 --- a/erts/emulator/beam/bs_instrs.tab +++ b/erts/emulator/beam/bs_instrs.tab @@ -2,7 +2,7 @@ // // %CopyrightBegin% // -// Copyright Ericsson AB 2017. All Rights Reserved. +// Copyright Ericsson AB 2017-2018. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/code_ix.c b/erts/emulator/beam/code_ix.c index 34e46f5f33..50352b4084 100644 --- a/erts/emulator/beam/code_ix.c +++ b/erts/emulator/beam/code_ix.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2012-2016. All Rights Reserved. + * Copyright Ericsson AB 2012-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/copy.c b/erts/emulator/beam/copy.c index 7769a914db..e7bfd04b73 100644 --- a/erts/emulator/beam/copy.c +++ b/erts/emulator/beam/copy.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1996-2017. All Rights Reserved. + * Copyright Ericsson AB 1996-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/dist.c b/erts/emulator/beam/dist.c index 70474898b2..146c00b07d 100644 --- a/erts/emulator/beam/dist.c +++ b/erts/emulator/beam/dist.c @@ -1200,21 +1200,8 @@ erts_dsig_send_group_leader(ErtsDSigData *dsdp, Eterm leader, Eterm remote) #include <valgrind/valgrind.h> #include <valgrind/memcheck.h> -#ifndef HAVE_VALGRIND_PRINTF_XML -#define VALGRIND_PRINTF_XML VALGRIND_PRINTF -#endif - # define PURIFY_MSG(msg) \ - do { \ - char buf__[1]; size_t bufsz__ = sizeof(buf__); \ - if (erts_sys_explicit_8bit_getenv("VALGRIND_LOG_XML", buf__, &bufsz__) >= 0) { \ - VALGRIND_PRINTF_XML("<erlang_error_log>" \ - "%s, line %d: %s</erlang_error_log>\n", \ - __FILE__, __LINE__, msg); \ - } else { \ - VALGRIND_PRINTF("%s, line %d: %s", __FILE__, __LINE__, msg); \ - } \ - } while (0) + VALGRIND_PRINTF("%s, line %d: %s", __FILE__, __LINE__, msg) #else # define PURIFY_MSG(msg) #endif @@ -3685,6 +3672,7 @@ int erts_auto_connect(DistEntry* dep, Process *proc, ErtsProcLocks proc_locks) dhandle = erts_build_dhandle(&hp, ohp, dep); msg = TUPLE4(hp, am_auto_connect, dep->sysname, make_small(conn_id), dhandle); + ERL_MESSAGE_TOKEN(mp) = am_undefined; erts_queue_proc_message(proc, net_kernel, nk_locks, mp, msg); erts_proc_unlock(net_kernel, nk_locks); } diff --git a/erts/emulator/beam/erl_afit_alloc.c b/erts/emulator/beam/erl_afit_alloc.c index 23efe3bba4..38289ea78a 100644 --- a/erts/emulator/beam/erl_afit_alloc.c +++ b/erts/emulator/beam/erl_afit_alloc.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2003-2016. All Rights Reserved. + * Copyright Ericsson AB 2003-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/erl_alloc.c b/erts/emulator/beam/erl_alloc.c index 575d6ca867..36c46fd7aa 100644 --- a/erts/emulator/beam/erl_alloc.c +++ b/erts/emulator/beam/erl_alloc.c @@ -64,9 +64,6 @@ # error "Too many schedulers; cannot create that many pref alloc instances" #endif -#define ERTS_ALC_FIX_TYPE_IX(T) \ - (ERTS_ALC_T2N((T)) - ERTS_ALC_N_MIN_A_FIXED_SIZE) - #define ERTS_ALC_DEFAULT_MAX_THR_PREF ERTS_MAX_NO_OF_SCHEDULERS #if defined(SMALL_MEMORY) || defined(PURIFY) || defined(VALGRIND) @@ -114,7 +111,7 @@ typedef union { char align_afa[ERTS_ALC_CACHE_LINE_ALIGN_SIZE(sizeof(AFAllctr_t))]; AOFFAllctr_t aoffa; char align_aoffa[ERTS_ALC_CACHE_LINE_ALIGN_SIZE(sizeof(AOFFAllctr_t))]; -} ErtsAllocatorState_t; +} ErtsAllocatorState_t erts_align_attribute(ERTS_CACHE_LINE_SIZE); static ErtsAllocatorState_t std_alloc_state; static ErtsAllocatorState_t ll_alloc_state; @@ -156,20 +153,13 @@ ERTS_SCHED_PREF_QUICK_ALLOC_IMPL(aireq, ErtsAlcType_t erts_fix_core_allocator_ix; -enum allctr_type { - GOODFIT, - BESTFIT, - AFIT, - FIRSTFIT -}; - struct au_init { int enable; int thr_spec; int disable_allowed; int thr_spec_allowed; int carrier_migration_allowed; - enum allctr_type atype; + ErtsAlcStrat_t astrat; struct { AllctrInit_t util; GFAllctrInit_t gf; @@ -219,7 +209,9 @@ typedef struct { struct au_init test_alloc; } erts_alc_hndl_args_init_t; -#define ERTS_AU_INIT__ {0, 0, 1, 1, 1, GOODFIT, DEFAULT_ALLCTR_INIT, {1,1,1,1}} +#define ERTS_AU_INIT__ {0, 0, 1, 1, 1, \ + ERTS_ALC_S_GOODFIT, DEFAULT_ALLCTR_INIT, \ + {1,1,1,1}} #define SET_DEFAULT_ALLOC_OPTS(IP) \ do { \ @@ -233,7 +225,7 @@ set_default_sl_alloc_opts(struct au_init *ip) SET_DEFAULT_ALLOC_OPTS(ip); ip->enable = AU_ALLOC_DEFAULT_ENABLE(1); ip->thr_spec = 1; - ip->atype = GOODFIT; + ip->astrat = ERTS_ALC_S_GOODFIT; ip->init.util.name_prefix = "sl_"; ip->init.util.alloc_no = ERTS_ALC_A_SHORT_LIVED; #ifndef SMALL_MEMORY @@ -252,7 +244,7 @@ set_default_std_alloc_opts(struct au_init *ip) SET_DEFAULT_ALLOC_OPTS(ip); ip->enable = AU_ALLOC_DEFAULT_ENABLE(1); ip->thr_spec = 1; - ip->atype = BESTFIT; + ip->astrat = ERTS_ALC_S_BESTFIT; ip->init.util.name_prefix = "std_"; ip->init.util.alloc_no = ERTS_ALC_A_STANDARD; #ifndef SMALL_MEMORY @@ -270,7 +262,7 @@ set_default_ll_alloc_opts(struct au_init *ip) SET_DEFAULT_ALLOC_OPTS(ip); ip->enable = AU_ALLOC_DEFAULT_ENABLE(1); ip->thr_spec = 0; - ip->atype = BESTFIT; + ip->astrat = ERTS_ALC_S_BESTFIT; ip->init.bf.ao = 1; ip->init.util.ramv = 0; ip->init.util.mmsbc = 0; @@ -299,7 +291,7 @@ set_default_literal_alloc_opts(struct au_init *ip) ip->disable_allowed = 0; ip->thr_spec_allowed = 0; ip->carrier_migration_allowed = 0; - ip->atype = BESTFIT; + ip->astrat = ERTS_ALC_S_BESTFIT; ip->init.bf.ao = 1; ip->init.util.ramv = 0; ip->init.util.mmsbc = 0; @@ -349,7 +341,7 @@ set_default_exec_alloc_opts(struct au_init *ip) ip->disable_allowed = 0; ip->thr_spec_allowed = 0; ip->carrier_migration_allowed = 0; - ip->atype = BESTFIT; + ip->astrat = ERTS_ALC_S_BESTFIT; ip->init.bf.ao = 1; ip->init.util.ramv = 0; ip->init.util.mmsbc = 0; @@ -378,7 +370,7 @@ set_default_temp_alloc_opts(struct au_init *ip) ip->thr_spec = 1; ip->disable_allowed = 0; ip->carrier_migration_allowed = 0; - ip->atype = AFIT; + ip->astrat = ERTS_ALC_S_AFIT; ip->init.util.name_prefix = "temp_"; ip->init.util.alloc_no = ERTS_ALC_A_TEMPORARY; #ifndef SMALL_MEMORY @@ -397,7 +389,7 @@ set_default_eheap_alloc_opts(struct au_init *ip) SET_DEFAULT_ALLOC_OPTS(ip); ip->enable = AU_ALLOC_DEFAULT_ENABLE(1); ip->thr_spec = 1; - ip->atype = GOODFIT; + ip->astrat = ERTS_ALC_S_GOODFIT; ip->init.util.name_prefix = "eheap_"; ip->init.util.alloc_no = ERTS_ALC_A_EHEAP; #ifndef SMALL_MEMORY @@ -416,7 +408,7 @@ set_default_binary_alloc_opts(struct au_init *ip) SET_DEFAULT_ALLOC_OPTS(ip); ip->enable = AU_ALLOC_DEFAULT_ENABLE(1); ip->thr_spec = 1; - ip->atype = BESTFIT; + ip->astrat = ERTS_ALC_S_BESTFIT; ip->init.util.name_prefix = "binary_"; ip->init.util.alloc_no = ERTS_ALC_A_BINARY; #ifndef SMALL_MEMORY @@ -435,7 +427,7 @@ set_default_ets_alloc_opts(struct au_init *ip) SET_DEFAULT_ALLOC_OPTS(ip); ip->enable = AU_ALLOC_DEFAULT_ENABLE(1); ip->thr_spec = 1; - ip->atype = BESTFIT; + ip->astrat = ERTS_ALC_S_BESTFIT; ip->init.util.name_prefix = "ets_"; ip->init.util.alloc_no = ERTS_ALC_A_ETS; #ifndef SMALL_MEMORY @@ -453,7 +445,7 @@ set_default_driver_alloc_opts(struct au_init *ip) SET_DEFAULT_ALLOC_OPTS(ip); ip->enable = AU_ALLOC_DEFAULT_ENABLE(1); ip->thr_spec = 1; - ip->atype = BESTFIT; + ip->astrat = ERTS_ALC_S_BESTFIT; ip->init.util.name_prefix = "driver_"; ip->init.util.alloc_no = ERTS_ALC_A_DRIVER; #ifndef SMALL_MEMORY @@ -473,7 +465,7 @@ set_default_fix_alloc_opts(struct au_init *ip, SET_DEFAULT_ALLOC_OPTS(ip); ip->enable = AU_ALLOC_DEFAULT_ENABLE(1); ip->thr_spec = 1; - ip->atype = BESTFIT; + ip->astrat = ERTS_ALC_S_BESTFIT; ip->init.bf.ao = 1; ip->init.util.name_prefix = "fix_"; ip->init.util.fix_type_size = fix_type_sizes; @@ -493,7 +485,7 @@ set_default_test_alloc_opts(struct au_init *ip) SET_DEFAULT_ALLOC_OPTS(ip); ip->enable = 0; /* Disabled by default */ ip->thr_spec = -1 * erts_no_schedulers; - ip->atype = FIRSTFIT; + ip->astrat = ERTS_ALC_S_FIRSTFIT; ip->init.aoff.crr_order = FF_AOFF; ip->init.aoff.blk_order = FF_BF; ip->init.util.name_prefix = "test_"; @@ -552,8 +544,8 @@ start_au_allocator(ErtsAlcType_t alctr_n, static void refuse_af_strategy(struct au_init *init) { - if (init->atype == AFIT) - init->atype = GOODFIT; + if (init->astrat == ERTS_ALC_S_AFIT) + init->astrat = ERTS_ALC_S_GOODFIT; } #ifdef HARD_DEBUG @@ -576,7 +568,10 @@ static void adjust_fix_alloc_sizes(UWord extra_block_size) for (i=0; i < tspec->size; i++) { Allctr_t* allctr = tspec->allctr[i]; for (j=0; j < ERTS_ALC_NO_FIXED_SIZES; ++j) { - allctr->fix[j].type_size += extra_block_size; + size_t size = allctr->fix[j].type_size; + size = MAX(size + extra_block_size, + sizeof(ErtsAllctrDDBlock_t)); + allctr->fix[j].type_size = size; } } } @@ -584,8 +579,11 @@ static void adjust_fix_alloc_sizes(UWord extra_block_size) { Allctr_t* allctr = erts_allctrs_info[ERTS_ALC_A_FIXED_SIZE].extra; for (j=0; j < ERTS_ALC_NO_FIXED_SIZES; ++j) { - allctr->fix[j].type_size += extra_block_size; - } + size_t size = allctr->fix[j].type_size; + size = MAX(size + extra_block_size, + sizeof(ErtsAllctrDDBlock_t)); + allctr->fix[j].type_size = size; + } } } } @@ -597,7 +595,7 @@ strategy_support_carrier_migration(struct au_init *auip) * Currently only aoff* and ageff* support carrier * migration, i.e, type AOFIRSTFIT. */ - return auip->atype == FIRSTFIT; + return auip->astrat == ERTS_ALC_S_FIRSTFIT; } static ERTS_INLINE void @@ -612,7 +610,7 @@ adjust_carrier_migration_support(struct au_init *auip) */ if (!strategy_support_carrier_migration(auip)) { /* Default to aoffcbf */ - auip->atype = FIRSTFIT; + auip->astrat = ERTS_ALC_S_FIRSTFIT; auip->init.aoff.crr_order = FF_AOFF; auip->init.aoff.blk_order = FF_BF; } @@ -1018,7 +1016,7 @@ start_au_allocator(ErtsAlcType_t alctr_n, int i; int size = 1; void *as0; - enum allctr_type atype; + ErtsAlcStrat_t astrat; ErtsAllocatorFunctions_t *af = &erts_allctrs[alctr_n]; ErtsAllocatorInfo_t *ai = &erts_allctrs_info[alctr_n]; ErtsAllocatorThrSpec_t *tspec = &erts_allctr_thr_spec[alctr_n]; @@ -1077,7 +1075,7 @@ start_au_allocator(ErtsAlcType_t alctr_n, for (i = 0; i < size; i++) { Allctr_t *as; - atype = init->atype; + astrat = init->astrat; if (!init->thr_spec) as0 = state; @@ -1094,8 +1092,8 @@ start_au_allocator(ErtsAlcType_t alctr_n, if (i != 0) init->init.util.ts = 0; else { - if (atype == AFIT) - atype = GOODFIT; + if (astrat == ERTS_ALC_S_AFIT) + astrat = ERTS_ALC_S_GOODFIT; init->init.util.ts = 1; } init->init.util.tspec = init->thr_spec + 1; @@ -1109,25 +1107,26 @@ start_au_allocator(ErtsAlcType_t alctr_n, (((char *) fix_lists) + fix_list_size)); } + init->init.util.alloc_strat = astrat; init->init.util.ix = i; - switch (atype) { - case GOODFIT: + switch (astrat) { + case ERTS_ALC_S_GOODFIT: as = erts_gfalc_start((GFAllctr_t *) as0, &init->init.gf, &init->init.util); break; - case BESTFIT: + case ERTS_ALC_S_BESTFIT: as = erts_bfalc_start((BFAllctr_t *) as0, &init->init.bf, &init->init.util); break; - case AFIT: + case ERTS_ALC_S_AFIT: as = erts_afalc_start((AFAllctr_t *) as0, &init->init.af, &init->init.util); break; - case FIRSTFIT: + case ERTS_ALC_S_FIRSTFIT: as = erts_aoffalc_start((AOFFAllctr_t *) as0, &init->init.aoff, &init->init.util); @@ -1363,51 +1362,59 @@ handle_au_arg(struct au_init *auip, else if(has_prefix("as", sub_param)) { char *alg = get_value(sub_param + 2, argv, ip); if (sys_strcmp("bf", alg) == 0) { - auip->atype = BESTFIT; + auip->astrat = ERTS_ALC_S_BESTFIT; auip->init.bf.ao = 0; } else if (sys_strcmp("aobf", alg) == 0) { - auip->atype = BESTFIT; + auip->astrat = ERTS_ALC_S_BESTFIT; auip->init.bf.ao = 1; } else if (sys_strcmp("gf", alg) == 0) { - auip->atype = GOODFIT; + auip->astrat = ERTS_ALC_S_GOODFIT; } else if (sys_strcmp("af", alg) == 0) { - auip->atype = AFIT; + auip->astrat = ERTS_ALC_S_AFIT; } else if (sys_strcmp("aoff", alg) == 0) { - auip->atype = FIRSTFIT; + auip->astrat = ERTS_ALC_S_FIRSTFIT; auip->init.aoff.crr_order = FF_AOFF; auip->init.aoff.blk_order = FF_AOFF; } else if (sys_strcmp("aoffcbf", alg) == 0) { - auip->atype = FIRSTFIT; + auip->astrat = ERTS_ALC_S_FIRSTFIT; auip->init.aoff.crr_order = FF_AOFF; auip->init.aoff.blk_order = FF_BF; } else if (sys_strcmp("aoffcaobf", alg) == 0) { - auip->atype = FIRSTFIT; + auip->astrat = ERTS_ALC_S_FIRSTFIT; auip->init.aoff.crr_order = FF_AOFF; auip->init.aoff.blk_order = FF_AOBF; } else if (sys_strcmp("ageffcaoff", alg) == 0) { - auip->atype = FIRSTFIT; + auip->astrat = ERTS_ALC_S_FIRSTFIT; auip->init.aoff.crr_order = FF_AGEFF; auip->init.aoff.blk_order = FF_AOFF; } else if (sys_strcmp("ageffcbf", alg) == 0) { - auip->atype = FIRSTFIT; + auip->astrat = ERTS_ALC_S_FIRSTFIT; auip->init.aoff.crr_order = FF_AGEFF; auip->init.aoff.blk_order = FF_BF; } else if (sys_strcmp("ageffcaobf", alg) == 0) { - auip->atype = FIRSTFIT; + auip->astrat = ERTS_ALC_S_FIRSTFIT; auip->init.aoff.crr_order = FF_AGEFF; auip->init.aoff.blk_order = FF_AOBF; } else { - bad_value(param, sub_param + 1, alg); + if (auip->init.util.alloc_no == ERTS_ALC_A_TEST + && sys_strcmp("chaosff", alg) == 0) { + auip->astrat = ERTS_ALC_S_FIRSTFIT; + auip->init.aoff.crr_order = FF_CHAOS; + auip->init.aoff.blk_order = FF_CHAOS; + } + else { + bad_value(param, sub_param + 1, alg); + } } if (!strategy_support_carrier_migration(auip)) auip->init.util.acul = 0; @@ -2030,33 +2037,55 @@ erts_realloc_n_enomem(ErtsAlcType_t n, void *ptr, Uint size) } static ERTS_INLINE UWord -alcu_size(ErtsAlcType_t ai, ErtsAlcUFixInfo_t *fi, int fisz) +alcu_size(ErtsAlcType_t alloc_no, ErtsAlcUFixInfo_t *fi, int fisz) { - UWord res = 0; + UWord res; + int ai; - ASSERT(erts_allctrs_info[ai].enabled); - ASSERT(erts_allctrs_info[ai].alloc_util); + if (!erts_allctrs_info[alloc_no].thr_spec) { + AllctrSize_t size; + Allctr_t *allctr; - if (!erts_allctrs_info[ai].thr_spec) { - Allctr_t *allctr = erts_allctrs_info[ai].extra; - AllctrSize_t asize; - erts_alcu_current_size(allctr, &asize, fi, fisz); - res += asize.blocks; + allctr = erts_allctrs_info[alloc_no].extra; + erts_alcu_current_size(allctr, &size, fi, fisz); + + return size.blocks; } - else { - ErtsAllocatorThrSpec_t *tspec = &erts_allctr_thr_spec[ai]; - int i; - ASSERT(tspec->enabled); + res = 0; - for (i = tspec->size - 1; i >= 0; i--) { - Allctr_t *allctr = tspec->allctr[i]; - AllctrSize_t asize; - if (allctr) { - erts_alcu_current_size(allctr, &asize, fi, fisz); - res += asize.blocks; - } - } + /* Thread-specific allocators can migrate carriers across types, so we have + * to visit every allocator type to gather information on blocks that were + * allocated by us. */ + for (ai = ERTS_ALC_A_MIN; ai < ERTS_ALC_A_MAX; ai++) { + ErtsAllocatorThrSpec_t *tspec; + Allctr_t *allctr; + int i; + + if (!erts_allctrs_info[ai].thr_spec) { + continue; + } + + tspec = &erts_allctr_thr_spec[ai]; + ASSERT(tspec->enabled); + + for (i = tspec->size - 1; i >= 0; i--) { + allctr = tspec->allctr[i]; + + if (allctr) { + AllctrSize_t size; + + if (ai == alloc_no) { + erts_alcu_current_size(allctr, &size, fi, fisz); + } else { + erts_alcu_foreign_size(allctr, alloc_no, &size); + } + + ASSERT(((SWord)size.blocks) >= 0); + + res += size.blocks; + } + } } return res; @@ -2400,6 +2429,7 @@ erts_memory(fmtfn_t *print_to_p, void *print_to_arg, void *proc, Eterm earg) } if (want_tot_or_sys) { + ASSERT(size.total >= size.processes); size.system = size.total - size.processes; } @@ -3459,29 +3489,29 @@ UWord erts_alc_test(UWord op, UWord a1, UWord a2, UWord a3) switch (op) { case 0xf00: if (((Allctr_t *) a1)->thread_safe) - return (UWord) erts_alcu_alloc_ts(ERTS_ALC_T_UNDEF, + return (UWord) erts_alcu_alloc_ts(ERTS_ALC_T_TEST, (void *) a1, (Uint) a2); else - return (UWord) erts_alcu_alloc(ERTS_ALC_T_UNDEF, + return (UWord) erts_alcu_alloc(ERTS_ALC_T_TEST, (void *) a1, (Uint) a2); case 0xf01: if (((Allctr_t *) a1)->thread_safe) - return (UWord) erts_alcu_realloc_ts(ERTS_ALC_T_UNDEF, + return (UWord) erts_alcu_realloc_ts(ERTS_ALC_T_TEST, (void *) a1, (void *) a2, (Uint) a3); else - return (UWord) erts_alcu_realloc(ERTS_ALC_T_UNDEF, + return (UWord) erts_alcu_realloc(ERTS_ALC_T_TEST, (void *) a1, (void *) a2, (Uint) a3); case 0xf02: if (((Allctr_t *) a1)->thread_safe) - erts_alcu_free_ts(ERTS_ALC_T_UNDEF, (void *) a1, (void *) a2); + erts_alcu_free_ts(ERTS_ALC_T_TEST, (void *) a1, (void *) a2); else - erts_alcu_free(ERTS_ALC_T_UNDEF, (void *) a1, (void *) a2); + erts_alcu_free(ERTS_ALC_T_TEST, (void *) a1, (void *) a2); return 0; case 0xf03: { Allctr_t *allctr; @@ -3489,8 +3519,10 @@ UWord erts_alc_test(UWord op, UWord a1, UWord a2, UWord a3) SET_DEFAULT_ALLOC_OPTS(&init); init.enable = 1; - init.atype = GOODFIT; + init.astrat = ERTS_ALC_S_GOODFIT; init.init.util.name_prefix = (char *) a1; + init.init.util.alloc_no = ERTS_ALC_A_TEST; + init.init.util.alloc_strat = init.astrat; init.init.util.ts = 1; if ((char **) a3) { char **argv = (char **) a3; @@ -3504,31 +3536,31 @@ UWord erts_alc_test(UWord op, UWord a1, UWord a2, UWord a3) } } - switch (init.atype) { - case GOODFIT: + switch (init.astrat) { + case ERTS_ALC_S_GOODFIT: allctr = erts_gfalc_start((GFAllctr_t *) - erts_alloc(ERTS_ALC_T_UNDEF, + erts_alloc(ERTS_ALC_T_TEST, sizeof(GFAllctr_t)), &init.init.gf, &init.init.util); break; - case BESTFIT: + case ERTS_ALC_S_BESTFIT: allctr = erts_bfalc_start((BFAllctr_t *) - erts_alloc(ERTS_ALC_T_UNDEF, + erts_alloc(ERTS_ALC_T_TEST, sizeof(BFAllctr_t)), &init.init.bf, &init.init.util); break; - case AFIT: + case ERTS_ALC_S_AFIT: allctr = erts_afalc_start((AFAllctr_t *) - erts_alloc(ERTS_ALC_T_UNDEF, + erts_alloc(ERTS_ALC_T_TEST, sizeof(AFAllctr_t)), &init.init.af, &init.init.util); break; - case FIRSTFIT: + case ERTS_ALC_S_FIRSTFIT: allctr = erts_aoffalc_start((AOFFAllctr_t *) - erts_alloc(ERTS_ALC_T_UNDEF, + erts_alloc(ERTS_ALC_T_TEST, sizeof(AOFFAllctr_t)), &init.init.aoff, &init.init.util); @@ -3544,7 +3576,7 @@ UWord erts_alc_test(UWord op, UWord a1, UWord a2, UWord a3) } case 0xf04: erts_alcu_stop((Allctr_t *) a1); - erts_free(ERTS_ALC_T_UNDEF, (void *) a1); + erts_free(ERTS_ALC_T_TEST, (void *) a1); break; case 0xf05: return (UWord) 1; case 0xf06: return (UWord) ((Allctr_t *) a1)->thread_safe; @@ -3554,7 +3586,7 @@ UWord erts_alc_test(UWord op, UWord a1, UWord a2, UWord a3) case 0xf07: return (UWord) ((Allctr_t *) a1)->thread_safe; #endif case 0xf08: { - ethr_mutex *mtx = erts_alloc(ERTS_ALC_T_UNDEF, sizeof(ethr_mutex)); + ethr_mutex *mtx = erts_alloc(ERTS_ALC_T_TEST, sizeof(ethr_mutex)); if (ethr_mutex_init(mtx) != 0) ERTS_ALC_TEST_ABORT; return (UWord) mtx; @@ -3563,7 +3595,7 @@ UWord erts_alc_test(UWord op, UWord a1, UWord a2, UWord a3) ethr_mutex *mtx = (ethr_mutex *) a1; if (ethr_mutex_destroy(mtx) != 0) ERTS_ALC_TEST_ABORT; - erts_free(ERTS_ALC_T_UNDEF, (void *) mtx); + erts_free(ERTS_ALC_T_TEST, (void *) mtx); break; } case 0xf0a: @@ -3573,7 +3605,7 @@ UWord erts_alc_test(UWord op, UWord a1, UWord a2, UWord a3) ethr_mutex_unlock((ethr_mutex *) a1); break; case 0xf0c: { - ethr_cond *cnd = erts_alloc(ERTS_ALC_T_UNDEF, sizeof(ethr_cond)); + ethr_cond *cnd = erts_alloc(ERTS_ALC_T_TEST, sizeof(ethr_cond)); if (ethr_cond_init(cnd) != 0) ERTS_ALC_TEST_ABORT; return (UWord) cnd; @@ -3582,7 +3614,7 @@ UWord erts_alc_test(UWord op, UWord a1, UWord a2, UWord a3) ethr_cond *cnd = (ethr_cond *) a1; if (ethr_cond_destroy(cnd) != 0) ERTS_ALC_TEST_ABORT; - erts_free(ERTS_ALC_T_UNDEF, (void *) cnd); + erts_free(ERTS_ALC_T_TEST, (void *) cnd); break; } case 0xf0e: @@ -3596,7 +3628,7 @@ UWord erts_alc_test(UWord op, UWord a1, UWord a2, UWord a3) break; } case 0xf10: { - ethr_tid *tid = erts_alloc(ERTS_ALC_T_UNDEF, sizeof(ethr_tid)); + ethr_tid *tid = erts_alloc(ERTS_ALC_T_TEST, sizeof(ethr_tid)); if (ethr_thr_create(tid, (void * (*)(void *)) a1, (void *) a2, @@ -3608,7 +3640,7 @@ UWord erts_alc_test(UWord op, UWord a1, UWord a2, UWord a3) ethr_tid *tid = (ethr_tid *) a1; if (ethr_thr_join(*tid, NULL) != 0) ERTS_ALC_TEST_ABORT; - erts_free(ERTS_ALC_T_UNDEF, (void *) tid); + erts_free(ERTS_ALC_T_TEST, (void *) tid); break; } case 0xf12: @@ -3960,9 +3992,10 @@ check_memory_fence(void *ptr, Uint *size, ErtsAlcType_t n, int func) static ErtsAllocatorFunctions_t real_allctrs[ERTS_ALC_A_MAX+1]; static void * -debug_alloc(ErtsAlcType_t n, void *extra, Uint size) +debug_alloc(ErtsAlcType_t type, void *extra, Uint size) { ErtsAllocatorFunctions_t *real_af = (ErtsAllocatorFunctions_t *) extra; + ErtsAlcType_t n; Uint dsize; void *res; @@ -3970,9 +4003,11 @@ debug_alloc(ErtsAlcType_t n, void *extra, Uint size) erts_hdbg_chk_blks(); #endif + n = ERTS_ALC_T2N(type); + ASSERT(ERTS_ALC_N_MIN <= n && n <= ERTS_ALC_N_MAX); dsize = size + FENCE_SZ; - res = (*real_af->alloc)(n, real_af->extra, dsize); + res = (*real_af->alloc)(type, real_af->extra, dsize); res = set_memory_fence(res, size, n); @@ -3986,14 +4021,17 @@ debug_alloc(ErtsAlcType_t n, void *extra, Uint size) static void * -debug_realloc(ErtsAlcType_t n, void *extra, void *ptr, Uint size) +debug_realloc(ErtsAlcType_t type, void *extra, void *ptr, Uint size) { ErtsAllocatorFunctions_t *real_af = (ErtsAllocatorFunctions_t *) extra; + ErtsAlcType_t n; Uint dsize; Uint old_size; void *dptr; void *res; + n = ERTS_ALC_T2N(type); + ASSERT(ERTS_ALC_N_MIN <= n && n <= ERTS_ALC_N_MAX); dsize = size + FENCE_SZ; @@ -4008,7 +4046,7 @@ debug_realloc(ErtsAlcType_t n, void *extra, void *ptr, Uint size) 0xf, sizeof(Uint) + old_size - size); - res = (*real_af->realloc)(n, real_af->extra, dptr, dsize); + res = (*real_af->realloc)(type, real_af->extra, dptr, dsize); res = set_memory_fence(res, size, n); @@ -4021,12 +4059,16 @@ debug_realloc(ErtsAlcType_t n, void *extra, void *ptr, Uint size) } static void -debug_free(ErtsAlcType_t n, void *extra, void *ptr) +debug_free(ErtsAlcType_t type, void *extra, void *ptr) { ErtsAllocatorFunctions_t *real_af = (ErtsAllocatorFunctions_t *) extra; + ErtsAlcType_t n; void *dptr; Uint size; - int free_pattern = n; + int free_pattern; + + n = ERTS_ALC_T2N(type); + free_pattern = n; ASSERT(ERTS_ALC_N_MIN <= n && n <= ERTS_ALC_N_MAX); @@ -4041,7 +4083,7 @@ debug_free(ErtsAlcType_t n, void *extra, void *ptr) #endif sys_memset((void *) dptr, free_pattern, size + FENCE_SZ); - (*real_af->free)(n, real_af->extra, dptr); + (*real_af->free)(type, real_af->extra, dptr); #ifdef PRINT_OPS fprintf(stderr, "free(%s, 0x%lx)\r\n", ERTS_ALC_N2TD(n), (Uint) ptr); diff --git a/erts/emulator/beam/erl_alloc.h b/erts/emulator/beam/erl_alloc.h index 578a3717d9..c13cf3f5b0 100644 --- a/erts/emulator/beam/erl_alloc.h +++ b/erts/emulator/beam/erl_alloc.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2002-2017. All Rights Reserved. + * Copyright Ericsson AB 2002-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -26,10 +26,23 @@ #define ERL_THR_PROGRESS_TSD_TYPE_ONLY #include "erl_thr_progress.h" #undef ERL_THR_PROGRESS_TSD_TYPE_ONLY -#include "erl_alloc_util.h" #include "erl_threads.h" #include "erl_mmap.h" +typedef enum { + ERTS_ALC_S_INVALID = 0, + + ERTS_ALC_S_GOODFIT, + ERTS_ALC_S_BESTFIT, + ERTS_ALC_S_AFIT, + ERTS_ALC_S_FIRSTFIT, + + ERTS_ALC_S_MIN = ERTS_ALC_S_GOODFIT, + ERTS_ALC_S_MAX = ERTS_ALC_S_FIRSTFIT +} ErtsAlcStrat_t; + +#include "erl_alloc_util.h" + #ifdef DEBUG # undef ERTS_ALC_WANT_INLINE # define ERTS_ALC_WANT_INLINE 0 @@ -52,6 +65,14 @@ #define ERTS_ALC_NO_FIXED_SIZES \ (ERTS_ALC_N_MAX_A_FIXED_SIZE - ERTS_ALC_N_MIN_A_FIXED_SIZE + 1) +#define ERTS_ALC_IS_FIX_TYPE(T) \ + (ERTS_ALC_T2N(T) >= ERTS_ALC_N_MIN_A_FIXED_SIZE && \ + ERTS_ALC_T2N(T) <= ERTS_ALC_N_MAX_A_FIXED_SIZE) + +#define ERTS_ALC_FIX_TYPE_IX(T) \ + (ASSERT(ERTS_ALC_IS_FIX_TYPE(T)), \ + ERTS_ALC_T2N((T)) - ERTS_ALC_N_MIN_A_FIXED_SIZE) + void erts_sys_alloc_init(void); void *erts_sys_alloc(ErtsAlcType_t, void *, Uint); void *erts_sys_realloc(ErtsAlcType_t, void *, void *, Uint); @@ -228,7 +249,7 @@ void *erts_alloc(ErtsAlcType_t type, Uint size) void *res; ERTS_MSACC_PUSH_AND_SET_STATE_X(ERTS_MSACC_STATE_ALLOC); res = (*erts_allctrs[ERTS_ALC_T2A(type)].alloc)( - ERTS_ALC_T2N(type), + type, erts_allctrs[ERTS_ALC_T2A(type)].extra, size); if (!res) @@ -243,7 +264,7 @@ void *erts_realloc(ErtsAlcType_t type, void *ptr, Uint size) void *res; ERTS_MSACC_PUSH_AND_SET_STATE_X(ERTS_MSACC_STATE_ALLOC); res = (*erts_allctrs[ERTS_ALC_T2A(type)].realloc)( - ERTS_ALC_T2N(type), + type, erts_allctrs[ERTS_ALC_T2A(type)].extra, ptr, size); @@ -258,7 +279,7 @@ void erts_free(ErtsAlcType_t type, void *ptr) { ERTS_MSACC_PUSH_AND_SET_STATE_X(ERTS_MSACC_STATE_ALLOC); (*erts_allctrs[ERTS_ALC_T2A(type)].free)( - ERTS_ALC_T2N(type), + type, erts_allctrs[ERTS_ALC_T2A(type)].extra, ptr); ERTS_MSACC_POP_STATE_X(); @@ -271,7 +292,7 @@ void *erts_alloc_fnf(ErtsAlcType_t type, Uint size) void *res; ERTS_MSACC_PUSH_AND_SET_STATE_X(ERTS_MSACC_STATE_ALLOC); res = (*erts_allctrs[ERTS_ALC_T2A(type)].alloc)( - ERTS_ALC_T2N(type), + type, erts_allctrs[ERTS_ALC_T2A(type)].extra, size); ERTS_MSACC_POP_STATE_X(); @@ -285,7 +306,7 @@ void *erts_realloc_fnf(ErtsAlcType_t type, void *ptr, Uint size) void *res; ERTS_MSACC_PUSH_AND_SET_STATE_X(ERTS_MSACC_STATE_ALLOC); res = (*erts_allctrs[ERTS_ALC_T2A(type)].realloc)( - ERTS_ALC_T2N(type), + type, erts_allctrs[ERTS_ALC_T2A(type)].extra, ptr, size); diff --git a/erts/emulator/beam/erl_alloc.types b/erts/emulator/beam/erl_alloc.types index 5409b89bab..f1e99820af 100644 --- a/erts/emulator/beam/erl_alloc.types +++ b/erts/emulator/beam/erl_alloc.types @@ -30,10 +30,10 @@ # name space). # * Types, allocators, classes, and descriptions have different name # spaces. -# * The type, allocator, and class names INVALID are reserved and can -# not be used. +# * The type, allocator, and class names INVALID are reserved and +# cannot be used. # * The descriptions invalid_allocator, invalid_class, and invalid_type -# are reserved and can not be used. +# are reserved and cannot be used. # * Declarations can be done conditionally by use of a # +if <boolean_variable> # diff --git a/erts/emulator/beam/erl_alloc_util.c b/erts/emulator/beam/erl_alloc_util.c index fdf355d503..b7a8b9c2d0 100644 --- a/erts/emulator/beam/erl_alloc_util.c +++ b/erts/emulator/beam/erl_alloc_util.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2002-2017. All Rights Reserved. + * Copyright Ericsson AB 2002-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -96,7 +96,6 @@ static int initialized = 0; #define MBC_REALLOC_ALWAYS_MOVES #endif - /* alloc_util global parameters */ static Uint sys_alloc_carrier_size; #if HAVE_ERTS_MSEG @@ -113,32 +112,38 @@ static int allow_sys_alloc_carriers; #define DEC_CC(CC) ((CC)--) -/* Multi block carrier (MBC) memory layout in R16: +/* Multi block carrier (MBC) memory layout in OTP 22: Empty MBC: -[Carrier_t|pad|Block_t L0T|fhdr| free... ] +[Carrier_t|pad|Block_t L0T0|fhdr| free... ] MBC after allocating first block: -[Carrier_t|pad|Block_t 000| udata |pad|Block_t L0T|fhdr| free... ] +[Carrier_t|pad|Block_t 0000| udata |pad|Block_t L0T0|fhdr| free... ] MBC after allocating second block: -[Carrier_t|pad|Block_t 000| udata |pad|Block_t 000| udata |pad|Block_t L0T|fhdr| free... ] +[Carrier_t|pad|Block_t 0000| udata |pad|Block_t 0000| udata |pad|Block_t L0T0|fhdr| free... ] MBC after deallocating first block: -[Carrier_t|pad|Block_t 00T|fhdr| free |FreeBlkFtr_t|Block_t 0P0| udata |pad|Block_t L0T|fhdr| free... ] +[Carrier_t|pad|Block_t 00T0|fhdr| free |FreeBlkFtr_t|Block_t 0P00| udata |pad|Block_t L0T0|fhdr| free... ] +MBC after allocating first block, with allocation tagging enabled: +[Carrier_t|pad|Block_t 000A| udata |atag|pad|Block_t L0T0|fhdr| free... ] udata = Allocated user data + atag = A tag with basic metadata about this allocation pad = Padding to ensure correct alignment for user data fhdr = Allocator specific header to keep track of free block free = Unused free memory T = This block is free (THIS_FREE_BLK_HDR_FLG) P = Previous block is free (PREV_FREE_BLK_HDR_FLG) L = Last block in carrier (LAST_BLK_HDR_FLG) + A = Block has an allocation tag footer, only valid for allocated blocks + (ATAG_BLK_HDR_FLG) */ /* Single block carrier (SBC): -[Carrier_t|pad|Block_t 111| udata... ] +[Carrier_t|pad|Block_t 1110| udata... ] +[Carrier_t|pad|Block_t 111A| udata | atag] */ /* Allocation tags ... @@ -154,20 +159,20 @@ MBC after deallocating first block: typedef UWord alcu_atag_t; -#define MAKE_ATAG(IdAtom, Type) \ - (ASSERT((Type) >= ERTS_ALC_N_MIN && (Type) <= ERTS_ALC_N_MAX), \ +#define MAKE_ATAG(IdAtom, TypeNum) \ + (ASSERT((TypeNum) >= ERTS_ALC_N_MIN && (TypeNum) <= ERTS_ALC_N_MAX), \ ASSERT(atom_val(IdAtom) <= MAX_ATAG_ATOM_ID), \ - (atom_val(IdAtom) << ERTS_ALC_N_BITS) | (Type)) + (atom_val(IdAtom) << ERTS_ALC_N_BITS) | (TypeNum)) #define ATAG_ID(AT) (make_atom((AT) >> ERTS_ALC_N_BITS)) #define ATAG_TYPE(AT) ((AT) & ERTS_ALC_N_MASK) #define MAX_ATAG_ATOM_ID (ERTS_UWORD_MAX >> ERTS_ALC_N_BITS) -#define DBG_IS_VALID_ATAG(Allocator, AT) \ +#define DBG_IS_VALID_ATAG(AT) \ (ATAG_TYPE(AT) >= ERTS_ALC_N_MIN && \ ATAG_TYPE(AT) <= ERTS_ALC_N_MAX && \ - (Allocator)->alloc_no == ERTS_ALC_T2A(ERTS_ALC_N2T(ATAG_TYPE(AT)))) + ATAG_ID(AT) <= MAX_ATAG_ATOM_ID) /* Blocks ... */ @@ -182,10 +187,15 @@ typedef UWord alcu_atag_t; #endif #define FBLK_FTR_SZ (sizeof(FreeBlkFtr_t)) +#define BLK_HAS_ATAG(B) \ + (!!((B)->bhdr & ATAG_BLK_HDR_FLG)) + #define GET_BLK_ATAG(B) \ - (((alcu_atag_t *) (((char *) (B)) + (BLK_SZ(B))))[-1]) + (ASSERT(BLK_HAS_ATAG(B)), \ + ((alcu_atag_t *) (((char *) (B)) + (BLK_SZ(B))))[-1]) #define SET_BLK_ATAG(B, T) \ - (((alcu_atag_t *) (((char *) (B)) + (BLK_SZ(B))))[-1] = (T)) + ((B)->bhdr |= ATAG_BLK_HDR_FLG, \ + ((alcu_atag_t *) (((char *) (B)) + (BLK_SZ(B))))[-1] = (T)) #define BLK_ATAG_SZ(AP) ((AP)->atags ? sizeof(alcu_atag_t) : 0) @@ -203,13 +213,13 @@ typedef UWord alcu_atag_t; (((FreeBlkFtr_t *) (((char *) (B)) + (SZ)))[-1] = (SZ)) #define SET_MBC_ABLK_SZ(B, SZ) \ - (ASSERT(((SZ) & FLG_MASK) == 0), \ + (ASSERT(((SZ) & BLK_FLG_MASK) == 0), \ (B)->bhdr = (((B)->bhdr) & ~MBC_ABLK_SZ_MASK) | (SZ)) #define SET_MBC_FBLK_SZ(B, SZ) \ - (ASSERT(((SZ) & FLG_MASK) == 0), \ + (ASSERT(((SZ) & BLK_FLG_MASK) == 0), \ (B)->bhdr = (((B)->bhdr) & ~MBC_FBLK_SZ_MASK) | (SZ)) #define SET_SBC_BLK_SZ(B, SZ) \ - (ASSERT(((SZ) & FLG_MASK) == 0), \ + (ASSERT(((SZ) & BLK_FLG_MASK) == 0), \ (B)->bhdr = (((B)->bhdr) & ~SBC_BLK_SZ_MASK) | (SZ)) #define SET_PREV_BLK_FREE(AP,B) \ (ASSERT(!IS_MBC_FIRST_BLK(AP,B)), \ @@ -235,12 +245,12 @@ typedef UWord alcu_atag_t; # define SET_MBC_ABLK_HDR(B, Sz, F, C) \ (ASSERT(((Sz) & ~MBC_ABLK_SZ_MASK) == 0), \ - ASSERT(!((UWord)(F) & (~FLG_MASK|THIS_FREE_BLK_HDR_FLG))), \ + ASSERT(!((UWord)(F) & (~BLK_FLG_MASK|THIS_FREE_BLK_HDR_FLG))), \ (B)->bhdr = ((Sz) | (F) | (BLK_CARRIER_OFFSET(B,C) << MBC_ABLK_OFFSET_SHIFT))) # define SET_MBC_FBLK_HDR(B, Sz, F, C) \ (ASSERT(((Sz) & ~MBC_FBLK_SZ_MASK) == 0), \ - ASSERT(((UWord)(F) & (~FLG_MASK|THIS_FREE_BLK_HDR_FLG|PREV_FREE_BLK_HDR_FLG)) == THIS_FREE_BLK_HDR_FLG), \ + ASSERT(((UWord)(F) & (~BLK_FLG_MASK|THIS_FREE_BLK_HDR_FLG|PREV_FREE_BLK_HDR_FLG)) == THIS_FREE_BLK_HDR_FLG), \ (B)->bhdr = ((Sz) | (F)), \ (B)->u.carrier = (C)) @@ -257,8 +267,8 @@ typedef UWord alcu_atag_t; # define SET_BLK_FREE(B) \ (ASSERT(!IS_PREV_BLK_FREE(B)), \ (B)->u.carrier = ABLK_TO_MBC(B), \ - (B)->bhdr |= THIS_FREE_BLK_HDR_FLG, \ - (B)->bhdr &= (MBC_ABLK_SZ_MASK|FLG_MASK)) + (B)->bhdr &= (MBC_ABLK_SZ_MASK|LAST_BLK_HDR_FLG), \ + (B)->bhdr |= THIS_FREE_BLK_HDR_FLG) # define SET_BLK_ALLOCED(B) \ (ASSERT(((B)->bhdr & (MBC_ABLK_OFFSET_MASK|THIS_FREE_BLK_HDR_FLG)) == THIS_FREE_BLK_HDR_FLG), \ @@ -270,15 +280,16 @@ typedef UWord alcu_atag_t; # define MBC_SZ_MAX_LIMIT ((UWord)~0) # define SET_MBC_ABLK_HDR(B, Sz, F, C) \ - (ASSERT(((Sz) & FLG_MASK) == 0), \ - ASSERT(!((UWord)(F) & (~FLG_MASK|THIS_FREE_BLK_HDR_FLG))), \ - ASSERT((UWord)(F) < SBC_BLK_HDR_FLG), \ + (ASSERT(((Sz) & BLK_FLG_MASK) == 0), \ + ASSERT(((F) & ~BLK_FLG_MASK) == 0), \ + ASSERT(!((UWord)(F) & (~BLK_FLG_MASK|THIS_FREE_BLK_HDR_FLG))), \ (B)->bhdr = ((Sz) | (F)), \ (B)->carrier = (C)) # define SET_MBC_FBLK_HDR(B, Sz, F, C) \ - (ASSERT(((Sz) & FLG_MASK) == 0), \ - ASSERT(((UWord)(F) & (~FLG_MASK|THIS_FREE_BLK_HDR_FLG|PREV_FREE_BLK_HDR_FLG)) == THIS_FREE_BLK_HDR_FLG), \ + (ASSERT(((Sz) & BLK_FLG_MASK) == 0), \ + ASSERT(((F) & ~BLK_FLG_MASK) == 0), \ + ASSERT(((UWord)(F) & (~BLK_FLG_MASK|THIS_FREE_BLK_HDR_FLG|PREV_FREE_BLK_HDR_FLG)) == THIS_FREE_BLK_HDR_FLG), \ (B)->bhdr = ((Sz) | (F)), \ (B)->carrier = (C)) @@ -297,7 +308,7 @@ typedef UWord alcu_atag_t; #endif /* !MBC_ABLK_OFFSET_BITS */ #define SET_SBC_BLK_HDR(B, Sz) \ - (ASSERT(((Sz) & FLG_MASK) == 0), (B)->bhdr = ((Sz) | (SBC_BLK_HDR_FLG))) + (ASSERT(((Sz) & BLK_FLG_MASK) == 0), (B)->bhdr = ((Sz) | (SBC_BLK_HDR_FLG))) #define BLK_UMEM_SZ(B) \ @@ -320,7 +331,7 @@ typedef UWord alcu_atag_t; #define GET_PREV_FREE_BLK_HDR_FLG(B) \ ((B)->bhdr & PREV_FREE_BLK_HDR_FLG) #define GET_BLK_HDR_FLGS(B) \ - ((B)->bhdr & FLG_MASK) + ((B)->bhdr & BLK_FLG_MASK) #define NXT_BLK(B) \ (ASSERT(IS_MBC_BLK(B)), \ @@ -419,7 +430,7 @@ do { \ #define SCH_SBC SBC_CARRIER_HDR_FLAG #define SET_CARRIER_HDR(C, Sz, F, AP) \ - (ASSERT(((Sz) & FLG_MASK) == 0), (C)->chdr = ((Sz) | (F)), \ + (ASSERT(((Sz) & CRR_FLG_MASK) == 0), (C)->chdr = ((Sz) | (F)), \ erts_atomic_init_nob(&(C)->allctr, (erts_aint_t) (AP))) #define BLK_TO_SBC(B) \ @@ -444,8 +455,8 @@ do { \ (!IS_SB_CARRIER((C))) #define SET_CARRIER_SZ(C, SZ) \ - (ASSERT(((SZ) & FLG_MASK) == 0), \ - ((C)->chdr = ((C)->chdr & FLG_MASK) | (SZ))) + (ASSERT(((SZ) & CRR_FLG_MASK) == 0), \ + ((C)->chdr = ((C)->chdr & CRR_FLG_MASK) | (SZ))) #define CFLG_SBC (1 << 0) #define CFLG_MBC (1 << 1) @@ -575,10 +586,12 @@ do { \ STAT_MSEG_MBC_ALLOC((AP), csz__); \ else \ STAT_SYS_ALLOC_MBC_ALLOC((AP), csz__); \ - (AP)->mbcs.blocks.curr.no += (CRR)->cpool.blocks; \ + set_new_allctr_abandon_limit(AP); \ + (AP)->mbcs.blocks.curr.no += (CRR)->cpool.blocks[(AP)->alloc_no]; \ if ((AP)->mbcs.blocks.max.no < (AP)->mbcs.blocks.curr.no) \ (AP)->mbcs.blocks.max.no = (AP)->mbcs.blocks.curr.no; \ - (AP)->mbcs.blocks.curr.size += (CRR)->cpool.blocks_size; \ + (AP)->mbcs.blocks.curr.size += \ + (CRR)->cpool.blocks_size[(AP)->alloc_no]; \ if ((AP)->mbcs.blocks.max.size < (AP)->mbcs.blocks.curr.size) \ (AP)->mbcs.blocks.max.size = (AP)->mbcs.blocks.curr.size; \ } while (0) @@ -601,25 +614,33 @@ do { \ DEBUG_CHECK_CARRIER_NO_SZ((AP)); \ } while (0) -#define STAT_MBC_ABANDON(AP, CRR) \ -do { \ - UWord csz__ = CARRIER_SZ((CRR)); \ - if (IS_MSEG_CARRIER((CRR))) \ - STAT_MSEG_MBC_FREE((AP), csz__); \ - else \ - STAT_SYS_ALLOC_MBC_FREE((AP), csz__); \ - ERTS_ALC_CPOOL_ASSERT((AP)->mbcs.blocks.curr.no \ - >= (CRR)->cpool.blocks); \ - (AP)->mbcs.blocks.curr.no -= (CRR)->cpool.blocks; \ - ERTS_ALC_CPOOL_ASSERT((AP)->mbcs.blocks.curr.size \ - >= (CRR)->cpool.blocks_size); \ - (AP)->mbcs.blocks.curr.size -= (CRR)->cpool.blocks_size; \ +#define STAT_MBC_FREE(AP, CRR) \ +do { \ + UWord csz__ = CARRIER_SZ((CRR)); \ + if (IS_MSEG_CARRIER((CRR))) { \ + STAT_MSEG_MBC_FREE((AP), csz__); \ + } else { \ + STAT_SYS_ALLOC_MBC_FREE((AP), csz__); \ + } \ + set_new_allctr_abandon_limit(AP); \ } while (0) -#define STAT_MBC_BLK_ALLOC_CRR(CRR, BSZ) \ +#define STAT_MBC_ABANDON(AP, CRR) \ +do { \ + STAT_MBC_FREE(AP, CRR); \ + ERTS_ALC_CPOOL_ASSERT((AP)->mbcs.blocks.curr.no \ + >= (CRR)->cpool.blocks[(AP)->alloc_no]); \ + (AP)->mbcs.blocks.curr.no -= (CRR)->cpool.blocks[(AP)->alloc_no]; \ + ERTS_ALC_CPOOL_ASSERT((AP)->mbcs.blocks.curr.size \ + >= (CRR)->cpool.blocks_size[(AP)->alloc_no]); \ + (AP)->mbcs.blocks.curr.size -= (CRR)->cpool.blocks_size[(AP)->alloc_no]; \ +} while (0) + +#define STAT_MBC_BLK_ALLOC_CRR(AP, CRR, BSZ) \ do { \ - (CRR)->cpool.blocks++; \ - (CRR)->cpool.blocks_size += (BSZ); \ + (CRR)->cpool.blocks[(AP)->alloc_no]++; \ + (CRR)->cpool.blocks_size[(AP)->alloc_no] += (BSZ); \ + (CRR)->cpool.total_blocks_size += (BSZ); \ } while (0) #define STAT_MBC_BLK_ALLOC(AP, CRR, BSZ, FLGS) \ @@ -631,50 +652,67 @@ do { \ cstats__->blocks.curr.size += (BSZ); \ if (cstats__->blocks.max.size < cstats__->blocks.curr.size) \ cstats__->blocks.max.size = cstats__->blocks.curr.size; \ - STAT_MBC_BLK_ALLOC_CRR((CRR), (BSZ)); \ + STAT_MBC_BLK_ALLOC_CRR((AP), (CRR), (BSZ)); \ } while (0) static ERTS_INLINE int stat_cpool_mbc_blk_free(Allctr_t *allctr, + ErtsAlcType_t type, Carrier_t *crr, Carrier_t **busy_pcrr_pp, UWord blksz) { + Allctr_t *orig_allctr; + int alloc_no; - ERTS_ALC_CPOOL_ASSERT(crr->cpool.blocks > 0); - crr->cpool.blocks--; - ERTS_ALC_CPOOL_ASSERT(crr->cpool.blocks_size >= blksz); - crr->cpool.blocks_size -= blksz; + alloc_no = ERTS_ALC_T2A(type); - if (!busy_pcrr_pp || !*busy_pcrr_pp) - return 0; + ERTS_ALC_CPOOL_ASSERT(crr->cpool.blocks[alloc_no] > 0); + crr->cpool.blocks[alloc_no]--; + ERTS_ALC_CPOOL_ASSERT(crr->cpool.blocks_size[alloc_no] >= blksz); + crr->cpool.blocks_size[alloc_no] -= blksz; + ERTS_ALC_CPOOL_ASSERT(crr->cpool.total_blocks_size >= blksz); + crr->cpool.total_blocks_size -= blksz; - ERTS_ALC_CPOOL_ASSERT(crr == *busy_pcrr_pp); + if (allctr->alloc_no == alloc_no && (!busy_pcrr_pp || !*busy_pcrr_pp)) { + /* This is a local block, so we should not update the pool + * statistics. */ + return 0; + } + + /* This is either a foreign block that's been fetched from the pool, or any + * block that's in the pool. The carrier's owner keeps the statistics for + * both pooled and foreign blocks. */ + + orig_allctr = crr->cpool.orig_allctr; + + ERTS_ALC_CPOOL_ASSERT(alloc_no != allctr->alloc_no || + (crr == *busy_pcrr_pp && allctr == orig_allctr)); #ifdef ERTS_ALC_CPOOL_DEBUG ERTS_ALC_CPOOL_ASSERT( - erts_atomic_dec_read_nob(&allctr->cpool.stat.no_blocks) >= 0); + erts_atomic_dec_read_nob(&orig_allctr->cpool.stat.no_blocks[alloc_no]) >= 0); ERTS_ALC_CPOOL_ASSERT( - erts_atomic_add_read_nob(&allctr->cpool.stat.blocks_size, + erts_atomic_add_read_nob(&orig_allctr->cpool.stat.blocks_size[alloc_no], -((erts_aint_t) blksz)) >= 0); #else - erts_atomic_dec_nob(&allctr->cpool.stat.no_blocks); - erts_atomic_add_nob(&allctr->cpool.stat.blocks_size, + erts_atomic_dec_nob(&orig_allctr->cpool.stat.no_blocks[alloc_no]); + erts_atomic_add_nob(&orig_allctr->cpool.stat.blocks_size[alloc_no], -((erts_aint_t) blksz)); #endif return 1; } -#define STAT_MBC_BLK_FREE(AP, CRR, BPCRRPP, BSZ, FLGS) \ -do { \ - if (!stat_cpool_mbc_blk_free((AP), (CRR), (BPCRRPP), (BSZ))) { \ - CarriersStats_t *cstats__ = &(AP)->mbcs; \ - ASSERT(cstats__->blocks.curr.no > 0); \ - cstats__->blocks.curr.no--; \ - ASSERT(cstats__->blocks.curr.size >= (BSZ)); \ - cstats__->blocks.curr.size -= (BSZ); \ - } \ +#define STAT_MBC_BLK_FREE(AP, TYPE, CRR, BPCRRPP, BSZ, FLGS) \ +do { \ + if (!stat_cpool_mbc_blk_free((AP), (TYPE), (CRR), (BPCRRPP), (BSZ))) { \ + CarriersStats_t *cstats__ = &(AP)->mbcs; \ + ASSERT(cstats__->blocks.curr.no > 0); \ + cstats__->blocks.curr.no--; \ + ASSERT(cstats__->blocks.curr.size >= (BSZ)); \ + cstats__->blocks.curr.size -= (BSZ); \ + } \ } while (0) /* Debug stuff... */ @@ -721,8 +759,8 @@ static void make_name_atoms(Allctr_t *allctr); static Block_t *create_carrier(Allctr_t *, Uint, UWord); static void destroy_carrier(Allctr_t *, Block_t *, Carrier_t **); -static void mbc_free(Allctr_t *allctr, void *p, Carrier_t **busy_pcrr_pp); -static void dealloc_block(Allctr_t *, void *, ErtsAlcFixList_t *, int); +static void mbc_free(Allctr_t *allctr, ErtsAlcType_t type, void *p, Carrier_t **busy_pcrr_pp); +static void dealloc_block(Allctr_t *, ErtsAlcType_t, Uint32, void *, ErtsAlcFixList_t *); static alcu_atag_t determine_alloc_tag(Allctr_t *allocator, ErtsAlcType_t type) { @@ -764,14 +802,14 @@ static alcu_atag_t determine_alloc_tag(Allctr_t *allocator, ErtsAlcType_t type) } } - return MAKE_ATAG(id, type); + return MAKE_ATAG(id, ERTS_ALC_T2N(type)); } static void set_alloc_tag(Allctr_t *allocator, void *p, alcu_atag_t tag) { Block_t *block; - ASSERT(DBG_IS_VALID_ATAG(allocator, tag)); + ASSERT(DBG_IS_VALID_ATAG(tag)); ASSERT(allocator->atags && p); (void)allocator; @@ -1308,28 +1346,9 @@ chk_fix_list(Allctr_t *allctr, ErtsAlcFixList_t *fix, int ix, int before) #define ERTS_DBG_CHK_FIX_LIST(A, FIX, IX, B) #endif +static ERTS_INLINE Allctr_t *get_pref_allctr(void *extra); static void *mbc_alloc(Allctr_t *allctr, Uint size); -typedef struct { - ErtsAllctrDDBlock_t ddblock__; /* must be first */ - ErtsAlcType_t fix_type; -} ErtsAllctrFixDDBlock_t; - -#define ERTS_ALC_FIX_NO_UNUSE (((ErtsAlcType_t) 1) << ERTS_ALC_N_BITS) - -static ERTS_INLINE void -dealloc_fix_block(Allctr_t *allctr, - ErtsAlcType_t type, - void *ptr, - ErtsAlcFixList_t *fix, - int dec_cc_on_redirect) -{ - /* May be redirected... */ - ASSERT((type & ERTS_ALC_FIX_NO_UNUSE) == 0); - ((ErtsAllctrFixDDBlock_t *) ptr)->fix_type = type | ERTS_ALC_FIX_NO_UNUSE; - dealloc_block(allctr, ptr, fix, dec_cc_on_redirect); -} - static ERTS_INLINE void sched_fix_shrink(Allctr_t *allctr, int on) { @@ -1371,7 +1390,7 @@ fix_cpool_check_shrink(Allctr_t *allctr, if (fix->u.cpool.min_list_size > fix->list_size) fix->u.cpool.min_list_size = fix->list_size; - dealloc_fix_block(allctr, type, p, fix, 0); + dealloc_block(allctr, type, DEALLOC_FLG_FIX_SHRINK, p, fix); } } } @@ -1382,11 +1401,9 @@ fix_cpool_alloc(Allctr_t *allctr, ErtsAlcType_t type, Uint size) void *res; ErtsAlcFixList_t *fix; - ASSERT(ERTS_ALC_N_MIN_A_FIXED_SIZE <= type - && type <= ERTS_ALC_N_MAX_A_FIXED_SIZE); - - fix = &allctr->fix[type - ERTS_ALC_N_MIN_A_FIXED_SIZE]; - ASSERT(size == fix->type_size); + fix = &allctr->fix[ERTS_ALC_FIX_TYPE_IX(type)]; + ASSERT(type == fix->type && size == fix->type_size); + ASSERT(size >= sizeof(ErtsAllctrDDBlock_t)); res = fix->list; if (res) { @@ -1415,21 +1432,39 @@ fix_cpool_alloc(Allctr_t *allctr, ErtsAlcType_t type, Uint size) static ERTS_INLINE void fix_cpool_free(Allctr_t *allctr, ErtsAlcType_t type, + Uint32 flags, void *p, - Carrier_t **busy_pcrr_pp, - int unuse) + Carrier_t **busy_pcrr_pp) { ErtsAlcFixList_t *fix; + Allctr_t *fix_allctr; + + /* If this isn't a fix allocator we need to update the fix list of our + * neighboring fix_alloc to keep the statistics consistent. */ + if (!allctr->fix) { + ErtsAllocatorThrSpec_t *tspec = &erts_allctr_thr_spec[ERTS_ALC_A_FIXED_SIZE]; + fix_allctr = get_pref_allctr(tspec); + ASSERT(!fix_allctr->thread_safe); + ASSERT(allctr != fix_allctr); + } + else { + fix_allctr = allctr; + } - ASSERT(ERTS_ALC_N_MIN_A_FIXED_SIZE <= type - && type <= ERTS_ALC_N_MAX_A_FIXED_SIZE); + ASSERT(ERTS_ALC_IS_CPOOL_ENABLED(fix_allctr)); + ASSERT(ERTS_ALC_IS_CPOOL_ENABLED(allctr)); - fix = &allctr->fix[type - ERTS_ALC_N_MIN_A_FIXED_SIZE]; + fix = &fix_allctr->fix[ERTS_ALC_FIX_TYPE_IX(type)]; + ASSERT(type == fix->type); - if (unuse) - fix->u.cpool.used--; + if (!(flags & DEALLOC_FLG_FIX_SHRINK)) { + fix->u.cpool.used--; + } - if ((!busy_pcrr_pp || !*busy_pcrr_pp) + /* We don't want foreign blocks to be long-lived, so we skip recycling if + * allctr != fix_allctr. */ + if (allctr == fix_allctr + && (!busy_pcrr_pp || !*busy_pcrr_pp) && !fix->u.cpool.shrink_list && fix->list_size < ERTS_ALCU_FIX_MAX_LIST_SZ) { *((void **) p) = fix->list; @@ -1442,7 +1477,7 @@ fix_cpool_free(Allctr_t *allctr, if (IS_SBC_BLK(blk)) destroy_carrier(allctr, blk, NULL); else - mbc_free(allctr, p, busy_pcrr_pp); + mbc_free(allctr, type, p, busy_pcrr_pp); fix->u.cpool.allocated--; fix_cpool_check_shrink(allctr, type, fix, busy_pcrr_pp); } @@ -1469,7 +1504,7 @@ fix_cpool_alloc_shrink(Allctr_t *allctr, erts_aint32_t flgs) fix->u.cpool.shrink_list = fix->u.cpool.min_list_size; fix->u.cpool.min_list_size = fix->list_size; } - type = (ErtsAlcType_t) (ix + ERTS_ALC_N_MIN_A_FIXED_SIZE); + type = ERTS_ALC_N2T((ErtsAlcType_t) (ix + ERTS_ALC_N_MIN_A_FIXED_SIZE)); for (o = 0; o < ERTS_ALC_FIX_MAX_SHRINK_OPS || flush; o++) { void *ptr; @@ -1483,7 +1518,7 @@ fix_cpool_alloc_shrink(Allctr_t *allctr, erts_aint32_t flgs) fix->list = *((void **) ptr); fix->list_size--; fix->u.cpool.shrink_list--; - dealloc_fix_block(allctr, type, ptr, fix, 0); + dealloc_block(allctr, type, DEALLOC_FLG_FIX_SHRINK, ptr, fix); } if (fix->u.cpool.min_list_size > fix->list_size) fix->u.cpool.min_list_size = fix->list_size; @@ -1509,11 +1544,9 @@ fix_nocpool_alloc(Allctr_t *allctr, ErtsAlcType_t type, Uint size) ErtsAlcFixList_t *fix; void *res; - ASSERT(ERTS_ALC_N_MIN_A_FIXED_SIZE <= type - && type <= ERTS_ALC_N_MAX_A_FIXED_SIZE); - - fix = &allctr->fix[type - ERTS_ALC_N_MIN_A_FIXED_SIZE]; - ASSERT(size == fix->type_size); + fix = &allctr->fix[ERTS_ALC_FIX_TYPE_IX(type)]; + ASSERT(type == fix->type && size == fix->type_size); + ASSERT(size >= sizeof(ErtsAllctrDDBlock_t)); ERTS_DBG_CHK_FIX_LIST(allctr, fix, ix, 1); fix->u.nocpool.used++; @@ -1530,7 +1563,7 @@ fix_nocpool_alloc(Allctr_t *allctr, ErtsAlcType_t type, Uint size) if (IS_SBC_BLK(blk)) destroy_carrier(allctr, blk, NULL); else - mbc_free(allctr, p, NULL); + mbc_free(allctr, type, p, NULL); fix->u.nocpool.allocated--; } ERTS_DBG_CHK_FIX_LIST(allctr, fix, ix, 0); @@ -1565,10 +1598,8 @@ fix_nocpool_free(Allctr_t *allctr, Block_t *blk; ErtsAlcFixList_t *fix; - ASSERT(ERTS_ALC_N_MIN_A_FIXED_SIZE <= type - && type <= ERTS_ALC_N_MAX_A_FIXED_SIZE); - - fix = &allctr->fix[type - ERTS_ALC_N_MIN_A_FIXED_SIZE]; + fix = &allctr->fix[ERTS_ALC_T2N(type) - ERTS_ALC_N_MIN_A_FIXED_SIZE]; + ASSERT(fix->type == type); ERTS_DBG_CHK_FIX_LIST(allctr, fix, ix, 1); fix->u.nocpool.used--; @@ -1587,7 +1618,7 @@ fix_nocpool_free(Allctr_t *allctr, if (IS_SBC_BLK(blk)) destroy_carrier(allctr, blk, NULL); else - mbc_free(allctr, p, NULL); + mbc_free(allctr, type, p, NULL); p = fix->list; fix->list = *((void **) p); fix->list_size--; @@ -1598,7 +1629,7 @@ fix_nocpool_free(Allctr_t *allctr, if (IS_SBC_BLK(blk)) destroy_carrier(allctr, blk, NULL); else - mbc_free(allctr, p, NULL); + mbc_free(allctr, type, p, NULL); ERTS_DBG_CHK_FIX_LIST(allctr, fix, ix, 0); } @@ -1639,7 +1670,7 @@ fix_nocpool_alloc_shrink(Allctr_t *allctr, erts_aint32_t flgs) ptr = fix->list; fix->list = *((void **) ptr); fix->list_size--; - dealloc_block(allctr, ptr, NULL, 0); + dealloc_block(allctr, fix->type, 0, ptr, NULL); fix->u.nocpool.allocated--; } if (fix->list_size != 0) { @@ -1681,6 +1712,7 @@ dealloc_mbc(Allctr_t *allctr, Carrier_t *crr) } +static UWord allctr_abandon_limit(Allctr_t *allctr); static void set_new_allctr_abandon_limit(Allctr_t*); static void abandon_carrier(Allctr_t*, Carrier_t*); static void poolify_my_carrier(Allctr_t*, Carrier_t*); @@ -1802,7 +1834,7 @@ get_used_allctr(Allctr_t *pref_allctr, int pref_lock, void *p, UWord *sizep, static void init_dd_queue(ErtsAllctrDDQueue_t *ddq) { - erts_atomic_init_nob(&ddq->tail.data.marker.atmc_next, ERTS_AINT_NULL); + erts_atomic_init_nob(&ddq->tail.data.marker.u.atmc_next, ERTS_AINT_NULL); erts_atomic_init_nob(&ddq->tail.data.last, (erts_aint_t) &ddq->tail.data.marker); erts_atomic_init_nob(&ddq->tail.data.um_refc[0], 0); @@ -1823,17 +1855,17 @@ ddq_managed_thread_enqueue(ErtsAllctrDDQueue_t *ddq, void *ptr, int cinit) erts_aint_t itmp; ErtsAllctrDDBlock_t *enq, *this = ptr; - erts_atomic_init_nob(&this->atmc_next, ERTS_AINT_NULL); + erts_atomic_init_nob(&this->u.atmc_next, ERTS_AINT_NULL); /* Enqueue at end of list... */ enq = (ErtsAllctrDDBlock_t *) erts_atomic_read_nob(&ddq->tail.data.last); - itmp = erts_atomic_cmpxchg_relb(&enq->atmc_next, + itmp = erts_atomic_cmpxchg_relb(&enq->u.atmc_next, (erts_aint_t) this, ERTS_AINT_NULL); if (itmp == ERTS_AINT_NULL) { /* We are required to move last pointer */ #ifdef DEBUG - ASSERT(ERTS_AINT_NULL == erts_atomic_read_nob(&this->atmc_next)); + ASSERT(ERTS_AINT_NULL == erts_atomic_read_nob(&this->u.atmc_next)); ASSERT(((erts_aint_t) enq) == erts_atomic_xchg_relb(&ddq->tail.data.last, (erts_aint_t) this)); @@ -1851,8 +1883,8 @@ ddq_managed_thread_enqueue(ErtsAllctrDDQueue_t *ddq, void *ptr, int cinit) while (1) { erts_aint_t itmp2; - erts_atomic_set_nob(&this->atmc_next, itmp); - itmp2 = erts_atomic_cmpxchg_relb(&enq->atmc_next, + erts_atomic_set_nob(&this->u.atmc_next, itmp); + itmp2 = erts_atomic_cmpxchg_relb(&enq->u.atmc_next, (erts_aint_t) this, itmp); if (itmp == itmp2) @@ -1861,7 +1893,7 @@ ddq_managed_thread_enqueue(ErtsAllctrDDQueue_t *ddq, void *ptr, int cinit) itmp = itmp2; else { enq = (ErtsAllctrDDBlock_t *) itmp2; - itmp = erts_atomic_read_acqb(&enq->atmc_next); + itmp = erts_atomic_read_acqb(&enq->u.atmc_next); ASSERT(itmp != ERTS_AINT_NULL); } i++; @@ -1877,8 +1909,8 @@ check_insert_marker(ErtsAllctrDDQueue_t *ddq, erts_aint_t ilast) erts_aint_t itmp; ErtsAllctrDDBlock_t *last = (ErtsAllctrDDBlock_t *) ilast; - erts_atomic_init_nob(&ddq->tail.data.marker.atmc_next, ERTS_AINT_NULL); - itmp = erts_atomic_cmpxchg_relb(&last->atmc_next, + erts_atomic_init_nob(&ddq->tail.data.marker.u.atmc_next, ERTS_AINT_NULL); + itmp = erts_atomic_cmpxchg_relb(&last->u.atmc_next, (erts_aint_t) &ddq->tail.data.marker, ERTS_AINT_NULL); if (itmp == ERTS_AINT_NULL) { @@ -1929,7 +1961,7 @@ ddq_dequeue(ErtsAllctrDDQueue_t *ddq) ASSERT(ddq->head.used_marker); ddq->head.used_marker = 0; blk = ((ErtsAllctrDDBlock_t *) - erts_atomic_read_nob(&blk->atmc_next)); + erts_atomic_read_nob(&blk->u.atmc_next)); if (blk == ddq->head.unref_end) { ddq->head.first = blk; return NULL; @@ -1937,7 +1969,7 @@ ddq_dequeue(ErtsAllctrDDQueue_t *ddq) } ddq->head.first = ((ErtsAllctrDDBlock_t *) - erts_atomic_read_nob(&blk->atmc_next)); + erts_atomic_read_nob(&blk->u.atmc_next)); ASSERT(ddq->head.first); @@ -1999,19 +2031,13 @@ check_pending_dealloc_carrier(Allctr_t *allctr, int *need_more_work); static void -handle_delayed_fix_dealloc(Allctr_t *allctr, void *ptr) +handle_delayed_fix_dealloc(Allctr_t *allctr, ErtsAlcType_t type, Uint32 flags, + void *ptr) { - ErtsAlcType_t type; - - type = ((ErtsAllctrFixDDBlock_t *) ptr)->fix_type; - - ASSERT(ERTS_ALC_N_MIN_A_FIXED_SIZE - <= (type & ~ERTS_ALC_FIX_NO_UNUSE)); - ASSERT((type & ~ERTS_ALC_FIX_NO_UNUSE) - <= ERTS_ALC_N_MAX_A_FIXED_SIZE); + ASSERT(ERTS_ALC_IS_FIX_TYPE(type)); if (!ERTS_ALC_IS_CPOOL_ENABLED(allctr)) - fix_nocpool_free(allctr, (type & ~ERTS_ALC_FIX_NO_UNUSE), ptr); + fix_nocpool_free(allctr, type, ptr); else { Block_t *blk = UMEM2BLK(ptr); Carrier_t *busy_pcrr_p; @@ -2026,20 +2052,24 @@ handle_delayed_fix_dealloc(Allctr_t *allctr, void *ptr) NULL, &busy_pcrr_p); if (used_allctr == allctr) { doit: - fix_cpool_free(allctr, (type & ~ERTS_ALC_FIX_NO_UNUSE), - ptr, &busy_pcrr_p, - !(type & ERTS_ALC_FIX_NO_UNUSE)); + fix_cpool_free(allctr, type, flags, ptr, &busy_pcrr_p); clear_busy_pool_carrier(allctr, busy_pcrr_p); } else { /* Carrier migrated; need to redirect block to new owner... */ - int cinit = used_allctr->dd.ix - allctr->dd.ix; + ErtsAllctrDDBlock_t *dd_block; + int cinit; + + dd_block = (ErtsAllctrDDBlock_t*)ptr; + dd_block->flags = flags; + dd_block->type = type; ERTS_ALC_CPOOL_ASSERT(!busy_pcrr_p); DEC_CC(allctr->calls.this_free); - ((ErtsAllctrFixDDBlock_t *) ptr)->fix_type = type; + cinit = used_allctr->dd.ix - allctr->dd.ix; + if (ddq_enqueue(&used_allctr->dd.q, ptr, cinit)) erts_alloc_notify_delayed_dealloc(used_allctr->ix); } @@ -2063,7 +2093,6 @@ handle_delayed_dealloc(Allctr_t *allctr, int need_mr_wrk = 0; int have_checked_incoming = 0; int ops = 0; - ErtsAlcFixList_t *fix; int res; ErtsAllctrDDQueue_t *ddq; @@ -2072,8 +2101,6 @@ handle_delayed_dealloc(Allctr_t *allctr, ERTS_ALCU_DBG_CHK_THR_ACCESS(allctr); - fix = allctr->fix; - ddq = &allctr->dd.q; res = 0; @@ -2162,16 +2189,27 @@ handle_delayed_dealloc(Allctr_t *allctr, } } else { + ErtsAllctrDDBlock_t *dd_block; + ErtsAlcType_t type; + Uint32 flags; + + dd_block = (ErtsAllctrDDBlock_t*)ptr; + flags = dd_block->flags; + type = dd_block->type; + + flags |= DEALLOC_FLG_REDIRECTED; + ASSERT(IS_SBC_BLK(blk) || (ABLK_TO_MBC(blk) != ErtsContainerStruct(blk, Carrier_t, cpool.homecoming_dd.blk))); INC_CC(allctr->calls.this_free); - if (fix) - handle_delayed_fix_dealloc(allctr, ptr); - else - dealloc_block(allctr, ptr, NULL, 1); + if (ERTS_ALC_IS_FIX_TYPE(type)) { + handle_delayed_fix_dealloc(allctr, type, flags, ptr); + } else { + dealloc_block(allctr, type, flags, ptr, NULL); + } } } @@ -2199,8 +2237,10 @@ enqueue_dealloc_other_instance(ErtsAlcType_t type, void *ptr, int cinit) { - if (allctr->fix) - ((ErtsAllctrFixDDBlock_t*) ptr)->fix_type = type; + ErtsAllctrDDBlock_t *dd_block = ((ErtsAllctrDDBlock_t*)ptr); + + dd_block->type = type; + dd_block->flags = 0; if (ddq_enqueue(&allctr->dd.q, ptr, cinit)) erts_alloc_notify_delayed_dealloc(allctr->ix); @@ -2230,10 +2270,7 @@ check_abandon_carrier(Allctr_t *allctr, Block_t *fblk, Carrier_t **busy_pcrr_pp) if (!ERTS_ALC_IS_CPOOL_ENABLED(allctr)) return; - allctr->cpool.check_limit_count--; - if (--allctr->cpool.check_limit_count <= 0) - set_new_allctr_abandon_limit(allctr); - + ASSERT(allctr->cpool.abandon_limit == allctr_abandon_limit(allctr)); ASSERT(erts_thr_progress_is_managed_thread()); if (allctr->cpool.disable_abandon) @@ -2251,7 +2288,7 @@ check_abandon_carrier(Allctr_t *allctr, Block_t *fblk, Carrier_t **busy_pcrr_pp) if (allctr->main_carrier == crr) return; - if (crr->cpool.blocks_size > crr->cpool.abandon_limit) + if (crr->cpool.total_blocks_size > crr->cpool.abandon_limit) return; if (crr->cpool.thr_prgr != ERTS_THR_PRGR_INVALID @@ -2287,24 +2324,26 @@ erts_alcu_check_delayed_dealloc(Allctr_t *allctr, ERTS_ALCU_DD_OPS_LIM_LOW, NULL, NULL, NULL) static void -dealloc_block(Allctr_t *allctr, void *ptr, ErtsAlcFixList_t *fix, int dec_cc_on_redirect) +dealloc_block(Allctr_t *allctr, ErtsAlcType_t type, Uint32 flags, void *ptr, + ErtsAlcFixList_t *fix) { Block_t *blk = UMEM2BLK(ptr); + ASSERT(!fix || type == fix->type); + ERTS_LC_ASSERT(!allctr->thread_safe || erts_lc_mtx_is_locked(&allctr->mutex)); if (IS_SBC_BLK(blk)) { destroy_carrier(allctr, blk, NULL); if (fix && ERTS_ALC_IS_CPOOL_ENABLED(allctr)) { - ErtsAlcType_t type = ((ErtsAllctrFixDDBlock_t *) ptr)->fix_type; - if (!(type & ERTS_ALC_FIX_NO_UNUSE)) + if (!(flags & DEALLOC_FLG_FIX_SHRINK)) fix->u.cpool.used--; fix->u.cpool.allocated--; } } else if (!ERTS_ALC_IS_CPOOL_ENABLED(allctr)) - mbc_free(allctr, ptr, NULL); + mbc_free(allctr, type, ptr, NULL); else { Carrier_t *busy_pcrr_p; Allctr_t *used_allctr; @@ -2313,22 +2352,29 @@ dealloc_block(Allctr_t *allctr, void *ptr, ErtsAlcFixList_t *fix, int dec_cc_on_ NULL, &busy_pcrr_p); if (used_allctr == allctr) { if (fix) { - ErtsAlcType_t type = ((ErtsAllctrFixDDBlock_t *) ptr)->fix_type; - if (!(type & ERTS_ALC_FIX_NO_UNUSE)) + if (!(flags & DEALLOC_FLG_FIX_SHRINK)) fix->u.cpool.used--; fix->u.cpool.allocated--; } - mbc_free(allctr, ptr, &busy_pcrr_p); + mbc_free(allctr, type, ptr, &busy_pcrr_p); clear_busy_pool_carrier(allctr, busy_pcrr_p); } else { /* Carrier migrated; need to redirect block to new owner... */ - int cinit = used_allctr->dd.ix - allctr->dd.ix; + ErtsAllctrDDBlock_t *dd_block; + int cinit; + + dd_block = (ErtsAllctrDDBlock_t*)ptr; + dd_block->flags = flags; + dd_block->type = type; ERTS_ALC_CPOOL_ASSERT(!busy_pcrr_p); - if (dec_cc_on_redirect) + if (flags & DEALLOC_FLG_REDIRECTED) DEC_CC(allctr->calls.this_free); + + cinit = used_allctr->dd.ix - allctr->dd.ix; + if (ddq_enqueue(&used_allctr->dd.q, ptr, cinit)) erts_alloc_notify_delayed_dealloc(used_allctr->ix); } @@ -2495,7 +2541,7 @@ mbc_alloc(Allctr_t *allctr, Uint size) } static void -mbc_free(Allctr_t *allctr, void *p, Carrier_t **busy_pcrr_pp) +mbc_free(Allctr_t *allctr, ErtsAlcType_t type, void *p, Carrier_t **busy_pcrr_pp) { Uint is_first_blk; Uint is_last_blk; @@ -2517,7 +2563,8 @@ mbc_free(Allctr_t *allctr, void *p, Carrier_t **busy_pcrr_pp) crr = ABLK_TO_MBC(blk); ERTS_ALC_CPOOL_FREE_OP(allctr); - STAT_MBC_BLK_FREE(allctr, crr, busy_pcrr_pp, blk_sz, alcu_flgs); + + STAT_MBC_BLK_FREE(allctr, type, crr, busy_pcrr_pp, blk_sz, alcu_flgs); is_first_blk = IS_MBC_FIRST_ABLK(allctr, blk); is_last_blk = IS_LAST_BLK(blk); @@ -2586,8 +2633,8 @@ mbc_free(Allctr_t *allctr, void *p, Carrier_t **busy_pcrr_pp) } static void * -mbc_realloc(Allctr_t *allctr, void *p, Uint size, Uint32 alcu_flgs, - Carrier_t **busy_pcrr_pp) +mbc_realloc(Allctr_t *allctr, ErtsAlcType_t type, void *p, Uint size, + Uint32 alcu_flgs, Carrier_t **busy_pcrr_pp) { void *new_p; Uint old_blk_sz; @@ -2625,7 +2672,7 @@ mbc_realloc(Allctr_t *allctr, void *p, Uint size, Uint32 alcu_flgs, new_blk = UMEM2BLK(new_p); ASSERT(!(IS_MBC_BLK(new_blk) && ABLK_TO_MBC(new_blk) == *busy_pcrr_pp)); sys_memcpy(new_p, p, MIN(size, old_blk_sz - ABLK_HDR_SZ)); - mbc_free(allctr, p, busy_pcrr_pp); + mbc_free(allctr, type, p, busy_pcrr_pp); return new_p; } @@ -2702,7 +2749,7 @@ mbc_realloc(Allctr_t *allctr, void *p, Uint size, Uint32 alcu_flgs, crr = ABLK_TO_MBC(blk); ERTS_ALC_CPOOL_REALLOC_OP(allctr); - STAT_MBC_BLK_FREE(allctr, crr, NULL, old_blk_sz, alcu_flgs); + STAT_MBC_BLK_FREE(allctr, type, crr, NULL, old_blk_sz, alcu_flgs); STAT_MBC_BLK_ALLOC(allctr, crr, blk_sz, alcu_flgs); ASSERT(MBC_BLK_SZ(blk) >= allctr->min_block_size); @@ -2806,7 +2853,7 @@ mbc_realloc(Allctr_t *allctr, void *p, Uint size, Uint32 alcu_flgs, } ERTS_ALC_CPOOL_REALLOC_OP(allctr); - STAT_MBC_BLK_FREE(allctr, crr, NULL, old_blk_sz, alcu_flgs); + STAT_MBC_BLK_FREE(allctr, type, crr, NULL, old_blk_sz, alcu_flgs); STAT_MBC_BLK_ALLOC(allctr, crr, blk_sz, alcu_flgs); ASSERT(IS_ALLOCED_BLK(blk)); @@ -2867,7 +2914,7 @@ mbc_realloc(Allctr_t *allctr, void *p, Uint size, Uint32 alcu_flgs, if (!new_p) return NULL; sys_memcpy(new_p, p, MIN(size, old_blk_sz - ABLK_HDR_SZ)); - mbc_free(allctr, p, busy_pcrr_pp); + mbc_free(allctr, type, p, busy_pcrr_pp); return new_p; @@ -2897,7 +2944,7 @@ mbc_realloc(Allctr_t *allctr, void *p, Uint size, Uint32 alcu_flgs, 1); new_p = BLK2UMEM(new_blk); sys_memcpy(new_p, p, MIN(size, old_blk_sz - ABLK_HDR_SZ)); - mbc_free(allctr, p, NULL); + mbc_free(allctr, type, p, NULL); return new_p; } else { @@ -2954,7 +3001,7 @@ mbc_realloc(Allctr_t *allctr, void *p, Uint size, Uint32 alcu_flgs, 0); ERTS_ALC_CPOOL_FREE_OP(allctr); - STAT_MBC_BLK_FREE(allctr, crr, NULL, old_blk_sz, alcu_flgs); + STAT_MBC_BLK_FREE(allctr, type, crr, NULL, old_blk_sz, alcu_flgs); return new_p; } @@ -2965,7 +3012,6 @@ mbc_realloc(Allctr_t *allctr, void *p, Uint size, Uint32 alcu_flgs, #define ERTS_ALC_MAX_DEALLOC_CARRIER 10 #define ERTS_ALC_CPOOL_MAX_FETCH_INSPECT 100 -#define ERTS_ALC_CPOOL_CHECK_LIMIT_COUNT 100 #define ERTS_ALC_CPOOL_MAX_FAILED_STAT_READS 3 #define ERTS_ALC_CPOOL_PTR_MOD_MRK (((erts_aint_t) 1) << 0) @@ -2992,14 +3038,11 @@ typedef union { # error "Carrier pool implementation assumes ERTS_ALC_A_MIN > ERTS_ALC_A_INVALID" #endif -/* - * The pool is only allowed to be manipulated by managed - * threads except in the alloc_SUITE:cpool case. In this - * test case carrier_pool[ERTS_ALC_A_INVALID] will be - * used. - */ +/* The pools are only allowed to be manipulated by managed threads except in + * the alloc_SUITE:cpool test, where only test_carrier_pool is used. */ -static ErtsAlcCrrPool_t carrier_pool[ERTS_ALC_A_MAX+1] erts_align_attribute(ERTS_CACHE_LINE_SIZE); +static ErtsAlcCrrPool_t firstfit_carrier_pool; +static ErtsAlcCrrPool_t test_carrier_pool; #define ERTS_ALC_CPOOL_MAX_BACKOFF (1 << 8) @@ -3020,12 +3063,12 @@ backoff(int n) static int cpool_dbg_is_in_pool(Allctr_t *allctr, Carrier_t *crr) { - ErtsAlcCPoolData_t *sentinel = &carrier_pool[allctr->alloc_no].sentinel; + ErtsAlcCPoolData_t *sentinel = allctr->cpool.sentinel; ErtsAlcCPoolData_t *cpdp = sentinel; Carrier_t *tmp_crr; while (1) { - cpdp = (ErtsAlcCPoolData_t *) (erts_atomic_read_ddrb(&cpdp->next) & ~FLG_MASK); + cpdp = (ErtsAlcCPoolData_t *) (erts_atomic_read_ddrb(&cpdp->next) & ~CRR_FLG_MASK); if (cpdp == sentinel) return 0; tmp_crr = (Carrier_t *) (((char *) cpdp) - offsetof(Carrier_t, cpool)); @@ -3037,7 +3080,7 @@ cpool_dbg_is_in_pool(Allctr_t *allctr, Carrier_t *crr) static int cpool_is_empty(Allctr_t *allctr) { - ErtsAlcCPoolData_t *sentinel = &carrier_pool[allctr->alloc_no].sentinel; + ErtsAlcCPoolData_t *sentinel = allctr->cpool.sentinel; return ((erts_atomic_read_rb(&sentinel->next) == (erts_aint_t) sentinel) && (erts_atomic_read_rb(&sentinel->prev) == (erts_aint_t) sentinel)); } @@ -3127,16 +3170,31 @@ cpool_insert(Allctr_t *allctr, Carrier_t *crr) { ErtsAlcCPoolData_t *cpd1p, *cpd2p; erts_aint_t val; - ErtsAlcCPoolData_t *sentinel = &carrier_pool[allctr->alloc_no].sentinel; + ErtsAlcCPoolData_t *sentinel = allctr->cpool.sentinel; Allctr_t *orig_allctr = crr->cpool.orig_allctr; - ERTS_ALC_CPOOL_ASSERT(allctr->alloc_no == ERTS_ALC_A_INVALID /* testcase */ + ERTS_ALC_CPOOL_ASSERT(allctr->alloc_no == ERTS_ALC_A_TEST /* testcase */ || erts_thr_progress_is_managed_thread()); - erts_atomic_add_nob(&orig_allctr->cpool.stat.blocks_size, - (erts_aint_t) crr->cpool.blocks_size); - erts_atomic_add_nob(&orig_allctr->cpool.stat.no_blocks, - (erts_aint_t) crr->cpool.blocks); + { + int alloc_no = allctr->alloc_no; + + ERTS_ALC_CPOOL_ASSERT( + erts_atomic_read_nob(&orig_allctr->cpool.stat.blocks_size[alloc_no]) >= 0 && + crr->cpool.blocks_size[alloc_no] >= 0); + + ERTS_ALC_CPOOL_ASSERT( + erts_atomic_read_nob(&orig_allctr->cpool.stat.no_blocks[alloc_no]) >= 0 && + crr->cpool.blocks[alloc_no] >= 0); + + /* We only modify the counter for our current type since the others are + * conceptually still in the pool. */ + erts_atomic_add_nob(&orig_allctr->cpool.stat.blocks_size[alloc_no], + ((erts_aint_t) crr->cpool.blocks_size[alloc_no])); + erts_atomic_add_nob(&orig_allctr->cpool.stat.no_blocks[alloc_no], + ((erts_aint_t) crr->cpool.blocks[alloc_no])); + } + erts_atomic_add_nob(&orig_allctr->cpool.stat.carriers_size, (erts_aint_t) CARRIER_SZ(crr)); erts_atomic_inc_nob(&orig_allctr->cpool.stat.no_carriers); @@ -3209,10 +3267,10 @@ cpool_delete(Allctr_t *allctr, Allctr_t *prev_allctr, Carrier_t *crr) ErtsAlcCPoolData_t *cpd1p, *cpd2p; erts_aint_t val; #ifdef ERTS_ALC_CPOOL_DEBUG - ErtsAlcCPoolData_t *sentinel = &carrier_pool[allctr->alloc_no].sentinel; + ErtsAlcCPoolData_t *sentinel = allctr->cpool.sentinel; #endif - ERTS_ALC_CPOOL_ASSERT(allctr->alloc_no == ERTS_ALC_A_INVALID /* testcase */ + ERTS_ALC_CPOOL_ASSERT(allctr->alloc_no == ERTS_ALC_A_TEST /* testcase */ || erts_thr_progress_is_managed_thread()); ERTS_ALC_CPOOL_ASSERT(sentinel != &crr->cpool); @@ -3288,28 +3346,43 @@ cpool_delete(Allctr_t *allctr, Allctr_t *prev_allctr, Carrier_t *crr) crr->cpool.thr_prgr = erts_thr_progress_later(NULL); - erts_atomic_add_nob(&prev_allctr->cpool.stat.blocks_size, - -((erts_aint_t) crr->cpool.blocks_size)); - erts_atomic_add_nob(&prev_allctr->cpool.stat.no_blocks, - -((erts_aint_t) crr->cpool.blocks)); - erts_atomic_add_nob(&prev_allctr->cpool.stat.carriers_size, + { + Allctr_t *orig_allctr = crr->cpool.orig_allctr; + int alloc_no = allctr->alloc_no; + + ERTS_ALC_CPOOL_ASSERT(orig_allctr == prev_allctr); + + ERTS_ALC_CPOOL_ASSERT(crr->cpool.blocks_size[alloc_no] <= + erts_atomic_read_nob(&orig_allctr->cpool.stat.blocks_size[alloc_no])); + + ERTS_ALC_CPOOL_ASSERT(crr->cpool.blocks[alloc_no] <= + erts_atomic_read_nob(&orig_allctr->cpool.stat.no_blocks[alloc_no])); + + /* We only modify the counters for our current type since the others + * were, conceptually, never taken out of the pool. */ + erts_atomic_add_nob(&orig_allctr->cpool.stat.blocks_size[alloc_no], + -((erts_aint_t) crr->cpool.blocks_size[alloc_no])); + erts_atomic_add_nob(&orig_allctr->cpool.stat.no_blocks[alloc_no], + -((erts_aint_t) crr->cpool.blocks[alloc_no])); + + erts_atomic_add_nob(&orig_allctr->cpool.stat.carriers_size, -((erts_aint_t) CARRIER_SZ(crr))); - erts_atomic_dec_wb(&prev_allctr->cpool.stat.no_carriers); + erts_atomic_dec_wb(&orig_allctr->cpool.stat.no_carriers); + } } static Carrier_t * cpool_fetch(Allctr_t *allctr, UWord size) { - enum { IGNORANT, HAS_SEEN_SENTINEL, THE_LAST_ONE } loop_state; - int i; + int i, seen_sentinel; Carrier_t *crr; Carrier_t *reinsert_crr = NULL; ErtsAlcCPoolData_t *cpdp; ErtsAlcCPoolData_t *cpool_entrance = NULL; ErtsAlcCPoolData_t *sentinel; - ERTS_ALC_CPOOL_ASSERT(allctr->alloc_no == ERTS_ALC_A_INVALID /* testcase */ + ERTS_ALC_CPOOL_ASSERT(allctr->alloc_no == ERTS_ALC_A_TEST /* testcase */ || erts_thr_progress_is_managed_thread()); i = ERTS_ALC_CPOOL_MAX_FETCH_INSPECT; @@ -3411,48 +3484,39 @@ cpool_fetch(Allctr_t *allctr, UWord size) /* * Finally search the shared pool and try employ foreign carriers */ - sentinel = &carrier_pool[allctr->alloc_no].sentinel; + sentinel = allctr->cpool.sentinel; if (cpool_entrance) { /* * We saw a pooled carried above, use it as entrance into the pool */ - cpdp = cpool_entrance; } else { /* - * No pooled carried seen above. Start search at cpool sentinel, + * No pooled carrier seen above. Start search at cpool sentinel, * but begin by passing one element before trying to fetch. * This in order to avoid contention with threads inserting elements. */ - cpool_entrance = sentinel; - cpdp = cpool_aint2cpd(cpool_read(&cpool_entrance->prev)); - if (cpdp == sentinel) + cpool_entrance = cpool_aint2cpd(cpool_read(&sentinel->prev)); + if (cpool_entrance == sentinel) goto check_dc_list; } - loop_state = IGNORANT; + cpdp = cpool_entrance; + seen_sentinel = 0; do { erts_aint_t exp; cpdp = cpool_aint2cpd(cpool_read(&cpdp->prev)); - if (cpdp == cpool_entrance) { - if (cpool_entrance == sentinel) { - cpdp = cpool_aint2cpd(cpool_read(&cpdp->prev)); - if (cpdp == sentinel) - break; - } - loop_state = THE_LAST_ONE; - } - else if (cpdp == sentinel) { - if (loop_state == HAS_SEEN_SENTINEL) { + if (cpdp == sentinel) { + if (seen_sentinel) { /* We been here before. cpool_entrance must have been removed */ INC_CC(allctr->cpool.stat.entrance_removed); break; } - cpdp = cpool_aint2cpd(cpool_read(&cpdp->prev)); - if (cpdp == sentinel) - break; - loop_state = HAS_SEEN_SENTINEL; + seen_sentinel = 1; + continue; } + ASSERT(cpdp != cpool_entrance || seen_sentinel); + crr = ErtsContainerStruct(cpdp, Carrier_t, cpool); exp = erts_atomic_read_rb(&crr->allctr); @@ -3485,7 +3549,7 @@ cpool_fetch(Allctr_t *allctr, UWord size) INC_CC(allctr->cpool.stat.fail_shared); return NULL; } - }while (loop_state != THE_LAST_ONE); + }while (cpdp != cpool_entrance); check_dc_list: /* Last; check our own pending dealloc carrier list... */ @@ -3664,8 +3728,9 @@ cpool_init_carrier_data(Allctr_t *allctr, Carrier_t *crr) crr->cpool.orig_allctr = allctr; crr->cpool.thr_prgr = ERTS_THR_PRGR_INVALID; erts_atomic_init_nob(&crr->cpool.max_size, 0); - crr->cpool.blocks = 0; - crr->cpool.blocks_size = 0; + sys_memset(&crr->cpool.blocks_size, 0, sizeof(crr->cpool.blocks_size)); + sys_memset(&crr->cpool.blocks, 0, sizeof(crr->cpool.blocks)); + crr->cpool.total_blocks_size = 0; if (!ERTS_ALC_IS_CPOOL_ENABLED(allctr)) crr->cpool.abandon_limit = 0; else { @@ -3680,14 +3745,14 @@ cpool_init_carrier_data(Allctr_t *allctr, Carrier_t *crr) crr->cpool.state = ERTS_MBC_IS_HOME; } -static void -set_new_allctr_abandon_limit(Allctr_t *allctr) + + +static UWord +allctr_abandon_limit(Allctr_t *allctr) { UWord limit; UWord csz; - allctr->cpool.check_limit_count = ERTS_ALC_CPOOL_CHECK_LIMIT_COUNT; - csz = allctr->mbcs.curr.norm.mseg.size; csz += allctr->mbcs.curr.norm.sys_alloc.size; @@ -3697,7 +3762,13 @@ set_new_allctr_abandon_limit(Allctr_t *allctr) else limit = (csz/100)*allctr->cpool.util_limit; - allctr->cpool.abandon_limit = limit; + return limit; +} + +static void ERTS_INLINE +set_new_allctr_abandon_limit(Allctr_t *allctr) +{ + allctr->cpool.abandon_limit = allctr_abandon_limit(allctr); } static void @@ -3709,7 +3780,6 @@ abandon_carrier(Allctr_t *allctr, Carrier_t *crr) unlink_carrier(&allctr->mbc_list, crr); allctr->remove_mbc(allctr, crr); - set_new_allctr_abandon_limit(allctr); cpool_insert(allctr, crr); @@ -3762,7 +3832,8 @@ poolify_my_carrier(Allctr_t *allctr, Carrier_t *crr) } static void -cpool_read_stat(Allctr_t *allctr, UWord *nocp, UWord *cszp, UWord *nobp, UWord *bszp) +cpool_read_stat(Allctr_t *allctr, int alloc_no, + UWord *nocp, UWord *cszp, UWord *nobp, UWord *bszp) { int i; UWord noc = 0, csz = 0, nob = 0, bsz = 0; @@ -3782,10 +3853,10 @@ cpool_read_stat(Allctr_t *allctr, UWord *nocp, UWord *cszp, UWord *nobp, UWord * ? erts_atomic_read_nob(&allctr->cpool.stat.carriers_size) : 0); tnob = (UWord) (nobp - ? erts_atomic_read_nob(&allctr->cpool.stat.no_blocks) + ? erts_atomic_read_nob(&allctr->cpool.stat.no_blocks[alloc_no]) : 0); tbsz = (UWord) (bszp - ? erts_atomic_read_nob(&allctr->cpool.stat.blocks_size) + ? erts_atomic_read_nob(&allctr->cpool.stat.blocks_size[alloc_no]) : 0); if (tnoc == noc && tcsz == csz && tnob == nob && tbsz == bsz) break; @@ -4040,6 +4111,7 @@ create_carrier(Allctr_t *allctr, Uint umem_sz, UWord flags) #if HAVE_ERTS_MSEG mbc_final_touch: #endif + set_new_allctr_abandon_limit(allctr); blk = MBC_TO_FIRST_BLK(allctr, crr); @@ -4258,7 +4330,6 @@ destroy_carrier(Allctr_t *allctr, Block_t *blk, Carrier_t **busy_pcrr_pp) else { ASSERT(IS_MBC_FIRST_FBLK(allctr, blk)); crr = FIRST_BLK_TO_MBC(allctr, blk); - crr_sz = CARRIER_SZ(crr); #ifdef DEBUG if (!allctr->stopped) { @@ -4290,15 +4361,7 @@ destroy_carrier(Allctr_t *allctr, Block_t *blk, Carrier_t **busy_pcrr_pp) else { unlink_carrier(&allctr->mbc_list, crr); -#if HAVE_ERTS_MSEG - if (IS_MSEG_CARRIER(crr)) { - ASSERT(crr_sz % ERTS_SACRR_UNIT_SZ == 0); - STAT_MSEG_MBC_FREE(allctr, crr_sz); - } - else -#endif - STAT_SYS_ALLOC_MBC_FREE(allctr, crr_sz); - + STAT_MBC_FREE(allctr, crr); if (allctr->remove_mbc) allctr->remove_mbc(allctr, crr); } @@ -4312,7 +4375,7 @@ destroy_carrier(Allctr_t *allctr, Block_t *blk, Carrier_t **busy_pcrr_pp) LTTNG5(carrier_destroy, ERTS_ALC_A2AD(allctr->alloc_no), allctr->ix, - crr_sz, + CARRIER_SZ(crr), mbc_stats, sbc_stats); } @@ -4390,6 +4453,8 @@ static struct { Eterm blocks_size; Eterm blocks; + Eterm foreign_blocks; + Eterm calls; Eterm sys_alloc; Eterm sys_free; @@ -4490,6 +4555,7 @@ init_atoms(Allctr_t *allctr) AM_INIT(carriers); AM_INIT(blocks_size); AM_INIT(blocks); + AM_INIT(foreign_blocks); AM_INIT(calls); AM_INIT(sys_alloc); @@ -4625,7 +4691,6 @@ sz_info_fix(Allctr_t *allctr, ErtsAlcFixList_t *fix = &allctr->fix[ix]; UWord alloced = fix->type_size * fix->u.cpool.allocated; UWord used = fix->type_size * fix->u.cpool.used; - ErtsAlcType_t n = ERTS_ALC_N_MIN_A_FIXED_SIZE + ix; if (print_to_p) { fmtfn_t to = *print_to_p; @@ -4633,14 +4698,14 @@ sz_info_fix(Allctr_t *allctr, erts_print(to, arg, "fix type internal: %s %bpu %bpu\n", - (char *) ERTS_ALC_N2TD(n), + (char *) ERTS_ALC_T2TD(fix->type), alloced, used); } if (hpp || szp) { add_3tup(hpp, szp, &res, - alloc_type_atoms[n], + alloc_type_atoms[ERTS_ALC_T2N(fix->type)], bld_unstable_uint(hpp, szp, alloced), bld_unstable_uint(hpp, szp, used)); } @@ -4653,7 +4718,6 @@ sz_info_fix(Allctr_t *allctr, ErtsAlcFixList_t *fix = &allctr->fix[ix]; UWord alloced = fix->type_size * fix->u.nocpool.allocated; UWord used = fix->type_size*fix->u.nocpool.used; - ErtsAlcType_t n = ERTS_ALC_N_MIN_A_FIXED_SIZE + ix; if (print_to_p) { fmtfn_t to = *print_to_p; @@ -4661,14 +4725,14 @@ sz_info_fix(Allctr_t *allctr, erts_print(to, arg, "fix type: %s %bpu %bpu\n", - (char *) ERTS_ALC_N2TD(n), + (char *) ERTS_ALC_T2TD(fix->type), alloced, used); } if (hpp || szp) { add_3tup(hpp, szp, &res, - alloc_type_atoms[n], + alloc_type_atoms[ERTS_ALC_T2N(fix->type)], bld_unstable_uint(hpp, szp, alloced), bld_unstable_uint(hpp, szp, used)); } @@ -4741,9 +4805,9 @@ info_cpool(Allctr_t *allctr, noc = csz = nob = bsz = ~0; if (print_to_p || hpp) { if (sz_only) - cpool_read_stat(allctr, NULL, &csz, NULL, &bsz); + cpool_read_stat(allctr, allctr->alloc_no, NULL, &csz, NULL, &bsz); else - cpool_read_stat(allctr, &noc, &csz, &nob, &bsz); + cpool_read_stat(allctr, allctr->alloc_no, &noc, &csz, &nob, &bsz); } if (print_to_p) { @@ -4758,6 +4822,10 @@ info_cpool(Allctr_t *allctr, } if (hpp || szp) { + Eterm foreign_blocks; + int i; + + foreign_blocks = NIL; res = NIL; if (!sz_only) { @@ -4804,22 +4872,61 @@ info_cpool(Allctr_t *allctr, add_3tup(hpp, szp, &res, am.entrance_removed, bld_unstable_uint(hpp, szp, ERTS_ALC_CC_GIGA_VAL(allctr->cpool.stat.entrance_removed)), bld_unstable_uint(hpp, szp, ERTS_ALC_CC_VAL(allctr->cpool.stat.entrance_removed))); + } add_2tup(hpp, szp, &res, am.carriers_size, bld_unstable_uint(hpp, szp, csz)); - } - if (!sz_only) - add_2tup(hpp, szp, &res, - am.carriers, - bld_unstable_uint(hpp, szp, noc)); + + if (!sz_only) { + add_2tup(hpp, szp, &res, + am.carriers, + bld_unstable_uint(hpp, szp, noc)); + } + add_2tup(hpp, szp, &res, am.blocks_size, bld_unstable_uint(hpp, szp, bsz)); - if (!sz_only) + + if (!sz_only) { add_2tup(hpp, szp, &res, am.blocks, bld_unstable_uint(hpp, szp, nob)); + } + + for (i = ERTS_ALC_A_MIN; i <= ERTS_ALC_A_MAX; i++) { + const char *name_str; + Eterm name, info; + + if (i == allctr->alloc_no) { + continue; + } + + cpool_read_stat(allctr, i, NULL, NULL, &nob, &bsz); + + if (bsz == 0 && (nob == 0 || sz_only)) { + continue; + } + + name_str = ERTS_ALC_A2AD(i); + info = NIL; + + add_2tup(hpp, szp, &info, + am.blocks_size, + bld_unstable_uint(hpp, szp, bsz)); + + if (!sz_only) { + add_2tup(hpp, szp, &info, + am.blocks, + bld_unstable_uint(hpp, szp, nob)); + } + + name = am_atom_put(name_str, sys_strlen(name_str)); + + add_2tup(hpp, szp, &foreign_blocks, name, info); + } + + add_2tup(hpp, szp, &res, am.foreign_blocks, foreign_blocks); } return res; @@ -5455,6 +5562,19 @@ erts_alcu_info(Allctr_t *allctr, return res; } +void +erts_alcu_foreign_size(Allctr_t *allctr, ErtsAlcType_t alloc_no, AllctrSize_t *size) +{ + if (ERTS_ALC_IS_CPOOL_ENABLED(allctr)) { + UWord csz, bsz; + cpool_read_stat(allctr, alloc_no, NULL, &csz, NULL, &bsz); + size->carriers = csz; + size->blocks = bsz; + } else { + size->carriers = 0; + size->blocks = 0; + } +} void erts_alcu_current_size(Allctr_t *allctr, AllctrSize_t *size, ErtsAlcUFixInfo_t *fi, int fisz) @@ -5473,7 +5593,7 @@ erts_alcu_current_size(Allctr_t *allctr, AllctrSize_t *size, ErtsAlcUFixInfo_t * if (ERTS_ALC_IS_CPOOL_ENABLED(allctr)) { UWord csz, bsz; - cpool_read_stat(allctr, NULL, &csz, NULL, &bsz); + cpool_read_stat(allctr, allctr->alloc_no, NULL, &csz, NULL, &bsz); size->blocks += bsz; size->carriers += csz; } @@ -5518,6 +5638,11 @@ do_erts_alcu_alloc(ErtsAlcType_t type, Allctr_t *allctr, Uint size) ERTS_ALCU_DBG_CHK_THR_ACCESS(allctr); + /* Reject sizes that can't fit into the header word. */ + if (size > ~BLK_FLG_MASK) { + return NULL; + } + #if ALLOC_ZERO_EQ_NULL if (!size) return NULL; @@ -5684,12 +5809,11 @@ do_erts_alcu_free(ErtsAlcType_t type, Allctr_t *allctr, void *p, ERTS_ALCU_DBG_CHK_THR_ACCESS(allctr); if (p) { - INC_CC(allctr->calls.this_free); - if (allctr->fix) { + if (ERTS_ALC_IS_FIX_TYPE(type)) { if (ERTS_ALC_IS_CPOOL_ENABLED(allctr)) - fix_cpool_free(allctr, type, p, busy_pcrr_pp, 1); + fix_cpool_free(allctr, type, 0, p, busy_pcrr_pp); else fix_nocpool_free(allctr, type, p); } @@ -5698,7 +5822,7 @@ do_erts_alcu_free(ErtsAlcType_t type, Allctr_t *allctr, void *p, if (IS_SBC_BLK(blk)) destroy_carrier(allctr, blk, NULL); else - mbc_free(allctr, p, busy_pcrr_pp); + mbc_free(allctr, type, p, busy_pcrr_pp); } } } @@ -5800,6 +5924,11 @@ do_erts_alcu_realloc(ErtsAlcType_t type, return res; } + /* Reject sizes that can't fit into the header word. */ + if (size > ~BLK_FLG_MASK) { + return NULL; + } + #if ALLOC_ZERO_EQ_NULL if (!size) { ASSERT(p); @@ -5816,7 +5945,7 @@ do_erts_alcu_realloc(ErtsAlcType_t type, if (size < allctr->sbc_threshold) { if (IS_MBC_BLK(blk)) - res = mbc_realloc(allctr, p, size, alcu_flgs, busy_pcrr_pp); + res = mbc_realloc(allctr, type, p, size, alcu_flgs, busy_pcrr_pp); else { Uint used_sz = SBC_HEADER_SIZE + ABLK_HDR_SZ + size; Uint crr_sz; @@ -5875,7 +6004,7 @@ do_erts_alcu_realloc(ErtsAlcType_t type, sys_memcpy((void *) res, (void *) p, MIN(MBC_ABLK_SZ(blk) - ABLK_HDR_SZ, size)); - mbc_free(allctr, p, busy_pcrr_pp); + mbc_free(allctr, type, p, busy_pcrr_pp); } else res = NULL; @@ -6243,6 +6372,7 @@ int erts_alcu_start(Allctr_t *allctr, AllctrInit_t *init) { /* erts_alcu_start assumes that allctr has been zeroed */ + int i; if (((UWord)allctr & ERTS_CRR_ALCTR_FLG_MASK) != 0) { erts_exit(ERTS_ABORT_EXIT, "%s:%d:erts_alcu_start: Alignment error\n", @@ -6266,6 +6396,11 @@ erts_alcu_start(Allctr_t *allctr, AllctrInit_t *init) allctr->ix = init->ix; allctr->alloc_no = init->alloc_no; + allctr->alloc_strat = init->alloc_strat; + + ASSERT(allctr->alloc_no >= ERTS_ALC_A_MIN && + allctr->alloc_no <= ERTS_ALC_A_MAX); + if (allctr->alloc_no < ERTS_ALC_A_MIN || ERTS_ALC_A_MAX < allctr->alloc_no) allctr->alloc_no = ERTS_ALC_A_INVALID; @@ -6318,8 +6453,7 @@ erts_alcu_start(Allctr_t *allctr, AllctrInit_t *init) + sizeof(FreeBlkFtr_t)); if (init->tpref) { Uint sz = ABLK_HDR_SZ; - sz += (init->fix ? - sizeof(ErtsAllctrFixDDBlock_t) : sizeof(ErtsAllctrDDBlock_t)); + sz += sizeof(ErtsAllctrDDBlock_t); sz = UNIT_CEILING(sz); if (sz > allctr->min_block_size) allctr->min_block_size = sz; @@ -6330,15 +6464,23 @@ erts_alcu_start(Allctr_t *allctr, AllctrInit_t *init) allctr->cpool.dc_list.last = NULL; allctr->cpool.abandon_limit = 0; allctr->cpool.disable_abandon = 0; - erts_atomic_init_nob(&allctr->cpool.stat.blocks_size, 0); - erts_atomic_init_nob(&allctr->cpool.stat.no_blocks, 0); + for (i = ERTS_ALC_A_MIN; i <= ERTS_ALC_A_MAX; i++) { + erts_atomic_init_nob(&allctr->cpool.stat.blocks_size[i], 0); + erts_atomic_init_nob(&allctr->cpool.stat.no_blocks[i], 0); + } erts_atomic_init_nob(&allctr->cpool.stat.carriers_size, 0); erts_atomic_init_nob(&allctr->cpool.stat.no_carriers, 0); - allctr->cpool.check_limit_count = ERTS_ALC_CPOOL_CHECK_LIMIT_COUNT; if (!init->ts && init->acul && init->acnl) { allctr->cpool.util_limit = init->acul; allctr->cpool.in_pool_limit = init->acnl; allctr->cpool.fblk_min_limit = init->acfml; + + if (allctr->alloc_strat == ERTS_ALC_S_FIRSTFIT) { + allctr->cpool.sentinel = &firstfit_carrier_pool.sentinel; + } + else if (allctr->alloc_no != ERTS_ALC_A_TEST) { + ERTS_INTERNAL_ERROR("Impossible carrier migration config."); + } } else { allctr->cpool.util_limit = 0; @@ -6346,6 +6488,12 @@ erts_alcu_start(Allctr_t *allctr, AllctrInit_t *init) allctr->cpool.fblk_min_limit = 0; } + /* The invasive tests don't really care whether the pool is enabled or not, + * so we need to set this unconditionally for this allocator type. */ + if (allctr->alloc_no == ERTS_ALC_A_TEST) { + allctr->cpool.sentinel = &test_carrier_pool.sentinel; + } + allctr->sbc_threshold = adjust_sbct(allctr, init->sbct); #if HAVE_ERTS_MSEG @@ -6457,9 +6605,9 @@ erts_alcu_start(Allctr_t *allctr, AllctrInit_t *init) allctr->fix_shrink_scheduled = 0; for (i = 0; i < ERTS_ALC_NO_FIXED_SIZES; i++) { allctr->fix[i].type_size = init->fix_type_size[i]; + allctr->fix[i].type = ERTS_ALC_N2T(i + ERTS_ALC_N_MIN_A_FIXED_SIZE); allctr->fix[i].list_size = 0; allctr->fix[i].list = NULL; - ASSERT(allctr->fix[i].type_size >= sizeof(ErtsAllctrFixDDBlock_t)); if (ERTS_ALC_IS_CPOOL_ENABLED(allctr)) { allctr->fix[i].u.cpool.min_list_size = 0; allctr->fix[i].u.cpool.shrink_list = 0; @@ -6508,12 +6656,16 @@ erts_alcu_stop(Allctr_t *allctr) void erts_alcu_init(AlcUInit_t *init) { - int i; - for (i = 0; i <= ERTS_ALC_A_MAX; i++) { - ErtsAlcCPoolData_t *sentinel = &carrier_pool[i].sentinel; - erts_atomic_init_nob(&sentinel->next, (erts_aint_t) sentinel); - erts_atomic_init_nob(&sentinel->prev, (erts_aint_t) sentinel); - } + ErtsAlcCPoolData_t *sentinel; + + sentinel = &firstfit_carrier_pool.sentinel; + erts_atomic_init_nob(&sentinel->next, (erts_aint_t) sentinel); + erts_atomic_init_nob(&sentinel->prev, (erts_aint_t) sentinel); + + sentinel = &test_carrier_pool.sentinel; + erts_atomic_init_nob(&sentinel->next, (erts_aint_t) sentinel); + erts_atomic_init_nob(&sentinel->prev, (erts_aint_t) sentinel); + ERTS_CT_ASSERT(SBC_BLK_SZ_MASK == MBC_FBLK_SZ_MASK); /* see BLK_SZ */ #if HAVE_ERTS_MSEG ASSERT(erts_mseg_unit_size() == ERTS_SACRR_UNIT_SZ); @@ -6695,7 +6847,7 @@ static int blockscan_cpool_yielding(blockscan_t *state) { ErtsAlcCPoolData_t *sentinel, *cursor; - sentinel = &carrier_pool[(state->allocator)->alloc_no].sentinel; + sentinel = (state->allocator)->cpool.sentinel; cursor = blockscan_restore_cpool_cursor(state); if (ERTS_PROC_IS_EXITING(state->process)) { @@ -6827,11 +6979,8 @@ static int blockscan_sweep_mbcs(blockscan_t *state) static int blockscan_sweep_cpool(blockscan_t *state) { if (state->current_op != blockscan_sweep_cpool) { - ErtsAlcCPoolData_t *sentinel; - SET_CARRIER_HDR(&state->dummy_carrier, 0, SCH_MBC, state->allocator); - sentinel = &carrier_pool[(state->allocator)->alloc_no].sentinel; - state->cpool_cursor = sentinel; + state->cpool_cursor = (state->allocator)->cpool.sentinel; } state->current_op = blockscan_sweep_cpool; @@ -7115,11 +7264,14 @@ static int gather_ahist_scan(Allctr_t *allocator, alcu_atag_t tag; block = SBC2BLK(allocator, carrier); - tag = GET_BLK_ATAG(block); - ASSERT(DBG_IS_VALID_ATAG(allocator, tag)); + if (BLK_HAS_ATAG(block)) { + tag = GET_BLK_ATAG(block); - gather_ahist_update(state, tag, SBC_BLK_SZ(block)); + ASSERT(DBG_IS_VALID_ATAG(tag)); + + gather_ahist_update(state, tag, SBC_BLK_SZ(block)); + } } else { UWord scanned_bytes = MBC_HEADER_SIZE(allocator); @@ -7130,10 +7282,10 @@ static int gather_ahist_scan(Allctr_t *allocator, while (1) { UWord block_size = MBC_BLK_SZ(block); - if (IS_ALLOCED_BLK(block)) { + if (IS_ALLOCED_BLK(block) && BLK_HAS_ATAG(block)) { alcu_atag_t tag = GET_BLK_ATAG(block); - ASSERT(DBG_IS_VALID_ATAG(allocator, tag)); + ASSERT(DBG_IS_VALID_ATAG(tag)); gather_ahist_update(state, tag, block_size); } @@ -7293,8 +7445,6 @@ int erts_alcu_gather_alloc_histograms(Process *p, int allocator_num, sched_id, &allocator)) { return 0; - } else if (!allocator->atags) { - return 0; } ensure_atoms_initialized(allocator); diff --git a/erts/emulator/beam/erl_alloc_util.h b/erts/emulator/beam/erl_alloc_util.h index cbae8ce98a..9ab8589bf3 100644 --- a/erts/emulator/beam/erl_alloc_util.h +++ b/erts/emulator/beam/erl_alloc_util.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2002-2017. All Rights Reserved. + * Copyright Ericsson AB 2002-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -24,6 +24,7 @@ #define ERTS_ALCU_VSN_STR "3.0" #include "erl_alloc_types.h" +#include "erl_alloc.h" #define ERL_THREADS_EMU_INTERNAL__ #include "erl_threads.h" @@ -44,6 +45,7 @@ typedef struct { typedef struct { char *name_prefix; ErtsAlcType_t alloc_no; + ErtsAlcStrat_t alloc_strat; int force; int ix; int ts; @@ -101,6 +103,7 @@ typedef struct { #define ERTS_DEFAULT_ALLCTR_INIT { \ NULL, \ ERTS_ALC_A_INVALID, /* (number) alloc_no: allocator number */\ + ERTS_ALC_S_INVALID, /* (number) alloc_strat: allocator strategy */\ 0, /* (bool) force: force enabled */\ 0, /* (number) ix: instance index */\ 1, /* (bool) ts: thread safe */\ @@ -138,6 +141,7 @@ typedef struct { #define ERTS_DEFAULT_ALLCTR_INIT { \ NULL, \ ERTS_ALC_A_INVALID, /* (number) alloc_no: allocator number */\ + ERTS_ALC_S_INVALID, /* (number) alloc_strat: allocator strategy */\ 0, /* (bool) force: force enabled */\ 0, /* (number) ix: instance index */\ 1, /* (bool) ts: thread safe */\ @@ -188,6 +192,7 @@ Eterm erts_alcu_info(Allctr_t *, int, int, fmtfn_t *, void *, Uint **, Uint *); void erts_alcu_init(AlcUInit_t *); void erts_alcu_current_size(Allctr_t *, AllctrSize_t *, ErtsAlcUFixInfo_t *, int); +void erts_alcu_foreign_size(Allctr_t *, ErtsAlcType_t, AllctrSize_t *); void erts_alcu_check_delayed_dealloc(Allctr_t *, int, int *, ErtsThrPrgrVal *, int *); erts_aint32_t erts_alcu_fix_alloc_shrink(Allctr_t *, erts_aint32_t); @@ -286,10 +291,18 @@ void erts_alcu_sched_spec_data_init(struct ErtsSchedulerData_ *esdp); #define UNIT_FLOOR(X) ((X) & UNIT_MASK) #define UNIT_CEILING(X) UNIT_FLOOR((X) + INV_UNIT_MASK) -#define FLG_MASK INV_UNIT_MASK -#define SBC_BLK_SZ_MASK UNIT_MASK -#define MBC_FBLK_SZ_MASK UNIT_MASK -#define CARRIER_SZ_MASK UNIT_MASK +/* We store flags in the bits that no one will ever use. Generally these are + * the bits below the alignment size, but for blocks we also steal the highest + * bit since the header's a size and no one can expect to be able to allocate + * objects that large. */ +#define HIGHEST_WORD_BIT (((UWord) 1) << (sizeof(UWord) * CHAR_BIT - 1)) + +#define BLK_FLG_MASK (INV_UNIT_MASK | HIGHEST_WORD_BIT) +#define SBC_BLK_SZ_MASK (~BLK_FLG_MASK) +#define MBC_FBLK_SZ_MASK (~BLK_FLG_MASK) + +#define CRR_FLG_MASK INV_UNIT_MASK +#define CRR_SZ_MASK UNIT_MASK #if ERTS_HAVE_MSEG_SUPER_ALIGNED \ || (!HAVE_ERTS_MSEG && ERTS_HAVE_ERTS_SYS_ALIGNED_ALLOC) @@ -299,9 +312,9 @@ void erts_alcu_sched_spec_data_init(struct ErtsSchedulerData_ *esdp); # define ERTS_SUPER_ALIGN_BITS 18 # endif # ifdef ARCH_64 -# define MBC_ABLK_OFFSET_BITS 24 +# define MBC_ABLK_OFFSET_BITS 23 # else -# define MBC_ABLK_OFFSET_BITS 9 +# define MBC_ABLK_OFFSET_BITS 8 /* Affects hard limits for sbct and lmbcs documented in erts_alloc.xml */ # endif # define ERTS_SACRR_UNIT_SHIFT ERTS_SUPER_ALIGN_BITS @@ -322,18 +335,17 @@ void erts_alcu_sched_spec_data_init(struct ErtsSchedulerData_ *esdp); #if MBC_ABLK_OFFSET_BITS # define MBC_ABLK_OFFSET_SHIFT (sizeof(UWord)*8 - MBC_ABLK_OFFSET_BITS) -# define MBC_ABLK_OFFSET_MASK (~((UWord)0) << MBC_ABLK_OFFSET_SHIFT) -# define MBC_ABLK_SZ_MASK (~MBC_ABLK_OFFSET_MASK & ~FLG_MASK) +# define MBC_ABLK_OFFSET_MASK ((~((UWord)0) << MBC_ABLK_OFFSET_SHIFT) & ~BLK_FLG_MASK) +# define MBC_ABLK_SZ_MASK (~MBC_ABLK_OFFSET_MASK & ~BLK_FLG_MASK) #else -# define MBC_ABLK_SZ_MASK (~FLG_MASK) +# define MBC_ABLK_SZ_MASK (~BLK_FLG_MASK) #endif #define MBC_ABLK_SZ(B) (ASSERT(!is_sbc_blk(B)), (B)->bhdr & MBC_ABLK_SZ_MASK) #define MBC_FBLK_SZ(B) (ASSERT(!is_sbc_blk(B)), (B)->bhdr & MBC_FBLK_SZ_MASK) #define SBC_BLK_SZ(B) (ASSERT(is_sbc_blk(B)), (B)->bhdr & SBC_BLK_SZ_MASK) -#define CARRIER_SZ(C) \ - ((C)->chdr & CARRIER_SZ_MASK) +#define CARRIER_SZ(C) ((C)->chdr & CRR_SZ_MASK) typedef union {char c[ERTS_ALLOC_ALIGN_BYTES]; long l; double d;} Unit_t; @@ -351,12 +363,20 @@ typedef struct { #endif } Block_t; -typedef union ErtsAllctrDDBlock_t_ ErtsAllctrDDBlock_t; +typedef struct ErtsAllctrDDBlock__ { + union { + struct ErtsAllctrDDBlock__ *ptr_next; + erts_atomic_t atmc_next; + } u; + ErtsAlcType_t type; + Uint32 flags; +} ErtsAllctrDDBlock_t; -union ErtsAllctrDDBlock_t_ { - erts_atomic_t atmc_next; - ErtsAllctrDDBlock_t *ptr_next; -}; +/* Deallocation was caused by shrinking a fix-list, so usage statistics has + * already been updated. */ +#define DEALLOC_FLG_FIX_SHRINK (1 << 0) +/* Deallocation was redirected to another instance. */ +#define DEALLOC_FLG_REDIRECTED (1 << 1) typedef struct { Block_t blk; @@ -365,11 +385,10 @@ typedef struct { #endif } ErtsFakeDDBlock_t; - - #define THIS_FREE_BLK_HDR_FLG (((UWord) 1) << 0) #define PREV_FREE_BLK_HDR_FLG (((UWord) 1) << 1) #define LAST_BLK_HDR_FLG (((UWord) 1) << 2) +#define ATAG_BLK_HDR_FLG HIGHEST_WORD_BIT #define SBC_BLK_HDR_FLG /* Special flag combo for (allocated) SBC blocks */\ (THIS_FREE_BLK_HDR_FLG | PREV_FREE_BLK_HDR_FLG | LAST_BLK_HDR_FLG) @@ -381,9 +400,9 @@ typedef struct { #define HOMECOMING_MBC_BLK_HDR (THIS_FREE_BLK_HDR_FLG | LAST_BLK_HDR_FLG) #define IS_FREE_LAST_MBC_BLK(B) \ - (((B)->bhdr & FLG_MASK) == (THIS_FREE_BLK_HDR_FLG | LAST_BLK_HDR_FLG)) + (((B)->bhdr & BLK_FLG_MASK) == (THIS_FREE_BLK_HDR_FLG | LAST_BLK_HDR_FLG)) -#define IS_SBC_BLK(B) (((B)->bhdr & FLG_MASK) == SBC_BLK_HDR_FLG) +#define IS_SBC_BLK(B) (((B)->bhdr & SBC_BLK_HDR_FLG) == SBC_BLK_HDR_FLG) #define IS_MBC_BLK(B) (!IS_SBC_BLK((B))) #define IS_FREE_BLK(B) (ASSERT(IS_MBC_BLK(B)), \ (B)->bhdr & THIS_FREE_BLK_HDR_FLG) @@ -394,7 +413,8 @@ typedef struct { # define ABLK_TO_MBC(B) \ (ASSERT(IS_MBC_BLK(B) && !IS_FREE_BLK(B)), \ (Carrier_t*)((ERTS_SACRR_UNIT_FLOOR((UWord)(B)) - \ - (((B)->bhdr >> MBC_ABLK_OFFSET_SHIFT) << ERTS_SACRR_UNIT_SHIFT)))) + ((((B)->bhdr & ~BLK_FLG_MASK) >> MBC_ABLK_OFFSET_SHIFT) \ + << ERTS_SACRR_UNIT_SHIFT)))) # define BLK_TO_MBC(B) (IS_FREE_BLK(B) ? FBLK_TO_MBC(B) : ABLK_TO_MBC(B)) #else # define FBLK_TO_MBC(B) ((B)->carrier) @@ -433,8 +453,9 @@ typedef struct { ErtsThrPrgrVal thr_prgr; erts_atomic_t max_size; UWord abandon_limit; - UWord blocks; - UWord blocks_size; + UWord blocks[ERTS_ALC_A_MAX + 1]; + UWord blocks_size[ERTS_ALC_A_MAX + 1]; + UWord total_blocks_size; enum { ERTS_MBC_IS_HOME, ERTS_MBC_WAS_POOLED, @@ -452,7 +473,7 @@ struct Carrier_t_ { }; #define ERTS_ALC_CARRIER_TO_ALLCTR(C) \ - ((Allctr_t *) (erts_atomic_read_nob(&(C)->allctr) & ~FLG_MASK)) + ((Allctr_t *) (erts_atomic_read_nob(&(C)->allctr) & ~CRR_FLG_MASK)) typedef struct { Carrier_t *first; @@ -530,7 +551,6 @@ typedef struct { } head; } ErtsAllctrDDQueue_t; - typedef struct { size_t type_size; SWord list_size; @@ -549,6 +569,7 @@ typedef struct { UWord used; } cpool; } u; + ErtsAlcType_t type; } ErtsAlcFixList_t; struct Allctr_t_ { @@ -569,6 +590,9 @@ struct Allctr_t_ { /* Allocator number */ ErtsAlcType_t alloc_no; + /* Allocator strategy */ + ErtsAlcStrat_t alloc_strat; + /* Instance index */ int ix; @@ -617,6 +641,9 @@ struct Allctr_t_ { AOFF_RBTree_t* pooled_tree; CarrierList_t dc_list; + /* the sentinel of the cpool we're attached to */ + ErtsAlcCPoolData_t *sentinel; + UWord abandon_limit; int disable_abandon; int check_limit_count; @@ -624,8 +651,8 @@ struct Allctr_t_ { UWord in_pool_limit; /* acnl */ UWord fblk_min_limit; /* acmfl */ struct { - erts_atomic_t blocks_size; - erts_atomic_t no_blocks; + erts_atomic_t blocks_size[ERTS_ALC_A_MAX + 1]; + erts_atomic_t no_blocks[ERTS_ALC_A_MAX + 1]; erts_atomic_t carriers_size; erts_atomic_t no_carriers; CallCounter_t fail_pooled; diff --git a/erts/emulator/beam/erl_ao_firstfit_alloc.c b/erts/emulator/beam/erl_ao_firstfit_alloc.c index ebbe4af53d..0e3e4c890a 100644 --- a/erts/emulator/beam/erl_ao_firstfit_alloc.c +++ b/erts/emulator/beam/erl_ao_firstfit_alloc.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2003-2016. All Rights Reserved. + * Copyright Ericsson AB 2003-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -107,9 +107,11 @@ typedef struct AOFF_Carrier_t_ AOFF_Carrier_t; struct AOFF_Carrier_t_ { Carrier_t crr; - AOFF_RBTree_t rbt_node; /* My node in the carrier tree */ - AOFF_RBTree_t* root; /* Root of my block tree */ + AOFF_RBTree_t rbt_node; /* My node in the carrier tree */ + AOFF_RBTree_t* root; /* Root of my block tree */ + enum AOFFSortOrder blk_order; }; + #define RBT_NODE_TO_MBC(PTR) ErtsContainerStruct((PTR), AOFF_Carrier_t, rbt_node) /* @@ -281,15 +283,28 @@ erts_aoffalc_start(AOFFAllctr_t *alc, sys_memcpy((void *) alc, (void *) &zero.allctr, sizeof(AOFFAllctr_t)); + if (aoffinit->blk_order == FF_CHAOS) { + const enum AOFFSortOrder orders[3] = {FF_AOFF, FF_AOBF, FF_BF}; + int index = init->ix % (sizeof(orders) / sizeof(orders[0])); + + ASSERT(init->alloc_no == ERTS_ALC_A_TEST); + aoffinit->blk_order = orders[index]; + } + + if (aoffinit->crr_order == FF_CHAOS) { + const enum AOFFSortOrder orders[2] = {FF_AGEFF, FF_AOFF}; + int index = init->ix % (sizeof(orders) / sizeof(orders[0])); + + ASSERT(init->alloc_no == ERTS_ALC_A_TEST); + aoffinit->crr_order = orders[index]; + } + alc->blk_order = aoffinit->blk_order; alc->crr_order = aoffinit->crr_order; allctr->mbc_header_size = sizeof(AOFF_Carrier_t); allctr->min_mbc_size = MIN_MBC_SZ; allctr->min_mbc_first_free_size = MIN_MBC_FIRST_FREE_SZ; - allctr->min_block_size = (aoffinit->blk_order == FF_BF - ? (offsetof(AOFF_RBTree_t, u.next) - + ErtsSizeofMember(AOFF_RBTree_t, u.next)) - : offsetof(AOFF_RBTree_t, u)); + allctr->min_block_size = sizeof(AOFF_RBTree_t); allctr->vsn_str = ERTS_ALC_AOFF_ALLOC_VSN_STR; @@ -512,14 +527,15 @@ tree_insert_fixup(AOFF_RBTree_t** root, AOFF_RBTree_t *blk) static void aoff_unlink_free_block(Allctr_t *allctr, Block_t *blk) { - AOFFAllctr_t* alc = (AOFFAllctr_t*)allctr; AOFF_RBTree_t* del = (AOFF_RBTree_t*)blk; AOFF_Carrier_t *crr = (AOFF_Carrier_t*) FBLK_TO_MBC(&del->hdr); + (void)allctr; + ASSERT(crr->rbt_node.hdr.bhdr == crr->root->max_sz); - HARD_CHECK_TREE(&crr->crr, alc->blk_order, crr->root, 0); + HARD_CHECK_TREE(&crr->crr, crr->blk_order, crr->root, 0); - if (alc->blk_order == FF_BF) { + if (crr->blk_order == FF_BF) { ASSERT(del->flags & IS_BF_FLG); if (IS_LIST_ELEM(del)) { /* Remove from list */ @@ -540,14 +556,14 @@ aoff_unlink_free_block(Allctr_t *allctr, Block_t *blk) replace(&crr->root, (AOFF_RBTree_t*)del, LIST_NEXT(del)); - HARD_CHECK_TREE(&crr->crr, alc->blk_order, crr->root, 0); + HARD_CHECK_TREE(&crr->crr, crr->blk_order, crr->root, 0); return; } } rbt_delete(&crr->root, (AOFF_RBTree_t*)del); - HARD_CHECK_TREE(&crr->crr, alc->blk_order, crr->root, 0); + HARD_CHECK_TREE(&crr->crr, crr->blk_order, crr->root, 0); /* Update the carrier tree with a potentially new (lower) max_sz */ @@ -737,17 +753,18 @@ rbt_delete(AOFF_RBTree_t** root, AOFF_RBTree_t* del) static void aoff_link_free_block(Allctr_t *allctr, Block_t *block) { - AOFFAllctr_t* alc = (AOFFAllctr_t*) allctr; AOFF_RBTree_t *blk = (AOFF_RBTree_t *) block; AOFF_RBTree_t *crr_node; AOFF_Carrier_t *blk_crr = (AOFF_Carrier_t*) FBLK_TO_MBC(block); Uint blk_sz = AOFF_BLK_SZ(blk); + (void)allctr; + ASSERT(allctr == ERTS_ALC_CARRIER_TO_ALLCTR(&blk_crr->crr)); ASSERT(blk_crr->rbt_node.hdr.bhdr == (blk_crr->root ? blk_crr->root->max_sz : 0)); - HARD_CHECK_TREE(&blk_crr->crr, alc->blk_order, blk_crr->root, 0); + HARD_CHECK_TREE(&blk_crr->crr, blk_crr->blk_order, blk_crr->root, 0); - rbt_insert(alc->blk_order, &blk_crr->root, blk); + rbt_insert(blk_crr->blk_order, &blk_crr->root, blk); /* * Update carrier tree with a potentially new (larger) max_sz @@ -891,7 +908,7 @@ aoff_get_free_block(Allctr_t *allctr, Uint size, /* Get block within carrier tree */ #ifdef HARD_DEBUG - dbg_blk = HARD_CHECK_TREE(&crr->crr, alc->blk_order, crr->root, size); + dbg_blk = HARD_CHECK_TREE(&crr->crr, crr->blk_order, crr->root, size); #endif blk = rbt_search(crr->root, size); @@ -904,7 +921,7 @@ aoff_get_free_block(Allctr_t *allctr, Uint size, if (!blk) return NULL; - if (cand_blk && cmp_cand_blk(alc->blk_order, cand_blk, blk) < 0) { + if (cand_blk && cmp_cand_blk(crr->blk_order, cand_blk, blk) < 0) { return NULL; /* cand_blk was better */ } @@ -927,21 +944,28 @@ static void aoff_creating_mbc(Allctr_t *allctr, Carrier_t *carrier) AOFFAllctr_t *alc = (AOFFAllctr_t *) allctr; AOFF_Carrier_t *crr = (AOFF_Carrier_t*) carrier; AOFF_RBTree_t **root = &alc->mbc_root; + Sint64 bt = get_birth_time(); HARD_CHECK_TREE(NULL, alc->crr_order, *root, 0); crr->rbt_node.hdr.bhdr = 0; - if (alc->crr_order == FF_AGEFF || IS_DEBUG) { - Sint64 bt = get_birth_time(); - crr->rbt_node.u.birth_time = bt; - crr->crr.cpool.pooled.u.birth_time = bt; - } + + /* While birth time is only used for FF_AGEFF, we have to set it for all + * types as we can be migrated to an instance that uses it and we don't + * want to mess its order up. */ + crr->rbt_node.u.birth_time = bt; + crr->crr.cpool.pooled.u.birth_time = bt; + rbt_insert(alc->crr_order, root, &crr->rbt_node); /* aoff_link_free_block will add free block later */ crr->root = NULL; HARD_CHECK_TREE(NULL, alc->crr_order, *root, 0); + + /* When a carrier has been migrated, its block order may differ from that + * of the allocator it's been migrated to. */ + crr->blk_order = alc->blk_order; } #define IS_CRR_IN_TREE(CRR,ROOT) \ diff --git a/erts/emulator/beam/erl_ao_firstfit_alloc.h b/erts/emulator/beam/erl_ao_firstfit_alloc.h index dad864801f..9c9b98da86 100644 --- a/erts/emulator/beam/erl_ao_firstfit_alloc.h +++ b/erts/emulator/beam/erl_ao_firstfit_alloc.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2003-2016. All Rights Reserved. + * Copyright Ericsson AB 2003-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -32,7 +32,12 @@ enum AOFFSortOrder { FF_AGEFF = 0, /* carrier trees only */ FF_AOFF = 1, FF_AOBF = 2, /* block trees only */ - FF_BF = 3 /* block trees only */ + FF_BF = 3, /* block trees only */ + + FF_CHAOS = -1 /* A test-specific sort order that picks any of the above + * after instance id. Used to test that carriers created + * under one order will work fine after being migrated + * to another. */ }; typedef struct { diff --git a/erts/emulator/beam/erl_arith.c b/erts/emulator/beam/erl_arith.c index b6625db0d3..144fb56ea5 100644 --- a/erts/emulator/beam/erl_arith.c +++ b/erts/emulator/beam/erl_arith.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1999-2016. All Rights Reserved. + * Copyright Ericsson AB 1999-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/erl_async.c b/erts/emulator/beam/erl_async.c index 3ceb2fd368..605a2b3461 100644 --- a/erts/emulator/beam/erl_async.c +++ b/erts/emulator/beam/erl_async.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2000-2016. All Rights Reserved. + * Copyright Ericsson AB 2000-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/erl_bestfit_alloc.c b/erts/emulator/beam/erl_bestfit_alloc.c index 85fc4c3a85..9cb1199c2a 100644 --- a/erts/emulator/beam/erl_bestfit_alloc.c +++ b/erts/emulator/beam/erl_bestfit_alloc.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2003-2016. All Rights Reserved. + * Copyright Ericsson AB 2003-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/erl_bif_binary.c b/erts/emulator/beam/erl_bif_binary.c index 469f6a1ea8..a2610bf2e1 100644 --- a/erts/emulator/beam/erl_bif_binary.c +++ b/erts/emulator/beam/erl_bif_binary.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2010-2017. All Rights Reserved. + * Copyright Ericsson AB 2010-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/erl_bif_chksum.c b/erts/emulator/beam/erl_bif_chksum.c index cf92687595..cce8472ccb 100644 --- a/erts/emulator/beam/erl_bif_chksum.c +++ b/erts/emulator/beam/erl_bif_chksum.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2008-2016. All Rights Reserved. + * Copyright Ericsson AB 2008-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/erl_bif_ddll.c b/erts/emulator/beam/erl_bif_ddll.c index 294bce115f..639aee29dc 100644 --- a/erts/emulator/beam/erl_bif_ddll.c +++ b/erts/emulator/beam/erl_bif_ddll.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2006-2017. All Rights Reserved. + * Copyright Ericsson AB 2006-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -829,7 +829,7 @@ BIF_RETTYPE erl_ddll_format_error_int_1(BIF_ALIST_1) "cannot be loaded/unloaded"; break; case am_permanent: - errstring = "DDLL driver is permanent an can not be unloaded/loaded"; + errstring = "DDLL driver is permanent an cannot be unloaded/loaded"; break; case am_not_loaded: errstring = "DDLL driver is not loaded"; diff --git a/erts/emulator/beam/erl_bif_info.c b/erts/emulator/beam/erl_bif_info.c index 5789fa8e71..7fada0d548 100644 --- a/erts/emulator/beam/erl_bif_info.c +++ b/erts/emulator/beam/erl_bif_info.c @@ -2093,17 +2093,6 @@ current_stacktrace(ErtsHeapFactory *hfact, Process* rp, return res; } -#if defined(VALGRIND) -static int check_if_xml(void) -{ - char buf[1]; - size_t bufsz = sizeof(buf); - return erts_sys_explicit_8bit_getenv("VALGRIND_LOG_XML", buf, &bufsz) >= 0; -} -#else -#define check_if_xml() 0 -#endif - /* * This function takes care of calls to erlang:system_info/1 when the argument * is a tuple. @@ -2200,15 +2189,9 @@ info_1_tuple(Process* BIF_P, /* Pointer to current process. */ #endif } else if (is_list(*tp)) { #if defined(PURIFY) -#define ERTS_ERROR_CHECKER_PRINTF purify_printf -#define ERTS_ERROR_CHECKER_PRINTF_XML purify_printf +# define ERTS_ERROR_CHECKER_PRINTF purify_printf #elif defined(VALGRIND) -#define ERTS_ERROR_CHECKER_PRINTF VALGRIND_PRINTF -# ifndef HAVE_VALGRIND_PRINTF_XML -# define ERTS_ERROR_CHECKER_PRINTF_XML VALGRIND_PRINTF -# else -# define ERTS_ERROR_CHECKER_PRINTF_XML VALGRIND_PRINTF_XML -# endif +# define ERTS_ERROR_CHECKER_PRINTF VALGRIND_PRINTF #endif ErlDrvSizeT buf_size = 8*1024; /* Try with 8KB first */ char *buf = erts_alloc(ERTS_ALC_T_TMP, buf_size); @@ -2224,12 +2207,7 @@ info_1_tuple(Process* BIF_P, /* Pointer to current process. */ ASSERT(r == buf_size - 1); } buf[buf_size - 1 - r] = '\0'; - if (check_if_xml()) { - ERTS_ERROR_CHECKER_PRINTF_XML("<erlang_info_log>" - "%s</erlang_info_log>\n", buf); - } else { - ERTS_ERROR_CHECKER_PRINTF("%s\n", buf); - } + ERTS_ERROR_CHECKER_PRINTF("%s\n", buf); erts_free(ERTS_ALC_T_TMP, (void *) buf); BIF_RET(am_true); #undef ERTS_ERROR_CHECKER_PRINTF diff --git a/erts/emulator/beam/erl_bif_re.c b/erts/emulator/beam/erl_bif_re.c index 4d769c2d46..bbc64eb9aa 100644 --- a/erts/emulator/beam/erl_bif_re.c +++ b/erts/emulator/beam/erl_bif_re.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2008-2017. All Rights Reserved. + * Copyright Ericsson AB 2008-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/erl_bif_trace.c b/erts/emulator/beam/erl_bif_trace.c index 9861483bf0..711e62c795 100644 --- a/erts/emulator/beam/erl_bif_trace.c +++ b/erts/emulator/beam/erl_bif_trace.c @@ -810,7 +810,7 @@ Eterm trace_info_2(BIF_ALIST_2) } erts_release_code_write_permission(); - if (is_internal_ref(res)) + if (is_value(res) && is_internal_ref(res)) BIF_TRAP1(erts_await_result, BIF_P, res); BIF_RET(res); diff --git a/erts/emulator/beam/erl_bif_unique.h b/erts/emulator/beam/erl_bif_unique.h index 6af9d4ac8c..40b70667c0 100644 --- a/erts/emulator/beam/erl_bif_unique.h +++ b/erts/emulator/beam/erl_bif_unique.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2014-2017. All Rights Reserved. + * Copyright Ericsson AB 2014-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/erl_binary.h b/erts/emulator/beam/erl_binary.h index 7dfd0c273a..08edb43c49 100644 --- a/erts/emulator/beam/erl_binary.h +++ b/erts/emulator/beam/erl_binary.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2000-2017. All Rights Reserved. + * Copyright Ericsson AB 2000-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/erl_bits.h b/erts/emulator/beam/erl_bits.h index a3816fa820..7beef5cfda 100644 --- a/erts/emulator/beam/erl_bits.h +++ b/erts/emulator/beam/erl_bits.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1999-2017. All Rights Reserved. + * Copyright Ericsson AB 1999-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/erl_db.c b/erts/emulator/beam/erl_db.c index 82e31f4cab..c009a3bde8 100644 --- a/erts/emulator/beam/erl_db.c +++ b/erts/emulator/beam/erl_db.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1996-2017. All Rights Reserved. + * Copyright Ericsson AB 1996-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -3647,6 +3647,7 @@ send_ets_transfer_message(Process *c_p, Process *proc, hd_copy = copy_struct(heir_data, hd_sz, &hp, ohp); sender = c_p->common.id; msg = TUPLE4(hp, am_ETS_TRANSFER, tid, sender, hd_copy); + ERL_MESSAGE_TOKEN(mp) = am_undefined; erts_queue_proc_message(c_p, proc, *locks, mp, msg); } diff --git a/erts/emulator/beam/erl_db.h b/erts/emulator/beam/erl_db.h index db1dec015c..23975d208f 100644 --- a/erts/emulator/beam/erl_db.h +++ b/erts/emulator/beam/erl_db.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1996-2016. All Rights Reserved. + * Copyright Ericsson AB 1996-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/erl_db_hash.c b/erts/emulator/beam/erl_db_hash.c index 74d63325e6..b988a19cf4 100644 --- a/erts/emulator/beam/erl_db_hash.c +++ b/erts/emulator/beam/erl_db_hash.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1998-2017. All Rights Reserved. + * Copyright Ericsson AB 1998-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/erl_db_hash.h b/erts/emulator/beam/erl_db_hash.h index 08e5b13db1..eae5537ba4 100644 --- a/erts/emulator/beam/erl_db_hash.h +++ b/erts/emulator/beam/erl_db_hash.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1998-2017. All Rights Reserved. + * Copyright Ericsson AB 1998-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/erl_db_tree.c b/erts/emulator/beam/erl_db_tree.c index 0692583dd4..2eb874b005 100644 --- a/erts/emulator/beam/erl_db_tree.c +++ b/erts/emulator/beam/erl_db_tree.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1998-2017. All Rights Reserved. + * Copyright Ericsson AB 1998-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -3126,7 +3126,7 @@ static int partly_bound_can_match_lesser(Eterm partly_bound_1, if (ret) erts_fprintf(stderr," can match lesser than "); else - erts_fprintf(stderr," can not match lesser than "); + erts_fprintf(stderr," cannot match lesser than "); erts_fprintf(stderr,"%T\n",partly_bound_2); #endif return ret; @@ -3144,7 +3144,7 @@ static int partly_bound_can_match_greater(Eterm partly_bound_1, if (ret) erts_fprintf(stderr," can match greater than "); else - erts_fprintf(stderr," can not match greater than "); + erts_fprintf(stderr," cannot match greater than "); erts_fprintf(stderr,"%T\n",partly_bound_2); #endif return ret; diff --git a/erts/emulator/beam/erl_db_tree.h b/erts/emulator/beam/erl_db_tree.h index dc1b93d410..54da2a6bc1 100644 --- a/erts/emulator/beam/erl_db_tree.h +++ b/erts/emulator/beam/erl_db_tree.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1998-2016. All Rights Reserved. + * Copyright Ericsson AB 1998-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/erl_db_util.h b/erts/emulator/beam/erl_db_util.h index 73d242449e..6ec3b4f98f 100644 --- a/erts/emulator/beam/erl_db_util.h +++ b/erts/emulator/beam/erl_db_util.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1998-2017. All Rights Reserved. + * Copyright Ericsson AB 1998-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/erl_dirty_bif.tab b/erts/emulator/beam/erl_dirty_bif.tab index 10c76d2579..086275fbe5 100644 --- a/erts/emulator/beam/erl_dirty_bif.tab +++ b/erts/emulator/beam/erl_dirty_bif.tab @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 2016. All Rights Reserved. +# Copyright Ericsson AB 2016-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/erl_drv_thread.c b/erts/emulator/beam/erl_drv_thread.c index 4cf42fce57..c5dbc87dee 100644 --- a/erts/emulator/beam/erl_drv_thread.c +++ b/erts/emulator/beam/erl_drv_thread.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2007-2016. All Rights Reserved. + * Copyright Ericsson AB 2007-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/erl_goodfit_alloc.c b/erts/emulator/beam/erl_goodfit_alloc.c index e3ba67f0af..01d4aa54ff 100644 --- a/erts/emulator/beam/erl_goodfit_alloc.c +++ b/erts/emulator/beam/erl_goodfit_alloc.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2003-2016. All Rights Reserved. + * Copyright Ericsson AB 2003-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/erl_init.c b/erts/emulator/beam/erl_init.c index 57c6c10c7f..5da7b43b9e 100644 --- a/erts/emulator/beam/erl_init.c +++ b/erts/emulator/beam/erl_init.c @@ -128,7 +128,7 @@ const Eterm etp_hole_marker = 0; static int modified_sched_thread_suggested_stack_size = 0; -Eterm erts_init_process_id; +Eterm erts_init_process_id = ERTS_INVALID_PID; /* * Note about VxWorks: All variables must be initialized by executable code, @@ -2258,6 +2258,7 @@ erl_start(int argc, char **argv) erts_init_process_id = erl_first_process_otp("otp_ring0", NULL, 0, boot_argc, boot_argv); + ASSERT(erts_init_process_id != ERTS_INVALID_PID); { /* diff --git a/erts/emulator/beam/erl_io_queue.c b/erts/emulator/beam/erl_io_queue.c index d779d1031a..2ae5b56b5c 100644 --- a/erts/emulator/beam/erl_io_queue.c +++ b/erts/emulator/beam/erl_io_queue.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2017. All Rights Reserved. + * Copyright Ericsson AB 2017-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/erl_lock_check.h b/erts/emulator/beam/erl_lock_check.h index 138bc810bd..d10e32985a 100644 --- a/erts/emulator/beam/erl_lock_check.h +++ b/erts/emulator/beam/erl_lock_check.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2005-2016. All Rights Reserved. + * Copyright Ericsson AB 2005-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/erl_lock_count.h b/erts/emulator/beam/erl_lock_count.h index 89d95a73cf..0d47b16e0b 100644 --- a/erts/emulator/beam/erl_lock_count.h +++ b/erts/emulator/beam/erl_lock_count.h @@ -532,7 +532,7 @@ ERTS_GLB_INLINE void lcnt_dec_lock_state__(ethr_atomic_t *l_state) { ethr_sint_t state = ethr_atomic_dec_read_acqb(l_state); - /* We can not assume that state is >= -1 here; unlock and unacquire might + /* We cannot assume that state is >= -1 here; unlock and unacquire might * bring it below -1 and race to increment it back. */ if(state < 0) { diff --git a/erts/emulator/beam/erl_map.c b/erts/emulator/beam/erl_map.c index 48154b5d0f..3d6c9eb43f 100644 --- a/erts/emulator/beam/erl_map.c +++ b/erts/emulator/beam/erl_map.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2014-2017. All Rights Reserved. + * Copyright Ericsson AB 2014-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -1505,25 +1505,6 @@ int hashmap_key_hash_cmp(Eterm* ap, Eterm* bp) return ap ? -1 : 1; } -/* maps:new/0 */ - -BIF_RETTYPE maps_new_0(BIF_ALIST_0) { - Eterm* hp; - Eterm tup; - flatmap_t *mp; - - hp = HAlloc(BIF_P, (MAP_HEADER_FLATMAP_SZ + 1)); - tup = make_tuple(hp); - *hp++ = make_arityval(0); - - mp = (flatmap_t*)hp; - mp->thing_word = MAP_HEADER_FLATMAP; - mp->size = 0; - mp->keys = tup; - - BIF_RET(make_flatmap(mp)); -} - /* maps:put/3 */ BIF_RETTYPE maps_put_3(BIF_ALIST_3) { @@ -1707,11 +1688,16 @@ int erts_maps_update(Process *p, Eterm key, Eterm value, Eterm map, Eterm *res) return 0; found_key: - *hp++ = value; - vs++; - if (++i < n) - sys_memcpy(hp, vs, (n - i)*sizeof(Eterm)); - *res = make_flatmap(shp); + if(*vs == value) { + HRelease(p, shp + MAP_HEADER_FLATMAP_SZ + n, shp); + *res = map; + } else { + *hp++ = value; + vs++; + if (++i < n) + sys_memcpy(hp, vs, (n - i)*sizeof(Eterm)); + *res = make_flatmap(shp); + } return 1; } @@ -1767,9 +1753,7 @@ Eterm erts_maps_put(Process *p, Eterm key, Eterm value, Eterm map) { if (is_immed(key)) { for( i = 0; i < n; i ++) { if (ks[i] == key) { - *hp++ = value; - vs++; - c = 1; + goto found_key; } else { *hp++ = *vs++; } @@ -1777,18 +1761,13 @@ Eterm erts_maps_put(Process *p, Eterm key, Eterm value, Eterm map) { } else { for( i = 0; i < n; i ++) { if (EQ(ks[i], key)) { - *hp++ = value; - vs++; - c = 1; + goto found_key; } else { *hp++ = *vs++; } } } - if (c) - return res; - /* the map will grow */ if (n >= MAP_SMALL_MAP_LIMIT) { @@ -1843,6 +1822,18 @@ Eterm erts_maps_put(Process *p, Eterm key, Eterm value, Eterm map) { */ *shp = make_pos_bignum_header(0); return res; + +found_key: + if(*vs == value) { + HRelease(p, shp + MAP_HEADER_FLATMAP_SZ + n, shp); + return map; + } else { + *hp++ = value; + vs++; + if (++i < n) + sys_memcpy(hp, vs, (n - i)*sizeof(Eterm)); + return res; + } } ASSERT(is_hashmap(map)); diff --git a/erts/emulator/beam/erl_message.c b/erts/emulator/beam/erl_message.c index 507cc989d2..a3274d7443 100644 --- a/erts/emulator/beam/erl_message.c +++ b/erts/emulator/beam/erl_message.c @@ -409,6 +409,11 @@ ErtsMessage* prepend_pending_sig_maybe(Process* sender, Process* receiver, * * @brief Send one message from *NOT* a local process. * + * seq_trace does not work with this type of messages + * to it is set to am_undefined which means that the + * receiving process will not remove the seq_trace token + * when it gets this message. + * */ void erts_queue_message(Process* receiver, ErtsProcLocks receiver_locks, @@ -417,11 +422,19 @@ erts_queue_message(Process* receiver, ErtsProcLocks receiver_locks, ASSERT(is_not_internal_pid(from)); ERL_MESSAGE_TERM(mp) = msg; ERL_MESSAGE_FROM(mp) = from; + ERL_MESSAGE_TOKEN(mp) = am_undefined; queue_messages(receiver, receiver_locks, mp, &mp->next, 1); } /** * @brief Send one message from a local process. + * + * It is up to the caller of this function to set the + * correct seq_trace. The general rule of thumb is that + * it should be set to am_undefined if the message + * cannot be traced using seq_trace, if it can be + * traced it should be set to the trace token. It should + * very rarely be explicitly set to NIL! */ void erts_queue_proc_message(Process* sender, @@ -584,8 +597,7 @@ void erts_send_message(Process* sender, Process* receiver, ErtsProcLocks *receiver_locks, - Eterm message, - unsigned flags) + Eterm message) { Uint msize; ErtsMessage* mp; @@ -619,7 +631,7 @@ erts_send_message(Process* sender, receiver_state = erts_atomic32_read_nob(&receiver->state); - if (SEQ_TRACE_TOKEN(sender) != NIL && !(flags & ERTS_SND_FLG_NO_SEQ_TRACE)) { + if (SEQ_TRACE_TOKEN(sender) != NIL) { Eterm* hp; Eterm stoken = SEQ_TRACE_TOKEN(sender); Uint seq_trace_size = 0; diff --git a/erts/emulator/beam/erl_message.h b/erts/emulator/beam/erl_message.h index d120111634..b2550814fd 100644 --- a/erts/emulator/beam/erl_message.h +++ b/erts/emulator/beam/erl_message.h @@ -405,8 +405,6 @@ typedef struct erl_trace_message_queue__ { #define SAVE_MESSAGE(p) \ (p)->sig_qs.save = &(*(p)->sig_qs.save)->next -#define ERTS_SND_FLG_NO_SEQ_TRACE (((unsigned) 1) << 0) - #define ERTS_HEAP_FRAG_SIZE(DATA_WORDS) \ (sizeof(ErlHeapFragment) - sizeof(Eterm) + (DATA_WORDS)*sizeof(Eterm)) @@ -429,7 +427,7 @@ typedef struct erl_trace_message_queue__ { do { \ (MP)->next = NULL; \ ERL_MESSAGE_TERM(MP) = THE_NON_VALUE; \ - ERL_MESSAGE_TOKEN(MP) = NIL; \ + ERL_MESSAGE_TOKEN(MP) = THE_NON_VALUE; \ ERL_MESSAGE_FROM(MP) = NIL; \ ERL_MESSAGE_DT_UTAG_INIT(MP); \ MP->data.attached = NULL; \ @@ -446,7 +444,7 @@ void erts_queue_proc_message(Process* from,Process* to, ErtsProcLocks,ErtsMessag void erts_queue_proc_messages(Process* from, Process* to, ErtsProcLocks, ErtsMessage*, ErtsMessage**, Uint); void erts_deliver_exit_message(Eterm, Process*, ErtsProcLocks *, Eterm, Eterm); -void erts_send_message(Process*, Process*, ErtsProcLocks*, Eterm, unsigned); +void erts_send_message(Process*, Process*, ErtsProcLocks*, Eterm); void erts_link_mbuf_to_proc(Process *proc, ErlHeapFragment *bp); Uint erts_msg_attached_data_size_aux(ErtsMessage *msg); diff --git a/erts/emulator/beam/erl_msacc.c b/erts/emulator/beam/erl_msacc.c index d13d6080e1..375b004b5b 100644 --- a/erts/emulator/beam/erl_msacc.c +++ b/erts/emulator/beam/erl_msacc.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2014-2017. All Rights Reserved. + * Copyright Ericsson AB 2014-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/erl_msacc.h b/erts/emulator/beam/erl_msacc.h index 895b1ae319..abea18b340 100644 --- a/erts/emulator/beam/erl_msacc.h +++ b/erts/emulator/beam/erl_msacc.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2014-2016. All Rights Reserved. + * Copyright Ericsson AB 2014-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/erl_mtrace.c b/erts/emulator/beam/erl_mtrace.c index 2807b443a1..6e0a0dcff7 100644 --- a/erts/emulator/beam/erl_mtrace.c +++ b/erts/emulator/beam/erl_mtrace.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2003-2017. All Rights Reserved. + * Copyright Ericsson AB 2003-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/erl_nfunc_sched.c b/erts/emulator/beam/erl_nfunc_sched.c index f97e86bf95..b8cf2bee0e 100644 --- a/erts/emulator/beam/erl_nfunc_sched.c +++ b/erts/emulator/beam/erl_nfunc_sched.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2016. All Rights Reserved. + * Copyright Ericsson AB 2016-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/erl_nfunc_sched.h b/erts/emulator/beam/erl_nfunc_sched.h index b8a4e4ebc3..1cb252eba5 100644 --- a/erts/emulator/beam/erl_nfunc_sched.h +++ b/erts/emulator/beam/erl_nfunc_sched.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2016. All Rights Reserved. + * Copyright Ericsson AB 2016-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/erl_nif.c b/erts/emulator/beam/erl_nif.c index 0fbf0eb03a..7339aa8874 100644 --- a/erts/emulator/beam/erl_nif.c +++ b/erts/emulator/beam/erl_nif.c @@ -453,8 +453,18 @@ static void full_flush_env(ErlNifEnv* env) static void full_cache_env(ErlNifEnv* env) { - if (env->proc->static_flags & ERTS_STC_FLG_SHADOW_PROC) + if (env->proc->static_flags & ERTS_STC_FLG_SHADOW_PROC) { erts_cache_dirty_shadow_proc(env->proc); + /* + * If shadow proc had heap fragments when flushed + * those have now been moved to the real proc. + * Ensure heap pointers do not point into a heap + * fragment on real proc... + */ + ASSERT(!env->proc->mbuf); + env->hp_end = HEAP_LIMIT(env->proc); + env->hp = HEAP_TOP(env->proc); + } cache_env(env); } @@ -744,8 +754,58 @@ int enif_send(ErlNifEnv* env, const ErlNifPid* to_pid, rp_locks = ERTS_PROC_LOCK_MAIN; if (menv) { + Eterm token = c_p ? SEQ_TRACE_TOKEN(c_p) : am_undefined; + if (token != NIL && token != am_undefined) { + /* This code is copied from erts_send_message */ + Eterm stoken = SEQ_TRACE_TOKEN(c_p); +#ifdef USE_VM_PROBES + DTRACE_CHARBUF(sender_name, 64); + DTRACE_CHARBUF(receiver_name, 64); + Sint tok_label = 0; + Sint tok_lastcnt = 0; + Sint tok_serial = 0; + Eterm utag = NIL; + *sender_name = *receiver_name = '\0'; + if (DTRACE_ENABLED(message_send)) { + erts_snprintf(sender_name, sizeof(DTRACE_CHARBUF_NAME(sender_name)), + "%T", c_p->common.id); + erts_snprintf(receiver_name, sizeof(DTRACE_CHARBUF_NAME(receiver_name)), + "%T", rp->common.id); + } +#endif + if (have_seqtrace(stoken)) { + seq_trace_update_send(c_p); + seq_trace_output(stoken, msg, SEQ_TRACE_SEND, + rp->common.id, c_p); + } +#ifdef USE_VM_PROBES + if (!(DT_UTAG_FLAGS(c_p) & DT_UTAG_SPREADING)) { + stoken = NIL; + } +#endif + token = enif_make_copy(msg_env, stoken); + +#ifdef USE_VM_PROBES + if (DT_UTAG_FLAGS(c_p) & DT_UTAG_SPREADING) { + if (is_immed(DT_UTAG(c_p))) + utag = DT_UTAG(c_p); + else + utag = enif_make_copy(msg_env, DT_UTAG(c_p)); + } + if (DTRACE_ENABLED(message_send)) { + if (have_seqtrace(stoken)) { + tok_label = SEQ_TRACE_T_DTRACE_LABEL(stoken); + tok_lastcnt = signed_val(SEQ_TRACE_T_LASTCNT(stoken)); + tok_serial = signed_val(SEQ_TRACE_T_SERIAL(stoken)); + } + DTRACE6(message_send, sender_name, receiver_name, + size_object(msg), tok_label, tok_lastcnt, tok_serial); + } +#endif + } flush_env(msg_env); mp = erts_alloc_message(0, NULL); + ERL_MESSAGE_TOKEN(mp) = token; mp->data.heap_frag = menv->env.heap_frag; ASSERT(mp->data.heap_frag == MBUF(&menv->phony_proc)); if (mp->data.heap_frag != NULL) { @@ -783,6 +843,7 @@ int enif_send(ErlNifEnv* env, const ErlNifPid* to_pid, ohp = &bp->off_heap; } } + ERL_MESSAGE_TOKEN(mp) = am_undefined; msg = copy_struct_litopt(msg, sz, &hp, ohp, &litarea); } @@ -826,6 +887,7 @@ int enif_send(ErlNifEnv* env, const ErlNifPid* to_pid, ERL_MESSAGE_TERM(mp) = msg; ERL_MESSAGE_FROM(mp) = from; + ERL_MESSAGE_TOKEN(mp) = am_undefined; if (!msgq) { msgq = erts_alloc(ERTS_ALC_T_TRACE_MSG_QUEUE, @@ -978,7 +1040,7 @@ ERL_NIF_TERM enif_make_copy(ErlNifEnv* dst_env, ERL_NIF_TERM src_term) Eterm* hp; /* * No preserved sharing allowed as long as literals are also preserved. - * Process independent environment can not be reached by purge. + * Process independent environment cannot be reached by purge. */ sz = size_object(src_term); hp = alloc_heap(dst_env, sz); diff --git a/erts/emulator/beam/erl_nif.h b/erts/emulator/beam/erl_nif.h index 1906da732b..4c09496ef1 100644 --- a/erts/emulator/beam/erl_nif.h +++ b/erts/emulator/beam/erl_nif.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2009-2017. All Rights Reserved. + * Copyright Ericsson AB 2009-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/erl_nif_api_funcs.h b/erts/emulator/beam/erl_nif_api_funcs.h index 61f8fcf6ed..81f64f2390 100644 --- a/erts/emulator/beam/erl_nif_api_funcs.h +++ b/erts/emulator/beam/erl_nif_api_funcs.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2009-2017. All Rights Reserved. + * Copyright Ericsson AB 2009-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/erl_node_container_utils.h b/erts/emulator/beam/erl_node_container_utils.h index 99e938266b..eb23e1eaa5 100644 --- a/erts/emulator/beam/erl_node_container_utils.h +++ b/erts/emulator/beam/erl_node_container_utils.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2001-2017. All Rights Reserved. + * Copyright Ericsson AB 2001-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/erl_port_task.c b/erts/emulator/beam/erl_port_task.c index 3953a4c2e9..4928d80f27 100644 --- a/erts/emulator/beam/erl_port_task.c +++ b/erts/emulator/beam/erl_port_task.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2006-2017. All Rights Reserved. + * Copyright Ericsson AB 2006-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/erl_printf_term.c b/erts/emulator/beam/erl_printf_term.c index 910f241a3a..990a01b96f 100644 --- a/erts/emulator/beam/erl_printf_term.c +++ b/erts/emulator/beam/erl_printf_term.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2005-2017. All Rights Reserved. + * Copyright Ericsson AB 2005-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/erl_proc_sig_queue.c b/erts/emulator/beam/erl_proc_sig_queue.c index 1aa390d94d..f90501cbf7 100644 --- a/erts/emulator/beam/erl_proc_sig_queue.c +++ b/erts/emulator/beam/erl_proc_sig_queue.c @@ -52,8 +52,8 @@ #define ERTS_SIG_Q_OP_MAX 13 -#define ERTS_SIG_Q_OP_EXIT 0 -#define ERTS_SIG_Q_OP_EXIT_LINKED 1 +#define ERTS_SIG_Q_OP_EXIT 0 /* Exit signal due to bif call */ +#define ERTS_SIG_Q_OP_EXIT_LINKED 1 /* Exit signal due to link break*/ #define ERTS_SIG_Q_OP_MONITOR_DOWN 2 #define ERTS_SIG_Q_OP_MONITOR 3 #define ERTS_SIG_Q_OP_DEMONITOR 4 @@ -1167,10 +1167,7 @@ erts_proc_sig_send_persistent_monitor_msg(Uint16 type, Eterm key, ERL_MESSAGE_TERM(mp) = ERTS_PROC_SIG_MAKE_TAG(ERTS_SIG_Q_OP_PERSISTENT_MON_MSG, type, 0); ERL_MESSAGE_FROM(mp) = from; - ERL_MESSAGE_TOKEN(mp) = NIL; -#ifdef USE_VM_PROBES - ERL_MESSAGE_DT_UTAG(mp) = NIL; -#endif + ERL_MESSAGE_TOKEN(mp) = am_undefined; if (!proc_queue_signal(NULL, to, (ErtsSignal *) mp, ERTS_SIG_Q_OP_PERSISTENT_MON_MSG)) { @@ -1564,11 +1561,6 @@ erts_proc_sig_send_is_alive_request(Process *c_p, Eterm to, Eterm ref) ERL_MESSAGE_TERM(mp) = ERTS_PROC_SIG_MAKE_TAG(ERTS_SIG_Q_OP_IS_ALIVE, ERTS_SIG_Q_TYPE_UNDEFINED, 0); - ERL_MESSAGE_TOKEN(mp) = NIL; - ERL_MESSAGE_FROM(mp) = am_system; -#ifdef USE_VM_PROBES - ERL_MESSAGE_DT_UTAG(mp) = NIL; -#endif if (proc_queue_signal(c_p, to, (ErtsSignal *) mp, ERTS_SIG_Q_OP_IS_ALIVE)) (void) maybe_elevate_sig_handling_prio(c_p, to); @@ -1672,11 +1664,6 @@ erts_proc_sig_send_sync_suspend(Process *c_p, Eterm to, Eterm tag, Eterm reply) ERL_MESSAGE_TERM(mp) = ERTS_PROC_SIG_MAKE_TAG(ERTS_SIG_Q_OP_SYNC_SUSPEND, ERTS_SIG_Q_TYPE_UNDEFINED, 0); - ERL_MESSAGE_TOKEN(mp) = NIL; - ERL_MESSAGE_FROM(mp) = am_system; -#ifdef USE_VM_PROBES - ERL_MESSAGE_DT_UTAG(mp) = NIL; -#endif if (proc_queue_signal(c_p, to, (ErtsSignal *) mp, ERTS_SIG_Q_OP_SYNC_SUSPEND)) (void) maybe_elevate_sig_handling_prio(c_p, to); @@ -1790,7 +1777,7 @@ handle_rpc(Process *c_p, ErtsProcSigRPC *rpc, int cnt, int limit, int *yieldp) msg = TUPLE2(hp, ref, res); mp->hfrag.next = bp; - + ERL_MESSAGE_TOKEN(mp) = am_undefined; erts_queue_proc_message(c_p, rp, 0, mp, msg); } @@ -2155,10 +2142,7 @@ handle_exit_signal(Process *c_p, ErtsSigRecvTracing *tracing, pid = STORE_NC(&hp, ohp, from); ERL_MESSAGE_TERM(mp) = TUPLE3(hp, am_EXIT, pid, reason); - ERL_MESSAGE_TOKEN(mp) = NIL; -#ifdef USE_VM_PROBES - ERL_MESSAGE_DT_UTAG(mp) = NIL; -#endif + ERL_MESSAGE_TOKEN(mp) = am_undefined; if (is_immed(pid)) ERL_MESSAGE_FROM(mp) = pid; else { @@ -2346,10 +2330,7 @@ convert_to_down_message(Process *c_p, type, from, reason); hp += 6; - ERL_MESSAGE_TOKEN(mp) = NIL; -#ifdef USE_VM_PROBES - ERL_MESSAGE_DT_UTAG(mp) = NIL; -#endif + ERL_MESSAGE_TOKEN(mp) = am_undefined; /* Replace original signal with the exit message... */ convert_to_msg(c_p, sig, mp, next_nm_sig); @@ -2397,10 +2378,7 @@ convert_to_nodedown_messages(Process *c_p, ERL_MESSAGE_TERM(mp) = TUPLE2(hp, am_nodedown, node); ERL_MESSAGE_FROM(mp) = am_system; - ERL_MESSAGE_TOKEN(mp) = NIL; -#ifdef USE_VM_PROBES - ERL_MESSAGE_DT_UTAG(mp) = NIL; -#endif + ERL_MESSAGE_TOKEN(mp) = am_undefined; mp->next = nd_first; nd_first = mp; if (!nd_last) @@ -2723,6 +2701,9 @@ handle_process_info(Process *c_p, ErtsSigRecvTracing *tracing, Uint reds = 0; Process *rp; + ASSERT(!!is_alive == !(erts_atomic32_read_nob(&c_p->state) + & ERTS_PSFLG_EXITING)); + if (pisig->msgq_len_offset != ERTS_PROC_SIG_PI_MSGQ_LEN_IGNORE) { /* * Request requires message queue data to be updated @@ -2827,6 +2808,7 @@ handle_process_info(Process *c_p, ErtsSigRecvTracing *tracing, if (is_alive) erts_factory_trim_and_close(&hfact, &msg, 1); + ERL_MESSAGE_TOKEN(mp) = am_undefined; erts_queue_proc_message(c_p, rp, locks, mp, msg); if (!is_alive && locks) @@ -2928,6 +2910,7 @@ sync_suspend_reply(Process *c_p, ErtsMessage *mp, erts_aint32_t state) tp[2] = ssusp->async ? am_not_suspended : am_internal_error; } } + ERL_MESSAGE_TOKEN(mp) = am_undefined; erts_queue_proc_message(c_p, rp, 0, mp, ssusp->message); } } @@ -3007,10 +2990,8 @@ erts_proc_sig_handle_incoming(Process *c_p, erts_aint32_t *statep, ERTS_HDBG_CHECK_SIGNAL_PRIV_QUEUE(c_p, 0); ERTS_LC_ASSERT(ERTS_PROC_LOCK_MAIN == erts_proc_lc_my_proc_locks(c_p)); - if (local_only) - state = -1; /* can never be a valid state... */ - else { - state = erts_atomic32_read_nob(&c_p->state); + state = erts_atomic32_read_nob(&c_p->state); + if (!local_only) { if (ERTS_PSFLG_SIG_IN_Q & state) { erts_proc_lock(c_p, ERTS_PROC_LOCK_MSGQ); erts_proc_sig_fetch(c_p); @@ -3023,13 +3004,15 @@ erts_proc_sig_handle_incoming(Process *c_p, erts_aint32_t *statep, yield = 0; if (!c_p->sig_qs.cont) { - if (state == -1) - *statep = erts_atomic32_read_nob(&c_p->state); - else - *statep = state; + *statep = state; return !0; } + if (state & ERTS_PSFLG_EXITING) { + *statep = state; + return 0; + } + next_nm_sig = &c_p->sig_qs.nmsigs.next; setup_tracing_state(c_p, &tracing); diff --git a/erts/emulator/beam/erl_process.c b/erts/emulator/beam/erl_process.c index 8253ec4f40..9386f79b56 100644 --- a/erts/emulator/beam/erl_process.c +++ b/erts/emulator/beam/erl_process.c @@ -3761,6 +3761,8 @@ dequeue_process(ErtsRunQueue *runq, int prio_q, erts_aint32_t *statep) ERTS_THR_DATA_DEPENDENCY_READ_MEMORY_BARRIER; state = erts_atomic32_read_nob(&p->state); + ASSERT(state & ERTS_PSFLG_IN_RUNQ); + if (statep) *statep = state; @@ -3768,8 +3770,7 @@ dequeue_process(ErtsRunQueue *runq, int prio_q, erts_aint32_t *statep) rqi = &runq->procs.prio_info[prio]; - if (p) - unqueue_process(runq, rpq, rqi, prio, NULL, p); + unqueue_process(runq, rpq, rqi, prio, NULL, p); return p; } @@ -4088,7 +4089,7 @@ evacuate_run_queue(ErtsRunQueue *rq, erts_runq_unlock(to_rq); smp_notify_inc_runq(to_rq); - erts_runq_lock(to_rq); + erts_runq_lock(rq); } if (rq->ports.start) { @@ -4157,22 +4158,17 @@ evacuate_run_queue(ErtsRunQueue *rq, free_proxy_proc(proc); else { erts_aint32_t clr_bits; -#ifdef DEBUG - erts_aint32_t old; -#endif clr_bits = ERTS_PSFLG_IN_RUNQ; clr_bits |= qbit << ERTS_PSFLGS_IN_PRQ_MASK_OFFSET; -#ifdef DEBUG - old = -#else - (void) -#endif - erts_atomic32_read_band_mb(&proc->state, - ~clr_bits); - ASSERT((old & clr_bits) == clr_bits); + state = erts_atomic32_read_band_mb(&proc->state, ~clr_bits); + ASSERT((state & clr_bits) == clr_bits); + if (state & ERTS_PSFLG_FREE) { + /* free and not queued by proxy */ + erts_proc_dec_refc(proc); + } } goto handle_next_proc; @@ -6208,13 +6204,14 @@ fin_dirty_enq_s_change(Process *p, /* Already enqueue by someone else... */ if (pstruct_reserved) { /* We reserved process struct for enqueue; clear it... */ -#ifdef DEBUG - erts_aint32_t old = -#else - (void) -#endif - erts_atomic32_read_band_nob(&p->state, ~ERTS_PSFLG_IN_RUNQ); - ASSERT(old & ERTS_PSFLG_IN_RUNQ); + erts_aint32_t state; + + state = erts_atomic32_read_band_nob(&p->state, ~ERTS_PSFLG_IN_RUNQ); + ASSERT(state & ERTS_PSFLG_IN_RUNQ); + + if (state & ERTS_PSFLG_FREE) { + erts_proc_dec_refc(p); + } } return 0; } @@ -6407,8 +6404,9 @@ schedule_out_process(ErtsRunQueue *c_rq, erts_aint32_t state, Process *p, == ERTS_PSFLG_ACTIVE)); n &= ~running_flgs; - if ((a & (ERTS_PSFLG_ACTIVE_SYS|ERTS_PSFLG_DIRTY_ACTIVE_SYS)) - || (a & (ERTS_PSFLG_ACTIVE|ERTS_PSFLG_SUSPENDED)) == ERTS_PSFLG_ACTIVE) { + if ((!!(a & (ERTS_PSFLG_ACTIVE_SYS|ERTS_PSFLG_DIRTY_ACTIVE_SYS)) + | ((a & (ERTS_PSFLG_ACTIVE|ERTS_PSFLG_SUSPENDED)) == ERTS_PSFLG_ACTIVE)) + & !(a & ERTS_PSFLG_FREE)) { enqueue = check_enqueue_in_prio_queue(p, &enq_prio, &n, a); } a = erts_atomic32_cmpxchg_mb(&p->state, n, e); @@ -6655,62 +6653,72 @@ erts_schedule_process(Process *p, erts_aint32_t state, ErtsProcLocks locks) schedule_process(p, state, locks); } +/* Enqueues the given sys task on the process and schedules it. The task may be + * NULL if only scheduling is desired. */ static ERTS_INLINE erts_aint32_t -active_sys_enqueue(Process *p, erts_aint32_t state, - erts_aint32_t enable_flags, int status_locked) -{ - /* - * This function may or may not be called with status locke held. - * It always returns without the status lock held! - */ - unsigned int prof_runnable_procs = erts_system_profile_flags.runnable_procs; - erts_aint32_t n, a = state, enq_prio = -1; - int slocked = status_locked; +active_sys_enqueue(Process *p, ErtsProcSysTask *sys_task, + erts_aint32_t task_prio, erts_aint32_t enable_flags, + erts_aint32_t state, erts_aint32_t *fail_state_p) +{ + int runnable_procs = erts_system_profile_flags.runnable_procs; + erts_aint32_t n, a, enq_prio, fail_state; + int already_scheduled; + int status_locked; int enqueue; /* < 0 -> use proxy */ - /* Status lock prevents out of order "runnable proc" trace msgs */ - ERTS_LC_ASSERT(slocked || !(ERTS_PROC_LOCK_STATUS & erts_proc_lc_my_proc_locks(p))); - ERTS_LC_ASSERT(!slocked || (ERTS_PROC_LOCK_STATUS & erts_proc_lc_my_proc_locks(p))); - - if (!prof_runnable_procs) { - if (slocked) { - erts_proc_unlock(p, ERTS_PROC_LOCK_STATUS); - slocked = 0; - } - } - else { - if (!slocked) { - erts_proc_lock(p, ERTS_PROC_LOCK_STATUS); - slocked = !0; - } - } + enable_flags |= ERTS_PSFLG_ACTIVE_SYS; + fail_state = *fail_state_p; + already_scheduled = 0; + status_locked = 0; + enq_prio = -1; + a = state; + ERTS_LC_ASSERT(!(ERTS_PROC_LOCK_STATUS & erts_proc_lc_my_proc_locks(p))); + ASSERT(fail_state & (ERTS_PSFLG_EXITING | ERTS_PSFLG_FREE)); + ASSERT(!(fail_state & enable_flags)); ASSERT(!(state & ERTS_PSFLG_PROXY)); + /* When runnable_procs is enabled, we need to take the status lock to + * prevent trace messages from being sent in the wrong order. The lock must + * be held over the call to add2runq. + * + * Otherwise, we only need to take it when we're enqueuing a task and can + * safely release it before add2runq. */ + if (sys_task || runnable_procs) { + erts_proc_lock(p, ERTS_PROC_LOCK_STATUS); + status_locked = 1; + } + while (1) { erts_aint32_t e; n = e = a; - if (a & ERTS_PSFLG_FREE) - goto cleanup; /* We don't want to schedule free processes... */ + if (a & fail_state) { + *fail_state_p = a & fail_state; + goto cleanup; + } enqueue = ERTS_ENQUEUE_NOT; - n |= enable_flags; - n |= ERTS_PSFLG_ACTIVE_SYS; + n |= enable_flags; + if (!(a & (ERTS_PSFLG_RUNNING | ERTS_PSFLG_RUNNING_SYS | ERTS_PSFLG_DIRTY_RUNNING - | ERTS_PSFLG_DIRTY_RUNNING_SYS))) + | ERTS_PSFLG_DIRTY_RUNNING_SYS))) { enqueue = check_enqueue_in_prio_queue(p, &enq_prio, &n, a); + } + a = erts_atomic32_cmpxchg_mb(&p->state, n, e); - if (a == e) + if (a == e) { break; - if (a == n && enqueue == ERTS_ENQUEUE_NOT) - goto cleanup; + } + else if (a == n && enqueue == ERTS_ENQUEUE_NOT) { + already_scheduled = 1; + break; + } } - if (prof_runnable_procs) { - + if (!already_scheduled && runnable_procs) { if (!(a & (ERTS_PSFLG_ACTIVE_SYS | ERTS_PSFLG_RUNNING | ERTS_PSFLG_RUNNING_SYS @@ -6720,19 +6728,56 @@ active_sys_enqueue(Process *p, erts_aint32_t state, /* We activated a prevously inactive process */ profile_runnable_proc(p, am_active); } - - erts_proc_unlock(p, ERTS_PROC_LOCK_STATUS); - slocked = 0; } - add2runq(enqueue, enq_prio, p, n, NULL); + if (sys_task) { + ErtsProcSysTaskQs *stqs = p->sys_task_qs; -cleanup: + if (!stqs) { + sys_task->next = sys_task->prev = sys_task; - if (slocked) - erts_proc_unlock(p, ERTS_PROC_LOCK_STATUS); + stqs = proc_sys_task_queues_alloc(); - ERTS_LC_ASSERT(!(ERTS_PROC_LOCK_STATUS & erts_proc_lc_my_proc_locks(p))); + stqs->qmask = 1 << task_prio; + stqs->ncount = 0; + stqs->q[PRIORITY_MAX] = NULL; + stqs->q[PRIORITY_HIGH] = NULL; + stqs->q[PRIORITY_NORMAL] = NULL; + stqs->q[PRIORITY_LOW] = NULL; + stqs->q[task_prio] = sys_task; + + p->sys_task_qs = stqs; + } + else { + if (!stqs->q[task_prio]) { + sys_task->next = sys_task->prev = sys_task; + + stqs->q[task_prio] = sys_task; + stqs->qmask |= 1 << task_prio; + } + else { + sys_task->next = stqs->q[task_prio]; + sys_task->prev = stqs->q[task_prio]->prev; + sys_task->next->prev = sys_task; + sys_task->prev->next = sys_task; + ASSERT(stqs->qmask & (1 << task_prio)); + } + } + } + + if (status_locked && !runnable_procs) { + erts_proc_unlock(p, ERTS_PROC_LOCK_STATUS); + status_locked = 0; + } + + if (!already_scheduled) { + add2runq(enqueue, enq_prio, p, n, NULL); + } + +cleanup: + if (status_locked) { + erts_proc_unlock(p, ERTS_PROC_LOCK_STATUS); + } return n; } @@ -6740,103 +6785,41 @@ cleanup: erts_aint32_t erts_proc_sys_schedule(Process *p, erts_aint32_t state, erts_aint32_t enable_flag) { - /* We are not allowed to call this function with status lock held... */ - return active_sys_enqueue(p, state, enable_flag, 0); + erts_aint32_t fail_state = ERTS_PSFLG_FREE; + + return active_sys_enqueue(p, NULL, 0, enable_flag, state, &fail_state); } static int schedule_process_sys_task(Process *p, erts_aint32_t prio, ErtsProcSysTask *st, erts_aint32_t *fail_state_p) { - int res; - int locked; - ErtsProcSysTaskQs *stqs, *free_stqs; erts_aint32_t fail_state, state; - fail_state = *fail_state_p; - - res = 1; /* prepare for success */ - st->next = st->prev = st; /* Prep for empty prio queue */ + /* Elevate priority if needed. */ state = erts_atomic32_read_nob(&p->state); - locked = 0; - free_stqs = NULL; - if (state & ERTS_PSFLG_SYS_TASKS) - stqs = NULL; - else { - alloc_qs: - stqs = proc_sys_task_queues_alloc(); - stqs->qmask = 1 << prio; - stqs->ncount = 0; - stqs->q[PRIORITY_MAX] = NULL; - stqs->q[PRIORITY_HIGH] = NULL; - stqs->q[PRIORITY_NORMAL] = NULL; - stqs->q[PRIORITY_LOW] = NULL; - stqs->q[prio] = st; - } - - if (!locked) { - locked = 1; - erts_proc_lock(p, ERTS_PROC_LOCK_STATUS); - - state = erts_atomic32_read_nob(&p->state); - if (state & fail_state) { - erts_proc_unlock(p, ERTS_PROC_LOCK_STATUS); - *fail_state_p = (state & fail_state); - free_stqs = stqs; - res = 0; - goto cleanup; - } - } - - if (!p->sys_task_qs) { - if (stqs) - p->sys_task_qs = stqs; - else - goto alloc_qs; - } - else { - free_stqs = stqs; - stqs = p->sys_task_qs; - if (!stqs->q[prio]) { - stqs->q[prio] = st; - stqs->qmask |= 1 << prio; - } - else { - st->next = stqs->q[prio]; - st->prev = stqs->q[prio]->prev; - st->next->prev = st; - st->prev->next = st; - ASSERT(stqs->qmask & (1 << prio)); - } - } - if (ERTS_PSFLGS_GET_ACT_PRIO(state) > prio) { - erts_aint32_t n, a, e; - /* Need to elevate actual prio */ + erts_aint32_t n, a, e; - a = state; - do { - if (ERTS_PSFLGS_GET_ACT_PRIO(a) <= prio) { - n = a; - break; - } - n = e = a; - n &= ~ERTS_PSFLGS_ACT_PRIO_MASK; - n |= (prio << ERTS_PSFLGS_ACT_PRIO_OFFSET); - a = erts_atomic32_cmpxchg_nob(&p->state, n, e); - } while (a != e); - state = n; - } - - /* active_sys_enqueue() always return with status lock unlocked */ - (void) active_sys_enqueue(p, state, ERTS_PSFLG_SYS_TASKS, locked); + a = state; + do { + if (ERTS_PSFLGS_GET_ACT_PRIO(a) <= prio) { + n = a; + break; + } + n = e = a; + n &= ~ERTS_PSFLGS_ACT_PRIO_MASK; + n |= (prio << ERTS_PSFLGS_ACT_PRIO_OFFSET); + a = erts_atomic32_cmpxchg_nob(&p->state, n, e); + } while (a != e); -cleanup: + state = n; + } - if (free_stqs) - proc_sys_task_queues_free(free_stqs); + fail_state = *fail_state_p; - return res; + return !(active_sys_enqueue(p, st, prio, ERTS_PSFLG_SYS_TASKS, + state, fail_state_p) & fail_state); } static ERTS_INLINE int @@ -9563,6 +9546,7 @@ Process *erts_schedule(ErtsSchedulerData *esdp, Process *p, int calls) } else if (state & ERTS_PSFLG_FREE) { /* free and not queued by proxy */ + ASSERT(state & ERTS_PSFLG_IN_RUNQ); erts_proc_dec_refc(p); } if (!is_normal_sched) @@ -9622,7 +9606,7 @@ Process *erts_schedule(ErtsSchedulerData *esdp, Process *p, int calls) ASSERT(!p->scheduler_data); p->scheduler_data = esdp; if ((!!(state & ERTS_PSFLGS_DIRTY_WORK)) - & (!(state & ERTS_PSFLG_ACTIVE_SYS))) { + & (!(state & ERTS_PSFLG_RUNNING_SYS))) { /* Migrate to dirty scheduler... */ sunlock_sched_out_proc: erts_proc_unlock(p, ERTS_PROC_LOCK_STATUS); @@ -9890,6 +9874,7 @@ notify_sys_task_executed(Process *c_p, ErtsProcSysTask *st, ASSERT(hp_start + hsz == hp); #endif + ERL_MESSAGE_TOKEN(mp) = am_undefined; erts_queue_proc_message(c_p, rp, rp_locks, mp, msg); if (c_p == rp) diff --git a/erts/emulator/beam/erl_process.h b/erts/emulator/beam/erl_process.h index a60e117bab..8d20ccdf90 100644 --- a/erts/emulator/beam/erl_process.h +++ b/erts/emulator/beam/erl_process.h @@ -293,7 +293,7 @@ typedef enum { * highest index... * * Remember to update description in erts_pre_init_process() - * when adding new flags... + * and etp-commands when adding new flags... */ typedef enum { diff --git a/erts/emulator/beam/erl_process_dict.c b/erts/emulator/beam/erl_process_dict.c index 38be3938cd..64ee483079 100644 --- a/erts/emulator/beam/erl_process_dict.c +++ b/erts/emulator/beam/erl_process_dict.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1999-2017. All Rights Reserved. + * Copyright Ericsson AB 1999-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/erl_process_dict.h b/erts/emulator/beam/erl_process_dict.h index b89b387f5a..3ff2354f91 100644 --- a/erts/emulator/beam/erl_process_dict.h +++ b/erts/emulator/beam/erl_process_dict.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1999-2016. All Rights Reserved. + * Copyright Ericsson AB 1999-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/erl_process_dump.c b/erts/emulator/beam/erl_process_dump.c index 243db4c734..706530023b 100644 --- a/erts/emulator/beam/erl_process_dump.c +++ b/erts/emulator/beam/erl_process_dump.c @@ -963,12 +963,16 @@ dump_module_literals(fmtfn_t to, void *to_arg, ErtsLiteralArea* lit_area) } erts_putc(to, to_arg, '\n'); } - } else if (is_export_header(w)) { + } else if (is_export_header(w) || is_fun_header(w)) { dump_externally(to, to_arg, term); erts_putc(to, to_arg, '\n'); } size = 1 + header_arity(w); switch (w & _HEADER_SUBTAG_MASK) { + case FUN_SUBTAG: + ASSERT(((ErlFunThing*)(htop))->num_free == 0); + size += 1; + break; case MAP_SUBTAG: if (is_flatmap_header(w)) { size += 1 + flatmap_get_size(htop); diff --git a/erts/emulator/beam/erl_sched_spec_pre_alloc.c b/erts/emulator/beam/erl_sched_spec_pre_alloc.c index 4a6e02281a..9766e76a83 100644 --- a/erts/emulator/beam/erl_sched_spec_pre_alloc.c +++ b/erts/emulator/beam/erl_sched_spec_pre_alloc.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2011-2016. All Rights Reserved. + * Copyright Ericsson AB 2011-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/erl_sched_spec_pre_alloc.h b/erts/emulator/beam/erl_sched_spec_pre_alloc.h index d232db0e69..b119c59ab3 100644 --- a/erts/emulator/beam/erl_sched_spec_pre_alloc.h +++ b/erts/emulator/beam/erl_sched_spec_pre_alloc.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2011-2016. All Rights Reserved. + * Copyright Ericsson AB 2011-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/erl_thr_progress.c b/erts/emulator/beam/erl_thr_progress.c index 96824dc06e..aa08eb40ec 100644 --- a/erts/emulator/beam/erl_thr_progress.c +++ b/erts/emulator/beam/erl_thr_progress.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2011-2016. All Rights Reserved. + * Copyright Ericsson AB 2011-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/erl_thr_progress.h b/erts/emulator/beam/erl_thr_progress.h index 8c029bcf99..8329995b24 100644 --- a/erts/emulator/beam/erl_thr_progress.h +++ b/erts/emulator/beam/erl_thr_progress.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2011-2016. All Rights Reserved. + * Copyright Ericsson AB 2011-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/erl_thr_queue.c b/erts/emulator/beam/erl_thr_queue.c index 548c2768e5..aab7c199d2 100644 --- a/erts/emulator/beam/erl_thr_queue.c +++ b/erts/emulator/beam/erl_thr_queue.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2011-2016. All Rights Reserved. + * Copyright Ericsson AB 2011-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/erl_thr_queue.h b/erts/emulator/beam/erl_thr_queue.h index 163a25318d..29b58063ac 100644 --- a/erts/emulator/beam/erl_thr_queue.h +++ b/erts/emulator/beam/erl_thr_queue.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2011-2016. All Rights Reserved. + * Copyright Ericsson AB 2011-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/erl_trace.c b/erts/emulator/beam/erl_trace.c index f4161b14f2..53a020e7a5 100644 --- a/erts/emulator/beam/erl_trace.c +++ b/erts/emulator/beam/erl_trace.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1999-2017. All Rights Reserved. + * Copyright Ericsson AB 1999-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/erl_trace.h b/erts/emulator/beam/erl_trace.h index 3228e19809..bccf31606e 100644 --- a/erts/emulator/beam/erl_trace.h +++ b/erts/emulator/beam/erl_trace.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2012-2016. All Rights Reserved. + * Copyright Ericsson AB 2012-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/erl_utils.h b/erts/emulator/beam/erl_utils.h index e4087e0ac8..b3bfa69052 100644 --- a/erts/emulator/beam/erl_utils.h +++ b/erts/emulator/beam/erl_utils.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2012-2017. All Rights Reserved. + * Copyright Ericsson AB 2012-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/erl_vm.h b/erts/emulator/beam/erl_vm.h index f8391fb665..4089fac48e 100644 --- a/erts/emulator/beam/erl_vm.h +++ b/erts/emulator/beam/erl_vm.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1996-2017. All Rights Reserved. + * Copyright Ericsson AB 1996-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/erlang_dtrace.d b/erts/emulator/beam/erlang_dtrace.d index c47a37eb62..8792138d53 100644 --- a/erts/emulator/beam/erlang_dtrace.d +++ b/erts/emulator/beam/erlang_dtrace.d @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Dustin Sallings, Michal Ptaszek, Scott Lystig Fritchie 2011-2016. + * Copyright Dustin Sallings, Michal Ptaszek, Scott Lystig Fritchie 2011-2018. * All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/erts/emulator/beam/erlang_lttng.h b/erts/emulator/beam/erlang_lttng.h index feb05f4f4c..9b93d77f6e 100644 --- a/erts/emulator/beam/erlang_lttng.h +++ b/erts/emulator/beam/erlang_lttng.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1996-2016. All Rights Reserved. + * Copyright Ericsson AB 1996-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/export.h b/erts/emulator/beam/export.h index 194e514b12..ae8dfa4cf8 100644 --- a/erts/emulator/beam/export.h +++ b/erts/emulator/beam/export.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1996-2016. All Rights Reserved. + * Copyright Ericsson AB 1996-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/external.c b/erts/emulator/beam/external.c index fb42969a8f..904993ceb6 100644 --- a/erts/emulator/beam/external.c +++ b/erts/emulator/beam/external.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1996-2017. All Rights Reserved. + * Copyright Ericsson AB 1996-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/hash.c b/erts/emulator/beam/hash.c index 6a31489473..8954dbb06c 100644 --- a/erts/emulator/beam/hash.c +++ b/erts/emulator/beam/hash.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1996-2016. All Rights Reserved. + * Copyright Ericsson AB 1996-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/instrs.tab b/erts/emulator/beam/instrs.tab index 6a531fcc09..42c1168f85 100644 --- a/erts/emulator/beam/instrs.tab +++ b/erts/emulator/beam/instrs.tab @@ -2,7 +2,7 @@ // // %CopyrightBegin% // -// Copyright Ericsson AB 2017. All Rights Reserved. +// Copyright Ericsson AB 2017-2018. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/io.c b/erts/emulator/beam/io.c index 2446b3c074..133ab485d9 100644 --- a/erts/emulator/beam/io.c +++ b/erts/emulator/beam/io.c @@ -3287,7 +3287,6 @@ static void deliver_read_message(Port* prt, erts_aint32_t state, Eterm to, if (trace_send) trace_port_send(prt, to, tuple, 1); - ERL_MESSAGE_TOKEN(mp) = am_undefined; erts_queue_message(rp, rp_locks, mp, tuple, prt->common.id); if (rp_locks) erts_proc_unlock(rp, rp_locks); @@ -3459,7 +3458,6 @@ deliver_vec_message(Port* prt, /* Port */ if (IS_TRACED_FL(prt, F_TRACE_SEND)) trace_port_send(prt, to, tuple, 1); - ERL_MESSAGE_TOKEN(mp) = am_undefined; erts_queue_message(rp, rp_locks, mp, tuple, prt->common.id); erts_proc_unlock(rp, rp_locks); if (!scheduler) @@ -5382,7 +5380,6 @@ void driver_report_exit(ErlDrvPort ix, int status) if (IS_TRACED_FL(prt, F_TRACE_SEND)) trace_port_send(prt, pid, tuple, 1); - ERL_MESSAGE_TOKEN(mp) = am_undefined; erts_queue_message(rp, rp_locks, mp, tuple, prt->common.id); erts_proc_unlock(rp, rp_locks); @@ -5988,8 +5985,6 @@ driver_deliver_term(Port *prt, Eterm to, ErlDrvTermData* data, int len) from = prt->common.id; } - /* send message */ - ERL_MESSAGE_TOKEN(factory.message) = am_undefined; erts_queue_message(rp, rp_locks, factory.message, mess, from); } else if (res == -2) { diff --git a/erts/emulator/beam/lttng-wrapper.h b/erts/emulator/beam/lttng-wrapper.h index e7f2971bf7..ad4852374a 100644 --- a/erts/emulator/beam/lttng-wrapper.h +++ b/erts/emulator/beam/lttng-wrapper.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1996-2016. All Rights Reserved. + * Copyright Ericsson AB 1996-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/module.c b/erts/emulator/beam/module.c index 1712dc803c..0642a06123 100644 --- a/erts/emulator/beam/module.c +++ b/erts/emulator/beam/module.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1996-2016. All Rights Reserved. + * Copyright Ericsson AB 1996-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/module.h b/erts/emulator/beam/module.h index a3f1ce1705..00efd129ff 100644 --- a/erts/emulator/beam/module.h +++ b/erts/emulator/beam/module.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1996-2016. All Rights Reserved. + * Copyright Ericsson AB 1996-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/ops.tab b/erts/emulator/beam/ops.tab index 8b2d9098a8..c51e4ef784 100644 --- a/erts/emulator/beam/ops.tab +++ b/erts/emulator/beam/ops.tab @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 1997-2017. All Rights Reserved. +# Copyright Ericsson AB 1997-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -1421,16 +1421,13 @@ get_map_elements Fail Src Size Rest=* | map_key_sort(Size, Rest) => \ i_get_map_elements f? s I -i_get_map_element Fail Src=xy Key=y Dst => \ - move Key x | i_get_map_element Fail Src x Dst - i_get_map_element_hash Fail Src=c Key Hash Dst => \ move Src x | i_get_map_element_hash Fail x Key Hash Dst i_get_map_element_hash f? xy c I xy i_get_map_element Fail Src=c Key Dst => \ move Src x | i_get_map_element Fail x Key Dst -i_get_map_element f? xy x xy +i_get_map_element f? xy xy xy # # Convert the plus operations to a generic plus instruction. diff --git a/erts/emulator/beam/packet_parser.c b/erts/emulator/beam/packet_parser.c index de1d481105..4b526887b5 100644 --- a/erts/emulator/beam/packet_parser.c +++ b/erts/emulator/beam/packet_parser.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2008-2016. All Rights Reserved. + * Copyright Ericsson AB 2008-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/register.c b/erts/emulator/beam/register.c index 92a0854ad3..c7e02c6d48 100644 --- a/erts/emulator/beam/register.c +++ b/erts/emulator/beam/register.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1996-2016. All Rights Reserved. + * Copyright Ericsson AB 1996-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/safe_hash.c b/erts/emulator/beam/safe_hash.c index 73306030ae..0d816a81a9 100644 --- a/erts/emulator/beam/safe_hash.c +++ b/erts/emulator/beam/safe_hash.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2008-2016. All Rights Reserved. + * Copyright Ericsson AB 2008-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/safe_hash.h b/erts/emulator/beam/safe_hash.h index af97b4cb4d..bd81e3022b 100644 --- a/erts/emulator/beam/safe_hash.h +++ b/erts/emulator/beam/safe_hash.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2008-2016. All Rights Reserved. + * Copyright Ericsson AB 2008-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/sys.h b/erts/emulator/beam/sys.h index be6ab57eeb..bb22548587 100644 --- a/erts/emulator/beam/sys.h +++ b/erts/emulator/beam/sys.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1996-2017. All Rights Reserved. + * Copyright Ericsson AB 1996-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/beam/utils.c b/erts/emulator/beam/utils.c index 87709edc15..996757ef43 100644 --- a/erts/emulator/beam/utils.c +++ b/erts/emulator/beam/utils.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1996-2017. All Rights Reserved. + * Copyright Ericsson AB 1996-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -1569,7 +1569,7 @@ make_hash2(Eterm term) * MUST BE USED AS INPUT FOR THE HASH. Two different terms must always have a * chance of hashing different when salted: hash([Salt|A]) vs hash([Salt|B]). * - * This is why we can not use cached hash values for atoms for example. + * This is why we cannot use cached hash values for atoms for example. * */ @@ -1946,7 +1946,7 @@ do_allocate_logger_message(Eterm gleader, ErtsMonotonicTime *ts, Eterm *pid, else sz += MAP4_SZ /* metadata map w gl w pid*/; - *ts = ERTS_MONOTONIC_TO_USEC(erts_get_monotonic_time(NULL)) + ERTS_MONOTONIC_OFFSET_USEC; + *ts = ERTS_MONOTONIC_TO_USEC(erts_get_monotonic_time(NULL) + erts_get_time_offset()); erts_bld_sint64(NULL, &sz, *ts); *bp = new_message_buffer(sz); @@ -3122,7 +3122,7 @@ tailrecur_ne: ASSERT(alen == blen); for (i = (Sint) alen - 1; i >= 0; i--) if (anum[i] != bnum[i]) - RETURN_NEQ((Sint32) (anum[i] - bnum[i])); + RETURN_NEQ(anum[i] < bnum[i] ? -1 : 1); goto pop_next; case (_TAG_HEADER_EXTERNAL_REF >> _TAG_PRIMARY_SIZE): if (is_internal_ref(b)) { diff --git a/erts/emulator/drivers/common/gzio.h b/erts/emulator/drivers/common/gzio.h index e331b5208b..20433a1a17 100644 --- a/erts/emulator/drivers/common/gzio.h +++ b/erts/emulator/drivers/common/gzio.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1999-2016. All Rights Reserved. + * Copyright Ericsson AB 1999-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/drivers/common/inet_drv.c b/erts/emulator/drivers/common/inet_drv.c index 1a68f65b52..91381bd60d 100644 --- a/erts/emulator/drivers/common/inet_drv.c +++ b/erts/emulator/drivers/common/inet_drv.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1997-2017. All Rights Reserved. + * Copyright Ericsson AB 1997-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -1034,6 +1034,7 @@ typedef struct { inet_async_op* oph; /* queue head or NULL */ inet_async_op* opt; /* queue tail or NULL */ inet_async_op op_queue[INET_MAX_ASYNC]; /* call queue */ + int op_ref; /* queue reference generator */ int active; /* 0 = passive, 1 = active, 2 = active once */ Sint16 active_count; /* counter for {active,N} */ @@ -1299,8 +1300,7 @@ static int packet_inet_input(udp_descriptor* udesc, HANDLE event); /* convert descriptor pointer to inet_descriptor pointer */ #define INETP(d) (&(d)->inet) -static int async_ref = 0; /* async reference id generator */ -#define NEW_ASYNC_ID() ((async_ref++) & 0xffff) +#define NEW_ASYNC_ID(desc) ((desc)->op_ref++ & 0xffff) /* check for transition from active to passive */ #define INET_CHECK_ACTIVE_TO_PASSIVE(inet) \ @@ -1549,6 +1549,8 @@ static void *realloc_wrapper(void *current, ErlDrvSizeT size){ # define LOAD_ASSOC_ID LOAD_UINT # define LOAD_ASSOC_ID_CNT LOAD_UINT_CNT # define SCTP_ANC_BUFF_SIZE INET_DEF_BUFFER/2 /* XXX: not very good... */ +#else +# define IS_SCTP(desc) 0 #endif #ifdef HAVE_UDP @@ -1785,6 +1787,7 @@ static void release_buffer(ErlDrvBinary* buf) #ifdef HAVE_UDP static ErlDrvBinary* realloc_buffer(ErlDrvBinary* buf, ErlDrvSizeT newsz) { + DEBUGF(("realloc_buffer: %ld -> %ld\r\n", (buf==NULL) ? 0 : buf->orig_size, newsz)); return driver_realloc_binary(buf, newsz); } #endif @@ -1955,7 +1958,7 @@ static void enq_multi_op(tcp_descriptor *desc, char *buf, int req, ErlDrvTermData caller, MultiTimerData *timeout, ErlDrvMonitor *monitorp) { - int id = NEW_ASYNC_ID(); + int id = NEW_ASYNC_ID(INETP(desc)); enq_old_multi_op(desc,id,req,caller,timeout,monitorp); if (buf != NULL) put_int16(id, buf); @@ -2024,7 +2027,7 @@ static int remove_multi_op(tcp_descriptor *desc, int *id_p, int *req_p, static int enq_async_w_tmo(inet_descriptor* desc, char* buf, int req, unsigned timeout, ErlDrvMonitor *monitorp) { - int id = NEW_ASYNC_ID(); + int id = NEW_ASYNC_ID(desc); inet_async_op* opp; if ((opp = desc->oph) == NULL) /* queue empty */ @@ -4390,7 +4393,7 @@ static void desc_close_read(inet_descriptor* desc) { if (desc->s != INVALID_SOCKET) { #ifdef __WIN32__ - /* This call can not be right??? + /* This call cannot be right??? * We want to turn off read events but keep any write events. * But on windows driver_select(...,READ,1) is only used as a * way to hook into the pollset. sock_select is used to control @@ -6435,7 +6438,12 @@ static int inet_set_opts(inet_descriptor* desc, char* ptr, int len) (long)desc->port, desc->s, res)); if (type == SO_RCVBUF) { /* make sure we have desc->bufsz >= SO_RCVBUF */ - if (ival > desc->bufsz) + if (ival > (1 << 16) && desc->stype == SOCK_DGRAM && !IS_SCTP(desc)) + /* For UDP we don't want to automatically + set the buffer size to be larger than + the theoretical max MTU */ + desc->bufsz = 1 << 16; + else if (ival > desc->bufsz) desc->bufsz = ival; } } @@ -8461,6 +8469,7 @@ static ErlDrvData inet_start(ErlDrvPort port, int size, int protocol) desc->delimiter = '\n'; /* line delimiting char */ desc->oph = NULL; desc->opt = NULL; + desc->op_ref = 0; desc->peer_ptr = NULL; desc->name_ptr = NULL; @@ -12041,15 +12050,11 @@ static int packet_inet_input(udp_descriptor* udesc, HANDLE event) sys_memzero((char *) &other, sizeof(other)); /* udesc->i_buf is only kept between SCTP fragments */ - if (udesc->i_buf == NULL) { - udesc->i_bufsz = desc->bufsz + len; - if ((udesc->i_buf = alloc_buffer(udesc->i_bufsz)) == NULL) - return packet_error(udesc, ENOMEM); - /* pointer to message start */ - udesc->i_ptr = udesc->i_buf->orig_bytes + len; - } else { - ErlDrvBinary* tmp; +#ifdef HAVE_SCTP + if (udesc->i_buf != NULL) { + ErlDrvBinary* tmp; int bufsz; + ASSERT(IS_SCTP(desc)); bufsz = desc->bufsz + (udesc->i_ptr - udesc->i_buf->orig_bytes); if ((tmp = realloc_buffer(udesc->i_buf, bufsz)) == NULL) { release_buffer(udesc->i_buf); @@ -12061,6 +12066,15 @@ static int packet_inet_input(udp_descriptor* udesc, HANDLE event) udesc->i_buf = tmp; udesc->i_bufsz = bufsz; } + } else +#endif + { + ASSERT(udesc->i_buf == NULL); + udesc->i_bufsz = desc->bufsz + len; + if ((udesc->i_buf = alloc_buffer(udesc->i_bufsz)) == NULL) + return packet_error(udesc, ENOMEM); + /* pointer to message start */ + udesc->i_ptr = udesc->i_buf->orig_bytes + len; } /* Note: On Windows NT, recvfrom() fails if the socket is connected. */ @@ -12119,6 +12133,14 @@ static int packet_inet_input(udp_descriptor* udesc, HANDLE event) ) { sock_select(desc,FD_READ,1); } +#ifdef HAVE_SCTP + if (!short_recv) { +#endif + release_buffer(udesc->i_buf); + udesc->i_buf = NULL; +#ifdef HAVE_SCTP + } +#endif return count; /* strange, not ready */ } diff --git a/erts/emulator/drivers/unix/ttsl_drv.c b/erts/emulator/drivers/unix/ttsl_drv.c index 7355df6059..11bb4373d8 100644 --- a/erts/emulator/drivers/unix/ttsl_drv.c +++ b/erts/emulator/drivers/unix/ttsl_drv.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1996-2016. All Rights Reserved. + * Copyright Ericsson AB 1996-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -31,7 +31,7 @@ static int ttysl_init(void); static ErlDrvData ttysl_start(ErlDrvPort, char*); -#ifdef HAVE_TERMCAP /* else make an empty driver that can not be opened */ +#ifdef HAVE_TERMCAP /* else make an empty driver that cannot be opened */ #ifndef WANT_NONBLOCKING #define WANT_NONBLOCKING diff --git a/erts/emulator/hipe/hipe_amd64.c b/erts/emulator/hipe/hipe_amd64.c index f23f341e6d..71cbb7c060 100644 --- a/erts/emulator/hipe/hipe_amd64.c +++ b/erts/emulator/hipe/hipe_amd64.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2004-2016. All Rights Reserved. + * Copyright Ericsson AB 2004-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/hipe/hipe_amd64_bifs.m4 b/erts/emulator/hipe/hipe_amd64_bifs.m4 index cf4c59c9af..4c866b3b68 100644 --- a/erts/emulator/hipe/hipe_amd64_bifs.m4 +++ b/erts/emulator/hipe/hipe_amd64_bifs.m4 @@ -2,7 +2,7 @@ changecom(`/*', `*/')dnl /* * %CopyrightBegin% * - * Copyright Ericsson AB 2004-2016. All Rights Reserved. + * Copyright Ericsson AB 2004-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/hipe/hipe_arm_bifs.m4 b/erts/emulator/hipe/hipe_arm_bifs.m4 index 554faa2567..421915dd72 100644 --- a/erts/emulator/hipe/hipe_arm_bifs.m4 +++ b/erts/emulator/hipe/hipe_arm_bifs.m4 @@ -2,7 +2,7 @@ changecom(`/*', `*/')dnl /* * %CopyrightBegin% * - * Copyright Ericsson AB 2005-2016. All Rights Reserved. + * Copyright Ericsson AB 2005-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/hipe/hipe_bif2.c b/erts/emulator/hipe/hipe_bif2.c index df377b2153..7e04f7d9c0 100644 --- a/erts/emulator/hipe/hipe_bif2.c +++ b/erts/emulator/hipe/hipe_bif2.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2001-2016. All Rights Reserved. + * Copyright Ericsson AB 2001-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/hipe/hipe_bif2.tab b/erts/emulator/hipe/hipe_bif2.tab index c4da44606a..8925291769 100644 --- a/erts/emulator/hipe/hipe_bif2.tab +++ b/erts/emulator/hipe/hipe_bif2.tab @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 2001-2016. All Rights Reserved. +# Copyright Ericsson AB 2001-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. diff --git a/erts/emulator/hipe/hipe_bif_list.m4 b/erts/emulator/hipe/hipe_bif_list.m4 index 33b3cc1ee5..7468860c37 100644 --- a/erts/emulator/hipe/hipe_bif_list.m4 +++ b/erts/emulator/hipe/hipe_bif_list.m4 @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2004-2016. All Rights Reserved. + * Copyright Ericsson AB 2004-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/hipe/hipe_debug.c b/erts/emulator/hipe/hipe_debug.c index 929b2a9432..138e4f7da3 100644 --- a/erts/emulator/hipe/hipe_debug.c +++ b/erts/emulator/hipe/hipe_debug.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2001-2016. All Rights Reserved. + * Copyright Ericsson AB 2001-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/hipe/hipe_gc.c b/erts/emulator/hipe/hipe_gc.c index aaedba1afd..7bd1de0117 100644 --- a/erts/emulator/hipe/hipe_gc.c +++ b/erts/emulator/hipe/hipe_gc.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2004-2017. All Rights Reserved. + * Copyright Ericsson AB 2004-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/hipe/hipe_mode_switch.c b/erts/emulator/hipe/hipe_mode_switch.c index 0a65e317ed..052cf9c263 100644 --- a/erts/emulator/hipe/hipe_mode_switch.c +++ b/erts/emulator/hipe/hipe_mode_switch.c @@ -2,7 +2,7 @@ * %CopyrightBegin% * - * Copyright Ericsson AB 2001-2017. All Rights Reserved. + * Copyright Ericsson AB 2001-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/hipe/hipe_native_bif.h b/erts/emulator/hipe/hipe_native_bif.h index ba42b126be..ce96778dea 100644 --- a/erts/emulator/hipe/hipe_native_bif.h +++ b/erts/emulator/hipe/hipe_native_bif.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2001-2016. All Rights Reserved. + * Copyright Ericsson AB 2001-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/hipe/hipe_ops.tab b/erts/emulator/hipe/hipe_ops.tab index 19a3820a6a..8dd81558f2 100644 --- a/erts/emulator/hipe/hipe_ops.tab +++ b/erts/emulator/hipe/hipe_ops.tab @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 2001-2016. All Rights Reserved. +# Copyright Ericsson AB 2001-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. diff --git a/erts/emulator/hipe/hipe_ppc_bifs.m4 b/erts/emulator/hipe/hipe_ppc_bifs.m4 index 283fbbb200..2ced443ce4 100644 --- a/erts/emulator/hipe/hipe_ppc_bifs.m4 +++ b/erts/emulator/hipe/hipe_ppc_bifs.m4 @@ -2,7 +2,7 @@ changecom(`/*', `*/')dnl /* * %CopyrightBegin% * - * Copyright Ericsson AB 2004-2016. All Rights Reserved. + * Copyright Ericsson AB 2004-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/hipe/hipe_primops.h b/erts/emulator/hipe/hipe_primops.h index c5f10672f3..f4a4b4a07c 100644 --- a/erts/emulator/hipe/hipe_primops.h +++ b/erts/emulator/hipe/hipe_primops.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2005-2016. All Rights Reserved. + * Copyright Ericsson AB 2005-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/hipe/hipe_process.h b/erts/emulator/hipe/hipe_process.h index 18354ba0a6..d412535968 100644 --- a/erts/emulator/hipe/hipe_process.h +++ b/erts/emulator/hipe/hipe_process.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2001-2017. All Rights Reserved. + * Copyright Ericsson AB 2001-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/hipe/hipe_risc_stack.c b/erts/emulator/hipe/hipe_risc_stack.c index bb93a918a2..b64afb1ba5 100644 --- a/erts/emulator/hipe/hipe_risc_stack.c +++ b/erts/emulator/hipe/hipe_risc_stack.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2008-2016. All Rights Reserved. + * Copyright Ericsson AB 2008-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/hipe/hipe_signal.h b/erts/emulator/hipe/hipe_signal.h index 524def11a4..14bc0ef360 100644 --- a/erts/emulator/hipe/hipe_signal.h +++ b/erts/emulator/hipe/hipe_signal.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2002-2016. All Rights Reserved. + * Copyright Ericsson AB 2002-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/hipe/hipe_sparc_bifs.m4 b/erts/emulator/hipe/hipe_sparc_bifs.m4 index 1b49fa57fd..54684fbfb9 100644 --- a/erts/emulator/hipe/hipe_sparc_bifs.m4 +++ b/erts/emulator/hipe/hipe_sparc_bifs.m4 @@ -2,7 +2,7 @@ changecom(`/*', `*/')dnl /* * %CopyrightBegin% * - * Copyright Ericsson AB 2001-2016. All Rights Reserved. + * Copyright Ericsson AB 2001-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/hipe/hipe_x86_bifs.m4 b/erts/emulator/hipe/hipe_x86_bifs.m4 index 9cb343d067..0f6b9a8c8a 100644 --- a/erts/emulator/hipe/hipe_x86_bifs.m4 +++ b/erts/emulator/hipe/hipe_x86_bifs.m4 @@ -2,7 +2,7 @@ changecom(`/*', `*/')dnl /* * %CopyrightBegin% * - * Copyright Ericsson AB 2001-2016. All Rights Reserved. + * Copyright Ericsson AB 2001-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/hipe/hipe_x86_stack.c b/erts/emulator/hipe/hipe_x86_stack.c index 615e07917a..8cfc541f0d 100644 --- a/erts/emulator/hipe/hipe_x86_stack.c +++ b/erts/emulator/hipe/hipe_x86_stack.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2001-2016. All Rights Reserved. + * Copyright Ericsson AB 2001-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/internal_doc/CarrierMigration.md b/erts/emulator/internal_doc/CarrierMigration.md index 3a796d11b7..bb3d8aac28 100644 --- a/erts/emulator/internal_doc/CarrierMigration.md +++ b/erts/emulator/internal_doc/CarrierMigration.md @@ -34,8 +34,7 @@ Solution -------- In order to prevent scenarios like this we've implemented support for -migration of multi-block carriers between allocator instances of the -same type. +migration of multi-block carriers between allocator instances. ### Management of Free Blocks ### @@ -130,10 +129,6 @@ threads may have references to it via the pool. ### Migration ### -There exists one pool for each allocator type enabling migration of -carriers between scheduler specific allocator instances of the same -allocator type. - Each allocator instance keeps track of the current utilization of its multi-block carriers. When the total utilization falls below the "abandon carrier utilization limit" it starts to inspect the utilization of the @@ -208,8 +203,8 @@ limited. We only inspect a limited number of carriers. If none of those carriers had a free block large enough to satisfy the allocation request, the search will fail. A carrier in the pool can also be BUSY if another thread is currently doing block deallocation work on the -carrier. A BUSY carrier will also be skipped by the search as it can -not satisfy the request. The pool is lock-free and we do not want to +carrier. A BUSY carrier will also be skipped by the search as it cannot +satisfy the request. The pool is lock-free and we do not want to block, waiting for the other thread to finish. ### The bad cluster problem ### @@ -287,11 +282,3 @@ reduced using the `aoffcbf` strategy. A trade off between memory consumption and performance is however inevitable, and it is up to the user to decide what is most important. -Further work ------------- - -It would be quite easy to extend this to allow migration of multi-block -carriers between all allocator types. More or less the only obstacle -is maintenance of the statistics information. - - diff --git a/erts/emulator/nifs/common/prim_file_nif.c b/erts/emulator/nifs/common/prim_file_nif.c index bbd9becb47..a05d50b333 100644 --- a/erts/emulator/nifs/common/prim_file_nif.c +++ b/erts/emulator/nifs/common/prim_file_nif.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson 2017. All Rights Reserved. + * Copyright Ericsson 2017-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/nifs/common/prim_file_nif.h b/erts/emulator/nifs/common/prim_file_nif.h index 4194cdc7d9..099c06c48c 100644 --- a/erts/emulator/nifs/common/prim_file_nif.h +++ b/erts/emulator/nifs/common/prim_file_nif.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson 2017. All Rights Reserved. + * Copyright Ericsson 2017-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/nifs/unix/unix_prim_file.c b/erts/emulator/nifs/unix/unix_prim_file.c index 2b112dda76..dea73db18a 100644 --- a/erts/emulator/nifs/unix/unix_prim_file.c +++ b/erts/emulator/nifs/unix/unix_prim_file.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson 2017. All Rights Reserved. + * Copyright Ericsson 2017-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/nifs/win32/win_prim_file.c b/erts/emulator/nifs/win32/win_prim_file.c index 044bee62cf..f7fae3c637 100644 --- a/erts/emulator/nifs/win32/win_prim_file.c +++ b/erts/emulator/nifs/win32/win_prim_file.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson 2017. All Rights Reserved. + * Copyright Ericsson 2017-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/sys/common/erl_check_io.c b/erts/emulator/sys/common/erl_check_io.c index 3e77dce1cd..9f115706dc 100644 --- a/erts/emulator/sys/common/erl_check_io.c +++ b/erts/emulator/sys/common/erl_check_io.c @@ -1493,7 +1493,6 @@ send_event_tuple(struct erts_nif_select_event* e, ErtsResource* resource, } tuple = TUPLE4(hp, am_select, resource_term, ref_term, event_atom); - ERL_MESSAGE_TOKEN(mp) = am_undefined; erts_queue_message(rp, rp_locks, mp, tuple, am_system); if (rp_locks) diff --git a/erts/emulator/sys/common/erl_mmap.c b/erts/emulator/sys/common/erl_mmap.c index 145503bea7..b0d9fc0776 100644 --- a/erts/emulator/sys/common/erl_mmap.c +++ b/erts/emulator/sys/common/erl_mmap.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2002-2016. All Rights Reserved. + * Copyright Ericsson AB 2002-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/sys/common/erl_mmap.h b/erts/emulator/sys/common/erl_mmap.h index c1f9668b18..539daea419 100644 --- a/erts/emulator/sys/common/erl_mmap.h +++ b/erts/emulator/sys/common/erl_mmap.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2013-2016. All Rights Reserved. + * Copyright Ericsson AB 2013-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/sys/common/erl_mseg.c b/erts/emulator/sys/common/erl_mseg.c index ced3d61525..030e5b00a7 100644 --- a/erts/emulator/sys/common/erl_mseg.c +++ b/erts/emulator/sys/common/erl_mseg.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2002-2017. All Rights Reserved. + * Copyright Ericsson AB 2002-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/sys/common/erl_mseg.h b/erts/emulator/sys/common/erl_mseg.h index af275c18be..ea9060ddac 100644 --- a/erts/emulator/sys/common/erl_mseg.h +++ b/erts/emulator/sys/common/erl_mseg.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2002-2016. All Rights Reserved. + * Copyright Ericsson AB 2002-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/sys/common/erl_os_monotonic_time_extender.c b/erts/emulator/sys/common/erl_os_monotonic_time_extender.c index 341845cc2a..5844e7eeb7 100644 --- a/erts/emulator/sys/common/erl_os_monotonic_time_extender.c +++ b/erts/emulator/sys/common/erl_os_monotonic_time_extender.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2015-2016. All Rights Reserved. + * Copyright Ericsson AB 2015-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/sys/common/erl_os_monotonic_time_extender.h b/erts/emulator/sys/common/erl_os_monotonic_time_extender.h index 53c32579d5..f6659fe973 100644 --- a/erts/emulator/sys/common/erl_os_monotonic_time_extender.h +++ b/erts/emulator/sys/common/erl_os_monotonic_time_extender.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2015. All Rights Reserved. + * Copyright Ericsson AB 2015-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/sys/common/erl_osenv.c b/erts/emulator/sys/common/erl_osenv.c index 9f54d1dff0..487ff87116 100644 --- a/erts/emulator/sys/common/erl_osenv.c +++ b/erts/emulator/sys/common/erl_osenv.c @@ -75,7 +75,15 @@ static int compare_env_keys(const erts_osenv_data_t a, const erts_osenv_data_t b #include "erl_rbtree.h" static int compare_env_keys(const erts_osenv_data_t a, const erts_osenv_data_t b) { - int relation = sys_memcmp(a.data, b.data, MIN(a.length, b.length)); + int relation; + +#ifdef __WIN32__ + /* Environment variables are case-insensitive on Windows. */ + relation = _wcsnicmp((const WCHAR*)a.data, (const WCHAR*)b.data, + MIN(a.length, b.length) / sizeof(WCHAR)); +#else + relation = sys_memcmp(a.data, b.data, MIN(a.length, b.length)); +#endif if(relation != 0) { return relation; diff --git a/erts/emulator/sys/common/erl_poll.c b/erts/emulator/sys/common/erl_poll.c index ced8a4a2a7..b4d1575ee5 100644 --- a/erts/emulator/sys/common/erl_poll.c +++ b/erts/emulator/sys/common/erl_poll.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2006-2016. All Rights Reserved. + * Copyright Ericsson AB 2006-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -141,12 +141,26 @@ typedef struct { size_t sz; fd_set* ptr; }ERTS_fd_set; -# define ERTS_FD_CLR(fd, fds) FD_CLR((fd), (fds)->ptr) -# define ERTS_FD_SET(fd, fds) FD_SET((fd), (fds)->ptr) -# define ERTS_FD_ISSET(fd,fds) FD_ISSET((fd), (fds)->ptr) + # define ERTS_FD_ZERO(fds) memset((fds)->ptr, 0, (fds)->sz) # define ERTS_FD_SIZE(n) ((((n)+NFDBITS-1)/NFDBITS)*sizeof(fd_mask)) +static ERTS_INLINE void ERTS_FD_CLR(int fd, ERTS_fd_set *fds) +{ + ASSERT(ERTS_FD_SIZE(fd+1) <= fds->sz); + FD_CLR(fd, fds->ptr); +} +static ERTS_INLINE void ERTS_FD_SET(int fd, ERTS_fd_set *fds) +{ + ASSERT(ERTS_FD_SIZE(fd+1) <= fds->sz); + FD_SET(fd, fds->ptr); +} +static ERTS_INLINE int ERTS_FD_ISSET(int fd, ERTS_fd_set *fds) +{ + ASSERT(ERTS_FD_SIZE(fd+1) <= fds->sz); + return FD_ISSET(fd, fds->ptr); +} + static void ERTS_FD_COPY(ERTS_fd_set *src, ERTS_fd_set *dst) { if (dst->sz != src->sz) { @@ -626,8 +640,15 @@ ensure_select_fds(int fd, ERTS_fd_set* in, ERTS_fd_set* out) grow_select_fds(fd, out); } } +static ERTS_INLINE int +check_select_fds(int fd, ERTS_fd_set* in, ERTS_fd_set* out) +{ + ASSERT(in->sz == out->sz); + return (ERTS_FD_SIZE(fd+1) <= in->sz); +} #else # define ensure_select_fds(fd, in, out) do {} while(0) +# define check_select_fds(fd, in, out) (1) #endif /* _DARWIN_UNLIMITED_SELECT */ #if !ERTS_POLL_USE_CONCURRENT_UPDATE @@ -782,6 +803,23 @@ update_pollset(ErtsPollSet *ps, int fd, ErtsPollOp op, ErtsPollEvents events) struct kevent evts[2]; struct timespec ts = {0, 0}; + if (op == ERTS_POLL_OP_ADD) { + /* This is a hack to make the "noshell" option work; kqueue can poll + * these fds but will not report EV_EOF, so we return NVAL to use the + * fallback instead. + * + * This may be common to all pipes but we have no way to tell whether + * an fd is a pipe or not. */ + switch (fd) { + case STDIN_FILENO: + case STDOUT_FILENO: + case STDERR_FILENO: + return ERTS_POLL_EV_NVAL; + default: + break; + } + } + #if defined(EV_DISPATCH) && !defined(__OpenBSD__) /* If we have EV_DISPATCH we use it, unless we are on OpenBSD as the behavior of EV_EOF seems to be edge triggered there and we need it @@ -1085,8 +1123,10 @@ static int update_pollset(ErtsPollSet *ps, ErtsPollResFd pr[], int fd) res++; } - ERTS_FD_CLR(fd, &ps->input_fds); - ERTS_FD_CLR(fd, &ps->output_fds); + if (check_select_fds(fd, &ps->input_fds, &ps->output_fds)) { + ERTS_FD_CLR(fd, &ps->input_fds); + ERTS_FD_CLR(fd, &ps->output_fds); + } if (ps->fds_status[fd].used_events) { erts_atomic_dec_nob(&ps->no_of_user_fds); diff --git a/erts/emulator/sys/common/erl_poll.h b/erts/emulator/sys/common/erl_poll.h index e9a667cac1..e1cea7eb8b 100644 --- a/erts/emulator/sys/common/erl_poll.h +++ b/erts/emulator/sys/common/erl_poll.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2006-2016. All Rights Reserved. + * Copyright Ericsson AB 2006-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/sys/common/erl_poll_api.h b/erts/emulator/sys/common/erl_poll_api.h index 04beb37d1c..1170a549b9 100644 --- a/erts/emulator/sys/common/erl_poll_api.h +++ b/erts/emulator/sys/common/erl_poll_api.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2006-2016. All Rights Reserved. + * Copyright Ericsson AB 2006-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/sys/common/erl_sys_common_misc.c b/erts/emulator/sys/common/erl_sys_common_misc.c index 41a6fcb7e1..2541ab5d31 100644 --- a/erts/emulator/sys/common/erl_sys_common_misc.c +++ b/erts/emulator/sys/common/erl_sys_common_misc.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2006-2016. All Rights Reserved. + * Copyright Ericsson AB 2006-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/sys/unix/erl_child_setup.c b/erts/emulator/sys/unix/erl_child_setup.c index 10601529a4..221ee2a69d 100644 --- a/erts/emulator/sys/unix/erl_child_setup.c +++ b/erts/emulator/sys/unix/erl_child_setup.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2002-2016. All Rights Reserved. + * Copyright Ericsson AB 2002-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/sys/unix/erl_unix_sys.h b/erts/emulator/sys/unix/erl_unix_sys.h index 5bfe5a8e2d..ae7a3ea23e 100644 --- a/erts/emulator/sys/unix/erl_unix_sys.h +++ b/erts/emulator/sys/unix/erl_unix_sys.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1997-2017. All Rights Reserved. + * Copyright Ericsson AB 1997-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/sys/unix/sys.c b/erts/emulator/sys/unix/sys.c index 189ca083d7..36579ffdb4 100644 --- a/erts/emulator/sys/unix/sys.c +++ b/erts/emulator/sys/unix/sys.c @@ -452,9 +452,9 @@ prepare_crash_dump(int secs) envsz = sizeof(env); i = erts_sys_explicit_8bit_getenv("ERL_CRASH_DUMP_NICE", env, &envsz); - if (i >= 0) { + if (i != 0) { int nice_val; - nice_val = i != 1 ? 0 : atoi(env); + nice_val = (i != 1) ? 0 : atoi(env); if (nice_val > 39) { nice_val = 39; } diff --git a/erts/emulator/sys/unix/sys_drivers.c b/erts/emulator/sys/unix/sys_drivers.c index 117855acf0..d0498f0bd5 100644 --- a/erts/emulator/sys/unix/sys_drivers.c +++ b/erts/emulator/sys/unix/sys_drivers.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1996-2017. All Rights Reserved. + * Copyright Ericsson AB 1996-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -685,7 +685,7 @@ static ErlDrvData spawn_start(ErlDrvPort port_num, char* name, /* we send the request to do the fork */ if ((res = writev(ofd[1], io_vector, iov_len > MAXIOV ? MAXIOV : iov_len)) < 0) { - if (errno == ERRNO_BLOCK) { + if (errno == ERRNO_BLOCK || errno == EINTR) { res = 0; } else { int err = errno; @@ -1257,6 +1257,8 @@ static int port_inp_failure(ErtsSysDriverData *dd, int res) } driver_failure_eof(dd->port_num); } else if (dd->ifd) { + if (dd->alive == -1) + errno = dd->status; erl_drv_init_ack(dd->port_num, ERL_DRV_ERROR_ERRNO); } else { driver_failure_posix(dd->port_num, err); @@ -1287,10 +1289,10 @@ static void ready_input(ErlDrvData e, ErlDrvEvent ready_fd) int res; if((res = read(ready_fd, &proto, sizeof(proto))) <= 0) { + if (res < 0 && (errno == ERRNO_BLOCK || errno == EINTR)) + return; /* hmm, child setup seems to have closed the pipe too early... we close the port as there is not much else we can do */ - if (res < 0 && errno == ERRNO_BLOCK) - return; driver_select(port_num, ready_fd, ERL_DRV_READ, 0); if (res == 0) errno = EPIPE; @@ -1424,7 +1426,7 @@ static void ready_input(ErlDrvData e, ErlDrvEvent ready_fd) continue; } else { /* The last message we got was split */ - char *buf = erts_alloc_fnf(ERTS_ALC_T_FD_ENTRY_BUF, h); + char *buf = erts_alloc_fnf(ERTS_ALC_T_FD_ENTRY_BUF, h); if (!buf) { errno = ENOMEM; port_inp_failure(dd, -1); @@ -1670,15 +1672,37 @@ static void forker_stop(ErlDrvData e) the port has been closed by the user. */ } +static ErlDrvSizeT forker_deq(ErlDrvPort port_num, ErtsSysForkerProto *proto) +{ + close(proto->u.start.fds[0]); + close(proto->u.start.fds[1]); + if (proto->u.start.fds[1] != proto->u.start.fds[2]) + close(proto->u.start.fds[2]); + + return driver_deq(port_num, sizeof(*proto)); +} + +static void forker_sigchld(Eterm port_id, int error) +{ + ErtsSysForkerProto *proto = erts_alloc(ERTS_ALC_T_DRV_CTRL_DATA, sizeof(*proto)); + proto->action = ErtsSysForkerProtoAction_SigChld; + proto->u.sigchld.error_number = error; + proto->u.sigchld.port_id = port_id; + + /* ideally this would be a port_command call, but as command is + already used by the spawn_driver, we use control instead. + Note that when using erl_drv_port_control it is an asynchronous + control. */ + erl_drv_port_control(port_id, 'S', (char*)proto, sizeof(*proto)); +} + static void forker_ready_input(ErlDrvData e, ErlDrvEvent fd) { int res; - ErtsSysForkerProto *proto; + ErtsSysForkerProto proto; - proto = erts_alloc(ERTS_ALC_T_DRV_CTRL_DATA, sizeof(*proto)); - - if ((res = read(fd, proto, sizeof(*proto))) < 0) { - if (errno == ERRNO_BLOCK) + if ((res = read(fd, &proto, sizeof(proto))) < 0) { + if (errno == ERRNO_BLOCK || errno == EINTR) return; erts_exit(ERTS_DUMP_EXIT, "Failed to read from erl_child_setup: %d\n", errno); } @@ -1686,10 +1710,10 @@ static void forker_ready_input(ErlDrvData e, ErlDrvEvent fd) if (res == 0) erts_exit(ERTS_DUMP_EXIT, "erl_child_setup closed\n"); - ASSERT(res == sizeof(*proto)); + ASSERT(res == sizeof(proto)); #ifdef FORKER_PROTO_START_ACK - if (proto->action == ErtsSysForkerProtoAction_StartAck) { + if (proto.action == ErtsSysForkerProtoAction_StartAck) { /* Ideally we would like to not have to ack each Start command being sent over the uds, but it would seem that some operating systems (only observed on FreeBSD) @@ -1699,28 +1723,15 @@ static void forker_ready_input(ErlDrvData e, ErlDrvEvent fd) ErlDrvPort port_num = (ErlDrvPort)e; int vlen; SysIOVec *iov = driver_peekq(port_num, &vlen); - ErtsSysForkerProto *proto = (ErtsSysForkerProto *)iov[0].iov_base; - - close(proto->u.start.fds[0]); - close(proto->u.start.fds[1]); - if (proto->u.start.fds[1] != proto->u.start.fds[2]) - close(proto->u.start.fds[2]); + ErtsSysForkerProto *qproto = (ErtsSysForkerProto *)iov[0].iov_base; - driver_deq(port_num, sizeof(*proto)); - - if (driver_sizeq(port_num) > 0) + if (forker_deq(port_num, qproto)) driver_select(port_num, forker_fd, ERL_DRV_WRITE|ERL_DRV_USE, 1); } else #endif { - ASSERT(proto->action == ErtsSysForkerProtoAction_SigChld); - - /* ideally this would be a port_command call, but as command is - already used by the spawn_driver, we use control instead. - Note that when using erl_drv_port_control it is an asynchronous - control. */ - erl_drv_port_control(proto->u.sigchld.port_id, 'S', - (char*)proto, sizeof(*proto)); + ASSERT(proto.action == ErtsSysForkerProtoAction_SigChld); + forker_sigchld(proto.u.sigchld.port_id, proto.u.sigchld.error_number); } } @@ -1730,7 +1741,8 @@ static void forker_ready_output(ErlDrvData e, ErlDrvEvent fd) ErlDrvPort port_num = (ErlDrvPort)e; #ifndef FORKER_PROTO_START_ACK - while (driver_sizeq(port_num) > 0) { + int loops = 10; + while (driver_sizeq(port_num) > 0 && --loops) { #endif int vlen; SysIOVec *iov = driver_peekq(port_num, &vlen); @@ -1738,20 +1750,24 @@ static void forker_ready_output(ErlDrvData e, ErlDrvEvent fd) ASSERT(iov[0].iov_len >= (sizeof(*proto))); if (sys_uds_write(forker_fd, (char*)proto, sizeof(*proto), proto->u.start.fds, 3, 0) < 0) { - if (errno == ERRNO_BLOCK) + if (errno == ERRNO_BLOCK || errno == EINTR) { return; - erts_exit(ERTS_DUMP_EXIT, "Failed to write to erl_child_setup: %d\n", errno); + } else if (errno == EMFILE) { + forker_sigchld(proto->u.start.port_id, errno); + if (forker_deq(port_num, proto) == 0) + driver_select(port_num, forker_fd, ERL_DRV_WRITE, 0); + return; + } else { + erts_exit(ERTS_DUMP_EXIT, "Failed to write to erl_child_setup: %d\n", errno); + } } #ifndef FORKER_PROTO_START_ACK - close(proto->u.start.fds[0]); - close(proto->u.start.fds[1]); - if (proto->u.start.fds[1] != proto->u.start.fds[2]) - close(proto->u.start.fds[2]); - driver_deq(port_num, sizeof(*proto)); + if (forker_deq(port_num, proto) == 0) + driver_select(port_num, forker_fd, ERL_DRV_WRITE, 0); } -#endif - +#else driver_select(port_num, forker_fd, ERL_DRV_WRITE, 0); +#endif } static ErlDrvSSizeT forker_control(ErlDrvData e, unsigned int cmd, char *buf, @@ -1777,20 +1793,21 @@ static ErlDrvSSizeT forker_control(ErlDrvData e, unsigned int cmd, char *buf, if ((res = sys_uds_write(forker_fd, (char*)proto, sizeof(*proto), proto->u.start.fds, 3, 0)) < 0) { - if (errno == ERRNO_BLOCK) { + if (errno == ERRNO_BLOCK || errno == EINTR) { driver_select(port_num, forker_fd, ERL_DRV_WRITE|ERL_DRV_USE, 1); return 0; + } else if (errno == EMFILE) { + forker_sigchld(proto->u.start.port_id, errno); + forker_deq(port_num, proto); + return 0; + } else { + erts_exit(ERTS_DUMP_EXIT, "Failed to write to erl_child_setup: %d\n", errno); } - erts_exit(ERTS_DUMP_EXIT, "Failed to write to erl_child_setup: %d\n", errno); } #ifndef FORKER_PROTO_START_ACK ASSERT(res == sizeof(*proto)); - close(proto->u.start.fds[0]); - close(proto->u.start.fds[1]); - if (proto->u.start.fds[1] != proto->u.start.fds[2]) - close(proto->u.start.fds[2]); - driver_deq(port_num, sizeof(*proto)); + forker_deq(port_num, proto); #endif return 0; diff --git a/erts/emulator/sys/unix/sys_float.c b/erts/emulator/sys/unix/sys_float.c index a82c15bd32..832074f679 100644 --- a/erts/emulator/sys/unix/sys_float.c +++ b/erts/emulator/sys/unix/sys_float.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2001-2016. All Rights Reserved. + * Copyright Ericsson AB 2001-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/sys/unix/sys_time.c b/erts/emulator/sys/unix/sys_time.c index ef05380d17..8ba575b7b6 100644 --- a/erts/emulator/sys/unix/sys_time.c +++ b/erts/emulator/sys/unix/sys_time.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2005-2016. All Rights Reserved. + * Copyright Ericsson AB 2005-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/sys/unix/sys_uds.c b/erts/emulator/sys/unix/sys_uds.c index 278c6b6ba1..39a4866065 100644 --- a/erts/emulator/sys/unix/sys_uds.c +++ b/erts/emulator/sys/unix/sys_uds.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2002-2016. All Rights Reserved. + * Copyright Ericsson AB 2002-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -132,7 +132,7 @@ sys_uds_writev(int fd, struct iovec *iov, size_t iov_len, struct msghdr msg; struct cmsghdr *cmsg = NULL; - int res, i; + int res, i, error; /* initialize socket message */ memset(&msg, 0, sizeof(struct msghdr)); @@ -173,11 +173,22 @@ sys_uds_writev(int fd, struct iovec *iov, size_t iov_len, res = sendmsg(fd, &msg, flags); +#ifdef ETOOMANYREFS + /* Linux may give ETOOMANYREFS when there are too many fds in transit. + We map this to EMFILE as bsd and other use this error code and we want + the behaviour to be the same on all OSs */ + if (errno == ETOOMANYREFS) + errno = EMFILE; +#endif + error = errno; + if (iov_len > MAXIOV) free(iov[0].iov_base); free(msg.msg_control); + errno = error; + return res; } diff --git a/erts/emulator/sys/unix/sys_uds.h b/erts/emulator/sys/unix/sys_uds.h index 26c91d6a00..49a4b39250 100644 --- a/erts/emulator/sys/unix/sys_uds.h +++ b/erts/emulator/sys/unix/sys_uds.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2002-2016. All Rights Reserved. + * Copyright Ericsson AB 2002-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/sys/win32/erl_poll.c b/erts/emulator/sys/win32/erl_poll.c index fd4c745c3b..fcf5a0d533 100644 --- a/erts/emulator/sys/win32/erl_poll.c +++ b/erts/emulator/sys/win32/erl_poll.c @@ -362,11 +362,11 @@ is_io_ready(ErtsPollSet *ps) } static ERTS_INLINE void -woke_up(ErtsPollSet *ps) +woke_up(ErtsPollSet *ps, int waketype) { if (erts_atomic32_read_nob(&ps->wakeup_state) == ERTS_POLL_NOT_WOKEN) erts_atomic32_cmpxchg_nob(&ps->wakeup_state, - ERTS_POLL_WOKEN_TIMEDOUT, + waketype, ERTS_POLL_NOT_WOKEN); #ifdef DEBUG { @@ -960,12 +960,12 @@ static int cancel_driver_select(ErtsPollSet *ps, HANDLE event) void erts_poll_interrupt(ErtsPollSet *ps, int set /* bool */) { - HARDTRACEF(("In erts_poll_interrupt(%d)",set)); + HARDTRACEF(("In erts_poll_interrupt(%p, %d)",ps,set)); if (!set) reset_interrupt(ps); else set_interrupt(ps); - HARDTRACEF(("Out erts_poll_interrupt(%d)",set)); + HARDTRACEF(("Out erts_poll_interrupt(%p, %d)",ps,set)); } @@ -1051,19 +1051,20 @@ int erts_poll_wait(ErtsPollSet *ps, if (!erts_atomic32_read_nob(&break_waiter_state)) { HANDLE harr[2] = {ps->event_io_ready, break_happened_event}; - int num_h = 2; + int num_h = 2, handle; ERTS_MSACC_PUSH_STATE(); HARDDEBUGF(("Start waiting %d [%d]",num_h, (int) timeout)); ERTS_POLLSET_UNLOCK(ps); erts_thr_progress_prepare_wait(NULL); ERTS_MSACC_SET_STATE_CACHED(ERTS_MSACC_STATE_SLEEP); - WaitForMultipleObjects(num_h, harr, FALSE, timeout); + handle = WaitForMultipleObjects(num_h, harr, FALSE, timeout); erts_thr_progress_finalize_wait(NULL); ERTS_MSACC_POP_STATE(); ERTS_POLLSET_LOCK(ps); HARDDEBUGF(("Stop waiting %d [%d]",num_h, (int) timeout)); - woke_up(ps); + if (handle == WAIT_OBJECT_0) + woke_up(ps, ERTS_POLL_WOKEN_TIMEDOUT); } ERTS_UNSET_BREAK_REQUESTED; @@ -1075,7 +1076,10 @@ int erts_poll_wait(ErtsPollSet *ps, erts_mtx_unlock(&break_waiter_lock); switch (break_state) { case BREAK_WAITER_GOT_BREAK: + woke_up(ps, ERTS_POLL_WOKEN_INTR); ERTS_SET_BREAK_REQUESTED; + /* Wake aux thread to get handle break */ + erts_aux_thread_poke(); break; case BREAK_WAITER_GOT_HALT: erts_exit(0,""); diff --git a/erts/emulator/sys/win32/erl_win32_sys_ddll.c b/erts/emulator/sys/win32/erl_win32_sys_ddll.c index fc2179328f..7fe1f5cc78 100644 --- a/erts/emulator/sys/win32/erl_win32_sys_ddll.c +++ b/erts/emulator/sys/win32/erl_win32_sys_ddll.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2006-2016. All Rights Reserved. + * Copyright Ericsson AB 2006-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/sys/win32/erl_win_dyn_driver.h b/erts/emulator/sys/win32/erl_win_dyn_driver.h index 0d1a6d4c87..c683e8cf49 100644 --- a/erts/emulator/sys/win32/erl_win_dyn_driver.h +++ b/erts/emulator/sys/win32/erl_win_dyn_driver.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2003-2016. All Rights Reserved. + * Copyright Ericsson AB 2003-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/sys/win32/erl_win_sys.h b/erts/emulator/sys/win32/erl_win_sys.h index 1f53452d17..b00ba287e2 100644 --- a/erts/emulator/sys/win32/erl_win_sys.h +++ b/erts/emulator/sys/win32/erl_win_sys.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1997-2016. All Rights Reserved. + * Copyright Ericsson AB 1997-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/sys/win32/sys_interrupt.c b/erts/emulator/sys/win32/sys_interrupt.c index 02aa50500f..cee269eed4 100644 --- a/erts/emulator/sys/win32/sys_interrupt.c +++ b/erts/emulator/sys/win32/sys_interrupt.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1997-2016. All Rights Reserved. + * Copyright Ericsson AB 1997-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/sys/win32/sys_time.c b/erts/emulator/sys/win32/sys_time.c index 25c2ad385c..a1dd14f871 100644 --- a/erts/emulator/sys/win32/sys_time.c +++ b/erts/emulator/sys/win32/sys_time.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1997-2016. All Rights Reserved. + * Copyright Ericsson AB 1997-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/test/alloc_SUITE.erl b/erts/emulator/test/alloc_SUITE.erl index 88ff2a7a92..4e0243c1cd 100644 --- a/erts/emulator/test/alloc_SUITE.erl +++ b/erts/emulator/test/alloc_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2003-2016. All Rights Reserved. +%% Copyright Ericsson AB 2003-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -71,7 +71,8 @@ migration(Cfg) -> %% Disable driver_alloc to avoid recursive alloc_util calls %% through enif_mutex_create() in my_creating_mbc(). drv_case(Cfg, concurrent, "+MZe true +MRe false"), - drv_case(Cfg, concurrent, "+MZe true +MRe false +MZas ageffcbf"). + drv_case(Cfg, concurrent, "+MZe true +MRe false +MZas ageffcbf"), + drv_case(Cfg, concurrent, "+MZe true +MRe false +MZas chaosff"). erts_mmap(Config) when is_list(Config) -> case {os:type(), mmsc_flags()} of diff --git a/erts/emulator/test/alloc_SUITE_data/migration.c b/erts/emulator/test/alloc_SUITE_data/migration.c index 1d974225fc..78f3a633e8 100644 --- a/erts/emulator/test/alloc_SUITE_data/migration.c +++ b/erts/emulator/test/alloc_SUITE_data/migration.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2014-2016. All Rights Reserved. + * Copyright Ericsson AB 2014-2018. All Rights Reserved. * * The contents of this file are subject to the Erlang Public License, * Version 1.1, (the "License"); you may not use this file except in diff --git a/erts/emulator/test/beam_SUITE.erl b/erts/emulator/test/beam_SUITE.erl index bdf8f6c34e..d3b3b96b14 100644 --- a/erts/emulator/test/beam_SUITE.erl +++ b/erts/emulator/test/beam_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1998-2016. All Rights Reserved. +%% Copyright Ericsson AB 1998-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/erts/emulator/test/beam_literals_SUITE.erl b/erts/emulator/test/beam_literals_SUITE.erl index b447ca0210..82a5e2b172 100644 --- a/erts/emulator/test/beam_literals_SUITE.erl +++ b/erts/emulator/test/beam_literals_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2016. All Rights Reserved. +%% Copyright Ericsson AB 1999-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/erts/emulator/test/bif_SUITE.erl b/erts/emulator/test/bif_SUITE.erl index 32bfcd5520..9e7bcd5255 100644 --- a/erts/emulator/test/bif_SUITE.erl +++ b/erts/emulator/test/bif_SUITE.erl @@ -36,7 +36,8 @@ error_stacktrace_during_call_trace/1, group_leader_prio/1, group_leader_prio_dirty/1, is_process_alive/1, - process_info_blast/1]). + process_info_blast/1, + os_env_case_sensitivity/1]). suite() -> [{ct_hooks,[ts_install_cth]}, @@ -51,7 +52,7 @@ all() -> erl_crash_dump_bytes, min_max, erlang_halt, is_builtin, error_stacktrace, error_stacktrace_during_call_trace, group_leader_prio, group_leader_prio_dirty, - is_process_alive, process_info_blast]. + is_process_alive, process_info_blast, os_env_case_sensitivity]. %% Uses erlang:display to test that erts_printf does not do deep recursion display(Config) when is_list(Config) -> @@ -443,6 +444,17 @@ os_env_long(Min, Max, Value) -> true = os:unsetenv(EnvVar), os_env_long(Min+1, Max, Value). +os_env_case_sensitivity(Config) when is_list(Config) -> + %% The keys in os:getenv/putenv must be case-insensitive on Windows, and + %% case-sensitive elsewhere. + true = os:putenv("os_env_gurka", "gaffel"), + Expected = case os:type() of + {win32, _} -> "gaffel"; + _ -> false + end, + Expected = os:getenv("OS_ENV_GURKA"), + ok. + %% Test that string:to_integer does not Halloc in wrong order. otp_7526(Config) when is_list(Config) -> ok = test_7526(256). diff --git a/erts/emulator/test/big_SUITE.erl b/erts/emulator/test/big_SUITE.erl index 5939d024ae..0a42b09903 100644 --- a/erts/emulator/test/big_SUITE.erl +++ b/erts/emulator/test/big_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2016. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/erts/emulator/test/binary_SUITE.erl b/erts/emulator/test/binary_SUITE.erl index a3c3daac15..23c675733c 100644 --- a/erts/emulator/test/binary_SUITE.erl +++ b/erts/emulator/test/binary_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2017. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/erts/emulator/test/call_trace_SUITE.erl b/erts/emulator/test/call_trace_SUITE.erl index d19f7f81ad..742592f88e 100644 --- a/erts/emulator/test/call_trace_SUITE.erl +++ b/erts/emulator/test/call_trace_SUITE.erl @@ -1395,7 +1395,7 @@ seq(M, N, R) when M =< N -> seq(M, N-1, [N|R]); seq(_, _, R) -> R. -%% lists:reverse can not be called since it is traced +%% lists:reverse cannot be called since it is traced reverse(L) -> reverse(L, []). %% diff --git a/erts/emulator/test/code_SUITE.erl b/erts/emulator/test/code_SUITE.erl index 9c6dc3ff83..7e690fd870 100644 --- a/erts/emulator/test/code_SUITE.erl +++ b/erts/emulator/test/code_SUITE.erl @@ -332,6 +332,7 @@ constant_pools(Config) when is_list(Config) -> A = literals:a(), B = literals:b(), C = literals:huge_bignum(), + D = literals:funs(), process_flag(trap_exit, true), Self = self(), @@ -345,7 +346,7 @@ constant_pools(Config) when is_list(Config) -> true = erlang:purge_module(literals), NoOldHeap ! done, receive - {'EXIT',NoOldHeap,{A,B,C}} -> + {'EXIT',NoOldHeap,{A,B,C,D}} -> ok; Other -> ct:fail({unexpected,Other}) @@ -362,7 +363,7 @@ constant_pools(Config) when is_list(Config) -> erlang:purge_module(literals), OldHeap ! done, receive - {'EXIT',OldHeap,{A,B,C,[1,2,3|_]=Seq}} when length(Seq) =:= 16 -> + {'EXIT',OldHeap,{A,B,C,D,[1,2,3|_]=Seq}} when length(Seq) =:= 16 -> ok end, @@ -390,7 +391,7 @@ constant_pools(Config) when is_list(Config) -> {'DOWN', Mon, process, Hib, Reason} -> {undef, [{no_module, no_function, - [{A,B,C,[1,2,3|_]=Seq}], _}]} = Reason, + [{A,B,C,D,[1,2,3|_]=Seq}], _}]} = Reason, 16 = length(Seq) end, HeapSz = TotHeapSz, %% Ensure restored to hibernated state... @@ -400,7 +401,9 @@ constant_pools(Config) when is_list(Config) -> no_old_heap(Parent) -> A = literals:a(), B = literals:b(), - Res = {A,B,literals:huge_bignum()}, + C = literals:huge_bignum(), + D = literals:funs(), + Res = {A,B,C,D}, Parent ! go, receive done -> @@ -410,7 +413,9 @@ no_old_heap(Parent) -> old_heap(Parent) -> A = literals:a(), B = literals:b(), - Res = {A,B,literals:huge_bignum(),lists:seq(1, 16)}, + C = literals:huge_bignum(), + D = literals:funs(), + Res = {A,B,C,D,lists:seq(1, 16)}, create_old_heap(), Parent ! go, receive @@ -421,7 +426,9 @@ old_heap(Parent) -> hibernated(Parent) -> A = literals:a(), B = literals:b(), - Res = {A,B,literals:huge_bignum(),lists:seq(1, 16)}, + C = literals:huge_bignum(), + D = literals:funs(), + Res = {A,B,C,D,lists:seq(1, 16)}, Parent ! go, erlang:hibernate(no_module, no_function, [Res]). @@ -755,7 +762,8 @@ t_copy_literals_frags(Config) when is_list(Config) -> 0, 1, 2, 3, 4, 5, 6, 7, 8, 9,10,11,12,13,14,15, 0, 1, 2, 3, 4, 5, 6, 7, - 8, 9,10,11,12,13,14,15>>}]), + 8, 9,10,11,12,13,14,15>>}, + {f, fun ?MODULE:all/0}]), {module, ?mod} = erlang:load_module(?mod, Bin), N = 6000, @@ -796,6 +804,7 @@ literal_receiver() -> C = ?mod:c(), D = ?mod:d(), E = ?mod:e(), + F = ?mod:f(), literal_receiver(); {Pid, sender_confirm} -> io:format("sender confirm ~w~n", [Pid]), @@ -811,7 +820,8 @@ literal_sender(N, Recv) -> ?mod:b(), ?mod:c(), ?mod:d(), - ?mod:e()]}, + ?mod:e(), + ?mod:f()]}, literal_sender(N - 1, Recv). literal_switcher() -> diff --git a/erts/emulator/test/code_SUITE_data/literals.erl b/erts/emulator/test/code_SUITE_data/literals.erl index 7c3b0ebe73..13c8b412b0 100644 --- a/erts/emulator/test/code_SUITE_data/literals.erl +++ b/erts/emulator/test/code_SUITE_data/literals.erl @@ -19,7 +19,8 @@ %% -module(literals). --export([a/0,b/0,huge_bignum/0,binary/0,unused_binaries/0,bits/0]). +-export([a/0,b/0,huge_bignum/0,funs/0, + binary/0,unused_binaries/0,bits/0]). -export([msg1/0,msg2/0,msg3/0,msg4/0,msg5/0]). a() -> @@ -108,3 +109,8 @@ msg2() -> {"hello","world"}. msg3() -> <<"halloj">>. msg4() -> #{ 1=> "hello", b => "world"}. msg5() -> {1,2,3,4,5,6}. + +funs() -> + %% Literal funs (in a non-literal list). + [fun ?MODULE:a/0, + fun() -> ok end]. %No environment. diff --git a/erts/emulator/test/decode_packet_SUITE.erl b/erts/emulator/test/decode_packet_SUITE.erl index 0ccdbd7ee8..ef13b515fb 100644 --- a/erts/emulator/test/decode_packet_SUITE.erl +++ b/erts/emulator/test/decode_packet_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2016. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/erts/emulator/test/dgawd_handler.erl b/erts/emulator/test/dgawd_handler.erl index 29b9d6ac7b..b66b5a073f 100644 --- a/erts/emulator/test/dgawd_handler.erl +++ b/erts/emulator/test/dgawd_handler.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2006-2016. All Rights Reserved. +%% Copyright Ericsson AB 2006-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/erts/emulator/test/distribution_SUITE.erl b/erts/emulator/test/distribution_SUITE.erl index 45dd922ff0..885c66331c 100644 --- a/erts/emulator/test/distribution_SUITE.erl +++ b/erts/emulator/test/distribution_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2017. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/erts/emulator/test/driver_SUITE.erl b/erts/emulator/test/driver_SUITE.erl index 40c7cc11e1..9ffb484eb4 100644 --- a/erts/emulator/test/driver_SUITE.erl +++ b/erts/emulator/test/driver_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2017. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -81,6 +81,7 @@ thr_msg_blast/1, consume_timeslice/1, env/1, + poll_pipe/1, z_test/1]). -export([bin_prefix/2]). @@ -168,6 +169,7 @@ all() -> %% Keep a_test first and z_test last... thr_msg_blast, consume_timeslice, env, + poll_pipe, z_test]. groups() -> @@ -2643,24 +2645,7 @@ wait_deallocations() -> driver_alloc_size() -> wait_deallocations(), - case erlang:system_info({allocator_sizes, driver_alloc}) of - false -> - undefined; - MemInfo -> - CS = lists:foldl( - fun ({instance, _, L}, Acc) -> - {value,{_,MBCS}} = lists:keysearch(mbcs, 1, L), - {value,{_,SBCS}} = lists:keysearch(sbcs, 1, L), - [MBCS,SBCS | Acc] - end, - [], - MemInfo), - lists:foldl( - fun(L, Sz0) -> - {value,{_,Sz,_,_}} = lists:keysearch(blocks_size, 1, L), - Sz0+Sz - end, 0, CS) - end. + erts_debug:alloc_blocks_size(driver_alloc). rpc(Config, Fun) -> case proplists:get_value(node, Config) of @@ -2693,3 +2678,25 @@ rpc(Config, Fun) -> ct:fail(Other) end end. + +poll_pipe(Config) when is_list(Config) -> + %% ERL-647; we wouldn't see any events on EOF when polling a pipe using + %% kqueue(2). + case os:type() of + {unix, _} -> + Command = "erl -noshell -eval " + "'\"DATA\n\" = io:get_line(\"\")," + "eof = io:get_line(\"\")," + "halt()' <<< 'DATA'", + Ref = make_ref(), + Self = self(), + Pid = spawn(fun() -> os:cmd(Command), Self ! Ref end), + receive + Ref -> ok + after 5000 -> + exit(Pid, kill), + ct:fail("Stuck reading from stdin.") + end; + _ -> + {skipped, "Unix-only test"} + end. diff --git a/erts/emulator/test/driver_SUITE_data/ioq_exit_drv.c b/erts/emulator/test/driver_SUITE_data/ioq_exit_drv.c index fa58e9d5ec..9e96923e17 100644 --- a/erts/emulator/test/driver_SUITE_data/ioq_exit_drv.c +++ b/erts/emulator/test/driver_SUITE_data/ioq_exit_drv.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2007-2016. All Rights Reserved. + * Copyright Ericsson AB 2007-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/test/dump_SUITE.erl b/erts/emulator/test/dump_SUITE.erl index 8d18d46d92..d0237b78cc 100644 --- a/erts/emulator/test/dump_SUITE.erl +++ b/erts/emulator/test/dump_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2017. All Rights Reserved. +%% Copyright Ericsson AB 2005-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/erts/emulator/test/efile_SUITE.erl b/erts/emulator/test/efile_SUITE.erl index 821381bf0d..7dcf302742 100644 --- a/erts/emulator/test/efile_SUITE.erl +++ b/erts/emulator/test/efile_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2017. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/erts/emulator/test/erts_debug_SUITE.erl b/erts/emulator/test/erts_debug_SUITE.erl index 6aa7a445b5..f39dbedd8f 100644 --- a/erts/emulator/test/erts_debug_SUITE.erl +++ b/erts/emulator/test/erts_debug_SUITE.erl @@ -22,8 +22,10 @@ -include_lib("common_test/include/ct.hrl"). -export([all/0, suite/0, - test_size/1,flat_size_big/1,df/1,term_type/1, - instructions/1, stack_check/1]). + test_size/1,flat_size_big/1,df/1,term_type/1, + instructions/1, stack_check/1, alloc_blocks_size/1]). + +-export([do_alloc_blocks_size/0]). suite() -> [{ct_hooks,[ts_install_cth]}, @@ -31,7 +33,7 @@ suite() -> all() -> [test_size, flat_size_big, df, instructions, term_type, - stack_check]. + stack_check, alloc_blocks_size]. test_size(Config) when is_list(Config) -> ConsCell1 = id([a|b]), @@ -210,5 +212,28 @@ instructions(Config) when is_list(Config) -> _ = [list_to_atom(I) || I <- Is], ok. +alloc_blocks_size(Config) when is_list(Config) -> + F = fun(Args) -> + Node = start_slave(Args), + ok = rpc:call(Node, ?MODULE, do_alloc_blocks_size, []), + true = test_server:stop_node(Node) + end, + F("+Meamax"), + F("+Meamin"), + F(""), + ok. + +do_alloc_blocks_size() -> + _ = erts_debug:alloc_blocks_size(binary_alloc), + ok. + +start_slave(Args) -> + Name = ?MODULE_STRING ++ "_slave", + Pa = filename:dirname(code:which(?MODULE)), + {ok, Node} = test_server:start_node(list_to_atom(Name), + slave, + [{args, "-pa " ++ Pa ++ " " ++ Args}]), + Node. + id(I) -> I. diff --git a/erts/emulator/test/exception_SUITE.erl b/erts/emulator/test/exception_SUITE.erl index 60d14ce841..aec66cb9a3 100644 --- a/erts/emulator/test/exception_SUITE.erl +++ b/erts/emulator/test/exception_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2017. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/erts/emulator/test/fun_SUITE.erl b/erts/emulator/test/fun_SUITE.erl index 7d29ebec52..f8a879182e 100644 --- a/erts/emulator/test/fun_SUITE.erl +++ b/erts/emulator/test/fun_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2016. All Rights Reserved. +%% Copyright Ericsson AB 1999-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -576,7 +576,7 @@ refc_dist(Config) when is_list(Config) -> process_flag(trap_exit, true), Pid = spawn_link(Node, fun() -> receive Fun when is_function(Fun) -> - 2 = fun_refc(Fun), + 3 = fun_refc(Fun), exit({normal,Fun}) end end), F = fun() -> 42 end, @@ -598,7 +598,7 @@ refc_dist_send(Node, F) -> Pid = spawn_link(Node, fun() -> receive {To,Fun} when is_function(Fun) -> wait_until(fun () -> - 2 =:= fun_refc(Fun) + 3 =:= fun_refc(Fun) end), To ! Fun end @@ -626,7 +626,7 @@ refc_dist_reg_send(Node, F) -> Me ! Ref, receive {Me,Fun} when is_function(Fun) -> - 2 = fun_refc(Fun), + 3 = fun_refc(Fun), Me ! Fun end end), @@ -806,11 +806,13 @@ verify_not_undef(Fun, Tag) -> ct:fail("tag ~w not defined in fun_info", [Tag]); {Tag,_} -> ok end. - + id(X) -> X. spawn_call(Node, AFun) -> + Parent = self(), + Init = erlang:whereis(init), Pid = spawn_link(Node, fun() -> receive @@ -821,8 +823,10 @@ spawn_call(Node, AFun) -> _ -> lists:seq(0, Arity-1) end, Res = apply(Fun, Args), - {pid,Creator} = erlang:fun_info(Fun, pid), - Creator ! {result,Res} + case erlang:fun_info(Fun, pid) of + {pid,Init} -> Parent ! {result,Res}; + {pid,Creator} -> Creator ! {result,Res} + end end end), Pid ! {AFun,AFun,AFun}, diff --git a/erts/emulator/test/iovec_SUITE.erl b/erts/emulator/test/iovec_SUITE.erl index 963b7e2501..d17a28d47f 100644 --- a/erts/emulator/test/iovec_SUITE.erl +++ b/erts/emulator/test/iovec_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2017. All Rights Reserved. +%% Copyright Ericsson AB 2017-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/erts/emulator/test/lcnt_SUITE.erl b/erts/emulator/test/lcnt_SUITE.erl index dfffd662e2..87b97037d6 100644 --- a/erts/emulator/test/lcnt_SUITE.erl +++ b/erts/emulator/test/lcnt_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2017. All Rights Reserved. +%% Copyright Ericsson AB 2017-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/erts/emulator/test/map_SUITE.erl b/erts/emulator/test/map_SUITE.erl index f93c637650..d0a6763fe5 100644 --- a/erts/emulator/test/map_SUITE.erl +++ b/erts/emulator/test/map_SUITE.erl @@ -3080,8 +3080,19 @@ y_regs(Config) when is_list(Config) -> true = is_map(Map2) andalso is_map(Map4), + gurka = y_regs_literal(0), + gaffel = y_regs_literal(1), + ok. +y_regs_literal(Key) when is_integer(Key) -> + %% Forces the key to be placed in a Y register. + lists:seq(1, 2), + case is_map_key(Key, #{ 0 => 0 }) of + true -> gurka; + false -> gaffel + end. + y_regs_update(Map0, Val0) -> Val1 = {t,Val0}, K1 = id({key,1}), diff --git a/erts/emulator/test/match_spec_SUITE.erl b/erts/emulator/test/match_spec_SUITE.erl index 4415d8d1b9..21de6b1002 100644 --- a/erts/emulator/test/match_spec_SUITE.erl +++ b/erts/emulator/test/match_spec_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2017. All Rights Reserved. +%% Copyright Ericsson AB 1999-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/erts/emulator/test/module_info_SUITE.erl b/erts/emulator/test/module_info_SUITE.erl index 46a3bba732..93f9de0c28 100644 --- a/erts/emulator/test/module_info_SUITE.erl +++ b/erts/emulator/test/module_info_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2016. All Rights Reserved. +%% Copyright Ericsson AB 2005-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/erts/emulator/test/monitor_SUITE.erl b/erts/emulator/test/monitor_SUITE.erl index c7250a9d26..27351dc5c1 100644 --- a/erts/emulator/test/monitor_SUITE.erl +++ b/erts/emulator/test/monitor_SUITE.erl @@ -314,7 +314,7 @@ local_remove_monitor(Config) when is_list(Config) -> remote_remove_monitor(Config) when is_list(Config) -> {ok, N} = test_server:start_node(demonitor_flush, slave, []), - Gs = generate(fun () -> start_remove_monitor_group(node()) end, + Gs = generate(fun () -> start_remove_monitor_group(N) end, ?RM_MON_GROUPS), {True, False} = lists:foldl(fun (G, {T, F}) -> receive diff --git a/erts/emulator/test/nif_SUITE.erl b/erts/emulator/test/nif_SUITE.erl index 100fa006e7..a2f3489943 100644 --- a/erts/emulator/test/nif_SUITE.erl +++ b/erts/emulator/test/nif_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2017. All Rights Reserved. +%% Copyright Ericsson AB 2010-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -45,8 +45,9 @@ api_macros/1, from_array/1, iolist_as_binary/1, resource/1, resource_binary/1, resource_takeover/1, - threading/1, send/1, send2/1, send3/1, send_threaded/1, neg/1, - is_checks/1, + threading/1, send/1, send2/1, send3/1, send_threaded/1, + send_trace/1, send_seq_trace/1, + neg/1, is_checks/1, get_length/1, make_atom/1, make_string/1, reverse_list_test/1, otp_9828/1, otp_9668/1, consume_timeslice/1, nif_schedule/1, @@ -1789,6 +1790,59 @@ send(Config) when is_list(Config) -> {ok,0} = send_list_seq(7, DeadPid), ok. + +%% Test tracing of enif_send +send_trace(Config) when is_list(Config) -> + ensure_lib_loaded(Config), + + Papa = self(), + N = 1500, + List = lists:seq(1,N), + + Tracer = spawn_link(fun F() -> receive get -> Papa ! receive_any(), F() end end), + + erlang:trace(self(), true, [send,'receive',{tracer,Tracer}]), + {ok,1} = send_list_seq(N, self()), + List = receive_any(), + timeout = receive_any(0), + Tracer ! get, + {trace,Papa,send,List,Papa} = receive_any(), + Tracer ! get, + {trace,Papa,'receive',List} = receive_any(). + +%% Test that seq_trace works with nif trace +send_seq_trace(Config) when is_list(Config) -> + ensure_lib_loaded(Config), + + Papa = self(), + N = 1500, + List = lists:seq(1,N), + Label = make_ref(), + + Tracer = spawn_link(fun F() -> receive get -> Papa ! receive_any(), F() end end), + + seq_trace:set_system_tracer(Tracer), + seq_trace:set_token(label,Label), + seq_trace:set_token(send,true), + seq_trace:set_token('receive',true), + + {ok,1} = send_list_seq(N, self()), + List = receive_any(), + timeout = receive_any(0), + {ok,1} = send_list_seq(N, self()), + List = receive_any(), + timeout = receive_any(0), + + Tracer ! get, + {seq_trace,Label,{send,{0,1},Papa,Papa,List}} = receive_any(), + Tracer ! get, + {seq_trace,Label,{'receive',{0,1},Papa,Papa,List}} = receive_any(), + Tracer ! get, + {seq_trace,Label,{send,{1,2},Papa,Papa,List}} = receive_any(), + Tracer ! get, + {seq_trace,Label,{'receive',{1,2},Papa,Papa,List}} = receive_any(). + + %% More NIF message sending send2(Config) when is_list(Config) -> ensure_lib_loaded(Config), diff --git a/erts/emulator/test/nif_SUITE_data/nif_SUITE.c b/erts/emulator/test/nif_SUITE_data/nif_SUITE.c index 155bda6df0..f2ce6dbe67 100644 --- a/erts/emulator/test/nif_SUITE_data/nif_SUITE.c +++ b/erts/emulator/test/nif_SUITE_data/nif_SUITE.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2009-2017. All Rights Reserved. + * Copyright Ericsson AB 2009-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/emulator/test/num_bif_SUITE.erl b/erts/emulator/test/num_bif_SUITE.erl index 290bb61fc8..700734cd0b 100644 --- a/erts/emulator/test/num_bif_SUITE.erl +++ b/erts/emulator/test/num_bif_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2017. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/erts/emulator/test/process_SUITE.erl b/erts/emulator/test/process_SUITE.erl index 585c5a1871..f4b1d885fe 100644 --- a/erts/emulator/test/process_SUITE.erl +++ b/erts/emulator/test/process_SUITE.erl @@ -59,6 +59,7 @@ no_priority_inversion2/1, system_task_blast/1, system_task_on_suspended/1, + system_task_failed_enqueue/1, gc_request_when_gc_disabled/1, gc_request_blast_when_gc_disabled/1]). -export([prio_server/2, prio_client/2, init/1, handle_event/2]). @@ -106,7 +107,7 @@ groups() -> otp_7738_resume]}, {system_task, [], [no_priority_inversion, no_priority_inversion2, - system_task_blast, system_task_on_suspended, + system_task_blast, system_task_on_suspended, system_task_failed_enqueue, gc_request_when_gc_disabled, gc_request_blast_when_gc_disabled]}]. init_per_suite(Config) -> @@ -2232,8 +2233,8 @@ processes_term_proc_list(Config) when is_list(Config) -> %% We have to run this test case with +S1 since instrument:allocations() %% will report a free()'d block as present until it's actually deallocated %% by its employer. - Run("+MSe true +MSatags false +S1"), - Run("+MSe true +MSatags true +S1"), + Run("+MSe true +Muatags false +S1"), + Run("+MSe true +Muatags true +S1"), ok. @@ -2241,10 +2242,12 @@ processes_term_proc_list(Config) when is_list(Config) -> chk_term_proc_list(?LINE, MC, XB)). chk_term_proc_list(Line, MustChk, ExpectBlks) -> - Allocs = instrument:allocations(#{ allocator_types => [sl_alloc] }), + Allocs = instrument:allocations(), case {MustChk, Allocs} of {false, {error, not_enabled}} -> not_enabled; + {false, {ok, {_Shift, _Unscanned, ByOrigin}}} when ByOrigin =:= #{} -> + not_enabled; {_, {ok, {_Shift, _Unscanned, ByOrigin}}} -> ByType = maps:get(system, ByOrigin, #{}), Hist = maps:get(ptab_list_deleted_el, ByType, {}), @@ -2625,6 +2628,57 @@ system_task_on_suspended(Config) when is_list(Config) -> ok end. +%% When a system task couldn't be enqueued due to the process being in an +%% incompatible state, it would linger in the system task list and get executed +%% anyway the next time the process was scheduled. This would result in a +%% double-free at best. +%% +%% This test continuously purges modules while other processes run dirty code, +%% which will provoke this error as ERTS_PSTT_CPC can't be enqueued while a +%% process is running dirty code. +system_task_failed_enqueue(Config) when is_list(Config) -> + case erlang:system_info(dirty_cpu_schedulers) of + N when N > 0 -> + system_task_failed_enqueue_1(Config); + _ -> + {skipped, "No dirty scheduler support"} + end. + +system_task_failed_enqueue_1(Config) -> + Priv = proplists:get_value(priv_dir, Config), + + Purgers = [spawn_link(fun() -> purge_loop(Priv, Id) end) + || Id <- lists:seq(1, erlang:system_info(schedulers))], + Hogs = [spawn_link(fun() -> dirty_loop() end) + || _ <- lists:seq(1, erlang:system_info(dirty_cpu_schedulers))], + + ct:sleep(5000), + + [begin + unlink(Pid), + exit(Pid, kill) + end || Pid <- (Purgers ++ Hogs)], + + ok. + +purge_loop(PrivDir, Id) -> + Mod = "failed_enq_" ++ integer_to_list(Id), + Path = PrivDir ++ "/" ++ Mod, + file:write_file(Path ++ ".erl", + "-module('" ++ Mod ++ "').\n" ++ + "-export([t/0]).\n" ++ + "t() -> ok."), + purge_loop_1(Path). +purge_loop_1(Path) -> + {ok, Mod} = compile:file(Path, []), + erlang:delete_module(Mod), + erts_code_purger:purge(Mod), + purge_loop_1(Path). + +dirty_loop() -> + ok = erts_debug:dirty_cpu(reschedule, 10000), + dirty_loop(). + gc_request_when_gc_disabled(Config) when is_list(Config) -> AIS = erts_debug:set_internal_state(available_internal_state, true), gc_request_when_gc_disabled_do(ref), diff --git a/erts/emulator/test/ref_SUITE.erl b/erts/emulator/test/ref_SUITE.erl index 5f519d522e..74df857c65 100644 --- a/erts/emulator/test/ref_SUITE.erl +++ b/erts/emulator/test/ref_SUITE.erl @@ -22,6 +22,7 @@ -export([all/0, suite/0]). -export([wrap_1/1]). +-export([compare_list/1, compare_ets/1]). -export([loop_ref/1]). @@ -32,7 +33,7 @@ suite() -> {timetrap, {minutes, 2}}]. all() -> - [wrap_1]. + [wrap_1, compare_list, compare_ets]. %% Check that refs don't wrap around easily. wrap_1(Config) when is_list(Config) -> @@ -53,3 +54,28 @@ loop_ref(Parent) -> loop_ref(R, R, _) -> ok; loop_ref(R0, _, N) -> loop_ref(R0, make_ref(), N+1). + +%% Check that ref ordering works +compare_list(Config) when is_list(Config) -> + %% Although this test uses external refs, it would apply the same to plain refs + ExtRef1 = <<131,114,0,3,100,0,3,110,64,98,3, 0,0,173,156, 0,216,0,4, 0,0,0,0>>, + ExtRef2 = <<131,114,0,3,100,0,3,110,64,98,3, 0,1,31,27, 129,4,0,1, 0,0,0,0>>, + + Ref1 = binary_to_term(ExtRef1), %% #Ref<[email protected]> + Ref2 = binary_to_term(ExtRef2), %% #Ref<[email protected]> + OrderedList = [Ref1, Ref2], + OrderedList = lists:sort(OrderedList), + ok. + +%% This is the scarier case since it makes terms "invisible" in ets or Mnesia +%% (the underlying fault cause is the same as compare_list/1) +compare_ets(Config) when is_list(Config) -> + W2s = [610350147,899574699,2994196869,686384822,2397690439, 923302211], + ExtRefBase = <<131,114,0,3,100,0,3,110,64,98,3>>, + ExtRefs = [<<ExtRefBase/binary, 1:32, W2:32, 0:32>> || W2 <- W2s], + Refs = [binary_to_term(Bin) || Bin <- ExtRefs], + + Ets = ets:new(refbug, [ordered_set]), + ets:insert(Ets, [{Ref,Ref} || Ref <- Refs]), + 0 = length([R || R <- ets:tab2list(Ets), ets:lookup(Ets, element(1,R)) == []]), + ok. diff --git a/erts/emulator/test/register_SUITE.erl b/erts/emulator/test/register_SUITE.erl index 49da94a775..a7c0acbf17 100644 --- a/erts/emulator/test/register_SUITE.erl +++ b/erts/emulator/test/register_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2016. All Rights Reserved. +%% Copyright Ericsson AB 2010-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/erts/emulator/test/sensitive_SUITE.erl b/erts/emulator/test/sensitive_SUITE.erl index 9b23a30e88..206d2c1bfc 100644 --- a/erts/emulator/test/sensitive_SUITE.erl +++ b/erts/emulator/test/sensitive_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2016. All Rights Reserved. +%% Copyright Ericsson AB 2007-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/erts/emulator/test/smoke_test_SUITE.erl b/erts/emulator/test/smoke_test_SUITE.erl index b3d34103f1..26c610e3a8 100644 --- a/erts/emulator/test/smoke_test_SUITE.erl +++ b/erts/emulator/test/smoke_test_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2011-2017. All Rights Reserved. +%% Copyright Ericsson AB 2011-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/erts/emulator/test/statistics_SUITE.erl b/erts/emulator/test/statistics_SUITE.erl index 3f2897242e..ae3099633a 100644 --- a/erts/emulator/test/statistics_SUITE.erl +++ b/erts/emulator/test/statistics_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2017. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/erts/emulator/test/system_info_SUITE.erl b/erts/emulator/test/system_info_SUITE.erl index 7309908337..8ea2d88ec4 100644 --- a/erts/emulator/test/system_info_SUITE.erl +++ b/erts/emulator/test/system_info_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2017. All Rights Reserved. +%% Copyright Ericsson AB 2005-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -457,11 +457,16 @@ cmp_memory(MWs, Str) -> %% Total, processes, processes_used, and system will seldom %% give us exactly the same result since the two readings %% aren't taken atomically. + %% + %% Torerance is scaled according to the number of schedulers + %% to match spawn_mem_workers. + + Tolerance = 1.05 + 0.01 * erlang:system_info(schedulers_online), - cmp_memory(total, EM, EDM, 1.05), - cmp_memory(processes, EM, EDM, 1.05), - cmp_memory(processes_used, EM, EDM, 1.05), - cmp_memory(system, EM, EDM, 1.05), + cmp_memory(total, EM, EDM, Tolerance), + cmp_memory(processes, EM, EDM, Tolerance), + cmp_memory(processes_used, EM, EDM, Tolerance), + cmp_memory(system, EM, EDM, Tolerance), ok. diff --git a/erts/emulator/test/system_profile_SUITE.erl b/erts/emulator/test/system_profile_SUITE.erl index ae27bfe9df..0c3844e90f 100644 --- a/erts/emulator/test/system_profile_SUITE.erl +++ b/erts/emulator/test/system_profile_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2017. All Rights Reserved. +%% Copyright Ericsson AB 2007-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/erts/emulator/test/timer_bif_SUITE.erl b/erts/emulator/test/timer_bif_SUITE.erl index fc11a04a31..15fe13c8c0 100644 --- a/erts/emulator/test/timer_bif_SUITE.erl +++ b/erts/emulator/test/timer_bif_SUITE.erl @@ -361,7 +361,7 @@ evil_timers(Config) when is_list(Config) -> %% %% 1. A timer started with erlang:start_timer(Time, Receiver, Msg), %% where Msg is a composite term, expires, and the receivers main - %% lock *can not* be acquired immediately (typically when the + %% lock *cannot* be acquired immediately (typically when the %% receiver *is* running). %% %% The wrap tuple ({timeout, TRef, Msg}) will in this case @@ -372,7 +372,7 @@ evil_timers(Config) when is_list(Config) -> RecvTimeOutMsgs0 = evil_recv_timeouts(200), %% 2. A timer started with erlang:start_timer(Time, Receiver, Msg), %% where Msg is an immediate term, expires, and the receivers main - %% lock *can not* be acquired immediately (typically when the + %% lock *cannot* be acquired immediately (typically when the %% receiver *is* running). %% %% The wrap tuple will in this case be allocated in a new diff --git a/erts/emulator/test/trace_SUITE.erl b/erts/emulator/test/trace_SUITE.erl index 138aefb29c..979b3185a5 100644 --- a/erts/emulator/test/trace_SUITE.erl +++ b/erts/emulator/test/trace_SUITE.erl @@ -38,7 +38,8 @@ system_monitor_long_gc_1/1, system_monitor_long_gc_2/1, system_monitor_large_heap_1/1, system_monitor_large_heap_2/1, system_monitor_long_schedule/1, - bad_flag/1, trace_delivered/1, trap_exit_self_receive/1]). + bad_flag/1, trace_delivered/1, trap_exit_self_receive/1, + trace_info_badarg/1]). -include_lib("common_test/include/ct.hrl"). @@ -62,7 +63,7 @@ all() -> system_monitor_long_gc_2, system_monitor_large_heap_1, system_monitor_long_schedule, system_monitor_large_heap_2, bad_flag, trace_delivered, - trap_exit_self_receive]. + trap_exit_self_receive, trace_info_badarg]. init_per_testcase(_Case, Config) -> [{receiver,spawn(fun receiver/0)}|Config]. @@ -1734,6 +1735,10 @@ trap_exit_self_receive(Config) -> receive_nothing(), ok. +trace_info_badarg(Config) when is_list(Config) -> + catch erlang:trace_info({a,b,c},d), + ok. + drop_trace_until_down(Proc, Mon) -> drop_trace_until_down(Proc, Mon, false, 0, 0). diff --git a/erts/emulator/test/tracer_SUITE.erl b/erts/emulator/test/tracer_SUITE.erl index 070462b0f1..5556953feb 100644 --- a/erts/emulator/test/tracer_SUITE.erl +++ b/erts/emulator/test/tracer_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2017. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/erts/emulator/test/tuple_SUITE.erl b/erts/emulator/test/tuple_SUITE.erl index baf41180e0..e03677a518 100644 --- a/erts/emulator/test/tuple_SUITE.erl +++ b/erts/emulator/test/tuple_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2016. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/erts/emulator/test/z_SUITE.erl b/erts/emulator/test/z_SUITE.erl index 103f9f1550..1c52e1a934 100644 --- a/erts/emulator/test/z_SUITE.erl +++ b/erts/emulator/test/z_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2006-2017. All Rights Reserved. +%% Copyright Ericsson AB 2006-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/erts/emulator/utils/make_driver_tab b/erts/emulator/utils/make_driver_tab index b7bca1dc3a..a000b9d415 100755 --- a/erts/emulator/utils/make_driver_tab +++ b/erts/emulator/utils/make_driver_tab @@ -2,7 +2,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 1999-2016. All Rights Reserved. +# Copyright Ericsson AB 1999-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. diff --git a/erts/emulator/utils/make_tables b/erts/emulator/utils/make_tables index 094a35ae4b..deee5c2344 100755 --- a/erts/emulator/utils/make_tables +++ b/erts/emulator/utils/make_tables @@ -2,7 +2,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 1999-2016. All Rights Reserved. +# Copyright Ericsson AB 1999-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. diff --git a/erts/etc/common/erlexec.c b/erts/etc/common/erlexec.c index 21a3f40c97..0cb01fd4ef 100644 --- a/erts/etc/common/erlexec.c +++ b/erts/etc/common/erlexec.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1996-2017. All Rights Reserved. + * Copyright Ericsson AB 1996-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/etc/common/escript.c b/erts/etc/common/escript.c index d739d21f12..078937e676 100644 --- a/erts/etc/common/escript.c +++ b/erts/etc/common/escript.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2007-2017. All Rights Reserved. + * Copyright Ericsson AB 2007-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -139,15 +139,6 @@ get_env(char *key) } static void -free_env_val(char *value) -{ -#ifdef __WIN32__ - if (value) - efree(value); -#endif -} - -static void set_env(char *key, char *value) { #ifdef __WIN32__ @@ -422,7 +413,6 @@ main(int argc, char** argv) int eargv_size; int eargc_base; /* How many arguments in the base of eargv. */ char* emulator; - char* env; char* basename; char* def_emu_lookup_path; char scriptname[PMAX]; @@ -504,7 +494,7 @@ main(int argc, char** argv) } /* Determine path to emulator */ - emulator = env = get_env("ESCRIPT_EMULATOR"); + emulator = get_env("ESCRIPT_EMULATOR"); if (emulator == NULL) { emulator = get_default_emulator(def_emu_lookup_path); @@ -518,7 +508,6 @@ main(int argc, char** argv) */ PUSH(emulator); - free_env_val(env); PUSH("+B"); PUSH2("-boot", "no_dot_erlang"); diff --git a/erts/etc/common/heart.c b/erts/etc/common/heart.c index 8f1e89b638..bd218ff725 100644 --- a/erts/etc/common/heart.c +++ b/erts/etc/common/heart.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1996-2016. All Rights Reserved. + * Copyright Ericsson AB 1996-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/etc/unix/cerl.src b/erts/etc/unix/cerl.src index 9dd8d85abe..2e034513b0 100644 --- a/erts/etc/unix/cerl.src +++ b/erts/etc/unix/cerl.src @@ -2,7 +2,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 2003-2016. All Rights Reserved. +# Copyright Ericsson AB 2003-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -69,15 +69,6 @@ cxargs_add() { done } -eeargs= -eeargs_add() { - while [ $# -gt 0 ]; do - cargs="$cargs $1" - eeargs="$eeargs $1" - shift - done -} - core= GDB= @@ -97,8 +88,6 @@ TARGET=%TARGET% PROGNAME=$ROOTDIR/bin/cerl EMU=beam -PRELOADED=$ROOTDIR/erts/preloaded/ebin - while [ $# -gt 0 ]; do case "$1" in @@ -255,7 +244,7 @@ EXEC=$BINDIR/erlexec PROGNAME="$PROGNAME$cargs" EMU="$EMU$TYPE" -EMU_NAME=`$EXEC -emu_name_exit $eeargs` +EMU_NAME=`$EXEC -emu_name_exit` if [ $skip_erlexec = yes ]; then emu_xargs=`echo $xargs | sed "s|+|-|g"` @@ -269,21 +258,20 @@ if [ $skip_erlexec = yes ]; then ' set -- $beam_args IFS="$SAVE_IFS" -else - xargs="$xargs -pz $PRELOADED --" fi if [ "x$GDB" = "x" ]; then if [ $run_valgrind = yes ]; then valversion=`valgrind --version` valmajor=`echo $valversion | sed 's,[a-z]*\-\([0-9]*\).*,\1,'` valminor=`echo $valversion | sed 's,[a-z]*\-[0-9]*.\([0-9]*\).*,\1,'` + valint=`echo "$valmajor * 1000 + $valminor" | bc` if [ "x$VALGRIND_LOG_XML" = "x" ]; then valgrind_xml= log_file_prefix="--log-file=" else export VALGRIND_LOG_XML valgrind_xml="--xml=yes" - if [ $valmajor -gt 2 -a $valminor -gt 4 ]; then + if [ $valint -gt 3004 ]; then log_file_prefix="--xml-file=" else log_file_prefix="--log-file=" @@ -292,7 +280,7 @@ if [ "x$GDB" = "x" ]; then if [ "x$VALGRIND_LOG_DIR" = "x" ]; then valgrind_log= else - if [ $valmajor -gt 2 -a $valminor -gt 4 ]; then + if [ $valint -gt 3004 ]; then valgrind_log="$log_file_prefix$VALGRIND_LOG_DIR/$VALGRIND_LOGFILE_PREFIX$VALGRIND_LOGFILE_INFIX$EMU_NAME.log.$$" else valgrind_log="$log_file_prefix$VALGRIND_LOG_DIR/$VALGRIND_LOGFILE_PREFIX$VALGRIND_LOGFILE_INFIX$EMU_NAME.log" @@ -316,12 +304,12 @@ if [ "x$GDB" = "x" ]; then sched_arg= fi - exec $taskset1 valgrind $valgrind_xml $valgrind_log $valgrind_misc_flags $BINDIR/$EMU_NAME $sched_arg $emu_xargs "$@" -pz $PRELOADED + exec $taskset1 valgrind $valgrind_xml $valgrind_log $valgrind_misc_flags $BINDIR/$EMU_NAME $sched_arg $emu_xargs "$@" elif [ $run_rr = yes ]; then - exec rr record --ignore-nested $BINDIR/$EMU_NAME $emu_xargs "$@" -pz $PRELOADED + exec rr record --ignore-nested $BINDIR/$EMU_NAME $emu_xargs "$@" else - exec $EXEC $eeargs $xargs ${1+"$@"} + exec $EXEC $xargs ${1+"$@"} fi elif [ "x$GDB" = "xgdb" ]; then case "x$core" in diff --git a/erts/etc/unix/dyn_erl.c b/erts/etc/unix/dyn_erl.c index 5c7c3cad38..c4a2f7217c 100644 --- a/erts/etc/unix/dyn_erl.c +++ b/erts/etc/unix/dyn_erl.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2009-2016. All Rights Reserved. + * Copyright Ericsson AB 2009-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/etc/unix/etp-commands.in b/erts/etc/unix/etp-commands.in index 39e378193a..b12a205ba7 100644 --- a/erts/etc/unix/etp-commands.in +++ b/erts/etc/unix/etp-commands.in @@ -1346,7 +1346,6 @@ define etp-sigq-int else if $etp_sig_save_last && *$etp_sig_save_last == $etp_sig printf " %% <== SAVED_LAST" - else end end if $etp_sig_next @@ -2735,25 +2734,37 @@ define etp-aux-work-flags printf " fix-alloc-lower-lim" end if ($arg0 & 0x10) - printf " async-ready" + printf " later-op" end if ($arg0 & 0x20) - printf " async-ready-clean" + printf " canceled-timers" end if ($arg0 & 0x40) - printf " misc-work-thr-prgr" + printf " canceled-timers-thr-prgr" end if ($arg0 & 0x80) - printf " misc-work" + printf " async-ready" end if ($arg0 & 0x100) - printf " check-children" + printf " async-ready-clean" end if ($arg0 & 0x200) - printf " set-tmo" + printf " misc-thr-prgr" end if ($arg0 & 0x400) - printf " mseg-cached-check" + printf " misc" + end + if ($arg0 & 0x800) + printf " set-tmo" + end + if ($arg0 & 0x1000) + printf " mseg-cache-check" + end + if ($arg0 & 0x2000) + printf " yield" + end + if ($arg0 & 0x1000) + printf " reap-ports" end if ($arg0 & ~0x7ff) printf " GARBAGE" diff --git a/erts/include/internal/erl_printf.h b/erts/include/internal/erl_printf.h index 7e9807f6a8..6881c9d4f1 100644 --- a/erts/include/internal/erl_printf.h +++ b/erts/include/internal/erl_printf.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2005-2016. All Rights Reserved. + * Copyright Ericsson AB 2005-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/include/internal/gcc/ethr_membar.h b/erts/include/internal/gcc/ethr_membar.h index 07960ce040..5cbda5582d 100644 --- a/erts/include/internal/gcc/ethr_membar.h +++ b/erts/include/internal/gcc/ethr_membar.h @@ -96,7 +96,7 @@ * issue an aquire memory barrier and an __atomic * builtin memory acess with the __ATOMIC_RELEASE * memory model must at least issue a release memory - * barrier. Otherwise the two can not be paired. + * barrier. Otherwise the two cannot be paired. * 4. All __atomic builtins accessing memory using the * __ATOMIC_CONSUME builtin can be used for the same * reason __ATOMIC_ACQUIRE can be used. The ethread diff --git a/erts/lib_src/Makefile.in b/erts/lib_src/Makefile.in index dfaf664a18..8e1f5b58c4 100644 --- a/erts/lib_src/Makefile.in +++ b/erts/lib_src/Makefile.in @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 2004-2016. All Rights Reserved. +# Copyright Ericsson AB 2004-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. diff --git a/erts/lib_src/common/erl_printf.c b/erts/lib_src/common/erl_printf.c index 1de0f81e84..259ba8c81d 100644 --- a/erts/lib_src/common/erl_printf.c +++ b/erts/lib_src/common/erl_printf.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2005-2016. All Rights Reserved. + * Copyright Ericsson AB 2005-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/lib_src/common/erl_printf_format.c b/erts/lib_src/common/erl_printf_format.c index 5a680d6f9d..8f9e0b4a90 100644 --- a/erts/lib_src/common/erl_printf_format.c +++ b/erts/lib_src/common/erl_printf_format.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2005-2016. All Rights Reserved. + * Copyright Ericsson AB 2005-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/erts/preloaded/ebin/erl_prim_loader.beam b/erts/preloaded/ebin/erl_prim_loader.beam Binary files differindex 495d306a23..0f5f5036f0 100644 --- a/erts/preloaded/ebin/erl_prim_loader.beam +++ b/erts/preloaded/ebin/erl_prim_loader.beam diff --git a/erts/preloaded/ebin/erl_tracer.beam b/erts/preloaded/ebin/erl_tracer.beam Binary files differindex cd2c0ac69d..6017112dac 100644 --- a/erts/preloaded/ebin/erl_tracer.beam +++ b/erts/preloaded/ebin/erl_tracer.beam diff --git a/erts/preloaded/ebin/erlang.beam b/erts/preloaded/ebin/erlang.beam Binary files differindex 99f11495a5..7a4167b0e9 100644 --- a/erts/preloaded/ebin/erlang.beam +++ b/erts/preloaded/ebin/erlang.beam diff --git a/erts/preloaded/ebin/erts_code_purger.beam b/erts/preloaded/ebin/erts_code_purger.beam Binary files differindex b6c69e3e67..c899b69a2c 100644 --- a/erts/preloaded/ebin/erts_code_purger.beam +++ b/erts/preloaded/ebin/erts_code_purger.beam diff --git a/erts/preloaded/ebin/erts_dirty_process_signal_handler.beam b/erts/preloaded/ebin/erts_dirty_process_signal_handler.beam Binary files differindex 1013b8de0c..9490a56758 100644 --- a/erts/preloaded/ebin/erts_dirty_process_signal_handler.beam +++ b/erts/preloaded/ebin/erts_dirty_process_signal_handler.beam diff --git a/erts/preloaded/ebin/erts_internal.beam b/erts/preloaded/ebin/erts_internal.beam Binary files differindex 73bd730eaa..15c59de80a 100644 --- a/erts/preloaded/ebin/erts_internal.beam +++ b/erts/preloaded/ebin/erts_internal.beam diff --git a/erts/preloaded/ebin/erts_literal_area_collector.beam b/erts/preloaded/ebin/erts_literal_area_collector.beam Binary files differindex 18f1f76055..e650a6b5af 100644 --- a/erts/preloaded/ebin/erts_literal_area_collector.beam +++ b/erts/preloaded/ebin/erts_literal_area_collector.beam diff --git a/erts/preloaded/ebin/init.beam b/erts/preloaded/ebin/init.beam Binary files differindex 1b458fc5da..858a9dc63e 100644 --- a/erts/preloaded/ebin/init.beam +++ b/erts/preloaded/ebin/init.beam diff --git a/erts/preloaded/ebin/otp_ring0.beam b/erts/preloaded/ebin/otp_ring0.beam Binary files differindex 69d809e325..0d194896c7 100644 --- a/erts/preloaded/ebin/otp_ring0.beam +++ b/erts/preloaded/ebin/otp_ring0.beam diff --git a/erts/preloaded/ebin/prim_buffer.beam b/erts/preloaded/ebin/prim_buffer.beam Binary files differindex e2f0d3f44d..4ad1380d0b 100644 --- a/erts/preloaded/ebin/prim_buffer.beam +++ b/erts/preloaded/ebin/prim_buffer.beam diff --git a/erts/preloaded/ebin/prim_eval.beam b/erts/preloaded/ebin/prim_eval.beam Binary files differindex e962fcfa17..2ae18846bf 100644 --- a/erts/preloaded/ebin/prim_eval.beam +++ b/erts/preloaded/ebin/prim_eval.beam diff --git a/erts/preloaded/ebin/prim_file.beam b/erts/preloaded/ebin/prim_file.beam Binary files differindex 2f1e55442a..3316e4348c 100644 --- a/erts/preloaded/ebin/prim_file.beam +++ b/erts/preloaded/ebin/prim_file.beam diff --git a/erts/preloaded/ebin/prim_inet.beam b/erts/preloaded/ebin/prim_inet.beam Binary files differindex 350cc343d5..c30c589d3a 100644 --- a/erts/preloaded/ebin/prim_inet.beam +++ b/erts/preloaded/ebin/prim_inet.beam diff --git a/erts/preloaded/ebin/prim_zip.beam b/erts/preloaded/ebin/prim_zip.beam Binary files differindex 1d50d32efe..4923cadbdc 100644 --- a/erts/preloaded/ebin/prim_zip.beam +++ b/erts/preloaded/ebin/prim_zip.beam diff --git a/erts/preloaded/ebin/zlib.beam b/erts/preloaded/ebin/zlib.beam Binary files differindex a328711702..07e7e97814 100644 --- a/erts/preloaded/ebin/zlib.beam +++ b/erts/preloaded/ebin/zlib.beam diff --git a/erts/preloaded/src/erl_prim_loader.erl b/erts/preloaded/src/erl_prim_loader.erl index 42c1f32a8e..ae5f86e017 100644 --- a/erts/preloaded/src/erl_prim_loader.erl +++ b/erts/preloaded/src/erl_prim_loader.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/erts/preloaded/src/erlang.erl b/erts/preloaded/src/erlang.erl index f4f31b1e4b..261b731900 100644 --- a/erts/preloaded/src/erlang.erl +++ b/erts/preloaded/src/erlang.erl @@ -86,15 +86,15 @@ | 'nano_seconds'. -opaque prepared_code() :: reference(). - -export_type([prepared_code/0]). --opaque dist_handle() :: atom(). +-opaque nif_resource() :: reference(). +-export_type([nif_resource/0]). +-opaque dist_handle() :: atom(). -export_type([dist_handle/0]). -type iovec() :: [binary()]. - -export_type([iovec/0]). %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -2316,7 +2316,7 @@ process_flag(_Flag, _Value) -> {min_heap_size, MinHeapSize :: non_neg_integer()} | {min_bin_vheap_size, MinBinVHeapSize :: non_neg_integer()} | {max_heap_size, MaxHeapSize :: max_heap_size()} | - {monitored_by, Pids :: [pid()]} | + {monitored_by, MonitoredBy :: [pid() | port() | nif_resource()]} | {monitors, Monitors :: [{process | port, Pid :: pid() | port() | {RegName :: atom(), Node :: node()}}]} | @@ -2681,11 +2681,12 @@ tuple_to_list(_Tuple) -> (dist_ctrl) -> {Node :: node(), ControllingEntity :: port() | pid()}; (driver_version) -> string(); - (dynamic_trace) -> none | dtrace | systemtap; + (dynamic_trace) -> none | dtrace | systemtap; (dynamic_trace_probes) -> boolean(); (end_time) -> non_neg_integer(); (elib_malloc) -> false; (eager_check_io) -> boolean(); + (ets_count) -> pos_integer(); (ets_limit) -> pos_integer(); (fullsweep_after) -> {fullsweep_after, non_neg_integer()}; (garbage_collection) -> [{atom(), integer()}]; @@ -3653,90 +3654,28 @@ memory() -> -spec erlang:memory(Type :: memory_type()) -> non_neg_integer(); (TypeList :: [memory_type()]) -> [{memory_type(), non_neg_integer()}]. memory(Type) when erlang:is_atom(Type) -> - {AA, ALCU, ChkSup, BadArgZero} = need_mem_info(Type), - case get_mem_data(ChkSup, ALCU, AA) of - notsup -> - erlang:error(notsup, [Type]); - Mem -> - Value = get_memval(Type, Mem), - case {BadArgZero, Value} of - {true, 0} -> erlang:error(badarg, [Type]); - _ -> Value - end + try + case aa_mem_data(au_mem_data(?ALL_NEEDED_ALLOCS)) of + notsup -> erlang:error(notsup); + Mem -> get_memval(Type, Mem) + end + catch + error:badarg -> erlang:error(badarg) end; memory(Types) when erlang:is_list(Types) -> - {AA, ALCU, ChkSup, BadArgZeroList} = need_mem_info_list(Types), - case get_mem_data(ChkSup, ALCU, AA) of - notsup -> - erlang:error(notsup, [Types]); - Mem -> - case memory_result_list(Types, BadArgZeroList, Mem) of - badarg -> erlang:error(badarg, [Types]); - Result -> Result - end - end. - -memory_result_list([], [], _Mem) -> - []; -memory_result_list([T|Ts], [BAZ|BAZs], Mem) -> - case memory_result_list(Ts, BAZs, Mem) of - badarg -> badarg; - TVs -> - V = get_memval(T, Mem), - case {BAZ, V} of - {true, 0} -> badarg; - _ -> [{T, V}| TVs] - end - end. - -get_mem_data(true, AlcUAllocs, NeedAllocatedAreas) -> - case memory_is_supported() of - false -> notsup; - true -> get_mem_data(false, AlcUAllocs, NeedAllocatedAreas) - end; -get_mem_data(false, AlcUAllocs, NeedAllocatedAreas) -> - AlcUMem = case AlcUAllocs of - [] -> #memory{}; - _ -> - au_mem_data(AlcUAllocs) - end, - case NeedAllocatedAreas of - true -> aa_mem_data(AlcUMem); - false -> AlcUMem + try + case aa_mem_data(au_mem_data(?ALL_NEEDED_ALLOCS)) of + notsup -> erlang:error(notsup); + Mem -> memory_1(Types, Mem) + end + catch + error:badarg -> erlang:error(badarg) end. -need_mem_info_list([]) -> - {false, [], false, []}; -need_mem_info_list([T|Ts]) -> - {MAA, MALCU, MChkSup, MBadArgZero} = need_mem_info_list(Ts), - {AA, ALCU, ChkSup, BadArgZero} = need_mem_info(T), - {case AA of - true -> true; - _ -> MAA - end, - ALCU ++ (MALCU -- ALCU), - case ChkSup of - true -> true; - _ -> MChkSup - end, - [BadArgZero|MBadArgZero]}. - -need_mem_info(Type) when Type == total; - Type == system -> - {true, ?ALL_NEEDED_ALLOCS, false, false}; -need_mem_info(Type) when Type == processes; - Type == processes_used -> - {true, [eheap_alloc, fix_alloc], true, false}; -need_mem_info(Type) when Type == atom; - Type == atom_used; - Type == code -> - {true, [], true, false}; -need_mem_info(binary) -> - {false, [binary_alloc], true, false}; -need_mem_info(ets) -> - {true, [ets_alloc], true, false}; -need_mem_info(_) -> - {false, [], false, true}. +memory_1([Type | Types], Mem) -> + [{Type, get_memval(Type, Mem)} | memory_1(Types, Mem)]; +memory_1([], _Mem) -> + []. get_memval(total, #memory{total = V}) -> V; get_memval(processes, #memory{processes = V}) -> V; @@ -3747,16 +3686,7 @@ get_memval(atom_used, #memory{atom_used = V}) -> V; get_memval(binary, #memory{binary = V}) -> V; get_memval(code, #memory{code = V}) -> V; get_memval(ets, #memory{ets = V}) -> V; -get_memval(_, #memory{}) -> 0. - -memory_is_supported() -> - {_, _, FeatureList, _} = erlang:system_info(allocator), - case ((erlang:system_info(alloc_util_allocators) - -- ?CARRIER_ALLOCS) - -- FeatureList) of - [] -> true; - _ -> false - end. +get_memval(_, #memory{}) -> erlang:error(badarg). get_blocks_size([{blocks_size, Sz, _, _} | Rest], Acc) -> get_blocks_size(Rest, Acc+Sz); @@ -3767,16 +3697,6 @@ get_blocks_size([_ | Rest], Acc) -> get_blocks_size([], Acc) -> Acc. - -blocks_size([{Carriers, SizeList} | Rest], Acc) when Carriers == mbcs; - Carriers == mbcs_pool; - Carriers == sbcs -> - blocks_size(Rest, get_blocks_size(SizeList, Acc)); -blocks_size([_ | Rest], Acc) -> - blocks_size(Rest, Acc); -blocks_size([], Acc) -> - Acc. - get_fix_proc([{ProcType, A1, U1}| Rest], {A0, U0}) when ProcType == proc; ProcType == monitor; ProcType == link; @@ -3801,64 +3721,78 @@ fix_proc([_ | Rest], Acc) -> fix_proc([], Acc) -> Acc. +au_mem_fix(#memory{ processes = Proc, + processes_used = ProcU, + system = Sys } = Mem, Data) -> + case fix_proc(Data, {0, 0}) of + {A, U} -> + Mem#memory{ processes = Proc+A, + processes_used = ProcU+U, + system = Sys-A }; + {Mask, A, U} -> + Mem#memory{ processes = Mask band (Proc+A), + processes_used = Mask band (ProcU+U), + system = Mask band (Sys-A) } + end. + +au_mem_acc(#memory{ total = Tot, + processes = Proc, + processes_used = ProcU } = Mem, + eheap_alloc, Data) -> + Sz = get_blocks_size(Data, 0), + Mem#memory{ total = Tot+Sz, + processes = Proc+Sz, + processes_used = ProcU+Sz}; +au_mem_acc(#memory{ total = Tot, + system = Sys, + ets = Ets } = Mem, ets_alloc, Data) -> + Sz = get_blocks_size(Data, 0), + Mem#memory{ total = Tot+Sz, + system = Sys+Sz, + ets = Ets+Sz }; +au_mem_acc(#memory{total = Tot, + system = Sys, + binary = Bin } = Mem, + binary_alloc, Data) -> + Sz = get_blocks_size(Data, 0), + Mem#memory{ total = Tot+Sz, + system = Sys+Sz, + binary = Bin+Sz}; +au_mem_acc(#memory{ total = Tot, + system = Sys } = Mem, + _Type, Data) -> + Sz = get_blocks_size(Data, 0), + Mem#memory{ total = Tot+Sz, + system = Sys+Sz }. + +au_mem_foreign(Mem, [{Type, SizeList} | Rest]) -> + au_mem_foreign(au_mem_acc(Mem, Type, SizeList), Rest); +au_mem_foreign(Mem, []) -> + Mem. + +au_mem_current(Mem0, Type, [{mbcs_pool, MBCS} | Rest]) -> + [Foreign] = [Foreign || {foreign_blocks, Foreign} <- MBCS], + SizeList = MBCS -- [Foreign], + Mem = au_mem_foreign(Mem0, Foreign), + au_mem_current(au_mem_acc(Mem, Type, SizeList), Type, Rest); +au_mem_current(Mem, Type, [{mbcs, SizeList} | Rest]) -> + au_mem_current(au_mem_acc(Mem, Type, SizeList), Type, Rest); +au_mem_current(Mem, Type, [{sbcs, SizeList} | Rest]) -> + au_mem_current(au_mem_acc(Mem, Type, SizeList), Type, Rest); +au_mem_current(Mem, Type, [_ | Rest]) -> + au_mem_current(Mem, Type, Rest); +au_mem_current(Mem, _Type, []) -> + Mem. + au_mem_data(notsup, _) -> notsup; au_mem_data(_, [{_, false} | _]) -> notsup; -au_mem_data(#memory{total = Tot, - processes = Proc, - processes_used = ProcU} = Mem, - [{eheap_alloc, _, Data} | Rest]) -> - Sz = blocks_size(Data, 0), - au_mem_data(Mem#memory{total = Tot+Sz, - processes = Proc+Sz, - processes_used = ProcU+Sz}, - Rest); -au_mem_data(#memory{total = Tot, - system = Sys, - ets = Ets} = Mem, - [{ets_alloc, _, Data} | Rest]) -> - Sz = blocks_size(Data, 0), - au_mem_data(Mem#memory{total = Tot+Sz, - system = Sys+Sz, - ets = Ets+Sz}, - Rest); -au_mem_data(#memory{total = Tot, - system = Sys, - binary = Bin} = Mem, - [{binary_alloc, _, Data} | Rest]) -> - Sz = blocks_size(Data, 0), - au_mem_data(Mem#memory{total = Tot+Sz, - system = Sys+Sz, - binary = Bin+Sz}, - Rest); -au_mem_data(#memory{total = Tot, - processes = Proc, - processes_used = ProcU, - system = Sys} = Mem, - [{fix_alloc, _, Data} | Rest]) -> - Sz = blocks_size(Data, 0), - case fix_proc(Data, {0, 0}) of - {A, U} -> - au_mem_data(Mem#memory{total = Tot+Sz, - processes = Proc+A, - processes_used = ProcU+U, - system = Sys+Sz-A}, - Rest); - {Mask, A, U} -> - au_mem_data(Mem#memory{total = Tot+Sz, - processes = Mask band (Proc+A), - processes_used = Mask band (ProcU+U), - system = Mask band (Sys+Sz-A)}, - Rest) - end; -au_mem_data(#memory{total = Tot, - system = Sys} = Mem, - [{_, _, Data} | Rest]) -> - Sz = blocks_size(Data, 0), - au_mem_data(Mem#memory{total = Tot+Sz, - system = Sys+Sz}, - Rest); +au_mem_data(#memory{} = Mem0, [{fix_alloc, _, Data} | Rest]) -> + Mem = au_mem_fix(Mem0, Data), + au_mem_data(au_mem_current(Mem, fix_alloc, Data), Rest); +au_mem_data(#memory{} = Mem, [{Type, _, Data} | Rest]) -> + au_mem_data(au_mem_current(Mem, Type, Data), Rest); au_mem_data(EMD, []) -> EMD. diff --git a/erts/preloaded/src/erts.app.src b/erts/preloaded/src/erts.app.src index 338f168158..1ccc4988c2 100644 --- a/erts/preloaded/src/erts.app.src +++ b/erts/preloaded/src/erts.app.src @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2013-2016. All Rights Reserved. +%% Copyright Ericsson AB 2013-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/erts/preloaded/src/init.erl b/erts/preloaded/src/init.erl index 9e3de87e08..253fcf7a1f 100644 --- a/erts/preloaded/src/init.erl +++ b/erts/preloaded/src/init.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2017. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/erts/preloaded/src/prim_file.erl b/erts/preloaded/src/prim_file.erl index 2921b9df1d..517ca74301 100644 --- a/erts/preloaded/src/prim_file.erl +++ b/erts/preloaded/src/prim_file.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2000-2017. All Rights Reserved. +%% Copyright Ericsson AB 2000-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -580,15 +580,20 @@ read_link_info(Name, Opts) -> read_info_1(Name, 0, proplist_get_value(time, Opts, local)). read_info_1(Name, FollowLinks, TimeType) -> - try read_info_nif(encode_path(Name), FollowLinks) of - {error, Reason} -> {error, Reason}; - FileInfo -> - CTime = from_posix_seconds(FileInfo#file_info.ctime, TimeType), - MTime = from_posix_seconds(FileInfo#file_info.mtime, TimeType), - ATime = from_posix_seconds(FileInfo#file_info.atime, TimeType), - {ok, FileInfo#file_info{ ctime = CTime, mtime = MTime, atime = ATime }} + try + case read_info_nif(encode_path(Name), FollowLinks) of + {error, Reason} -> + {error, Reason}; + FileInfo -> + CTime = from_posix_seconds(FileInfo#file_info.ctime, TimeType), + MTime = from_posix_seconds(FileInfo#file_info.mtime, TimeType), + ATime = from_posix_seconds(FileInfo#file_info.atime, TimeType), + {ok, FileInfo#file_info{ ctime = CTime, + mtime = MTime, + atime = ATime }} + end catch - error:badarg -> {error, badarg} + error:_ -> {error, badarg} end. write_file_info(Filename, Info) -> diff --git a/erts/preloaded/src/prim_inet.erl b/erts/preloaded/src/prim_inet.erl index 2a3605260d..bcb54cca42 100644 --- a/erts/preloaded/src/prim_inet.erl +++ b/erts/preloaded/src/prim_inet.erl @@ -1557,7 +1557,7 @@ type_opt_1(O) when is_atom(O) -> undefined. %% Get. No supplied value. type_value(get, undefined) -> false; % Undefined type -%% These two clauses can not happen since they are only used +%% These two clauses cannot happen since they are only used %% in record fields - from record fields they must have a %% value though it might be 'undefined', so record fields %% calls type_value/3, not type_value/2. @@ -1723,7 +1723,7 @@ type_value_2(_, _) -> false. %% Get. No supplied value. %% -%% These two clauses can not happen since they are only used +%% These two clauses cannot happen since they are only used %% in record fields - from record fields they must have a %% value though it might be 'undefined', so record fields %% calls enc_value/3, not enc_value/2. diff --git a/erts/preloaded/src/prim_zip.erl b/erts/preloaded/src/prim_zip.erl index 5cc15b7acd..ca5cfec0e3 100644 --- a/erts/preloaded/src/prim_zip.erl +++ b/erts/preloaded/src/prim_zip.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2016. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/erts/test/erlc_SUITE.erl b/erts/test/erlc_SUITE.erl index 622c4ec06b..0c5b9f8358 100644 --- a/erts/test/erlc_SUITE.erl +++ b/erts/test/erlc_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2016. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/erts/test/erlexec_SUITE.erl b/erts/test/erlexec_SUITE.erl index 73ed0ac56a..602dc5ce2e 100644 --- a/erts/test/erlexec_SUITE.erl +++ b/erts/test/erlexec_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2016. All Rights Reserved. +%% Copyright Ericsson AB 2007-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/erts/test/nt_SUITE.erl b/erts/test/nt_SUITE.erl index 3081b58835..b2a0445ec1 100644 --- a/erts/test/nt_SUITE.erl +++ b/erts/test/nt_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1998-2016. All Rights Reserved. +%% Copyright Ericsson AB 1998-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/erts/test/otp_SUITE.erl b/erts/test/otp_SUITE.erl index 3abe45c141..2372e8b9ac 100644 --- a/erts/test/otp_SUITE.erl +++ b/erts/test/otp_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2000-2016. All Rights Reserved. +%% Copyright Ericsson AB 2000-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/erts/test/run_erl_SUITE.erl b/erts/test/run_erl_SUITE.erl index 08edd930b4..7c6f58a93a 100644 --- a/erts/test/run_erl_SUITE.erl +++ b/erts/test/run_erl_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2016. All Rights Reserved. +%% Copyright Ericsson AB 2005-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/erts/test/upgrade_SUITE.erl b/erts/test/upgrade_SUITE.erl index 73d221cfab..f92c25bdb4 100644 --- a/erts/test/upgrade_SUITE.erl +++ b/erts/test/upgrade_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2014-2017. All Rights Reserved. +%% Copyright Ericsson AB 2014-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -25,7 +25,7 @@ -define(upgr_sname,otp_upgrade). -%% Applications that are excluded from this test because they can not +%% Applications that are excluded from this test because they cannot %% just be started in a new node with out specific configuration. -define(start_exclude, [cosEvent,cosEventDomain,cosFileTransfer,cosNotification, diff --git a/erts/test/z_SUITE.erl b/erts/test/z_SUITE.erl index bb1afecafc..6a34299dd2 100644 --- a/erts/test/z_SUITE.erl +++ b/erts/test/z_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2016. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/erts/vsn.mk b/erts/vsn.mk index 687c62343e..c0444fa483 100644 --- a/erts/vsn.mk +++ b/erts/vsn.mk @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 1997-2017. All Rights Reserved. +# Copyright Ericsson AB 1997-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -18,7 +18,7 @@ # %CopyrightEnd% # -VSN = 9.3.2 +VSN = 10.0.5 # Port number 4365 in 4.2 # Port number 4366 in 4.3 diff --git a/lib/Makefile b/lib/Makefile index 0053238826..cdb3f3f3dc 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -58,7 +58,7 @@ else SUB_DIRECTORIES = hipe parsetools asn1/src else ifdef TERTIARY_BOOTSTRAP - SUB_DIRECTORIES = snmp sasl jinterface ic syntax_tools wx + SUB_DIRECTORIES = snmp sasl jinterface syntax_tools wx else ifdef DOC_BOOTSTRAP SUB_DIRECTORIES = xmerl edoc erl_docgen diff --git a/lib/asn1/c_src/asn1_erl_nif.c b/lib/asn1/c_src/asn1_erl_nif.c index d5aaadb89b..797be6d4f8 100644 --- a/lib/asn1/c_src/asn1_erl_nif.c +++ b/lib/asn1/c_src/asn1_erl_nif.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2002-2017. All Rights Reserved. + * Copyright Ericsson AB 2002-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -949,6 +949,12 @@ static int ber_decode_value(ErlNifEnv* env, ERL_NIF_TERM *value, unsigned char * unsigned char *tmp_out_buff; ERL_NIF_TERM term = 0, curr_head = 0; + /* Recursion depth limitation, borrow a signed int: maybe_ret */ + maybe_ret = (int) (ErlNifSInt) ((char *)value - (char *)ib_index); + maybe_ret = maybe_ret < 0 ? -maybe_ret : maybe_ret; + if (maybe_ret >= sizeof(void *) * 8192) /* 8 k pointer words */ + return ASN1_ERROR; + if (((in_buf[*ib_index]) & 0x80) == ASN1_SHORT_DEFINITE_LENGTH) { len = in_buf[*ib_index]; } else if (in_buf[*ib_index] == ASN1_INDEFINITE_LENGTH) { @@ -993,7 +999,7 @@ static int ber_decode_value(ErlNifEnv* env, ERL_NIF_TERM *value, unsigned char * while (*ib_index < end_index) { if ((maybe_ret = ber_decode(env, &term, in_buf, ib_index, - in_buf_len)) <= ASN1_ERROR + *ib_index + len)) <= ASN1_ERROR ) return maybe_ret; curr_head = enif_make_list_cell(env, term, curr_head); diff --git a/lib/asn1/doc/src/Makefile b/lib/asn1/doc/src/Makefile index 2b5d9467d9..9c0d865884 100644 --- a/lib/asn1/doc/src/Makefile +++ b/lib/asn1/doc/src/Makefile @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 1997-2016. All Rights Reserved. +# Copyright Ericsson AB 1997-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -107,6 +107,7 @@ html: gifs $(HTML_REF_MAN_FILE) clean clean_docs: rm -rf $(HTMLDIR)/* + rm -rf $(XMLDIR) rm -f $(MAN3DIR)/* rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) rm -f $(GEN_XML) errs core *~ diff --git a/lib/asn1/doc/src/notes.xml b/lib/asn1/doc/src/notes.xml index bb15c9ff5f..c86fa79c2c 100644 --- a/lib/asn1/doc/src/notes.xml +++ b/lib/asn1/doc/src/notes.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2004</year><year>2017</year> + <year>2004</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -32,6 +32,43 @@ <p>This document describes the changes made to the asn1 application.</p> +<section><title>Asn1 5.0.6</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Update to use the new string api instead of the old.</p> + <p> + Own Id: OTP-15036</p> + </item> + </list> + </section> + +</section> + +<section><title>Asn1 5.0.5.1</title> + + <section><title>Known Bugs and Problems</title> + <list> + <item> + <p>A bug in ASN.1 BER decoding has been fixed. When + decoding a recursively enclosed term the length was not + propagated to that term decoding, so if the length of the + enclosed term was longer than the enclosing that error + was not dectected</p> <p>A hard coded C stack limitation + for decoding recursive ASN.1 terms has been introduced. + This is currently set to 8 kWords giving a nesting depth + of about 1000 levels. Deeper terms can not be decoded, + which should not be much of a real world limitation.</p> + <p> + Own Id: OTP-14440 Aux Id: ERIERL-220 </p> + </item> + </list> + </section> + +</section> + <section><title>Asn1 5.0.5</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/asn1/src/asn1ct.erl b/lib/asn1/src/asn1ct.erl index e9e9f6eb42..cd50d30aa8 100644 --- a/lib/asn1/src/asn1ct.erl +++ b/lib/asn1/src/asn1ct.erl @@ -2,7 +2,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2017. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/asn1/src/asn1ct_check.erl b/lib/asn1/src/asn1ct_check.erl index 321980e5e4..9ec0d93e93 100644 --- a/lib/asn1/src/asn1ct_check.erl +++ b/lib/asn1/src/asn1ct_check.erl @@ -5770,7 +5770,7 @@ format_error({missing_ocft,Component}) -> format_error(multiple_uniqs) -> "implementation limitation: only one UNIQUE field is allowed in CLASS"; format_error({namelist_redefinition,Name}) -> - io_lib:format("the name '~s' can not be redefined", [Name]); + io_lib:format("the name '~s' cannot be redefined", [Name]); format_error({param_bad_type, Ref}) -> io_lib:format("'~p' is not a parameterized type", [Ref]); format_error(param_wrong_number_of_arguments) -> diff --git a/lib/asn1/src/asn1ct_gen.erl b/lib/asn1/src/asn1ct_gen.erl index ee039dfbab..28836ff264 100644 --- a/lib/asn1/src/asn1ct_gen.erl +++ b/lib/asn1/src/asn1ct_gen.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2017. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/asn1/src/asn1rtt_real_common.erl b/lib/asn1/src/asn1rtt_real_common.erl index 81c1f54d74..0d53447eb7 100644 --- a/lib/asn1/src/asn1rtt_real_common.erl +++ b/lib/asn1/src/asn1rtt_real_common.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2012-2016. All Rights Reserved. +%% Copyright Ericsson AB 2012-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/asn1/test/asn1_SUITE.erl b/lib/asn1/test/asn1_SUITE.erl index 8cf6745103..5506923341 100644 --- a/lib/asn1/test/asn1_SUITE.erl +++ b/lib/asn1/test/asn1_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2001-2017. All Rights Reserved. +%% Copyright Ericsson AB 2001-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -132,7 +132,8 @@ groups() -> testSeq2738, % Uses 'Constructed' {group, [], [constructed, - ber_decode_error]}, + ber_decode_error, + otp_14440]}, testSeqSetIndefinite, testChoiceIndefinite, per_open_type, @@ -736,6 +737,36 @@ ber_decode_error(Config, Rule, Opts) -> asn1_test_lib:compile("Constructed", Config, [Rule|Opts]), ber_decode_error:run(Opts). +otp_14440(_Config) -> + Args = " -pa \"" ++ filename:dirname(code:which(?MODULE)) ++ "\"", + {ok,N} = slave:start(hostname(), otp_14440, Args), + Result = rpc:call(N, ?MODULE, otp_14440_decode, []), + io:format("Decode result = ~p~n", [Result]), + case Result of + {exit,{error,{asn1,{invalid_value,5}}}} -> + ok = slave:stop(N); + %% We get this if stack depth limit kicks in: + {exit,{error,{asn1,{unknown,_}}}} -> + ok = slave:stop(N); + _ -> + _ = slave:stop(N), + ?t:fail(Result) + end. +%% +otp_14440_decode() -> + Data = + iolist_to_binary( + lists:duplicate( + 32, list_to_binary(lists:duplicate(1024, 16#7f)))), + try asn1rt_nif:decode_ber_tlv(Data) of + Result -> + {unexpected_return,Result} + catch + Class:Reason -> + {Class,Reason} + end. + + h323test(Config) -> test(Config, fun h323test/3). h323test(Config, Rule, Opts) -> Files = ["H235-SECURITY-MESSAGES", "H323-MESSAGES", @@ -1350,7 +1381,7 @@ xref_export_all(_Config) -> {ok,_} = xref:q(S, Def), {ok,Unused} = xref:q(S, "X - Called - range (closure E | Called)"), xref:stop(S), - case Unused of + case Unused -- [{?MODULE,otp_14440_decode,0}] of [] -> ok; [_|_] -> @@ -1385,3 +1416,11 @@ all_called_1([F|T]) when is_atom(F) -> L ++ all_called_1(T); all_called_1([]) -> []. + +hostname() -> + hostname(atom_to_list(node())). + +hostname([$@ | Hostname]) -> + list_to_atom(Hostname); +hostname([_C | Cs]) -> + hostname(Cs). diff --git a/lib/asn1/test/asn1_SUITE_data/IN-CS-1-Datatypes.asn b/lib/asn1/test/asn1_SUITE_data/IN-CS-1-Datatypes.asn index ff0361f5c5..fb092f3f9c 100644 --- a/lib/asn1/test/asn1_SUITE_data/IN-CS-1-Datatypes.asn +++ b/lib/asn1/test/asn1_SUITE_data/IN-CS-1-Datatypes.asn @@ -1152,7 +1152,7 @@ FilteringCriteria ::= CHOICE { -- In case calledAddressValue is specified, the numbers to be filtered are from calledAddressValue
-- up to and including calledAddressValue + maximumNumberOfCounters-1.
--- The last two digits of calledAddressvalue can not exceed 100-maximumNumberOfCounters.
+-- The last two digits of calledAddressvalue cannot exceed 100-maximumNumberOfCounters.
FilteringTimeOut ::= CHOICE {
duration [0] Duration,
stopTime [1] DateAndTime
diff --git a/lib/asn1/test/testUniqueObjectSets.erl b/lib/asn1/test/testUniqueObjectSets.erl index c75a673c4b..0c61da96a1 100644 --- a/lib/asn1/test/testUniqueObjectSets.erl +++ b/lib/asn1/test/testUniqueObjectSets.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2014-2017. All Rights Reserved. +%% Copyright Ericsson AB 2014-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/asn1/test/test_modified_x420.erl b/lib/asn1/test/test_modified_x420.erl index 15f7c70978..a4b335026d 100644 --- a/lib/asn1/test/test_modified_x420.erl +++ b/lib/asn1/test/test_modified_x420.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2016. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/asn1/vsn.mk b/lib/asn1/vsn.mk index 39dfe8f4fb..30cbca3773 100644 --- a/lib/asn1/vsn.mk +++ b/lib/asn1/vsn.mk @@ -1 +1 @@ -ASN1_VSN = 5.0.5 +ASN1_VSN = 5.0.6 diff --git a/lib/common_test/doc/src/Makefile b/lib/common_test/doc/src/Makefile index 293ef591cb..4d6161d3ae 100644 --- a/lib/common_test/doc/src/Makefile +++ b/lib/common_test/doc/src/Makefile @@ -140,6 +140,7 @@ debug opt: clean clean_docs: rm -rf $(HTMLDIR)/* + rm -rf $(XMLDIR) rm -f $(MAN1DIR)/* rm -f $(MAN3DIR)/* rm -f $(MAN6DIR)/* diff --git a/lib/common_test/doc/src/ct.xml b/lib/common_test/doc/src/ct.xml index 3d35ae4f54..c0380c4142 100644 --- a/lib/common_test/doc/src/ct.xml +++ b/lib/common_test/doc/src/ct.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>2010</year><year>2017</year> + <year>2010</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -1522,7 +1522,7 @@ <v>Hours = integer()</v> <v>Mins = integer()</v> <v>Secs = integer()</v> - <v>Millisecs = integer() | float()</v> + <v>Millisecs = integer()</v> <v>Func = {M, F, A} | function()</v> <v>M = atom()</v> <v>F = atom()</v> diff --git a/lib/common_test/doc/src/ct_ftp.xml b/lib/common_test/doc/src/ct_ftp.xml index a6f01dd58e..592c5eb05d 100644 --- a/lib/common_test/doc/src/ct_ftp.xml +++ b/lib/common_test/doc/src/ct_ftp.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>2010</year><year>2016</year> + <year>2010</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/common_test/doc/src/ct_hooks.xml b/lib/common_test/doc/src/ct_hooks.xml index 954be0ffba..2f853d133d 100644 --- a/lib/common_test/doc/src/ct_hooks.xml +++ b/lib/common_test/doc/src/ct_hooks.xml @@ -368,7 +368,7 @@ <seealso marker="common_test#Module:end_per_testcase-2"><c>end_per_testcase</c></seealso> instead.</p> - <p>This function can not change the result of the test case by returning skip or fail + <p>This function cannot change the result of the test case by returning skip or fail tuples, but it may insert items in <c>Config</c> that can be read in <c>end_per_testcase/2</c> or in <c>post_end_per_testcase/5</c>.</p> diff --git a/lib/common_test/doc/src/notes.xml b/lib/common_test/doc/src/notes.xml index 7e909b24cd..df22896b27 100644 --- a/lib/common_test/doc/src/notes.xml +++ b/lib/common_test/doc/src/notes.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2004</year><year>2017</year> + <year>2004</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -33,6 +33,28 @@ <file>notes.xml</file> </header> +<section><title>Common_Test 1.16</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p>Use the compiler option <c>nowarn_export_all</c> to + disable <c>export_all</c> warnings when automatically + compiling test suites.</p> + <p> + Own Id: OTP-14810</p> + </item> + <item> + <p> + Use uri_string module instead of http_uri.</p> + <p> + Own Id: OTP-14902</p> + </item> + </list> + </section> + +</section> + <section><title>Common_Test 1.15.4</title> <section><title>Fixed Bugs and Malfunctions</title> @@ -2067,7 +2089,7 @@ ct_netconfc:close_session sometimes returned {error,closed} because the ssh connection was closed (from the server side) before the rpc-reply was received - by the client. This is normal and can not be helped. It + by the client. This is normal and cannot be helped. It has been corrected so the return will be 'ok' in this case. Other error situations will still give {error,Reason}.</p> @@ -2081,7 +2103,7 @@ {error,{process_down,Pid,normal}} because the ssh connection was closed (from the server side) before the rpc-reply was received by the client. This is normal and - can not be helped. It has been corrected so the return + cannot be helped. It has been corrected so the return will be 'ok' in this situation.</p> <p> Own Id: OTP-10570</p> @@ -2733,8 +2755,8 @@ The info function for <c>init/end_per_suite(Config)</c> is <c>init/end_per_suite()</c>, and for <c>init/end_per_group(GroupName,Config)</c> it's - <c>init/end_per_group(GroupName)</c>. Info functions can - not be used with <c>init/end_per_testcase(TestCase, + <c>init/end_per_group(GroupName)</c>. Info functions + cannot be used with <c>init/end_per_testcase(TestCase, Config)</c>, since these configuration functions execute on the test case process and will use the same properties as the test case (i.e. properties set by the test case @@ -3818,7 +3840,7 @@ If the Erlang runtime system was started without access to an erlang shell (e.g. -noshell), compilation errors would cause a crash in the Common Test application. - Without access to a shell, Common Test can not prompt the + Without access to a shell, Common Test cannot prompt the user to choose to continue or abort the test session, but must assume that the session should proceed.</p> <p> diff --git a/lib/common_test/src/Makefile b/lib/common_test/src/Makefile index 9adcf2f13b..80eaed70bd 100644 --- a/lib/common_test/src/Makefile +++ b/lib/common_test/src/Makefile @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 2003-2016. All Rights Reserved. +# Copyright Ericsson AB 2003-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. diff --git a/lib/common_test/src/common_test.app.src b/lib/common_test/src/common_test.app.src index f686003637..efebea896c 100644 --- a/lib/common_test/src/common_test.app.src +++ b/lib/common_test/src/common_test.app.src @@ -1,7 +1,7 @@ % This is an -*- erlang -*- file. %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009-2017. All Rights Reserved. +%% Copyright Ericsson AB 2009-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/common_test/src/ct_config.erl b/lib/common_test/src/ct_config.erl index e33b47b0e8..a10d939919 100644 --- a/lib/common_test/src/ct_config.erl +++ b/lib/common_test/src/ct_config.erl @@ -1,7 +1,7 @@ %%-------------------------------------------------------------------- %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2017. All Rights Reserved. +%% Copyright Ericsson AB 2010-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/common_test/src/ct_config_plain.erl b/lib/common_test/src/ct_config_plain.erl index d525019f7b..7b68ac6597 100644 --- a/lib/common_test/src/ct_config_plain.erl +++ b/lib/common_test/src/ct_config_plain.erl @@ -1,7 +1,7 @@ %%-------------------------------------------------------------------- %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2016. All Rights Reserved. +%% Copyright Ericsson AB 2010-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/common_test/src/ct_default_gl.erl b/lib/common_test/src/ct_default_gl.erl index 9ae430c546..cf1bcc058d 100644 --- a/lib/common_test/src/ct_default_gl.erl +++ b/lib/common_test/src/ct_default_gl.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2012-2016. All Rights Reserved. +%% Copyright Ericsson AB 2012-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/common_test/src/ct_framework.erl b/lib/common_test/src/ct_framework.erl index 134ae0e1cc..7e98e6395f 100644 --- a/lib/common_test/src/ct_framework.erl +++ b/lib/common_test/src/ct_framework.erl @@ -1168,7 +1168,7 @@ get_all(Mod, ConfTests) -> case code:which(Mod) of non_existing -> list_to_atom(atom_to_list(Mod)++ - " can not be compiled or loaded"); + " cannot be compiled or loaded"); _ -> list_to_atom(atom_to_list(Mod)++":all/0 is missing") end, diff --git a/lib/common_test/src/ct_release_test.erl b/lib/common_test/src/ct_release_test.erl index 9629473794..ac3dcab7c9 100644 --- a/lib/common_test/src/ct_release_test.erl +++ b/lib/common_test/src/ct_release_test.erl @@ -152,10 +152,10 @@ %% returned configuration must therefore also be returned from %% the calling `init_per_*'. %% -%% If the initialization fails, e.g. if a required release can -%% not be found, the function returns `{skip,Reason}'. In +%% If the initialization fails, e.g. if a required release +%% cannot be found, the function returns `{skip,Reason}'. In %% this case the other test support functions in this mudule -%% can not be used. +%% cannot be used. %% %% Example: %% @@ -426,7 +426,7 @@ init_upgrade_test(Level) -> case OldRel of false -> ct:log("Release ~tp is not available." - " Upgrade on '~p' level can not be tested.", + " Upgrade on '~p' level cannot be tested.", [FromVsn,Level]), undefined; _ -> diff --git a/lib/common_test/src/ct_run.erl b/lib/common_test/src/ct_run.erl index 223aa06e64..c9d406f1fd 100644 --- a/lib/common_test/src/ct_run.erl +++ b/lib/common_test/src/ct_run.erl @@ -1909,7 +1909,8 @@ auto_compile(TestSuites) -> SuiteMakeErrors = lists:flatmap(fun({TestDir,Suite} = TS) -> case run_make(suites, TestDir, - Suite, UserInclude) of + Suite, UserInclude, + [nowarn_export_all]) of {error,{make_failed,Bad}} -> [{TS,Bad}]; {error,_} -> @@ -1927,7 +1928,7 @@ auto_compile(TestSuites) -> case lists:member(Dir, Done) of false -> Failed1 = - case run_make(helpmods, Dir, Suite, UserInclude) of + case run_make(helpmods, Dir, Suite, UserInclude, []) of {error,{make_failed,BadMods}} -> [{{Dir,all},BadMods}|Failed]; {error,_} -> @@ -2669,12 +2670,12 @@ get_name(Dir) -> run_make(TestDir, Mod, UserInclude) -> - run_make(suites, TestDir, Mod, UserInclude). + run_make(suites, TestDir, Mod, UserInclude, [nowarn_export_all]). -run_make(Targets, TestDir0, Mod, UserInclude) when is_list(Mod) -> - run_make(Targets, TestDir0, list_to_atom(Mod), UserInclude); +run_make(Targets, TestDir0, Mod, UserInclude, COpts) when is_list(Mod) -> + run_make(Targets, TestDir0, list_to_atom(Mod), UserInclude, COpts); -run_make(Targets, TestDir0, Mod, UserInclude) -> +run_make(Targets, TestDir0, Mod, UserInclude, COpts) -> case locate_test_dir(TestDir0, Mod) of {ok,TestDir} -> %% send a start_make notification which may suspend @@ -2689,7 +2690,7 @@ run_make(Targets, TestDir0, Mod, UserInclude) -> XmerlInclude = get_dir(xmerl, "include"), ErlFlags = UserInclude ++ [{i,CtInclude}, {i,XmerlInclude}, - debug_info], + debug_info] ++ COpts, Result = if Mod == all ; Targets == helpmods -> case (catch ct_make:all([noexec|ErlFlags])) of diff --git a/lib/common_test/src/ct_telnet.erl b/lib/common_test/src/ct_telnet.erl index f9abecfd38..58a29edace 100644 --- a/lib/common_test/src/ct_telnet.erl +++ b/lib/common_test/src/ct_telnet.erl @@ -1085,7 +1085,7 @@ match_line(Name,Pid,Line,[Pattern|Patterns],FoundPrompt,Term,EO,RetTag) -> end; match_line(Name,Pid,Line,[],FoundPrompt,Term,EO,match) -> match_line(Name,Pid,Line,EO#eo.haltpatterns,FoundPrompt,Term,EO,halt); -%% print any terminated line that can not be matched +%% print any terminated line that cannot be matched match_line(Name,Pid,Line,[],_FoundPrompt,true,_EO,halt) -> log(name_or_pid(Name,Pid)," ~ts",[Line]), nomatch; diff --git a/lib/common_test/src/ct_webtool.erl b/lib/common_test/src/ct_webtool.erl index c7174b13f5..32d4255217 100644 --- a/lib/common_test/src/ct_webtool.erl +++ b/lib/common_test/src/ct_webtool.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2001-2017. All Rights Reserved. +%% Copyright Ericsson AB 2001-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/common_test/src/ct_webtool_sup.erl b/lib/common_test/src/ct_webtool_sup.erl index 6c6dbde0a6..04fbbf8745 100644 --- a/lib/common_test/src/ct_webtool_sup.erl +++ b/lib/common_test/src/ct_webtool_sup.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2001-2016. All Rights Reserved. +%% Copyright Ericsson AB 2001-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/common_test/src/cth_log_redirect.erl b/lib/common_test/src/cth_log_redirect.erl index 8cf0b39101..86081369b9 100644 --- a/lib/common_test/src/cth_log_redirect.erl +++ b/lib/common_test/src/cth_log_redirect.erl @@ -137,7 +137,7 @@ start_log_handler() -> init([]) -> {ok, #eh_state{log_func = tc_log_async}}. -log(#{msg:={report,Msg},meta:=#{domain:=[beam,erlang,otp,sasl]}}=Log,Config) -> +log(#{msg:={report,Msg},meta:=#{domain:=[otp,sasl]}}=Log,Config) -> case whereis(sasl_sup) of undefined -> ok; % sasl application is not started @@ -163,7 +163,7 @@ log(#{msg:={report,Msg},meta:=#{domain:=[beam,erlang,otp,sasl]}}=Log,Config) -> do_log(add_log_category(Log,sasl),Config) end end; -log(#{meta:=#{domain:=[beam,erlang,otp]}}=Log,Config) -> +log(#{meta:=#{domain:=[otp]}}=Log,Config) -> do_log(add_log_category(Log,error_logger),Config); log(#{meta:=#{domain:=_}},_) -> ok; @@ -185,21 +185,12 @@ handle_call({log,#{meta:=#{gl:=GL}},_}, _From, {reply, ok, State}; handle_call({log, - #{msg:=Msg0, - meta:=#{?MODULE:=#{category:=Category}}=Meta}=Log, + #{meta:=#{?MODULE:=#{category:=Category}}}=Log, #{formatter:={Formatter,FConfig}}}, _From, #eh_state{log_func=LogFunc}=State) -> Header = format_header(State), - Msg = - case Msg0 of - {report,R} -> - Fun=maps:get(report_cb,Meta,fun logger:format_report/1), - Fun(R); - _ -> - Msg0 - end, - String = Formatter:format(Log#{msg=>Msg},FConfig), + String = Formatter:format(Log,FConfig), case LogFunc of tc_log -> ct_logs:tc_log(Category, ?STD_IMPORTANCE, diff --git a/lib/common_test/src/test_server.erl b/lib/common_test/src/test_server.erl index 9469619aa9..a896a0551b 100644 --- a/lib/common_test/src/test_server.erl +++ b/lib/common_test/src/test_server.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2017. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/common_test/src/test_server_ctrl.erl b/lib/common_test/src/test_server_ctrl.erl index 67645cac08..34f2feb33c 100644 --- a/lib/common_test/src/test_server_ctrl.erl +++ b/lib/common_test/src/test_server_ctrl.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2002-2017. All Rights Reserved. +%% Copyright Ericsson AB 2002-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -1393,7 +1393,7 @@ temp_nodename([Chr|Base], Acc) -> %% %% Counts the test cases that are about to run and returns that number. %% If there's a conf group in TestSpec with a repeat property, the total number -%% of cases can not be calculated and NoOfCases = unknown. +%% of cases cannot be calculated and NoOfCases = unknown. count_test_cases(TopCases, SkipCases) when is_list(TopCases) -> case collect_all_cases(TopCases, SkipCases) of {error,_Why} = Error -> @@ -4906,7 +4906,7 @@ collect_files(Dir, Pattern, St, Mode) -> fullname_to_mod(Path) when is_list(Path) -> %% If this is called with a binary, then we are probably in +fnu %% mode and have found a beam file with name encoded as latin1. We - %% will let this crash since it can not work to load such a module + %% will let this crash since it cannot work to load such a module %% anyway. It should be removed or renamed! list_to_atom(filename:rootname(filename:basename(Path))). diff --git a/lib/common_test/src/test_server_node.erl b/lib/common_test/src/test_server_node.erl index 76588e6887..ea7ad8538e 100644 --- a/lib/common_test/src/test_server_node.erl +++ b/lib/common_test/src/test_server_node.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2002-2017. All Rights Reserved. +%% Copyright Ericsson AB 2002-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/common_test/src/test_server_sup.erl b/lib/common_test/src/test_server_sup.erl index 139621141d..26e7534c6c 100644 --- a/lib/common_test/src/test_server_sup.erl +++ b/lib/common_test/src/test_server_sup.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1998-2017. All Rights Reserved. +%% Copyright Ericsson AB 1998-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/common_test/src/vts.erl b/lib/common_test/src/vts.erl index 2bf3e82ade..38e549d2d6 100644 --- a/lib/common_test/src/vts.erl +++ b/lib/common_test/src/vts.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2003-2017. All Rights Reserved. +%% Copyright Ericsson AB 2003-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/common_test/test/ct_auto_clean_SUITE.erl b/lib/common_test/test/ct_auto_clean_SUITE.erl index fd81430d0d..a89c90eb79 100644 --- a/lib/common_test/test/ct_auto_clean_SUITE.erl +++ b/lib/common_test/test/ct_auto_clean_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009-2016. All Rights Reserved. +%% Copyright Ericsson AB 2009-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/common_test/test/ct_auto_clean_SUITE_data/ac_SUITE.erl b/lib/common_test/test/ct_auto_clean_SUITE_data/ac_SUITE.erl index dae7c1e22c..e779f70693 100644 --- a/lib/common_test/test/ct_auto_clean_SUITE_data/ac_SUITE.erl +++ b/lib/common_test/test/ct_auto_clean_SUITE_data/ac_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009-2016. All Rights Reserved. +%% Copyright Ericsson AB 2009-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/common_test/test/ct_auto_clean_SUITE_data/cth_auto_clean.erl b/lib/common_test/test/ct_auto_clean_SUITE_data/cth_auto_clean.erl index 137c81969d..3f8d3957cc 100644 --- a/lib/common_test/test/ct_auto_clean_SUITE_data/cth_auto_clean.erl +++ b/lib/common_test/test/ct_auto_clean_SUITE_data/cth_auto_clean.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009-2016. All Rights Reserved. +%% Copyright Ericsson AB 2009-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/common_test/test/ct_auto_compile_SUITE.erl b/lib/common_test/test/ct_auto_compile_SUITE.erl index dface99b8f..f88f13c889 100644 --- a/lib/common_test/test/ct_auto_compile_SUITE.erl +++ b/lib/common_test/test/ct_auto_compile_SUITE.erl @@ -169,7 +169,7 @@ test_events(ac_flag) -> {?eh,start_info,{1,1,3}}, {?eh,tc_start,{ct_framework,error_in_suite}}, {?eh,tc_done,{ct_framework,error_in_suite, - {failed,{error,'bad_SUITE can not be compiled or loaded'}}}}, + {failed,{error,'bad_SUITE cannot be compiled or loaded'}}}}, {?eh,tc_start,{dummy_SUITE,init_per_suite}}, {?eh,tc_done,{dummy_SUITE,init_per_suite,ok}}, {?eh,test_stats,{1,1,{1,0}}}, @@ -186,7 +186,7 @@ test_events(ac_spec) -> {?eh,start_info,{1,1,3}}, {?eh,tc_start,{ct_framework,error_in_suite}}, {?eh,tc_done,{ct_framework,error_in_suite, - {failed,{error,'bad_SUITE can not be compiled or loaded'}}}}, + {failed,{error,'bad_SUITE cannot be compiled or loaded'}}}}, {?eh,tc_start,{dummy_SUITE,init_per_suite}}, {?eh,tc_done,{dummy_SUITE,init_per_suite,ok}}, {?eh,test_stats,{1,1,{1,0}}}, diff --git a/lib/common_test/test/ct_hooks_SUITE.erl b/lib/common_test/test/ct_hooks_SUITE.erl index 6228524a88..0f5636a789 100644 --- a/lib/common_test/test/ct_hooks_SUITE.erl +++ b/lib/common_test/test/ct_hooks_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009-2017. All Rights Reserved. +%% Copyright Ericsson AB 2009-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/common_test/test/ct_hooks_SUITE_data/cth/tests/cth_log_SUITE.erl b/lib/common_test/test/ct_hooks_SUITE_data/cth/tests/cth_log_SUITE.erl index a0cd77b88b..eda190b682 100644 --- a/lib/common_test/test/ct_hooks_SUITE_data/cth/tests/cth_log_SUITE.erl +++ b/lib/common_test/test/ct_hooks_SUITE_data/cth/tests/cth_log_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2016. All Rights Reserved. +%% Copyright Ericsson AB 2010-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/common_test/test/ct_log_SUITE.erl b/lib/common_test/test/ct_log_SUITE.erl index 9655b6f09a..214eb17e1a 100644 --- a/lib/common_test/test/ct_log_SUITE.erl +++ b/lib/common_test/test/ct_log_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009-2016. All Rights Reserved. +%% Copyright Ericsson AB 2009-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/common_test/test/ct_netconfc_SUITE_data/netconfc1_SUITE.erl b/lib/common_test/test/ct_netconfc_SUITE_data/netconfc1_SUITE.erl index a2fa099a8c..0a374d7404 100644 --- a/lib/common_test/test/ct_netconfc_SUITE_data/netconfc1_SUITE.erl +++ b/lib/common_test/test/ct_netconfc_SUITE_data/netconfc1_SUITE.erl @@ -205,7 +205,7 @@ hello_required_exists(Config) -> SshDir = ?config(ssh_dir,Config), {ok,_Client1} = open_configured_success(my_named_connection,SshDir), - %% Check that same name can not be used twice + %% Check that same name cannot be used twice {error,{connection_exists,_Client1}} = ct_netconfc:open(my_named_connection,[{user_dir,SshDir}]), @@ -385,7 +385,7 @@ timeout_get(Config) -> %% received, the timeout message might already be sent when the timer %% is cancelled. This test checks that the timeout message is flushed %% from the message queue. If it isn't, the client crashes and the -%% session can not be closed afterwards. +%% session cannot be closed afterwards. %% Note that we can only hope that the test case triggers the problem %% every now and then, as it is very timing dependent... flush_timeout_get(Config) -> diff --git a/lib/common_test/test/ct_netconfc_SUITE_data/ns.erl b/lib/common_test/test/ct_netconfc_SUITE_data/ns.erl index 975f6cafcb..656fdb4a40 100644 --- a/lib/common_test/test/ct_netconfc_SUITE_data/ns.erl +++ b/lib/common_test/test/ct_netconfc_SUITE_data/ns.erl @@ -1,7 +1,7 @@ %%-------------------------------------------------------------------- %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2012-2017. All Rights Reserved. +%% Copyright Ericsson AB 2012-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/common_test/test/ct_pre_post_test_io_SUITE.erl b/lib/common_test/test/ct_pre_post_test_io_SUITE.erl index 538fd822c1..bca2d5f3de 100644 --- a/lib/common_test/test/ct_pre_post_test_io_SUITE.erl +++ b/lib/common_test/test/ct_pre_post_test_io_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2012-2016. All Rights Reserved. +%% Copyright Ericsson AB 2012-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/common_test/test/ct_priv_dir_SUITE_data/priv_dir_SUITE.erl b/lib/common_test/test/ct_priv_dir_SUITE_data/priv_dir_SUITE.erl index b1d191873d..0e10ec187d 100644 --- a/lib/common_test/test/ct_priv_dir_SUITE_data/priv_dir_SUITE.erl +++ b/lib/common_test/test/ct_priv_dir_SUITE_data/priv_dir_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009-2016. All Rights Reserved. +%% Copyright Ericsson AB 2009-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/common_test/test/ct_unicode_SUITE_data/unicode_atoms_SUITE.erl b/lib/common_test/test/ct_unicode_SUITE_data/unicode_atoms_SUITE.erl index 041c13cdbd..4f9dd20e3e 100644 --- a/lib/common_test/test/ct_unicode_SUITE_data/unicode_atoms_SUITE.erl +++ b/lib/common_test/test/ct_unicode_SUITE_data/unicode_atoms_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009-2017. All Rights Reserved. +%% Copyright Ericsson AB 2009-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/common_test/test/ct_userconfig_callback.erl b/lib/common_test/test/ct_userconfig_callback.erl index 14e3d9a688..e48d338dd5 100644 --- a/lib/common_test/test/ct_userconfig_callback.erl +++ b/lib/common_test/test/ct_userconfig_callback.erl @@ -1,7 +1,7 @@ %%-------------------------------------------------------------------- %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2016. All Rights Reserved. +%% Copyright Ericsson AB 2010-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/common_test/test/test_server_SUITE.erl b/lib/common_test/test/test_server_SUITE.erl index 05737cfac9..6e52f24025 100644 --- a/lib/common_test/test/test_server_SUITE.erl +++ b/lib/common_test/test/test_server_SUITE.erl @@ -260,7 +260,7 @@ translate_filename(Filename,EncodingOnTestNode) -> end. get_latest_run_dir(Dir) -> - %% For the time being, filelib:wildcard can not take a binary + %% For the time being, filelib:wildcard cannot take a binary %% argument, so we avoid using this here. case file:list_dir(Dir) of {ok,Files} -> @@ -315,7 +315,7 @@ generate_and_run_unicode_test(Config0,Encoding) -> DataDir = ?config(data_dir,Config0), Suite = create_unicode_test_suite(DataDir,Encoding), - %% We can not run this test on default node since it must be + %% We cannot run this test on default node since it must be %% started with correct file name mode (+fnu/+fnl). %% OBS: the node are stopped by end_per_testcase/2 Config1 = lists:keydelete(node,1,Config0), diff --git a/lib/common_test/test/test_server_test_lib.erl b/lib/common_test/test/test_server_test_lib.erl index 9ee946af0b..58b3aaee9b 100644 --- a/lib/common_test/test/test_server_test_lib.erl +++ b/lib/common_test/test/test_server_test_lib.erl @@ -22,7 +22,7 @@ -export([parse_suite/1]). -export([init/2, pre_init_per_testcase/3, post_end_per_testcase/4]). -%% for test_server_SUITE when node can not be started as slave +%% for test_server_SUITE when node cannot be started as slave -export([prepare_tester_node/2]). -include("test_server_test_lib.hrl"). diff --git a/lib/common_test/test_server/ts.erl b/lib/common_test/test_server/ts.erl index 330652e73f..179380a562 100644 --- a/lib/common_test/test_server/ts.erl +++ b/lib/common_test/test_server/ts.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2016. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/common_test/test_server/ts_autoconf_win32.erl b/lib/common_test/test_server/ts_autoconf_win32.erl index 6f6caaeb70..1179dfe0e5 100644 --- a/lib/common_test/test_server/ts_autoconf_win32.erl +++ b/lib/common_test/test_server/ts_autoconf_win32.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2016. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/common_test/test_server/ts_erl_config.erl b/lib/common_test/test_server/ts_erl_config.erl index e37fa844bb..537628e39a 100644 --- a/lib/common_test/test_server/ts_erl_config.erl +++ b/lib/common_test/test_server/ts_erl_config.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2016. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/common_test/test_server/ts_lib.erl b/lib/common_test/test_server/ts_lib.erl index da8d676b18..7c1c14f467 100644 --- a/lib/common_test/test_server/ts_lib.erl +++ b/lib/common_test/test_server/ts_lib.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2016. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/common_test/test_server/ts_run.erl b/lib/common_test/test_server/ts_run.erl index 5dbbaca916..7e12b9652c 100644 --- a/lib/common_test/test_server/ts_run.erl +++ b/lib/common_test/test_server/ts_run.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2016. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -411,9 +411,9 @@ make_common_test_args(Args0, Options0, _Vars) -> end, ConfigFiles = [{config,[filename:join(ConfigPath,File) || File <- get_config_files()]}], - io_lib:format("~100000p",[[{abort_if_missing_suites,true} | - Args0++Trace++Cover++Logdir++ - ConfigFiles++Options++TimeTrap]]). + io_lib:format("~0p",[[{abort_if_missing_suites,true} | + Args0++Trace++Cover++Logdir++ + ConfigFiles++Options++TimeTrap]]). to_list(X) when is_atom(X) -> atom_to_list(X); diff --git a/lib/common_test/vsn.mk b/lib/common_test/vsn.mk index ea3e9871cb..2dc1965878 100644 --- a/lib/common_test/vsn.mk +++ b/lib/common_test/vsn.mk @@ -1 +1 @@ -COMMON_TEST_VSN = 1.15.4 +COMMON_TEST_VSN = 1.16 diff --git a/lib/compiler/doc/src/Makefile b/lib/compiler/doc/src/Makefile index 13210de040..661415899f 100644 --- a/lib/compiler/doc/src/Makefile +++ b/lib/compiler/doc/src/Makefile @@ -89,6 +89,7 @@ debug opt: clean clean_docs: rm -rf $(HTMLDIR)/* + rm -rf $(XMLDIR) rm -f $(MAN3DIR)/* rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) rm -f errs core *~ diff --git a/lib/compiler/doc/src/compile.xml b/lib/compiler/doc/src/compile.xml index e4491288a6..1a71c83521 100644 --- a/lib/compiler/doc/src/compile.xml +++ b/lib/compiler/doc/src/compile.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1996</year><year>2017</year> + <year>1996</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/compiler/doc/src/notes.xml b/lib/compiler/doc/src/notes.xml index bc1f68337b..671126b73b 100644 --- a/lib/compiler/doc/src/notes.xml +++ b/lib/compiler/doc/src/notes.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2004</year><year>2017</year> + <year>2004</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -32,6 +32,305 @@ <p>This document describes the changes made to the Compiler application.</p> +<section><title>Compiler 7.2.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p>Fixed an issue where files compiled with the + <c>+deterministic</c> option differed if they were + compiled in a different directory but were otherwise + identical.</p> + <p> + Own Id: OTP-15204 Aux Id: ERL-679 </p> + </item> + </list> + </section> + +</section> + +<section><title>Compiler 7.2.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p>In rare cases involving matching of binary literal + strings, the compiler could optimize away code that + should be executed.</p> + <p> + Own Id: OTP-15156 Aux Id: ERL-655 </p> + </item> + <item> + <p>There could be an internal consistency check failure + when compiling code that called <c>map_get(Key, Map)</c> + and then updated the same map.</p> + <p> + Own Id: OTP-15157</p> + </item> + <item> + <p>In rare circumstances, the compiler could crash in + <c>beam_jump</c> when compiling a floating point + operation.</p> + <p> + Own Id: OTP-15166 Aux Id: ERL-660 </p> + </item> + </list> + </section> + +</section> + +<section><title>Compiler 7.2.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p>The could could crash when compiling a complicated + function that used the binary syntax.</p> + <p> + Own Id: OTP-15150 Aux Id: ERL-650 </p> + </item> + </list> + </section> + +</section> + +<section><title>Compiler 7.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p>Fixed an error in an optimization pass that caused + impossible tuple matching.</p> + <p> + Own Id: OTP-14855 Aux Id: ERL-549 </p> + </item> + <item> + <p>The exception thrown when a list comprehension was + given a non-list term was not always correct.</p> + <p> + Own Id: OTP-14992 Aux Id: ERL-572 </p> + </item> + <item> + <p>The compiler could produce incorrect code in rare + circumstances when the <c>[{inline,F/A}]</c> option was + used.</p> + <p> + Own Id: OTP-15115 Aux Id: PR-1831 </p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p>Changed the default behaviour of <c>.erlang</c> + loading: <c>.erlang</c> is no longer loaded from the + current directory. <c>c:erlangrc(PathList)</c> can be + used to search and load an <c>.erlang</c> file from user + specified directories.</p> <p><c>escript</c>, + <c>erlc</c>, <c>dialyzer</c> and <c>typer</c> no longer + load an <c>.erlang</c> at all.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-14439</p> + </item> + <item> + <p>Support for "tuple calls" have been removed from the + run-time system. Tuple calls was an undocumented and + unsupported feature which allowed the module argument for + an apply operation to be a tuple: <c>Var = dict:new(), + Var:size()</c>. This "feature" frequently caused + confusion, especially when such call failed. The + stacktrace would point out functions that don't exist in + the source code.</p> + <p>For legacy code that need to use parameterized modules + or tuple calls for some other reason, there is a new + compiler option called <c>tuple_calls</c>. When this + option is given, the compiler will generate extra code + that emulates the old behavior for calls where the module + is a variable.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-14497</p> + </item> + <item> + <p>In code such as <c>example({ok, Val}) -> {ok, + Val}.</c> a tuple would be built. The compiler will now + automatically rewrite the code to + <c>example({ok,Val}=Tuple) -> Tuple.</c> which will + reduce code size, execution time, and remove GC + pressure.</p> + <p> + Own Id: OTP-14505</p> + </item> + <item> + <p>The optimization of <c>case</c> expression where only + one of the case arms can execute successfully has been + improved.</p> + <p> + Own Id: OTP-14525</p> + </item> + <item> + <p>Some uses of binary matching has been slightly + improved, eliminating unnecessary register shuffling.</p> + <p> + Own Id: OTP-14594 Aux Id: ERL-444 </p> + </item> + <item> + <p>There is a new <c>{compile_info,Info}</c> option for + the compiler that allows BEAM-based languages such as + Elixir and LFE to add their own compiler versions.</p> + <p> + Own Id: OTP-14615 Aux Id: PR-1558 </p> + </item> + <item> + <p>Loaded BEAM code in a 64-bit system requires less + memory because of better packing of operands for + instructions.</p> + <p>These memory savings were achieved by major + improvements to the <c>beam_makeops</c> scripts used when + building the run time system and BEAM compiler. There is + also new for documentation for <c>beam_makeops</c> that + describes how new BEAM instructions and loader + transformations can be implemented. The documentation is + found in here in a source directory or git repository: + erts/emulator/internal_doc/beam_makeops.md. An online + version can be found here: + https://github.com/erlang/otp/blob/master/erts/emulator/internal_doc/beam_makeops.md</p> + <p> + Own Id: OTP-14626</p> + </item> + <item> + <p>Size calculations for binary constructions has been + somewhat optimized, producing smaller code.</p> + <p> + Own Id: OTP-14654</p> + </item> + <item> + <p>When the value returned from a '<c>catch</c>' + expression is ignored, no stacktrace will be built if an + exception is caught. That will save time and produce less + garbage. There are also some minor optimizations of + '<c>try</c>/<c>catch</c>' both in the compiler and + run-time system.</p> + <p> + Own Id: OTP-14683</p> + </item> + <item> + <p>There is a new syntax in '<c>try/catch</c>' for + retrieving the stacktrace without calling + '<c>erlang:get_stacktrace/0</c>'. See the reference + manual for a description of the new syntax. The + '<c>erlang:get_stacktrace/0</c>' BIF is now + deprecated.</p> + <p> + Own Id: OTP-14692</p> + </item> + <item> + <p>The following is an internal change in the compiler, + that is not noticeable for normal use of the compiler: + The module <c>v3_life</c> has been removed. Its + functionality has been simplified and integrated into + <c>v3_codegen</c>.</p> + <p> + Own Id: OTP-14712</p> + </item> + <item> + <p>The optimization of binary matching that delays + creation of sub binaries (see the Efficiency Guide) could + be thwarted by the argument order and could be necessary + to change the argument order. The compiler has now become + smarter and can handle any argument order.</p> + <p> + Own Id: OTP-14774</p> + </item> + <item> + <p>When the compiler was faced with complex case + expressions it would unnecessarily allocate stack + elements and shuffle data between x and y registers. + Improved code generation to only allocate a stack frame + when strictly necessary.</p> + <p> + Own Id: OTP-14808 Aux Id: ERL-514 </p> + </item> + <item> + <p>There is a new option '<c>makedep_side_effect</c>' for + the compiler and <c>-MMD</c> for '<c>erlc</c>' that + generates dependencies and continues to compile as + normal.</p> + <p> + Own Id: OTP-14830</p> + </item> + <item> + <p>When compiling modules with huge functions, the + compiler would generate a lot of atoms for its internal, + sometimes so many that the atom table would overflow. The + compiler has been rewritten to generate far less internal + atoms to avoid filling the atom table.</p> + <p> + Own Id: OTP-14968 Aux Id: ERL-563 </p> + </item> + <item> + <p>External funs with literal values for module, name, + and arity (e.g. <c>erlang:abs/1</c>) are now treated as + literals. That means more efficient code that produces + less garbage on the heap.</p> + <p> + Own Id: OTP-15003</p> + </item> + <item> + <p>Two new guards BIFs operating on maps have been added: + <c>map_get/2</c> and <c>is_map_key/2</c>. They do the + same as <c>maps:get/2</c> and <c>maps:is_key/2</c>, + respectively, except that they are allowed to be used in + guards.</p> + <p> + Own Id: OTP-15037 Aux Id: PR-1784, PR-1802 </p> + </item> + <item> + <p>A call or apply of a literal external fun will be + replaced with a direct call.</p> + <p> + Own Id: OTP-15044 Aux Id: ERL-614 </p> + </item> + <item> + <p>Part of EEP-44 has been implemented.</p> + <p>There is a new predefined macro called + <c>OTP_RELEASE</c> which is an integer indicating the OTP + release number (its value is <c>21</c> in this + release).</p> + <p>There are new preprocessor directives + <c>-if(Condition).</c> and <c>-elif(Condition).</c>. The + <c>if/elif</c> supports the builtin function + <c>defined(Symbol)</c>.</p> + <p> + Own Id: OTP-15087 Aux Id: PR-1810 </p> + </item> + </list> + </section> + +</section> + +<section><title>Compiler 7.1.5.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p>Fixed an issue where files compiled with the + <c>+deterministic</c> option differed if they were + compiled in a different directory but were otherwise + identical.</p> + <p> + Own Id: OTP-15204 Aux Id: ERL-679 </p> + </item> + </list> + </section> + +</section> + <section><title>Compiler 7.1.5</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/compiler/src/Makefile b/lib/compiler/src/Makefile index c81b81e82b..2408c76b48 100644 --- a/lib/compiler/src/Makefile +++ b/lib/compiler/src/Makefile @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 1996-2017. All Rights Reserved. +# Copyright Ericsson AB 1996-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. diff --git a/lib/compiler/src/beam_a.erl b/lib/compiler/src/beam_a.erl index 91acb19971..6fd4ace540 100644 --- a/lib/compiler/src/beam_a.erl +++ b/lib/compiler/src/beam_a.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2012-2016. All Rights Reserved. +%% Copyright Ericsson AB 2012-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/compiler/src/beam_block.erl b/lib/compiler/src/beam_block.erl index 8cd271e1dc..fe43163455 100644 --- a/lib/compiler/src/beam_block.erl +++ b/lib/compiler/src/beam_block.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2016. All Rights Reserved. +%% Copyright Ericsson AB 1999-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/compiler/src/beam_bs.erl b/lib/compiler/src/beam_bs.erl index 14cedbb582..5f1b9ed488 100644 --- a/lib/compiler/src/beam_bs.erl +++ b/lib/compiler/src/beam_bs.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2016. All Rights Reserved. +%% Copyright Ericsson AB 1999-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/compiler/src/beam_bsm.erl b/lib/compiler/src/beam_bsm.erl index 9f3b9d788f..abc6e96c85 100644 --- a/lib/compiler/src/beam_bsm.erl +++ b/lib/compiler/src/beam_bsm.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2016. All Rights Reserved. +%% Copyright Ericsson AB 2007-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -310,7 +310,18 @@ btb_reaches_match_2([{test,bs_start_match2,{f,F},Live,[Bin,_],Ctx}|Is], end; btb_reaches_match_2([{test,_,{f,F},Ss}=I|Is], Regs, D0) -> btb_ensure_not_used(Ss, I, Regs), - D = btb_follow_branch(F, Regs, D0), + D1 = btb_follow_branch(F, Regs, D0), + D = case Is of + [{bs_context_to_binary,_}|_] -> + %% bs_context_to_binary following a test instruction + %% probably needs the current position to be saved as + %% the new start position, but we can't be sure. + %% Therefore, conservatively disable the optimization + %% (instead of forcing a saving of the position). + D1#btb{must_save=true,must_not_save=true}; + _ -> + D1 + end, btb_reaches_match_1(Is, Regs, D); btb_reaches_match_2([{test,_,{f,F},_,Ss,_}=I|Is], Regs, D0) -> btb_ensure_not_used(Ss, I, Regs), diff --git a/lib/compiler/src/beam_clean.erl b/lib/compiler/src/beam_clean.erl index 955c128699..207f1c4deb 100644 --- a/lib/compiler/src/beam_clean.erl +++ b/lib/compiler/src/beam_clean.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2000-2016. All Rights Reserved. +%% Copyright Ericsson AB 2000-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/compiler/src/beam_dead.erl b/lib/compiler/src/beam_dead.erl index 762c7bdf9e..efad082152 100644 --- a/lib/compiler/src/beam_dead.erl +++ b/lib/compiler/src/beam_dead.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2002-2016. All Rights Reserved. +%% Copyright Ericsson AB 2002-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/compiler/src/beam_disasm.erl b/lib/compiler/src/beam_disasm.erl index a68c4b5367..6cee9acae4 100644 --- a/lib/compiler/src/beam_disasm.erl +++ b/lib/compiler/src/beam_disasm.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2000-2016. All Rights Reserved. +%% Copyright Ericsson AB 2000-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/compiler/src/beam_disasm.hrl b/lib/compiler/src/beam_disasm.hrl index c3326c15a0..e8ebfc4cfc 100644 --- a/lib/compiler/src/beam_disasm.hrl +++ b/lib/compiler/src/beam_disasm.hrl @@ -2,7 +2,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2016. All Rights Reserved. +%% Copyright Ericsson AB 2007-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/compiler/src/beam_except.erl b/lib/compiler/src/beam_except.erl index abd39c661d..05c0f4fbc7 100644 --- a/lib/compiler/src/beam_except.erl +++ b/lib/compiler/src/beam_except.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2011-2016. All Rights Reserved. +%% Copyright Ericsson AB 2011-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/compiler/src/beam_flatten.erl b/lib/compiler/src/beam_flatten.erl index c60211f516..20bd23a912 100644 --- a/lib/compiler/src/beam_flatten.erl +++ b/lib/compiler/src/beam_flatten.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2016. All Rights Reserved. +%% Copyright Ericsson AB 1999-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/compiler/src/beam_jump.erl b/lib/compiler/src/beam_jump.erl index 576d20505d..43084ad588 100644 --- a/lib/compiler/src/beam_jump.erl +++ b/lib/compiler/src/beam_jump.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2016. All Rights Reserved. +%% Copyright Ericsson AB 1999-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -156,41 +156,51 @@ function({function,Name,Arity,CLabel,Asm0}) -> %%% share(Is0) -> - %% We will get more sharing if we never fall through to a label. - Is = eliminate_fallthroughs(Is0, []), - share_1(Is, #{}, [], []). + Is1 = eliminate_fallthroughs(Is0, []), + Is2 = find_fixpoint(fun(Is) -> + share_1(Is, #{}, #{}, [], []) + end, Is1), + reverse(Is2). -share_1([{label,L}=Lbl|Is], Dict0, [_|_]=Seq, Acc) -> +share_1([{label,L}=Lbl|Is], Dict0, Lbls0, [_|_]=Seq, Acc) -> case maps:find(Seq, Dict0) of - error -> - Dict = maps:put(Seq, L, Dict0), - share_1(Is, Dict, [], [Lbl|Seq ++ Acc]); - {ok,Label} -> - share_1(Is, Dict0, [], [Lbl,{jump,{f,Label}}|Acc]) + error -> + Dict = maps:put(Seq, L, Dict0), + share_1(Is, Dict, Lbls0, [], [[Lbl|Seq]|Acc]); + {ok,Label} -> + Lbls = maps:put(L, Label, Lbls0), + share_1(Is, Dict0, Lbls, [], [[Lbl,{jump,{f,Label}}]|Acc]) end; -share_1([{func_info,_,_,_}=I|Is], _, [], Acc) -> - reverse(Is, [I|Acc]); -share_1([{'catch',_,_}=I|Is], Dict0, Seq, Acc) -> - Dict = clean_non_sharable(Dict0), - share_1(Is, Dict, [I|Seq], Acc); -share_1([{'try',_,_}=I|Is], Dict0, Seq, Acc) -> - Dict = clean_non_sharable(Dict0), - share_1(Is, Dict, [I|Seq], Acc); -share_1([{try_case,_}=I|Is], Dict0, Seq, Acc) -> - Dict = clean_non_sharable(Dict0), - share_1(Is, Dict, [I|Seq], Acc); -share_1([{catch_end,_}=I|Is], Dict0, Seq, Acc) -> - Dict = clean_non_sharable(Dict0), - share_1(Is, Dict, [I|Seq], Acc); -share_1([I|Is], Dict, Seq, Acc) -> +share_1([{func_info,_,_,_}|_]=Is0, _, Lbls, [], Acc0) when Lbls =/= #{} -> + lists:foldl(fun(Is, Acc) -> + beam_utils:replace_labels(Is, Acc, Lbls, fun(Old) -> Old end) + end, Is0, Acc0); +share_1([{func_info,_,_,_}|_]=Is, _, Lbls, [], Acc) when Lbls =:= #{} -> + lists:foldl(fun lists:reverse/2, Is, Acc); +share_1([{'catch',_,_}=I|Is], Dict0, Lbls0, Seq, Acc) -> + {Dict,Lbls} = clean_non_sharable(Dict0, Lbls0), + share_1(Is, Dict, Lbls, [I|Seq], Acc); +share_1([{'try',_,_}=I|Is], Dict0, Lbls0, Seq, Acc) -> + {Dict,Lbls} = clean_non_sharable(Dict0, Lbls0), + share_1(Is, Dict, Lbls, [I|Seq], Acc); +share_1([{try_case,_}=I|Is], Dict0, Lbls0, Seq, Acc) -> + {Dict,Lbls} = clean_non_sharable(Dict0, Lbls0), + share_1(Is, Dict, Lbls, [I|Seq], Acc); +share_1([{catch_end,_}=I|Is], Dict0, Lbls0, Seq, Acc) -> + {Dict,Lbls} = clean_non_sharable(Dict0, Lbls0), + share_1(Is, Dict, Lbls, [I|Seq], Acc); +share_1([{jump,{f,To}}=I,{label,L}=Lbl|Is], Dict0, Lbls0, _Seq, Acc) -> + Lbls = maps:put(L, To, Lbls0), + share_1(Is, Dict0, Lbls, [], [[Lbl,I]|Acc]); +share_1([I|Is], Dict, Lbls, Seq, Acc) -> case is_unreachable_after(I) of false -> - share_1(Is, Dict, [I|Seq], Acc); + share_1(Is, Dict, Lbls, [I|Seq], Acc); true -> - share_1(Is, Dict, [I], Acc) + share_1(Is, Dict, Lbls, [I], Acc) end. -clean_non_sharable(Dict) -> +clean_non_sharable(Dict0, Lbls0) -> %% We are passing in or out of a 'catch' or 'try' block. Remove %% sequences that should not be shared over the boundaries of the %% block. Since the end of the sequence must match, the only @@ -198,7 +208,17 @@ clean_non_sharable(Dict) -> %% the 'catch'/'try' block is a sequence that ends with an %% instruction that causes an exception. Any sequence that causes %% an exception must contain a line/1 instruction. - maps:filter(fun(K, _V) -> sharable_with_try(K) end, Dict). + Dict1 = maps:to_list(Dict0), + Lbls1 = maps:to_list(Lbls0), + {Dict2,Lbls2} = foldl(fun({K, V}, {Dict,Lbls}) -> + case sharable_with_try(K) of + true -> + {[{K,V}|Dict],lists:keydelete(V, 2, Lbls)}; + false -> + {Dict,Lbls} + end + end, {[],Lbls1}, Dict1), + {maps:from_list(Dict2),maps:from_list(Lbls2)}. sharable_with_try([{line,_}|_]) -> %% This sequence may cause an exception and may potentially diff --git a/lib/compiler/src/beam_listing.erl b/lib/compiler/src/beam_listing.erl index 73c6501fe5..518b958794 100644 --- a/lib/compiler/src/beam_listing.erl +++ b/lib/compiler/src/beam_listing.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2016. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/compiler/src/beam_peep.erl b/lib/compiler/src/beam_peep.erl index 920fb00397..2b8dd40e29 100644 --- a/lib/compiler/src/beam_peep.erl +++ b/lib/compiler/src/beam_peep.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2016. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/compiler/src/beam_receive.erl b/lib/compiler/src/beam_receive.erl index 3c8efa577c..ddbe67605a 100644 --- a/lib/compiler/src/beam_receive.erl +++ b/lib/compiler/src/beam_receive.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2016. All Rights Reserved. +%% Copyright Ericsson AB 2010-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/compiler/src/beam_reorder.erl b/lib/compiler/src/beam_reorder.erl index 63bb57a1ac..8d2ef5a431 100644 --- a/lib/compiler/src/beam_reorder.erl +++ b/lib/compiler/src/beam_reorder.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2016. All Rights Reserved. +%% Copyright Ericsson AB 1999-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/compiler/src/beam_split.erl b/lib/compiler/src/beam_split.erl index 52dd89b5bb..809e49b3d0 100644 --- a/lib/compiler/src/beam_split.erl +++ b/lib/compiler/src/beam_split.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2011-2016. All Rights Reserved. +%% Copyright Ericsson AB 2011-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/compiler/src/beam_type.erl b/lib/compiler/src/beam_type.erl index 12da8c9446..b5c979e529 100644 --- a/lib/compiler/src/beam_type.erl +++ b/lib/compiler/src/beam_type.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2017. All Rights Reserved. +%% Copyright Ericsson AB 1999-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -559,7 +559,7 @@ update({bs_save2,_,_}, Ts) -> update({bs_restore2,_,_}, Ts) -> Ts; update({bs_context_to_binary,Dst}, Ts) -> - tdb_store(Dst, {binary,1}, Ts); + tdb_store(Dst, any, Ts); update({test,bs_start_match2,_,_,[Src,_],Dst}, Ts0) -> Ts = tdb_meet(Src, {binary,1}, Ts0), tdb_copy(Src, Dst, Ts); diff --git a/lib/compiler/src/beam_utils.erl b/lib/compiler/src/beam_utils.erl index f57a7af1ab..3bb671f034 100644 --- a/lib/compiler/src/beam_utils.erl +++ b/lib/compiler/src/beam_utils.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2016. All Rights Reserved. +%% Copyright Ericsson AB 2007-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -355,6 +355,9 @@ split_even(Rs) -> split_even(Rs, [], []). %% exit BIF will raise an exception %% used - Reg is used +check_liveness({fr,_}, _, St) -> + %% Conservatively always consider the floating point register used. + {used,St}; check_liveness(R, [{block,Blk}|Is], St0) -> case check_liveness_block(R, Blk, St0) of {transparent,St1} -> @@ -1105,8 +1108,12 @@ defs([{bif,_,{f,Fail},_Src,Dst}=I|Is], Regs0, D) -> defs([{block,Block0}|Is], Regs0, D0) -> {Block,Regs,D} = defs_list(Block0, Regs0, D0), [{block,[make_anno({def,Regs0})|Block]}|defs(Is, Regs, D)]; -defs([{bs_init,{f,L},_,_,_,Dst}=I|Is], Regs0, D) -> - Regs = def_regs([Dst], Regs0), +defs([{bs_init,{f,L},_,Live,_,Dst}=I|Is], Regs0, D) -> + Regs1 = case Live of + none -> Regs0; + _ -> init_def_regs(Live) + end, + Regs = def_regs([Dst], Regs1), [I|defs(Is, Regs, update_regs(L, Regs, D))]; defs([{bs_put,{f,L},_,_}=I|Is], Regs, D) -> [I|defs(Is, Regs, update_regs(L, Regs, D))]; diff --git a/lib/compiler/src/beam_validator.erl b/lib/compiler/src/beam_validator.erl index 86aa12e19b..c09dbadd7f 100644 --- a/lib/compiler/src/beam_validator.erl +++ b/lib/compiler/src/beam_validator.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2017. All Rights Reserved. +%% Copyright Ericsson AB 2004-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -280,7 +280,7 @@ valfun_1({try_case_end,Src}, Vst) -> verify_y_init(Vst), assert_term(Src, Vst), kill_state(Vst); -%% Instructions that can not cause exceptions +%% Instructions that cannot cause exceptions valfun_1({bs_context_to_binary,Ctx}, #vst{current=#st{x=Xs}}=Vst) -> case Ctx of {Tag,X} when Tag =:= x; Tag =:= y -> @@ -451,6 +451,19 @@ valfun_1({try_case,Reg}, #vst{current=#st{ct=[Fail|Fails]}}=Vst0) -> Type -> error({bad_type,Type}) end; +valfun_1({get_list,Src,D1,D2}, Vst0) -> + assert_type(cons, Src, Vst0), + Vst = set_type_reg(term, Src, D1, Vst0), + set_type_reg(term, Src, D2, Vst); +valfun_1({get_hd,Src,Dst}, Vst) -> + assert_type(cons, Src, Vst), + set_type_reg(term, Src, Dst, Vst); +valfun_1({get_tl,Src,Dst}, Vst) -> + assert_type(cons, Src, Vst), + set_type_reg(term, Src, Dst, Vst); +valfun_1({get_tuple_element,Src,I,Dst}, Vst) -> + assert_type({tuple_element,I+1}, Src, Vst), + set_type_reg(term, Src, Dst, Vst); valfun_1(I, Vst) -> valfun_2(I, Vst). @@ -546,6 +559,12 @@ valfun_4({bif,raise,{f,0},Src,_Dst}, Vst) -> kill_state(Vst); valfun_4(raw_raise=I, Vst) -> call(I, 3, Vst); +valfun_4({bif,map_get,{f,Fail},[_Key,Map]=Src,Dst}, Vst0) -> + validate_src(Src, Vst0), + Vst1 = branch_state(Fail, Vst0), + Vst = set_type(map, Map, Vst1), + Type = propagate_fragility(term, Src, Vst), + set_type_reg(Type, Dst, Vst); valfun_4({bif,Op,{f,Fail},Src,Dst}, Vst0) -> validate_src(Src, Vst0), Vst = branch_state(Fail, Vst0), @@ -603,19 +622,6 @@ valfun_4({select_val,Src,{f,Fail},{list,Choices}}, Vst) -> valfun_4({select_tuple_arity,Tuple,{f,Fail},{list,Choices}}, Vst) -> assert_type(tuple, Tuple, Vst), kill_state(branch_arities(Choices, Tuple, branch_state(Fail, Vst))); -valfun_4({get_list,Src,D1,D2}, Vst0) -> - assert_type(cons, Src, Vst0), - Vst = set_type_reg(term, Src, D1, Vst0), - set_type_reg(term, Src, D2, Vst); -valfun_4({get_hd,Src,Dst}, Vst) -> - assert_type(cons, Src, Vst), - set_type_reg(term, Src, Dst, Vst); -valfun_4({get_tl,Src,Dst}, Vst) -> - assert_type(cons, Src, Vst), - set_type_reg(term, Src, Dst, Vst); -valfun_4({get_tuple_element,Src,I,Dst}, Vst) -> - assert_type({tuple_element,I+1}, Src, Vst), - set_type_reg(term, Src, Dst, Vst); %% New bit syntax matching instructions. valfun_4({test,bs_start_match2,{f,Fail},Live,[Ctx,NeedSlots],Ctx}, Vst0) -> diff --git a/lib/compiler/src/beam_z.erl b/lib/compiler/src/beam_z.erl index 6c3a6995d7..1c9d762eb1 100644 --- a/lib/compiler/src/beam_z.erl +++ b/lib/compiler/src/beam_z.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2012-2016. All Rights Reserved. +%% Copyright Ericsson AB 2012-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/compiler/src/compile.erl b/lib/compiler/src/compile.erl index a37b2064b2..e1c1f7338e 100644 --- a/lib/compiler/src/compile.erl +++ b/lib/compiler/src/compile.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2017. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -1432,16 +1432,30 @@ encrypt_debug_info(DebugInfo, Key, Opts) -> end. cleanup_compile_options(Opts) -> - lists:filter(fun keep_compile_option/1, Opts). - + IsDeterministic = lists:member(deterministic, Opts), + lists:filter(fun(Opt) -> + keep_compile_option(Opt, IsDeterministic) + end, Opts). + +%% Include paths and current directory don't affect compilation, but they might +%% be helpful so we include them unless we're doing a deterministic build. +keep_compile_option({i, _}, Deterministic) -> + not Deterministic; +keep_compile_option({cwd, _}, Deterministic) -> + not Deterministic; %% We are storing abstract, not asm or core. -keep_compile_option(from_asm) -> false; -keep_compile_option(from_core) -> false; +keep_compile_option(from_asm, _Deterministic) -> + false; +keep_compile_option(from_core, _Deterministic) -> + false; %% Parse transform and macros have already been applied. -keep_compile_option({parse_transform, _}) -> false; -keep_compile_option({d, _, _}) -> false; +keep_compile_option({parse_transform, _}, _Deterministic) -> + false; +keep_compile_option({d, _, _}, _Deterministic) -> + false; %% Do not affect compilation result on future calls. -keep_compile_option(Option) -> effects_code_generation(Option). +keep_compile_option(Option, _Deterministic) -> + effects_code_generation(Option). start_crypto() -> try crypto:start() of @@ -1589,6 +1603,7 @@ effects_code_generation(Option) -> binary -> false; verbose -> false; {cwd,_} -> false; + {i,_} -> false; {outdir, _} -> false; _ -> true end. diff --git a/lib/compiler/src/core_lint.erl b/lib/compiler/src/core_lint.erl index 6ded8fe78f..3f69cb03a9 100644 --- a/lib/compiler/src/core_lint.erl +++ b/lib/compiler/src/core_lint.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2015. All Rights Reserved. +%% Copyright Ericsson AB 1999-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/compiler/src/core_parse.yrl b/lib/compiler/src/core_parse.yrl index 11c4cd8b50..10d8c44dd3 100644 --- a/lib/compiler/src/core_parse.yrl +++ b/lib/compiler/src/core_parse.yrl @@ -1,7 +1,7 @@ %% -*-Erlang-*- %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2016. All Rights Reserved. +%% Copyright Ericsson AB 1999-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/compiler/src/core_pp.erl b/lib/compiler/src/core_pp.erl index f247722b4c..cb3f24fd08 100644 --- a/lib/compiler/src/core_pp.erl +++ b/lib/compiler/src/core_pp.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2017. All Rights Reserved. +%% Copyright Ericsson AB 1999-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/compiler/src/erl_bifs.erl b/lib/compiler/src/erl_bifs.erl index a7452aebc8..71ab0e872a 100644 --- a/lib/compiler/src/erl_bifs.erl +++ b/lib/compiler/src/erl_bifs.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2001-2017. All Rights Reserved. +%% Copyright Ericsson AB 2001-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -91,6 +91,7 @@ is_pure(erlang, is_bitstring, 1) -> true; %% erlang:is_builtin/3 depends on the state (i.e. the version of the emulator). is_pure(erlang, is_float, 1) -> true; is_pure(erlang, is_function, 1) -> true; +is_pure(erlang, is_function, 2) -> true; is_pure(erlang, is_integer, 1) -> true; is_pure(erlang, is_list, 1) -> true; is_pure(erlang, is_map, 1) -> true; diff --git a/lib/compiler/src/genop.tab b/lib/compiler/src/genop.tab index a47d4e8cf7..02dead9e92 100755 --- a/lib/compiler/src/genop.tab +++ b/lib/compiler/src/genop.tab @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 1998-2017. All Rights Reserved. +# Copyright Ericsson AB 1998-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. diff --git a/lib/compiler/src/sys_core_alias.erl b/lib/compiler/src/sys_core_alias.erl index 1bce1577d1..3326c6a2a8 100644 --- a/lib/compiler/src/sys_core_alias.erl +++ b/lib/compiler/src/sys_core_alias.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2016. All Rights Reserved. +%% Copyright Ericsson AB 1999-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/compiler/src/sys_core_bsm.erl b/lib/compiler/src/sys_core_bsm.erl index 65580f79e3..d7b26c3a56 100644 --- a/lib/compiler/src/sys_core_bsm.erl +++ b/lib/compiler/src/sys_core_bsm.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2017. All Rights Reserved. +%% Copyright Ericsson AB 2017-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/compiler/src/sys_core_dsetel.erl b/lib/compiler/src/sys_core_dsetel.erl index 9e2df69b33..9ab83c210f 100644 --- a/lib/compiler/src/sys_core_dsetel.erl +++ b/lib/compiler/src/sys_core_dsetel.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2002-2016. All Rights Reserved. +%% Copyright Ericsson AB 2002-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/compiler/src/sys_core_fold.erl b/lib/compiler/src/sys_core_fold.erl index 47042c2393..d848cd8f19 100644 --- a/lib/compiler/src/sys_core_fold.erl +++ b/lib/compiler/src/sys_core_fold.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2017. All Rights Reserved. +%% Copyright Ericsson AB 1999-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -99,7 +99,7 @@ t=#{} :: map(), %Types in_guard=false}). %In guard or not. --type type_info() :: cerl:cerl() | 'bool' | 'integer'. +-type type_info() :: cerl:cerl() | 'bool' | 'integer' | {'fun', pos_integer()}. -type yes_no_maybe() :: 'yes' | 'no' | 'maybe'. -type sub() :: #sub{}. @@ -115,13 +115,6 @@ module(#c_module{defs=Ds0}=Mod, Opts) -> {ok,Mod#c_module{defs=Ds1},get_warnings()}. function_1({#c_var{name={F,Arity}}=Name,B0}) -> - %% Find a suitable starting value for the variable counter. Note - %% that this pass assumes that new_var_name/1 returns a variable - %% name distinct from any variable used in the entire body of - %% the function. We use integers as variable names to avoid - %% filling up the atom table when compiling huge functions. - Count = cerl_trees:next_free_variable_name(B0), - put(new_var_num, Count), try %% Find a suitable starting value for the variable %% counter. Note that this pass assumes that new_var_name/1 @@ -352,7 +345,12 @@ expr(#c_letrec{body=#c_var{}}=Letrec, effect, _Sub) -> void(); expr(#c_letrec{defs=Fs0,body=B0}=Letrec, Ctxt, Sub) -> Fs1 = map(fun ({Name,Fb}) -> - {Name,expr(Fb, {letrec,Ctxt}, Sub)} + case Ctxt =:= effect andalso is_fun_effect_safe(Name, B0) of + true -> + {Name,expr(Fb, {letrec, effect}, Sub)}; + false -> + {Name,expr(Fb, {letrec, value}, Sub)} + end end, Fs0), B1 = body(B0, Ctxt, Sub), Letrec#c_letrec{defs=Fs1,body=B1}; @@ -480,9 +478,101 @@ expr(#c_try{anno=A,arg=E0,vars=Vs0,body=B0,evars=Evs0,handler=H0}=Try, _, Sub0) false -> {Evs1,Sub2} = var_list(Evs0, Sub0), H1 = body(H0, value, Sub2), - Try#c_try{arg=E1,vars=Vs1,body=B1,evars=Evs1,handler=H1} + H2 = opt_try_handler(H1, lists:last(Evs1)), + Try#c_try{arg=E1,vars=Vs1,body=B1,evars=Evs1,handler=H2} end. +%% Attempts to convert old erlang:get_stacktrace/0 calls into the new +%% three-argument catch, with possibility of further optimisations. +opt_try_handler(#c_call{anno=A,module=#c_literal{val=erlang},name=#c_literal{val=get_stacktrace},args=[]}, Var) -> + #c_primop{anno=A,name=#c_literal{val=build_stacktrace},args=[Var]}; +opt_try_handler(#c_case{clauses=Cs0} = Case, Var) -> + Cs = [C#c_clause{body=opt_try_handler(B, Var)} || #c_clause{body=B} = C <- Cs0], + Case#c_case{clauses=Cs}; +opt_try_handler(#c_let{arg=Arg} = Let, Var) -> + Let#c_let{arg=opt_try_handler(Arg, Var)}; +opt_try_handler(X, _) -> X. + +%% If a fun or its application is used as an argument, then it's unsafe to +%% handle it in effect context as the side-effects may rely on its return +%% value. The following is a minimal example of where it can go wrong: +%% +%% do letrec 'f'/0 = fun () -> ... whatever ... +%% in call 'side':'effect'(apply 'f'/0()) +%% 'ok' +%% +%% This function returns 'true' if Body definitely does not rely on a +%% value produced by FVar, or 'false' if Body depends on or might depend on +%% a value produced by FVar. + +is_fun_effect_safe(#c_var{}=FVar, Body) -> + ifes_1(FVar, Body, true). + +ifes_1(FVar, #c_alias{pat=Pat}, _Safe) -> + ifes_1(FVar, Pat, false); +ifes_1(FVar, #c_apply{op=Op,args=Args}, Safe) -> + %% FVar(...) is safe as long its return value is ignored, but it's never + %% okay to pass FVar as an argument. + ifes_list(FVar, Args, false) andalso ifes_1(FVar, Op, Safe); +ifes_1(FVar, #c_binary{segments=Segments}, _Safe) -> + ifes_list(FVar, Segments, false); +ifes_1(FVar, #c_bitstr{val=Val,size=Size,unit=Unit}, _Safe) -> + ifes_list(FVar, [Val, Size, Unit], false); +ifes_1(FVar, #c_call{args=Args}, _Safe) -> + ifes_list(FVar, Args, false); +ifes_1(FVar, #c_case{arg=Arg,clauses=Clauses}, Safe) -> + ifes_1(FVar, Arg, false) andalso ifes_list(FVar, Clauses, Safe); +ifes_1(FVar, #c_catch{body=Body}, _Safe) -> + ifes_1(FVar, Body, false); +ifes_1(FVar, #c_clause{pats=Pats,guard=Guard,body=Body}, Safe) -> + ifes_list(FVar, Pats, false) andalso + ifes_1(FVar, Guard, false) andalso + ifes_1(FVar, Body, Safe); +ifes_1(FVar, #c_cons{hd=Hd,tl=Tl}, _Safe) -> + ifes_1(FVar, Hd, false) andalso ifes_1(FVar, Tl, false); +ifes_1(FVar, #c_fun{body=Body}, _Safe) -> + ifes_1(FVar, Body, false); +ifes_1(FVar, #c_let{arg=Arg,body=Body}, Safe) -> + ifes_1(FVar, Arg, false) andalso ifes_1(FVar, Body, Safe); +ifes_1(FVar, #c_letrec{defs=Defs,body=Body}, Safe) -> + Funs = [Fun || {_,Fun} <- Defs], + ifes_list(FVar, Funs, false) andalso ifes_1(FVar, Body, Safe); +ifes_1(_FVar, #c_literal{}, _Safe) -> + true; +ifes_1(FVar, #c_map{arg=Arg,es=Elements}, _Safe) -> + ifes_1(FVar, Arg, false) andalso ifes_list(FVar, Elements, false); +ifes_1(FVar, #c_map_pair{key=Key,val=Val}, _Safe) -> + ifes_1(FVar, Key, false) andalso ifes_1(FVar, Val, false); +ifes_1(FVar, #c_primop{args=Args}, _Safe) -> + ifes_list(FVar, Args, false); +ifes_1(FVar, #c_receive{timeout=Timeout,action=Action,clauses=Clauses}, Safe) -> + ifes_1(FVar, Timeout, false) andalso + ifes_1(FVar, Action, Safe) andalso + ifes_list(FVar, Clauses, Safe); +ifes_1(FVar, #c_seq{arg=Arg,body=Body}, Safe) -> + %% Arg of a #c_seq{} has no effect so it's okay to use FVar there even if + %% Safe=false. + ifes_1(FVar, Arg, true) andalso ifes_1(FVar, Body, Safe); +ifes_1(FVar, #c_try{arg=Arg,handler=Handler,body=Body}, Safe) -> + ifes_1(FVar, Arg, false) andalso + ifes_1(FVar, Handler, Safe) andalso + ifes_1(FVar, Body, Safe); +ifes_1(FVar, #c_tuple{es=Elements}, _Safe) -> + ifes_list(FVar, Elements, false); +ifes_1(FVar, #c_values{es=Elements}, _Safe) -> + ifes_list(FVar, Elements, false); +ifes_1(#c_var{name=Name}, #c_var{name=Name}, Safe) -> + %% It's safe to return FVar if it's unused. + Safe; +ifes_1(_FVar, #c_var{}, _Safe) -> + true. + +ifes_list(FVar, [E|Es], Safe) -> + ifes_1(FVar, E, Safe) andalso ifes_list(FVar, Es, Safe); +ifes_list(_FVar, [], _Safe) -> + true. + + expr_list(Es, Ctxt, Sub) -> [expr(E, Ctxt, Sub) || E <- Es]. @@ -883,6 +973,10 @@ fold_non_lit_args(Call, erlang, setelement, [Arg1,Arg2,Arg3], _) -> eval_setelement(Call, Arg1, Arg2, Arg3); fold_non_lit_args(Call, erlang, is_record, [Arg1,Arg2,Arg3], Sub) -> eval_is_record(Call, Arg1, Arg2, Arg3, Sub); +fold_non_lit_args(Call, erlang, is_function, [Arg1], Sub) -> + eval_is_function_1(Call, Arg1, Sub); +fold_non_lit_args(Call, erlang, is_function, [Arg1,Arg2], Sub) -> + eval_is_function_2(Call, Arg1, Arg2, Sub); fold_non_lit_args(Call, erlang, N, Args, Sub) -> NumArgs = length(Args), case erl_internal:comp_op(N, NumArgs) of @@ -898,6 +992,22 @@ fold_non_lit_args(Call, erlang, N, Args, Sub) -> end; fold_non_lit_args(Call, _, _, _, _) -> Call. +eval_is_function_1(Call, Arg1, Sub) -> + case get_type(Arg1, Sub) of + none -> Call; + {'fun',_} -> #c_literal{anno=cerl:get_ann(Call),val=true}; + _ -> #c_literal{anno=cerl:get_ann(Call),val=false} + end. + +eval_is_function_2(Call, Arg1, #c_literal{val=Arity}, Sub) + when is_integer(Arity), Arity > 0 -> + case get_type(Arg1, Sub) of + none -> Call; + {'fun',Arity} -> #c_literal{anno=cerl:get_ann(Call),val=true}; + _ -> #c_literal{anno=cerl:get_ann(Call),val=false} + end; +eval_is_function_2(Call, _Arg1, _Arg2, _Sub) -> Call. + %% Evaluate a relational operation using type information. eval_rel_op(Call, Op, [#c_var{name=V},#c_var{name=V}], _) -> Bool = erlang:Op(same, same), @@ -3105,6 +3215,10 @@ update_types_2(V, [#c_tuple{}=P], Types) -> Types#{V=>P}; update_types_2(V, [#c_literal{val=Bool}], Types) when is_boolean(Bool) -> Types#{V=>bool}; +update_types_2(V, [#c_fun{vars=Vars}], Types) -> + Types#{V=>{'fun',length(Vars)}}; +update_types_2(V, [#c_var{name={_,Arity}}], Types) -> + Types#{V=>{'fun',Arity}}; update_types_2(V, [Type], Types) when is_atom(Type) -> Types#{V=>Type}; update_types_2(_, _, Types) -> Types. @@ -3123,6 +3237,8 @@ kill_types2(V, [{_,#c_tuple{}=Tuple}=Entry|Tdb]) -> false -> [Entry|kill_types2(V, Tdb)]; true -> kill_types2(V, Tdb) end; +kill_types2(V, [{_, {'fun',_}}=Entry|Tdb]) -> + [Entry|kill_types2(V, Tdb)]; kill_types2(V, [{_,Atom}=Entry|Tdb]) when is_atom(Atom) -> [Entry|kill_types2(V, Tdb)]; kill_types2(_, []) -> []. diff --git a/lib/compiler/src/sys_core_inline.erl b/lib/compiler/src/sys_core_inline.erl index eee3594922..5a6cc45e4a 100644 --- a/lib/compiler/src/sys_core_inline.erl +++ b/lib/compiler/src/sys_core_inline.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2000-2016. All Rights Reserved. +%% Copyright Ericsson AB 2000-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/compiler/src/v3_codegen.erl b/lib/compiler/src/v3_codegen.erl index 9652a8476d..e9152ba88f 100644 --- a/lib/compiler/src/v3_codegen.erl +++ b/lib/compiler/src/v3_codegen.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2016. All Rights Reserved. +%% Copyright Ericsson AB 1999-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -1621,11 +1621,6 @@ test_cg(is_boolean, [#k_atom{val=Val}], Fail, I, Vdb, Bef, St) -> false -> [{jump,{f,Fail}}] end, {Is,Aft,St}; -test_cg(is_map_key, As, Fail, I, Vdb, Bef, St) -> - [Key,Map] = cg_reg_args(As, Bef), - Aft = clear_dead(Bef, I, Vdb), - F = {f,Fail}, - {[{test,is_map,F,[Map]},{test,has_map_fields,F,Map,{list,[Key]}}],Aft,St}; test_cg(Test, As, Fail, I, Vdb, Bef, St) -> Args = cg_reg_args(As, Bef), Aft = clear_dead(Bef, I, Vdb), diff --git a/lib/compiler/src/v3_core.erl b/lib/compiler/src/v3_core.erl index 4799105d05..3b746ab5bf 100644 --- a/lib/compiler/src/v3_core.erl +++ b/lib/compiler/src/v3_core.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2017. All Rights Reserved. +%% Copyright Ericsson AB 1999-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -1501,7 +1501,7 @@ bc_initial_size(E0, Q, St0) -> end. bc_elem_size({bin,_,El}, St0) -> - case bc_elem_size_1(El, 0, []) of + case bc_elem_size_1(El, ordsets:new(), 0, []) of {Bits,[]} -> {#c_literal{val=Bits},[],[],St0}; {Bits,Vars0} -> @@ -1515,19 +1515,33 @@ bc_elem_size(_, _) -> throw(impossible). bc_elem_size_1([{bin_element,_,{string,_,String},{integer,_,N},_}=El|Es], - Bits, Vars) -> + DefVars, Bits, SizeVars) -> U = get_unit(El), - bc_elem_size_1(Es, Bits+U*N*length(String), Vars); -bc_elem_size_1([{bin_element,_,_,{integer,_,N},_}=El|Es], Bits, Vars) -> + bc_elem_size_1(Es, DefVars, Bits+U*N*length(String), SizeVars); +bc_elem_size_1([{bin_element,_,Expr,{integer,_,N},_}=El|Es], + DefVars0, Bits, SizeVars) -> U = get_unit(El), - bc_elem_size_1(Es, Bits+U*N, Vars); -bc_elem_size_1([{bin_element,_,_,{var,_,Var},_}=El|Es], Bits, Vars) -> - U = get_unit(El), - bc_elem_size_1(Es, Bits, [{U,#c_var{name=Var}}|Vars]); -bc_elem_size_1([_|_], _, _) -> + DefVars = bc_elem_size_def_var(Expr, DefVars0), + bc_elem_size_1(Es, DefVars, Bits+U*N, SizeVars); +bc_elem_size_1([{bin_element,_,Expr,{var,_,Src},_}=El|Es], + DefVars0, Bits, SizeVars) -> + case ordsets:is_element(Src, DefVars0) of + false -> + U = get_unit(El), + DefVars = bc_elem_size_def_var(Expr, DefVars0), + bc_elem_size_1(Es, DefVars, Bits, [{U,#c_var{name=Src}}|SizeVars]); + true -> + throw(impossible) + end; +bc_elem_size_1([_|_], _, _, _) -> throw(impossible); -bc_elem_size_1([], Bits, Vars) -> - {Bits,Vars}. +bc_elem_size_1([], _DefVars, Bits, SizeVars) -> + {Bits,SizeVars}. + +bc_elem_size_def_var({var,_,Var}, DefVars) -> + ordsets:add_element(Var, DefVars); +bc_elem_size_def_var(_Expr, DefVars) -> + DefVars. bc_elem_size_combine([{U,V}|T], U, UVars, Acc) -> bc_elem_size_combine(T, U, [V|UVars], Acc); diff --git a/lib/compiler/src/v3_kernel.erl b/lib/compiler/src/v3_kernel.erl index 4e3ceedbc0..aef0b6cc9f 100644 --- a/lib/compiler/src/v3_kernel.erl +++ b/lib/compiler/src/v3_kernel.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2017. All Rights Reserved. +%% Copyright Ericsson AB 1999-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/compiler/src/v3_kernel.hrl b/lib/compiler/src/v3_kernel.hrl index 7cd30b25a8..e6f0d3c1f7 100644 --- a/lib/compiler/src/v3_kernel.hrl +++ b/lib/compiler/src/v3_kernel.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2016. All Rights Reserved. +%% Copyright Ericsson AB 1999-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/compiler/src/v3_kernel_pp.erl b/lib/compiler/src/v3_kernel_pp.erl index e9cbe81088..c12c301ee2 100644 --- a/lib/compiler/src/v3_kernel_pp.erl +++ b/lib/compiler/src/v3_kernel_pp.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2017. All Rights Reserved. +%% Copyright Ericsson AB 1999-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/compiler/test/andor_SUITE.erl b/lib/compiler/test/andor_SUITE.erl index 05c087104d..721f77f0f6 100644 --- a/lib/compiler/test/andor_SUITE.erl +++ b/lib/compiler/test/andor_SUITE.erl @@ -29,7 +29,6 @@ suite() -> [{ct_hooks,[ts_install_cth]}]. all() -> - test_lib:recompile(?MODULE), [{group,p}]. groups() -> @@ -38,6 +37,7 @@ groups() -> combined,in_case,slow_compilation]}]. init_per_suite(Config) -> + test_lib:recompile(?MODULE), Config. end_per_suite(_Config) -> diff --git a/lib/compiler/test/apply_SUITE.erl b/lib/compiler/test/apply_SUITE.erl index cca92e4713..be49cff9b9 100644 --- a/lib/compiler/test/apply_SUITE.erl +++ b/lib/compiler/test/apply_SUITE.erl @@ -29,13 +29,13 @@ suite() -> [{ct_hooks,[ts_install_cth]}]. all() -> - test_lib:recompile(?MODULE), [mfa, fun_apply]. groups() -> []. init_per_suite(Config) -> + test_lib:recompile(?MODULE), Config. end_per_suite(_Config) -> diff --git a/lib/compiler/test/beam_block_SUITE.erl b/lib/compiler/test/beam_block_SUITE.erl index 38ead96cc8..40a30b65d7 100644 --- a/lib/compiler/test/beam_block_SUITE.erl +++ b/lib/compiler/test/beam_block_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2015-2016. All Rights Reserved. +%% Copyright Ericsson AB 2015-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -31,7 +31,6 @@ suite() -> [{ct_hooks,[ts_install_cth]}]. all() -> - test_lib:recompile(?MODULE), [{group,p}]. groups() -> @@ -46,6 +45,7 @@ groups() -> ]}]. init_per_suite(Config) -> + test_lib:recompile(?MODULE), Config. end_per_suite(_Config) -> diff --git a/lib/compiler/test/beam_except_SUITE.erl b/lib/compiler/test/beam_except_SUITE.erl index 47367d6eab..1eb07c8c85 100644 --- a/lib/compiler/test/beam_except_SUITE.erl +++ b/lib/compiler/test/beam_except_SUITE.erl @@ -26,7 +26,6 @@ suite() -> [{ct_hooks,[ts_install_cth]}]. all() -> - test_lib:recompile(?MODULE), [{group,p}]. groups() -> @@ -35,6 +34,7 @@ groups() -> coverage]}]. init_per_suite(Config) -> + test_lib:recompile(?MODULE), Config. end_per_suite(_Config) -> diff --git a/lib/compiler/test/beam_jump_SUITE.erl b/lib/compiler/test/beam_jump_SUITE.erl index 088f63606c..488c30919b 100644 --- a/lib/compiler/test/beam_jump_SUITE.erl +++ b/lib/compiler/test/beam_jump_SUITE.erl @@ -27,7 +27,6 @@ suite() -> [{ct_hooks,[ts_install_cth]}]. all() -> - test_lib:recompile(?MODULE), [{group,p}]. groups() -> @@ -37,6 +36,7 @@ groups() -> ]}]. init_per_suite(Config) -> + test_lib:recompile(?MODULE), Config. end_per_suite(_Config) -> diff --git a/lib/compiler/test/beam_reorder_SUITE.erl b/lib/compiler/test/beam_reorder_SUITE.erl index 27ce51eec3..33b27b9f9f 100644 --- a/lib/compiler/test/beam_reorder_SUITE.erl +++ b/lib/compiler/test/beam_reorder_SUITE.erl @@ -26,7 +26,6 @@ suite() -> [{ct_hooks,[ts_install_cth]}]. all() -> - test_lib:recompile(?MODULE), [{group,p}]. groups() -> @@ -36,6 +35,7 @@ groups() -> ]}]. init_per_suite(Config) -> + test_lib:recompile(?MODULE), Config. end_per_suite(_Config) -> diff --git a/lib/compiler/test/beam_type_SUITE.erl b/lib/compiler/test/beam_type_SUITE.erl index 541075af2a..061076b3ff 100644 --- a/lib/compiler/test/beam_type_SUITE.erl +++ b/lib/compiler/test/beam_type_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2015-2017. All Rights Reserved. +%% Copyright Ericsson AB 2015-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -23,12 +23,11 @@ init_per_group/2,end_per_group/2, integers/1,coverage/1,booleans/1,setelement/1,cons/1, tuple/1,record_float/1,binary_float/1,float_compare/1, - arity_checks/1,elixir_binaries/1]). + arity_checks/1,elixir_binaries/1,find_best/1]). suite() -> [{ct_hooks,[ts_install_cth]}]. all() -> - test_lib:recompile(?MODULE), [{group,p}]. groups() -> @@ -43,10 +42,12 @@ groups() -> binary_float, float_compare, arity_checks, - elixir_binaries + elixir_binaries, + find_best ]}]. init_per_suite(Config) -> + test_lib:recompile(?MODULE), Config. end_per_suite(_Config) -> @@ -292,6 +293,33 @@ elixir_bitstring_3(Bar) when is_bitstring(Bar) -> list_to_bitstring(Rewrite) end/bitstring>>. +find_best(_Config) -> + ok = find_best([a], nil), + ok = find_best([<<"a">>], nil), + {error,_} = find_best([], nil), + ok. + +%% Failed because beam_type assumed that the operand +%% for bs_context_binary must be a binary. Not true! +find_best([a|Tail], Best) -> + find_best(Tail, + case Best of + X when X =:= nil orelse X =:= false -> a; + X -> X + end); +find_best([<<"a">>|Tail], Best) -> + find_best(Tail, + case Best of + X when X =:= nil orelse X =:= false -> <<"a">>; + X -> X + end); +find_best([], a) -> + ok; +find_best([], <<"a">>) -> + ok; +find_best([], nil) -> + {error,<<"should not get here">>}. + id(I) -> I. diff --git a/lib/compiler/test/beam_utils_SUITE.erl b/lib/compiler/test/beam_utils_SUITE.erl index b2a5cada3d..ac19305d69 100644 --- a/lib/compiler/test/beam_utils_SUITE.erl +++ b/lib/compiler/test/beam_utils_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2015-2016. All Rights Reserved. +%% Copyright Ericsson AB 2015-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -25,13 +25,13 @@ is_not_killed/1,is_not_used_at/1, select/1,y_catch/1,otp_8949_b/1,liveopt/1,coverage/1, y_registers/1,user_predef/1,scan_f/1,cafu/1, - receive_label/1,read_size_file_version/1,not_used/1]). + receive_label/1,read_size_file_version/1,not_used/1, + is_used_fr/1]). -export([id/1]). suite() -> [{ct_hooks,[ts_install_cth]}]. all() -> - test_lib:recompile(?MODULE), [{group,p}]. groups() -> @@ -52,10 +52,12 @@ groups() -> scan_f, cafu, read_size_file_version, - not_used + not_used, + is_used_fr ]}]. init_per_suite(Config) -> + test_lib:recompile(?MODULE), Config. end_per_suite(_Config) -> @@ -132,6 +134,15 @@ bs_init(_Config) -> <<"foo/foo">> = do_bs_init_4(<<"foo">>, true), error = do_bs_init_4([], not_boolean), + Id = 17575, + Domain = -8798798, + [<<10,1:16,Id:16/signed>>,<<8,2:16,Domain:32/signed>>] = + do_bs_init_5(#{tag=>value,id=>Id,domain=>Domain}), + {'EXIT',{{required,id},[_|_]}} = + (catch do_bs_init_5(#{tag=>value,id=>nil,domain=>Domain})), + {'EXIT',{{required,domain},[_|_]}} = + (catch do_bs_init_5(#{tag=>value,id=>Id,domain=>nil})), + ok. do_bs_init_1([?MODULE], Sz) -> @@ -189,6 +200,20 @@ do_bs_init_4(Arg1, Arg2) -> error end. +do_bs_init_5(#{tag := value, id := Id, domain := Domain}) -> + [case Id of + nil -> + error(id({required, id})); + _ -> + <<10, 1:16/signed, Id:16/signed>> + end, + case Domain of + nil -> + error(id({required, domain})); + _ -> + <<8, 2:16/signed, Domain:32/signed>> + end]. + bs_save(_Config) -> {a,30,<<>>} = do_bs_save(<<1:1,30:5>>), {b,127,<<>>} = do_bs_save(<<1:1,31:5,0:1,127:7>>), @@ -527,5 +552,24 @@ not_used_p(_C, S, K, L) when is_record(K, k) -> id(K) end. +is_used_fr(Config) -> + 1 = is_used_fr(self(), self()), + 1 = is_used_fr(self(), other), + receive 1 -> ok end, + receive 1 -> ok end, + receive 1 -> ok end, + receive 1 -> ok end, + ok. + +is_used_fr(X, Y) -> + %% beam_utils:is_used({fr,R}, Code) would crash. + _ = 0 / (X ! 1), + _ = case Y of + X -> ok; + _ -> error + end, + X ! 1. + + %% The identity function. id(I) -> I. diff --git a/lib/compiler/test/beam_validator_SUITE.erl b/lib/compiler/test/beam_validator_SUITE.erl index 41f2957dc1..d3e544a9cc 100644 --- a/lib/compiler/test/beam_validator_SUITE.erl +++ b/lib/compiler/test/beam_validator_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2017. All Rights Reserved. +%% Copyright Ericsson AB 2004-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -49,7 +49,6 @@ suite() -> {timetrap,{minutes,10}}]. all() -> - test_lib:recompile(?MODULE), [{group,p}]. groups() -> @@ -67,6 +66,7 @@ groups() -> receive_stacked]}]. init_per_suite(Config) -> + test_lib:recompile(?MODULE), Config. end_per_suite(_Config) -> diff --git a/lib/compiler/test/bif_SUITE.erl b/lib/compiler/test/bif_SUITE.erl index bba2058f2f..c4c709eb3a 100644 --- a/lib/compiler/test/bif_SUITE.erl +++ b/lib/compiler/test/bif_SUITE.erl @@ -29,7 +29,6 @@ suite() -> [{ct_hooks,[ts_install_cth]}]. all() -> - test_lib:recompile(?MODULE), [{group,p}]. groups() -> @@ -40,6 +39,7 @@ groups() -> ]}]. init_per_suite(Config) -> + test_lib:recompile(?MODULE), Config. end_per_suite(_Config) -> diff --git a/lib/compiler/test/bs_bincomp_SUITE.erl b/lib/compiler/test/bs_bincomp_SUITE.erl index dd1d245f88..a5d49020a9 100644 --- a/lib/compiler/test/bs_bincomp_SUITE.erl +++ b/lib/compiler/test/bs_bincomp_SUITE.erl @@ -26,22 +26,22 @@ init_per_group/2,end_per_group/2, byte_aligned/1,bit_aligned/1,extended_byte_aligned/1, extended_bit_aligned/1,mixed/1,filters/1,trim_coverage/1, - nomatch/1,sizes/1,general_expressions/1]). + nomatch/1,sizes/1,general_expressions/1,matched_out_size/1]). -include_lib("common_test/include/ct.hrl"). suite() -> [{ct_hooks,[ts_install_cth]}]. all() -> - test_lib:recompile(?MODULE), [byte_aligned, bit_aligned, extended_byte_aligned, extended_bit_aligned, mixed, filters, trim_coverage, - nomatch, sizes, general_expressions]. + nomatch, sizes, general_expressions, matched_out_size]. groups() -> []. init_per_suite(Config) -> + test_lib:recompile(?MODULE), Config. end_per_suite(_Config) -> @@ -338,6 +338,13 @@ general_expressions(_) -> -undef(BAD). +matched_out_size(Config) when is_list(Config) -> + <<1, 2>> = matched_out_size_1(<<4, 1:4, 4, 2:4>>), + ok. + +matched_out_size_1(Binary) -> + << <<X>> || <<S, X:S>> <= Binary>>. + cs_init() -> erts_debug:set_internal_state(available_internal_state, true), ok. diff --git a/lib/compiler/test/bs_bit_binaries_SUITE.erl b/lib/compiler/test/bs_bit_binaries_SUITE.erl index 208d8c5487..17faa012bc 100644 --- a/lib/compiler/test/bs_bit_binaries_SUITE.erl +++ b/lib/compiler/test/bs_bit_binaries_SUITE.erl @@ -34,7 +34,6 @@ suite() -> [{ct_hooks,[ts_install_cth]}]. all() -> - test_lib:recompile(?MODULE), [{group,p}]. groups() -> @@ -43,9 +42,10 @@ groups() -> asymmetric_tests,big_asymmetric_tests, binary_to_and_from_list,big_binary_to_and_from_list, send_and_receive,send_and_receive_alot]}]. - + init_per_suite(Config) -> + test_lib:recompile(?MODULE), Config. end_per_suite(_Config) -> diff --git a/lib/compiler/test/bs_construct_SUITE.erl b/lib/compiler/test/bs_construct_SUITE.erl index 7c5ad97f7e..ccc49df005 100644 --- a/lib/compiler/test/bs_construct_SUITE.erl +++ b/lib/compiler/test/bs_construct_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2017. All Rights Reserved. +%% Copyright Ericsson AB 2004-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -38,11 +38,10 @@ suite() -> [{ct_hooks,[ts_install_cth]}, {timetrap,{minutes,1}}]. -all() -> - test_lib:recompile(?MODULE), +all() -> [{group,p}]. -groups() -> +groups() -> [{p,[parallel], [two,test1,fail,float_bin,in_guard,in_catch, nasty_literals,side_effect,opt,otp_7556,float_arith, @@ -50,6 +49,7 @@ groups() -> init_per_suite(Config) -> + test_lib:recompile(?MODULE), Config. end_per_suite(_Config) -> diff --git a/lib/compiler/test/bs_match_SUITE.erl b/lib/compiler/test/bs_match_SUITE.erl index 3b6ffa8d68..7814738449 100644 --- a/lib/compiler/test/bs_match_SUITE.erl +++ b/lib/compiler/test/bs_match_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2017. All Rights Reserved. +%% Copyright Ericsson AB 2005-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -40,7 +40,7 @@ map_and_binary/1,unsafe_branch_caching/1, bad_literals/1,good_literals/1,constant_propagation/1, parse_xml/1,get_payload/1,escape/1,num_slots_different/1, - beam_bsm/1,guard/1,is_ascii/1,non_opt_eq/1]). + beam_bsm/1,guard/1,is_ascii/1,non_opt_eq/1,erl_689/1]). -export([coverage_id/1,coverage_external_ignore/2]). @@ -53,7 +53,6 @@ suite() -> {timetrap,{minutes,1}}]. all() -> - test_lib:recompile(?MODULE), [{group,p}]. groups() -> @@ -73,10 +72,11 @@ groups() -> map_and_binary,unsafe_branch_caching, bad_literals,good_literals,constant_propagation,parse_xml, get_payload,escape,num_slots_different, - beam_bsm,guard,is_ascii,non_opt_eq]}]. + beam_bsm,guard,is_ascii,non_opt_eq,erl_689]}]. init_per_suite(Config) -> + test_lib:recompile(?MODULE), Config. end_per_suite(_Config) -> @@ -1688,6 +1688,35 @@ non_opt_eq([_|_], <<_,_/binary>>) -> non_opt_eq([], <<>>) -> true. +%% ERL-689 + +erl_689(Config) -> + {{0, 0, 0}, <<>>} = do_erl_689_1(<<0>>, ?MODULE), + {{2018, 8, 7}, <<>>} = do_erl_689_1(<<4,2018:16/little,8,7>>, ?MODULE), + {{0, 0, 0}, <<>>} = do_erl_689_2(?MODULE, <<0>>), + {{2018, 8, 7}, <<>>} = do_erl_689_2(?MODULE, <<4,2018:16/little,8,7>>), + ok. + +do_erl_689_1(<<Length, Data/binary>>, _) -> + case {Data, Length} of + {_, 0} -> + %% bs_context_to_binary would incorrectly set Data to the original + %% binary (before matching in the function head). + {{0, 0, 0}, Data}; + {<<Y:16/little, M, D, Rest/binary>>, 4} -> + {{Y, M, D}, Rest} + end. + +do_erl_689_2(_, <<Length, Data/binary>>) -> + case {Length, Data} of + {0, _} -> + %% bs_context_to_binary would incorrectly set Data to the original + %% binary (before matching in the function head). + {{0, 0, 0}, Data}; + {4, <<Y:16/little, M, D, Rest/binary>>} -> + {{Y, M, D}, Rest} + end. + check(F, R) -> R = F(). diff --git a/lib/compiler/test/bs_utf_SUITE.erl b/lib/compiler/test/bs_utf_SUITE.erl index ef3fc54b37..4330677260 100644 --- a/lib/compiler/test/bs_utf_SUITE.erl +++ b/lib/compiler/test/bs_utf_SUITE.erl @@ -31,7 +31,6 @@ suite() -> [{ct_hooks,[ts_install_cth]}]. all() -> - test_lib:recompile(?MODULE), [utf8_roundtrip, unused_utf_char, utf16_roundtrip, utf32_roundtrip, guard, extreme_tripping, literals, coverage]. @@ -40,6 +39,7 @@ groups() -> []. init_per_suite(Config) -> + test_lib:recompile(?MODULE), Config. end_per_suite(_Config) -> diff --git a/lib/compiler/test/compilation_SUITE.erl b/lib/compiler/test/compilation_SUITE.erl index a4de125d32..3ba3ce7cdf 100644 --- a/lib/compiler/test/compilation_SUITE.erl +++ b/lib/compiler/test/compilation_SUITE.erl @@ -66,7 +66,6 @@ suite() -> {timetrap,{minutes,10}}]. all() -> - test_lib:recompile(?MODULE), [self_compile_old_inliner,self_compile, {group,p}]. @@ -88,6 +87,7 @@ groups() -> string_table,otp_8949_a,split_cases]}]. init_per_suite(Config) -> + test_lib:recompile(?MODULE), Config. end_per_suite(_Config) -> diff --git a/lib/compiler/test/compilation_SUITE_data/opt_crash.erl b/lib/compiler/test/compilation_SUITE_data/opt_crash.erl index c65ec31593..4643ce61f6 100644 --- a/lib/compiler/test/compilation_SUITE_data/opt_crash.erl +++ b/lib/compiler/test/compilation_SUITE_data/opt_crash.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2016. All Rights Reserved. +%% Copyright Ericsson AB 2005-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/compiler/test/compile_SUITE.erl b/lib/compiler/test/compile_SUITE.erl index a1de8961bd..1ecae06128 100644 --- a/lib/compiler/test/compile_SUITE.erl +++ b/lib/compiler/test/compile_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2017. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -36,7 +36,7 @@ core_roundtrip/1, asm/1, optimized_guards/1, sys_pre_attributes/1, dialyzer/1, warnings/1, pre_load_check/1, env_compiler_options/1, - bc_options/1 + bc_options/1, deterministic_include/1 ]). suite() -> [{ct_hooks,[ts_install_cth]}]. @@ -46,7 +46,6 @@ suite() -> [{ct_hooks,[ts_install_cth]}]. -spec all() -> all_return_type(). all() -> - test_lib:recompile(?MODULE), [app_test, appup_test, file_1, forms_2, module_mismatch, big_file, outdir, binary, makedep, cond_and_ifdef, listings, listings_big, other_output, kernel_listing, encrypted_abstr, tuple_calls, @@ -54,12 +53,13 @@ all() -> cover, env, core_pp, core_roundtrip, asm, optimized_guards, sys_pre_attributes, dialyzer, warnings, pre_load_check, env_compiler_options, custom_debug_info, bc_options, - custom_compile_info]. + custom_compile_info, deterministic_include]. groups() -> []. init_per_suite(Config) -> + test_lib:recompile(?MODULE), Config. end_per_suite(_Config) -> @@ -1514,6 +1514,23 @@ highest_opcode(DataDir, Mod, Opt) -> <<16:32,0:32,HighestOpcode:32,_/binary>> = Code, HighestOpcode. +deterministic_include(Config) when is_list(Config) -> + DataDir = proplists:get_value(data_dir, Config), + Simple = filename:join(DataDir, "simple"), + + %% Files without +deterministic should differ if their include paths do, + %% as their debug info will be different. + {ok,_,NonDetA} = compile:file(Simple, [binary, {i,"gurka"}]), + {ok,_,NonDetB} = compile:file(Simple, [binary, {i,"gaffel"}]), + true = NonDetA =/= NonDetB, + + %% ... but files with +deterministic shouldn't. + {ok,_,DetC} = compile:file(Simple, [binary, deterministic, {i,"gurka"}]), + {ok,_,DetD} = compile:file(Simple, [binary, deterministic, {i,"gaffel"}]), + true = DetC =:= DetD, + + ok. + %%% %%% Utilities. %%% diff --git a/lib/compiler/test/compile_SUITE_data/big.erl b/lib/compiler/test/compile_SUITE_data/big.erl index 1db07755a1..765c71c07d 100644 --- a/lib/compiler/test/compile_SUITE_data/big.erl +++ b/lib/compiler/test/compile_SUITE_data/big.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2016. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/compiler/test/core_SUITE.erl b/lib/compiler/test/core_SUITE.erl index d07cd3b8b7..e5611e99d1 100644 --- a/lib/compiler/test/core_SUITE.erl +++ b/lib/compiler/test/core_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2006-2016. All Rights Reserved. +%% Copyright Ericsson AB 2006-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -29,7 +29,7 @@ bs_shadowed_size_var/1, cover_v3_kernel_1/1,cover_v3_kernel_2/1,cover_v3_kernel_3/1, cover_v3_kernel_4/1,cover_v3_kernel_5/1, - non_variable_apply/1,name_capture/1]). + non_variable_apply/1,name_capture/1,fun_letrec_effect/1]). -include_lib("common_test/include/ct.hrl"). @@ -47,7 +47,6 @@ suite() -> {timetrap,{minutes,5}}]. all() -> - test_lib:recompile(?MODULE), [{group,p}]. groups() -> @@ -58,11 +57,12 @@ groups() -> bs_shadowed_size_var, cover_v3_kernel_1,cover_v3_kernel_2,cover_v3_kernel_3, cover_v3_kernel_4,cover_v3_kernel_5, - non_variable_apply,name_capture + non_variable_apply,name_capture,fun_letrec_effect ]}]. init_per_suite(Config) -> + test_lib:recompile(?MODULE), Config. end_per_suite(_Config) -> @@ -94,6 +94,7 @@ end_per_group(_GroupName, Config) -> ?comp(cover_v3_kernel_5). ?comp(non_variable_apply). ?comp(name_capture). +?comp(fun_letrec_effect). try_it(Mod, Conf) -> Src = filename:join(proplists:get_value(data_dir, Conf), diff --git a/lib/compiler/test/core_SUITE_data/fun_letrec_effect.core b/lib/compiler/test/core_SUITE_data/fun_letrec_effect.core new file mode 100644 index 0000000000..ab6f5b7940 --- /dev/null +++ b/lib/compiler/test/core_SUITE_data/fun_letrec_effect.core @@ -0,0 +1,25 @@ +module 'fun_letrec_effect' ['fun_letrec_effect'/0, 'ok'/0, 'wat'/0] +attributes [] + +'fun_letrec_effect'/0 = + fun () -> + do apply 'wat'/0() + receive + <'bar'> when 'true' -> 'ok' + <_0> when 'true' -> 'failed' + after 'infinity' -> + 'true' + +%% The return value (bar) of the fun was optimized away because the result of +%% the `letrec ... in` was unused, despite the fun's return value being +%% relevant for the side-effect of the expression. +'wat'/0 = + fun () -> + let <Self> = call 'erlang':'self'() in + do letrec 'f'/0 = fun () -> + do call 'maps':'put'('foo', 'bar', ~{}~) + 'bar' + in call 'erlang':'send'(Self, apply 'f'/0()) + 'undefined' + +end diff --git a/lib/compiler/test/core_alias_SUITE.erl b/lib/compiler/test/core_alias_SUITE.erl index f3f15ef0f8..4f96576621 100644 --- a/lib/compiler/test/core_alias_SUITE.erl +++ b/lib/compiler/test/core_alias_SUITE.erl @@ -28,7 +28,6 @@ suite() -> [{ct_hooks,[ts_install_cth]}]. all() -> - test_lib:recompile(?MODULE), [{group,p}]. groups() -> @@ -36,6 +35,7 @@ groups() -> [tuples, cons]}]. init_per_suite(Config) -> + test_lib:recompile(?MODULE), Config. end_per_suite(_Config) -> diff --git a/lib/compiler/test/core_fold_SUITE.erl b/lib/compiler/test/core_fold_SUITE.erl index ab7f36abf7..30d145582a 100644 --- a/lib/compiler/test/core_fold_SUITE.erl +++ b/lib/compiler/test/core_fold_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2016. All Rights Reserved. +%% Copyright Ericsson AB 2007-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -28,7 +28,7 @@ mixed_matching_clauses/1,unnecessary_building/1, no_no_file/1,configuration/1,supplies/1, redundant_stack_frame/1,export_from_case/1, - empty_values/1]). + empty_values/1,cover_letrec_effect/1]). -export([foo/0,foo/1,foo/2,foo/3]). @@ -37,7 +37,6 @@ suite() -> [{ct_hooks,[ts_install_cth]}]. all() -> - test_lib:recompile(?MODULE), [{group,p}]. groups() -> @@ -49,10 +48,11 @@ groups() -> mixed_matching_clauses,unnecessary_building, no_no_file,configuration,supplies, redundant_stack_frame,export_from_case, - empty_values]}]. + empty_values,cover_letrec_effect]}]. init_per_suite(Config) -> + test_lib:recompile(?MODULE), Config. end_per_suite(_Config) -> @@ -278,6 +278,8 @@ coverage(Config) when is_list(Config) -> a = cover_remove_non_vars_alias({a,b,c}), error = cover_will_match_lit_list(), {ok,[a]} = cover_is_safe_bool_expr(a), + false = cover_is_safe_bool_expr2(a), + ok = cover_eval_is_function(fun id/1), ok = cover_opt_guard_try(#cover_opt_guard_try{list=[a]}), error = cover_opt_guard_try(#cover_opt_guard_try{list=[]}), @@ -341,6 +343,15 @@ cover_is_safe_bool_expr(X) -> false end. +cover_is_safe_bool_expr2(X) -> + try + V = [X], + is_function(V, 1) + catch + _:_ -> + false + end. + cover_opt_guard_try(Msg) -> if length(Msg#cover_opt_guard_try.list) =/= 1 -> @@ -349,6 +360,12 @@ cover_opt_guard_try(Msg) -> ok end. +cover_eval_is_function(X) -> + case X of + {a,_} -> is_function(X); + _ -> ok + end. + bsm_an_inlined(<<_:8>>, _) -> ok; bsm_an_inlined(_, _) -> error. @@ -598,5 +615,25 @@ empty_values(_Config) -> do_empty_values() when (#{})#{} -> c. +cover_letrec_effect(_Config) -> + self() ! {tag,42}, + _ = try + try + ignore + after + receive + {tag,Int}=Term -> + Res = #{k => {Term,<<Int:16>>}}, + self() ! Res + end + end + after + ok + end, + receive + Any -> + #{k := {{tag,42},<<42:16>>}} = Any + end, + ok. id(I) -> I. diff --git a/lib/compiler/test/error_SUITE.erl b/lib/compiler/test/error_SUITE.erl index 01c779b181..da291bdc8b 100644 --- a/lib/compiler/test/error_SUITE.erl +++ b/lib/compiler/test/error_SUITE.erl @@ -31,7 +31,6 @@ suite() -> [{ct_hooks,[ts_install_cth]}]. all() -> - test_lib:recompile(?MODULE), [{group,p}]. groups() -> @@ -40,6 +39,7 @@ groups() -> transforms,maps_warnings,bad_utf8]}]. init_per_suite(Config) -> + test_lib:recompile(?MODULE), Config. end_per_suite(_Config) -> diff --git a/lib/compiler/test/float_SUITE.erl b/lib/compiler/test/float_SUITE.erl index 08c3dd8593..39867021cb 100644 --- a/lib/compiler/test/float_SUITE.erl +++ b/lib/compiler/test/float_SUITE.erl @@ -27,7 +27,6 @@ suite() -> [{ct_hooks,[ts_install_cth]}]. all() -> - test_lib:recompile(?MODULE), [pending, bif_calls, math_functions, mixed_float_and_int]. @@ -35,6 +34,7 @@ groups() -> []. init_per_suite(Config) -> + test_lib:recompile(?MODULE), Config. end_per_suite(_Config) -> diff --git a/lib/compiler/test/fun_SUITE.erl b/lib/compiler/test/fun_SUITE.erl index 3c272a35a6..e00885fcd6 100644 --- a/lib/compiler/test/fun_SUITE.erl +++ b/lib/compiler/test/fun_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2000-2016. All Rights Reserved. +%% Copyright Ericsson AB 2000-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -32,7 +32,6 @@ suite() -> [{ct_hooks,[ts_install_cth]}]. all() -> - test_lib:recompile(?MODULE), [{group,p}]. groups() -> @@ -41,6 +40,7 @@ groups() -> eep37_dup,badarity,badfun]}]. init_per_suite(Config) -> + test_lib:recompile(?MODULE), Config. end_per_suite(_Config) -> diff --git a/lib/compiler/test/guard_SUITE.erl b/lib/compiler/test/guard_SUITE.erl index 0d6f8c6f98..99dc06b525 100644 --- a/lib/compiler/test/guard_SUITE.erl +++ b/lib/compiler/test/guard_SUITE.erl @@ -41,7 +41,6 @@ suite() -> [{ct_hooks,[ts_install_cth]}]. all() -> - test_lib:recompile(?MODULE), [{group,p}]. groups() -> @@ -59,6 +58,7 @@ groups() -> cover_beam_dead]}]. init_per_suite(Config) -> + test_lib:recompile(?MODULE), Config. end_per_suite(_Config) -> diff --git a/lib/compiler/test/inline_SUITE.erl b/lib/compiler/test/inline_SUITE.erl index ae59cc8026..fdf2fe88b4 100644 --- a/lib/compiler/test/inline_SUITE.erl +++ b/lib/compiler/test/inline_SUITE.erl @@ -32,7 +32,6 @@ suite() -> [{ct_hooks,[ts_install_cth]}]. all() -> - test_lib:recompile(?MODULE), [{group,p}]. groups() -> @@ -42,6 +41,7 @@ groups() -> coverage]}]. init_per_suite(Config) -> + test_lib:recompile(?MODULE), Pa = "-pa " ++ filename:dirname(code:which(?MODULE)), {ok,Node} = start_node(compiler, Pa), [{testing_node,Node}|Config]. diff --git a/lib/compiler/test/lc_SUITE.erl b/lib/compiler/test/lc_SUITE.erl index 699081470d..c80b7cc59e 100644 --- a/lib/compiler/test/lc_SUITE.erl +++ b/lib/compiler/test/lc_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2001-2017. All Rights Reserved. +%% Copyright Ericsson AB 2001-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -33,7 +33,6 @@ suite() -> {timetrap,{minutes,1}}]. all() -> - test_lib:recompile(?MODULE), [{group,p}]. groups() -> @@ -48,6 +47,7 @@ groups() -> ]}]. init_per_suite(Config) -> + test_lib:recompile(?MODULE), Config. end_per_suite(_Config) -> diff --git a/lib/compiler/test/map_SUITE.erl b/lib/compiler/test/map_SUITE.erl index 6badc7a8b8..c004dca834 100644 --- a/lib/compiler/test/map_SUITE.erl +++ b/lib/compiler/test/map_SUITE.erl @@ -76,7 +76,6 @@ suite() -> []. all() -> - test_lib:recompile(?MODULE), [ %% literals t_build_and_match_literals, t_build_and_match_literals_large, @@ -130,8 +129,12 @@ all() -> groups() -> []. -init_per_suite(Config) -> Config. -end_per_suite(_Config) -> ok. +init_per_suite(Config) -> + test_lib:recompile(?MODULE), + Config. + +end_per_suite(_Config) -> + ok. init_per_group(_GroupName, Config) -> Config. end_per_group(_GroupName, Config) -> Config. @@ -706,6 +709,12 @@ t_map_get(Config) when is_list(Config) -> {'EXIT',{{badmap,[]},_}} = (catch map_get(a, [])), {'EXIT',{{badmap,<<1,2,3>>},_}} = (catch map_get(a, <<1,2,3>>)), {'EXIT',{{badmap,1},_}} = (catch map_get(a, 1)), + + %% Test that beam_validator understands that NewMap is + %% a map after seeing map_get(a, NewMap). + NewMap = id(#{a=>b}), + b = map_get(a, NewMap), + #{a:=z} = NewMap#{a:=z}, ok. check_map_value(Map, Key, Value) when map_get(Key, Map) =:= Value -> true; @@ -1203,10 +1212,25 @@ t_guard_bifs(Config) when is_list(Config) -> true = map_guard_empty_2(), true = map_guard_head(#{a=>1}), false = map_guard_head([]), + true = map_get_head(#{a=>1}), + false = map_get_head(#{}), + false = map_get_head([]), + + true = map_get_head_not(#{a=>false}), + false = map_get_head_not(#{a=>true}), + false = map_get_head(#{}), false = map_get_head([]), + true = map_is_key_head(#{a=>1}), false = map_is_key_head(#{}), + false = map_is_key_head(not_a_map), + + false = map_is_key_head_not(#{a=>1}), + true = map_is_key_head_not(#{b=>1}), + true = map_is_key_head_not(#{}), + false = map_is_key_head_not(not_a_map), + true = map_guard_body(#{a=>1}), false = map_guard_body({}), true = map_guard_pattern(#{a=>1, <<"hi">> => "hi" }), @@ -1215,6 +1239,25 @@ t_guard_bifs(Config) when is_list(Config) -> true = map_guard_ill_map_size(), true = map_field_check_sequence(#{a=>1}), false = map_field_check_sequence(#{}), + + %% The guard BIFs used in a body. + + v = map_get(a, id(#{a=>v})), + {'EXIT',{{badkey,a},_}} = + (catch map_get(a, id(#{}))), + {'EXIT',{{badmap,not_a_map},_}} = + (catch map_get(a, id(not_a_map))), + + true = is_map_key(a, id(#{a=>1})), + false = is_map_key(b, id(#{a=>1})), + false = is_map_key(b, id(#{})), + {'EXIT',{{badmap,not_a_map},_}} = + (catch is_map_key(b, id(not_a_map))), + + {true,v} = erl_699(#{k=>v}), + {'EXIT',{{badkey,k},_}} = (catch erl_699(#{})), + {'EXIT',{{badmap,not_a_map},_}} = (catch erl_699(not_a_map)), + ok. map_guard_empty() when is_map(#{}); false -> true. @@ -1227,8 +1270,14 @@ map_guard_head(_) -> false. map_get_head(M) when map_get(a, M) =:= 1 -> true; map_get_head(_) -> false. +map_get_head_not(M) when not map_get(a, M) -> true; +map_get_head_not(_) -> false. + map_is_key_head(M) when is_map_key(a, M) -> true; -map_is_key_head(M) -> false. +map_is_key_head(_) -> false. + +map_is_key_head_not(M) when not is_map_key(a, M) -> true; +map_is_key_head_not(_) -> false. map_guard_body(M) -> is_map(M). @@ -1245,6 +1294,10 @@ map_field_check_sequence(M) map_field_check_sequence(_) -> false. +erl_699(M) -> + %% Used to cause an internal consistency failure. + {is_map_key(k, M),maps:get(k, M)}. + t_guard_sequence(Config) when is_list(Config) -> {1, "a"} = map_guard_sequence_1(#{seq=>1,val=>id("a")}), {2, "b"} = map_guard_sequence_1(#{seq=>2,val=>id("b")}), diff --git a/lib/compiler/test/match_SUITE.erl b/lib/compiler/test/match_SUITE.erl index 4b26a8dcdc..e3f842b668 100644 --- a/lib/compiler/test/match_SUITE.erl +++ b/lib/compiler/test/match_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2016. All Rights Reserved. +%% Copyright Ericsson AB 2004-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -32,7 +32,6 @@ suite() -> [{ct_hooks,[ts_install_cth]}]. all() -> - test_lib:recompile(?MODULE), [{group,p}]. groups() -> @@ -45,6 +44,7 @@ groups() -> init_per_suite(Config) -> + test_lib:recompile(?MODULE), Config. end_per_suite(_Config) -> diff --git a/lib/compiler/test/misc_SUITE.erl b/lib/compiler/test/misc_SUITE.erl index 4e39f4663e..a1d931b994 100644 --- a/lib/compiler/test/misc_SUITE.erl +++ b/lib/compiler/test/misc_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2006-2017. All Rights Reserved. +%% Copyright Ericsson AB 2006-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -61,7 +61,6 @@ suite() -> -spec all() -> misc_SUITE_test_cases(). all() -> - test_lib:recompile(?MODULE), [{group,p}]. groups() -> @@ -70,6 +69,7 @@ groups() -> confused_literals,integer_encoding,override_bif]}]. init_per_suite(Config) -> + test_lib:recompile(?MODULE), Config. end_per_suite(_Config) -> diff --git a/lib/compiler/test/overridden_bif_SUITE.erl b/lib/compiler/test/overridden_bif_SUITE.erl index ce18916515..a46abe8dcf 100644 --- a/lib/compiler/test/overridden_bif_SUITE.erl +++ b/lib/compiler/test/overridden_bif_SUITE.erl @@ -36,7 +36,6 @@ suite() -> {timetrap,{minutes,1}}]. all() -> - test_lib:recompile(?MODULE), [{group,p}]. groups() -> @@ -45,6 +44,7 @@ groups() -> ]}]. init_per_suite(Config) -> + test_lib:recompile(?MODULE), Config. end_per_suite(_Config) -> diff --git a/lib/compiler/test/receive_SUITE.erl b/lib/compiler/test/receive_SUITE.erl index 5e386790c0..37e737084a 100644 --- a/lib/compiler/test/receive_SUITE.erl +++ b/lib/compiler/test/receive_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2016. All Rights Reserved. +%% Copyright Ericsson AB 2004-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -25,7 +25,7 @@ init_per_group/2,end_per_group/2, init_per_testcase/2,end_per_testcase/2, export/1,recv/1,coverage/1,otp_7980/1,ref_opt/1, - wait/1]). + wait/1,recv_in_try/1]). -include_lib("common_test/include/ct.hrl"). @@ -40,15 +40,16 @@ suite() -> {timetrap,{minutes,2}}]. all() -> - test_lib:recompile(?MODULE), [{group,p}]. groups() -> [{p,test_lib:parallel(), - [recv,coverage,otp_7980,ref_opt,export,wait]}]. + [recv,coverage,otp_7980,ref_opt,export,wait, + recv_in_try]}]. init_per_suite(Config) -> + test_lib:recompile(?MODULE), Config. end_per_suite(_Config) -> @@ -305,4 +306,48 @@ wait_1(r, _, _) -> wait_1(A, B, C) -> {A,B,C}. +recv_in_try(_Config) -> + self() ! {ok,fh}, {ok,fh} = recv_in_try(infinity, native), + self() ! {ok,ignored}, {ok,42} = recv_in_try(infinity, plain), + self() ! {error,ignored}, nok = recv_in_try(infinity, plain), + timeout = recv_in_try(1, plain), + ok. + +recv_in_try(Timeout, Format) -> + try + receive + {Status,History} -> + %% {test,is_tuple,{f,148},[{x,0}]}. + %% {test,test_arity,{f,148},[{x,0},2]}. + %% {get_tuple_element,{x,0},0,{y,1}}. %y1 is fragile. + %% + %% %% Here the fragility of y1 would be be progated to + %% %% the 'catch' below. Incorrect, since get_tuple_element + %% %% can't fail. + %% {get_tuple_element,{x,0},1,{x,2}}. + %% + %% remove_message. %y1 fragility cleared. + FH = case Format of + native -> + id(History); + plain -> + id(42) + end, + case Status of + ok -> + {ok,FH}; + error -> + nok + end + after Timeout -> + timeout + end + catch + %% The fragility of y1 incorrectly propagated to here. + %% beam_validator would complain. + throw:{error,Reason} -> + {nok,Reason} + end. + + id(I) -> I. diff --git a/lib/compiler/test/record_SUITE.erl b/lib/compiler/test/record_SUITE.erl index 5546765f26..118e0a241c 100644 --- a/lib/compiler/test/record_SUITE.erl +++ b/lib/compiler/test/record_SUITE.erl @@ -41,7 +41,6 @@ suite() -> {timetrap,{minutes,2}}]. all() -> - test_lib:recompile(?MODULE), [{group,p}]. groups() -> @@ -52,6 +51,7 @@ groups() -> init_per_suite(Config) -> + test_lib:recompile(?MODULE), Config. end_per_suite(_Config) -> diff --git a/lib/compiler/test/regressions_SUITE.erl b/lib/compiler/test/regressions_SUITE.erl index 7a6fe08c73..f448d54933 100644 --- a/lib/compiler/test/regressions_SUITE.erl +++ b/lib/compiler/test/regressions_SUITE.erl @@ -21,13 +21,29 @@ -module(regressions_SUITE). -include_lib("common_test/include/ct.hrl"). --export([all/0,groups/0,init_per_testcase/2,end_per_testcase/2,suite/0]). +-export([all/0,groups/0,init_per_testcase/2,end_per_testcase/2, + init_per_group/2,end_per_group/2, + init_per_testcase/2,end_per_testcase/2, + suite/0]). -export([maps/1]). groups() -> [{p,test_lib:parallel(), [maps]}]. +init_per_suite(Config) -> + test_lib:recompile(?MODULE), + Config. + +end_per_suite(_Config) -> + ok. + +init_per_group(_GroupName, Config) -> + Config. + +end_per_group(_GroupName, Config) -> + Config. + init_per_testcase(_Case, Config) -> Config. @@ -39,7 +55,6 @@ suite() -> {timetrap,{minutes,2}}]. all() -> - test_lib:recompile(?MODULE), [{group,p}]. %%% test cases diff --git a/lib/compiler/test/trycatch_SUITE.erl b/lib/compiler/test/trycatch_SUITE.erl index d5a1dc642f..1b7ef4ddb0 100644 --- a/lib/compiler/test/trycatch_SUITE.erl +++ b/lib/compiler/test/trycatch_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2003-2016. All Rights Reserved. +%% Copyright Ericsson AB 2003-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -34,7 +34,6 @@ suite() -> [{ct_hooks,[ts_install_cth]}]. all() -> - test_lib:recompile(?MODULE), [{group,p}]. groups() -> @@ -48,6 +47,7 @@ groups() -> init_per_suite(Config) -> + test_lib:recompile(?MODULE), Config. end_per_suite(_Config) -> diff --git a/lib/compiler/test/warnings_SUITE.erl b/lib/compiler/test/warnings_SUITE.erl index 857995b6a6..42ff4f6133 100644 --- a/lib/compiler/test/warnings_SUITE.erl +++ b/lib/compiler/test/warnings_SUITE.erl @@ -55,7 +55,6 @@ suite() -> {timetrap,{minutes,2}}]. all() -> - test_lib:recompile(?MODULE), [{group,p}]. groups() -> @@ -68,6 +67,7 @@ groups() -> underscore,no_warnings,bit_syntax,inlining]}]. init_per_suite(Config) -> + test_lib:recompile(?MODULE), Config. end_per_suite(_Config) -> diff --git a/lib/compiler/test/z_SUITE.erl b/lib/compiler/test/z_SUITE.erl index cd95d0e733..bfa8e279e8 100644 --- a/lib/compiler/test/z_SUITE.erl +++ b/lib/compiler/test/z_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2015. All Rights Reserved. +%% Copyright Ericsson AB 2015-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -26,13 +26,13 @@ suite() -> [{ct_hooks,[ts_install_cth]}]. all() -> - test_lib:recompile(?MODULE), [loaded]. groups() -> []. init_per_suite(Config) -> + test_lib:recompile(?MODULE), Config. end_per_suite(_Config) -> diff --git a/lib/compiler/vsn.mk b/lib/compiler/vsn.mk index ee75ee27fd..355113a94d 100644 --- a/lib/compiler/vsn.mk +++ b/lib/compiler/vsn.mk @@ -1 +1 @@ -COMPILER_VSN = 7.1.5 +COMPILER_VSN = 7.2.3 diff --git a/lib/crypto/c_src/crypto.c b/lib/crypto/c_src/crypto.c index e4eb2bc3b6..a9fef65e03 100644 --- a/lib/crypto/c_src/crypto.c +++ b/lib/crypto/c_src/crypto.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2010-2017. All Rights Reserved. + * Copyright Ericsson AB 2010-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -179,6 +179,7 @@ #if OPENSSL_VERSION_NUMBER >= PACKED_OPENSSL_VERSION_PLAIN(1,1,0) # ifndef HAS_LIBRESSL # define HAVE_CHACHA20_POLY1305 +# define HAVE_RSA_OAEP_MD # endif #endif @@ -3068,202 +3069,187 @@ static ERL_NIF_TERM rsa_generate_key_nif(ErlNifEnv* env, int argc, const ERL_NIF static ERL_NIF_TERM dh_generate_key_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) {/* (PrivKey|undefined, DHParams=[P,G], Mpint, Len|0) */ - DH* dh_params = NULL; - int pub_len, prv_len; - unsigned char *pub_ptr, *prv_ptr; - ERL_NIF_TERM ret, ret_pub, ret_prv, head, tail; + DH *dh_params = NULL; int mpint; /* 0 or 4 */ - BIGNUM *priv_key_in = NULL; - BIGNUM *dh_p = NULL, *dh_g = NULL; - unsigned long len = 0; -#ifdef HAS_EVP_PKEY_CTX - EVP_PKEY_CTX *ctx = NULL; - EVP_PKEY *dhkey = NULL, - *params = NULL; -#endif - if (!(get_bn_from_bin(env, argv[0], &priv_key_in) - || argv[0] == atom_undefined) - || !enif_get_list_cell(env, argv[1], &head, &tail) - || !get_bn_from_bin(env, head, &dh_p) - || !enif_get_list_cell(env, tail, &head, &tail) - || !get_bn_from_bin(env, head, &dh_g) - || !enif_is_empty_list(env, tail) - || !enif_get_int(env, argv[2], &mpint) || (mpint & ~4) - || !enif_get_ulong(env, argv[3], &len) - - /* Load dh_params with values to use by the generator. - Mem mgmnt transfered from dh_p etc to dh_params */ - || !(dh_params = DH_new()) - || (priv_key_in && !DH_set0_key(dh_params, NULL, priv_key_in)) - || !DH_set0_pqg(dh_params, dh_p, NULL, dh_g) - ) { - if (priv_key_in) BN_free(priv_key_in); - if (dh_p) BN_free(dh_p); - if (dh_g) BN_free(dh_g); - if (dh_params) DH_free(dh_params); - return enif_make_badarg(env); - } - - if (len) { - if (len < BN_num_bits(dh_p)) - DH_set_length(dh_params, len); - else { + { + ERL_NIF_TERM head, tail; + BIGNUM + *dh_p = NULL, + *dh_g = NULL, + *priv_key_in = NULL; + unsigned long + len = 0; + + if (!(get_bn_from_bin(env, argv[0], &priv_key_in) + || argv[0] == atom_undefined) + || !enif_get_list_cell(env, argv[1], &head, &tail) + || !get_bn_from_bin(env, head, &dh_p) + || !enif_get_list_cell(env, tail, &head, &tail) + || !get_bn_from_bin(env, head, &dh_g) + || !enif_is_empty_list(env, tail) + || !enif_get_int(env, argv[2], &mpint) || (mpint & ~4) + || !enif_get_ulong(env, argv[3], &len) + + /* Load dh_params with values to use by the generator. + Mem mgmnt transfered from dh_p etc to dh_params */ + || !(dh_params = DH_new()) + || (priv_key_in && !DH_set0_key(dh_params, NULL, priv_key_in)) + || !DH_set0_pqg(dh_params, dh_p, NULL, dh_g) + ) { if (priv_key_in) BN_free(priv_key_in); if (dh_p) BN_free(dh_p); if (dh_g) BN_free(dh_g); if (dh_params) DH_free(dh_params); return enif_make_badarg(env); } + + if (len) { + if (len < BN_num_bits(dh_p)) + DH_set_length(dh_params, len); + else { + if (priv_key_in) BN_free(priv_key_in); + if (dh_p) BN_free(dh_p); + if (dh_g) BN_free(dh_g); + if (dh_params) DH_free(dh_params); + return enif_make_badarg(env); + } + } } #ifdef HAS_EVP_PKEY_CTX - if ((dhkey = EVP_PKEY_new()) - && (params = EVP_PKEY_new()) - && EVP_PKEY_set1_DH(params, dh_params) /* set the key referenced by params to dh_params. - dh_params (and params) must be freed */ - && (ctx = EVP_PKEY_CTX_new(params, NULL)) - && EVP_PKEY_keygen_init(ctx) - && EVP_PKEY_keygen(ctx, &dhkey) /* "performs a key generation operation, the - generated key is written to ppkey." (=last arg) */ - && (dh_params = EVP_PKEY_get1_DH(dhkey)) /* return the referenced key. dh_params and dhkey must be freed */ - ) { + { + EVP_PKEY_CTX *ctx; + EVP_PKEY *dhkey, *params; + int success; + + params = EVP_PKEY_new(); + success = EVP_PKEY_set1_DH(params, dh_params); /* set the key referenced by params to dh_params... */ + DH_free(dh_params); /* ...dh_params (and params) must be freed */ + if (!success) return atom_error; + + ctx = EVP_PKEY_CTX_new(params, NULL); + EVP_PKEY_free(params); + if (!ctx) { + return atom_error; + } + + if (!EVP_PKEY_keygen_init(ctx)) { + /* EVP_PKEY_CTX_free(ctx); */ + return atom_error; + } + + dhkey = EVP_PKEY_new(); + if (!EVP_PKEY_keygen(ctx, &dhkey)) { /* "performs a key generation operation, the ... */ + /*... generated key is written to ppkey." (=last arg) */ + /* EVP_PKEY_CTX_free(ctx); */ + /* EVP_PKEY_free(dhkey); */ + return atom_error; + } + + dh_params = EVP_PKEY_get1_DH(dhkey); /* return the referenced key. dh_params and dhkey must be freed */ + EVP_PKEY_free(dhkey); + if (!dh_params) { + /* EVP_PKEY_CTX_free(ctx); */ + return atom_error; + } + EVP_PKEY_CTX_free(ctx); + } #else - if (DH_generate_key(dh_params)) { -#endif + if (!DH_generate_key(dh_params)) return atom_error; +#endif + { + unsigned char *pub_ptr, *prv_ptr; + int pub_len, prv_len; + ERL_NIF_TERM ret_pub, ret_prv; const BIGNUM *pub_key_gen, *priv_key_gen; - - DH_get0_key(dh_params, - &pub_key_gen, &priv_key_gen); /* Get pub_key_gen and priv_key_gen. - "The values point to the internal representation of - the public key and private key values. This memory + + DH_get0_key(dh_params, + &pub_key_gen, &priv_key_gen); /* Get pub_key_gen and priv_key_gen. + "The values point to the internal representation of + the public key and private key values. This memory should not be freed directly." says man */ - pub_len = BN_num_bytes(pub_key_gen); - prv_len = BN_num_bytes(priv_key_gen); - pub_ptr = enif_make_new_binary(env, pub_len+mpint, &ret_pub); - prv_ptr = enif_make_new_binary(env, prv_len+mpint, &ret_prv); - if (mpint) { - put_int32(pub_ptr, pub_len); pub_ptr += 4; - put_int32(prv_ptr, prv_len); prv_ptr += 4; - } - BN_bn2bin(pub_key_gen, pub_ptr); - BN_bn2bin(priv_key_gen, prv_ptr); - ERL_VALGRIND_MAKE_MEM_DEFINED(pub_ptr, pub_len); - ERL_VALGRIND_MAKE_MEM_DEFINED(prv_ptr, prv_len); - ret = enif_make_tuple2(env, ret_pub, ret_prv); - } - else { - ret = atom_error; - } + pub_len = BN_num_bytes(pub_key_gen); + prv_len = BN_num_bytes(priv_key_gen); + pub_ptr = enif_make_new_binary(env, pub_len+mpint, &ret_pub); + prv_ptr = enif_make_new_binary(env, prv_len+mpint, &ret_prv); + if (mpint) { + put_int32(pub_ptr, pub_len); pub_ptr += 4; + put_int32(prv_ptr, prv_len); prv_ptr += 4; + } + BN_bn2bin(pub_key_gen, pub_ptr); + BN_bn2bin(priv_key_gen, prv_ptr); + ERL_VALGRIND_MAKE_MEM_DEFINED(pub_ptr, pub_len); + ERL_VALGRIND_MAKE_MEM_DEFINED(prv_ptr, prv_len); - DH_free(dh_params); -#ifdef HAS_EVP_PKEY_CTX - if (ctx) EVP_PKEY_CTX_free(ctx); - if (dhkey) EVP_PKEY_free(dhkey); - if (params) EVP_PKEY_free(params); -#endif - return ret; + DH_free(dh_params); + + return enif_make_tuple2(env, ret_pub, ret_prv); + } } static ERL_NIF_TERM dh_compute_key_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) {/* (OthersPublicKey, MyPrivateKey, DHParams=[P,G]) */ - BIGNUM *dummy_pub_key = NULL, - *priv_key = NULL, - *other_pub_key = NULL, + BIGNUM *other_pub_key = NULL, *dh_p = NULL, *dh_g = NULL; - ErlNifBinary ret_bin; - ERL_NIF_TERM ret, head, tail; - DH *dh_priv = DH_new(), *dh_pub = DH_new(); -#ifdef HAS_EVP_PKEY_CTX - EVP_PKEY_CTX *ctx = NULL; - EVP_PKEY *my_priv_key = NULL, *peer_pub_key = NULL; - size_t skeylen; -#else - int i; -#endif + DH *dh_priv = DH_new(); - if (!get_bn_from_bin(env, argv[0], &other_pub_key) - || !get_bn_from_bin(env, argv[1], &priv_key) - || !enif_get_list_cell(env, argv[2], &head, &tail) - || !get_bn_from_bin(env, head, &dh_p) - || !enif_get_list_cell(env, tail, &head, &tail) - || !get_bn_from_bin(env, head, &dh_g) - || !enif_is_empty_list(env, tail) - - /* Note: DH_set0_key() does not allow setting only the - * private key, although DH_compute_key() does not use the - * public key. Work around this limitation by setting - * the public key to a copy of the private key. - */ - || !(dummy_pub_key = BN_dup(priv_key)) - || !DH_set0_key(dh_priv, dummy_pub_key, priv_key) - || !DH_set0_pqg(dh_priv, dh_p, NULL, dh_g) - ) { - if (dh_p) BN_free(dh_p); - if (dh_g) BN_free(dh_g); - if (other_pub_key) BN_free(other_pub_key); - if (dummy_pub_key) BN_free(dummy_pub_key); - if (priv_key) BN_free(priv_key); - return enif_make_badarg(env); - } - -#ifdef HAS_EVP_PKEY_CTX - if (!(my_priv_key = EVP_PKEY_new()) - || !EVP_PKEY_set1_DH(my_priv_key, dh_priv) /* set the key referenced by my_priv_key to dh_priv. - dh_priv (and my_priv_key) must be freed by us*/ - - || !(peer_pub_key = EVP_PKEY_new()) - || !DH_set0_key(dh_pub, other_pub_key, NULL) - || !DH_set0_pqg(dh_pub, dh_p, NULL, dh_g) - || !EVP_PKEY_set1_DH(peer_pub_key, dh_pub) - - || !(ctx = EVP_PKEY_CTX_new(my_priv_key, NULL)) - || (EVP_PKEY_derive_init(ctx) <= 0) - || (EVP_PKEY_derive_set_peer(ctx, peer_pub_key) <= 0) - || (EVP_PKEY_derive(ctx, NULL, &skeylen) <= 0)) { + /* Check the arguments and get + my private key (dh_priv), + the peer's public key (other_pub_key), + the parameters p & q + */ - ret = atom_error; - } - else { - enif_alloc_binary(skeylen, &ret_bin); - - if ((EVP_PKEY_derive(ctx, ret_bin.data, &skeylen) > 0) - && (ret_bin.size >= skeylen)) { - /* Derivation succeded */ - if (ret_bin.size > skeylen) enif_realloc_binary(&ret_bin, skeylen); - ret = enif_make_binary(env, &ret_bin); + { + BIGNUM *dummy_pub_key = NULL, + *priv_key = NULL; + ERL_NIF_TERM head, tail; + + if (!get_bn_from_bin(env, argv[0], &other_pub_key) + || !get_bn_from_bin(env, argv[1], &priv_key) + || !enif_get_list_cell(env, argv[2], &head, &tail) + || !get_bn_from_bin(env, head, &dh_p) + || !enif_get_list_cell(env, tail, &head, &tail) + || !get_bn_from_bin(env, head, &dh_g) + || !enif_is_empty_list(env, tail) + + /* Note: DH_set0_key() does not allow setting only the + * private key, although DH_compute_key() does not use the + * public key. Work around this limitation by setting + * the public key to a copy of the private key. + */ + || !(dummy_pub_key = BN_dup(priv_key)) + || !DH_set0_key(dh_priv, dummy_pub_key, priv_key) + || !DH_set0_pqg(dh_priv, dh_p, NULL, dh_g) + ) { + if (dh_p) BN_free(dh_p); + if (dh_g) BN_free(dh_g); + if (other_pub_key) BN_free(other_pub_key); + if (dummy_pub_key) BN_free(dummy_pub_key); + if (priv_key) BN_free(priv_key); + return enif_make_badarg(env); } - else { + } + { + ErlNifBinary ret_bin; + int size; + + enif_alloc_binary(DH_size(dh_priv), &ret_bin); + size = DH_compute_key(ret_bin.data, other_pub_key, dh_priv); + BN_free(other_pub_key); + DH_free(dh_priv); + if (size<=0) { enif_release_binary(&ret_bin); - ret = atom_error; + return atom_error; } - } -#else - enif_alloc_binary(DH_size(dh_priv), &ret_bin); - i = DH_compute_key(ret_bin.data, other_pub_key, dh_priv); - if (i > 0) { - if (i != ret_bin.size) enif_realloc_binary(&ret_bin, i); - ret = enif_make_binary(env, &ret_bin); + if (size != ret_bin.size) enif_realloc_binary(&ret_bin, size); + return enif_make_binary(env, &ret_bin); } - else { - enif_release_binary(&ret_bin); - ret = atom_error; - } -#endif - - if (other_pub_key) BN_free(other_pub_key); - if (dh_priv) DH_free(dh_priv); - if (dh_pub) DH_free(dh_pub); -#ifdef HAS_EVP_PKEY_CTX - if (ctx) EVP_PKEY_CTX_free(ctx); - if (my_priv_key) EVP_PKEY_free(my_priv_key); - /* if (peer_pub_key) EVP_PKEY_free(peer_pub_key); */ -#endif - return ret; } + static ERL_NIF_TERM srp_value_B_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) {/* (Multiplier, Verifier, Generator, Exponent, Prime) */ BIGNUM *bn_verifier = NULL; @@ -3825,7 +3811,7 @@ static ERL_NIF_TERM ecdh_compute_key_nif(ErlNifEnv* env, int argc, const ERL_NIF int i; EC_GROUP *group; const BIGNUM *priv_key; - EC_POINT *my_ecpoint; + EC_POINT *my_ecpoint = NULL; EC_KEY *other_ecdh = NULL; if (!get_ec_key(env, argv[1], argv[2], atom_undefined, &key)) @@ -3919,13 +3905,6 @@ static ERL_NIF_TERM evp_compute_key_nif(ErlNifEnv* env, int argc, const ERL_NIF_ } return enif_make_binary(env, &key_bin); - -/* Importing the other side's public key from raw binary format can be done with the EVP_PKEY_new_raw_public_key() function. Man page here: */ -/* https://www.openssl.org/docs/man1.1.1/man3/EVP_PKEY_new_raw_public_key.html */ - - -/* You need two EVP_PKEY objects. One containing your private/public key pair (i.e. the one you generated in the EVP_PKEY_keygen() call in your question), and one containing the public key of the peer (e.g. created using EVP_PKEY_new_raw_public_key()). To generate the X25519 shared secret you then call EVP_PKEY_derive(). See the example on the man page: openssl.org/docs/man1.1.1/man3/EVP_PKEY_derive.html – Matt Caswell May 15 at 20:39 */ - #else return atom_notsup; #endif @@ -3951,20 +3930,6 @@ static ERL_NIF_TERM evp_generate_key_nif(ErlNifEnv* env, int argc, const ERL_NIF if (!EVP_PKEY_keygen_init(ctx)) return enif_make_atom(env,"EVP_PKEY_keygen_init failed"); if (!EVP_PKEY_keygen(ctx, &pkey)) return enif_make_atom(env,"EVP_PKEY_keygen failed"); - /* - int EVP_PKEY_get_raw_private_key(const EVP_PKEY *pkey, unsigned char *priv, size_t *len) - int EVP_PKEY_get_raw_public_key(const EVP_PKEY *pkey, unsigned char *pub, size_t *len) - - +EVP_PKEY_get_raw_private_key() fills the buffer provided by B<priv> with raw - +private key data. The number of bytes written is populated in B<*len>. If the - +buffer B<priv> is NULL then B<*len> is populated with the number of bytes - +required in the buffer. The calling application is responsible for ensuring that - +the buffer is large enough to receive the private key data. This function only - +works for algorithms that support raw private keys. Currently this is: - +B<EVP_PKEY_HMAC>, B<EVP_PKEY_POLY1305>, B<EVP_PKEY_SIPHASH>, B<EVP_PKEY_X25519>, - +B<EVP_PKEY_ED25519>, B<EVP_PKEY_X448> or B<EVP_PKEY_ED448>. - */ - if (!EVP_PKEY_get_raw_public_key(pkey, NULL, &key_len)) return enif_make_atom(env,"EVP_PKEY_get_raw_public_key 1 failed"); if (!EVP_PKEY_get_raw_public_key(pkey, diff --git a/lib/crypto/c_src/otp_test_engine.c b/lib/crypto/c_src/otp_test_engine.c index d0e23a2a3e..b6c9067964 100644 --- a/lib/crypto/c_src/otp_test_engine.c +++ b/lib/crypto/c_src/otp_test_engine.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2017-2017. All Rights Reserved. + * Copyright Ericsson AB 2017-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/lib/crypto/doc/src/Makefile b/lib/crypto/doc/src/Makefile index aa987d2b39..2148062e78 100644 --- a/lib/crypto/doc/src/Makefile +++ b/lib/crypto/doc/src/Makefile @@ -89,6 +89,7 @@ debug opt valgrind: clean clean_docs clean_tex: rm -rf $(HTMLDIR)/* + rm -rf $(XMLDIR) rm -f $(MAN3DIR)/* rm -f $(MAN6DIR)/* rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) diff --git a/lib/crypto/doc/src/crypto.xml b/lib/crypto/doc/src/crypto.xml index af676d9021..85d5ccaa34 100644 --- a/lib/crypto/doc/src/crypto.xml +++ b/lib/crypto/doc/src/crypto.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1999</year><year>2017</year> + <year>1999</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -830,7 +830,7 @@ _FloatValue = rand:uniform(). % [0.0; 1.0[</pre> </p> <note> <p> - The state returned from this function can not be used + The state returned from this function cannot be used to get a reproducable random sequence as from the other <seealso marker="stdlib:rand">rand</seealso> @@ -920,7 +920,7 @@ _FloatValue = rand:uniform(). % [0.0; 1.0[</pre> </p> <note> <p> - The state returned from this function can not be used + The state returned from this function cannot be used to get a reproducable random sequence as from the other <seealso marker="stdlib:rand">rand</seealso> diff --git a/lib/crypto/doc/src/notes.xml b/lib/crypto/doc/src/notes.xml index 66619c9e11..5e0851f6b8 100644 --- a/lib/crypto/doc/src/notes.xml +++ b/lib/crypto/doc/src/notes.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>1999</year><year>2017</year> + <year>1999</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -31,6 +31,85 @@ </header> <p>This document describes the changes made to the Crypto application.</p> +<section><title>Crypto 4.3.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p>Fixed a node crash in <c>crypto:compute_key(ecdh, + ...)</c> when passing a wrongly typed Others + argument.</p> + <p> + Own Id: OTP-15194 Aux Id: ERL-673 </p> + </item> + </list> + </section> + +</section> + +<section><title>Crypto 4.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Removed two undocumented and erroneous functions + (<c>crypto:dh_generate_parameters/2</c> and + <c>crypto:dh_check/1</c>).</p> + <p> + Own Id: OTP-14956 Aux Id: ERL-579 </p> + </item> + <item> + <p> + Fixed bug causing VM crash if doing runtime upgrade of a + crypto module built against OpenSSL older than 0.9.8h. + Bug exists since OTP-20.2.</p> + <p> + Own Id: OTP-15088</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + A new <c>rand</c> plugin algorithm has been implemented + in <c>crypto</c>, that is: <c>crypto_cache</c>. It uses + strong random bytes as randomness source and caches them + to get good speed. See <c>crypto:rand_seed_alg/1</c>.</p> + <p> + Own Id: OTP-13370 Aux Id: PR-1573 </p> + </item> + <item> + <p> + Diffie-Hellman key functions are re-written with the + EVP_PKEY api.</p> + <p> + Own Id: OTP-14864</p> + </item> + </list> + </section> + +</section> + +<section><title>Crypto 4.2.2.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p>Fixed a node crash in <c>crypto:compute_key(ecdh, + ...)</c> when passing a wrongly typed Others + argument.</p> + <p> + Own Id: OTP-15194 Aux Id: ERL-673 </p> + </item> + </list> + </section> + +</section> + <section><title>Crypto 4.2.2</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/crypto/doc/src/usersguide.xml b/lib/crypto/doc/src/usersguide.xml index e2ba1fe160..0124121433 100644 --- a/lib/crypto/doc/src/usersguide.xml +++ b/lib/crypto/doc/src/usersguide.xml @@ -4,7 +4,7 @@ <part xmlns:xi="http://www.w3.org/2001/XInclude"> <header> <copyright> - <year>2003</year><year>2016</year> + <year>2003</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/crypto/src/Makefile b/lib/crypto/src/Makefile index edad0e6b61..1753ba4f36 100644 --- a/lib/crypto/src/Makefile +++ b/lib/crypto/src/Makefile @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 1999-2016. All Rights Reserved. +# Copyright Ericsson AB 1999-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. diff --git a/lib/crypto/src/crypto.erl b/lib/crypto/src/crypto.erl index 85b484011c..714cba58b9 100644 --- a/lib/crypto/src/crypto.erl +++ b/lib/crypto/src/crypto.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2017. All Rights Reserved. +%% Copyright Ericsson AB 1999-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/crypto/test/blowfish_SUITE.erl b/lib/crypto/test/blowfish_SUITE.erl index c9033ac4f8..a931ebb47e 100644 --- a/lib/crypto/test/blowfish_SUITE.erl +++ b/lib/crypto/test/blowfish_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009-2016. All Rights Reserved. +%% Copyright Ericsson AB 2009-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/crypto/test/crypto_SUITE.erl b/lib/crypto/test/crypto_SUITE.erl index d148fa3856..aa2278dbdd 100644 --- a/lib/crypto/test/crypto_SUITE.erl +++ b/lib/crypto/test/crypto_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2017. All Rights Reserved. +%% Copyright Ericsson AB 1999-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -31,6 +31,7 @@ suite() -> [{ct_hooks,[ts_install_cth]}]. all() -> [app, + {group, api_errors}, appup, {group, fips}, {group, non_fips}, @@ -170,7 +171,8 @@ groups() -> {no_aes_ige256, [], [no_support, no_block]}, {no_chacha20_poly1305, [], [no_support, no_aead]}, {no_rc2_cbc, [], [no_support, no_block]}, - {no_rc4, [], [no_support, no_stream]} + {no_rc4, [], [no_support, no_stream]}, + {api_errors, [], [api_errors_ecdh]} ]. %%------------------------------------------------------------------- @@ -238,6 +240,8 @@ init_per_group(non_fips, Config) -> _NotEnabled -> NonFIPSConfig end; +init_per_group(api_errors, Config) -> + Config; init_per_group(GroupName, Config) -> case atom_to_list(GroupName) of "no_" ++ TypeStr -> @@ -836,26 +840,69 @@ negative_verify(Type, Hash, Msg, Signature, Public, Options) -> ok end. +-define(PUB_PRIV_ENC_DEC_CATCH(Type,Padding), + CC:EE -> + ct:log("~p:~p in ~p:~p/~p, line ~p.~n" + "Type = ~p~nPadding = ~p", + [CC,EE,?MODULE,?FUNCTION_NAME,?FUNCTION_ARITY,?LINE,(Type),(Padding)]), + MaybeUnsupported = + case crypto:info_lib() of + [{<<"OpenSSL">>,_,_}] -> + is_list(Padding) andalso + lists:any(fun(P) -> lists:member(P,(Padding)) end, + [{rsa_padding, rsa_pkcs1_oaep_padding}, + {rsa_padding, rsa_sslv23_padding}, + {rsa_padding, rsa_x931_padding}]); + _ -> + false + end, + case CC of + error when MaybeUnsupported -> + ct:comment("Padding unsupported?",[]); + _ -> + ct:fail({?FUNCTION_NAME,CC,EE,(Type),(Padding)}) + end + ). + do_public_encrypt({Type, Public, Private, Msg, Padding}) -> - PublicEcn = (catch crypto:public_encrypt(Type, Msg, Public, Padding)), - case crypto:private_decrypt(Type, PublicEcn, Private, Padding) of - Msg -> - ok; - Other -> - ct:fail({{crypto, private_decrypt, [Type, PublicEcn, Private, Padding]}, {expected, Msg}, {got, Other}}) + try + crypto:public_encrypt(Type, Msg, Public, Padding) + of + PublicEcn -> + try + crypto:private_decrypt(Type, PublicEcn, Private, Padding) + of + Msg -> + ok; + Other -> + ct:fail({{crypto, private_decrypt, [Type, PublicEcn, Private, Padding]}, {expected, Msg}, {got, Other}}) + catch + ?PUB_PRIV_ENC_DEC_CATCH(Type, Padding) + end + catch + ?PUB_PRIV_ENC_DEC_CATCH(Type, Padding) end. -do_private_encrypt({_Type, _Public, _Private, _Msg, rsa_pkcs1_oaep_padding}) -> - ok; %% Not supported by openssl + do_private_encrypt({Type, Public, Private, Msg, Padding}) -> - PrivEcn = (catch crypto:private_encrypt(Type, Msg, Private, Padding)), - case crypto:public_decrypt(Type, PrivEcn, Public, Padding) of - Msg -> - ok; - Other -> - ct:fail({{crypto, public_decrypt, [Type, PrivEcn, Public, Padding]}, {expected, Msg}, {got, Other}}) + try + crypto:private_encrypt(Type, Msg, Private, Padding) + of + PrivEcn -> + try + crypto:public_decrypt(Type, PrivEcn, Public, Padding) + of + Msg -> + ok; + Other -> + ct:fail({{crypto, public_decrypt, [Type, PrivEcn, Public, Padding]}, {expected, Msg}, {got, Other}}) + catch + ?PUB_PRIV_ENC_DEC_CATCH(Type, Padding) + end + catch + ?PUB_PRIV_ENC_DEC_CATCH(Type, Padding) end. - + do_generate_compute({srp = Type, UserPrivate, UserGenParams, UserComParams, HostPublic, HostPrivate, HostGenParams, HostComParam, SessionKey}) -> {UserPublic, UserPrivate} = crypto:generate_key(Type, UserGenParams, UserPrivate), @@ -1241,7 +1288,12 @@ group_config(rsa = Type, Config) -> end, MsgPubEnc = <<"7896345786348 Asldi">>, PubPrivEnc = [{rsa, PublicS, PrivateS, MsgPubEnc, rsa_pkcs1_padding}, + {rsa, PublicS, PrivateS, MsgPubEnc, [{rsa_padding, rsa_pkcs1_padding}]}, + {rsa, PublicS, PrivateS, MsgPubEnc, [{rsa_padding, rsa_sslv23_padding}]}, + {rsa, PublicS, PrivateS, MsgPubEnc, [{rsa_padding, rsa_x931_padding}]}, rsa_oaep(), + rsa_oaep_label(), + rsa_oaep256(), no_padding() ], Generate = [{rsa, 1024, 3}, {rsa, 2048, 17}, {rsa, 3072, 65537}], @@ -2334,7 +2386,32 @@ rsa_oaep() -> hexstr2bin("4f456c502493bdc0ed2ab756a3a6ed4d67352a697d4216e93212b127a63d5411ce6fa98d5dbefd73263e3728142743818166ed7dd63687dd2a8ca1d2f4fbd8e1")], %%Msg = hexstr2bin("6628194e12073db03ba94cda9ef9532397d50dba79b987004afefe34"), Msg = hexstr2bin("750c4047f547e8e41411856523298ac9bae245efaf1397fbe56f9dd5"), - {rsa, Public, Private, Msg, rsa_pkcs1_oaep_padding}. + {rsa, Public, Private, Msg, [{rsa_padding, rsa_pkcs1_oaep_padding}]}. + +rsa_oaep_label() -> + Public = [hexstr2bin("010001"), + hexstr2bin("a8b3b284af8eb50b387034a860f146c4919f318763cd6c5598c8ae4811a1e0abc4c7e0b082d693a5e7fced675cf4668512772c0cbc64a742c6c630f533c8cc72f62ae833c40bf25842e984bb78bdbf97c0107d55bdb662f5c4e0fab9845cb5148ef7392dd3aaff93ae1e6b667bb3d4247616d4f5ba10d4cfd226de88d39f16fb")], + Private = Public ++ [hexstr2bin("53339cfdb79fc8466a655c7316aca85c55fd8f6dd898fdaf119517ef4f52e8fd8e258df93fee180fa0e4ab29693cd83b152a553d4ac4d1812b8b9fa5af0e7f55fe7304df41570926f3311f15c4d65a732c483116ee3d3d2d0af3549ad9bf7cbfb78ad884f84d5beb04724dc7369b31def37d0cf539e9cfcdd3de653729ead5d1"), + hexstr2bin("d32737e7267ffe1341b2d5c0d150a81b586fb3132bed2f8d5262864a9cb9f30af38be448598d413a172efb802c21acf1c11c520c2f26a471dcad212eac7ca39d"), + hexstr2bin("cc8853d1d54da630fac004f471f281c7b8982d8224a490edbeb33d3e3d5cc93c4765703d1dd791642f1f116a0dd852be2419b2af72bfe9a030e860b0288b5d77"), + hexstr2bin("0e12bf1718e9cef5599ba1c3882fe8046a90874eefce8f2ccc20e4f2741fb0a33a3848aec9c9305fbecbd2d76819967d4671acc6431e4037968db37878e695c1"), + hexstr2bin("95297b0f95a2fa67d00707d609dfd4fc05c89dafc2ef6d6ea55bec771ea333734d9251e79082ecda866efef13c459e1a631386b7e354c899f5f112ca85d71583"), + hexstr2bin("4f456c502493bdc0ed2ab756a3a6ed4d67352a697d4216e93212b127a63d5411ce6fa98d5dbefd73263e3728142743818166ed7dd63687dd2a8ca1d2f4fbd8e1")], + Msg = hexstr2bin("750c4047f547e8e41411856523298ac9bae245efaf1397fbe56f9dd5"), + Lbl = hexstr2bin("1332a67ca7088f75c9b8fb5e3d072882"), + {rsa, Public, Private, Msg, [{rsa_padding, rsa_pkcs1_oaep_padding}, {rsa_oaep_label, Lbl}]}. + +rsa_oaep256() -> + Public = [hexstr2bin("010001"), + hexstr2bin("a8b3b284af8eb50b387034a860f146c4919f318763cd6c5598c8ae4811a1e0abc4c7e0b082d693a5e7fced675cf4668512772c0cbc64a742c6c630f533c8cc72f62ae833c40bf25842e984bb78bdbf97c0107d55bdb662f5c4e0fab9845cb5148ef7392dd3aaff93ae1e6b667bb3d4247616d4f5ba10d4cfd226de88d39f16fb")], + Private = Public ++ [hexstr2bin("53339cfdb79fc8466a655c7316aca85c55fd8f6dd898fdaf119517ef4f52e8fd8e258df93fee180fa0e4ab29693cd83b152a553d4ac4d1812b8b9fa5af0e7f55fe7304df41570926f3311f15c4d65a732c483116ee3d3d2d0af3549ad9bf7cbfb78ad884f84d5beb04724dc7369b31def37d0cf539e9cfcdd3de653729ead5d1"), + hexstr2bin("d32737e7267ffe1341b2d5c0d150a81b586fb3132bed2f8d5262864a9cb9f30af38be448598d413a172efb802c21acf1c11c520c2f26a471dcad212eac7ca39d"), + hexstr2bin("cc8853d1d54da630fac004f471f281c7b8982d8224a490edbeb33d3e3d5cc93c4765703d1dd791642f1f116a0dd852be2419b2af72bfe9a030e860b0288b5d77"), + hexstr2bin("0e12bf1718e9cef5599ba1c3882fe8046a90874eefce8f2ccc20e4f2741fb0a33a3848aec9c9305fbecbd2d76819967d4671acc6431e4037968db37878e695c1"), + hexstr2bin("95297b0f95a2fa67d00707d609dfd4fc05c89dafc2ef6d6ea55bec771ea333734d9251e79082ecda866efef13c459e1a631386b7e354c899f5f112ca85d71583"), + hexstr2bin("4f456c502493bdc0ed2ab756a3a6ed4d67352a697d4216e93212b127a63d5411ce6fa98d5dbefd73263e3728142743818166ed7dd63687dd2a8ca1d2f4fbd8e1")], + Msg = hexstr2bin("750c4047f547e8e41411856523298ac9bae245efaf1397fbe56f9dd5"), + {rsa, Public, Private, Msg, [{rsa_padding, rsa_pkcs1_oaep_padding}, {rsa_oaep_md, sha256}]}. ecc() -> %% http://point-at-infinity.org/ecc/nisttv @@ -2510,3 +2587,14 @@ parse_rsp_cmac(Type, Key0, Msg0, Mlen0, Tlen, MAC0, Next, Acc) -> I -> parse_rsp(Type, Next, [{Type, Key, Msg, I, MAC}|Acc]) end. + +api_errors_ecdh(Config) when is_list(Config) -> + %% Check that we don't segfault when fed garbage. + Test = fun(Others, Curve) -> + {_Pub, Priv} = crypto:generate_key(ecdh, Curve), + crypto:compute_key(ecdh, Others, Priv, Curve) + end, + Others = [gurka, 0, <<0>>], + Curves = [gaffel, 0, sect571r1], + [_= (catch Test(O, C)) || O <- Others, C <- Curves], + ok. diff --git a/lib/crypto/test/engine_SUITE.erl b/lib/crypto/test/engine_SUITE.erl index 3f176f165d..f708d7272d 100644 --- a/lib/crypto/test/engine_SUITE.erl +++ b/lib/crypto/test/engine_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2017-2017. All Rights Reserved. +%% Copyright Ericsson AB 2017-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/crypto/vsn.mk b/lib/crypto/vsn.mk index 778aff9d13..0d7b0e5575 100644 --- a/lib/crypto/vsn.mk +++ b/lib/crypto/vsn.mk @@ -1 +1 @@ -CRYPTO_VSN = 4.2.2 +CRYPTO_VSN = 4.3.1 diff --git a/lib/debugger/doc/src/Makefile b/lib/debugger/doc/src/Makefile index cc0b8861d3..56d6085e9c 100644 --- a/lib/debugger/doc/src/Makefile +++ b/lib/debugger/doc/src/Makefile @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 1997-2016. All Rights Reserved. +# Copyright Ericsson AB 1997-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -99,6 +99,7 @@ gifs: $(GIF_FILES:%=$(HTMLDIR)/%) clean clean_docs: rm -rf $(HTMLDIR)/* + rm -rf $(XMLDIR) rm -f $(MAN3DIR)/* rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) rm -f errs core *~ diff --git a/lib/debugger/doc/src/notes.xml b/lib/debugger/doc/src/notes.xml index e71746e30d..9f7214ac69 100644 --- a/lib/debugger/doc/src/notes.xml +++ b/lib/debugger/doc/src/notes.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2004</year><year>2017</year> + <year>2004</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -33,6 +33,21 @@ <p>This document describes the changes made to the Debugger application.</p> +<section><title>Debugger 4.2.5</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> Fix a bug where calling a fun inside a binary would + crash the Debugger. </p> + <p> + Own Id: OTP-14957 Aux Id: PR-1741 </p> + </item> + </list> + </section> + +</section> + <section><title>Debugger 4.2.4</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/debugger/src/dbg_debugged.erl b/lib/debugger/src/dbg_debugged.erl index 8b64a60a9c..5296b8d892 100644 --- a/lib/debugger/src/dbg_debugged.erl +++ b/lib/debugger/src/dbg_debugged.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1998-2016. All Rights Reserved. +%% Copyright Ericsson AB 1998-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/debugger/src/dbg_icmd.erl b/lib/debugger/src/dbg_icmd.erl index 55cbada53b..0eb258567f 100644 --- a/lib/debugger/src/dbg_icmd.erl +++ b/lib/debugger/src/dbg_icmd.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1998-2017. All Rights Reserved. +%% Copyright Ericsson AB 1998-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -248,7 +248,7 @@ handle_int_msg({attached, AttPid}, Status, _Bs, tell_attached({attached, M, Line, get(trace)}), %% Give info about status and call level as well - %% In this case, Status can not be exit_at + %% In this case, Status cannot be exit_at Msg = case Status of idle -> {func_at,M,Line,Le}; break -> {break_at,M,Line,Le}; diff --git a/lib/debugger/src/dbg_ieval.erl b/lib/debugger/src/dbg_ieval.erl index 9840cebc1a..b6703d5d9e 100644 --- a/lib/debugger/src/dbg_ieval.erl +++ b/lib/debugger/src/dbg_ieval.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1998-2017. All Rights Reserved. +%% Copyright Ericsson AB 1998-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/debugger/src/dbg_wx_mon.erl b/lib/debugger/src/dbg_wx_mon.erl index 00aee62a87..08b20d3f69 100644 --- a/lib/debugger/src/dbg_wx_mon.erl +++ b/lib/debugger/src/dbg_wx_mon.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2016. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/debugger/src/dbg_wx_trace.erl b/lib/debugger/src/dbg_wx_trace.erl index 505d53005f..25f32ca7e7 100644 --- a/lib/debugger/src/dbg_wx_trace.erl +++ b/lib/debugger/src/dbg_wx_trace.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2017. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -518,8 +518,8 @@ gui_cmd({user_command, Cmd}, State) -> gui_cmd({edit, {Var, Value}}, State) -> Window = dbg_wx_trace_win:get_window(State#state.win), Val = case State#state.strings of - [] -> dbg_wx_win:to_string("~999999lp",[Value]); - [str_on] -> dbg_wx_win:to_string("~999999tp",[Value]) + [] -> dbg_wx_win:to_string("~0lp",[Value]); + [str_on] -> dbg_wx_win:to_string("~0tp",[Value]) end, case dbg_wx_win:entry(Window, "Edit variable", Var, {term, Val}) of cancel -> diff --git a/lib/debugger/src/dbg_wx_win.erl b/lib/debugger/src/dbg_wx_win.erl index fea94156c1..4391ad1598 100644 --- a/lib/debugger/src/dbg_wx_win.erl +++ b/lib/debugger/src/dbg_wx_win.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2017. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/debugger/src/debugger.app.src b/lib/debugger/src/debugger.app.src index 37a41c1a56..c8a8b0fd43 100644 --- a/lib/debugger/src/debugger.app.src +++ b/lib/debugger/src/debugger.app.src @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2015. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/debugger/test/guard_SUITE.erl b/lib/debugger/test/guard_SUITE.erl index e781ea932f..02a1bbd66b 100644 --- a/lib/debugger/test/guard_SUITE.erl +++ b/lib/debugger/test/guard_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2016. All Rights Reserved. +%% Copyright Ericsson AB 1999-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/debugger/test/int_eval_SUITE.erl b/lib/debugger/test/int_eval_SUITE.erl index da1d6734f8..0542e45142 100644 --- a/lib/debugger/test/int_eval_SUITE.erl +++ b/lib/debugger/test/int_eval_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2016. All Rights Reserved. +%% Copyright Ericsson AB 1999-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/debugger/test/int_eval_SUITE_data/my_int_eval_module.erl b/lib/debugger/test/int_eval_SUITE_data/my_int_eval_module.erl index 384d61f051..aa26a44686 100644 --- a/lib/debugger/test/int_eval_SUITE_data/my_int_eval_module.erl +++ b/lib/debugger/test/int_eval_SUITE_data/my_int_eval_module.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2016. All Rights Reserved. +%% Copyright Ericsson AB 1999-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/debugger/vsn.mk b/lib/debugger/vsn.mk index 57da7e5618..d62e5915a7 100644 --- a/lib/debugger/vsn.mk +++ b/lib/debugger/vsn.mk @@ -1 +1 @@ -DEBUGGER_VSN = 4.2.4 +DEBUGGER_VSN = 4.2.5 diff --git a/lib/dialyzer/doc/src/Makefile b/lib/dialyzer/doc/src/Makefile index 3463b589e6..3ce777392b 100644 --- a/lib/dialyzer/doc/src/Makefile +++ b/lib/dialyzer/doc/src/Makefile @@ -91,6 +91,7 @@ debug opt: clean clean_docs: rm -rf $(HTMLDIR)/* + rm -rf $(XMLDIR) rm -f $(MAN3DIR)/* rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) rm -f errs core *~ diff --git a/lib/dialyzer/doc/src/notes.xml b/lib/dialyzer/doc/src/notes.xml index 8d11252bff..4ec72eecd9 100644 --- a/lib/dialyzer/doc/src/notes.xml +++ b/lib/dialyzer/doc/src/notes.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2006</year><year>2017</year> + <year>2006</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -32,6 +32,60 @@ <p>This document describes the changes made to the Dialyzer application.</p> +<section><title>Dialyzer 3.3</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p>Changed the default behaviour of <c>.erlang</c> + loading: <c>.erlang</c> is no longer loaded from the + current directory. <c>c:erlangrc(PathList)</c> can be + used to search and load an <c>.erlang</c> file from user + specified directories.</p> <p><c>escript</c>, + <c>erlc</c>, <c>dialyzer</c> and <c>typer</c> no longer + load an <c>.erlang</c> at all.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-14439</p> + </item> + <item> + <p> Dialyzer can no longer read BEAM files created with + OTP 19 or earlier. </p> + <p> + Own Id: OTP-14493 Aux Id: PR-1434 </p> + </item> + <item> + <p> Speed up the computation of MD5 sums. </p> + <p> + Own Id: OTP-14937 Aux Id: PR-1719 </p> + </item> + <item> + <p> Fix a situation where Dialyzer unnecessarily + discarded contract information, resulting in missed + warnings. </p> + <p> + Own Id: OTP-14970 Aux Id: PR-1722 </p> + </item> + <item> + <p> The (not recommended) option <c>-Woverspecs</c> is + somewhat refined, and generates warnings in a few more + cases. </p> + <p> + Own Id: OTP-14982 Aux Id: OTP-14970, PR-1722 </p> + </item> + <item> + <p> Do not emit warnings for fun expressions residing in + code that cannot be run. This is consistent with how + Dialyzer treats other code that cannot be run. </p> + <p> + Own Id: OTP-15079 Aux Id: ERL-593 </p> + </item> + </list> + </section> + +</section> + <section><title>Dialyzer 3.2.4</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/dialyzer/src/typer.erl b/lib/dialyzer/src/typer.erl index 4b99f5f72e..5a1b8619c9 100644 --- a/lib/dialyzer/src/typer.erl +++ b/lib/dialyzer/src/typer.erl @@ -980,7 +980,7 @@ fatal_error(Slogan) -> mode_error(OldMode, NewMode) -> Msg = io_lib:format("Mode was previously set to '~s'; " - "can not set it to '~s' now", + "cannot set it to '~s' now", [OldMode, NewMode]), fatal_error(Msg). diff --git a/lib/dialyzer/test/options1_SUITE_data/src/compiler/beam_validator.erl b/lib/dialyzer/test/options1_SUITE_data/src/compiler/beam_validator.erl index ea92613781..3606b21932 100644 --- a/lib/dialyzer/test/options1_SUITE_data/src/compiler/beam_validator.erl +++ b/lib/dialyzer/test/options1_SUITE_data/src/compiler/beam_validator.erl @@ -272,7 +272,7 @@ valfun_1(if_end, Vst) -> valfun_1({try_case_end,Src}, Vst) -> assert_term(Src, Vst), kill_state(Vst); -%% Instructions that can not cause exceptions +%% Instructions that cannot cause exceptions valfun_1({move,Src,Dst}, Vst) -> Type = get_term_type(Src, Vst), set_type_reg(Type, Dst, Vst); diff --git a/lib/dialyzer/test/r9c_SUITE_data/src/asn1/asn1ct_gen_ber.erl b/lib/dialyzer/test/r9c_SUITE_data/src/asn1/asn1ct_gen_ber.erl index 1d065b3723..88b464a721 100644 --- a/lib/dialyzer/test/r9c_SUITE_data/src/asn1/asn1ct_gen_ber.erl +++ b/lib/dialyzer/test/r9c_SUITE_data/src/asn1/asn1ct_gen_ber.erl @@ -300,7 +300,7 @@ gen_encode_prim(_Erules,D,DoTag,Value) when record(D,type) -> 'ASN1_OPEN_TYPE' -> emit_encode_func('open_type', Value,DoTag); XX -> - exit({'can not encode' ,XX}) + exit({'cannot encode' ,XX}) end. @@ -562,7 +562,7 @@ gen_dec_prim(Erules,Att,BytesVar,DoTag,TagIn,Length,_Form,OptOrMand) -> BytesVar,","]), false; Other -> - exit({'can not decode' ,Other}) + exit({'cannot decode' ,Other}) end, NewLength = case DoLength of diff --git a/lib/dialyzer/test/r9c_SUITE_data/src/asn1/asn1ct_gen_ber_bin_v2.erl b/lib/dialyzer/test/r9c_SUITE_data/src/asn1/asn1ct_gen_ber_bin_v2.erl index 9164ec6551..573bb15c92 100644 --- a/lib/dialyzer/test/r9c_SUITE_data/src/asn1/asn1ct_gen_ber_bin_v2.erl +++ b/lib/dialyzer/test/r9c_SUITE_data/src/asn1/asn1ct_gen_ber_bin_v2.erl @@ -290,7 +290,7 @@ gen_encode_prim(_Erules,D,DoTag,Value) when record(D,type) -> 'ASN1_OPEN_TYPE' -> emit_encode_func('open_type', Value,DoTag); XX -> - exit({'can not encode' ,XX}) + exit({'cannot encode' ,XX}) end. @@ -602,7 +602,7 @@ gen_dec_prim(_Erules,Att,BytesVar,DoTag,_TagIn,_Form,_OptOrMand) -> BytesVar,","]), add_func({decode_open_type_as_binary,2}); Other -> - exit({'can not decode' ,Other}) + exit({'cannot decode' ,Other}) end, case {DoTag,NewTypeName} of diff --git a/lib/dialyzer/test/r9c_SUITE_data/src/mnesia/mnesia_lib.erl b/lib/dialyzer/test/r9c_SUITE_data/src/mnesia/mnesia_lib.erl index 4008f8d789..9e0cdac2dc 100644 --- a/lib/dialyzer/test/r9c_SUITE_data/src/mnesia/mnesia_lib.erl +++ b/lib/dialyzer/test/r9c_SUITE_data/src/mnesia/mnesia_lib.erl @@ -779,7 +779,7 @@ error_desc(no_transaction) -> "Operation not allowed outside transactions"; error_desc(combine_error) -> "Table options were ilegally combined"; error_desc(bad_index) -> "Index already exists or was out of bounds"; error_desc(already_exists) -> "Some schema option we try to set is already on"; -error_desc(index_exists)-> "Some ops can not be performed on tabs with index"; +error_desc(index_exists)-> "Some ops cannot be performed on tabs with index"; error_desc(no_exists)-> "Tried to perform op on non-existing (non alive) item"; error_desc(system_limit) -> "Some system_limit was exhausted"; error_desc(mnesia_down) -> "A transaction involving objects at some remote " diff --git a/lib/dialyzer/test/r9c_SUITE_data/src/mnesia/mnesia_tm.erl b/lib/dialyzer/test/r9c_SUITE_data/src/mnesia/mnesia_tm.erl index af49ceff72..b4e9edb7ce 100644 --- a/lib/dialyzer/test/r9c_SUITE_data/src/mnesia/mnesia_tm.erl +++ b/lib/dialyzer/test/r9c_SUITE_data/src/mnesia/mnesia_tm.erl @@ -1529,7 +1529,7 @@ commit_participant(Coord, Tid, Bin, C0, DiscNs, _RamNs) -> ?eval_debug_fun({?MODULE, commit_participant, pre}, [{tid, Tid}]), case catch mnesia_schema:prepare_commit(Tid, C0, {part, Coord}) of {Modified, C, DumperMode} when record(C, commit) -> - %% If we can not find any local unclear decision + %% If we cannot find any local unclear decision %% we should presume abort at startup recovery case lists:member(node(), DiscNs) of false -> diff --git a/lib/dialyzer/test/small_SUITE_data/results/maps_sum b/lib/dialyzer/test/small_SUITE_data/results/maps_sum index b29ac77d88..daa099e490 100644 --- a/lib/dialyzer/test/small_SUITE_data/results/maps_sum +++ b/lib/dialyzer/test/small_SUITE_data/results/maps_sum @@ -1,4 +1,4 @@ -maps_sum.erl:15: Invalid type specification for function maps_sum:wrong1/1. The success typing is (maps:iterator() | map()) -> any() +maps_sum.erl:15: Invalid type specification for function maps_sum:wrong1/1. The success typing is (maps:iterator(_,_) | map()) -> any() maps_sum.erl:26: Function wrong2/1 has no local return maps_sum.erl:27: The call lists:foldl(fun((_,_,_) -> any()),0,Data::any()) will never return since it differs in the 1st argument from the success typing arguments: (fun((_,_) -> any()),any(),[any()]) diff --git a/lib/dialyzer/vsn.mk b/lib/dialyzer/vsn.mk index fa58adc2db..7240b66fb5 100644 --- a/lib/dialyzer/vsn.mk +++ b/lib/dialyzer/vsn.mk @@ -1 +1 @@ -DIALYZER_VSN = 3.2.4 +DIALYZER_VSN = 3.3 diff --git a/lib/diameter/bin/diameterc b/lib/diameter/bin/diameterc index 3dbd238c19..4d415ece78 100755 --- a/lib/diameter/bin/diameterc +++ b/lib/diameter/bin/diameterc @@ -4,7 +4,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2016. All Rights Reserved. +%% Copyright Ericsson AB 2010-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -78,8 +78,8 @@ compile(#argv{file = File, options = Opts, output = Out}) -> error_msg(diameter_make:format_error(Reason), []), 1 catch - error: Reason -> - error_msg("ERROR: ~p~n ~p", [Reason, erlang:get_stacktrace()]), + error: Reason: Stack -> + error_msg("ERROR: ~p~n ~p", [Reason, Stack]), 2 end. diff --git a/lib/diameter/doc/src/Makefile b/lib/diameter/doc/src/Makefile index 7a7546fc4d..7672598060 100644 --- a/lib/diameter/doc/src/Makefile +++ b/lib/diameter/doc/src/Makefile @@ -77,6 +77,7 @@ clean_man: clean_html: rm -rf $(HTMLDIR)/* + rm -rf $(XMLDIR) gifs: $(GIF_FILES:%=$(HTMLDIR)/%) diff --git a/lib/diameter/doc/src/notes.xml b/lib/diameter/doc/src/notes.xml index f74a4ca2a2..3232c3380a 100644 --- a/lib/diameter/doc/src/notes.xml +++ b/lib/diameter/doc/src/notes.xml @@ -11,7 +11,7 @@ <header> <copyright> <year>2011</year> -<year>2017</year> +<year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -43,6 +43,21 @@ first.</p> <!-- ===================================================================== --> +<section><title>diameter 2.1.5</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fix documentation typos.</p> + <p> + Own Id: OTP-15045</p> + </item> + </list> + </section> + +</section> + <section><title>diameter 2.1.4</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/diameter/examples/code/node.erl b/lib/diameter/examples/code/node.erl index fc5830f8e2..77810bf893 100644 --- a/lib/diameter/examples/code/node.erl +++ b/lib/diameter/examples/code/node.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2016. All Rights Reserved. +%% Copyright Ericsson AB 2010-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/diameter/src/Makefile b/lib/diameter/src/Makefile index 3af856f63e..98636ed6e2 100644 --- a/lib/diameter/src/Makefile +++ b/lib/diameter/src/Makefile @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 2010-2017. All Rights Reserved. +# Copyright Ericsson AB 2010-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -149,7 +149,7 @@ gen/$(DICT_YRL).erl: compiler/$(DICT_YRL).yrl $(ERLC) -Werror -o $(@D) $< # Generate the app file. -$(APP_TARGET): $(APP_SRC) ../vsn.mk modules.mk app.sed +$(APP_TARGET): $(APP_SRC) ../vsn.mk modules.mk $(gen_verbose) \ M=`echo $(notdir $(APP_MODULES)) | tr ' ' ,`; \ C=`echo $(COMPILER_MODULES) | tr ' ' ,`; \ @@ -160,8 +160,7 @@ $(APP_TARGET): $(APP_SRC) ../vsn.mk modules.mk app.sed -e "s;%COMPILER%;$$C;" \ -e "s;%INFO%;$$I;" \ -e "s;%REGISTERED%;$$R;" \ - $< \ - | sed -f app.sed > $@ + $< > $@ $(APPUP_TARGET): $(APPUP_SRC) ../vsn.mk $(vsn_verbose) \ diff --git a/lib/diameter/src/app.sed b/lib/diameter/src/app.sed deleted file mode 100644 index dd3806f5f1..0000000000 --- a/lib/diameter/src/app.sed +++ /dev/null @@ -1,41 +0,0 @@ -# -# %CopyrightBegin% -# -# Copyright Ericsson AB 2014-2016. All Rights Reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# -# %CopyrightEnd% - -# -# Generate runtime_dependencies from applications to avoid having to -# specify the same application more than once. -# - -/{runtime_dependencies,/b v -/{[-a-z]*, "[0-9.]*"}/!b -/{vsn,/b - -/%%/!H -s/{\([^,]*\)[^}]*}/\1/g -s/%%/%,/ -b - -:v - -p -x -s/\n// -s/%//g -s/\n */ /g -s/{\([^,]*\), "\([^"]*"\)}/"\1-\2/g diff --git a/lib/diameter/src/base/diameter_codec.erl b/lib/diameter/src/base/diameter_codec.erl index 2dd2c906a2..493a6ab1e3 100644 --- a/lib/diameter/src/base/diameter_codec.erl +++ b/lib/diameter/src/base/diameter_codec.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2017. All Rights Reserved. +%% Copyright Ericsson AB 2010-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -92,8 +92,8 @@ encode(Mod, Opts, #diameter_packet{} = Pkt) -> %% count encode errors. ?LOG(encode_error, {Reason, Stack, H}), exit({?MODULE, encode, T}); - error: Reason -> - T = {Reason, diameter_lib:get_stacktrace()}, + error: Reason: Stack -> + T = {Reason, diameter_lib:stacktrace(Stack)}, ?LOG(encode_error, T), exit({?MODULE, encode, T}) end; @@ -134,8 +134,8 @@ enc(_, Opts, #diameter_packet{msg = [#diameter_header{} = Hdr | As]} Eid:32, Bin/binary>>} catch - error: Reason -> - exit({Reason, diameter_lib:get_stacktrace(), Hdr}) + error: Reason: Stack -> + exit({Reason, diameter_lib:stacktrace(Stack), Hdr}) end; enc(Mod, Opts, #diameter_packet{header = Hdr0, msg = Msg} = Pkt) -> @@ -179,14 +179,14 @@ enc(Mod, Opts, #diameter_packet{header = Hdr0, msg = Msg} = Pkt) -> Eid:32, Bin/binary>>} catch - error: Reason -> + error: Reason: Stack -> Hdr = Hdr0#diameter_header{cmd_code = Code, application_id = Aid, is_request = RB, is_proxiable = PB, is_error = EB, is_retransmitted = TB}, - exit({Reason, diameter_lib:get_stacktrace(), Hdr}) + exit({Reason, diameter_lib:stacktrace(Stack), Hdr}) end. %% values/1 diff --git a/lib/diameter/src/base/diameter_config.erl b/lib/diameter/src/base/diameter_config.erl index 90a9282349..36ae4c2276 100644 --- a/lib/diameter/src/base/diameter_config.erl +++ b/lib/diameter/src/base/diameter_config.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2017. All Rights Reserved. +%% Copyright Ericsson AB 2010-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -633,8 +633,8 @@ opt(service, {K, F}) Nodes -> is_list(Nodes) orelse {error, Nodes} catch - E:R -> - {error, {E, R, ?STACK}} + E:R:Stack -> + {error, {E, R, Stack}} end; opt(service, {sequence, {H,N}}) -> @@ -651,8 +651,8 @@ opt(service = S, {sequence = K, F}) -> V -> {error, V} catch - E:R -> - {error, {E, R, ?STACK}} + E:R:Stack -> + {error, {E, R, Stack}} end; opt(transport, {transport_module, M}) -> @@ -932,8 +932,8 @@ cb(M,F) -> try M:F() of V -> V catch - E: Reason -> - ?THROW({callback, E, Reason, ?STACK}) + E: Reason: Stack -> + ?THROW({callback, E, Reason, Stack}) end. %% call/1 diff --git a/lib/diameter/src/base/diameter_gen.erl b/lib/diameter/src/base/diameter_gen.erl index 93ebe57685..d110a3015e 100644 --- a/lib/diameter/src/base/diameter_gen.erl +++ b/lib/diameter/src/base/diameter_gen.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2017. All Rights Reserved. +%% Copyright Ericsson AB 2010-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -76,8 +76,7 @@ encode_avps(Name, Vals, #{module := Mod} = Opts) -> ?LINE, {Reason, Name, Vals, Mod}), erlang:error(list_to_tuple(Reason ++ [Name])); - error: Reason -> - Stack = erlang:get_stacktrace(), + error: Reason: Stack -> diameter_lib:log({encode, failure}, ?MODULE, ?LINE, @@ -555,8 +554,8 @@ dec(Data, Name, AvpName, Type, Mod, Dict, Fmt, Failed, Opts, Avp) -> catch throw: {?MODULE, T} -> decode_error(Failed, Fmt, T, Avp); - error: Reason -> - decode_error(Failed, Reason, Name, Mod, Opts, Avp) + error: Reason: Stack -> + decode_error(Failed, Reason, Stack, Name, Mod, Opts, Avp) end. %% dec_AVP/7 @@ -623,19 +622,19 @@ set(none, Avp, _Name) -> set(_, Avp, Rec) -> Avp#diameter_avp{value = Rec}. -%% decode_error/6 +%% decode_error/7 %% %% Error when decoding a non-grouped AVP. -decode_error(true, _, _, _, _, Avp) -> +decode_error(true, _, _, _, _, _, Avp) -> Avp; -decode_error(false, Reason, Name, Mod, Opts, Avp) -> - Stack = diameter_lib:get_stacktrace(), +decode_error(false, Reason, Stack, Name, Mod, Opts, Avp) -> + Z = diameter_lib:stacktrace(Stack), diameter_lib:log(decode_error, ?MODULE, ?LINE, - {Reason, Name, Avp#diameter_avp.name, Mod, Stack}), + {Reason, Name, Avp#diameter_avp.name, Mod, Z}), case Reason of {'DIAMETER', 5014 = RC, _} -> %% Length error communicated from diameter_types or a diff --git a/lib/diameter/src/base/diameter_internal.hrl b/lib/diameter/src/base/diameter_internal.hrl index a0f4a8567d..4a678643c2 100644 --- a/lib/diameter/src/base/diameter_internal.hrl +++ b/lib/diameter/src/base/diameter_internal.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2016. All Rights Reserved. +%% Copyright Ericsson AB 2010-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -35,9 +35,6 @@ %% A corresponding error when failure is the best option. -define(ERROR(T), erlang:error({T, ?MODULE, ?LINE})). -%% Failure reports always get a stack trace. --define(STACK, erlang:get_stacktrace()). - %% Warning report for unexpected messages in various processes. -define(UNEXPECTED(F,A), diameter_lib:warning_report(unexpected, {?MODULE, F, A})). diff --git a/lib/diameter/src/base/diameter_lib.erl b/lib/diameter/src/base/diameter_lib.erl index 1c1ea42cb5..edd9d5a4ce 100644 --- a/lib/diameter/src/base/diameter_lib.erl +++ b/lib/diameter/src/base/diameter_lib.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2017. All Rights Reserved. +%% Copyright Ericsson AB 2010-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -33,7 +33,7 @@ time/1, eval/1, eval_name/1, - get_stacktrace/0, + stacktrace/1, ipaddr/1, spawn_opts/2, wait/1, @@ -42,16 +42,13 @@ log/4]). %% --------------------------------------------------------------------------- -%% # get_stacktrace/0 +%% # stacktrace/1 %% --------------------------------------------------------------------------- %% Return a stacktrace with a leading, potentially large, argument -%% list replaced by an arity. Trace on stacktrace/0 to see the +%% list replaced by an arity. Trace on stacktrace/1 to see the %% original. -get_stacktrace() -> - stacktrace(erlang:get_stacktrace()). - stacktrace([{M,F,A,L} | T]) when is_list(A) -> [{M, F, length(A), L} | T]; stacktrace(L) -> @@ -268,8 +265,8 @@ ipaddr(Addr) -> try ip(Addr) catch - error: _ -> - erlang:error({invalid_address, erlang:get_stacktrace()}) + error: _: Stack -> + erlang:error({invalid_address, Stack}) end. %% Already a tuple: ensure non-negative integers of the right size. diff --git a/lib/diameter/src/base/diameter_service.erl b/lib/diameter/src/base/diameter_service.erl index cbe66ef27a..77d184cfc7 100644 --- a/lib/diameter/src/base/diameter_service.erl +++ b/lib/diameter/src/base/diameter_service.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2017. All Rights Reserved. +%% Copyright Ericsson AB 2010-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -808,8 +808,8 @@ remotes(F) -> error_report(invalid_return, share_peers, F), [] catch - E:R -> - ?LOG(failure, {E, R, F, diameter_lib:get_stacktrace()}), + E:R:S -> + ?LOG(failure, {E, R, F, diameter_lib:stacktrace(S)}), error_report(failure, share_peers, F), [] end. @@ -1146,11 +1146,11 @@ peer_cb(App, F, A) -> mod_state(App#diameter_app.alias, ModS), true catch - E:R -> + E:R:S -> %% Don't include arguments since a #diameter_caps{} strings %% from the peer, which could be anything (especially, large). [Mod|X] = App#diameter_app.module, - ?LOG(failure, {E, R, Mod, F, diameter_lib:get_stacktrace()}), + ?LOG(failure, {E, R, Mod, F, diameter_lib:stacktrace(S)}), error_report(failure, F, {Mod, F, A ++ X}), false end. @@ -1376,9 +1376,9 @@ cm([#diameter_app{alias = Alias} = App], Req, From, Svc) -> ?LOG(invalid_return, {ModX, handle_call, Args, T}), invalid catch - E: Reason -> + E: Reason: S -> ModX = App#diameter_app.module, - Stack = diameter_lib:get_stacktrace(), + Stack = diameter_lib:stacktrace(S), ?LOG(failure, {E, Reason, ModX, handle_call, Stack}), failure end; @@ -1585,10 +1585,10 @@ pick_peer(Local, ?LOG(invalid_return, {ModX, pick_peer, T}), false catch - E: Reason when M -> + E: Reason: Stack when M -> ModX = App#diameter_app.module, - Stack = diameter_lib:get_stacktrace(), - ?LOG(failure, {E, Reason, ModX, pick_peer, Stack}), + Z = diameter_lib:stacktrace(Stack), + ?LOG(failure, {E, Reason, ModX, pick_peer, Z}), false end. diff --git a/lib/diameter/src/compiler/diameter_exprecs.erl b/lib/diameter/src/compiler/diameter_exprecs.erl index 143dede037..ded07f2353 100644 --- a/lib/diameter/src/compiler/diameter_exprecs.erl +++ b/lib/diameter/src/compiler/diameter_exprecs.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2016. All Rights Reserved. +%% Copyright Ericsson AB 2010-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/diameter/src/compiler/diameter_make.erl b/lib/diameter/src/compiler/diameter_make.erl index eae40dbafd..03cfc03edc 100644 --- a/lib/diameter/src/compiler/diameter_make.erl +++ b/lib/diameter/src/compiler/diameter_make.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2016. All Rights Reserved. +%% Copyright Ericsson AB 2010-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -271,6 +271,6 @@ make(File, Opts, Dict, Mode) -> try diameter_codegen:from_dict(File, Dict, Opts, Mode) catch - error: Reason -> - erlang:error({Reason, Mode, erlang:get_stacktrace()}) + error: Reason: Stack -> + erlang:error({Reason, Mode, Stack}) end. diff --git a/lib/diameter/src/diameter.app.src b/lib/diameter/src/diameter.app.src index 9a6e47006b..18202f033e 100644 --- a/lib/diameter/src/diameter.app.src +++ b/lib/diameter/src/diameter.app.src @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2017. All Rights Reserved. +%% Copyright Ericsson AB 2010-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -28,14 +28,21 @@ ]}, {registered, [%REGISTERED%]}, {applications, [ - {stdlib, "2.4"}, {kernel, "3.2"}%, {erts, "6.4"} - %% {syntax-tools, "1.6,18"} - %% {runtime-tools, "1.8.16"} - %, {ssl, "6.0"} + stdlib, + kernel + %, ssl + %, syntax-tools + %, runtime-tools ]}, {env, []}, {mod, {diameter_app, []}}, {runtime_dependencies, [ + "erts-10.0", + "stdlib-2.4", + "kernel-3.2", + "ssl-9.0" + %, "syntax-tools-1.6.18" + %, "runtime-tools-1.8.16" ]} %% %% Note that ssl is only required if configured on TCP transports, diff --git a/lib/diameter/src/transport/diameter_tcp.erl b/lib/diameter/src/transport/diameter_tcp.erl index a8639baa11..da059fa7d6 100644 --- a/lib/diameter/src/transport/diameter_tcp.erl +++ b/lib/diameter/src/transport/diameter_tcp.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2017. All Rights Reserved. +%% Copyright Ericsson AB 2010-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -716,7 +716,7 @@ tls_handshake(_, false, S) -> tls(connect, Sock, Opts) -> ssl:connect(Sock, Opts); tls(accept, Sock, Opts) -> - ssl:ssl_accept(Sock, Opts). + ssl:handshake(Sock, Opts). %% assume no handshake option %% recv/2 %% @@ -839,7 +839,7 @@ start_fragment_timer(#transport{timeout = Tmo} = S) -> accept(ssl, LSock) -> case ssl:transport_accept(LSock) of {ok, Sock} -> - {ssl:ssl_accept(Sock), Sock}; + ssl:handshake(Sock); {error, _} = No -> No end; diff --git a/lib/diameter/test/diameter_app_SUITE.erl b/lib/diameter/test/diameter_app_SUITE.erl index 71256020f5..ef4a28d3f4 100644 --- a/lib/diameter/test/diameter_app_SUITE.erl +++ b/lib/diameter/test/diameter_app_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2015. All Rights Reserved. +%% Copyright Ericsson AB 2010-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -208,9 +208,9 @@ xref(Config) -> CTmods = CTmods -- Mods, %% Ensure that runtime modules only call other runtime modules, or - %% applications declared as in runtime_dependencies in the app - %% file. Note that the declared application versions are ignored - %% since we only know what we can see now. + %% applications declared in runtime_dependencies in the app file. + %% The declared application versions are ignored since we only + %% know what we see now. [] = lists:filter(fun(M) -> not lists:member(app(M), Deps) end, RTdeps -- Mods). @@ -261,8 +261,12 @@ app(Mod) -> case code:which(Mod) of preloaded -> "erts"; + Reason when is_atom(Reason) -> + error({Reason, Mod}); Path -> - unversion(lists:nth(3, lists:reverse(filename:split(Path)))) + %% match to identify an unexpectedly short path + {_, _, [_,_,_|_] = Split} = {Mod, Path, filename:split(Path)}, + unversion(lists:nth(3, lists:reverse(Split))) end. add_application(XRef, App) -> diff --git a/lib/diameter/test/diameter_traffic_SUITE.erl b/lib/diameter/test/diameter_traffic_SUITE.erl index c224f9a27e..434aef01dd 100644 --- a/lib/diameter/test/diameter_traffic_SUITE.erl +++ b/lib/diameter/test/diameter_traffic_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2017. All Rights Reserved. +%% Copyright Ericsson AB 2010-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -1924,8 +1924,8 @@ compile_and_load() -> {module, nas4005} = code:load_binary(nas4005, "nas4005", Bin), true catch - E:R -> - {E, R, erlang:get_stacktrace()} + E:R:Stack -> + {E, R, Stack} end. here() -> diff --git a/lib/diameter/test/diameter_watchdog_SUITE.erl b/lib/diameter/test/diameter_watchdog_SUITE.erl index 39c4f051a5..f3f168e671 100644 --- a/lib/diameter/test/diameter_watchdog_SUITE.erl +++ b/lib/diameter/test/diameter_watchdog_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2017. All Rights Reserved. +%% Copyright Ericsson AB 2010-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -638,10 +638,9 @@ run1([F|A]) -> apply(?MODULE, F, A), ok catch - E:R -> - S = erlang:get_stacktrace(), - ?WARN("~p", [{A, E, R, S}]), - S + E:R:Stack -> + ?WARN("~p", [{A, E, R, Stack}]), + Stack end. %% jitter/2 diff --git a/lib/edoc/doc/src/Makefile b/lib/edoc/doc/src/Makefile index 71de42795a..aba94a6802 100644 --- a/lib/edoc/doc/src/Makefile +++ b/lib/edoc/doc/src/Makefile @@ -114,6 +114,7 @@ debug opt: clean clean_docs: rm -rf $(HTMLDIR)/* + rm -rf $(XMLDIR) rm -f $(MAN3DIR)/* rm -f $(XML_REF3_FILES) $(XML_CHAPTER_FILES) *.html rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) diff --git a/lib/edoc/doc/src/notes.xml b/lib/edoc/doc/src/notes.xml index 240789e876..2ef941b06d 100644 --- a/lib/edoc/doc/src/notes.xml +++ b/lib/edoc/doc/src/notes.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2007</year><year>2017</year> + <year>2007</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -32,6 +32,33 @@ <p>This document describes the changes made to the EDoc application.</p> +<section><title>Edoc 0.9.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> Types and function specifications including the + <c>map()</c> type are pretty-printed correctly. </p> + <p> + Own Id: OTP-15117</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Update to use the new string api instead of the old.</p> + <p> + Own Id: OTP-15036</p> + </item> + </list> + </section> + +</section> + <section><title>Edoc 0.9.2</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/edoc/src/edoc_layout.erl b/lib/edoc/src/edoc_layout.erl index baa147410b..47ff7b21fc 100644 --- a/lib/edoc/src/edoc_layout.erl +++ b/lib/edoc/src/edoc_layout.erl @@ -1193,6 +1193,8 @@ ot_name(Es, T) -> {atom,0,list_to_atom(Atom)},T]}; "tuple" when T =:= [] -> {type,0,tuple,any}; + "map" when T =:= [] -> + {type,0,map,any}; Atom -> {type,0,list_to_atom(Atom),T} end. diff --git a/lib/edoc/vsn.mk b/lib/edoc/vsn.mk index 2f6d469536..bea9277e1f 100644 --- a/lib/edoc/vsn.mk +++ b/lib/edoc/vsn.mk @@ -1 +1 @@ -EDOC_VSN = 0.9.2 +EDOC_VSN = 0.9.3 diff --git a/lib/eldap/doc/src/Makefile b/lib/eldap/doc/src/Makefile index aff1da4a9a..bf1eca267a 100644 --- a/lib/eldap/doc/src/Makefile +++ b/lib/eldap/doc/src/Makefile @@ -89,6 +89,7 @@ debug opt valgrind: clean clean_docs clean_tex: rm -rf $(HTMLDIR)/* + rm -rf $(XMLDIR) rm -f $(MAN3DIR)/* rm -f $(MAN6DIR)/* rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) diff --git a/lib/eldap/doc/src/notes.xml b/lib/eldap/doc/src/notes.xml index 8b066671ee..ef7dda510c 100644 --- a/lib/eldap/doc/src/notes.xml +++ b/lib/eldap/doc/src/notes.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2012</year><year>2016</year> + <year>2012</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -31,6 +31,21 @@ </header> <p>This document describes the changes made to the Eldap application.</p> +<section><title>Eldap 1.2.4</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Update to use the new string api instead of the old.</p> + <p> + Own Id: OTP-15036</p> + </item> + </list> + </section> + +</section> + <section><title>Eldap 1.2.3</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/eldap/test/eldap_basic_SUITE.erl b/lib/eldap/test/eldap_basic_SUITE.erl index 4bfb0dd291..b871eda19c 100644 --- a/lib/eldap/test/eldap_basic_SUITE.erl +++ b/lib/eldap/test/eldap_basic_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2012-2016. All Rights Reserved. +%% Copyright Ericsson AB 2012-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/eldap/vsn.mk b/lib/eldap/vsn.mk index 1636b6bb6d..f2b3cd47f2 100644 --- a/lib/eldap/vsn.mk +++ b/lib/eldap/vsn.mk @@ -1 +1 @@ -ELDAP_VSN = 1.2.3 +ELDAP_VSN = 1.2.4 diff --git a/lib/erl_docgen/doc/src/Makefile b/lib/erl_docgen/doc/src/Makefile index a9110e4635..d6d2550425 100644 --- a/lib/erl_docgen/doc/src/Makefile +++ b/lib/erl_docgen/doc/src/Makefile @@ -106,6 +106,7 @@ html: gifs examples $(HTML_REF_MAN_FILE) clean clean_docs: rm -rf $(HTMLDIR)/* + rm -rf $(XMLDIR) rm -f $(MAN6DIR)/* rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) rm -f errs core *~ diff --git a/lib/erl_docgen/doc/src/notes.xml b/lib/erl_docgen/doc/src/notes.xml index f75d2af5c4..7a09b62351 100644 --- a/lib/erl_docgen/doc/src/notes.xml +++ b/lib/erl_docgen/doc/src/notes.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2004</year><year>2017</year> + <year>2004</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -31,7 +31,28 @@ </header> <p>This document describes the changes made to the <em>erl_docgen</em> application.</p> - <section><title>Erl_Docgen 0.7.3</title> + <section><title>Erl_Docgen 0.8</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Add a hoverable element to the titles in the html + documentation with a link to github where the + documentation can be edited.</p> + <p> + Make the anchors in the html User's Guide and system + documentation use the title of the sections instead of a + generated id.</p> + <p> + Own Id: OTP-14979</p> + </item> + </list> + </section> + +</section> + +<section><title>Erl_Docgen 0.7.3</title> <section><title>Fixed Bugs and Malfunctions</title> <list> diff --git a/lib/erl_docgen/priv/bin/codeline_preprocessing.escript b/lib/erl_docgen/priv/bin/codeline_preprocessing.escript index 67966b79e6..ce9303f86e 100755 --- a/lib/erl_docgen/priv/bin/codeline_preprocessing.escript +++ b/lib/erl_docgen/priv/bin/codeline_preprocessing.escript @@ -2,7 +2,7 @@ %% -*- erlang -*- %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009-2016. All Rights Reserved. +%% Copyright Ericsson AB 2009-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/erl_docgen/priv/bin/github_link.escript b/lib/erl_docgen/priv/bin/github_link.escript index 1b36fca202..e0f5a2f471 100755 --- a/lib/erl_docgen/priv/bin/github_link.escript +++ b/lib/erl_docgen/priv/bin/github_link.escript @@ -2,7 +2,7 @@ %% -*- erlang -*- %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2016. All Rights Reserved. +%% Copyright Ericsson AB 2010-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/erl_docgen/priv/bin/xml_from_edoc.escript b/lib/erl_docgen/priv/bin/xml_from_edoc.escript index b0e3764fae..38e4f2d923 100755 --- a/lib/erl_docgen/priv/bin/xml_from_edoc.escript +++ b/lib/erl_docgen/priv/bin/xml_from_edoc.escript @@ -2,7 +2,7 @@ %% -*- erlang -*- %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2016. All Rights Reserved. +%% Copyright Ericsson AB 2010-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/erl_docgen/priv/xsl/db_man.xsl b/lib/erl_docgen/priv/xsl/db_man.xsl index a5ad7ed5ae..27b2bd4066 100644 --- a/lib/erl_docgen/priv/xsl/db_man.xsl +++ b/lib/erl_docgen/priv/xsl/db_man.xsl @@ -3,7 +3,7 @@ # # %CopyrightBegin% # - # Copyright Ericsson AB 2009-2016. All Rights Reserved. + # Copyright Ericsson AB 2009-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. diff --git a/lib/erl_docgen/src/docgen_edoc_xml_cb.erl b/lib/erl_docgen/src/docgen_edoc_xml_cb.erl index b065c18cda..d562cfddcc 100644 --- a/lib/erl_docgen/src/docgen_edoc_xml_cb.erl +++ b/lib/erl_docgen/src/docgen_edoc_xml_cb.erl @@ -182,7 +182,7 @@ chapter_title(#xmlElement{content=Es}) -> % name = h3 | h4 %% otp_xmlify(Es1) -> Es2 %% Es1 = Es2 = [#xmlElement{} | #xmlText{}] %% Fix things that are allowed in XHTML but not in chapter/erlref DTDs. -%% 1) lists (<ul>, <ol>, <dl>) and code snippets (<pre>) can not occur +%% 1) lists (<ul>, <ol>, <dl>) and code snippets (<pre>) cannot occur %% within a <p>, such a <p> must be splitted into a sequence of <p>, %% <ul>, <ol>, <dl> and <pre>. %% 2) <a> must only have either a href attribute (corresponds to a diff --git a/lib/erl_docgen/src/docgen_otp_specs.erl b/lib/erl_docgen/src/docgen_otp_specs.erl index 9f2b401f93..311ec1471a 100644 --- a/lib/erl_docgen/src/docgen_otp_specs.erl +++ b/lib/erl_docgen/src/docgen_otp_specs.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2017. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -658,6 +658,8 @@ ot_name(Es, T) -> {atom,0,list_to_atom(Atom)},T]}; "tuple" when T =:= [] -> {type,0,tuple,any}; + "map" when T =:= [] -> + {type,0,map,any}; Atom -> {type,0,list_to_atom(Atom),T} end. diff --git a/lib/erl_docgen/vsn.mk b/lib/erl_docgen/vsn.mk index a556b73103..0b24604daa 100644 --- a/lib/erl_docgen/vsn.mk +++ b/lib/erl_docgen/vsn.mk @@ -1 +1 @@ -ERL_DOCGEN_VSN = 0.7.3 +ERL_DOCGEN_VSN = 0.8 diff --git a/lib/erl_interface/configure.in b/lib/erl_interface/configure.in index 696ebf5ca0..a155ceef7e 100644 --- a/lib/erl_interface/configure.in +++ b/lib/erl_interface/configure.in @@ -1,7 +1,7 @@ # -*- Autoconf -*- # %CopyrightBegin% # -# Copyright Ericsson AB 2000-2016. All Rights Reserved. +# Copyright Ericsson AB 2000-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. diff --git a/lib/erl_interface/doc/src/Makefile b/lib/erl_interface/doc/src/Makefile index 8ef7e9648c..173bd2e83b 100644 --- a/lib/erl_interface/doc/src/Makefile +++ b/lib/erl_interface/doc/src/Makefile @@ -100,6 +100,7 @@ debug opt: clean clean_docs clean_tex: rm -rf $(HTMLDIR)/* + rm -rf $(XMLDIR) rm -f $(MAN1DIR)/* rm -f $(MAN3DIR)/* rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) diff --git a/lib/erl_interface/doc/src/notes.xml b/lib/erl_interface/doc/src/notes.xml index f165dde259..1438317d8d 100644 --- a/lib/erl_interface/doc/src/notes.xml +++ b/lib/erl_interface/doc/src/notes.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2004</year><year>2017</year> + <year>2004</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -31,6 +31,62 @@ </header> <p>This document describes the changes made to the Erl_interface application.</p> +<section><title>Erl_Interface 3.10.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fix bug where calling erl_init on certain platforms could + result in a buffer overflow bug.</p> + <p> + Own Id: OTP-15033</p> + </item> + <item> + <p> + Fixed <c>erl_call -m</c> to not deallocate module source + binary before it has been read.</p> + <p> + Own Id: OTP-15105 Aux Id: ERL-629 </p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> The program <c>erl_call</c> calls + <c>erl_eval:eval_str/1</c> when it used to call + <c>lib:eval_str/1</c>. This means that <c>erl_call</c> + will fail when trying interact with an Erlang node + running Erlang/OTP 20 or earlier. </p> + <p> + Own Id: OTP-15114 Aux Id: OTP-15072, ERL-634 </p> + </item> + </list> + </section> + +</section> + +<section><title>Erl_Interface 3.10.2.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Make <c>ei_connect</c> and friends also accept state + <c>ok_simultaneous</c> during handshake, which means the + other node has initiated a connection setup that will be + cancelled in favor of this connection.</p> + <p> + Own Id: OTP-15161 Aux Id: ERIERL-191 </p> + </item> + </list> + </section> + +</section> + <section><title>Erl_Interface 3.10.2</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/erl_interface/src/INSTALL b/lib/erl_interface/src/INSTALL index b42a17ac46..bf3ca8b6a5 100644 --- a/lib/erl_interface/src/INSTALL +++ b/lib/erl_interface/src/INSTALL @@ -122,10 +122,10 @@ you can use the `configure' options `--x-includes=DIR' and Specifying the System Type ========================== - There may be some features `configure' can not figure out + There may be some features `configure' cannot figure out automatically, but needs to determine by the type of host the package will run on. Usually `configure' can figure that out, but if it prints -a message saying it can not guess the host type, give it the +a message saying it cannot guess the host type, give it the `--host=TYPE' option. TYPE can either be a short name for the system type, such as `sun4', or a canonical name with three fields: CPU-COMPANY-SYSTEM diff --git a/lib/erl_interface/src/Makefile b/lib/erl_interface/src/Makefile index 135522397b..800557fbfe 100644 --- a/lib/erl_interface/src/Makefile +++ b/lib/erl_interface/src/Makefile @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 1996-2016. All Rights Reserved. +# Copyright Ericsson AB 1996-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. diff --git a/lib/erl_interface/src/Makefile.in b/lib/erl_interface/src/Makefile.in index 69b5b6003d..614e7325a9 100644 --- a/lib/erl_interface/src/Makefile.in +++ b/lib/erl_interface/src/Makefile.in @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 1997-2016. All Rights Reserved. +# Copyright Ericsson AB 1997-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. diff --git a/lib/erl_interface/src/connect/ei_connect.c b/lib/erl_interface/src/connect/ei_connect.c index 5c01223e3d..9df4fa3b6c 100644 --- a/lib/erl_interface/src/connect/ei_connect.c +++ b/lib/erl_interface/src/connect/ei_connect.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2000-2017. All Rights Reserved. + * Copyright Ericsson AB 2000-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -138,6 +138,11 @@ static int recv_name(int fd, unsigned *version, unsigned *flags, ErlConnect *namebuf, unsigned ms); +static struct hostent* +dyn_gethostbyname_r(const char *name, struct hostent *hostp, char **buffer_p, + int buflen, int *h_errnop); + + /*************************************************************************** * @@ -480,10 +485,14 @@ int ei_connect_xinit(ei_cnode* ec, const char *thishostname, int ei_connect_init(ei_cnode* ec, const char* this_node_name, const char *cookie, short creation) { - struct hostent *hp; char thishostname[EI_MAXHOSTNAMELEN+1]; char thisnodename[MAXNODELEN+1]; char thisalivename[EI_MAXALIVELEN+1]; + struct hostent host, *hp; + char buffer[1024]; + char *buf = buffer; + int ei_h_errno; + int res; #ifdef __WIN32__ if (!initWinSock()) { @@ -517,10 +526,13 @@ int ei_connect_init(ei_cnode* ec, const char* this_node_name, strcpy(thisalivename, this_node_name); } - if ((hp = ei_gethostbyname(thishostname)) == 0) { + hp = dyn_gethostbyname_r(thishostname,&host,&buf,sizeof(buffer),&ei_h_errno); + if (hp == NULL) { /* Looking up IP given hostname fails. We must be on a standalone host so let's use loopback for communication instead. */ - if ((hp = ei_gethostbyname("localhost")) == 0) { + hp = dyn_gethostbyname_r("localhost", &host, &buf, sizeof(buffer), + &ei_h_errno); + if (hp == NULL) { #ifdef __WIN32__ char reason[1024]; @@ -549,8 +561,11 @@ int ei_connect_init(ei_cnode* ec, const char* this_node_name, sprintf(thisnodename, "%s@%s", this_node_name, hp->h_name); } } - return ei_connect_xinit(ec, thishostname, thisalivename, thisnodename, - (struct in_addr *)*hp->h_addr_list, cookie, creation); + res = ei_connect_xinit(ec, thishostname, thisalivename, thisnodename, + (struct in_addr *)*hp->h_addr_list, cookie, creation); + if (buf != buffer) + free(buf); + return res; } @@ -595,6 +610,13 @@ struct hostent *dyn_gethostbyname_r(const char *name, int buflen, int *h_errnop) { +#ifdef __WIN32__ + /* + * Apparently ei_gethostbyname_r not implemented for Windows (?) + * Fall back on ei_gethostbyname like before. + */ + return ei_gethostbyname(name); +#else char* buf = *buffer_p; struct hostent *hp; @@ -629,6 +651,7 @@ struct hostent *dyn_gethostbyname_r(const char *name, } } return hp; +#endif } /* @@ -999,7 +1022,7 @@ int ei_do_receive_msg(int fd, int staticbuffer_p, erl_errno = EMSGSIZE; return ERL_ERROR; } - x->index = x->buffsz; + x->index = msglen; switch (msg->msgtype) { /* FIXME does not handle trace tokens and monitors */ case ERL_SEND: case ERL_REG_SEND: @@ -1357,11 +1380,14 @@ static int recv_status(int fd, unsigned ms) "<- RECV_STATUS socket read failed (%d)", rlen); goto error; } - if (rlen == 3 && buf[0] == 's' && buf[1] == 'o' && - buf[2] == 'k') { + + EI_TRACE_CONN2("recv_status", + "<- RECV_STATUS (%.*s)", (rlen>20 ? 20 : rlen), buf); + + if (rlen >= 3 && buf[0] == 's' && buf[1] == 'o' && buf[2] == 'k') { + /* Expecting "sok" or "sok_simultaneous" */ if (!is_static) free(buf); - EI_TRACE_CONN0("recv_status","<- RECV_STATUS (ok)"); return 0; } error: diff --git a/lib/erl_interface/src/connect/ei_resolve.c b/lib/erl_interface/src/connect/ei_resolve.c index 2757735d39..022a43d255 100644 --- a/lib/erl_interface/src/connect/ei_resolve.c +++ b/lib/erl_interface/src/connect/ei_resolve.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1997-2016. All Rights Reserved. + * Copyright Ericsson AB 1997-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/lib/erl_interface/src/decode/decode_atom.c b/lib/erl_interface/src/decode/decode_atom.c index 87cd75b1be..2bf8ab0552 100644 --- a/lib/erl_interface/src/decode/decode_atom.c +++ b/lib/erl_interface/src/decode/decode_atom.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1998-2016. All Rights Reserved. + * Copyright Ericsson AB 1998-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/lib/erl_interface/src/legacy/erl_marshal.c b/lib/erl_interface/src/legacy/erl_marshal.c index 6435754823..c18067b9bc 100644 --- a/lib/erl_interface/src/legacy/erl_marshal.c +++ b/lib/erl_interface/src/legacy/erl_marshal.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1996-2017. All Rights Reserved. + * Copyright Ericsson AB 1996-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/lib/erl_interface/src/misc/ei_pthreads.c b/lib/erl_interface/src/misc/ei_pthreads.c index 25608edeec..8c7e86555d 100644 --- a/lib/erl_interface/src/misc/ei_pthreads.c +++ b/lib/erl_interface/src/misc/ei_pthreads.c @@ -78,7 +78,7 @@ static void tls_init_once(void) errno_tls_index = TlsAlloc(); if (errno_tls_index == TLS_OUT_OF_INDEXES) { fprintf(stderr, - "FATAL ERROR: can not allocate TLS index for " + "FATAL ERROR: cannot allocate TLS index for " "erl_errno (error code = %d)!\n",GetLastError()); exit(1); } @@ -206,6 +206,7 @@ volatile int *__erl_errno_place(void) use_fallback = 1; return &fallback_errno; } + *erl_errno_p = 0; if (pthread_setspecific(erl_errno_key, erl_errno_p) != 0 || (erl_errno_p = pthread_getspecific(erl_errno_key)) == NULL) { diff --git a/lib/erl_interface/test/ei_accept_SUITE.erl b/lib/erl_interface/test/ei_accept_SUITE.erl index 6676e73e21..78a433d21b 100644 --- a/lib/erl_interface/test/ei_accept_SUITE.erl +++ b/lib/erl_interface/test/ei_accept_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2001-2016. All Rights Reserved. +%% Copyright Ericsson AB 2001-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -25,6 +25,7 @@ -include("ei_accept_SUITE_data/ei_accept_test_cases.hrl"). -export([all/0, suite/0, + init_per_testcase/2, ei_accept/1, ei_threaded_accept/1, monitor_ei_process/1]). @@ -38,8 +39,11 @@ all() -> [ei_accept, ei_threaded_accept, monitor_ei_process]. +init_per_testcase(Case, Config) -> + runner:init_per_testcase(?MODULE, Case, Config). + ei_accept(Config) when is_list(Config) -> - P = runner:start(?interpret), + P = runner:start(Config, ?interpret), 0 = ei_connect_init(P, 42, erlang:get_cookie(), 0), Myname = hd(tl(string:tokens(atom_to_list(node()), "@"))), @@ -91,7 +95,7 @@ ei_threaded_accept(Config) when is_list(Config) -> %% Test erlang:monitor toward erl_interface "processes" monitor_ei_process(Config) when is_list(Config) -> - P = runner:start(?interpret), + P = runner:start(Config, ?interpret), 0 = ei_connect_init(P, 42, erlang:get_cookie(), 0), Myname = hd(tl(string:tokens(atom_to_list(node()), "@"))), @@ -99,7 +103,6 @@ monitor_ei_process(Config) when is_list(Config) -> EINode = list_to_atom("c42@"++Myname), io:format("EINode ~p ~n", [EINode]), - Self = self(), Port = 6543, {ok, ListenFd} = ei_publish(P, Port), MRef1 = erlang:monitor(process, {any, EINode}), diff --git a/lib/erl_interface/test/ei_accept_SUITE_data/ei_accept_test.c b/lib/erl_interface/test/ei_accept_SUITE_data/ei_accept_test.c index 2b62d7c3a4..50df848b69 100644 --- a/lib/erl_interface/test/ei_accept_SUITE_data/ei_accept_test.c +++ b/lib/erl_interface/test/ei_accept_SUITE_data/ei_accept_test.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2001-2016. All Rights Reserved. + * Copyright Ericsson AB 2001-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -75,11 +75,7 @@ TESTCASE(interpret) ei_term term; ei_x_new(&x); - for (;;) { - if (get_bin_term(&x, &term)) { - report(1); - return; - } else { + while (get_bin_term(&x, &term) == 0) { char* buf = x.buff, func[MAXATOMLEN]; int index = x.index, arity; if (term.ei_type != ERL_SMALL_TUPLE_EXT || term.arity != 2) @@ -100,8 +96,9 @@ TESTCASE(interpret) message("\"%d\" \n", func); fail("bad command"); } - } - } + } + report(1); + ei_x_free(&x); } static void cmd_ei_connect_init(char* buf, int len) diff --git a/lib/erl_interface/test/ei_connect_SUITE.erl b/lib/erl_interface/test/ei_connect_SUITE.erl index 66498deadc..24cd384295 100644 --- a/lib/erl_interface/test/ei_connect_SUITE.erl +++ b/lib/erl_interface/test/ei_connect_SUITE.erl @@ -25,6 +25,7 @@ -include("ei_connect_SUITE_data/ei_connect_test_cases.hrl"). -export([all/0, suite/0, + init_per_testcase/2, ei_send/1, ei_reg_send/1, ei_format_pid/1, @@ -44,8 +45,11 @@ all() -> [ei_send, ei_reg_send, ei_rpc, ei_format_pid, ei_send_funs, ei_threaded_send, ei_set_get_tracelevel]. +init_per_testcase(Case, Config) -> + runner:init_per_testcase(?MODULE, Case, Config). + ei_send(Config) when is_list(Config) -> - P = runner:start(?interpret), + P = runner:start(Config, ?interpret), 0 = ei_connect_init(P, 42, erlang:get_cookie(), 0), {ok,Fd} = ei_connect(P, node()), @@ -58,7 +62,7 @@ ei_send(Config) when is_list(Config) -> ei_format_pid(Config) when is_list(Config) -> S = self(), - P = runner:start(?interpret), + P = runner:start(Config, ?interpret), 0 = ei_connect_init(P, 42, erlang:get_cookie(), 0), {ok,Fd} = ei_connect(P, node()), @@ -70,7 +74,7 @@ ei_format_pid(Config) when is_list(Config) -> ok. ei_send_funs(Config) when is_list(Config) -> - P = runner:start(?interpret), + P = runner:start(Config, ?interpret), 0 = ei_connect_init(P, 42, erlang:get_cookie(), 0), {ok,Fd} = ei_connect(P, node()), @@ -88,7 +92,7 @@ ei_send_funs(Config) when is_list(Config) -> ok. ei_reg_send(Config) when is_list(Config) -> - P = runner:start(?interpret), + P = runner:start(Config, ?interpret), 0 = ei_connect_init(P, 42, erlang:get_cookie(), 0), {ok,Fd} = ei_connect(P, node()), @@ -137,7 +141,7 @@ start_einode(Einode, N, Host) -> ok. ei_rpc(Config) when is_list(Config) -> - P = runner:start(?interpret), + P = runner:start(Config, ?interpret), 0 = ei_connect_init(P, 42, erlang:get_cookie(), 0), {ok,Fd} = ei_connect(P, node()), @@ -150,7 +154,7 @@ ei_rpc(Config) when is_list(Config) -> ok. ei_set_get_tracelevel(Config) when is_list(Config) -> - P = runner:start(?interpret), + P = runner:start(Config, ?interpret), 5 = ei_set_get_tracelevel(P, 5), 0 = ei_connect_init(P, 42, erlang:get_cookie(), 0), {ok,Fd} = ei_connect(P, node()), diff --git a/lib/erl_interface/test/ei_connect_SUITE_data/ei_connect_test.c b/lib/erl_interface/test/ei_connect_SUITE_data/ei_connect_test.c index 6a3796dd24..fbd86cdb50 100644 --- a/lib/erl_interface/test/ei_connect_SUITE_data/ei_connect_test.c +++ b/lib/erl_interface/test/ei_connect_SUITE_data/ei_connect_test.c @@ -74,11 +74,7 @@ TESTCASE(interpret) ei_term term; ei_x_new(&x); - for (;;) { - if (get_bin_term(&x, &term)) { - report(1); - return; - } else { + while (get_bin_term(&x, &term) == 0) { char* buf = x.buff, func[MAXATOMLEN]; int index = x.index, arity; if (term.ei_type != ERL_SMALL_TUPLE_EXT || term.arity != 2) @@ -99,8 +95,10 @@ TESTCASE(interpret) message("\"%d\" \n", func); fail("bad command"); } - } - } + } + report(1); + ei_x_free(&x); + return; } diff --git a/lib/erl_interface/test/ei_decode_SUITE.erl b/lib/erl_interface/test/ei_decode_SUITE.erl index 74fb9b8916..499f10611e 100644 --- a/lib/erl_interface/test/ei_decode_SUITE.erl +++ b/lib/erl_interface/test/ei_decode_SUITE.erl @@ -25,6 +25,7 @@ -include("ei_decode_SUITE_data/ei_decode_test_cases.hrl"). -export([all/0, suite/0, + init_per_testcase/2, test_ei_decode_long/1, test_ei_decode_ulong/1, test_ei_decode_longlong/1, @@ -42,6 +43,9 @@ all() -> test_ei_decode_char, test_ei_decode_nonoptimal, test_ei_decode_misc, test_ei_decode_utf8_atom]. +init_per_testcase(Case, Config) -> + runner:init_per_testcase(?MODULE, Case, Config). + %% --------------------------------------------------------------------------- % NOTE: for historical reasons we don't pach as tight as we can, @@ -51,7 +55,7 @@ all() -> %% ######################################################################## %% test_ei_decode_long(Config) when is_list(Config) -> - P = runner:start(?test_ei_decode_long), + P = runner:start(Config, ?test_ei_decode_long), send_integers(P), runner:recv_eot(P), ok. @@ -60,7 +64,7 @@ test_ei_decode_long(Config) when is_list(Config) -> %% ######################################################################## %% test_ei_decode_ulong(Config) when is_list(Config) -> - P = runner:start(?test_ei_decode_ulong), + P = runner:start(Config, ?test_ei_decode_ulong), send_integers(P), runner:recv_eot(P), ok. @@ -77,7 +81,7 @@ test_ei_decode_longlong(Config) when is_list(Config) -> vxworks -> {skip,"Skipped on VxWorks"}; _ -> - P = runner:start(?test_ei_decode_longlong), + P = runner:start(Config, ?test_ei_decode_longlong), send_integers2(P), runner:recv_eot(P), ok @@ -91,7 +95,7 @@ test_ei_decode_ulonglong(Config) when is_list(Config) -> vxworks -> {skip,"Skipped on VxWorks"}; _ -> - P = runner:start(?test_ei_decode_ulonglong), + P = runner:start(Config, ?test_ei_decode_ulonglong), send_integers2(P), runner:recv_eot(P), ok @@ -104,7 +108,7 @@ test_ei_decode_ulonglong(Config) when is_list(Config) -> %% FIXME maybe the API should change to use "unsigned char" to be clear?! test_ei_decode_char(Config) when is_list(Config) -> - P = runner:start(?test_ei_decode_char), + P = runner:start(Config, ?test_ei_decode_char), send_term_as_binary(P,0), send_term_as_binary(P,16#7f), @@ -119,7 +123,7 @@ test_ei_decode_char(Config) when is_list(Config) -> %% ######################################################################## %% test_ei_decode_nonoptimal(Config) when is_list(Config) -> - P = runner:start(?test_ei_decode_nonoptimal), + P = runner:start(Config, ?test_ei_decode_nonoptimal), send_non_optimal_pos(P), % decode_char send_non_optimal(P), % decode_long @@ -168,7 +172,7 @@ send_non_optimal_neg(P) -> %% ######################################################################## %% test_ei_decode_misc(Config) when is_list(Config) -> - P = runner:start(?test_ei_decode_misc), + P = runner:start(Config, ?test_ei_decode_misc), send_term_as_binary(P,0.0), send_term_as_binary(P,-1.0), @@ -199,7 +203,7 @@ test_ei_decode_misc(Config) when is_list(Config) -> %% ######################################################################## %% test_ei_decode_utf8_atom(Config) -> - P = runner:start(?test_ei_decode_utf8_atom), + P = runner:start(Config, ?test_ei_decode_utf8_atom), send_latin1_atom_as_binary(P,"å"), send_latin1_atom_as_binary(P,"ä"), diff --git a/lib/erl_interface/test/ei_decode_SUITE_data/ei_decode_test.c b/lib/erl_interface/test/ei_decode_SUITE_data/ei_decode_test.c index b7a2c4bb8b..5a9be1e9a2 100644 --- a/lib/erl_interface/test/ei_decode_SUITE_data/ei_decode_test.c +++ b/lib/erl_interface/test/ei_decode_SUITE_data/ei_decode_test.c @@ -105,6 +105,7 @@ int ei_decode_my_string(const char *buf, int *index, char *to, fail1("size of encoded data (%d) is incorrect", size1); \ return; \ } \ + free_packet(buf); \ } \ #define EI_DECODE_2_FAIL(FUNC,SIZE,TYPE,VAL) \ @@ -148,6 +149,7 @@ int ei_decode_my_string(const char *buf, int *index, char *to, fail("size of encoded data should be 0"); \ return; \ } \ + free_packet(buf); \ } \ #define dump(arr, num) { \ @@ -205,6 +207,7 @@ int ei_decode_my_string(const char *buf, int *index, char *to, fail("size of encoded data is incorrect"); \ return; \ } \ + free_packet(buf); \ } \ #define EI_DECODE_STRING(FUNC,SIZE,VAL) \ @@ -248,6 +251,7 @@ int ei_decode_my_string(const char *buf, int *index, char *to, fail("size of encoded data should be 0"); \ return; \ } \ + free_packet(buf); \ } \ //#define EI_DECODE_UTF8_STRING(FUNC,SIZE,VAL) @@ -310,6 +314,7 @@ int ei_decode_my_string(const char *buf, int *index, char *to, fail("size of encoded data is incorrect"); \ return; \ } \ + free_packet(buf); \ } \ /* ******************************************************************** */ diff --git a/lib/erl_interface/test/ei_decode_encode_SUITE.erl b/lib/erl_interface/test/ei_decode_encode_SUITE.erl index 160720b413..6476e92be9 100644 --- a/lib/erl_interface/test/ei_decode_encode_SUITE.erl +++ b/lib/erl_interface/test/ei_decode_encode_SUITE.erl @@ -25,6 +25,7 @@ -include("ei_decode_encode_SUITE_data/ei_decode_encode_test_cases.hrl"). -export([all/0, suite/0, + init_per_testcase/2, test_ei_decode_encode/1]). suite() -> @@ -33,6 +34,9 @@ suite() -> all() -> [test_ei_decode_encode]. +init_per_testcase(Case, Config) -> + runner:init_per_testcase(?MODULE, Case, Config). + %% --------------------------------------------------------------------------- % NOTE: these types have no meaning on the C side so we pass them @@ -42,7 +46,7 @@ all() -> %% ######################################################################## %% test_ei_decode_encode(Config) when is_list(Config) -> - P = runner:start(?test_ei_decode_encode), + P = runner:start(Config, ?test_ei_decode_encode), Fun = fun (X) -> {X,true} end, Pid = self(), diff --git a/lib/erl_interface/test/ei_decode_encode_SUITE_data/ei_decode_encode_test.c b/lib/erl_interface/test/ei_decode_encode_SUITE_data/ei_decode_encode_test.c index 467f789fdb..6285b5e199 100644 --- a/lib/erl_interface/test/ei_decode_encode_SUITE_data/ei_decode_encode_test.c +++ b/lib/erl_interface/test/ei_decode_encode_SUITE_data/ei_decode_encode_test.c @@ -466,6 +466,7 @@ void decode_encode_big(struct Type* t) send_buffer(arg.buff, arg.index); ei_x_free(&arg); ei_free_big(p); + free_packet(buf); } diff --git a/lib/erl_interface/test/ei_encode_SUITE.erl b/lib/erl_interface/test/ei_encode_SUITE.erl index 8857b092f3..8c3b0e193b 100644 --- a/lib/erl_interface/test/ei_encode_SUITE.erl +++ b/lib/erl_interface/test/ei_encode_SUITE.erl @@ -25,6 +25,7 @@ -include("ei_encode_SUITE_data/ei_encode_test_cases.hrl"). -export([all/0, suite/0, + init_per_testcase/2, test_ei_encode_long/1, test_ei_encode_ulong/1, test_ei_encode_longlong/1, @@ -45,6 +46,9 @@ all() -> test_ei_encode_fails, test_ei_encode_utf8_atom, test_ei_encode_utf8_atom_len]. +init_per_testcase(Case, Config) -> + runner:init_per_testcase(?MODULE, Case, Config). + %% --------------------------------------------------------------------------- @@ -55,7 +59,7 @@ all() -> %% ######################################################################## %% test_ei_encode_long(Config) when is_list(Config) -> - P = runner:start(?test_ei_encode_long), + P = runner:start(Config, ?test_ei_encode_long), {<<97,0>> ,0} = get_buf_and_term(P), {<<97,255>> ,255} = get_buf_and_term(P), @@ -77,7 +81,7 @@ test_ei_encode_long(Config) when is_list(Config) -> %% ######################################################################## %% test_ei_encode_ulong(Config) when is_list(Config) -> - P = runner:start(?test_ei_encode_ulong), + P = runner:start(Config, ?test_ei_encode_ulong), {<<97,0>> ,0} = get_buf_and_term(P), {<<97,255>> ,255} = get_buf_and_term(P), @@ -101,7 +105,7 @@ test_ei_encode_longlong(Config) when is_list(Config) -> vxworks -> {skip,"Skipped on VxWorks"}; _ -> - P = runner:start(?test_ei_encode_longlong), + P = runner:start(Config, ?test_ei_encode_longlong), {<<97,0>> ,0} = get_buf_and_term(P), {<<97,255>> ,255} = get_buf_and_term(P), @@ -132,7 +136,7 @@ test_ei_encode_ulonglong(Config) when is_list(Config) -> vxworks -> {skip,"Skipped on VxWorks"}; _ -> - P = runner:start(?test_ei_encode_ulonglong), + P = runner:start(Config, ?test_ei_encode_ulonglong), {<<97,0>> ,0} = get_buf_and_term(P), {<<97,255>> ,255} = get_buf_and_term(P), @@ -158,7 +162,7 @@ test_ei_encode_ulonglong(Config) when is_list(Config) -> %% FIXME maybe the API should change to use "unsigned char" to be clear?! test_ei_encode_char(Config) when is_list(Config) -> - P = runner:start(?test_ei_encode_char), + P = runner:start(Config, ?test_ei_encode_char), {<<97, 0>>,0} = get_buf_and_term(P), {<<97,127>>,16#7f} = get_buf_and_term(P), @@ -171,7 +175,7 @@ test_ei_encode_char(Config) when is_list(Config) -> %% ######################################################################## %% test_ei_encode_misc(Config) when is_list(Config) -> - P = runner:start(?test_ei_encode_misc), + P = runner:start(Config, ?test_ei_encode_misc), <<131>> = get_binaries(P), @@ -217,7 +221,7 @@ test_ei_encode_misc(Config) when is_list(Config) -> %% ######################################################################## %% test_ei_encode_fails(Config) when is_list(Config) -> - P = runner:start(?test_ei_encode_fails), + P = runner:start(Config, ?test_ei_encode_fails), XAtom = list_to_atom(lists:duplicate(255, $x)), YAtom = list_to_atom(lists:duplicate(255, $y)), @@ -236,7 +240,7 @@ test_ei_encode_fails(Config) when is_list(Config) -> %% ######################################################################## %% test_ei_encode_utf8_atom(Config) -> - P = runner:start(?test_ei_encode_utf8_atom), + P = runner:start(Config, ?test_ei_encode_utf8_atom), {<<119,2,195,133>>,'Å'} = get_buf_and_term(P), {<<119,2,195,133>>,'Å'} = get_buf_and_term(P), @@ -251,7 +255,7 @@ test_ei_encode_utf8_atom(Config) -> %% ######################################################################## %% test_ei_encode_utf8_atom_len(Config) -> - P = runner:start(?test_ei_encode_utf8_atom_len), + P = runner:start(Config, ?test_ei_encode_utf8_atom_len), {<<119,2,195,133>>,'Å'} = get_buf_and_term(P), {<<119,4,195,133,195,132>>,'ÅÄ'} = get_buf_and_term(P), diff --git a/lib/erl_interface/test/ei_format_SUITE.erl b/lib/erl_interface/test/ei_format_SUITE.erl index 07ee479b1f..b11a6ff164 100644 --- a/lib/erl_interface/test/ei_format_SUITE.erl +++ b/lib/erl_interface/test/ei_format_SUITE.erl @@ -26,6 +26,7 @@ -export([format_wo_ver/1, all/0, suite/0, + init_per_testcase/2, atoms/1, tuples/1, lists/1]). @@ -41,10 +42,13 @@ suite() -> all() -> [format_wo_ver, atoms, tuples, lists]. +init_per_testcase(Case, Config) -> + runner:init_per_testcase(?MODULE, Case, Config). + %% Tests formatting various atoms. atoms(Config) when is_list(Config) -> - P = runner:start(?atoms), + P = runner:start(Config, ?atoms), {term, ''} = get_term(P), {term, 'a'} = get_term(P), @@ -84,7 +88,7 @@ atoms(Config) when is_list(Config) -> %% Tests formatting various tuples tuples(Config) when is_list(Config) -> - P = runner:start(?tuples), + P = runner:start(Config, ?tuples), {term, {}} = get_term(P), {term, {a}} = get_term(P), @@ -105,7 +109,7 @@ tuples(Config) when is_list(Config) -> %% Tests formatting various lists lists(Config) when is_list(Config) -> - P = runner:start(?lists), + P = runner:start(Config, ?lists), {term, []} = get_term(P), {term, [a]} = get_term(P), @@ -146,7 +150,7 @@ lists(Config) when is_list(Config) -> format_wo_ver(Config) when is_list(Config) -> - P = runner:start(?format_wo_ver), + P = runner:start(Config, ?format_wo_ver), {term, [-1, 2, $c, {a, "b"}, {c, 10}]} = get_term(P), diff --git a/lib/erl_interface/test/ei_print_SUITE.erl b/lib/erl_interface/test/ei_print_SUITE.erl index 6d5c341eae..cad2686018 100644 --- a/lib/erl_interface/test/ei_print_SUITE.erl +++ b/lib/erl_interface/test/ei_print_SUITE.erl @@ -25,6 +25,7 @@ -include("ei_print_SUITE_data/ei_print_test_cases.hrl"). -export([all/0, suite/0, + init_per_testcase/2, atoms/1, tuples/1, lists/1, strings/1]). -import(runner, [get_term/1]). @@ -38,10 +39,13 @@ suite() -> all() -> [atoms, tuples, lists, strings]. +init_per_testcase(Case, Config) -> + runner:init_per_testcase(?MODULE, Case, Config). + %% Tests formatting various atoms. atoms(Config) when is_list(Config) -> - P = runner:start(?atoms), + P = runner:start(Config, ?atoms), {term, "''"} = get_term(P), {term, "a"} = get_term(P), @@ -79,7 +83,7 @@ atoms(Config) when is_list(Config) -> %% Tests formatting various tuples tuples(Config) when is_list(Config) -> - P = runner:start(?tuples), + P = runner:start(Config, ?tuples), {term, "{}"} = get_term(P), {term, "{a}"} = get_term(P), @@ -100,7 +104,7 @@ tuples(Config) when is_list(Config) -> %% Tests formatting various lists lists(Config) when is_list(Config) -> - P = runner:start(?lists), + P = runner:start(Config, ?lists), {term, "[]"} = get_term(P), {term, "[a]"} = get_term(P), @@ -125,7 +129,7 @@ lists(Config) when is_list(Config) -> ok. strings(Config) when is_list(Config) -> - P = runner:start(?strings), + P = runner:start(Config, ?strings), {term, "\"\\n\""} = get_term(P), {term, "\"\\r\\n\""} = get_term(P), diff --git a/lib/erl_interface/test/ei_tmo_SUITE.erl b/lib/erl_interface/test/ei_tmo_SUITE.erl index 003fe20594..1e76a99e1e 100644 --- a/lib/erl_interface/test/ei_tmo_SUITE.erl +++ b/lib/erl_interface/test/ei_tmo_SUITE.erl @@ -39,14 +39,16 @@ all() -> [framework_check, ei_accept_tmo, ei_connect_tmo, ei_send_tmo, ei_recv_tmo]. -init_per_testcase(_Case, Config) -> +init_per_testcase(Case, Config) -> + Config1 = runner:init_per_testcase(?MODULE, Case, Config), + % test if platform is vxworks_simso {_,Host} = split(node()), Bool = case atom_to_list(Host) of [$v,$x,$s,$i,$m | _] -> true; _ -> false end, - [{vxsim,Bool}|Config]. + [{vxsim,Bool} | Config1]. end_per_testcase(_Case, _Config) -> ok. @@ -55,7 +57,7 @@ end_per_testcase(_Case, _Config) -> framework_check(Config) when is_list(Config) -> %%dbg:tracer(), %%dbg:p(self()), - P = runner:start(?framework_check), + P = runner:start(Config, ?framework_check), runner:send_term(P,{hello,world}), {term, {hello,world}} = runner:get_term(P), runner:recv_eot(P), @@ -71,7 +73,7 @@ ei_recv_tmo(Config) when is_list(Config) -> do_one_recv(Config,CNode) -> {_,Host} = split(node()), - P1 = runner:start(?recv_tmo), + P1 = runner:start(Config, ?recv_tmo), runner:send_term(P1,{CNode, erlang:get_cookie(), node()}), @@ -84,7 +86,7 @@ do_one_recv(Config,CNode) -> runner:recv_eot(P1). do_one_recv_failure(Config,CNode) -> - P1 = runner:start(?recv_tmo), + P1 = runner:start(Config, ?recv_tmo), runner:send_term(P1,{CNode, erlang:get_cookie(), node()}), @@ -110,7 +112,7 @@ ei_send_tmo(Config) when is_list(Config) -> do_one_send(Config,From,CNode) -> {_,Host} = split(node()), - P1 = runner:start(?send_tmo), + P1 = runner:start(Config, ?send_tmo), runner:send_term(P1,{CNode, erlang:get_cookie(), node()}), @@ -139,7 +141,7 @@ do_one_send_failure(Config,From,FakeName,CName,VxSim) -> exit(Else) end, EpmdSocket = register(OurName, LSocket, 1, 5), - P3 = runner:start(?send_tmo), + P3 = runner:start(Config, ?send_tmo), Cookie = kaksmula_som_ingen_bryr_sig_om, runner:send_term(P3,{CName, Cookie, @@ -202,7 +204,7 @@ ei_connect_tmo(Config) when is_list(Config) -> %dbg:p(self()), VxSim = proplists:get_value(vxsim, Config), DummyNode = make_and_check_dummy(), - P = runner:start(?connect_tmo), + P = runner:start(Config, ?connect_tmo), runner:send_term(P,{c_nod_connect_tmo_1, kaksmula_som_ingen_bryr_sig_om, DummyNode}), @@ -219,7 +221,7 @@ ei_connect_tmo(Config) when is_list(Config) -> end end, runner:recv_eot(P), - P2 = runner:start(?connect_tmo), + P2 = runner:start(Config, ?connect_tmo), runner:send_term(P2,{c_nod_connect_tmo_2, erlang:get_cookie(), node()}), @@ -237,7 +239,7 @@ ei_connect_tmo(Config) when is_list(Config) -> exit(Else) end, EpmdSocket = register(OurName, LSocket, 1, 5), - P3 = runner:start(?connect_tmo), + P3 = runner:start(Config, ?connect_tmo), Cookie = kaksmula_som_ingen_bryr_sig_om, runner:send_term(P3,{c_nod_connect_tmo_3, Cookie, @@ -266,12 +268,12 @@ ei_connect_tmo(Config) when is_list(Config) -> ei_accept_tmo(Config) when is_list(Config) -> %%dbg:tracer(), %%dbg:p(self()), - P = runner:start(?accept_tmo), + P = runner:start(Config, ?accept_tmo), runner:send_term(P,{c_nod_som_ingen_kontaktar_1, kaksmula_som_ingen_bryr_sig_om}), {term,{-1,ETimedout,ETimedout}} = runner:get_term(P, 10000), runner:recv_eot(P), - P2 = runner:start(?accept_tmo), + P2 = runner:start(Config, ?accept_tmo), runner:send_term(P2,{c_nod_som_vi_kontaktar_1, erlang:get_cookie()}), receive after 1000 -> ok end, @@ -280,7 +282,7 @@ ei_accept_tmo(Config) when is_list(Config) -> {term, X} = runner:get_term(P2, 10000), runner:recv_eot(P2), true = is_integer(X), - P3 = runner:start(?accept_tmo), + P3 = runner:start(Config, ?accept_tmo), runner:send_term(P3,{c_nod_som_vi_kontaktar_2, erlang:get_cookie()}), receive after 1000 -> ok end, diff --git a/lib/erl_interface/test/ei_tmo_SUITE_data/ei_tmo_test.c b/lib/erl_interface/test/ei_tmo_SUITE_data/ei_tmo_test.c index 8e0f2807a4..39846e4a58 100644 --- a/lib/erl_interface/test/ei_tmo_SUITE_data/ei_tmo_test.c +++ b/lib/erl_interface/test/ei_tmo_SUITE_data/ei_tmo_test.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2003-2016. All Rights Reserved. + * Copyright Ericsson AB 2003-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/lib/erl_interface/test/erl_connect_SUITE.erl b/lib/erl_interface/test/erl_connect_SUITE.erl index cd73f07b8f..ad4e34c548 100644 --- a/lib/erl_interface/test/erl_connect_SUITE.erl +++ b/lib/erl_interface/test/erl_connect_SUITE.erl @@ -25,6 +25,7 @@ -include("erl_connect_SUITE_data/erl_connect_test_cases.hrl"). -export([all/0, suite/0, + init_per_testcase/2, erl_send/1, erl_reg_send/1, erl_send_cookie_file/1]). @@ -38,8 +39,11 @@ all() -> [erl_send, erl_reg_send, erl_send_cookie_file]. +init_per_testcase(Case, Config) -> + runner:init_per_testcase(?MODULE, Case, Config). + erl_send(Config) when is_list(Config) -> - P = runner:start(?interpret), + P = runner:start(Config, ?interpret), 1 = erl_connect_init(P, 42, erlang:get_cookie(), 0), {ok,Fd} = erl_connect(P, node()), @@ -56,7 +60,7 @@ erl_send_cookie_file(Config) when is_list(Config) -> vxworks -> {skip,"Skipped on VxWorks"}; _ -> - P = runner:start(?interpret), + P = runner:start(Config, ?interpret), 1 = erl_connect_init(P, 42, '', 0), {ok,Fd} = erl_connect(P, node()), @@ -70,7 +74,7 @@ erl_send_cookie_file(Config) when is_list(Config) -> end. erl_reg_send(Config) when is_list(Config) -> - P = runner:start(?interpret), + P = runner:start(Config, ?interpret), 1 = erl_connect_init(P, 42, erlang:get_cookie(), 0), {ok,Fd} = erl_connect(P, node()), diff --git a/lib/erl_interface/test/erl_eterm_SUITE.erl b/lib/erl_interface/test/erl_eterm_SUITE.erl index 3d1e33081b..84b58fed54 100644 --- a/lib/erl_interface/test/erl_eterm_SUITE.erl +++ b/lib/erl_interface/test/erl_eterm_SUITE.erl @@ -35,6 +35,7 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -export([all/0, suite/0, + init_per_testcase/2, build_terms/1, round_trip_conversion/1, decode_terms/1, decode_float/1, t_erl_mk_int/1, t_erl_mk_list/1, @@ -94,6 +95,9 @@ all() -> high_chaparal, broken_data, cnode_1]. +init_per_testcase(Case, Config) -> + runner:init_per_testcase(?MODULE, Case, Config). + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%% %%% 1. B a s i c t e s t s @@ -104,7 +108,7 @@ all() -> %% a list and verifies that the result is as expected. build_terms(Config) when is_list(Config) -> - P = runner:start(?build_terms), + P = runner:start(Config, ?build_terms), {term, Term} = get_term(P), io:format("Received: ~p", [Term]), [ARefLN, ARef, APortLN, APort, APidLN, APid, @@ -136,7 +140,7 @@ build_terms(Config) when is_list(Config) -> %% This test is run entirely in C code. round_trip_conversion(Config) when is_list(Config) -> - runner:test(?round_trip_conversion), + runner:test(Config, ?round_trip_conversion), ok. %% This test sends a list of all data types to the C code function, @@ -156,7 +160,7 @@ decode_terms(Config) when is_list(Config) -> {element1, 42, 767}, "A string", 1, -1, 0, 3.0, ABinary, 'I am an atom'], - P = runner:start(?decode_terms), + P = runner:start(Config, ?decode_terms), runner:send_term(P, Terms), runner:recv_eot(P), @@ -165,7 +169,7 @@ decode_terms(Config) when is_list(Config) -> %% Decodes the floating point number 3.1415. decode_float(Config) when is_list(Config) -> - P = runner:start(?decode_float), + P = runner:start(Config, ?decode_float), runner:send_term(P, 3.1415), runner:recv_eot(P), ok. @@ -173,7 +177,7 @@ decode_float(Config) when is_list(Config) -> %% Tests the erl_free_compound() function. t_erl_free_compound(Config) when is_list(Config) -> - runner:test(?t_erl_free_compound), + runner:test(Config, ?t_erl_free_compound), ok. @@ -186,7 +190,7 @@ t_erl_free_compound(Config) when is_list(Config) -> %% This tests the erl_mk_list() function. t_erl_mk_list(Config) when is_list(Config) -> - P = runner:start(?t_erl_mk_list), + P = runner:start(Config, ?t_erl_mk_list), {term, []} = get_term(P), {term, [abc]} = get_term(P), @@ -200,7 +204,7 @@ t_erl_mk_list(Config) when is_list(Config) -> %% This tests the erl_mk_int() function. t_erl_mk_int(Config) when is_list(Config) -> - P = runner:start(?t_erl_mk_int), + P = runner:start(Config, ?t_erl_mk_int), {term, 0} = get_term(P), {term, 127} = get_term(P), @@ -255,14 +259,14 @@ t_erl_mk_int(Config) when is_list(Config) -> %% Basic test of erl_copy_term(). basic_copy(Config) when is_list(Config) -> - runner:test(?basic_copy), + runner:test(Config, ?basic_copy), ok. %% This tests the erl_mk_tuple() function. t_erl_mk_tuple(Config) when is_list(Config) -> - P = runner:start(?t_erl_mk_tuple), + P = runner:start(Config, ?t_erl_mk_tuple), {term, {madonna, 21, 'mad donna', 12}} = get_term(P), {term, {'Madonna',21,{children,{"Isabella",2}}, @@ -275,7 +279,7 @@ t_erl_mk_tuple(Config) when is_list(Config) -> %% This tests the erl_mk_atom() function. t_erl_mk_atom(Config) when is_list(Config) -> - P = runner:start(?t_erl_mk_atom), + P = runner:start(Config, ?t_erl_mk_atom), {term, madonna} = (get_term(P)), {term, 'Madonna'} = (get_term(P)), @@ -295,7 +299,7 @@ t_erl_mk_atom(Config) when is_list(Config) -> %% This tests the erl_mk_binary() function. t_erl_mk_binary(Config) when is_list(Config) -> - P = runner:start(?t_erl_mk_binary), + P = runner:start(Config, ?t_erl_mk_binary), {term, Bin} = (get_term(P)), "{madonna,21,'mad donna',1234.567.890, !#$%&/()=?+-@, \" \\}" = binary_to_list(Bin), @@ -307,7 +311,7 @@ t_erl_mk_binary(Config) when is_list(Config) -> %% This tests the erl_mk_empty_list() function. t_erl_mk_empty_list(Config) when is_list(Config) -> - P = runner:start(?t_erl_mk_empty_list), + P = runner:start(Config, ?t_erl_mk_empty_list), {term, []} = get_term(P), @@ -322,7 +326,7 @@ t_erl_mk_float(Config) when is_list(Config) -> vxworks -> {skipped, "Floating point numbers never compare equal on PPC"}; _ -> - P = runner:start(?t_erl_mk_float), + P = runner:start(Config, ?t_erl_mk_float), {term, {3.1415, 1.999999, 2.000000, 2.000001, 2.000002, 12345.67890}} = get_term(P), runner:recv_eot(P), @@ -333,7 +337,7 @@ t_erl_mk_float(Config) when is_list(Config) -> %% This tests the erl_mk_pid() function. t_erl_mk_pid(Config) when is_list(Config) -> - P = runner:start(?t_erl_mk_pid), + P = runner:start(Config, ?t_erl_mk_pid), {term, A_pid} = (get_term(P)), {pid, kalle@localhost, 3, 2} = nc2vinfo(A_pid), @@ -342,7 +346,7 @@ t_erl_mk_pid(Config) when is_list(Config) -> ok. t_erl_mk_xpid(Config) when is_list(Config) -> - P = runner:start(?t_erl_mk_xpid), + P = runner:start(Config, ?t_erl_mk_xpid), {term, A_pid} = (get_term(P)), {pid, kalle@localhost, 32767, 8191} = nc2vinfo(A_pid), @@ -354,7 +358,7 @@ t_erl_mk_xpid(Config) when is_list(Config) -> %% This tests the erl_mk_port() function. t_erl_mk_port(Config) when is_list(Config) -> - P = runner:start(?t_erl_mk_port), + P = runner:start(Config, ?t_erl_mk_port), {term, A_port} = (get_term(P)), {port, kalle@localhost, 4} = nc2vinfo(A_port), @@ -363,7 +367,7 @@ t_erl_mk_port(Config) when is_list(Config) -> ok. t_erl_mk_xport(Config) when is_list(Config) -> - P = runner:start(?t_erl_mk_xport), + P = runner:start(Config, ?t_erl_mk_xport), {term, A_port} = (get_term(P)), {port, kalle@localhost, 268435455} = nc2vinfo(A_port), @@ -375,7 +379,7 @@ t_erl_mk_xport(Config) when is_list(Config) -> %% This tests the erl_mk_ref() function. t_erl_mk_ref(Config) when is_list(Config) -> - P = runner:start(?t_erl_mk_ref), + P = runner:start(Config, ?t_erl_mk_ref), {term, A_ref} = (get_term(P)), {ref, kalle@localhost, _Length, [6]} = nc2vinfo(A_ref), @@ -384,7 +388,7 @@ t_erl_mk_ref(Config) when is_list(Config) -> ok. t_erl_mk_long_ref(Config) when is_list(Config) -> - P = runner:start(?t_erl_mk_long_ref), + P = runner:start(Config, ?t_erl_mk_long_ref), {term, A_ref} = (get_term(P)), {ref, kalle@localhost, _Length, [4294967295,4294967295,262143]} @@ -397,7 +401,7 @@ t_erl_mk_long_ref(Config) when is_list(Config) -> %% This tests the erl_mk_string() function. t_erl_mk_string(Config) when is_list(Config) -> - P = runner:start(?t_erl_mk_string), + P = runner:start(Config, ?t_erl_mk_string), {term, "madonna"} = (get_term(P)), {term, "Madonna"} = (get_term(P)), @@ -417,7 +421,7 @@ t_erl_mk_string(Config) when is_list(Config) -> %% This tests the erl_mk_estring() function. t_erl_mk_estring(Config) when is_list(Config) -> - P = runner:start(?t_erl_mk_estring), + P = runner:start(Config, ?t_erl_mk_estring), {term, "madonna"} = (get_term(P)), {term, "Madonna"} = (get_term(P)), @@ -437,7 +441,7 @@ t_erl_mk_estring(Config) when is_list(Config) -> %% This tests the erl_mk_uint() function. t_erl_mk_uint(Config) when is_list(Config) -> - P = runner:start(?t_erl_mk_uint), + P = runner:start(Config, ?t_erl_mk_uint), {term, 54321} = (get_term(P)), {term, 2147483647} = (get_term(P)), @@ -453,7 +457,7 @@ t_erl_mk_uint(Config) when is_list(Config) -> %% This tests the erl_mk_var() function. t_erl_mk_var(Config) when is_list(Config) -> - P = runner:start(?t_erl_mk_var), + P = runner:start(Config, ?t_erl_mk_var), {term, 1} = (get_term(P)), {term, 0} = (get_term(P)), @@ -470,7 +474,7 @@ t_erl_mk_var(Config) when is_list(Config) -> %% This tests the erl_cons() function. t_erl_cons(Config) when is_list(Config) -> - P = runner:start(?t_erl_cons), + P = runner:start(Config, ?t_erl_cons), {term, [madonna, 21]} = get_term(P), @@ -490,7 +494,7 @@ t_erl_cons(Config) when is_list(Config) -> %% Tests the erl_length() function. t_erl_length(Config) when is_list(Config) -> - P = runner:start(?t_erl_length), + P = runner:start(Config, ?t_erl_length), 0 = erl_length(P, []), 1 = erl_length(P, [a]), @@ -513,7 +517,7 @@ erl_length(Port, List) -> %% Tests the erl_hd() function. t_erl_hd(Config) when is_list(Config) -> - P = runner:start(?t_erl_hd), + P = runner:start(Config, ?t_erl_hd), 'NULL' = erl_hd(P, 42), 'NULL' = erl_hd(P, abc), @@ -537,7 +541,7 @@ erl_hd(Port, List) -> %% Tests the erl_tail() function. t_erl_tl(Config) when is_list(Config) -> - P = runner:start(?t_erl_tl), + P = runner:start(Config, ?t_erl_tl), 'NULL' = erl_tl(P, 42), 'NULL' = erl_tl(P, abc), @@ -561,20 +565,20 @@ erl_tl(Port, List) -> %% Tests the type checking macros (done in the C program). type_checks(Config) when is_list(Config) -> - runner:test(?type_checks), + runner:test(Config, ?type_checks), ok. %% Tests the extractor macros (done in the C program). extractor_macros(Config) when is_list(Config) -> - runner:test(?extractor_macros), + runner:test(Config, ?extractor_macros), ok. %% This tests the erl_size() function. t_erl_size(Config) when is_list(Config) -> - P = runner:start(?t_erl_size), + P = runner:start(Config, ?t_erl_size), {term, 0} = (get_term(P)), {term, 4} = (get_term(P)), @@ -589,7 +593,7 @@ t_erl_size(Config) when is_list(Config) -> %% This tests the erl_var_content() function. t_erl_var_content(Config) when is_list(Config) -> - P = runner:start(?t_erl_var_content), + P = runner:start(Config, ?t_erl_var_content), {term, 17} = (get_term(P)), {term, "http://www.madonna.com"} = (get_term(P)), @@ -604,7 +608,7 @@ t_erl_var_content(Config) when is_list(Config) -> %% This tests the erl_element() function. t_erl_element(Config) when is_list(Config) -> - P = runner:start(?t_erl_element), + P = runner:start(Config, ?t_erl_element), {term, madonna} = get_term(P), {term, 21} = get_term(P), @@ -630,7 +634,7 @@ t_erl_element(Config) when is_list(Config) -> %% Tests the erl_iolist_length() function. t_erl_iolist_length(Config) when is_list(Config) -> - P = runner:start(?t_erl_iolist_length), + P = runner:start(Config, ?t_erl_iolist_length), %% Flat lists. @@ -697,7 +701,7 @@ erl_iolist_length(Port, List) -> %% Tests the erl_iolist_to_binary() function. t_erl_iolist_to_binary(Config) when is_list(Config) -> - P = runner:start(?t_erl_iolist_to_binary), + P = runner:start(Config, ?t_erl_iolist_to_binary), %% Flat lists. @@ -768,7 +772,7 @@ iolist_to_list(Port, Term) -> %% Tests the erl_iolist_to_string() function. t_erl_iolist_to_string(Config) when is_list(Config) -> - P = runner:start(?t_erl_iolist_to_string), + P = runner:start(Config, ?t_erl_iolist_to_string), %% Flat lists. @@ -947,14 +951,14 @@ collect_line1([C|Rest], Result) -> %% Test case submitted by Per Lundgren, ERV. high_chaparal(Config) when is_list(Config) -> - P = runner:start(?high_chaparal), + P = runner:start(Config, ?high_chaparal), {term, [hello, world]} = get_term(P), runner:recv_eot(P), ok. %% OTP-7448 broken_data(Config) when is_list(Config) -> - P = runner:start(?broken_data), + P = runner:start(Config, ?broken_data), runner:recv_eot(P), ok. diff --git a/lib/erl_interface/test/erl_ext_SUITE.erl b/lib/erl_interface/test/erl_ext_SUITE.erl index afaba1fd93..806339b122 100644 --- a/lib/erl_interface/test/erl_ext_SUITE.erl +++ b/lib/erl_interface/test/erl_ext_SUITE.erl @@ -25,6 +25,7 @@ -include("erl_ext_SUITE_data/ext_test_cases.hrl"). -export([all/0, suite/0, + init_per_testcase/2, compare_tuple/1, compare_list/1, compare_string/1, @@ -40,28 +41,30 @@ all() -> [compare_tuple, compare_list, compare_string, compare_list_string, compare_nc_ext]. +init_per_testcase(Case, Config) -> + runner:init_per_testcase(?MODULE, Case, Config). compare_tuple(Config) when is_list(Config) -> - P = runner:start(?compare_tuple), + P = runner:start(Config, ?compare_tuple), runner:recv_eot(P), ok. compare_list(Config) when is_list(Config) -> - P = runner:start(?compare_list), + P = runner:start(Config, ?compare_list), runner:recv_eot(P), ok. compare_string(Config) when is_list(Config) -> - P = runner:start(?compare_string), + P = runner:start(Config, ?compare_string), runner:recv_eot(P), ok. compare_list_string(Config) when is_list(Config) -> - P = runner:start(?compare_list_string), + P = runner:start(Config, ?compare_list_string), runner:recv_eot(P), ok. compare_nc_ext(Config) when is_list(Config) -> - P = runner:start(?compare_nc_ext), + P = runner:start(Config, ?compare_nc_ext), runner:recv_eot(P), ok. diff --git a/lib/erl_interface/test/erl_format_SUITE.erl b/lib/erl_interface/test/erl_format_SUITE.erl index c1a7d8377e..d984dcb08e 100644 --- a/lib/erl_interface/test/erl_format_SUITE.erl +++ b/lib/erl_interface/test/erl_format_SUITE.erl @@ -25,6 +25,7 @@ -include("erl_format_SUITE_data/format_test_cases.hrl"). -export([all/0, suite/0, + init_per_testcase/2, atoms/1, tuples/1, lists/1]). -import(runner, [get_term/1]). @@ -38,10 +39,13 @@ suite() -> all() -> [atoms, tuples, lists]. +init_per_testcase(Case, Config) -> + runner:init_per_testcase(?MODULE, Case, Config). + %% Tests formatting various atoms. atoms(Config) when is_list(Config) -> - P = runner:start(?atoms), + P = runner:start(Config, ?atoms), {term, ''} = get_term(P), {term, 'a'} = get_term(P), @@ -79,7 +83,7 @@ atoms(Config) when is_list(Config) -> %% Tests formatting various tuples tuples(Config) when is_list(Config) -> - P = runner:start(?tuples), + P = runner:start(Config, ?tuples), {term, {}} = get_term(P), {term, {a}} = get_term(P), @@ -100,7 +104,7 @@ tuples(Config) when is_list(Config) -> %% Tests formatting various lists lists(Config) when is_list(Config) -> - P = runner:start(?lists), + P = runner:start(Config, ?lists), {term, []} = get_term(P), {term, [a]} = get_term(P), diff --git a/lib/erl_interface/test/erl_global_SUITE.erl b/lib/erl_interface/test/erl_global_SUITE.erl index ecc6753c7f..560afd58ba 100644 --- a/lib/erl_interface/test/erl_global_SUITE.erl +++ b/lib/erl_interface/test/erl_global_SUITE.erl @@ -25,6 +25,7 @@ -include("erl_global_SUITE_data/erl_global_test_cases.hrl"). -export([all/0,suite/0, + init_per_testcase/2, erl_global_registration/1, erl_global_whereis/1, erl_global_names/1]). @@ -39,9 +40,11 @@ suite() -> [{ct_hooks,[ts_install_cth]}, {timetrap, {seconds, 30}}]. +init_per_testcase(Case, Config) -> + runner:init_per_testcase(?MODULE, Case, Config). erl_global_registration(Config) when is_list(Config) -> - P = runner:start(?interpret), + P = runner:start(Config, ?interpret), {ok, Fd} = erl_connect(P, node(), 42, erlang:get_cookie(), 0), ok = erl_global_register(P, Fd, ?GLOBAL_NAME), @@ -53,7 +56,7 @@ erl_global_registration(Config) when is_list(Config) -> ok. erl_global_whereis(Config) when is_list(Config) -> - P = runner:start(?interpret), + P = runner:start(Config, ?interpret), {ok, Fd} = erl_connect(P, node(), 42, erlang:get_cookie(), 0), Self = self(), @@ -66,7 +69,7 @@ erl_global_whereis(Config) when is_list(Config) -> ok. erl_global_names(Config) when is_list(Config) -> - P = runner:start(?interpret), + P = runner:start(Config, ?interpret), {ok, Fd} = erl_connect(P, node(), 42, erlang:get_cookie(), 0), Self = self(), diff --git a/lib/erl_interface/test/erl_match_SUITE.erl b/lib/erl_interface/test/erl_match_SUITE.erl index 5566714092..f1f6892ae0 100644 --- a/lib/erl_interface/test/erl_match_SUITE.erl +++ b/lib/erl_interface/test/erl_match_SUITE.erl @@ -25,6 +25,7 @@ -include("erl_match_SUITE_data/match_test_cases.hrl"). -export([all/0, suite/0, + init_per_testcase/2, atoms/1, lists/1, tuples/1, references/1, pids/1, ports/1, bind/1, integers/1, floats/1, binaries/1, strings/1]). @@ -40,6 +41,8 @@ all() -> [atoms, lists, tuples, references, pids, ports, bind, integers, floats, binaries, strings]. +init_per_testcase(Case, Config) -> + runner:init_per_testcase(?MODULE, Case, Config). atoms(Config) when is_list(Config) -> P = start_matcher(Config), @@ -239,7 +242,7 @@ bind(Config) when is_list(Config) -> ok. start_bind(Config) -> - runner:start(?erl_match_bind). + runner:start(Config, ?erl_match_bind). bind_ok(Port, Bind, Term) -> true = erl_bind(Port, Bind, Term). @@ -258,7 +261,7 @@ erl_bind(Port, Pattern, Term) -> start_matcher(Config) -> - runner:start(?erl_match_server). + runner:start(Config, ?erl_match_server). eq(Port, Pattern, Term) -> true = erl_match(Port, Pattern, Term). diff --git a/lib/erl_interface/test/port_call_SUITE.erl b/lib/erl_interface/test/port_call_SUITE.erl index fb10bd895f..d31b2372ab 100644 --- a/lib/erl_interface/test/port_call_SUITE.erl +++ b/lib/erl_interface/test/port_call_SUITE.erl @@ -32,7 +32,9 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% --export([all/0, suite/0, basic/1]). +-export([all/0, suite/0, + init_per_testcase/2, + basic/1]). % Private exports -include_lib("common_test/include/ct.hrl"). @@ -44,6 +46,8 @@ suite() -> all() -> [basic]. +init_per_testcase(Case, Config) -> + runner:init_per_testcase(?MODULE, Case, Config). basic(Config) when is_list(Config) -> case os:type() of diff --git a/lib/erl_interface/test/runner.erl b/lib/erl_interface/test/runner.erl index cbdeb3c8a9..484890006e 100644 --- a/lib/erl_interface/test/runner.erl +++ b/lib/erl_interface/test/runner.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2016. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -21,8 +21,9 @@ %% -module(runner). --export([test/1, test/2, - start/1, send_term/2, finish/1, send_eot/1, recv_eot/1, +-export([test/2, test/3, + init_per_testcase/3, + start/2, send_term/2, finish/1, send_eot/1, recv_eot/1, get_term/1, get_term/2]). -define(default_timeout, 5000). @@ -32,11 +33,11 @@ %% This function is useful for test cases written in C which requires %% no further input, and only returns a result by calling report(). -test(Tc) -> - test(Tc, ?default_timeout). +test(Config, Tc) -> + test(Config, Tc, ?default_timeout). -test(Tc, Timeout) -> - Port = start(Tc), +test(Config, Tc, Timeout) -> + Port = start(Config, Tc), case get_term(Port, Timeout) of eot -> @@ -54,12 +55,51 @@ test(Tc, Timeout) -> %% %% Returns: {ok, Port} -start({Prog, Tc}) when is_list(Prog), is_integer(Tc) -> - Port = open_port({spawn, Prog}, [{packet, 4}, exit_status]), +start(Config, {Prog, Tc}) when is_list(Prog), is_integer(Tc) -> + Port = open_port({spawn, prog_cmd(Config, Prog)}, + [{packet, 4}, exit_status]), Command = [Tc div 256, Tc rem 256], Port ! {self(), {command, Command}}, Port. +prog_cmd(Config, Prog) -> + case proplists:get_value(valgrind_cmd_fun, Config) of + undefined -> + Prog; + Fun when is_function(Fun) -> + Fun(Prog) + end. + +init_per_testcase(Suite, Case, Config) -> + case os:getenv("VALGRIND_LOG_DIR") of + false -> + Config; + LogDir -> + Valgrind = case os:find_executable("valgrind") of + false -> + ct:fail("VALGRIND_LOG_DIR set, " + "but no valgrind executable found"); + VG -> VG + end, + + LogFileOpt = case os:getenv("VALGRIND_LOG_XML") of + false -> + " --log-file="; + "yes" -> + " --xml=yes --xml-file=" + end, + Fun = fun(Prog) -> + LogFile = io_lib:format("erl_interface-~w.~w-~s.log.%p", + [Suite, Case, filename:basename(Prog)]), + Valgrind + ++ LogFileOpt ++ filename:join(LogDir,LogFile) + ++ " " ++ os:getenv("VALGRIND_MISC_FLAGS","") + ++ " " ++ Prog + end, + [{valgrind_cmd_fun, Fun} | Config] + end. + + %% Finishes a test case by send an 'eot' message to the C program %% and waiting for an 'eot'. %% diff --git a/lib/erl_interface/vsn.mk b/lib/erl_interface/vsn.mk index 8b6e91757d..586b23c5b3 100644 --- a/lib/erl_interface/vsn.mk +++ b/lib/erl_interface/vsn.mk @@ -1,2 +1,2 @@ -EI_VSN = 3.10.2 +EI_VSN = 3.10.3 ERL_INTERFACE_VSN = $(EI_VSN) diff --git a/lib/et/doc/src/Makefile b/lib/et/doc/src/Makefile index 162d36e274..93e2f8eeee 100644 --- a/lib/et/doc/src/Makefile +++ b/lib/et/doc/src/Makefile @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 2002-2016. All Rights Reserved. +# Copyright Ericsson AB 2002-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -85,6 +85,7 @@ clean clean_docs: fi \ done rm -rf $(HTMLDIR)/* + rm -rf $(XMLDIR) rm -f $(MAN3DIR)/* rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) rm -f errs core *~ diff --git a/lib/et/doc/src/files.mk b/lib/et/doc/src/files.mk index 7437da7ce3..c9041caa81 100644 --- a/lib/et/doc/src/files.mk +++ b/lib/et/doc/src/files.mk @@ -1,7 +1,7 @@ #-*-makefile-*- ; force emacs to enter makefile-mode # %CopyrightBegin% # -# Copyright Ericsson AB 2002-2016. All Rights Reserved. +# Copyright Ericsson AB 2002-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. diff --git a/lib/et/doc/src/notes.xml b/lib/et/doc/src/notes.xml index f0995b7c19..e7cec937b3 100644 --- a/lib/et/doc/src/notes.xml +++ b/lib/et/doc/src/notes.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2002</year><year>2016</year> + <year>2002</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -37,6 +37,21 @@ one section in this document. The title of each section is the version number of <c>Event Tracer (ET)</c>.</p> +<section><title>ET 1.6.2</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> Calls to <c>erlang:get_stacktrace()</c> are removed. + </p> + <p> + Own Id: OTP-14861</p> + </item> + </list> + </section> + +</section> + <section><title>ET 1.6.1</title> <section><title>Improvements and New Features</title> diff --git a/lib/et/src/et.app.src b/lib/et/src/et.app.src index f4e32f734d..8cea1ba842 100644 --- a/lib/et/src/et.app.src +++ b/lib/et/src/et.app.src @@ -2,7 +2,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2002-2016. All Rights Reserved. +%% Copyright Ericsson AB 2002-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/et/src/et_wx_contents_viewer.erl b/lib/et/src/et_wx_contents_viewer.erl index 7ca41da3e8..580c921139 100644 --- a/lib/et/src/et_wx_contents_viewer.erl +++ b/lib/et/src/et_wx_contents_viewer.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2000-2017. All Rights Reserved. +%% Copyright Ericsson AB 2000-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/et/test/et_test_lib.erl b/lib/et/test/et_test_lib.erl index fc469f646a..4addcd7ca8 100644 --- a/lib/et/test/et_test_lib.erl +++ b/lib/et/test/et_test_lib.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009-2016. All Rights Reserved. +%% Copyright Ericsson AB 2009-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/et/test/ett.erl b/lib/et/test/ett.erl index 2b276eab1a..dd274ec317 100644 --- a/lib/et/test/ett.erl +++ b/lib/et/test/ett.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009-2016. All Rights Reserved. +%% Copyright Ericsson AB 2009-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/et/vsn.mk b/lib/et/vsn.mk index aab63a402e..08ded8b6f1 100644 --- a/lib/et/vsn.mk +++ b/lib/et/vsn.mk @@ -1 +1 @@ -ET_VSN = 1.6.1 +ET_VSN = 1.6.2 diff --git a/lib/eunit/doc/src/Makefile b/lib/eunit/doc/src/Makefile index e91d947592..117542cb37 100644 --- a/lib/eunit/doc/src/Makefile +++ b/lib/eunit/doc/src/Makefile @@ -146,6 +146,7 @@ debug opt: clean clean_docs: rm -rf $(HTMLDIR)/* + rm -rf $(XMLDIR) rm -f $(MAN3DIR)/* rm -f $(XML_REF3_FILES) $(XML_CHAPTER_FILES) *.html rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) diff --git a/lib/eunit/doc/src/notes.xml b/lib/eunit/doc/src/notes.xml index b38cb2e70e..0d2d61d553 100644 --- a/lib/eunit/doc/src/notes.xml +++ b/lib/eunit/doc/src/notes.xml @@ -5,7 +5,7 @@ <header> <copyright> <year>2008</year> - <year>2017</year> + <year>2018</year> <holder>Ericsson AB, All Rights Reserved</holder> </copyright> <legalnotice> @@ -33,6 +33,21 @@ </header> <p>This document describes the changes made to the EUnit application.</p> +<section><title>Eunit 2.3.6</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> Calls to <c>erlang:get_stacktrace()</c> are removed. + </p> + <p> + Own Id: OTP-14861</p> + </item> + </list> + </section> + +</section> + <section><title>Eunit 2.3.5</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/eunit/vsn.mk b/lib/eunit/vsn.mk index 2ed9eaac16..054e21f6ad 100644 --- a/lib/eunit/vsn.mk +++ b/lib/eunit/vsn.mk @@ -1 +1 @@ -EUNIT_VSN = 2.3.5 +EUNIT_VSN = 2.3.6 diff --git a/lib/ftp/Makefile b/lib/ftp/Makefile index 555f8b0dea..e0c9de42e4 100644 --- a/lib/ftp/Makefile +++ b/lib/ftp/Makefile @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 1996-2016. All Rights Reserved. +# Copyright Ericsson AB 1996-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. diff --git a/lib/ftp/doc/src/Makefile b/lib/ftp/doc/src/Makefile index e96a9c032f..20fbbc73a9 100644 --- a/lib/ftp/doc/src/Makefile +++ b/lib/ftp/doc/src/Makefile @@ -103,6 +103,7 @@ pdf: $(TOP_PDF_FILE) html: gifs $(HTML_REF_MAN_FILE) clean clean_docs: clean_html clean_man clean_pdf + rm -rf $(XMLDIR) rm -f errs core *~ man: $(MAN3_FILES) diff --git a/lib/ftp/doc/src/ftp.xml b/lib/ftp/doc/src/ftp.xml index 18770ebcb4..23a9e3fec7 100644 --- a/lib/ftp/doc/src/ftp.xml +++ b/lib/ftp/doc/src/ftp.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1997</year><year>2016</year> + <year>1997</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -546,11 +546,12 @@ <v>start_option() = {verbose, verbose()} | {debug, debug()}</v> <v>verbose() = boolean() (default is false)</v> <v>debug() = disable | debug | trace (default is disable)</v> - <v>open_option() = {ipfamily, ipfamily()} | {port, port()} | {mode, mode()} | {tls, tls_options()} | {timeout, timeout()} | {dtimeout, dtimeout()} | {progress, progress()}</v> + <v>open_option() = {ipfamily, ipfamily()} | {port, port()} | {mode, mode()} | {tls, tls_options()} | {timeout, timeout()} | {dtimeout, dtimeout()} | {progress, progress() | {sock_ctrl, sock_opts()} | {sock_data_act, sock_opts()} | {sock_data_pass, sock_opts()} }</v> <v>ipfamily() = inet | inet6 | inet6fb4 (default is inet)</v> <v>port() = integer() > 0 (default is 21)</v> <v>mode() = active | passive (default is passive)</v> <v>tls_options() = [<seealso marker="ssl:ssl#type-ssloption">ssl:ssloption()</seealso>]</v> + <v>sock_opts() = [<seealso marker="kernel:gen_tcp#type-option">gen_tcp:option()</seealso> except for ipv6_v6only, active, packet, mode, packet_size and header</v> <v>timeout() = integer() > 0 (default is 60000 milliseconds)</v> <v>dtimeout() = integer() > 0 | infinity (default is infinity)</v> <v>pogress() = ignore | {module(), function(), initial_data()} (default is ignore)</v> @@ -573,6 +574,11 @@ is used for securing both the control connection and the data sessions. </p> + <p>The options <c>sock_ctrl</c>, <c>sock_data_act</c> and <c>sock_data_pass</c> passes options down to + the underlying transport layer (tcp). The default value for <c>sock_ctrl</c> is <c>[]</c>. Both + <c>sock_data_act</c> and <c>sock_data_pass</c> uses the value of <c>sock_ctrl</c> as default value. + </p> + <p>A session opened in this way is closed using function <seealso marker="#close">close</seealso>.</p> diff --git a/lib/ftp/src/Makefile b/lib/ftp/src/Makefile index 6a6df6bde4..93e3c4a8e5 100644 --- a/lib/ftp/src/Makefile +++ b/lib/ftp/src/Makefile @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 1999-2017. All Rights Reserved. +# Copyright Ericsson AB 1999-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. diff --git a/lib/ftp/src/ftp.erl b/lib/ftp/src/ftp.erl index 8790bfec13..40f6b53fa3 100644 --- a/lib/ftp/src/ftp.erl +++ b/lib/ftp/src/ftp.erl @@ -101,6 +101,9 @@ %% data needed further on. caller = undefined, % term() ipfamily, % inet | inet6 | inet6fb4 + sockopts_ctrl = [], + sockopts_data_passive = [], + sockopts_data_active = [], progress = ignore, % ignore | pid() dtimeout = ?DATA_ACCEPT_TIMEOUT, % non_neg_integer() | infinity tls_upgrading_data_connection = false, @@ -135,9 +138,10 @@ start_standalone(Options) -> try {ok, StartOptions} = start_options(Options), {ok, OpenOptions} = open_options(Options), + {ok, SocketOptions} = socket_options(Options), case start_link(StartOptions, []) of {ok, Pid} -> - call(Pid, {open, ip_comm, OpenOptions}, plain); + call(Pid, {open, ip_comm, OpenOptions, SocketOptions}, plain); Error1 -> Error1 end @@ -149,10 +153,11 @@ start_standalone(Options) -> start_service(Options) -> try {ok, StartOptions} = start_options(Options), - {ok, OpenOptions} = open_options(Options), + {ok, OpenOptions} = open_options(Options), + {ok, SocketOptions} = socket_options(Options), case ftp_sup:start_child([[[{client, self()} | StartOptions], []]]) of {ok, Pid} -> - call(Pid, {open, ip_comm, OpenOptions}, plain); + call(Pid, {open, ip_comm, OpenOptions, SocketOptions}, plain); Error1 -> Error1 end @@ -200,9 +205,10 @@ open({option_list, Options}) when is_list(Options) -> try {ok, StartOptions} = start_options(Options), {ok, OpenOptions} = open_options(Options), + {ok, SockOpts} = socket_options(Options), case ftp_sup:start_child([[[{client, self()} | StartOptions], []]]) of {ok, Pid} -> - call(Pid, {open, ip_comm, OpenOptions}, plain); + call(Pid, {open, ip_comm, OpenOptions, SockOpts}, plain); Error1 -> Error1 end @@ -227,9 +233,10 @@ open(Host, Opts) when is_list(Opts) -> try {ok, StartOptions} = start_options(Opts), {ok, OpenOptions} = open_options([{host, Host}|Opts]), + {ok, SocketOptions} = socket_options(Opts), case start_link(StartOptions, []) of {ok, Pid} -> - do_open(Pid, OpenOptions, tls_options(Opts)); + do_open(Pid, OpenOptions, SocketOptions, tls_options(Opts)); Error1 -> Error1 end @@ -238,8 +245,8 @@ open(Host, Opts) when is_list(Opts) -> Error2 end. -do_open(Pid, OpenOptions, TLSOpts) -> - case call(Pid, {open, ip_comm, OpenOptions}, plain) of +do_open(Pid, OpenOptions, SocketOptions, TLSOpts) -> + case call(Pid, {open, ip_comm, OpenOptions, SocketOptions}, plain) of {ok, Pid} -> maybe_tls_upgrade(Pid, TLSOpts); Error -> @@ -1026,7 +1033,7 @@ handle_call({_,latest_ctrl_response}, _, #state{latest_ctrl_response=Resp} = Sta handle_call({Pid, _}, _, #state{owner = Owner} = State) when Owner =/= Pid -> {reply, {error, not_connection_owner}, State}; -handle_call({_, {open, ip_comm, Opts}}, From, State) -> +handle_call({_, {open, ip_comm, Opts, {CtrlOpts, DataPassOpts, DataActOpts}}}, From, State) -> case key_search(host, Opts, undefined) of undefined -> {stop, normal, {error, ehost}, State}; @@ -1043,6 +1050,9 @@ handle_call({_, {open, ip_comm, Opts}}, From, State) -> mode = Mode, progress = progress(Progress), ipfamily = IpFamily, + sockopts_ctrl = CtrlOpts, + sockopts_data_passive = DataPassOpts, + sockopts_data_active = DataActOpts, dtimeout = DTimeout, ftp_extension = FtpExt}, @@ -1055,28 +1065,6 @@ handle_call({_, {open, ip_comm, Opts}}, From, State) -> end end; -handle_call({_, {open, ip_comm, Host, Opts}}, From, State) -> - Mode = key_search(mode, Opts, ?DEFAULT_MODE), - Port = key_search(port, Opts, ?FTP_PORT), - Timeout = key_search(timeout, Opts, ?CONNECTION_TIMEOUT), - DTimeout = key_search(dtimeout, Opts, ?DATA_ACCEPT_TIMEOUT), - Progress = key_search(progress, Opts, ignore), - FtpExt = key_search(ftp_extension, Opts, ?FTP_EXT_DEFAULT), - - State2 = State#state{client = From, - mode = Mode, - progress = progress(Progress), - dtimeout = DTimeout, - ftp_extension = FtpExt}, - - case setup_ctrl_connection(Host, Port, Timeout, State2) of - {ok, State3, WaitTimeout} -> - {noreply, State3, WaitTimeout}; - {error, _Reason} -> - gen_server:reply(From, {error, ehost}), - {stop, normal, State2#state{client = undefined}} - end; - handle_call({_, {open, tls_upgrade, TLSOptions}}, From, State) -> _ = send_ctrl_message(State, mk_cmd("AUTH TLS", [])), activate_ctrl_connection(State), @@ -1659,11 +1647,12 @@ handle_ctrl_result({pos_compl, Lines}, client = From, caller = {setup_data_connection, Caller}, csock = CSock, + sockopts_data_passive = SockOpts, timeout = Timeout} = State) -> [_, PortStr | _] = lists:reverse(string:tokens(Lines, "|")), {ok, {IP, _}} = peername(CSock), - case connect(IP, list_to_integer(PortStr), Timeout, State) of + case connect(IP, list_to_integer(PortStr), SockOpts, Timeout, State) of {ok, _, Socket} -> handle_caller(State#state{caller = Caller, dsock = {tcp, Socket}}); {error, _Reason} = Error -> @@ -1676,7 +1665,8 @@ handle_ctrl_result({pos_compl, Lines}, ipfamily = inet, client = From, caller = {setup_data_connection, Caller}, - timeout = Timeout, + timeout = Timeout, + sockopts_data_passive = SockOpts, ftp_extension = false} = State) -> {_, [?LEFT_PAREN | Rest]} = @@ -1690,7 +1680,7 @@ handle_ctrl_result({pos_compl, Lines}, Port = (P1 * 256) + P2, ?DBG('<--data tcp connect to ~p:~p, Caller=~p~n',[IP,Port,Caller]), - case connect(IP, Port, Timeout, State) of + case connect(IP, Port, SockOpts, Timeout, State) of {ok, _, Socket} -> handle_caller(State#state{caller = Caller, dsock = {tcp,Socket}}); {error, _Reason} = Error -> @@ -1705,13 +1695,14 @@ handle_ctrl_result({pos_compl, Lines}, caller = {setup_data_connection, Caller}, csock = CSock, timeout = Timeout, + sockopts_data_passive = SockOpts, ftp_extension = true} = State) -> [_, PortStr | _] = lists:reverse(string:tokens(Lines, "|")), {ok, {IP, _}} = peername(CSock), ?DBG('<--data tcp connect to ~p:~p, Caller=~p~n',[IP,PortStr,Caller]), - case connect(IP, list_to_integer(PortStr), Timeout, State) of + case connect(IP, list_to_integer(PortStr), SockOpts, Timeout, State) of {ok, _, Socket} -> handle_caller(State#state{caller = Caller, dsock = {tcp, Socket}}); {error, _Reason} = Error -> @@ -2075,9 +2066,9 @@ handle_caller(#state{caller = {transfer_data, {Cmd, Bin, RemoteFile}}} = %% Connect to FTP server at Host (default is TCP port 21) %% in order to establish a control connection. -setup_ctrl_connection(Host, Port, Timeout, State) -> +setup_ctrl_connection(Host, Port, Timeout, #state{sockopts_ctrl = SockOpts} = State) -> MsTime = erlang:monotonic_time(), - case connect(Host, Port, Timeout, State) of + case connect(Host, Port, SockOpts, Timeout, State) of {ok, IpFam, CSock} -> NewState = State#state{csock = {tcp, CSock}, ipfamily = IpFam}, activate_ctrl_connection(NewState), @@ -2095,12 +2086,15 @@ setup_ctrl_connection(Host, Port, Timeout, State) -> setup_data_connection(#state{mode = active, caller = Caller, csock = CSock, + sockopts_data_active = SockOpts, ftp_extension = FtpExt} = State) -> case (catch sockname(CSock)) of - {ok, {{_, _, _, _, _, _, _, _} = IP, _}} -> + {ok, {{_, _, _, _, _, _, _, _} = IP0, _}} -> + IP = proplists:get_value(ip, SockOpts, IP0), {ok, LSock} = gen_tcp:listen(0, [{ip, IP}, {active, false}, - inet6, binary, {packet, 0}]), + inet6, binary, {packet, 0} | + lists:keydelete(ip,1,SockOpts)]), {ok, {_, Port}} = sockname({tcp,LSock}), IpAddress = inet_parse:ntoa(IP), Cmd = mk_cmd("EPRT |2|~s|~p|", [IpAddress, Port]), @@ -2108,9 +2102,11 @@ setup_data_connection(#state{mode = active, activate_ctrl_connection(State), {noreply, State#state{caller = {setup_data_connection, {LSock, Caller}}}}; - {ok, {{_,_,_,_} = IP, _}} -> + {ok, {{_,_,_,_} = IP0, _}} -> + IP = proplists:get_value(ip, SockOpts, IP0), {ok, LSock} = gen_tcp:listen(0, [{ip, IP}, {active, false}, - binary, {packet, 0}]), + binary, {packet, 0} | + lists:keydelete(ip,1,SockOpts)]), {ok, Port} = inet:port(LSock), _ = case FtpExt of false -> @@ -2149,41 +2145,41 @@ setup_data_connection(#state{mode = passive, ipfamily = inet, activate_ctrl_connection(State), {noreply, State#state{caller = {setup_data_connection, Caller}}}. -connect(Host, Port, Timeout, #state{ipfamily = inet = IpFam}) -> - connect2(Host, Port, IpFam, Timeout); +connect(Host, Port, SockOpts, Timeout, #state{ipfamily = inet = IpFam}) -> + connect2(Host, Port, IpFam, SockOpts, Timeout); -connect(Host, Port, Timeout, #state{ipfamily = inet6 = IpFam}) -> - connect2(Host, Port, IpFam, Timeout); +connect(Host, Port, SockOpts, Timeout, #state{ipfamily = inet6 = IpFam}) -> + connect2(Host, Port, IpFam, SockOpts, Timeout); -connect(Host, Port, Timeout, #state{ipfamily = inet6fb4}) -> +connect(Host, Port, SockOpts, Timeout, #state{ipfamily = inet6fb4}) -> case inet:getaddr(Host, inet6) of {ok, {0, 0, 0, 0, 0, 16#ffff, _, _} = IPv6} -> case inet:getaddr(Host, inet) of {ok, IPv4} -> IpFam = inet, - connect2(IPv4, Port, IpFam, Timeout); + connect2(IPv4, Port, IpFam, SockOpts, Timeout); _ -> IpFam = inet6, - connect2(IPv6, Port, IpFam, Timeout) + connect2(IPv6, Port, IpFam, SockOpts, Timeout) end; {ok, IPv6} -> IpFam = inet6, - connect2(IPv6, Port, IpFam, Timeout); + connect2(IPv6, Port, IpFam, SockOpts, Timeout); _ -> case inet:getaddr(Host, inet) of {ok, IPv4} -> IpFam = inet, - connect2(IPv4, Port, IpFam, Timeout); + connect2(IPv4, Port, IpFam, SockOpts, Timeout); Error -> Error end end. -connect2(Host, Port, IpFam, Timeout) -> - Opts = [IpFam, binary, {packet, 0}, {active, false}], +connect2(Host, Port, IpFam, SockOpts, Timeout) -> + Opts = [IpFam, binary, {packet, 0}, {active, false} | SockOpts], case gen_tcp:connect(Host, Port, Opts, Timeout) of {ok, Sock} -> {ok, IpFam, Sock}; @@ -2553,6 +2549,32 @@ open_options(Options) -> {ftp_extension, ValidateFtpExtension, false, ?FTP_EXT_DEFAULT}], validate_options(Options, ValidOptions, []). +socket_options(Options) -> + CtrlOpts = proplists:get_value(sock_ctrl, Options, []), + DataActOpts = proplists:get_value(sock_data_act, Options, CtrlOpts), + DataPassOpts = proplists:get_value(sock_data_pass, Options, CtrlOpts), + case [O || O <- lists:usort(CtrlOpts++DataPassOpts++DataActOpts), + not valid_socket_option(O)] of + [] -> + {ok, {CtrlOpts, DataPassOpts, DataActOpts}}; + Invalid -> + throw({error,{sock_opts,Invalid}}) + end. + + +valid_socket_option(inet ) -> false; +valid_socket_option(inet6 ) -> false; +valid_socket_option({ipv6_v6only, _}) -> false; +valid_socket_option({active,_} ) -> false; +valid_socket_option({packet,_} ) -> false; +valid_socket_option({mode,_} ) -> false; +valid_socket_option(binary ) -> false; +valid_socket_option(list ) -> false; +valid_socket_option({header,_} ) -> false; +valid_socket_option({packet_size,_} ) -> false; +valid_socket_option(_) -> true. + + tls_options(Options) -> %% Options will be validated by ssl application proplists:get_value(tls, Options, undefined). diff --git a/lib/ftp/src/ftp_progress.erl b/lib/ftp/src/ftp_progress.erl index a6263e5cd7..64c612519d 100644 --- a/lib/ftp/src/ftp_progress.erl +++ b/lib/ftp/src/ftp_progress.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2016. All Rights Reserved. +%% Copyright Ericsson AB 2005-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ftp/src/ftp_response.erl b/lib/ftp/src/ftp_response.erl index d54d97dc91..8d00153ba8 100644 --- a/lib/ftp/src/ftp_response.erl +++ b/lib/ftp/src/ftp_response.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2016. All Rights Reserved. +%% Copyright Ericsson AB 2005-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ftp/test/erl_make_certs.erl b/lib/ftp/test/erl_make_certs.erl index 2db95825bc..e10ecf01f9 100644 --- a/lib/ftp/test/erl_make_certs.erl +++ b/lib/ftp/test/erl_make_certs.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2011-2016. All Rights Reserved. +%% Copyright Ericsson AB 2011-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ftp/test/ftp_SUITE.erl b/lib/ftp/test/ftp_SUITE.erl index 92d2c36a86..7c87d5cbdb 100644 --- a/lib/ftp/test/ftp_SUITE.erl +++ b/lib/ftp/test/ftp_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2017. All Rights Reserved. +%% Copyright Ericsson AB 2004-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ftp/test/ftp_format_SUITE.erl b/lib/ftp/test/ftp_format_SUITE.erl index 95d594a44b..e1d0de2390 100644 --- a/lib/ftp/test/ftp_format_SUITE.erl +++ b/lib/ftp/test/ftp_format_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2016. All Rights Reserved. +%% Copyright Ericsson AB 2005-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ftp/test/ftp_property_test_SUITE.erl b/lib/ftp/test/ftp_property_test_SUITE.erl index 46ed6959a8..cd43613b9e 100644 --- a/lib/ftp/test/ftp_property_test_SUITE.erl +++ b/lib/ftp/test/ftp_property_test_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2016. All Rights Reserved. +%% Copyright Ericsson AB 2004-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ftp/test/property_test/ftp_simple_client_server.erl b/lib/ftp/test/property_test/ftp_simple_client_server.erl index 1bc54128f6..d304478b47 100644 --- a/lib/ftp/test/property_test/ftp_simple_client_server.erl +++ b/lib/ftp/test/property_test/ftp_simple_client_server.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2016. All Rights Reserved. +%% Copyright Ericsson AB 2004-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/hipe/cerl/Makefile b/lib/hipe/cerl/Makefile index b6116c4276..f653dce36f 100644 --- a/lib/hipe/cerl/Makefile +++ b/lib/hipe/cerl/Makefile @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 2003-2016. All Rights Reserved. +# Copyright Ericsson AB 2003-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. diff --git a/lib/hipe/doc/src/Makefile b/lib/hipe/doc/src/Makefile index 1c774d3357..bd6a7b2f74 100644 --- a/lib/hipe/doc/src/Makefile +++ b/lib/hipe/doc/src/Makefile @@ -94,6 +94,7 @@ debug opt: clean clean_docs: rm -rf $(HTMLDIR)/* + rm -rf $(XMLDIR) rm -f $(MAN3DIR)/* rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) rm -f errs core *~ diff --git a/lib/hipe/doc/src/hipe_app.xml b/lib/hipe/doc/src/hipe_app.xml index fc42ecd97d..99759a2f2c 100644 --- a/lib/hipe/doc/src/hipe_app.xml +++ b/lib/hipe/doc/src/hipe_app.xml @@ -4,7 +4,7 @@ <appref> <header> <copyright> - <year>1997</year><year>2016</year> + <year>1997</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -35,6 +35,14 @@ <app>HiPE</app> <appsummary>The HiPE Application</appsummary> <description> + <note> + <p> + HiPE and execution of HiPE compiled code only have limited support by + the OTP team at Ericsson. The OTP team only does limited maintenance + of HiPE and does not actively develop HiPE. HiPE is mainly supported + by the HiPE team at Uppsala University. + </p> + </note> <p> The normal way to native-compile an Erlang module using HiPE is to include the atom native in the Erlang compiler options, as in:</p> @@ -70,12 +78,12 @@ </item> <tag>NIFs</tag> - <item><p>Modules compiled with HiPE can not call <seealso marker="erts:erlang#load_nif-2"> + <item><p>Modules compiled with HiPE cannot call <seealso marker="erts:erlang#load_nif-2"> <c>erlang:load_nif/2</c></seealso> to load NIFs.</p> </item> <tag>-on_load</tag> - <item><p>Modules compiled with HiPE can not use + <item><p>Modules compiled with HiPE cannot use <seealso marker="doc/reference_manual:code_loading#on_load"><c>-on_load()</c></seealso> directives.</p> </item> @@ -108,7 +116,7 @@ queue when the reference was created will be bypassed, as they cannot possibly contain the reference. HiPE currently has an optimization similar this, but it is not guaranteed to - bypass all messages. In the worst case scenario it, cannot + bypass all messages. In the worst case scenario, it cannot bypass any messages at all. </p> <p> @@ -117,21 +125,55 @@ </p> </item> + <tag>Garbage collection after BIFs</tag> + <item> + <p> + The condition for determining whether a garbage collection + is needed or not has changed in later releases. HiPE has not + been updated regarding this which may cause premature garbage + collections after BIF calls. + </p> + </item> + </taglist> </section> <section> <title>Stability Issues</title> <taglist> - <tag>Not yielding in <c>receive</c> statements</tag> + <tag>Not checking reduction count on function returns</tag> <item> - <p>HiPE will not yield in <c>receive</c> statements where - appropriate. If a process have lots of signals in its signal - queue and execute a HiPE compiled <c>receive</c> statement, - the scheduler thread performing the execution may be stuck - in the <c>receive</c> statement for a very long time. This - can in turn cause various severe issues such as for example - prevent the runtime system from being able to release - memory. + <p> + BEAM checks the reduction count and schedules out the executing + process if needed both when calling a function and when returning + from a function call that was not called using a tail call. + HiPE only checks the reduction count when calling a function. + </p> + <p> + The runtime system might need to schedule out a process + in order to reclaim memory. If the process isn't scheduled + out soon after the process has entered this state, memory + consumption will quickly grow. Maintaining this state is also + quite expensive performance wise. + </p> + <p> + Processes executing code that performs large recursions and + produce data after returning from recursive calls may have to + be scheduled out when returning from a function call. Since + HiPE does not check reductions on returns, processes executing + such HiPE compiled code may cause huge peeks in memory + consumption as well as severe performance degradation. + </p> + </item> + + <tag>Not bumping appropriate amount of reductions in <c>receive</c> statements</tag> + <item> + <p> + The process signaling improvements made in ERTS version + 10.0 moved potentially significant amounts of work into the + receive statement from other places. In order to account for + this work, the reduction count should be bumped on the + executing process. Reductions are not bumped when entering + the <c>receive</c> statement from HiPE compiled code. </p> </item> </taglist> diff --git a/lib/hipe/doc/src/notes.xml b/lib/hipe/doc/src/notes.xml index c190a89260..f47868296a 100644 --- a/lib/hipe/doc/src/notes.xml +++ b/lib/hipe/doc/src/notes.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2006</year><year>2017</year> + <year>2006</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -31,6 +31,51 @@ </header> <p>This document describes the changes made to HiPE.</p> +<section><title>Hipe 3.18</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Optimize <c>receive</c> statements that are only waiting + for messages containing a reference created before the + receive. All messages that existed in the queue when the + reference was created will be bypassed, as they cannot + possibly contain the reference. This optimization has + existed for vanilla BEAM since OTP R14.</p> + <p> + Own Id: OTP-14785 Aux Id: PR-1632 </p> + </item> + <item> + <p> + Add validation pass to hipe compiler to detect internal + errors causing primop calls that may trigger an unsafe GC + at run-time. The pass can be disabled with option + <c>no_verify_gcsafe</c>.</p> + <p> + Own Id: OTP-14900 Aux Id: PR-1685, PR-1621 </p> + </item> + <item> + <p> + Make hipe compiled code work on x86_64 (amd64) with OS + security feature PIE, where executable code can be loaded + into a random location. Old behavior, if hipe was + enabled, was to disable PIE build options for the VM.</p> + <p> + Own Id: OTP-14903</p> + </item> + <item> + <p> + Inline more type test BIFs; <c>is_number</c>, + <c>is_bitstring</c>, <c>is_map</c>.</p> + <p> + Own Id: OTP-14941 Aux Id: PR-1718 </p> + </item> + </list> + </section> + +</section> + <section><title>Hipe 3.17.1</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/hipe/icode/hipe_icode_range.erl b/lib/hipe/icode/hipe_icode_range.erl index 34b18acccd..098a7a8d8c 100644 --- a/lib/hipe/icode/hipe_icode_range.erl +++ b/lib/hipe/icode/hipe_icode_range.erl @@ -1633,7 +1633,7 @@ inf_bsl(_, pos_inf) -> neg_inf; inf_bsl(Number, neg_inf) when is_integer(Number), Number >= 0 -> 0; inf_bsl(_Number, neg_inf) -> -1; inf_bsl(Number1, Number2) when is_integer(Number1), is_integer(Number2) -> - %% We can not shift left with a number which is not a fixnum. We + %% We cannot shift left with a number which is not a fixnum. We %% don't have enough memory. Bits = ?BITS, if Number2 > (Bits bsl 1) -> inf_bsl(Number1, pos_inf); diff --git a/lib/hipe/main/hipe.app.src b/lib/hipe/main/hipe.app.src index 7350e873aa..4684ab49ea 100644 --- a/lib/hipe/main/hipe.app.src +++ b/lib/hipe/main/hipe.app.src @@ -2,7 +2,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2002-2017. All Rights Reserved. +%% Copyright Ericsson AB 2002-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/hipe/rtl/hipe_icode2rtl.erl b/lib/hipe/rtl/hipe_icode2rtl.erl index 6da8a76d34..1ab41f4deb 100644 --- a/lib/hipe/rtl/hipe_icode2rtl.erl +++ b/lib/hipe/rtl/hipe_icode2rtl.erl @@ -215,7 +215,7 @@ gen_enter(I, VarMap, ConstTab) -> {Code1, ConstTab2} = case hipe_icode:enter_type(I) of primop -> - IsGuard = false, % enter can not happen in a guard + IsGuard = false, % enter cannot happen in a guard hipe_rtl_primops:gen_enter_primop({Fun, Args}, IsGuard, ConstTab1); Type -> Call = gen_enter_1(Fun, Args, Type), diff --git a/lib/hipe/rtl/hipe_rtl_arith.inc b/lib/hipe/rtl/hipe_rtl_arith.inc index c05b7aa160..575f10b542 100644 --- a/lib/hipe/rtl/hipe_rtl_arith.inc +++ b/lib/hipe/rtl/hipe_rtl_arith.inc @@ -118,8 +118,8 @@ eval_alu(Op, Arg1, Arg2) -> %% Björn & Bjarni: %% We need to be able to do evaluations based only on the bits, since -%% there are cases where we can evaluate a subset of the bits, but can -%% not do a full eval-alub call (eg. a + 0 gives no carry) +%% there are cases where we can evaluate a subset of the bits, but +%% cannot do a full eval-alub call (eg. a + 0 gives no carry) %% -spec eval_cond_bits(hipe_rtl:alub_cond(), boolean(), boolean(), boolean(), boolean()) -> boolean(). diff --git a/lib/hipe/rtl/hipe_rtl_lcm.erl b/lib/hipe/rtl/hipe_rtl_lcm.erl index af39c9a0a4..2c8cc80e56 100644 --- a/lib/hipe/rtl/hipe_rtl_lcm.erl +++ b/lib/hipe/rtl/hipe_rtl_lcm.erl @@ -267,14 +267,17 @@ try_insert_expr_last(CFG0, Label, Instr) -> %% with the new code inserted second to last (assuming the last expression %% is a branch operation). insert_expr_last_work(_Instr, [#call{}]) -> - %% Call instructions clobber all expressions; we musn't insert the expression - %% before it + %% Call instructions clobber all expressions; we must not insert the + %% expression before it not_safe; insert_expr_last_work(Instr, [Code1]) -> %% We insert the code next to last. [Instr, Code1]; insert_expr_last_work(Instr, [Code|Codes]) -> - [Code|insert_expr_last_work(Instr, Codes)]. + case insert_expr_last_work(Instr, Codes) of + not_safe -> not_safe; + NewCodes -> [Code|NewCodes] + end. %%============================================================================= %% Inserts expression first in the block for the given label. diff --git a/lib/hipe/rtl/hipe_rtl_ssa_const_prop.erl b/lib/hipe/rtl/hipe_rtl_ssa_const_prop.erl index cad43e2df5..72373e536d 100644 --- a/lib/hipe/rtl/hipe_rtl_ssa_const_prop.erl +++ b/lib/hipe/rtl/hipe_rtl_ssa_const_prop.erl @@ -31,11 +31,11 @@ %% %% Some things to note: %% -%% 1. All precoloured registers are assumed to contain bottom. We can not +%% 1. All precoloured registers are assumed to contain bottom. We cannot %% do anything with them since they are not in SSA-form. This might be %% possible to resolve in some way, but we decided to not go there. %% -%% 2. const_labels are assumed to be bottom, we can not find the address +%% 2. const_labels are assumed to be bottom, we cannot find the address %% in any nice way (that I know of, maybe someone can help ?). I %% suppose they don't get a value until linking (or some step that %% resembles it). They are only affecting bignums and floats (at least @@ -579,7 +579,7 @@ visit_multimove(Inst, Env) -> %% Procedure : visit_call/2 %% Purpose : execute a call-instruction. All calls return bottom. We make %% this assumption since the icode-leel have taken care of BIF's -%% and we belive that we are left with the things that can not be +%% and we belive that we are left with the things that cannot be %% done att compile time. %% Arguments : Inst - The instruction %% Env - The environment diff --git a/lib/hipe/test/basic_SUITE_data/basic_issues_hipe.erl b/lib/hipe/test/basic_SUITE_data/basic_issues_hipe.erl index e71045bfe2..fc87abb54e 100644 --- a/lib/hipe/test/basic_SUITE_data/basic_issues_hipe.erl +++ b/lib/hipe/test/basic_SUITE_data/basic_issues_hipe.erl @@ -8,8 +8,9 @@ -export([test/0]). -%% functions that need to be exported so that they are retained. --export([auth/4]). +%% functions that need to be exported so that they are retained and/or +%% not specialized away by the compiler. +-export([auth/4, wxSizer_replace/2, parent_class/1]). test() -> ok = test_dominance_trees(), @@ -18,6 +19,7 @@ test() -> ok = test_bif_fails(), ok = test_find_catches(), ok = test_heap_allocate_trim(), + ok = wxSizer_replace(), ok. %%-------------------------------------------------------------------- @@ -151,3 +153,25 @@ get_next_retry(Error, Count) -> end. pair(A, B) -> {A, B}. + +%%-------------------------------------------------------------------- +%% Date: June 11, 2018 +%% +%% Stripped down test case (from `wxSizer') that crashed the lazy code +%% motion pass of the HiPE compiler in a pre-release of Erlang/OTP 21. +%% A similar crash existed in `ssl_correction'. +%%-------------------------------------------------------------------- + +wxSizer_replace() -> + wxSizer_replace(?MODULE, ?MODULE). + +-define(CLASS(Type, Class), ((Type) =:= Class) orelse (Type):parent_class(Class)). + +wxSizer_replace(OldwinT, NewwinT) -> % this function was the culprit + ?CLASS(OldwinT, ?MODULE), + ?CLASS(NewwinT, ?MODULE), + ok. + +parent_class(wxWindow) -> true; +parent_class(wxEvtHandler) -> true; +parent_class(_Class) -> erlang:error({badtype, ?MODULE}). diff --git a/lib/hipe/vsn.mk b/lib/hipe/vsn.mk index 0c517f9a7a..b081cb0c26 100644 --- a/lib/hipe/vsn.mk +++ b/lib/hipe/vsn.mk @@ -1 +1 @@ -HIPE_VSN = 3.17.1 +HIPE_VSN = 3.18 diff --git a/lib/inets/doc/src/Makefile b/lib/inets/doc/src/Makefile index 90c1258d4a..cbc0e384d8 100644 --- a/lib/inets/doc/src/Makefile +++ b/lib/inets/doc/src/Makefile @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 1997-2015. All Rights Reserved. +# Copyright Ericsson AB 1997-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -115,6 +115,7 @@ pdf: $(TOP_PDF_FILE) html: gifs $(HTML_REF_MAN_FILE) clean clean_docs: clean_html clean_man clean_pdf + rm -rf $(XMLDIR) rm -f errs core *~ man: $(MAN3_FILES) diff --git a/lib/inets/doc/src/http_client.xml b/lib/inets/doc/src/http_client.xml index 15e383ec77..c31a47f4f4 100644 --- a/lib/inets/doc/src/http_client.xml +++ b/lib/inets/doc/src/http_client.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2004</year><year>2016</year> + <year>2004</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/inets/doc/src/http_uri.xml b/lib/inets/doc/src/http_uri.xml index f57214a7ce..2dec5acbf9 100644 --- a/lib/inets/doc/src/http_uri.xml +++ b/lib/inets/doc/src/http_uri.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>2012</year><year>2017</year> + <year>2012</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/inets/doc/src/httpc.xml b/lib/inets/doc/src/httpc.xml index 521ad6a015..a2871f3b95 100644 --- a/lib/inets/doc/src/httpc.xml +++ b/lib/inets/doc/src/httpc.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>2004</year><year>2017</year> + <year>2004</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/inets/doc/src/httpd.xml b/lib/inets/doc/src/httpd.xml index edf8731a82..e5adb4eb20 100644 --- a/lib/inets/doc/src/httpd.xml +++ b/lib/inets/doc/src/httpd.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1997</year><year>2016</year> + <year>1997</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -173,7 +173,7 @@ <item> <p>For <c>ip_comm</c> configuration options, see <seealso marker="kernel:gen_tcp#listen-2">gen_tcp:listen/2</seealso>, some options - that are used internally by httpd can not be set.</p> + that are used internally by httpd cannot be set.</p> <p>For <c>SSL</c> configuration options, see <seealso marker="ssl:ssl#listen-2">ssl:listen/2</seealso>.</p> <p>Default is <c>ip_comm</c>.</p> diff --git a/lib/inets/doc/src/inets.xml b/lib/inets/doc/src/inets.xml index eb4e51584f..9b0ffaad5e 100644 --- a/lib/inets/doc/src/inets.xml +++ b/lib/inets/doc/src/inets.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>2007</year><year>2016</year> + <year>2007</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/inets/doc/src/mod_esi.xml b/lib/inets/doc/src/mod_esi.xml index d024c8afa8..ede7dc8f7d 100644 --- a/lib/inets/doc/src/mod_esi.xml +++ b/lib/inets/doc/src/mod_esi.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1997</year><year>2016</year> + <year>1997</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/inets/doc/src/notes.xml b/lib/inets/doc/src/notes.xml index 1e22809b3a..b2776846c8 100644 --- a/lib/inets/doc/src/notes.xml +++ b/lib/inets/doc/src/notes.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2002</year><year>2017</year> + <year>2002</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -33,7 +33,100 @@ <file>notes.xml</file> </header> - <section><title>Inets 6.5.2</title> + <section><title>Inets 7.0</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fixed HTTP content injection bug in httpc (ERL-456).</p> + <p> + Own Id: OTP-14726</p> + </item> + <item> + <p> + Fixed support for URI-references in HTTP 'Location' + header (ERL-333).</p> + <p> + Own Id: OTP-14729</p> + </item> + <item> + <p> + Fix broken 'Content-Type' handling in httpc (ERL-536).</p> + <p> + Own Id: OTP-15006</p> + </item> + <item> + <p> + Fix handling of relative paths in the script_alias + property of httpd (ERL-574).</p> + <p> + Own Id: OTP-15021</p> + </item> + <item> + <p> + Fix httpd:reload_config/2 with path() as the first + argument (ERL-578).</p> + <p> + Own Id: OTP-15025</p> + </item> + <item> + <p> + Improved gracefulness.</p> + <p> + Own Id: OTP-15042</p> + </item> + </list> + </section> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Split inets and create separate ftp and tftp apps.</p> + <p> + Own Id: OTP-14113</p> + </item> + </list> + </section> + + </section> + +<section><title>Inets 6.5.2.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Enhance error handling, that is mod_get will return 403 + if a path is a directory and not a file.</p> + <p> + Own Id: OTP-15192</p> + </item> + </list> + </section> + +</section> + + <section><title>Inets 6.5.2.1</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Options added for setting low-level properties on the + underlying TCP connections. The options are: + <c>sock_ctrl</c>, <c>sock_data_act</c> and + <c>sock_data_pass</c>. See the manual for details.</p> + <p> + Own Id: OTP-15120 Aux Id: ERIERL-192 </p> + </item> + </list> + </section> + + </section> + +<section><title>Inets 6.5.2</title> <section><title>Fixed Bugs and Malfunctions</title> <list> @@ -1089,7 +1182,7 @@ Add option {ftp_extension, boolean} to enable use of extended commands EPSV and EPRT, as specified in RFC 2428, for IPv4 instead of using the legacy commands. Ipv6 - can not be supported without the extended commands.</p> + cannot be supported without the extended commands.</p> <p> Own Id: OTP-12255</p> </item> diff --git a/lib/inets/doc/src/notes_history.xml b/lib/inets/doc/src/notes_history.xml index c12899e614..1523827db9 100644 --- a/lib/inets/doc/src/notes_history.xml +++ b/lib/inets/doc/src/notes_history.xml @@ -1149,8 +1149,8 @@ <list type="bulleted"> <item> <p>When further testing the functionality of https requests - that goes through a proxy. We realised that alas this can - not currently be supported as it requires features from + that goes through a proxy. We realised that alas this + cannot currently be supported as it requires features from the ssl implementation that is not currently available. So for now an error message will be returned when trying to use this functionality.</p> diff --git a/lib/inets/src/http_client/httpc.erl b/lib/inets/src/http_client/httpc.erl index a73503a5ce..24a205ced9 100644 --- a/lib/inets/src/http_client/httpc.erl +++ b/lib/inets/src/http_client/httpc.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009-2017. All Rights Reserved. +%% Copyright Ericsson AB 2009-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/inets/src/http_client/httpc_cookie.erl b/lib/inets/src/http_client/httpc_cookie.erl index cbf428ab3e..2e647a1438 100644 --- a/lib/inets/src/http_client/httpc_cookie.erl +++ b/lib/inets/src/http_client/httpc_cookie.erl @@ -271,7 +271,7 @@ lookup_cookies(CookieDb, Host, Path) -> lookup_domain_cookies(_CookieDb, [], AccCookies) -> lists:flatten(AccCookies); -%% Top domains can not have cookies +%% Top domains cannot have cookies lookup_domain_cookies(_CookieDb, [_], AccCookies) -> lists:flatten(AccCookies); diff --git a/lib/inets/src/http_client/httpc_handler.erl b/lib/inets/src/http_client/httpc_handler.erl index c9763507d1..5e05b8170a 100644 --- a/lib/inets/src/http_client/httpc_handler.erl +++ b/lib/inets/src/http_client/httpc_handler.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2002-2017. All Rights Reserved. +%% Copyright Ericsson AB 2002-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/inets/src/http_client/httpc_internal.hrl b/lib/inets/src/http_client/httpc_internal.hrl index c5fe439722..a14d8aa71b 100644 --- a/lib/inets/src/http_client/httpc_internal.hrl +++ b/lib/inets/src/http_client/httpc_internal.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2016. All Rights Reserved. +%% Copyright Ericsson AB 2005-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/inets/src/http_client/httpc_manager.erl b/lib/inets/src/http_client/httpc_manager.erl index c3404dbb37..0333442bf2 100644 --- a/lib/inets/src/http_client/httpc_manager.erl +++ b/lib/inets/src/http_client/httpc_manager.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2002-2016. All Rights Reserved. +%% Copyright Ericsson AB 2002-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/inets/src/http_client/httpc_request.erl b/lib/inets/src/http_client/httpc_request.erl index 641b6559de..9b81bd7a80 100644 --- a/lib/inets/src/http_client/httpc_request.erl +++ b/lib/inets/src/http_client/httpc_request.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2016. All Rights Reserved. +%% Copyright Ericsson AB 2004-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/inets/src/http_client/httpc_response.erl b/lib/inets/src/http_client/httpc_response.erl index 92dc9b0e02..bb6b76da89 100644 --- a/lib/inets/src/http_client/httpc_response.erl +++ b/lib/inets/src/http_client/httpc_response.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2017. All Rights Reserved. +%% Copyright Ericsson AB 2004-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -398,7 +398,7 @@ redirect(Response = {_, Headers, _}, Request) -> THost = http_util:maybe_add_brackets(maps:get(host, URIMap), Brackets), TPort = maps:get(port, URIMap), TPath = maps:get(path, URIMap), - TQuery = maps:get(query, URIMap, ""), + TQuery = add_question_mark(maps:get(query, URIMap, "")), NewURI = uri_string:normalize( uri_string:recompose(URIMap)), HostPort = http_request:normalize_host(TScheme, THost, TPort), @@ -417,29 +417,38 @@ redirect(Response = {_, Headers, _}, Request) -> end end. +add_question_mark(<<>>) -> + <<>>; +add_question_mark([]) -> + []; +add_question_mark(Comp) when is_binary(Comp) -> + <<$?, Comp/binary>>; +add_question_mark(Comp) when is_list(Comp) -> + [$?|Comp]. %% RFC3986 - 5.2.2. Transform References resolve_uri(Scheme, Host, Port, Path, Query, URI) -> resolve_uri(Scheme, Host, Port, Path, Query, URI, #{}). %% resolve_uri(Scheme, Host, Port, Path, Query, URI, Map0) -> - case maps:is_key(scheme, URI) of - true -> - Port = get_port(URI), + case maps:get(scheme, URI, undefined) of + undefined -> + Port0 = get_port(Scheme, URI), + Map = Map0#{scheme => Scheme, + port => Port0}, + resolve_authority(Host, Port, Path, Query, URI, Map); + URIScheme -> + Port0 = get_port(URIScheme, URI), maybe_add_query( - Map0#{scheme => maps:get(scheme, URI), - host => maps:get(host, URI), - port => Port, - path => maps:get(path, URI)}, - URI); - false -> - Map = Map0#{scheme => Scheme}, - resolve_authority(Host, Port, Path, Query, URI, Map) + Map0#{scheme => URIScheme, + host => maps:get(host, URI), + port => Port0, + path => maps:get(path, URI)}, + URI) end. -get_port(URI) -> - Scheme = maps:get(scheme, URI), +get_port(Scheme, URI) -> case maps:get(port, URI, undefined) of undefined -> get_default_port(Scheme); @@ -457,15 +466,13 @@ get_default_port("https") -> resolve_authority(Host, Port, Path, Query, RelURI, Map) -> case maps:is_key(host, RelURI) of true -> - Port = get_port(RelURI), maybe_add_query( Map#{host => maps:get(host, RelURI), - port => Port, path => maps:get(path, RelURI)}, RelURI); false -> Map1 = Map#{host => Host, - port => Port}, + port => Port}, resolve_path(Path, Query, RelURI, Map1) end. diff --git a/lib/inets/src/http_lib/http_request.erl b/lib/inets/src/http_lib/http_request.erl index 8ca1542164..2b1a0bd40f 100644 --- a/lib/inets/src/http_lib/http_request.erl +++ b/lib/inets/src/http_lib/http_request.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2017. All Rights Reserved. +%% Copyright Ericsson AB 2005-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/inets/src/http_lib/http_uri.erl b/lib/inets/src/http_lib/http_uri.erl index bc588fd390..6805b0293d 100644 --- a/lib/inets/src/http_lib/http_uri.erl +++ b/lib/inets/src/http_lib/http_uri.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2006-2017. All Rights Reserved. +%% Copyright Ericsson AB 2006-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/inets/src/http_server/httpd.erl b/lib/inets/src/http_server/httpd.erl index 1eaa1c930a..f4b53ce129 100644 --- a/lib/inets/src/http_server/httpd.erl +++ b/lib/inets/src/http_server/httpd.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2016. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/inets/src/http_server/httpd_esi.erl b/lib/inets/src/http_server/httpd_esi.erl index f5493f6fad..3b66b86348 100644 --- a/lib/inets/src/http_server/httpd_esi.erl +++ b/lib/inets/src/http_server/httpd_esi.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2016. All Rights Reserved. +%% Copyright Ericsson AB 2005-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/inets/src/http_server/httpd_example.erl b/lib/inets/src/http_server/httpd_example.erl index 47a8c48d01..52f5fa03a9 100644 --- a/lib/inets/src/http_server/httpd_example.erl +++ b/lib/inets/src/http_server/httpd_example.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2016. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/inets/src/http_server/httpd_file.erl b/lib/inets/src/http_server/httpd_file.erl index 4d419172d0..fb71834e95 100644 --- a/lib/inets/src/http_server/httpd_file.erl +++ b/lib/inets/src/http_server/httpd_file.erl @@ -33,6 +33,9 @@ handle_error(enoent, Op, ModData, Path) -> handle_error(enotdir, Op, ModData, Path) -> handle_error(404, Op, ModData, Path, ": A component of the file name is not a directory"); +handle_error(eisdir, Op, ModData, Path) -> + handle_error(403, Op, ModData, Path, + ":Ilegal operation expected a file not a directory"); handle_error(emfile, Op, _ModData, Path) -> handle_error(500, Op, none, Path, ": Too many open files"); handle_error({enfile,_}, Op, _ModData, Path) -> diff --git a/lib/inets/src/http_server/httpd_request.erl b/lib/inets/src/http_server/httpd_request.erl index e513eb8a3a..9d7538a13d 100644 --- a/lib/inets/src/http_server/httpd_request.erl +++ b/lib/inets/src/http_server/httpd_request.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2015. All Rights Reserved. +%% Copyright Ericsson AB 2005-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/inets/src/http_server/httpd_response.erl b/lib/inets/src/http_server/httpd_response.erl index 57ce162922..3ee8665a54 100644 --- a/lib/inets/src/http_server/httpd_response.erl +++ b/lib/inets/src/http_server/httpd_response.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2016. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/inets/src/http_server/httpd_script_env.erl b/lib/inets/src/http_server/httpd_script_env.erl index d7c92c59ef..055edca211 100644 --- a/lib/inets/src/http_server/httpd_script_env.erl +++ b/lib/inets/src/http_server/httpd_script_env.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2016. All Rights Reserved. +%% Copyright Ericsson AB 2005-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/inets/src/http_server/mod_alias.erl b/lib/inets/src/http_server/mod_alias.erl index 8f0b92710e..68a3de0229 100644 --- a/lib/inets/src/http_server/mod_alias.erl +++ b/lib/inets/src/http_server/mod_alias.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2016. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/inets/src/http_server/mod_disk_log.erl b/lib/inets/src/http_server/mod_disk_log.erl index 2023546f01..190cf91416 100644 --- a/lib/inets/src/http_server/mod_disk_log.erl +++ b/lib/inets/src/http_server/mod_disk_log.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2016. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/inets/src/http_server/mod_log.erl b/lib/inets/src/http_server/mod_log.erl index ec570504be..0323918578 100644 --- a/lib/inets/src/http_server/mod_log.erl +++ b/lib/inets/src/http_server/mod_log.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2016. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/inets/src/inets_app/Makefile b/lib/inets/src/inets_app/Makefile index fad2fefe2f..ec1ae70305 100644 --- a/lib/inets/src/inets_app/Makefile +++ b/lib/inets/src/inets_app/Makefile @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 2005-2016. All Rights Reserved. +# Copyright Ericsson AB 2005-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. diff --git a/lib/inets/src/inets_app/inets.app.src b/lib/inets/src/inets_app/inets.app.src index 5b34018def..e5780b2a4c 100644 --- a/lib/inets/src/inets_app/inets.app.src +++ b/lib/inets/src/inets_app/inets.app.src @@ -1,7 +1,7 @@ %% This is an -*- erlang -*- file. %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2016. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/inets/src/inets_app/inets.appup.src b/lib/inets/src/inets_app/inets.appup.src index a86413147c..0dcf66265e 100644 --- a/lib/inets/src/inets_app/inets.appup.src +++ b/lib/inets/src/inets_app/inets.appup.src @@ -1,7 +1,7 @@ %% -*- erlang -*- %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2017. All Rights Reserved. +%% Copyright Ericsson AB 1999-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/inets/src/inets_app/inets.erl b/lib/inets/src/inets_app/inets.erl index 450adf1a02..ab2bc5b784 100644 --- a/lib/inets/src/inets_app/inets.erl +++ b/lib/inets/src/inets_app/inets.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2006-2016. All Rights Reserved. +%% Copyright Ericsson AB 2006-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/inets/src/inets_app/inets_internal.hrl b/lib/inets/src/inets_app/inets_internal.hrl index e0f59bba5f..38b7e68638 100644 --- a/lib/inets/src/inets_app/inets_internal.hrl +++ b/lib/inets/src/inets_app/inets_internal.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2016. All Rights Reserved. +%% Copyright Ericsson AB 2005-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/inets/src/inets_app/inets_sup.erl b/lib/inets/src/inets_app/inets_sup.erl index 22c928f9f9..0d3c7475fc 100644 --- a/lib/inets/src/inets_app/inets_sup.erl +++ b/lib/inets/src/inets_app/inets_sup.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2016. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/inets/test/Makefile b/lib/inets/test/Makefile index 0e33b72095..6ab9771a8f 100644 --- a/lib/inets/test/Makefile +++ b/lib/inets/test/Makefile @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 1997-2016. All Rights Reserved. +# Copyright Ericsson AB 1997-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. diff --git a/lib/inets/test/http_format_SUITE.erl b/lib/inets/test/http_format_SUITE.erl index 647eff4f7c..0a5aed67d5 100644 --- a/lib/inets/test/http_format_SUITE.erl +++ b/lib/inets/test/http_format_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2016. All Rights Reserved. +%% Copyright Ericsson AB 2004-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -435,7 +435,7 @@ http_request(Config) when is_list(Config) -> [<<>>, Length1], HttpBody1)). %%------------------------------------------------------------------------- validate_request_line() -> - [{doc, "Test httpd_request:validate/3. Makes sure you can not get past" + [{doc, "Test httpd_request:validate/3. Makes sure you cannot get past" " the server_root and that the request is recognized by the server" " and protcol version."}]. validate_request_line(Config) when is_list(Config) -> diff --git a/lib/inets/test/httpc_SUITE.erl b/lib/inets/test/httpc_SUITE.erl index 84d07bc97f..6e048a4d56 100644 --- a/lib/inets/test/httpc_SUITE.erl +++ b/lib/inets/test/httpc_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2017. All Rights Reserved. +%% Copyright Ericsson AB 2004-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -59,7 +59,8 @@ all() -> {group, http_unix_socket}, {group, https}, {group, sim_https}, - {group, misc} + {group, misc}, + {group, sim_mixed} % HTTP and HTTPS sim servers ]. groups() -> @@ -74,7 +75,8 @@ groups() -> {http_unix_socket, [], simulated_unix_socket()}, {https, [], real_requests()}, {sim_https, [], only_simulated()}, - {misc, [], misc()} + {misc, [], misc()}, + {sim_mixed, [], sim_mixed()} ]. real_requests()-> @@ -170,6 +172,12 @@ misc() -> wait_for_whole_response ]. +sim_mixed() -> + [ + redirect_http_to_https, + redirect_relative_different_port + ]. + %%-------------------------------------------------------------------- init_per_suite(Config) -> @@ -195,7 +203,8 @@ init_per_group(misc = Group, Config) -> Config; -init_per_group(Group, Config0) when Group =:= sim_https; Group =:= https-> +init_per_group(Group, Config0) when Group =:= sim_https; Group =:= https; + Group =:= sim_mixed -> catch crypto:stop(), try crypto:start() of ok -> @@ -238,6 +247,13 @@ end_per_group(http_unix_socket,_Config) -> end_per_group(_, _Config) -> ok. +do_init_per_group(Group=sim_mixed, Config0) -> + % The mixed group uses two server ports (http and https), so we use + % different config names here. + Config1 = init_ssl(Config0), + Config2 = proplists:delete(http_port, proplists:delete(https_port, Config1)), + {HttpPort, HttpsPort} = server_start(Group, server_config(sim_https, Config2)), + [{http_port, HttpPort} | [{https_port, HttpsPort} | Config2]]; do_init_per_group(Group, Config0) -> Config1 = case Group of @@ -734,6 +750,48 @@ redirect_loop(Config) when is_list(Config) -> = httpc:request(get, {URL, []}, [], []). %%------------------------------------------------------------------------- +redirect_http_to_https() -> + [{doc, "Test that a 30X redirect from one scheme to another is handled " + "correctly."}]. +redirect_http_to_https(Config) when is_list(Config) -> + URL301 = mixed_url(http, "/301_custom_url.html", Config), + TargetUrl = mixed_url(https, "/dummy.html", Config), + Headers = [{"x-test-301-url", TargetUrl}], + + {ok, {{_,200,_}, [_ | _], [_|_]}} + = httpc:request(get, {URL301, Headers}, [], []), + + {ok, {{_,200,_}, [_ | _], []}} + = httpc:request(head, {URL301, Headers}, [], []), + + {ok, {{_,200,_}, [_ | _], [_|_]}} + = httpc:request(post, {URL301, Headers, "text/plain", "foobar"}, + [], []). +%%------------------------------------------------------------------------- +redirect_relative_different_port() -> + [{doc, "Test that a 30X redirect with a relative target, but different " + "port, is handled correctly."}]. +redirect_relative_different_port(Config) when is_list(Config) -> + URL301 = mixed_url(http, "/301_custom_url.html", Config), + + % We need an extra server of the same protocol here, so spawn a new + % HTTP-protocol one + Port = server_start(sim_http, []), + {ok, Host} = inet:gethostname(), + % Prefix the URI with '/' instead of a scheme + TargetUrl = "//" ++ Host ++ ":" ++ integer_to_list(Port) ++ "/dummy.html", + Headers = [{"x-test-301-url", TargetUrl}], + + {ok, {{_,200,_}, [_ | _], [_|_]}} + = httpc:request(get, {URL301, Headers}, [], []), + + {ok, {{_,200,_}, [_ | _], []}} + = httpc:request(head, {URL301, Headers}, [], []), + + {ok, {{_,200,_}, [_ | _], [_|_]}} + = httpc:request(post, {URL301, Headers, "text/plain", "foobar"}, + [], []). +%%------------------------------------------------------------------------- cookie() -> [{doc, "Test cookies."}]. cookie(Config) when is_list(Config) -> @@ -1559,6 +1617,21 @@ url(sim_http, UserInfo, End, Config) -> url(sim_https, UserInfo, End, Config) -> url(https, UserInfo, End, Config). +% Only for use in the `mixed` test group, where both http and https +% URLs are possible. +mixed_url(http, End, Config) -> + mixed_url(http_port, End, Config); +mixed_url(https, End, Config) -> + mixed_url(https_port, End, Config); +mixed_url(PortType, End, Config) -> + Port = proplists:get_value(PortType, Config), + {ok, Host} = inet:gethostname(), + Start = case PortType of + http_port -> ?URL_START; + https_port -> ?TLS_URL_START + end, + Start ++ Host ++ ":" ++ integer_to_list(Port) ++ End. + group_name(Config) -> GroupProp = proplists:get_value(tc_group_properties, Config), proplists:get_value(name, GroupProp). @@ -1587,6 +1660,9 @@ server_start(http_ipv6, HttpdConfig) -> Serv = inets:services_info(), {value, {_, _, Info}} = lists:keysearch(Pid, 2, Serv), proplists:get_value(port, Info); +server_start(sim_mixed, Config) -> + % For the mixed http/https case, we start two servers and return both ports. + {server_start(sim_http, []), server_start(sim_https, Config)}; server_start(_, HttpdConfig) -> {ok, Pid} = inets:start(httpd, HttpdConfig), Serv = inets:services_info(), @@ -1645,6 +1721,8 @@ start_apps(https) -> inets_test_lib:start_apps([crypto, public_key, ssl]); start_apps(sim_https) -> inets_test_lib:start_apps([crypto, public_key, ssl]); +start_apps(sim_mixed) -> + inets_test_lib:start_apps([crypto, public_key, ssl]); start_apps(_) -> ok. @@ -2089,6 +2167,20 @@ handle_uri(_,"/301_rel_uri.html",_,_,_,_) -> "Content-Length:" ++ integer_to_list(length(Body)) ++ "\r\n\r\n" ++ Body; +handle_uri("HEAD","/301_custom_url.html",_,Headers,_,_) -> + NewUri = proplists:get_value("x-test-301-url", Headers), + "HTTP/1.1 301 Moved Permanently\r\n" ++ + "Location:" ++ NewUri ++ "\r\n" ++ + "Content-Length:0\r\n\r\n"; + +handle_uri(_,"/301_custom_url.html",_,Headers,_,_) -> + NewUri = proplists:get_value("x-test-301-url", Headers), + Body = "<HTML><BODY><a href=" ++ NewUri ++ + ">New place</a></BODY></HTML>", + "HTTP/1.1 301 Moved Permanently\r\n" ++ + "Location:" ++ NewUri ++ "\r\n" ++ + "Content-Length:" ++ integer_to_list(length(Body)) + ++ "\r\n\r\n" ++ Body; handle_uri("HEAD","/302.html",Port,_,Socket,_) -> NewUri = url_start(Socket) ++ diff --git a/lib/inets/test/httpc_proxy_SUITE.erl b/lib/inets/test/httpc_proxy_SUITE.erl index 198b245399..3ee7981660 100644 --- a/lib/inets/test/httpc_proxy_SUITE.erl +++ b/lib/inets/test/httpc_proxy_SUITE.erl @@ -534,7 +534,7 @@ init_local_proxy(Config) -> ct:fail({local_proxy_start_failed,Error}) end; _ -> - {skip,"Platform can not run local proxy start script"} + {skip,"Platform cannot run local proxy start script"} end. init_local_proxy_string(String, Config) -> diff --git a/lib/inets/test/httpd_SUITE.erl b/lib/inets/test/httpd_SUITE.erl index 251547dcb0..dc4f607750 100644 --- a/lib/inets/test/httpd_SUITE.erl +++ b/lib/inets/test/httpd_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2013-2017. All Rights Reserved. +%% Copyright Ericsson AB 2013-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -448,8 +448,19 @@ get(Config) when is_list(Config) -> {header, "Content-Type", "text/html"}, {header, "Date"}, {header, "Server"}, + {version, Version}]), + + ok = httpd_test_lib:verify_request(proplists:get_value(type, Config), Host, + proplists:get_value(port, Config), + transport_opts(Type, Config), + proplists:get_value(node, Config), + http_request("GET /open/ ", Version, Host), + [{statuscode, 403}, + {header, "Content-Type", "text/html"}, + {header, "Date"}, + {header, "Server"}, {version, Version}]). - + basic_auth_1_1(Config) when is_list(Config) -> basic_auth([{http_version, "HTTP/1.1"} | Config]). @@ -1992,7 +2003,7 @@ head_status(_) -> basic_conf() -> [{modules, [mod_alias, mod_range, mod_responsecontrol, - mod_trace, mod_esi, mod_cgi, mod_dir, mod_get, mod_head]}]. + mod_trace, mod_esi, mod_cgi, mod_get, mod_head]}]. auth_access_conf() -> [{modules, [mod_alias, mod_htaccess, mod_dir, mod_get, mod_head]}, diff --git a/lib/inets/test/httpd_bench_SUITE.erl b/lib/inets/test/httpd_bench_SUITE.erl index 9d8cbf9ae2..4b549dcb5b 100644 --- a/lib/inets/test/httpd_bench_SUITE.erl +++ b/lib/inets/test/httpd_bench_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2012. All Rights Reserved. +%% Copyright Ericsson AB 2012-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/inets/test/httpd_mod.erl b/lib/inets/test/httpd_mod.erl index 2035b50248..6e3635001a 100644 --- a/lib/inets/test/httpd_mod.erl +++ b/lib/inets/test/httpd_mod.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2016. All Rights Reserved. +%% Copyright Ericsson AB 2005-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/inets/test/inets_SUITE.erl b/lib/inets/test/inets_SUITE.erl index 07ce594a1d..e7964ff7f1 100644 --- a/lib/inets/test/inets_SUITE.erl +++ b/lib/inets/test/inets_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2016. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/inets/test/inets_socketwrap_SUITE.erl b/lib/inets/test/inets_socketwrap_SUITE.erl index fc87c595a9..b88cff4e90 100644 --- a/lib/inets/test/inets_socketwrap_SUITE.erl +++ b/lib/inets/test/inets_socketwrap_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2016. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/inets/test/inets_sup_SUITE.erl b/lib/inets/test/inets_sup_SUITE.erl index 727e91e987..b90d55dadf 100644 --- a/lib/inets/test/inets_sup_SUITE.erl +++ b/lib/inets/test/inets_sup_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2016. All Rights Reserved. +%% Copyright Ericsson AB 2004-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/inets/test/inets_test_lib.erl b/lib/inets/test/inets_test_lib.erl index d1f238ba0e..1cc4e11e45 100644 --- a/lib/inets/test/inets_test_lib.erl +++ b/lib/inets/test/inets_test_lib.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2001-2015. All Rights Reserved. +%% Copyright Ericsson AB 2001-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/inets/test/uri_SUITE.erl b/lib/inets/test/uri_SUITE.erl index 8e00e6f565..db6a8bdfc3 100644 --- a/lib/inets/test/uri_SUITE.erl +++ b/lib/inets/test/uri_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2017. All Rights Reserved. +%% Copyright Ericsson AB 2004-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/inets/vsn.mk b/lib/inets/vsn.mk index 9bbcd06914..b76390ad66 100644 --- a/lib/inets/vsn.mk +++ b/lib/inets/vsn.mk @@ -2,7 +2,7 @@ # %CopyrightBegin% # -# Copyright Ericsson AB 2001-2017. All Rights Reserved. +# Copyright Ericsson AB 2001-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -19,6 +19,6 @@ # %CopyrightEnd% APPLICATION = inets -INETS_VSN = 6.5.2 +INETS_VSN = 7.0 PRE_VSN = APP_VSN = "$(APPLICATION)-$(INETS_VSN)$(PRE_VSN)" diff --git a/lib/jinterface/doc/src/Makefile b/lib/jinterface/doc/src/Makefile index 37de0a35c5..6f1ecc8dea 100644 --- a/lib/jinterface/doc/src/Makefile +++ b/lib/jinterface/doc/src/Makefile @@ -129,6 +129,7 @@ html: gifs $(HTML_REF_MAN_FILE) clean clean_docs: rm -rf $(HTMLDIR)/* + rm -rf $(XMLDIR) rm -f $(MAN3DIR)/* rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) rm -f errs core *~ diff --git a/lib/jinterface/doc/src/notes.xml b/lib/jinterface/doc/src/notes.xml index 346d467c2d..640712ad98 100644 --- a/lib/jinterface/doc/src/notes.xml +++ b/lib/jinterface/doc/src/notes.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2000</year><year>2017</year> + <year>2000</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -31,6 +31,21 @@ </header> <p>This document describes the changes made to the Jinterface application.</p> +<section><title>Jinterface 1.9</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Add module package name for Java 9</p> + <p> + Own Id: OTP-14844</p> + </item> + </list> + </section> + +</section> + <section><title>Jinterface 1.8.1</title> <section><title>Fixed Bugs and Malfunctions</title> @@ -690,7 +705,7 @@ <item> <p><c>OtpMbox.receive()</c> and <c>OtpMbox.receive(long timeout)</c> can now throw <c>OtpErlangDecodeException</c> if the received message - can not be decoded. <c>null</c> is now only returned from + cannot be decoded. <c>null</c> is now only returned from <c>OtpMbox.receive(long timeout)</c> if a timeout occurs. <c>OtpMbox.receive()</c> will never return <c>null</c>.</p> <p>*** POTENTIAL INCOMPATIBILITY ***</p> diff --git a/lib/jinterface/java_src/com/ericsson/otp/erlang/Makefile b/lib/jinterface/java_src/com/ericsson/otp/erlang/Makefile index 001acfdd2e..ee616f3d7e 100644 --- a/lib/jinterface/java_src/com/ericsson/otp/erlang/Makefile +++ b/lib/jinterface/java_src/com/ericsson/otp/erlang/Makefile @@ -3,7 +3,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 2000-2016. All Rights Reserved. +# Copyright Ericsson AB 2000-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. diff --git a/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpEpmd.java b/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpEpmd.java index 363fdb950a..fffb8475d3 100644 --- a/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpEpmd.java +++ b/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpEpmd.java @@ -102,7 +102,7 @@ public class OtpEpmd { /** * Set the port number to be used to contact the epmd process. Only needed * when the default port is not desired and system environment variable - * ERL_EPMD_PORT can not be read (applet). + * ERL_EPMD_PORT cannot be read (applet). */ public static void useEpmdPort(final int port) { EpmdPort.set(port); diff --git a/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpInputStream.java b/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpInputStream.java index ded8f6e1e5..6d81ce630b 100644 --- a/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpInputStream.java +++ b/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpInputStream.java @@ -601,7 +601,7 @@ public class OtpInputStream extends ByteArrayInputStream { * @return the integer value. * * @exception OtpErlangDecodeException - * if the next term in the stream can not be represented as a + * if the next term in the stream cannot be represented as a * positive integer. */ public int read_uint() throws OtpErlangDecodeException { @@ -622,7 +622,7 @@ public class OtpInputStream extends ByteArrayInputStream { * @return the integer value. * * @exception OtpErlangDecodeException - * if the next term in the stream can not be represented as + * if the next term in the stream cannot be represented as * an integer. */ public int read_int() throws OtpErlangDecodeException { @@ -643,7 +643,7 @@ public class OtpInputStream extends ByteArrayInputStream { * @return the short value. * * @exception OtpErlangDecodeException - * if the next term in the stream can not be represented as a + * if the next term in the stream cannot be represented as a * positive short. */ public short read_ushort() throws OtpErlangDecodeException { @@ -664,7 +664,7 @@ public class OtpInputStream extends ByteArrayInputStream { * @return the short value. * * @exception OtpErlangDecodeException - * if the next term in the stream can not be represented as a + * if the next term in the stream cannot be represented as a * short. */ public short read_short() throws OtpErlangDecodeException { @@ -685,7 +685,7 @@ public class OtpInputStream extends ByteArrayInputStream { * @return the long value. * * @exception OtpErlangDecodeException - * if the next term in the stream can not be represented as a + * if the next term in the stream cannot be represented as a * positive long. */ public long read_ulong() throws OtpErlangDecodeException { @@ -698,7 +698,7 @@ public class OtpInputStream extends ByteArrayInputStream { * @return the long value. * * @exception OtpErlangDecodeException - * if the next term in the stream can not be represented as a + * if the next term in the stream cannot be represented as a * long. */ public long read_long() throws OtpErlangDecodeException { diff --git a/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpMbox.java b/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpMbox.java index 42e178c3f6..29a8bc1540 100644 --- a/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpMbox.java +++ b/lib/jinterface/java_src/com/ericsson/otp/erlang/OtpMbox.java @@ -156,7 +156,7 @@ public class OtpMbox { * of the next message waiting in this mailbox. * * @exception OtpErlangDecodeException - * if the message can not be decoded. + * if the message cannot be decoded. * * @exception OtpErlangExit * if a linked {@link OtpErlangPid pid} has exited or has @@ -184,7 +184,7 @@ public class OtpMbox { * of the next message waiting in this mailbox. * * @exception OtpErlangDecodeException - * if the message can not be decoded. + * if the message cannot be decoded. * * @exception OtpErlangExit * if a linked {@link OtpErlangPid pid} has exited or has diff --git a/lib/jinterface/test/jinterface_SUITE.erl b/lib/jinterface/test/jinterface_SUITE.erl index 8c6a6368a9..468981a557 100644 --- a/lib/jinterface/test/jinterface_SUITE.erl +++ b/lib/jinterface/test/jinterface_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2016. All Rights Reserved. +%% Copyright Ericsson AB 2004-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/jinterface/test/nc_SUITE.erl b/lib/jinterface/test/nc_SUITE.erl index daa17195ee..4f225a396e 100644 --- a/lib/jinterface/test/nc_SUITE.erl +++ b/lib/jinterface/test/nc_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2016. All Rights Reserved. +%% Copyright Ericsson AB 2004-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/jinterface/vsn.mk b/lib/jinterface/vsn.mk index 0a8a1190ec..f527a83092 100644 --- a/lib/jinterface/vsn.mk +++ b/lib/jinterface/vsn.mk @@ -1 +1 @@ -JINTERFACE_VSN = 1.8.1 +JINTERFACE_VSN = 1.9 diff --git a/lib/kernel/doc/src/.gitignore b/lib/kernel/doc/src/.gitignore new file mode 100644 index 0000000000..c2813ac866 --- /dev/null +++ b/lib/kernel/doc/src/.gitignore @@ -0,0 +1 @@ +*.eps
\ No newline at end of file diff --git a/lib/kernel/doc/src/Makefile b/lib/kernel/doc/src/Makefile index 29dc73a523..f8867ccf25 100644 --- a/lib/kernel/doc/src/Makefile +++ b/lib/kernel/doc/src/Makefile @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 1997-2016. All Rights Reserved. +# Copyright Ericsson AB 1997-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -84,6 +84,7 @@ XML_CHAPTER_FILES = \ BOOK_FILES = book.xml +# The .png file is generated from a .dia file with target 'update_png' IMAGE_FILES = \ logger_arch.png @@ -112,6 +113,17 @@ SPECS_FILES = $(XML_REF3_FILES:%.xml=$(SPECDIR)/specs_%.xml) TOP_SPECS_FILE = specs.xml # ---------------------------------------------------- +# FIGURES +# ---------------------------------------------------- +# In order to update the figures you have to have both dia +# and imagemagick installed. +# The generated .png file must be committed. + +update_png: + dia --export=logger_arch.eps logger_arch.dia + convert logger_arch.eps -resize 65% logger_arch.png + +# ---------------------------------------------------- # FLAGS # ---------------------------------------------------- XML_FLAGS += @@ -142,12 +154,13 @@ debug opt: clean clean_docs: rm -rf $(HTMLDIR)/* + rm -rf $(XMLDIR) rm -f $(MAN3DIR)/* rm -f $(MAN4DIR)/* rm -f $(MAN6DIR)/* rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) rm -f $(SPECDIR)/* - rm -f errs core *~ + rm -f errs core *~ *.eps $(SPECDIR)/specs_erl_prim_loader_stub.xml: $(gen_verbose)escript $(SPECS_EXTRACTOR) $(SPECS_FLAGS) \ diff --git a/lib/kernel/doc/src/application.xml b/lib/kernel/doc/src/application.xml index 886286b76d..38c7b5acf1 100644 --- a/lib/kernel/doc/src/application.xml +++ b/lib/kernel/doc/src/application.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1996</year><year>2016</year> + <year>1996</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -318,8 +318,13 @@ Nodes = [cp1@cave, {cp2@cave, cp3@cave}]</code> <c>{error,{not_started,App}}</c> is returned, where <c>App</c> is the name of the missing application.</p> <p>The application controller then creates an <em>application master</em> - for the application. The application master is - the group leader of all the processes in the application. + for the application. The application master becomes the + group leader of all the processes in the application. I/O is + forwarded to the previous group leader, though, this is just + a way to identify processes that belong to the application. + Used for example to find itself from any process, or, + reciprocally, to kill them all when it terminates.</p> + <p> The application master starts the application by calling the application callback function <c>Module:start/2</c> as defined by the application specification key <c>mod</c>.</p> @@ -608,4 +613,3 @@ Nodes = [cp1@cave, {cp2@cave, cp3@cave}]</code> <seealso marker="app">app(4)</seealso></p> </section> </erlref> - diff --git a/lib/kernel/doc/src/book.xml b/lib/kernel/doc/src/book.xml index 0b69b547e7..4b3573b9fe 100644 --- a/lib/kernel/doc/src/book.xml +++ b/lib/kernel/doc/src/book.xml @@ -4,7 +4,7 @@ <book xmlns:xi="http://www.w3.org/2001/XInclude"> <header titlestyle="normal"> <copyright> - <year>1997</year><year>2016</year> + <year>1997</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/kernel/doc/src/code.xml b/lib/kernel/doc/src/code.xml index bd95819636..69ce4da61c 100644 --- a/lib/kernel/doc/src/code.xml +++ b/lib/kernel/doc/src/code.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1996</year><year>2017</year> + <year>1996</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -538,7 +538,7 @@ zip:create("mnesia-4.4.7.ez", </item> <tag><c>not_purged</c></tag> <item> - <p>The object code can not be loaded because an old version + <p>The object code cannot be loaded because an old version of the code already exists.</p> </item> <tag><c>sticky_directory</c></tag> @@ -611,7 +611,7 @@ ok = code:finish_loading(Prepared), <taglist> <tag><c>not_purged</c></tag> <item> - <p>The object code can not be loaded because an old version + <p>The object code cannot be loaded because an old version of the code already exists.</p> </item> <tag><c>sticky_directory</c></tag> diff --git a/lib/kernel/doc/src/config.xml b/lib/kernel/doc/src/config.xml index 8850c1736b..714af93f4d 100644 --- a/lib/kernel/doc/src/config.xml +++ b/lib/kernel/doc/src/config.xml @@ -4,7 +4,7 @@ <fileref> <header> <copyright> - <year>1997</year><year>2017</year> + <year>1997</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -86,8 +86,13 @@ <tag><c>File = string()</c></tag> <item>Name of another <c>.config</c> file. Extension <c>.config</c> can be omitted. It is - recommended to use absolute paths. A relative path is - relative the current working directory of the emulator.</item> + recommended to use absolute paths. If a relative path is used, + <c>File</c> is searched, first, relative from <c>sys.config</c> directory, then relative + to the current working directory of the emulator, for backward compatibility. + This allow to use a <c>sys.config</c> pointing out other <c>.config</c> files in a release + or in a node started manually using <c>-config ...</c> with same result whatever + the current working directory. + </item> </taglist> <p>When traversing the contents of <c>sys.config</c> and a filename is encountered, its contents are read and merged with the result diff --git a/lib/kernel/doc/src/error_logger.xml b/lib/kernel/doc/src/error_logger.xml index f418aa5bbe..c3d68fd79f 100644 --- a/lib/kernel/doc/src/error_logger.xml +++ b/lib/kernel/doc/src/error_logger.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1996</year><year>2017</year> + <year>1996</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -305,14 +305,22 @@ ok</pre> <type name="open_error"/> <desc> <p>Enables or disables printout of standard events to a file.</p> - <p>This is done by adding or deleting the standard event handler - for output to file. Thus, calling this function overrides - the value of the Kernel <c>error_logger</c> configuration - parameter.</p> - <p>Enabling file logging can be used together with calling - <c>tty(false)</c>, to have a silent system where - all standard events are logged to a file only. - Only one log file can be active at a time.</p> + <p>This is done by adding or deleting + the <c>error_logger_file_h</c> event handler, and thus + indirectly adding <c>error_logger</c> as a Logger + handler.</p> + <p>Notice that this function does not manipulate the Logger + configuration directly, meaning that if the default Logger + handler is already logging to a file, this function can + potentially cause logging to a second file.</p> + <p>This function is useful as a shortcut during development + and testing, but must not be used in a production + system. See + section <seealso marker="logger_chapter">Logging</seealso> + in the Kernel User's Guide, and + the <seealso marker="logger"><c>logger(3)</c></seealso> + manual page for information about how to configure Logger + for live systems.</p> <p><c>Request</c> is one of the following:</p> <taglist> <tag><c>{open, <anno>Filename</anno>}</c></tag> @@ -344,9 +352,14 @@ ok</pre> <p>Enables (<c><anno>Flag</anno> == true</c>) or disables (<c><anno>Flag</anno> == false</c>) printout of standard events to the terminal.</p> - <p>This is done by adding or deleting the standard event handler - for output to the terminal. Thus, calling this function overrides - the value of the Kernel <c>error_logger</c> configuration parameter.</p> + <p>This is done by manipulating the Logger configuration. The + function is useful as a shortcut during development and + testing, but must not be used in a production system. See + section <seealso marker="logger_chapter">Logging</seealso> + in the Kernel User's Guide, and + the <seealso marker="logger"><c>logger(3)</c></seealso> + manual page for information about how to configure Logger + for live systems.</p> </desc> </func> <func> diff --git a/lib/kernel/doc/src/file.xml b/lib/kernel/doc/src/file.xml index 1b72769ce3..9acaf6b41e 100644 --- a/lib/kernel/doc/src/file.xml +++ b/lib/kernel/doc/src/file.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1996</year><year>2016</year> + <year>1996</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/kernel/doc/src/heart.xml b/lib/kernel/doc/src/heart.xml index 46c7ce60b6..ad1a2ffeb9 100644 --- a/lib/kernel/doc/src/heart.xml +++ b/lib/kernel/doc/src/heart.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1996</year><year>2016</year> + <year>1996</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/kernel/doc/src/inet.xml b/lib/kernel/doc/src/inet.xml index e6a7962c5a..f281d61459 100644 --- a/lib/kernel/doc/src/inet.xml +++ b/lib/kernel/doc/src/inet.xml @@ -734,22 +734,23 @@ get_tcpi_sacked(Sock) -> </item> <tag><c>{buffer, Size}</c></tag> <item> - <p>The size of the user-level software buffer used by - the driver. - Not to be confused with options <c>sndbuf</c> + <p>The size of the user-level buffer used by + the driver. Not to be confused with options <c>sndbuf</c> and <c>recbuf</c>, which correspond to the - Kernel socket buffers. It is recommended - to have <c>val(buffer) >= max(val(sndbuf),val(recbuf))</c> to + Kernel socket buffers. For TCP it is recommended + to have <c>val(buffer) >= val(recbuf)</c> to avoid performance issues because of unnecessary copying. + For UDP the same recommendation applies, but the max should + not be larger than the MTU of the network path. <c>val(buffer)</c> is automatically set to the above - maximum when values <c>sndbuf</c> or <c>recbuf</c> are set. - However, as the sizes set for <c>sndbuf</c> and <c>recbuf</c> + maximum when <c>recbuf</c> is set. + However, as the size set for <c>recbuf</c> usually become larger, you are encouraged to use <seealso marker="#getopts/2"><c>getopts/2</c></seealso> to analyze the behavior of your operating system.</p> <p>Note that this is also the maximum amount of data that can be - received from a single recv call. If you are using higher than - normal MTU consider setting buffer higher.</p> + received from a single recv call. If you are using higher than + normal MTU consider setting buffer higher.</p> </item> <tag><c>{delay_send, Boolean}</c></tag> <item> diff --git a/lib/kernel/doc/src/introduction_chapter.xml b/lib/kernel/doc/src/introduction_chapter.xml index 2eadc70abf..d02b1a2ee5 100644 --- a/lib/kernel/doc/src/introduction_chapter.xml +++ b/lib/kernel/doc/src/introduction_chapter.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2017</year> + <year>2017</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/kernel/doc/src/kernel_app.xml b/lib/kernel/doc/src/kernel_app.xml index e2a6d30249..15dbdb47dc 100644 --- a/lib/kernel/doc/src/kernel_app.xml +++ b/lib/kernel/doc/src/kernel_app.xml @@ -54,9 +54,9 @@ <p>Two standard logger handlers are defined in the Kernel application. These are described in the <seealso marker="logger_chapter">Kernel User's Guide</seealso>, - and in <seealso marker="logger_std_h"><c>logger_std_h(3)</c></seealso> + and in the <seealso marker="logger_std_h"><c>logger_std_h(3)</c></seealso> and <seealso marker="logger_disk_log_h"><c>logger_disk_log_h(3)</c> - </seealso>.</p> + </seealso> manual pages.</p> </section> <section> @@ -166,57 +166,71 @@ </item> <tag><marker id="logger"/><c>logger = [Config]</c></tag> <item> - <p>Specifies how <seealso marker="logger"><c>logger</c></seealso> should be - configured.</p> - <p>For more details and examples, see the <seealso marker="logger_chapter#logger"> - Configuration</seealso> section in the <seealso marker="logger_chapter"> - Logger User's Guide</seealso>. - </p> + <p>Specifies the configuration + for <seealso marker="logger">Logger</seealso>, except the + primary log level, which is specified + with <seealso marker="#logger_level"><c>logger_level</c></seealso>, + and the compatibility + with <seealso marker="sasl:error_logging">SASL Error + Logging</seealso>, which is specified + with <seealso marker="#logger_sasl_compatible"> + <c>logger_sasl_compatible</c></seealso>.</p> + <p>The <c>logger </c> parameter is described in + section <seealso marker="logger_chapter#logger_parameter"> + Logging</seealso> in the Kernel User's Guide.</p> </item> <tag><marker id="logger_level"/><c>logger_level = Level</c></tag> <item> - <p><c>Level = emergency | alert | critical | error | warning | - notice | info | debug</c></p> - <p>This parameter specifies which log levels to log. The - specified level, and all levels that are more severe, will - be logged.</p> - <p>The default value is <c>info</c>.</p> - <p>To change the global log level at run-time, use - <seealso marker="logger#set_logger_config/2"> - <c>logger:set_logger_config(level, error)</c></seealso>.</p> + <p>Specifies the primary log level for Logger. Log events with + the same, or a more severe level, pass through the primary + log level check. See + section <seealso marker="logger_chapter">Logging</seealso> + in the Kernel User's Guide for more information about Logger + and log levels.</p> + <p><c>Level = emergency | alert | critical | error | warning | + notice | info | debug | all | none</c></p> + <p>To change the primary log level at runtime, use + <seealso marker="logger#set_primary_config/2"> + <c>logger:set_primary_config(level, Level)</c></seealso>.</p> + <p>Defaults to <c>notice</c>.</p> </item> <tag><marker id="logger_sasl_compatible"/> - <c>logger_sasl_compatible = boolean()</c></tag> - <item> - <p>If this parameter is set to true, then the <c>default</c> logger handler - will not log any progress-, crash-, or supervisor reports. - If the SASL application is started, - these log events will be sent to a second handler instance - named <c>sasl</c>, according to values of the SASL - configuration parameter <c>sasl_error_logger</c> - and <c>sasl_errlog_type</c>, see - <seealso marker="sasl:sasl_app#deprecated_error_logger_config">sasl(6) - </seealso></p> - <p>The default value is <c>false</c>.</p> - <p>See chapter <seealso marker="logger_chapter#compatibility">Backwards - compatibility with error_logger</seealso> for more - information about handling of the so called SASL reports.</p> - <note><p>This configuration option only effects the <c>default</c> - and <c>sasl</c> handler. Any other handlers are uneffected.</p></note> - </item> - <tag><marker id="logger_progress_reports"/> - <c>logger_progress_reports = stop | log</c></tag> - <item> - <p>If <c>logger_sasl_compatible = false</c>, - then <c>logger_progress_reports</c> specifies if progress - reports from <c>supervisor</c> - and <c>application_controller</c> shall be logged by the - default logger.</p> - <p>If <c>logger_sasl_compatible = true</c>, - then <c>logger_progress_reports</c> is ignored.</p> - <p>The default value is <c>stop</c></p> - <note><p>This configuration option only effects the <c>default</c> - and <c>sasl</c> handler. Any other handlers are uneffected.</p></note> + <c>logger_sasl_compatible = true | false</c></tag> + <item> + <p>Specifies if Logger behaves backwards compatible with the + SASL error logging functionality from releases prior to + Erlang/OTP 21.0.</p> + <p>If this parameter is set to <c>true</c>, the default Logger + handler does not log any progress-, crash-, or supervisor + reports. If the SASL application is then started, it adds a + Logger handler named <c>sasl</c>, which logs these events + according to values of the SASL configuration + parameter <c>sasl_error_logger</c> + and <c>sasl_errlog_type</c>.</p> + <p>See section + <seealso marker="sasl:sasl_app#deprecated_error_logger_config"> + Deprecated Error Logger Event Handlers and + Configuration</seealso> in the sasl(6) manual page for + information about the SASL configuration parameters.</p> + <p>See section <seealso marker="sasl:error_logging">SASL Error + Logging</seealso> in the SASL User's Guide, and + section <seealso marker="logger_chapter#compatibility">Backwards + Compatibility with error_logger</seealso> in the Kernel + User's Guide for information about the SASL error logging + functionality, and how Logger can be backwards compatible + with this.</p> + <p>Defaults to <c>false</c>.</p> + <note> + <p>If this parameter is set to <c>true</c>, + <c>sasl_errlog_type</c> indicates that progress reports + shall be logged, and the configured primary log level + is <c>notice</c> or more severe, then SASL automatically + sets the primary log level to <c>info</c>. That is, this + setting can potentially overwrite the value of the Kernel + configuration parameter <c>logger_level</c>. This is to + allow progress reports, which have log level <c>info</c>, + to be forwarded to the handlers.</p> + </note> </item> <tag><c>global_groups = [GroupTuple]</c></tag> <item> @@ -269,9 +283,8 @@ </item> <tag><c>inet_parse_error_log = silent</c></tag> <item> - <p>If set, no - <c>logger</c> messages are generated when erroneous - lines are found and skipped in the various Inet configuration + <p>If set, no log events are issued when erroneous lines are + found and skipped in the various Inet configuration files.</p> </item> <tag><c>inetrc = Filename</c></tag> @@ -297,24 +310,31 @@ <tag><c>net_ticktime = TickTime</c></tag> <item> <marker id="net_ticktime"></marker> - <p>Specifies the <c>net_kernel</c> tick time. <c>TickTime</c> - is specified in seconds. Once every <c>TickTime/4</c> second, all - connected nodes are ticked (if anything else is written - to a node). If nothing is received from another node - within the last four tick times, that node is considered - to be down. This ensures that nodes that are not responding, - for reasons such as hardware errors, are considered to be - down.</p> - <p>The time <c>T</c>, in which a node that is not responding is - detected, is calculated as <c><![CDATA[MinT < T < MaxT]]></c>, where:</p> + <p>Specifies the <c>net_kernel</c> tick time in seconds. This is the + approximate time a connected node may be unresponsive until it is + considered down and thereby disconnected.</p> + <p>Once every <c>TickTime/4</c> seconds, each connected node is ticked + if nothing has been sent to it during that last <c>TickTime/4</c> + interval. A tick is a small package sent on the connection. A connected + node is considered to be down if no ticks or payload packages have been + received during the last four <c>TickTime/4</c> intervals. This ensures + that nodes that are not responding, for reasons such as hardware errors, + are considered to be down.</p> + <p>As the availability is only checked every <c>TickTime/4</c> seconds, + the actual time <c>T</c> a node have been unresponsive when + detected may vary between <c>MinT</c> and <c>MaxT</c>, + where:</p> <code type="none"> MinT = TickTime - TickTime / 4 MaxT = TickTime + TickTime / 4</code> - <p><c>TickTime</c> defaults to <c>60</c> (seconds). Thus, - <c><![CDATA[45 < T < 75]]></c> seconds.</p> - <p>Notice that <em>all</em> communicating nodes are to have the <em>same</em> - <c>TickTime</c> value specified.</p> - <p>Normally, a terminating node is detected immediately.</p> + <p><c>TickTime</c> defaults to <c>60</c> seconds. Thus, + <c><![CDATA[45 < T < 75]]></c> seconds.</p> + <p>Notice that <em>all</em> communicating nodes are to have the + <em>same</em> <c>TickTime</c> value specified, as it determines both the + frequency of outgoing ticks and the expected frequency of incominging + ticks.</p> + <p>Normally, a terminating node is detected immediately by the transport + protocol (like TCP/IP).</p> </item> <tag><c>shutdown_timeout = integer() | infinity</c></tag> <item> @@ -494,7 +514,7 @@ MaxT = TickTime + TickTime / 4</code> <seealso marker="logger_std_h#type"><c>logger_std_h</c></seealso> to the same value. Example: <code type="none"> -erl -kernel logger '[{handler,default,logger_std_h,#{logger_std_h=>#{type=>{file,"/tmp/erlang.log"}}}}]' +erl -kernel logger '[{handler,default,logger_std_h,#{config=>#{type=>{file,"/tmp/erlang.log"}}}}]' </code> </item> <tag><c>error_logger_format_depth</c></tag> diff --git a/lib/kernel/doc/src/logger.xml b/lib/kernel/doc/src/logger.xml index 911eb158da..f1830a8224 100644 --- a/lib/kernel/doc/src/logger.xml +++ b/lib/kernel/doc/src/logger.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>2017</year> + <year>2017</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -33,73 +33,151 @@ <file>logger.xml</file> </header> <module>logger</module> - <modulesummary>API module for logging in Erlang/OTP.</modulesummary> + <modulesummary>API module for Logger, the standard logging facility + in Erlang/OTP.</modulesummary> <description> - <p> - This module is the main API for logging in Erlang/OTP. It - contains functions that allow applications to use a single log - API and the system to manage those log events independently. Use - the <seealso marker="#emergency-1">API functions</seealso> or the log - <seealso marker="#macros">macros</seealso> to log events. For instance, - to log a new error event:</p> - <code> -?LOG_ERROR("error happened because: ~p",[Reason]). %% With macro -logger:error("error happened because: ~p",[Reason]). %% Without macro - </code> - <p>This log event is then sent to the configured log handlers which - by default means that it is be printed to the console. If you want - your systems logs to be printed to a file instead of the console you - must configure the default handler to do so. The simplest way is - to include the following in your <seealso marker="config"><c>sys.config</c></seealso>.</p> + <p>This module implements the main API for logging in + Erlang/OTP. To create a log event, use the + <seealso marker="#logging_API">API functions</seealso> or the + log + <seealso marker="#macros">macros</seealso>, for example:</p> + <code> +?LOG_ERROR("error happened because: ~p", [Reason]). % With macro +logger:error("error happened because: ~p", [Reason]). % Without macro + </code> + <p>To configure the Logger backend, + use <seealso marker="kernel_app#logger">Kernel configuration + parameters</seealso> + or <seealso marker="#configuration_API">configuration + functions</seealso> in the Logger API.</p> + + <p>By default, the Kernel application installs one log handler at + system start. This handler is named <c>default</c>. It receives + and processes standard log events produced by the Erlang runtime + system, standard behaviours and different Erlang/OTP + applications. The log events are by default printed to the + terminal.</p> + <p>If you want your systems logs to be printed to a file instead, + you must configure the default handler to do so. The simplest + way is to include the following in + your <seealso marker="config"><c>sys.config</c></seealso>:</p> <code> [{kernel, [{logger, - [{handler,default,logger_std_h, - #{logger_std_h=>#{type=>{file,"path/to/file.log"}}}}]}]}]. + [{handler, default, logger_std_h, + #{config => #{type => {file, "path/to/file.log"}}}}]}]}]. </code> <p> For more information about: </p> <list type="bulleted"> - <item>how to use the API, - see <seealso marker="logger_chapter">the User's Guide</seealso>.</item> - <item>how to configure Logger, - see the <seealso marker="logger_chapter#configuration">Configuration</seealso> + <item>the Logger facility in general, see + the <seealso marker="logger_chapter">User's + Guide</seealso>.</item> + <item>how to configure Logger, see + the <seealso marker="logger_chapter#configuration">Configuration</seealso> section in the User's Guide.</item> - <item>the convinience macros in logger.hrl, - see <seealso marker="#macros">the macro section</seealso>.</item> - <item>what the builtin formatter can do, - see <seealso marker="logger_formatter">logger_formatter</seealso>.</item> - <item>what the builtin handlers can do, + <item>the built-in handlers, see <seealso marker="logger_std_h">logger_std_h</seealso> and <seealso marker="logger_disk_log_h">logger_disk_log_h</seealso>.</item> - <item>what builtin filters are available, + <item>the built-in formatter, + see <seealso marker="logger_formatter">logger_formatter</seealso>.</item> + <item>built-in filters, see <seealso marker="logger_filters">logger_filters</seealso>.</item> </list> + + <note> + <p>Since Logger is new in Erlang/OTP 21.0, we do reserve the right + to introduce changes to the Logger API and functionality in + patches following this release. These changes might or might not + be backwards compatible with the initial version.</p> + </note> + </description> <datatypes> <datatype> - <name name="level"/> + <name name="filter"/> <desc> - <p>The severity level for the message to be logged.</p> + <p>A filter which can be installed as a handler filter, or as + a primary filter in Logger.</p> </desc> </datatype> <datatype> - <name name="log_event"/> + <name name="filter_arg"/> <desc> - <p></p> + <p>The second argument to the filter fun.</p> </desc> </datatype> <datatype> - <name name="report"/> + <name name="filter_id"/> <desc> - <p></p> + <p>A unique identifier for a filter.</p> </desc> </datatype> <datatype> - <name name="msg_fun"/> + <name name="filter_return"/> + <desc> + <p>The return value from the filter fun.</p> + </desc> + </datatype> + <datatype> + <name name="formatter_config"/> + <desc> + <p>Configuration data for the + formatter. See <seealso marker="logger_formatter"> + <c>logger_formatter(3)</c></seealso> + for an example of a formatter implementation.</p> + </desc> + </datatype> + <datatype> + <name name="handler_config"/> + <desc> + <p>Handler configuration data for Logger. The following + default values apply:</p> + <list> + <item><c>level => all</c></item> + <item><c>filter_default => log</c></item> + <item><c>filters => []</c></item> + <item><c>formatter => {logger_formatter, DefaultFormatterConfig</c>}</item> + </list> + <p>In addition to these, the following fields are + automatically inserted by Logger, values taken from the + two first parameters + to <seealso marker="#add_handler-3"><c>add_handler/3</c></seealso>:</p> + <list> + <item><c>id => HandlerId</c></item> + <item><c>module => Module</c></item> + </list> + <p>These are read-only and cannot be changed in runtime.</p> + <p>Handler specific configuration data is inserted by the + handler callback itself, in a sub structure associated with + the field named <c>config</c>. See + the <seealso marker="logger_std_h"><c>logger_std_h(3)</c></seealso> + and <seealso marker="logger_disk_log_h"><c>logger_disk_log_h</c></seealso> + manual pages for information about the specifc configuration + for these handlers.</p> + <p>See the <seealso marker="logger_formatter#type-config"> + <c>logger_formatter(3)</c></seealso> manual page for + information about the default configuration for this + formatter.</p> + </desc> + </datatype> + <datatype> + <name name="handler_id"/> + <desc> + <p>A unique identifier for a handler instance.</p> + </desc> + </datatype> + <datatype> + <name name="level"/> + <desc> + <p>The severity level for the message to be logged.</p> + </desc> + </datatype> + <datatype> + <name name="log_event"/> <desc> <p></p> </desc> @@ -117,7 +195,7 @@ logger:error("error happened because: ~p",[Reason]). %% Without macro <p>When a log macro is used, Logger also inserts location information:</p> <list> - <item><c>mfa => {?MODULE,?FUNCTION_NAME,?FUNCTION_ARITY}</c></item> + <item><c>mfa => {?MODULE, ?FUNCTION_NAME, ?FUNCTION_ARITY}</c></item> <item><c>file => ?FILE</c></item> <item><c>line => ?LINE</c></item> </list> @@ -149,7 +227,10 @@ logger:error("error happened because: ~p",[Reason]). %% Without macro a <seealso marker="#type-report"><c>report()</c></seealso>, the <c>report_cb</c> key can be associated with a fun (report callback) that converts the report to a format - string and arguments. See + string and arguments, or directly to a string. See the + type definition + of <seealso marker="#type-report_cb"><c>report_cb()</c></seealso>, + and section <seealso marker="logger_chapter#log_message">Log Message</seealso> in the User's Guide for more information about report callbacks.</p> @@ -158,51 +239,43 @@ logger:error("error happened because: ~p",[Reason]). %% Without macro </desc> </datatype> <datatype> - <name name="config"/> + <name name="msg_fun"/> + <desc> + <p></p> + </desc> + </datatype> + <datatype> + <name name="primary_config"/> <desc> - <p>Configuration data for the logger part of Logger, or for a handler.</p> - <p>The following default values apply:</p> + <p>Primary configuration data for Logger. The following + default values apply:</p> <list> <item><c>level => info</c></item> <item><c>filter_default => log</c></item> <item><c>filters => []</c></item> - <item><c>formatter => {logger_formatter,DefaultFormatterConfig</c>}</item> </list> - <p>See the <seealso marker="logger_formatter#type-config"> - <c>logger_formatter(3)</c></seealso> manual page for - information about the default configuration for this - formatter.</p> - </desc> - </datatype> - <datatype> - <name name="handler_id"/> - <desc> - <p>A unique identifier for a handler instance.</p> </desc> </datatype> <datatype> - <name name="filter_id"/> - <desc> - <p>A unique identifier for a filter.</p> - </desc> - </datatype> - <datatype> - <name name="filter"/> + <name name="report"/> <desc> - <p>A filter which can be installed for the logger part of - Logger, or for a handler.</p> + <p></p> </desc> </datatype> <datatype> - <name name="filter_arg"/> + <name name="report_cb"/> <desc> - <p>The second argument to the filter fun.</p> + <p>A fun which converts a <seealso marker="#type-report"><c>report()</c> + </seealso> to a format string and arguments, or directly to a string. + See section <seealso marker="logger_chapter#log_message">Log + Message</seealso> in the User's Guide for more + information.</p> </desc> </datatype> <datatype> - <name name="filter_return"/> + <name name="report_cb_config"/> <desc> - <p>The return value from the filter fun.</p> + <p></p> </desc> </datatype> <datatype> @@ -213,15 +286,6 @@ logger:error("error happened because: ~p",[Reason]). %% Without macro <c>erlang:system_time(microsecond)</c></seealso>.</p> </desc> </datatype> - <datatype> - <name name="formatter_config"/> - <desc> - <p>Configuration data for the - formatter. See <seealso marker="logger_formatter"> - <c>logger_formatter(3)</c></seealso> - for an example of a formatter implementation.</p> - </desc> - </datatype> </datatypes> <section> @@ -245,19 +309,26 @@ logger:error("error happened because: ~p",[Reason]). %% Without macro <item><c>?LOG_INFO(FunOrFormat,Args[,Metadata])</c></item> <item><c>?LOG_DEBUG(StringOrReport[,Metadata])</c></item> <item><c>?LOG_DEBUG(FunOrFormat,Args[,Metadata])</c></item> + <item><c>?LOG(Level,StringOrReport[,Metadata])</c></item> + <item><c>?LOG(Level,FunOrFormat,Args[,Metadata])</c></item> </list> <p>All macros expand to a call to Logger, where <c>Level</c> is - taken from the macro name, and location data is added to the - metadata. See the description of + taken from the macro name, or from the first argument in the + case of the <c>?LOG</c> macro. Location data is added to the + metadata as described under the <seealso marker="#type-metadata"><c>metadata()</c></seealso> - type for more information about the location data.</p> + type definition.</p> <p>The call is wrapped in a case statement and will be evaluated only if <c>Level</c> is equal to or below the configured log level.</p> </section> + <section> + <marker id="logging_API"/> + <title>Logging API functions</title> + </section> <funcs> <func> <name>emergency(StringOrReport[,Metadata])</name> @@ -317,7 +388,7 @@ logger:error("error happened because: ~p",[Reason]). %% Without macro <func> <name>notice(StringOrReport[,Metadata])</name> <name>notice(Format,Args[,Metadata])</name> - <name>notice(Fun,FunArgs[,Metadata])</name> + <name>notice(Fun,FunArgs[,Metadata])</name> <fsummary>Logs the given message as level <c>notice</c>.</fsummary> <desc> <p>Equivalent to @@ -366,136 +437,21 @@ logger:error("error happened because: ~p",[Reason]). %% Without macro <p>Log the given message.</p> </desc> </func> + </funcs> + <section> + <marker id="configuration_API"/> + <title>Configuration API functions</title> + </section> + <funcs> <func> - <name name="get_logger_config" arity="0"/> - <fsummary>Look up the current configuration for the logger part - of Logger.</fsummary> - <desc> - <p>Look up the current configuration for the logger part of - Logger.</p> - </desc> - </func> - - <func> - <name name="get_handler_config" arity="1"/> - <fsummary>Look up the current configuration for the given - handler.</fsummary> - <desc> - <p>Look up the current configuration for the given handler.</p> - </desc> - </func> - - <func> - <name name="i" arity="0"/> - <fsummary>Get all Logger configurations</fsummary> - <desc> - <p>Same as <seealso marker="#i/1"><c>logger:i(term)</c></seealso></p> - </desc> - </func> - - <func> - <name name="i" arity="1" clause_i="1"/> - <name name="i" arity="1" clause_i="2"/> - <name name="i" arity="1" clause_i="3"/> - <fsummary>Get all Logger configurations</fsummary> - <desc> - <p>Display or return all current Logger configurations.</p> - <taglist> - <tag><c><anno>Action</anno> = string</c></tag> - <item> - <p>Return the pretty printed current Logger configuration - as iodata.</p> - </item> - <tag><c><anno>Action</anno> = term</c></tag> - <item> - <p>Return the current Logger configuration as a term. The - format of this term may change between releases. For a - stable format use <seealso marker="#get_handler_config/1"> - <c>logger:get_handler_config/1</c></seealso> - and <seealso marker="#get_logger_config/0"> - <c>logger:get_logger_config/0</c></seealso>.</p> - <p>The same as calling <c>logger:i()</c>.</p> - </item> - <tag><c><anno>Action</anno> = print</c></tag> - <item> - <p>Pretty print all the current Logger configuration to - standard out. Example:</p> - <code><![CDATA[1> logger:i(print). -Current logger configuration: - Level: info - Filter Default: log - Filters: - Handlers: - Id: default - Module: logger_std_h - Level: info - Formatter: - Module: logger_formatter - Config: #{legacy_header => true,single_line => false, - template => [{logger_formatter,header},"\n",msg,"\n"]} - Filter Default: stop - Filters: - Id: stop_progress - Fun: fun logger_filters:progress/2 - Config: stop - Id: remote_gl - Fun: fun logger_filters:remote_gl/2 - Config: stop - Id: domain - Fun: fun logger_filters:domain/2 - Config: {log,super,[beam,erlang,otp,sasl]} - Id: no_domain - Fun: fun logger_filters:domain/2 - Config: {log,undefined,[]} - Handler Config: - logger_std_h: #{type => standard_io} - Level set per module: - Module: my_module - Level: debug]]></code> - </item> - </taglist> - </desc> - </func> - - <func> - <name name="add_logger_filter" arity="2"/> - <fsummary>Add a filter to the logger part of Logger.</fsummary> + <name name="add_handler" arity="3"/> + <fsummary>Add a handler with the given configuration.</fsummary> <desc> - <p>Add a filter to the logger part of Logger.</p> - <p>The filter fun is called with the log event as the first - parameter, and the specified <c>filter_args()</c> as the - second parameter.</p> - <p>The return value of the fun specifies if a log event is to - be discarded or forwarded to the handlers:</p> - <taglist> - <tag><c>log_event()</c></tag> - <item> - <p>The filter <em>passed</em>. The next logger filter, if - any, is applied. If no more logger filters exist, the - log event is forwarded to the handler part of Logger, - where handler filters are applied.</p> - </item> - <tag><c>stop</c></tag> - <item> - <p>The filter <em>did not pass</em>, and the log event is - immediately discarded.</p> - </item> - <tag><c>ignore</c></tag> - <item> - <p>The filter has no knowledge of the log event. The next - logger filter, if any, is applied. If no more logger - filters exist, the value of the <c>filter_default</c> - configuration parameter for the logger part specifies if - the log event shall be discarded or forwarded to the - handler part.</p> - </item> - </taglist> - <p>See section <seealso marker="logger_chapter#filters"> - Filters</seealso> in the User's Guide for more information - about filters.</p> - <p>Some built-in filters exist. These are defined - in <seealso marker="logger_filters"><c>logger_filters</c></seealso>.</p> + <p>Add a handler with the given configuration.</p> + <p><c><anno>HandlerId</anno></c> is a unique identifier which + must be used in all subsequent calls referring to this + handler.</p> </desc> </func> @@ -541,98 +497,12 @@ Current logger configuration: </func> <func> - <name name="remove_logger_filter" arity="1"/> - <fsummary>Remove a filter from the logger part of Logger.</fsummary> - <desc> - <p>Remove the filter identified - by <c><anno>FilterId</anno></c> from the logger part of Logger.</p> - </desc> - </func> - - <func> - <name name="remove_handler_filter" arity="2"/> - <fsummary>Remove a filter from the specified handler.</fsummary> - <desc> - <p>Remove the filter identified - by <c><anno>FilterId</anno></c> from the handler identified - by <c><anno>HandlerId</anno></c>.</p> - </desc> - </func> - - <func> - <name name="add_handler" arity="3"/> - <fsummary>Add a handler with the given configuration.</fsummary> - <desc> - <p>Add a handler with the given configuration.</p> - <p><c><anno>HandlerId</anno></c> is a unique identifier which - must be used in all subsequent calls referring to this - handler.</p> - </desc> - </func> - - <func> - <name name="remove_handler" arity="1"/> - <fsummary>Remove the handler with the specified identity.</fsummary> - <desc> - <p>Remove the handler identified by <c><anno>HandlerId</anno></c>.</p> - </desc> - </func> - - <func> - <name name="set_module_level" arity="2"/> - <fsummary>Set the log level for the specified module.</fsummary> - <desc> - <p>Set the log level for the - specified <c><anno>Module</anno></c>.</p> - <p>The log level for a module overrides the global log level - of Logger for log events originating from the module in - question. Notice, however, that it does not override the - level configuration for any handler.</p> - <p>For example: Assume that the global log level for Logger - is <c>info</c>, and there is one handler, <c>h1</c>, with - level <c>info</c> and one handler, <c>h2</c>, with - level <c>debug</c>.</p> - <p>With this configuration, no debug messages will be logged, - since they are all stopped by the global log level.</p> - <p>If the level for <c>mymodule</c> is now set - to <c>debug</c>, then debug events from this module will be - logged by the handler <c>h2</c>, but not by - handler <c>h1</c>.</p> - <p>Debug events from other modules are still not logged.</p> - <p>To change the global log level for Logger, use - <seealso marker="#set_logger_config/2"> - <c>logger:set_logger_config(level,Level)</c></seealso>.</p> - <p>To change the log level for a handler, use - <seealso marker="#set_handler_config/3"> - <c>logger:set_handler_config(HandlerId,level,Level)</c></seealso>.</p> - <note> - <p>The originating module for a log event is only detected - if <c>mfa=>{Module,Function,Arity}</c> exists in the - metadata. When log macros are used, this association is - automatically added to all log events. If an API function - is called directly, without using a macro, the logging - client must explicitly add this information if module - levels shall have any effect.</p> - </note> - </desc> - </func> - - <func> - <name name="unset_module_level" arity="1"/> - <fsummary>Remove a module specific log setting.</fsummary> - <desc> - <p>Remove a module specific log setting. After this, the - global log level is used for the specified module.</p> - </desc> - </func> - - <func> <name name="add_handlers" arity="1" clause_i="1"/> <fsummary>Set up log handlers from the application's configuration parameters.</fsummary> <desc> <p>Reads the application configuration parameter <c>logger</c> and - calls <c>logger:add_handlers/1</c> with its contents.</p> + calls <c>add_handlers/1</c> with its contents.</p> </desc> </func> @@ -670,61 +540,158 @@ start(_, []) -> [{kernel, [{logger, %% Disable the default Kernel handler - [{handler,default,undefined}]}]}, + [{handler, default, undefined}]}]}, {my_app, [{logger, %% Enable this handler as the default - [{handler,default,my_handler,#{}}]}]}]. + [{handler, default, my_handler, #{}}]}]}]. </code> </desc> </func> <func> - <name name="set_logger_config" arity="1"/> - <fsummary>Set configuration data for the logger part of Logger.</fsummary> + <name name="add_primary_filter" arity="2"/> + <fsummary>Add a primary filter to Logger.</fsummary> <desc> - <p>Set configuration data for the logger part of Logger. This - overwrites the current logger configuration.</p> - <p>To modify the existing configuration, - use <seealso marker="#update_logger_config-1"> - <c>update_logger_config/1</c></seealso>, or, if a more - complex merge is needed, read the current configuration - with <seealso marker="#get_logger_config-0"><c>get_logger_config/0</c> - </seealso>, then do the merge before writing the new - configuration back with this function.</p> - <p>If a key is removed compared to the current configuration, - the default value is used.</p> + <p>Add a primary filter to Logger.</p> + <p>The filter fun is called with the log event as the first + parameter, and the specified <c>filter_args()</c> as the + second parameter.</p> + <p>The return value of the fun specifies if a log event is to + be discarded or forwarded to the handlers:</p> + <taglist> + <tag><c>log_event()</c></tag> + <item> + <p>The filter <em>passed</em>. The next primary filter, if + any, is applied. If no more primary filters exist, the + log event is forwarded to the handler part of Logger, + where handler filters are applied.</p> + </item> + <tag><c>stop</c></tag> + <item> + <p>The filter <em>did not pass</em>, and the log event is + immediately discarded.</p> + </item> + <tag><c>ignore</c></tag> + <item> + <p>The filter has no knowledge of the log event. The next + primary filter, if any, is applied. If no more primary + filters exist, the value of the + primary <c>filter_default</c> configuration parameter + specifies if the log event shall be discarded or + forwarded to the handler part.</p> + </item> + </taglist> + <p>See section <seealso marker="logger_chapter#filters"> + Filters</seealso> in the User's Guide for more information + about filters.</p> + <p>Some built-in filters exist. These are defined + in <seealso marker="logger_filters"><c>logger_filters</c></seealso>.</p> </desc> </func> <func> - <name name="set_logger_config" arity="2"/> - <fsummary>Add or update configuration data for the logger part - of Logger.</fsummary> + <name name="get_config" arity="0"/> + <fsummary>Look up the current Logger configuration</fsummary> <desc> - <p>Add or update configuration data for the logger part of - Logger. If the given <c><anno>Key</anno></c> already exists, - its associated value will be changed - to <c><anno>Value</anno></c>. If it does not exist, it will - be added.</p> + <p>Look up all current Logger configuration, including primary + and handler configuration, and module level settings.</p> </desc> </func> <func> - <name name="update_logger_config" arity="1"/> - <fsummary>Update configuration data for the logger part of - Logger.</fsummary> + <name name="get_handler_config" arity="0"/> + <fsummary>Look up the current configuration for all handlers.</fsummary> <desc> - <p>Update configuration data for the logger part of - Logger. This function behaves as if it was implemented as - follows:</p> - <code type="erl"> -{ok,Old} = logger:get_logger_config(), -logger:set_logger_config(maps:merge(Old,Config)). - </code> - <p>To overwrite the existing configuration without any merge, - use <seealso marker="#set_logger_config-1"><c>set_logger_config/1</c> - </seealso>.</p> + <p>Look up the current configuration for all handlers.</p> + </desc> + </func> + + <func> + <name name="get_handler_config" arity="1"/> + <fsummary>Look up the current configuration for the given + handler.</fsummary> + <desc> + <p>Look up the current configuration for the given handler.</p> + </desc> + </func> + + <func> + <name name="get_handler_ids" arity="0"/> + <fsummary>Look up the identities for all installed handlers.</fsummary> + <desc> + <p>Look up the identities for all installed handlers.</p> + </desc> + </func> + + <func> + <name name="get_primary_config" arity="0"/> + <fsummary>Look up the current primary configuration for Logger.</fsummary> + <desc> + <p>Look up the current primary configuration for Logger.</p> + </desc> + </func> + + <func> + <name name="get_module_level" arity="0"/> + <fsummary>Look up all current module levels.</fsummary> + <desc> + <p>Look up all current module levels. Returns a list + containing one <c>{Module,Level}</c> element for each module + for which the module level was previously set + with <seealso marker="#set_module_level-2"> + <c>set_module_level/2</c></seealso>.</p> + </desc> + </func> + + <func> + <name name="get_module_level" arity="1"/> + <fsummary>Look up the current level for the given modules.</fsummary> + <desc> + <p>Look up the current level for the given modules. Returns a + list containing one <c>{Module,Level}</c> element for each + of the given modules for which the module level was + previously set with <seealso marker="#set_module_level-2"> + <c>set_module_level/2</c></seealso>.</p> + </desc> + </func> + + <func> + <name name="get_process_metadata" arity="0"/> + <fsummary>Retrieve data set with set_process_metadata/1.</fsummary> + <desc> + <p>Retrieve data set + with <seealso marker="#set_process_metadata-1"> + <c>set_process_metadata/1</c></seealso> or + <seealso marker="#update_process_metadata-1"> + <c>update_process_metadata/1</c></seealso>.</p> + </desc> + </func> + + <func> + <name name="remove_handler" arity="1"/> + <fsummary>Remove the handler with the specified identity.</fsummary> + <desc> + <p>Remove the handler identified by <c><anno>HandlerId</anno></c>.</p> + </desc> + </func> + + <func> + <name name="remove_handler_filter" arity="2"/> + <fsummary>Remove a filter from the specified handler.</fsummary> + <desc> + <p>Remove the filter identified + by <c><anno>FilterId</anno></c> from the handler identified + by <c><anno>HandlerId</anno></c>.</p> + </desc> + </func> + + <func> + <name name="remove_primary_filter" arity="1"/> + <fsummary>Remove a primary filter from Logger.</fsummary> + <desc> + <p>Remove the primary filter identified + by <c><anno>FilterId</anno></c> from Logger.</p> </desc> </func> @@ -750,31 +717,155 @@ logger:set_logger_config(maps:merge(Old,Config)). </func> <func> - <name name="set_handler_config" arity="3"/> + <name name="set_handler_config" arity="3" clause_i="1"/> + <name name="set_handler_config" arity="3" clause_i="2"/> + <name name="set_handler_config" arity="3" clause_i="3"/> + <name name="set_handler_config" arity="3" clause_i="4"/> + <name name="set_handler_config" arity="3" clause_i="5"/> <fsummary>Add or update configuration data for the specified handler.</fsummary> + <type variable="HandlerId"/> + <type variable="Level" name_i="1"/> + <type variable="FilterDefault" name_i="2"/> + <type variable="Filters" name_i="3"/> + <type variable="Formatter" name_i="4"/> + <type variable="Config" name_i="5"/> + <type variable="Return"/> <desc> <p>Add or update configuration data for the specified handler. If the given <c><anno>Key</anno></c> already exists, its associated value will be changed - to <c><anno>Value</anno></c>. If it does not exist, it will + to the given value. If it does not exist, it will be added.</p> + <p>See the definition of + the <seealso marker="#type-handler_config"> + <c>handler_config()</c></seealso> type for more + information about the different parameters.</p> </desc> </func> <func> - <name name="update_handler_config" arity="2"/> - <fsummary>Update configuration data for the specified handler.</fsummary> + <name name="set_primary_config" arity="1"/> + <fsummary>Set primary configuration data for Logger.</fsummary> <desc> - <p>Update configuration data for the specified handler. This function - behaves as if it was implemented as follows:</p> - <code type="erl"> -{ok,{_,Old}} = logger:get_handler_config(HandlerId), -logger:set_handler_config(HandlerId,maps:merge(Old,Config)). - </code> - <p>To overwrite the existing configuration without any merge, - use <seealso marker="#set_handler_config-2"><c>set_handler_config/2</c> + <p>Set primary configuration data for Logger. This + overwrites the current configuration.</p> + <p>To modify the existing configuration, + use <seealso marker="#update_primary_config-1"> + <c>update_primary_config/1</c></seealso>, or, if a more + complex merge is needed, read the current configuration + with <seealso marker="#get_primary_config-0"><c>get_primary_config/0</c> + </seealso>, then do the merge before writing the new + configuration back with this function.</p> + <p>If a key is removed compared to the current configuration, + the default value is used.</p> + </desc> + </func> + + <func> + <name name="set_primary_config" arity="2" clause_i="1"/> + <name name="set_primary_config" arity="2" clause_i="2"/> + <name name="set_primary_config" arity="2" clause_i="3"/> + <fsummary>Add or update primary configuration data for Logger.</fsummary> + <type variable="Level" name_i="1"/> + <type variable="FilterDefault" name_i="2"/> + <type variable="Filters" name_i="3"/> + <desc> + <p>Add or update primary configuration data for Logger. If the + given <c><anno>Key</anno></c> already exists, its associated + value will be changed to the given value. If it does not + exist, it will be added.</p> + </desc> + </func> + + <func> + <name name="set_module_level" arity="2"/> + <fsummary>Set the log level for the specified modules.</fsummary> + <desc> + <p>Set the log level for the + specified modules.</p> + <p>The log level for a module overrides the primary log level + of Logger for log events originating from the module in + question. Notice, however, that it does not override the + level configuration for any handler.</p> + <p>For example: Assume that the primary log level for Logger + is <c>info</c>, and there is one handler, <c>h1</c>, with + level <c>info</c> and one handler, <c>h2</c>, with + level <c>debug</c>.</p> + <p>With this configuration, no debug messages will be logged, + since they are all stopped by the primary log level.</p> + <p>If the level for <c>mymodule</c> is now set + to <c>debug</c>, then debug events from this module will be + logged by the handler <c>h2</c>, but not by + handler <c>h1</c>.</p> + <p>Debug events from other modules are still not logged.</p> + <p>To change the primary log level for Logger, use + <seealso marker="#set_primary_config/2"> + <c>set_primary_config(level, Level)</c></seealso>.</p> + <p>To change the log level for a handler, use + <seealso marker="#set_handler_config/3"> + <c>set_handler_config(HandlerId, level, Level)</c> </seealso>.</p> + <note> + <p>The originating module for a log event is only detected + if the key <c>mfa</c> exists in the metadata, and is + associated with <c>{Module, Function, Arity}</c>. When log + macros are used, this association is automatically added + to all log events. If an API function is called directly, + without using a macro, the logging client must explicitly + add this information if module levels shall have any + effect.</p> + </note> + </desc> + </func> + + <func> + <name name="set_process_metadata" arity="1"/> + <fsummary>Set metadata to use when logging from current process.</fsummary> + <desc> + <p>Set metadata which Logger shall automatically insert in + all log events produced on the current process.</p> + <p>Location data produced by the log macros, and/or metadata + given as argument to the log call (API function or macro), + are merged with the process metadata. If the same keys + occur, values from the metadata argument to the log call + overwrite values from the process metadata, which in turn + overwrite values from the location data.</p> + <p>Subsequent calls to this function overwrites previous data + set. To update existing data instead of overwriting it, + see <seealso marker="#update_process_metadata-1"> + <c>update_process_metadata/1</c></seealso>.</p> + </desc> + </func> + + <func> + <name name="unset_module_level" arity="0"/> + <fsummary>Remove module specific log settings for all modules.</fsummary> + <desc> + <p>Remove module specific log settings. After this, the + primary log level is used for all modules.</p> + </desc> + </func> + + <func> + <name name="unset_module_level" arity="1"/> + <fsummary>Remove module specific log settings for the given + modules.</fsummary> + <desc> + <p>Remove module specific log settings. After this, the + primary log level is used for the specified modules.</p> + </desc> + </func> + + <func> + <name name="unset_process_metadata" arity="0"/> + <fsummary>Delete data set with set_process_metadata/1.</fsummary> + <desc> + <p>Delete data set + with <seealso marker="#set_process_metadata-1"> + <c>set_process_metadata/1</c></seealso> or + <seealso marker="#update_process_metadata-1"> + <c>update_process_metadata/1</c></seealso>.</p> </desc> </func> @@ -786,9 +877,10 @@ logger:set_handler_config(HandlerId,maps:merge(Old,Config)). <p>The new configuration is merged with the existing formatter configuration.</p> <p>To overwrite the existing configuration without any merge, - use <seealso marker="#set_handler_config-3"> - <c>set_handler_config(HandlerId,formatter, - {FormatterModule,FormatterConfig})</c></seealso>.</p> + use</p> + <pre> +<seealso marker="#set_handler_config-3">set_handler_config(HandlerId, formatter, + {FormatterModule, FormatterConfig})</seealso>.</pre> </desc> </func> @@ -797,39 +889,41 @@ logger:set_handler_config(HandlerId,maps:merge(Old,Config)). <fsummary>Update the formatter configuration for the specified handler.</fsummary> <desc> <p>Update the formatter configuration for the specified handler.</p> - <p>This is equivalent - to <br/><seealso marker="#update_formatter_config-2"> - <c>update_formatter_config(<anno>HandlerId</anno>,#{<anno>Key</anno>=><anno>Value</anno>})</c></seealso></p> + <p>This is equivalent to</p> + <pre> +<seealso marker="#update_formatter_config-2">update_formatter_config(<anno>HandlerId</anno>, #{<anno>Key</anno> => <anno>Value</anno>})</seealso></pre> </desc> </func> <func> - <name name="compare_levels" arity="2"/> - <fsummary>Compare the severity of two log levels.</fsummary> + <name name="update_handler_config" arity="2"/> + <fsummary>Update configuration data for the specified handler.</fsummary> <desc> - <p>Compare the severity of two log levels. Returns <c>gt</c> - if <c>Level1</c> is more severe than - <c>Level2</c>, <c>lt</c> if <c>Level1</c> is less severe, - and <c>eq</c> if the levels are equal.</p> + <p>Update configuration data for the specified handler. This function + behaves as if it was implemented as follows:</p> + <code type="erl"> +{ok, {_, Old}} = logger:get_handler_config(HandlerId), +logger:set_handler_config(HandlerId, maps:merge(Old, Config)). + </code> + <p>To overwrite the existing configuration without any merge, + use <seealso marker="#set_handler_config-2"><c>set_handler_config/2</c> + </seealso>.</p> </desc> </func> <func> - <name name="set_process_metadata" arity="1"/> - <fsummary>Set metadata to use when logging from current process.</fsummary> + <name name="update_primary_config" arity="1"/> + <fsummary>Update primary configuration data for Logger.</fsummary> <desc> - <p>Set metadata which Logger shall automatically insert in - all log events produced on the current process.</p> - <p>Location data produced by the log macros, and/or metadata - given as argument to the log call (API function or macro), - are merged with the process metadata. If the same keys - occur, values from the metadata argument to the log call - overwrite values from the process metadata, which in turn - overwrite values from the location data.</p> - <p>Subsequent calls to this function overwrites previous data - set. To update existing data instead of overwriting it, - see <seealso marker="#update_process_metadata-1"> - <c>update_process_metadata/1</c></seealso>.</p> + <p>Update primary configuration data for Logger. This function + behaves as if it was implemented as follows:</p> + <code type="erl"> +Old = logger:get_primary_config(), +logger:set_primary_config(maps:merge(Old, Config)). + </code> + <p>To overwrite the existing configuration without any merge, + use <seealso marker="#set_primary_config-1"><c>set_primary_config/1</c> + </seealso>.</p> </desc> </func> @@ -843,7 +937,7 @@ logger:set_handler_config(HandlerId,maps:merge(Old,Config)). <p>If process metadata exists for the current process, this function behaves as if it was implemented as follows:</p> <code type="erl"> -logger:set_process_metadata(maps:merge(logger:get_process_metadata(),Meta)). +logger:set_process_metadata(maps:merge(logger:get_process_metadata(), Meta)). </code> <p>If no process metadata exists, the function behaves as <seealso marker="#set_process_metadata-1"> @@ -851,48 +945,44 @@ logger:set_process_metadata(maps:merge(logger:get_process_metadata(),Meta)). </seealso>.</p> </desc> </func> + </funcs> + <section> + <marker id="misc_API"/> + <title>Miscellaneous API functions</title> + </section> + <funcs> <func> - <name name="get_process_metadata" arity="0"/> - <fsummary>Retrieve data set with set_process_metadata/1.</fsummary> - <desc> - <p>Retrieve data set - with <seealso marker="#set_process_metadata-1"> - <c>set_process_metadata/1</c></seealso> or - <seealso marker="#update_process_metadata-1"> - <c>update_process_metadata/1</c></seealso>.</p> - </desc> - </func> - - <func> - <name name="unset_process_metadata" arity="0"/> - <fsummary>Delete data set with set_process_metadata/1.</fsummary> + <name name="compare_levels" arity="2"/> + <fsummary>Compare the severity of two log levels.</fsummary> <desc> - <p>Delete data set - with <seealso marker="#set_process_metadata-1"> - <c>set_process_metadata/1</c></seealso> or - <seealso marker="#update_process_metadata-1"> - <c>update_process_metadata/1</c></seealso>.</p> + <p>Compare the severity of two log levels. Returns <c>gt</c> + if <c>Level1</c> is more severe than + <c>Level2</c>, <c>lt</c> if <c>Level1</c> is less severe, + and <c>eq</c> if the levels are equal.</p> </desc> </func> <func> <name name="format_report" arity="1"/> - <fsummary>Convert a log message on report form to {Format,Args}.</fsummary> + <fsummary>Convert a log message on report form to {Format, Args}.</fsummary> <desc> - <p>Convert a log message on report form to <c>{Format,Args}</c>.</p> - <p>This is the default report callback used + <p>Convert a log message on report form to <c>{Format, + Args}</c>. This is the default report callback used by <seealso marker="logger_formatter"> <c>logger_formatter</c></seealso> when no custom report - callback is found.</p> + callback is found. See + section <seealso marker="logger_chapter#log_message">Log + Message</seealso> in the Kernel User's Guide for + information about report callbacks and valid forms of log + messages.</p> <p>The function produces lines of <c>Key: Value</c> from key-value lists. Strings are printed with <c>~ts</c> and other terms with <c>~tp</c>.</p> - <p>If the <c><anno>Report</anno></c> is a map, it is - converted to a key-value list before formatting as such.</p> + <p>If <c><anno>Report</anno></c> is a map, it is converted to + a key-value list before formatting as such.</p> </desc> </func> - </funcs> <section> @@ -904,18 +994,20 @@ logger:set_process_metadata(maps:merge(logger:get_process_metadata(),Meta)). <funcs> <func> - <name>HModule:adding_handler(Config1) -> {ok,Config2} | {error,Reason}</name> + <name>HModule:adding_handler(Config1) -> {ok, Config2} | {error, + Reason}</name> <fsummary>An instance of this handler is about to be added.</fsummary> <type> <v>Config1 = Config2 = - <seealso marker="#type-config">config()</seealso></v> + <seealso marker="#type-handler_config">handler_config()</seealso></v> <v>Reason = term()</v> </type> <desc> <p>This callback function is optional.</p> - <p>The function is called when an new handler is about to be - added, and the purpose is to verify the configuration and - initiate all resources needed by the handler.</p> + <p>The function is called on a temporary process when an new + handler is about to be added. The purpose is to verify the + configuration and initiate all resources needed by the + handler.</p> <p>The handler identity is associated with the <c>id</c> key in <c>Config1</c>.</p> <p>If everything succeeds, the callback function can add @@ -928,18 +1020,18 @@ logger:set_process_metadata(maps:merge(logger:get_process_metadata(),Meta)). </func> <func> - <name>HModule:changing_config(Config1,Config2) -> {ok,Config3} | {error,Reason}</name> + <name>HModule:changing_config(Config1, Config2) -> {ok, Config3} | {error, Reason}</name> <fsummary>The configuration for this handler is about to change.</fsummary> <type> <v>Config1 = Config2 = Config3 = - <seealso marker="#type-config">config()</seealso></v> + <seealso marker="#type-handler_config">handler_config()</seealso></v> <v>Reason = term()</v> </type> <desc> <p>This callback function is optional.</p> - <p>The function is called when the configuration for a handler - is about to change, and the purpose is to verify and act on - the new configuration.</p> + <p>The function is called on a temporary process when the + configuration for a handler is about to change. The purpose + is to verify and act on the new configuration.</p> <p><c>Config1</c> is the existing configuration and <c>Config2</c> is the new configuration.</p> <p>The handler identity is associated with the <c>id</c> key @@ -952,19 +1044,20 @@ logger:set_process_metadata(maps:merge(logger:get_process_metadata(),Meta)). </func> <func> - <name>HModule:log(LogEvent,Config) -> void()</name> + <name>HModule:log(LogEvent, Config) -> void()</name> <fsummary>Log the given log event.</fsummary> <type> <v>LogEvent = <seealso marker="#type-log_event">log_event()</seealso></v> <v>Config = - <seealso marker="#type-config">config()</seealso></v> + <seealso marker="#type-handler_config">handler_config()</seealso></v> </type> <desc> <p>This callback function is mandatory.</p> - <p>The function is called when all global filters and all + <p>The function is called when all primary filters and all handler filters for the handler in question have passed for - the given log event.</p> + the given log event. It is called on the client process, that + is, the process that issued the log event.</p> <p>The handler identity is associated with the <c>id</c> key in <c>Config</c>.</p> <p>The handler must log the event.</p> @@ -978,13 +1071,13 @@ logger:set_process_metadata(maps:merge(logger:get_process_metadata(),Meta)). <fsummary>The given handler is about to be removed.</fsummary> <type> <v>Config = - <seealso marker="#type-config">config()</seealso></v> + <seealso marker="#type-handler_config">handler_config()</seealso></v> </type> <desc> <p>This callback function is optional.</p> - <p>The function is called when a handler is about to be - removed, and the purpose is to release all resources used by - the handler.</p> + <p>The function is called on a temporary process when a + handler is about to be removed. The purpose is to release + all resources used by the handler.</p> <p>The handler identity is associated with the <c>id</c> key in <c>Config</c>.</p> <p>The return value is ignored by Logger.</p> @@ -1002,18 +1095,30 @@ logger:set_process_metadata(maps:merge(logger:get_process_metadata(),Meta)). <funcs> <func> - <name>FModule:check_config(FConfig) -> ok | {error,term()}</name> + <name>FModule:check_config(FConfig) -> ok | {error, Reason}</name> <fsummary>Validate the given formatter configuration.</fsummary> <type> <v>FConfig = <seealso marker="#type-formatter_config">formatter_config()</seealso></v> + <v>Reason = term()</v> </type> <desc> <p>This callback function is optional.</p> <p>The function is called by a Logger when formatter configuration is set or modified. The formatter must validate the given configuration and return <c>ok</c> if it - is correct, and <c>{error,term()}</c> if it is faulty.</p> + is correct, and <c>{error,Reason}</c> if it is faulty.</p> + <p>The following Logger API functions can trigger this callback:</p> + <list> + <item><seealso marker="logger#add_handler-3"> + <c>logger:add_handler/3</c></seealso></item> + <item><seealso marker="logger#set_handler_config-2"> + <c>logger:set_handler_config/2,3</c></seealso></item> + <item><seealso marker="logger#update_handler_config-2"> + <c>logger:updata_handler_config/2</c></seealso></item> + <item><seealso marker="logger#update_formatter_config-2"> + <c>logger:update_formatter_config/2</c></seealso></item> + </list> <p>See <seealso marker="logger_formatter"> <c>logger_formatter(3)</c></seealso> for an example implementation. <c>logger_formatter</c> is the @@ -1021,7 +1126,7 @@ logger:set_process_metadata(maps:merge(logger:get_process_metadata(),Meta)). </desc> </func> <func> - <name>FModule:format(LogEvent,FConfig) -> FormattedLogEntry</name> + <name>FModule:format(LogEvent, FConfig) -> FormattedLogEntry</name> <fsummary>Format the given log event.</fsummary> <type> <v>LogEvent = @@ -1045,6 +1150,20 @@ logger:set_process_metadata(maps:merge(logger:get_process_metadata(),Meta)). </desc> </func> </funcs> + + <section> + <title>See Also</title> + <p> + <seealso marker="config"><c>config(4)</c></seealso>, + <seealso marker="erts:erlang"><c>erlang(3)</c></seealso>, + <seealso marker="stdlib:io"><c>io(3)</c></seealso>, + <seealso marker="logger_disk_log_h"><c>logger_disk_log_h(3)</c></seealso>, + <seealso marker="logger_filters"><c>logger_filters(3)</c></seealso>, + <seealso marker="logger_formatter"><c>logger_formatter(3)</c></seealso>, + <seealso marker="logger_std_h"><c>logger_std_h(3)</c></seealso>, + <seealso marker="stdlib:unicode"><c>unicode(3)</c></seealso> + </p> + </section> </erlref> diff --git a/lib/kernel/doc/src/logger_arch.dia b/lib/kernel/doc/src/logger_arch.dia Binary files differnew file mode 100644 index 0000000000..97be31856e --- /dev/null +++ b/lib/kernel/doc/src/logger_arch.dia diff --git a/lib/kernel/doc/src/logger_arch.png b/lib/kernel/doc/src/logger_arch.png Binary files differindex 901122193a..70933a5a41 100644 --- a/lib/kernel/doc/src/logger_arch.png +++ b/lib/kernel/doc/src/logger_arch.png diff --git a/lib/kernel/doc/src/logger_chapter.xml b/lib/kernel/doc/src/logger_chapter.xml index a3eec7bd4b..c2cdf38a64 100644 --- a/lib/kernel/doc/src/logger_chapter.xml +++ b/lib/kernel/doc/src/logger_chapter.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2017</year> + <year>2017</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -30,7 +30,7 @@ <file>logger_chapter.xml</file> </header> - <p>Erlang/OTP 21.0 provides a new standard API for logging + <p>Erlang/OTP 21.0 provides a standard API for logging through <c>Logger</c>, which is part of the Kernel application. Logger consists of the API for issuing log events, and a customizable backend where log handlers, filters and @@ -44,23 +44,33 @@ <p>You can also configure the system so that the default handler prints log events to a single file, or to a set of wrap logs via <seealso marker="disk_log"><c>disk_log</c></seealso>.</p> - <p>By confiugration, you can aslo modify or disable the default + <p>By configuration, you can also modify or disable the default handler, replace it by a custom handler, and install additional handlers.</p> + <note> + <p>Since Logger is new in Erlang/OTP 21.0, we do reserve the right + to introduce changes to the Logger API and functionality in + patches following this release. These changes might or might not + be backwards compatible with the initial version.</p> + </note> + <section> <title>Overview</title> <p>A <em>log event</em> consists of a <em>log level</em>, the <em>message</em> to be logged, and <em>metadata</em>.</p> <p>The Logger backend forwards log events from the API, first - through a set of <em>global filters</em>, then through a set - of <em>handler filters</em> for each log handler.</p> + through a set of <em>primary filters</em>, then through a set of + secondary filters attached to each log handler. The secondary + filters are in the following named <em>handler filters</em>.</p> <p>Each filter set consists of a <em>log level check</em>, followed by zero or more <em>filter functions</em>.</p> - <p>The following figure show a conseptual overview of Logger. The + <p>The following figure shows a conceptual overview of Logger. The figure shows two log handlers, but any number of handlers can be installed.</p> + <!-- The image is edited with dia in logger_arch.dia file, + and .png file generated with make target 'png'. --> <image file="logger_arch.png"> <icaption>Conceptual Overview</icaption> </image> @@ -69,11 +79,11 @@ atoms are mapped to integer values, and a log event passes the log level check if the integer value of its log level is less than or equal to the currently configured log level. That is, - the check pases if the event is equally or more severe than the + the check passes if the event is equally or more severe than the configured level. See section <seealso marker="#log_level">Log Level</seealso> for a listing and description of all log levels.</p> - <p>The global log level can be overridden by a log level + <p>The primary log level can be overridden by a log level configured per module. This is to, for instance, allow more verbose logging from a specific part of the system.</p> <p>Filter functions can be used for more sophisticated filtering @@ -82,12 +92,12 @@ also modify all parts of the log event. See see section <seealso marker="#filters">Filters</seealso> for more details.</p> - <p>If a log event passes through all global filters and all - handler filters for a specific handler, Logger forwards the event - to the handler callback. The handler formats and prints the - event to its destination. See - section <seealso marker="#handlers">Handlers</seealso> for - more details.</p> + <p>If a log event passes through all primary filters and all + handler filters for a specific handler, Logger forwards the + event to the <em>handler callback</em>. The handler formats and + prints the event to its destination. See + section <seealso marker="#handlers">Handlers</seealso> for more + details.</p> <p>Everything up to and including the call to the handler callbacks is executed on the client process, that is, the process where the log event was issued. It is up to the handler @@ -96,6 +106,7 @@ defined.</p> </section> <section> + <marker id="logger_api"/> <title>Logger API</title> <p>The API for logging consists of a set of <seealso marker="logger#macros">macros</seealso>, and a set @@ -107,15 +118,16 @@ functions is that macros add location (originator) information to the metadata, and performs lazy evaluation by wrapping the logger call in a case statement, so it is only evaluated if the - log level of the event passes the global log level check.</p> + log level of the event passes the primary log level check.</p> <section> <marker id="log_level"/> <title>Log Level</title> <p>The log level indicates the severity of a event. In - accordance with the Syslog protocol, RFC-5424, eight log - levels can be specified. The following table lists all - possible log levels by name (atom), integer value, and - description:</p> + accordance with the Syslog protocol, + <url href="https://www.ietf.org/rfc/rfc5424.txt">RFC + 5424</url>, eight log levels can be specified. The following + table lists all possible log levels by name (atom), integer + value, and description:</p> <table align="left"> <row> @@ -136,7 +148,7 @@ <row> <cell>critical</cell> <cell align="center">2</cell> - <cell>critical contidions</cell> + <cell>critical conditions</cell> </row> <row> <cell>error</cell> @@ -178,14 +190,30 @@ message can consist of a format string and arguments (given as two separate parameters in the Logger API), a string or a report. The latter, which is either a map or a key-value list, - can be accompanied by a report callback specified in the log - event's <seealso marker="#metadata">metadata</seealso>. The - report callback is a convenience function that + can be accompanied by a <em>report callback</em> specified in + the log event's <seealso marker="#metadata">metadata</seealso>. + The report callback is a convenience function that the <seealso marker="#formatters">formatter</seealso> can use - to convert the report to a format string and arguments. The + to convert the report to a format string and arguments, or + directly to a string. The formatter can also use its own conversion function, if no callback is provided, or if a customized formatting is desired.</p> + <p>The report callback must be a fun with one or two + arguments. If it takes one argument, this is the report + itself, and the fun returns a format string and arguments:</p> + <pre>fun((<seealso marker="logger#type-report"><c>logger:report()</c></seealso>) -> {<seealso marker="stdlib:io#type-format"><c>io:format()</c></seealso>,[term()]})</pre> + <p>If it takes two arguments, the first is the report, and the + second is a map containing extra data that allows direct + coversion to a string:</p> + <pre>fun((<seealso marker="logger#type-report"><c>logger:report()</c></seealso>,<seealso marker="logger#type-report_cb_config"><c>logger:report_cb_config()</c></seealso>) -> <seealso marker="stdlib:unicode#type-chardata"><c>unicode:chardata()</c></seealso>) + </pre> + <p>The fun must obey the <c>encoding</c>, <c>depth</c> + and <c>chars_limit</c> parameters provided in the second + argument, as the formatter cannot do anything useful of these + parameters with the returned string. This variant is used when + the formatting of the report depends on the size and encoding + parameters.</p> <p>Example, format string and arguments:</p> <code>logger:error("The file does not exist: ~ts",[Filename])</code> <p>Example, string:</p> @@ -195,7 +223,7 @@ logger:debug(#{got => connection_request, id => Id, state => State}, #{report_cb => fun(R) -> {"~p",[R]} end})</code> <p>The log message can also be provided through a fun for lazy - evaluation. The fun is only evaluated if the global log level + evaluation. The fun is only evaluated if the primary log level check passes, and is therefore recommended if it is expensive to generate the message. The lazy fun must return a string, a report, or a tuple with format string and arguments.</p> @@ -212,14 +240,14 @@ logger:debug(#{got => connection_request, id => Id, state => State}, with <seealso marker="logger#set_process_metadata-1"> <c>logger:set_process_metadata/1</c></seealso> and <seealso marker="logger#update_process_metadata-1"> - <c>logger:update_process metadata/1</c></seealso>, + <c>logger:update_process_metadata/1</c></seealso>, respectively. This metadata applies to the process on which these calls are made, and Logger adds the metadata to all log events issued on that process.</p> </item> - <tag>Add metadata to a specifc log event</tag> + <tag>Add metadata to a specific log event</tag> <item> - <p>Metadata associated with one specifc log event is given + <p>Metadata associated with one specific log event is given as the last parameter to the log macro or Logger API function when the event is issued. For example:</p> <code>?LOG_ERROR("Connection closed",#{context => server})</code> @@ -235,8 +263,8 @@ logger:debug(#{got => connection_request, id => Id, state => State}, <section> <marker id="filter"/> <title>Filters</title> - <p>Filters can be global, or attached to a specific - handler. Logger calls the global filters first, and if they all + <p>Filters can be primary, or attached to a specific + handler. Logger calls the primary filters first, and if they all pass, it calls the handler filters for each handler. Logger calls the handler callback only if all filters attached to the handler in question also pass.</p> @@ -251,7 +279,7 @@ logger:debug(#{got => connection_request, id => Id, state => State}, <p>The filter function can return <c>stop</c>, <c>ignore</c> or the (possibly modified) log event.</p> <p>If <c>stop</c> is returned, the log event is immediately - discarded. If the filter is global, no handler filters or + discarded. If the filter is primary, no handler filters or callbacks are called. If it is a handler filter, the corresponding handler callback is not called, but the log event is forwarded to filters attached to the next handler, if @@ -265,23 +293,21 @@ logger:debug(#{got => connection_request, id => Id, state => State}, <p>If the filter function returns <c>ignore</c>, it means that it did not recognize the log event, and thus leaves to other filters to decide the event's destiny.</p> - <p>The configuration - option <seealso marker="#filter_default"><c>filter_default</c></seealso> - specifies the behaviour if all filter functions - return <c>ignore</c>, or if no filters - exist. <c>filter_default</c> is by default set to <c>log</c>, - meaning that if all existing filters ignore a log event, Logger - forwards the event to the handler + <p>The configuration option <c>filter_default</c> specifies the + behaviour if all filter functions return <c>ignore</c>, or if no + filters exist. <c>filter_default</c> is by default set + to <c>log</c>, meaning that if all existing filters ignore a log + event, Logger forwards the event to the handler callback. If <c>filter_default</c> is set to <c>stop</c>, Logger discards such events.</p> - <p>Global filters are added - with <seealso marker="logger#add_logger_filter-2"> - <c>logger:add_logger_filter/2</c></seealso> + <p>Primary filters are added + with <seealso marker="logger#add_primary_filter-2"> + <c>logger:add_primary_filter/2</c></seealso> and removed - with <seealso marker="logger#remove_logger_filter-1"> - <c>logger:remove_logger_filter/1</c></seealso>. They can also + with <seealso marker="logger#remove_primary_filter-1"> + <c>logger:remove_primary_filter/1</c></seealso>. They can also be added at system start via the Kernel configuration - parameter <seealso marker="#logger"><c>logger</c></seealso>.</p> + parameter <seealso marker="#logger_parameter"><c>logger</c></seealso>.</p> <p>Handler filters are added with <seealso marker="logger#add_handler_filter-3"> <c>logger:add_handler_filter/3</c></seealso> @@ -292,12 +318,13 @@ logger:debug(#{got => connection_request, id => Id, state => State}, with <seealso marker="logger#add_handler/3"> <c>logger:add_handler/3</c></seealso> or via the Kernel configuration - parameter <seealso marker="#logger"><c>logger</c></seealso>.</p> + parameter <seealso marker="#logger_parameter"><c>logger</c></seealso>.</p> <p>To see which filters are currently installed in the system, - use <seealso marker="logger#i-0"><c>logger:i/0</c></seealso>, - or <seealso marker="logger#get_logger_config-0"> - <c>logger:get_logger_config/0</c></seealso> + use <seealso marker="logger#get_config-0"> + <c>logger:get_config/0</c></seealso>, + or <seealso marker="logger#get_primary_config-0"> + <c>logger:get_primary_config/0</c></seealso> and <seealso marker="logger#get_handler_config-1"> <c>logger:get_handler_config/1</c></seealso>. Filters are listed in the order they are applied, that is, the first @@ -337,12 +364,12 @@ logger:debug(#{got => connection_request, id => Id, state => State}, <marker id="handlers"/> <title>Handlers</title> <p>A handler is defined as a module exporting at least the - following function:</p> + following callback function:</p> <pre><seealso marker="logger#HModule:log-2">log(LogEvent, Config) -> void()</seealso></pre> <p>This function is called when a log event has passed through all - global filters, and all handler filters attached to the handler + primary filters, and all handler filters attached to the handler in question. The function call is executed on the client process, and it is up to the handler implementation if other processes are involved or not.</p> @@ -358,8 +385,8 @@ logger:debug(#{got => connection_request, id => Id, state => State}, functions <c>adding_handler/1</c>, <c>changing_config/2</c> and <c>removing_handler/1</c>. See section <seealso marker="logger#handler_callback_functions">Handler - Callback Functions</seealso> in the logger(3) manual for more - information about these function.</p> + Callback Functions</seealso> in the logger(3) manual page for + more information about these function.</p> <p>The following built-in handlers exist:</p> @@ -404,7 +431,7 @@ logger:debug(#{got => connection_request, id => Id, state => State}, which is passed as the second argument to <seealso marker="logger#HModule:log-2"> <c>HModule:log/2</c></seealso>.</p> - <p>The formatter information consits of a formatter + <p>The formatter information consist of a formatter module, <c>FModule</c> and its configuration, <c>FConfig</c>. <c>FModule</c> must export the following function, which can be called by the handler:</p> @@ -425,152 +452,92 @@ logger:debug(#{got => connection_request, id => Id, state => State}, function when the formatter information is set or modified, to verify the validity of the formatter configuration.</p> <p>If no formatter information is specified for a handler, Logger - uses <seealso marker="logger_formatter"> - <c>logger_formatter(3)</c></seealso> as default.</p> + uses <c>logger_formatter</c> as default. See + the <seealso marker="logger_formatter"><c>logger_formatter(3)</c></seealso> + manual page for more information about this module.</p> </section> <section> <title>Configuration</title> - <p>Logger can be configured either when the system starts through - <seealso marker="config">configuration parameters</seealso>, - or at run-time by using the <seealso marker="logger">logger(3)</seealso> - API. The recommended approach is to do the initial configuration in - the <c>sys.config</c> file and then use the API when some configuration - has to be changed at runtime, such as the log level.</p> + <p>At system start, Logger is configured through Kernel + configuration parameters. The parameters that apply to Logger + are described in + section <seealso marker="#kernel_config_params">Kernel + Configuration Parameters</seealso>. Examples are found in + section <seealso marker="#config_examples">Configuration + Examples</seealso>.</p> + <p>During runtime, Logger configuration is changed via API + functions. See + section <seealso marker="logger#configuration_API">Configuration + API Functions</seealso> in the <c>logger(3)</c> manual page.</p> <section> - <title>Kernel Configuration Parameters</title> - <p>Logger is best configured by using the configuration parameters - of Kernel. There are four possible configuration parameters: - <seealso marker="#logger"><c>logger</c></seealso>, - <seealso marker="kernel_app#logger_level"><c>logger_level</c></seealso>, - <seealso marker="kernel_app#logger_sasl_compatible"><c>logger_sasl_compatible</c></seealso> and - <seealso marker="kernel_app#logger_progress_reports"><c>logger_progress_reports</c></seealso>. - <c>logger_level</c>, <c>logger_sasl_compatible</c> and <c>logger_progress_reports</c> are described in the - <seealso marker="kernel_app#configuration">Kernel Configuration</seealso>, - while <c>logger</c> is described below.</p> - - <marker id="logger"/> - <p><em>logger</em></p> - <p>The application configuration parameter <c>logger</c> is used to configure - three different Logger aspects; handlers, logger filters and module levels. - The configuration is a list containing tagged tuples that look like this:</p> - <taglist> - <tag><c>DisableHandler = {handler,default,undefined}</c></tag> - <item> - <p>Disable the default handler. This allows another application - to add its own default handler. See <seealso marker="logger#add_handlers/1"> - <c>logger:add_handlers/1</c></seealso> for more details.</p> - <p>Only one entry of this option is allowed.</p></item> - <tag><c>AddHandler = {handler,HandlerId,Module,HandlerConfig}</c></tag> - <item> - <p>Add a handler as if <seealso marker="logger:add_handler/3"> - <c>logger:add_handler(HandlerId,Module,HandlerConfig)</c></seealso> is - called.</p> - <p>It is allowed to have multiple entries of this option.</p></item> - <tag><c>Filters = {filters, default, [Filter]}</c><br/> - <c>FilterDefault = log | stop</c><br/> - <c>Filter = {FilterId, {FilterFun, FilterConfig}}</c></tag> - <item> - <p>Add the specified <seealso marker="logger#add_logger_filter/2"> - logger filters</seealso>.</p> - <p>Only one entry of this option is allowed.</p></item> - <tag><c>ModuleLevel = {module_level, Level, [Module]}</c></tag> - <item> - <p>This option configures <seealso marker="logger#set_module_level/2"> - module log level</seealso>.</p> - <p>It is allowed to have multiple entries of this option.</p></item> - </taglist> - <p>Examples:</p> - <list> - <item> - <p>Output logs into the file "logs/erlang.log"</p> - <code> -[{kernel, - [{logger, - [{handler, default, logger_std_h, - #{ logger_std_h => #{ type => {file,"log/erlang.log"}}}}]}]}]. - </code> - </item> - <item> - <p>Output logs in single line format</p> - <code> -[{kernel, - [{logger, - [{handler, default, logger_std_h, - #{ formatter => { logger_formatter,#{ single_line => true}}}}]}]}]. - </code> - </item> - <item> - <p>Add the pid to each log event</p> - <code> -[{kernel, - [{logger, - [{handler, default, logger_std_h, - #{ formatter => { logger_formatter, - #{ template => [time," ",pid," ",msg,"\n"]}} - }}]}]}]. - </code> - </item> - <item> - <p>Use a different file for debug logging</p> - <code> -[{kernel, - [{logger, - [{handler, default, logger_std_h, - #{ level => error, - logger_std_h => #{ type => {file, "log/erlang.log"}}}}, - {handler, info, logger_std_h, - #{ level => debug, - logger_std_h => #{ type => {file, "log/debug.log"}}}} - ]}]}]. - </code> - </item> - </list> - </section> - - <section> - <title>Global Logger Configuration</title> - + <title>Primary Logger Configuration</title> + <p>Logger API functions that apply to the primary Logger + configuration are:</p> + <list> + <item><seealso marker="logger#get_primary_config-0"> + <c>get_primary_config/0</c></seealso></item> + <item><seealso marker="logger#set_primary_config-1"> + <c>set_primary_config/1,2</c></seealso></item> + <item><seealso marker="logger#update_primary_config-1"> + <c>update_primary_config/1</c></seealso></item> + <item><seealso marker="logger#add_primary_filter-2"> + <c>add_primary_filter/2</c></seealso></item> + <item><seealso marker="logger#remove_primary_filter-1"> + <c>remove_primary_filter/1</c></seealso></item> + </list> + <p>The primary Logger configuration is a map with the following + keys:</p> <taglist> - <tag><c>level = </c><seealso marker="logger#type-level"> - <c>logger:level()</c></seealso></tag> + <tag><marker id="primary_level"/> + <c>level = </c><seealso marker="logger#type-level"> + <c>logger:level()</c></seealso><c> | all | none</c></tag> <item> - <p>Specifies the global log level to log.</p> + <p>Specifies the primary log level, that is, log event that + are equally or more severe than this level, are forwarded + to the primary filters. Less severe log events are + immediately discarded.</p> <p>See section <seealso marker="#log_level">Log Level</seealso> for a listing and description of possible log levels.</p> <p>The initial value of this option is set by the Kernel - configuration - parameter <seealso marker="kernel_app#logger_level"> - <c>logger_level</c></seealso>. It can be changed during - runtime - with <seealso marker="logger#set_logger_config-2"> - <c>logger:set_logger_config(level,NewLevel)</c></seealso>.</p> + configuration parameter <seealso marker="#logger_level"> + <c>logger_level</c></seealso>. It is changed during + runtime with <seealso marker="logger#set_primary_config-2"> + <c>logger:set_primary_config(level,Level)</c></seealso>.</p> + <p>Defaults to <c>info</c>.</p> </item> - <tag><c>filters = [{</c><seealso marker="logger#type-filter_id"> - <c>logger:filter_id()</c></seealso><c>,</c> - <seealso marker="logger#type-filter"> - <c>logger:filter()</c></seealso><c>}]</c></tag> + <tag><c>filters = [{FilterId,Filter}]</c></tag> <item> - <p>Global filters are added and removed with - <seealso marker="logger#add_logger_filter-2"> - <c>logger:add_logger_filter/2</c></seealso> and - <seealso marker="logger#remove_logger_filter-1"> - <c>logger:remove_logger_filter/1</c></seealso>, + <p>Specifies the primary filters.</p> + <list> + <item><c>FilterId = </c><seealso marker="logger#type-filter_id"> + <c>logger:filter_id()</c></seealso></item> + <item><c>Filter = </c><seealso marker="logger#type-filter"> + <c>logger:filter()</c></seealso></item> + </list> + <p>The initial value of this option is set by the Kernel + configuration + parameter <seealso marker="#logger_parameter"><c>logger</c></seealso>. + During runtime, primary filters are added and removed with + <seealso marker="logger#add_primary_filter-2"> + <c>logger:add_primary_filter/2</c></seealso> and + <seealso marker="logger#remove_primary_filter-1"> + <c>logger:remove_primary_filter/1</c></seealso>, respectively.</p> <p>See section <seealso marker="#filters">Filters</seealso> - for more information.</p> - <p>Default is <c>[]</c>, that is, no filters exist.</p> + for more detailed information.</p> + <p>Defaults to <c>[]</c>.</p> </item> - <tag><marker id="filter_default"/><c>filter_default = log | stop</c></tag> + <tag><c>filter_default = log | stop</c></tag> <item> - <p>Specifies what to do with an event if all filters + <p>Specifies what happens to a log event if all filters return <c>ignore</c>, or if no filters exist.</p> <p>See section <seealso marker="#filters">Filters</seealso> for more information about how this option is used.</p> - <p>Default is <c>log</c>.</p> + <p>Defaults to <c>log</c>.</p> </item> </taglist> </section> @@ -578,26 +545,64 @@ logger:debug(#{got => connection_request, id => Id, state => State}, <section> <marker id="handler_configuration"/> <title>Handler Configuration</title> + <p>Logger API functions that apply to handler configuration + are:</p> + <list> + <item><seealso marker="logger#get_handler_config-0"> + <c>get_handler_config/0,1</c></seealso></item> + <item><seealso marker="logger#set_handler_config-2"> + <c>set_handler_config/2,3</c></seealso></item> + <item><seealso marker="logger#update_handler_config-2"> + <c>update_handler_config/2</c></seealso></item> + <item><seealso marker="logger#add_handler_filter-3"> + <c>add_handler_filter/3</c></seealso></item> + <item><seealso marker="logger#remove_handler_filter-2"> + <c>remove_handler_filter/2</c></seealso></item> + <item><seealso marker="logger#update_formatter_config-2"> + <c>update_formatter_config/2,3</c></seealso></item> + </list> + <p>The configuration for a handler is a map with the following keys:</p> <taglist> + <tag><c>id = </c><seealso marker="logger#type-handler_id"> + <c>logger:handler_id()</c></seealso></tag> + <item> + <p>Automatically inserted by Logger. The value is the same + as the <c>HandlerId</c> specified when adding the handler, + and it cannot be changed.</p> + </item> + <tag><c>module = module()</c></tag> + <item> + <p>Automatically inserted by Logger. The value is the same + as the <c>Module</c> specified when adding the handler, + and it cannot be changed.</p> + </item> <tag><c>level = </c><seealso marker="logger#type-level"> - <c>logger:level()</c></seealso></tag> + <c>logger:level()</c></seealso><c> | all | none</c></tag> <item> - <p>Specifies the log level which the handler logs.</p> + <p>Specifies the log level for the handler, that is, log + events that are equally or more severe than this level, + are forwarded to the handler filters for this + handler.</p> <p>See section <seealso marker="#log_level">Log Level</seealso> for a listing and description of possible log levels.</p> - <p>The log level can be specified when adding the handler, - or changed during runtime with, for + <p>The log level is specified when adding the handler, or + changed during runtime with, for instance, <seealso marker="logger#set_handler_config/3"> - <c>logger:set_handler_config/3</c></seealso>.</p> - <p>Default is <c>info</c>.</p> + <c>logger:set_handler_config(HandlerId,level,Level)</c></seealso>. + </p> + <p>Defaults to <c>all</c>.</p> </item> - <tag><c>filters = [{</c><seealso marker="logger#type-filter_id"> - <c>logger:filter_id()</c></seealso><c>,</c> - <seealso marker="logger#type-filter"> - <c>logger:filter()</c></seealso><c>}]</c></tag> + <tag><c>filters = [{FilterId,Filter}]</c></tag> <item> - <p>Handler filters can be specified when adding the handler, + <p>Specifies the handler filters.</p> + <list> + <item><c>FilterId = </c><seealso marker="logger#type-filter_id"> + <c>logger:filter_id()</c></seealso></item> + <item><c>Filter = </c><seealso marker="logger#type-filter"> + <c>logger:filter()</c></seealso></item> + </list> + <p>Handler filters are specified when adding the handler, or added or removed during runtime with <seealso marker="logger#add_handler_filter-3"> <c>logger:add_handler_filter/3</c></seealso> and @@ -605,49 +610,214 @@ logger:debug(#{got => connection_request, id => Id, state => State}, <c>logger:remove_handler_filter/2</c></seealso>, respectively.</p> <p>See <seealso marker="#filters">Filters</seealso> for more - information.</p> - <p>Default is <c>[]</c>, that is, no filters exist.</p> + detailed information.</p> + <p>Defaults to <c>[]</c>.</p> </item> - <tag><marker id="filter_default"/><c>filter_default = log | stop</c></tag> + <tag><c>filter_default = log | stop</c></tag> <item> - <p>Specifies what to do with an event if all filters + <p>Specifies what happens to a log event if all filters return <c>ignore</c>, or if no filters exist.</p> <p>See section <seealso marker="#filters">Filters</seealso> for more information about how this option is used.</p> - <p>Default is <c>log</c>.</p> + <p>Defaults to <c>log</c>.</p> </item> - <tag><c>formatter = {module(),</c><seealso marker="logger#type-formatter_config"> - <c>logger:formatter_config()</c></seealso><c>}</c></tag> + <tag><c>formatter = {FormatterModule,FormatterConfig}</c></tag> <item> - <p>The formatter which the handler can use for converting - the log event term to a printable string.</p> - <p>See <seealso marker="#formatters">Formatters</seealso> for more - information.</p> - <p>Default - is <c>{logger_formatter,DefaultFormatterConfig}</c>, see + <p>Specifies a formatter that the handler can use for + converting the log event term to a printable string.</p> + <list> + <item><c>FormatterModule = module()</c></item> + <item><c>FormatterConfig = </c> + <seealso marker="logger#type-formatter_config"> + <c>logger:formatter_config()</c></seealso></item> + </list> + <p>The formatter information is specified when adding the + handler. The formatter configuration can be changed during + runtime + with <seealso marker="logger#update_formatter_config-2"> + <c>logger:update_formatter_config/2,3</c></seealso>, + or the complete formatter information can be overwritten + with, for + instance, <seealso marker="logger#set_handler_config-3"> + <c>logger:set_handler_config/3</c></seealso>.</p> + <p>See + section <seealso marker="#formatters">Formatters</seealso> + for more detailed information.</p> + <p>Defaults + to <c>{logger_formatter,DefaultFormatterConfig}</c>. See the <seealso marker="logger_formatter"> - <c>logger_formatter(3)</c></seealso> - manual for information about this formatter and its - default configuration.</p> + <c>logger_formatter(3)</c></seealso> manual page for + information about this formatter and its default + configuration.</p> </item> - <tag><c>HandlerConfig, atom() = term()</c></tag> - <item> - <p>Any keys not listed above are considered to be handler - specific configuration. The configuration of the Kernel - handlers can be found in + <tag><c>config = term()</c></tag> + <item> + <p>Handler specific configuration, that is, configuration + data related to a specific handler implementation.</p> + <p>The configuration for the built-in handlers is described + in the <seealso marker="logger_std_h"><c>logger_std_h(3)</c></seealso> and <seealso marker="logger_disk_log_h"><c>logger_disk_log_h(3)</c> </seealso> manual pages.</p> - </item> + </item> </taglist> <p>Notice that <c>level</c> and <c>filters</c> are obeyed by Logger itself before forwarding the log events to each - handler, while <c>formatter</c> and all handle specific + handler, while <c>formatter</c> and all handler specific options are left to the handler implementation.</p> - <p>All Logger's built-in handlers will call the given formatter - before printing.</p> + </section> + + <section> + <marker id="kernel_config_params"/> + <title>Kernel Configuration Parameters</title> + + <p>The following Kernel configuration parameters apply to + Logger:</p> + <taglist> + <tag><marker id="logger_parameter"/><c>logger = [Config]</c></tag> + <item> + <p>Specifies the configuration + for <seealso marker="logger">Logger</seealso>, except the + primary log level, which is specified + with <seealso marker="#logger_level"><c>logger_level</c></seealso>, + and the compatibility + with <seealso marker="sasl:error_logging">SASL Error + Logging</seealso>, which is specified + with <seealso marker="#logger_sasl_compatible"> + <c>logger_sasl_compatible</c></seealso>.</p> + <p>With this parameter, you can modify or disable the default + handler, add custom handlers and primary logger filters, and + set log levels per module.</p> + <p><c>Config</c> is any (zero or more) of the following:</p> + <taglist> + <tag><c>{handler, default, undefined}</c></tag> + <item> + <p>Disables the default handler. This allows another + application to add its own default handler.</p> + <p>Only one entry of this type is allowed.</p> + </item> + <tag><c>{handler, HandlerId, Module, HandlerConfig}</c></tag> + <item> + <p>If <c>HandlerId</c> is <c>default</c>, then this entry + modifies the default handler, equivalent to calling</p> + <pre><seealso marker="logger#set_handler_config-2"> + logger:set_handler_config(default, Module, HandlerConfig) + </seealso></pre> + <p>For all other values of <c>HandlerId</c>, this entry + adds a new handler, equivalent to calling</p> + <pre><seealso marker="logger:add_handler/3"> + logger:add_handler(HandlerId, Module, HandlerConfig) + </seealso></pre> + <p>Multiple entries of this type are allowed.</p></item> + <tag><c>{filters, FilterDefault, [Filter]}</c></tag> + <item> + <p>Adds the specified primary filters.</p> + <list> + <item><c>FilterDefault = log | stop</c></item> + <item><c>Filter = {FilterId, {FilterFun, FilterConfig}}</c></item> + </list> + <p>Equivalent to calling</p> + <pre><seealso marker="logger#add_primary_filter/2"> + logger:add_primary_filter(FilterId, {FilterFun, FilterConfig}) + </seealso></pre> + <p>for each <c>Filter</c>.</p> + <p><c>FilterDefault</c> specifies the behaviour if all + primary filters return <c>ignore</c>, see + section <seealso marker="#filters">Filters</seealso>.</p> + <p>Only one entry of this type is allowed.</p> + </item> + <tag><c>{module_level, Level, [Module]}</c></tag> + <item> + <p>Sets module log level for the given modules. Equivalent + to calling</p> + <pre><seealso marker="logger#set_module_level/2"> + logger:set_module_level(Module, Level)</seealso></pre> + <p>for each <c>Module</c>.</p> + <p>Multiple entries of this type are allowed.</p> + </item> + </taglist> + <p>See + section <seealso marker="#config_examples">Configuration + Examples</seealso> for examples using the <c>logger</c> + parameter for system configuration.</p> + </item> + <tag><marker id="logger_level"/> + <c>logger_level = Level</c></tag> + <item> + <p>Specifies the primary log level. See + the <seealso marker="kernel_app#logger_level"><c>kernel(6)</c></seealso> + manual page for more information about this parameter.</p> + </item> + <tag><marker id="logger_sasl_compatible"/> + <c>logger_sasl_compatible = true | false</c></tag> + <item> + <p>Specifies Logger's compatibility + with <seealso marker="sasl:error_logging">SASL Error + Logging</seealso>. See + the <seealso marker="kernel_app#logger_sasl_compatible"> + <c>kernel(6)</c></seealso> manual page for more + information about this parameter.</p> + </item> + </taglist> + </section> + + <section> + <marker id="config_examples"/> + <title>Configuration Examples</title> + <p>The value of the Kernel configuration parameter <c>logger</c> + is a list of tuples. It is possible to write the term on the + command line when starting an erlang node, but as the term + grows, a better approach is to use the system configuration + file. See + the <seealso marker="config"><c>config(4)</c></seealso> manual + page for more information about this file.</p> + <p>Each of the following examples shows a simple system + configuration file that configures Logger according to the + description.</p> + <p>Modify the default handler to print to a file instead of + <c>standard_io</c>:</p> + <code> +[{kernel, + [{logger, + [{handler, default, logger_std_h, % {handler, HandlerId, Module, + #{config => #{type => {file,"log/erlang.log"}}}} % Config} + ]}]}]. + </code> + <p>Modify the default handler to print each log event as a + single line:</p> + <code> +[{kernel, + [{logger, + [{handler, default, logger_std_h, + #{formatter => {logger_formatter, #{single_line => true}}}} + ]}]}]. + </code> + <p>Modify the default handler to print the pid of the logging + process for each log event:</p> + <code> +[{kernel, + [{logger, + [{handler, default, logger_std_h, + #{formatter => {logger_formatter, + #{template => [time," ",pid," ",msg,"\n"]}}}} + ]}]}]. + </code> + <p>Modify the default handler to only print errors and more + severe log events to "log/erlang.log", and add another handler + to print all log events to "log/debug.log".</p> + <code> +[{kernel, + [{logger, + [{handler, default, logger_std_h, + #{level => error, + config => #{type => {file, "log/erlang.log"}}}}, + {handler, info, logger_std_h, + #{level => debug, + config => #{type => {file, "log/debug.log"}}}} + ]}]}]. + </code> </section> </section> @@ -687,8 +857,9 @@ logger:debug(#{got => connection_request, id => Id, state => State}, <p>To get log events on the same format as produced by <c>error_logger_tty_h</c> and <c>error_logger_file_h</c>, use the default formatter, <c>logger_formatter</c>, with - configuration parameter <c>legacy_header => true</c>. This is - also the default.</p> + configuration parameter <c>legacy_header</c> set + to <c>true</c>. This is the default configuration of + the <c>default</c> handler started by Kernel.</p> </item> <tag>Default Format of Log Events from OTP</tag> <item> @@ -700,12 +871,11 @@ logger:debug(#{got => connection_request, id => Id, state => State}, <item> <p>By SASL reports we mean supervisor reports, crash reports and progress reports.</p> - <p>In earlier releases, these reports were only logged when - the SASL application was running, and they were printed - trough specific event handlers - named <c>sasl_report_tty_h</c> + <p>Prior to Erlang/OTP 21.0, these reports were only logged + when the SASL application was running, and they were printed + trough SASL's own event handlers <c>sasl_report_tty_h</c> and <c>sasl_report_file_h</c>.</p> - <p>The destination of these log events were configured by + <p>The destination of these log events was configured by <seealso marker="sasl:sasl_app#deprecated_error_logger_config">SASL configuration parameters</seealso>.</p> <p>Due to the specific event handlers, the output format @@ -716,17 +886,20 @@ logger:debug(#{got => connection_request, id => Id, state => State}, <list> <item>Supervisor reports, crash reports, and progress reports are no longer connected to the SASL application.</item> - <item>Supervisor reports and crash reports are logged by - default.</item> - <item>Progress reports are not logged by default, but can be - enabled with the Kernel configuration - parameter <seealso marker="kernel_app#logger_progress_reports"> - <c>logger_progress_reports</c></seealso>.</item> + <item>Supervisor reports and crash reports are issued + as <c>error</c> level log events, and are logged through + the default handler started by Kernel.</item> + <item>Progress reports are issued as <c>info</c> level log + events, and since the default primary log level + is <c>notice</c>, these are not logged by default. To + enable printing of progress reports, set + the <seealso marker="#primary_level">primary log + level</seealso> to <c>info</c>.</item> <item>The output format is the same for all log events.</item> </list> - <p>If the old behaviour is preferred, the Kernel configuation - parameter <seealso marker="kernel_app:logger_sasl_compatible"> + <p>If the old behaviour is preferred, the Kernel configuration + parameter <seealso marker="kernel_app#logger_sasl_compatible"> <c>logger_sasl_compatible</c></seealso> can be set to <c>true</c>. The <seealso marker="sasl:sasl_app#deprecated_error_logger_config">SASL @@ -734,11 +907,10 @@ logger:debug(#{got => connection_request, id => Id, state => State}, before, and the SASL reports will only be printed if the SASL application is running, through a second log handler named <c>sasl</c>.</p> - <p>All SASL reports have a metadata - field <c>domain => [beam,erlang,otp,sasl]</c>, which can be - used, for example, by filters to stop or allow the - log events.</p> - <p>See the <seealso marker="sasl:error_logging">SASL User's + <p>All SASL reports have a metadata field <c>domain</c> which + is set to <c>[otp,sasl]</c>. This field can be + used by filters to stop or allow the log events.</p> + <p>See section <seealso marker="sasl:error_logging">SASL User's Guide</seealso> for more information about the old SASL error logging functionality.</p> </item> @@ -749,21 +921,22 @@ logger:debug(#{got => connection_request, id => Id, state => State}, <code> error_logger:add_report_handler/1,2. </code> - <p>This will automatically start the <c>error_logger</c> - event manager, and add <c>error_logger</c> as a - handler to <c>logger</c>, with configuration</p> + <p>This automatically starts the error logger event manager, + and adds <c>error_logger</c> as a handler to Logger, with + the following configuration:</p> <code> #{level => info, filter_default => log, filters => []}. </code> - <p>Notice that this handler will ignore events that do not - originate from the <c>error_logger</c> API, or from within - OTP. This means that if your code uses the Logger API for - logging, then your log events will be discarded by this - handler.</p> - <p>Also notice that <c>error_logger</c> is not overload - protected.</p> + <note> + <p>This handler ignores events that do not originate from + the <c>error_logger</c> API, or from within OTP. This + means that if your code uses the Logger API for logging, + then your log events will be discarded by this + handler.</p> + <p>The handler is not overload protected.</p> + </note> </item> </taglist> </section> @@ -771,361 +944,375 @@ error_logger:add_report_handler/1,2. <section> <title>Error Handling</title> - <p>Log data is expected to be either a format string and - arguments, a string - (<seealso marker="stdlib:unicode#type-chardata"> - <c>unicode:chardata()</c></seealso>), or a report (map or - key-value list) which can be converted to a format string and - arguments by the handler. If a report is given, a default report - callback can be included in the log event's metadata. The - handler can use this callback for converting the report to a - format string and arguments. If the format obtained by the - provided callback is not desired, or if there is no provided - callback, the handler must do a custom conversion.</p> - <p>Logger does, to a certain extent, check its input data - before forwarding a log event to the handlers, but it does not - evaluate conversion funs or check the validity of format strings - and arguments. This means that any filter or handler must be - careful when formatting the data of a log event, making sure - that it does not crash due to bad input data or faulty - callbacks.</p> + <p>Logger does, to a certain extent, check its input data before + forwarding a log event to filters and handlers. It does, + however, not evaluate report callbacks, or check the validity of + format strings and arguments. This means that all filters and + handlers must be careful when formatting the data of a log + event, making sure that it does not crash due to bad input data + or faulty callbacks.</p> <p>If a filter or handler still crashes, Logger will remove the filter or handler in question from the configuration, and print a short error message to the terminal. A debug event containing - the crash reason and other details is also issued, and can be - seen if a handler logging debug events is installed.</p> + the crash reason and other details is also issued.</p> + <p>See section <seealso marker="#log_message">Log + Message</seealso> for more information about report callbacks + and valid forms of log messages.</p> </section> <section> - <title>Example: add a handler to log debug events to file</title> + <title>Example: Add a handler to log info events to file</title> <p>When starting an Erlang node, the default behaviour is that all - log events with level info and above are logged to the - terminal. In order to also log debug events, you can either - change the global log level to <c>debug</c> or add a separate - handler to take care of this. In this example we will add a new - handler which prints the debug events to a separate file.</p> - <p>First, we add an instance of <c>logger_std_h</c> with - type <c>{file,File}</c>, and we set the handler's level - to <c>debug</c>:</p> + log events on level <c>notice</c> or more severe, are logged to + the terminal via the default handler. To also log info events, + you can either change the primary log level to <c>info</c>:</p> <pre> -1> <input>Config = #{level => debug, logger_std_h => #{type => {file,"./debug.log"}}}.</input> -#{logger_std_h => #{type => {file,"./debug.log"}}, - level => debug} -2> <input>logger:add_handler(debug_handler,logger_std_h,Config).</input> +1> <input>logger:set_primary_config(level, info).</input> ok</pre> - <p>By default, the handler receives all events - (<c>filter_default=log</c>, see - section <seealso marker="#filters">Filters</seealso> for more - details), so we need to add a filter to stop all non-debug - events. The built-in - filter <seealso marker="logger_filters#level-2"> - <c>logger_filters:level/2</c></seealso> - is used for this:</p> + <p>or set the level for one or a few modules only:</p> <pre> -3> <input>logger:add_handler_filter(debug_handler,stop_non_debug, - {fun logger_filters:level/2,{stop,neq,debug}}).</input> +2> <input>logger:set_module_level(mymodule, info).</input> ok</pre> - <p>And finally, we need to make sure that Logger itself allows - debug events. This can either be done by setting the global - log level:</p> + <p>This allows info events to pass through to the default handler, + and be printed to the terminal as well. If there are many info + events, it can be useful to print these to a file instead.</p> + <p>First, set the log level of the default handler + to <c>notice</c>, preventing it from printing info events to the + terminal:</p> <pre> -4> <input>logger:set_logger_config(level,debug).</input> +3> <input>logger:set_handler_config(default, level, notice).</input> ok</pre> - <p>Or by allowing debug events from one or a few modules only:</p> + <p>Then, add a new handler which prints to file. You can use the + handler + module <seealso marker="logger_std_h"><c>logger_std_h</c></seealso>, + and specify type <c>{file,File}</c>.:</p> <pre> -5> <input>logger:set_module_level(mymodule,debug).</input> +4> <input>Config = #{config => #{type => {file,"./info.log"}}, level => info}.</input> +#{config => #{type => {file,"./info.log"}},level => info} +5> <input>logger:add_handler(myhandler, logger_std_h, Config).</input> ok</pre> + <p>Since <c>filter_default</c> defaults to <c>log</c>, this + handler now receives all log events. If you want info events + only in the file, you must add a filter to stop all non-info + events. The built-in + filter <seealso marker="logger_filters#level-2"> + <c>logger_filters:level/2</c></seealso> + can do this:</p> + <pre> +6> <input>logger:add_handler_filter(myhandler, stop_non_info, + {fun logger_filters:level/2, {stop, neq, info}}).</input> +ok</pre> + <p>See section <seealso marker="#filters">Filters</seealso> for + more information about the filters and the <c>filter_default</c> + configuration parameter.</p> </section> <section> - <title>Example: implement a handler</title> - <p>The only requirement that a handler MUST fulfill is to export - the following function:</p> - <code>log(logger:log_event(),logger:config()) -> ok</code> - <p>It can optionally also implement the following callbacks:</p> - <code> -adding_handler(logger:config()) -> {ok,logger:config()} | {error,term()} -removing_handler(logger:config()) -> ok -changing_config(logger:config(),logger:config()) -> {ok,logger:config()} | {error,term()} - </code> - <p>When <c>logger:add_handler(Id,Module,Config)</c> is called, + <title>Example: Implement a handler</title> + <p>Section <seealso marker="logger#handler_callback_functions">Handler + Callback Functions</seealso> in the logger(3) manual page + describes the callback functions that can be implemented for a + Logger handler.</p> + <p>A handler callback module must export:</p> + <list> + <item><c>log(Log, Config)</c></item> + </list> + <p>It can optionally also export some, or all, of the following:</p> + <list> + <item><c>adding_handler(Config)</c></item> + <item><c>removing_handler(Config)</c></item> + <item><c>changing_config(OldConfig, NewConfig)</c></item> + </list> + <p>When a handler is added, by for example a call + to <seealso marker="logger#add_handler-3"> + <c>logger:add_handler(Id, HModule, Config)</c></seealso>, Logger first calls <c>HModule:adding_handler(Config)</c>. If - this function returns <c>{ok,NewConfig}</c>, Logger - writes <c>NewConfig</c> to the configuration database, and + this function returns <c>{ok,Config1}</c>, Logger + writes <c>Config1</c> to the configuration database, and the <c>logger:add_handler/3</c> call returns. After this, the handler is installed and must be ready to receive log events as calls to <c>HModule:log/2</c>.</p> <p>A handler can be removed by calling - <c>logger:remove_handler(Id)</c>. Logger calls + <seealso marker="logger#remove_handler-1"> + <c>logger:remove_handler(Id)</c></seealso>. Logger calls <c>HModule:removing_handler(Config)</c>, and removes the handler's configuration from the configuration database.</p> - <p>When <c>logger:set_handler_config/2,3</c> - or <c>logger:update_handler_config/2</c> is called, Logger - calls <c>HModule:changing_config(OldConfig,NewConfig)</c>. If - this function returns <c>{ok,NewConfig}</c>, Logger - writes <c>NewConfig</c> to the configuration database.</p> + <p>When <seealso marker="logger#set_handler_config-2"> + <c>logger:set_handler_config/2,3</c></seealso> + or <seealso marker="logger#update_handler_config/2"> + <c>logger:update_handler_config/2</c></seealso> is called, + Logger + calls <c>HModule:changing_config(OldConfig, NewConfig)</c>. If + this function returns <c>{ok,NewConfig1}</c>, Logger + writes <c>NewConfig1</c> to the configuration database.</p> <p>A simple handler that prints to the terminal can be implemented as follows:</p> <code> --module(myhandler). +-module(myhandler1). -export([log/2]). -log(LogEvent,#{formatter:={FModule,FConfig}) -> - io:put_chars(FModule:format(LogEvent,FConfig)). +log(LogEvent, #{formatter := {FModule, FConfig}}) -> + io:put_chars(FModule:format(LogEvent, FConfig)). </code> - <p>A simple handler which prints to file could be implemented like - this:</p> - <code> --module(myhandler). --export([adding_handler/1, removing_handler/1, log/2]). --export([init/1, handle_call/3, handle_cast/2, terminate/2]). - -adding_handler(Config) -> - {ok,Fd} = file:open(File,[append,{encoding,utf8}]), - {ok,Config#{myhandler_fd => Fd}}. - -removing_handler(#{myhandler_fd:=Fd}) -> - _ = file:close(Fd), - ok. - -log(LogEvent,#{myhandler_fd:=Fd,formatter:={FModule,FConfig}}) -> - io:put_chars(Fd,FModule:format(LogEvent,FConfig)). - </code> - - <note><p>The above handlers do not have any overload + <p>Notice that the above handler does not have any overload protection, and all log events are printed directly from the - client process.</p></note> - + client process.</p> <p>For information and examples of overload protection, please refer to section <seealso marker="#overload_protection">Protecting the - Handler from Overload</seealso>, and the implementation - of <seealso marker="logger_std_h"><c>logger_std_h(3)</c></seealso> - and <seealso marker="logger_disk_log_h"><c>logger_disk_log_h(3)</c> + Handler from Overload</seealso>, and the implementation + of <seealso marker="logger_std_h"><c>logger_std_h</c></seealso> + and <seealso marker="logger_disk_log_h"><c>logger_disk_log_h</c> </seealso>.</p> - - <p>Below is a simpler example of a handler which logs through one - single process.</p> + <p>The following is a simpler example of a handler which logs to a + file through one single process:</p> <code> --module(myhandler). +-module(myhandler2). -export([adding_handler/1, removing_handler/1, log/2]). -export([init/1, handle_call/3, handle_cast/2, terminate/2]). adding_handler(Config) -> - {ok,Pid} = gen_server:start(?MODULE,Config), - {ok,Config#{myhandler_pid => Pid}}. + MyConfig = maps:get(config,Config,#{file => "myhandler2.log"}), + {ok, Pid} = gen_server:start(?MODULE, MyConfig, []), + {ok, Config#{config => MyConfig#{pid => Pid}}}. -removing_handler(#{myhandler_pid:=Pid}) -> +removing_handler(#{config := #{pid := Pid}}) -> gen_server:stop(Pid). -log(LogEvent,#{myhandler_pid:=Pid} = Config) -> - gen_server:cast(Pid,{log,LogEvent,Config}). +log(LogEvent,#{config := #{pid := Pid}} = Config) -> + gen_server:cast(Pid, {log, LogEvent, Config}). -init(#{myhandler_file:=File}) -> - {ok,Fd} = file:open(File,[append,{encoding,utf8}]), - {ok,#{file => File, fd => Fd}}. +init(#{file := File}) -> + {ok, Fd} = file:open(File, [append, {encoding, utf8}]), + {ok, #{file => File, fd => Fd}}. -handle_call(_,_,State) -> - {reply,{error,bad_request},State}. +handle_call(_, _, State) -> + {reply, {error, bad_request}, State}. -handle_cast({log,LogEvent,Config},#{fd:=Fd} = State) -> - do_log(Fd,LogEvent,Config), - {noreply,State}. +handle_cast({log, LogEvent, Config}, #{fd := Fd} = State) -> + do_log(Fd, LogEvent, Config), + {noreply, State}. -terminate(Reason,#{fd:=Fd}) -> +terminate(_Reason, #{fd := Fd}) -> _ = file:close(Fd), ok. -do_log(Fd,LogEvent,#{formatter:={FModule,FConfig}}) -> - String = FModule:format(LogEvent,FConfig), - io:put_chars(Fd,String). +do_log(Fd, LogEvent, #{formatter := {FModule, FConfig}}) -> + String = FModule:format(LogEvent, FConfig), + io:put_chars(Fd, String). </code> </section> <section> <marker id="overload_protection"/> <title>Protecting the Handler from Overload</title> - <p>In order for the built-in handlers to survive, and stay responsive, - during periods of high load (i.e. when huge numbers of incoming - log requests must be handled), a mechanism for overload protection - has been implemented in the - <seealso marker="logger_std_h"><c>logger_std_h</c></seealso> - and <seealso marker="logger_disk_log_h"><c>logger_disk_log_h</c> - </seealso> handler. The mechanism, used by both handlers, works - as follows:</p> + <p>The default handlers, <seealso marker="logger_std_h"> + <c>logger_std_h</c></seealso> and <seealso marker="logger_disk_log_h"> + <c>logger_disk_log_h</c></seealso>, feature an overload protection + mechanism, which makes it possible for the handlers to survive, + and stay responsive, during periods of high load (when huge + numbers of incoming log requests must be handled). + The mechanism works as follows:</p> <section> <title>Message Queue Length</title> <p>The handler process keeps track of the length of its message - queue and reacts in different ways depending on the current status. - The purpose is to keep the handler in, or (as quickly as possible), - get the handler into, a state where it can keep up with the pace - of incoming log requests. The memory usage of the handler must never - keep growing larger and larger, since that would eventually cause the - handler to crash. Three thresholds with associated actions have been - defined:</p> + queue and takes some form of action when the current length exceeds a + configurable threshold. The purpose is to keep the handler in, or to + as quickly as possible get the handler into, a state where it can + keep up with the pace of incoming log events. The memory use of the + handler must never grow larger and larger, since that will eventually + cause the handler to crash. These three thresholds, with associated + actions, exist:</p> <taglist> - <tag><c>toggle_sync_qlen</c></tag> + <tag><c>sync_mode_qlen</c></tag> <item> - <p>The default value of this level is <c>10</c> messages, - and as long as the length of the message queue is lower, all log - requests are handled asynchronously. This simply means that the - process sending the log request (by calling a log function in the - Logger API) does not wait for a response from the handler but - continues executing immediately after the request (i.e. it will not - be affected by the time it takes the handler to print to the log - device). If the message queue grows larger than this value, however, - the handler starts handling the log requests synchronously instead, - meaning the process sending the request will have to wait for a - response. When the handler manages to reduce the message queue to a - level below the <c>toggle_sync_qlen</c> threshold, asynchronous + <p>As long as the length of the message queue is lower than this + value, all log events are handled asynchronously. This means that + the client process sending the log event, by calling a log function + in the <seealso marker="logger_chapter#logger_api">Logger API</seealso>, + does not wait for a response from the handler but continues + executing immediately after the event is sent. It is not affected + by the time it takes the handler to print the event to the log + device. If the message queue grows larger than this value, + the handler starts handling log events synchronously instead, + meaning that the client process sending the event must wait for a + response. When the handler reduces the message queue to a + level below the <c>sync_mode_qlen</c> threshold, asynchronous operation is resumed. The switch from asynchronous to synchronous - mode will force the logging tempo of few busy senders to slow down, - but can not protect the handler sufficiently in situations of many - concurrent senders.</p> + mode can slow down the logging tempo of one, or a few, busy senders, + but cannot protect the handler sufficiently in a situation of many + busy concurrent senders.</p> + <p>Defaults to <c>10</c> messages.</p> </item> - <tag><c>drop_new_reqs_qlen</c></tag> + <tag><c>drop_mode_qlen</c></tag> <item> - <p>When the message queue has grown larger than this threshold, which - defaults to <c>200</c> messages, the handler switches to a mode in - which it drops any new requests being made. Dropping a message in - this state means that the log function never actually sends a message - to the handler. The log call simply returns without an action. When - the length of the message queue has been reduced to a level below this - threshold, synchronous or asynchronous request handling mode is - resumed.</p> + <p>When the message queue grows larger than this threshold, the + handler switches to a mode in which it drops all new events that + senders want to log. Dropping an event in this mode means that the + call to the log function never results in a message being sent to + the handler, but the function returns without taking any action. + The handler keeps logging the events that are already in its message + queue, and when the length of the message queue is reduced to a level + below the threshold, synchronous or asynchronous mode is resumed. + Notice that when the handler activates or deactivates drop mode, + information about it is printed in the log.</p> + <p>Defaults to <c>200</c> messages.</p> </item> - <tag><c>flush_reqs_qlen</c></tag> + <tag><c>flush_qlen</c></tag> <item> - <p>Above this threshold, which defaults to <c>1000</c> messages, a - flush operation takes place, in which all messages buffered in the - process mailbox get deleted without any logging actually taking - place. (Processes waiting for a response from a synchronous log request - will receive a reply indicating that the request has been dropped).</p> + <p>If the length of the message queue grows larger than this threshold, + a flush (delete) operation takes place. To flush events, the handler + discards the messages in the message queue by receiving them in a + loop without logging. Client processes waiting for a response from a + synchronous log request receive a reply from the handler indicating + that the request is dropped. The handler process increases its + priority during the flush loop to make sure that no new events + are received during the operation. Notice that after the flush operation + is performed, the handler prints information in the log about how many + events have been deleted.</p> + <p>Defaults to <c>1000</c> messages.</p> </item> </taglist> <p>For the overload protection algorithm to work properly, it is required that:</p> - <p><c>toggle_sync_qlen =< drop_new_reqs_qlen =< flush_reqs_qlen</c></p> + <p><c>sync_mode_qlen =< drop_mode_qlen =< flush_qlen</c></p> <p>and that:</p> - <p><c>drop_new_reqs_qlen > 1</c></p> + <p><c>drop_mode_qlen > 1</c></p> - <p>If <c>toggle_sync_qlen</c> is set to <c>0</c>, the handler will handle all - requests synchronously. Setting the value of <c>toggle_sync_qlen</c> to the same - as <c>drop_new_reqs_qlen</c>, disables the synchronous mode. Likewise, setting - the value of <c>drop_new_reqs_qlen</c> to the same as <c>flush_reqs_qlen</c>, - disables the drop mode.</p> + <p>To disable certain modes, do the following:</p> + <list> + <item>If <c>sync_mode_qlen</c> is set to <c>0</c>, all log events are handled + synchronously. That is, asynchronous logging is disabled.</item> + <item>If <c>sync_mode_qlen</c> is set to the same value as + <c>drop_mode_qlen</c>, synchronous mode is disabled. That is, the handler + always runs in asynchronous mode, unless dropping or flushing is invoked.</item> + <item>If <c>drop_mode_qlen</c> is set to the same value as <c>flush_qlen</c>, + drop mode is disabled and can never occur.</item> + </list> <p>During high load scenarios, the length of the handler message queue rarely grows in a linear and predictable way. Instead, whenever the - handler process gets scheduled in, it can have an almost arbitrary number - of messages waiting in the mailbox. It's for this reason that the overload - protection mechanism is focused on acting quickly and quite drastically - (such as immediately dropping or flushing messages) as soon as a large - queue length is detected. </p> - - <p>The thresholds listed above may be modified by the user if, e.g, a handler - shouldn't drop or flush messages unless the message queue length grows - extremely large. (The handler must be allowed to use large amounts of memory - under such circumstances however). Another example of when the user might want - to change the settings is if, for performance reasons, the logging processes must - never get blocked by synchronous log requests, while dropping or flushing requests - is perfectly acceptable (since it doesn't affect the performance of the - loggers).</p> + handler process is scheduled in, it can have an almost arbitrary number + of messages waiting in the message queue. It is for this reason that the overload + protection mechanism is focused on acting quickly, and quite drastically, + such as immediately dropping or flushing messages, when a large queue length + is detected.</p> + + <p>The values of the previously listed thresholds can be specified by the user. + This way, a handler can be configured to, for example, not drop or flush + messages unless the message queue length of the handler process grows extremely + large. Notice that large amounts of memory can be required for the node under such + circumstances. Another example of user configuration is when, for performance + reasons, the client processes must never be blocked by synchronous log requests. + It is possible, perhaps, that dropping or flushing events is still acceptable, since + it does not affect the performance of the client processes sending the log events.</p> <p>A configuration example:</p> <code type="none"> logger:add_handler(my_standard_h, logger_std_h, - #{logger_std_h => - #{type => {file,"./system_info.log"}, - toggle_sync_qlen => 100, - drop_new_reqs_qlen => 1000, - flush_reqs_qlen => 2000}}). + #{config => #{type => {file,"./system_info.log"}, + sync_mode_qlen => 100, + drop_mode_qlen => 1000, + flush_qlen => 2000}}). </code> </section> <section> <title>Controlling Bursts of Log Requests</title> - <p>A potential problem with large bursts of log requests, is that log files - may get full or wrapped too quickly (in the latter case overwriting - previously logged data that could be of great importance). For this reason, - both built-in handlers offer the possibility to set a maximum level of how - many requests to process with a certain time frame. With this burst control - feature enabled, the handler will take care of bursts of log requests - without choking log files, or the terminal, with massive amounts of - printouts. These are the configuration parameters:</p> - + <p>Large bursts of log events - many events received by the handler + under a short period of time - can potentially cause problems, such as:</p> + <list> + <item>Log files grow very large, very quickly.</item> + <item>Circular logs wrap too quickly so that important data is overwritten.</item> + <item>Write buffers grow large, which slows down file sync operations.</item> + </list> + + <p>For this reason, both built-in handlers offer the possibility to specify the + maximum number of events to be handled within a certain time frame. + With this burst control feature enabled, the handler can avoid choking the log with + massive amounts of printouts. The configuration parameters are:</p> <taglist> - <tag><c>enable_burst_limit</c></tag> + <tag><c>burst_limit_enable</c></tag> <item> - <p>This is set to <c>true</c> by default. The value <c>false</c> - disables the burst control feature.</p> + <p>Value <c>true</c> enables burst control and <c>false</c> disables it.</p> + <p>Defaults to <c>true</c>.</p> </item> - <tag><c>burst_limit_size</c></tag> + <tag><c>burst_limit_max_count</c></tag> <item> - <p>This is how many requests should be processed within the - <c>burst_window_time</c> time frame. After this maximum has been - reached, successive requests will be dropped until the end of the - time frame. The default value is <c>500</c> messages.</p> + <p>This is the maximum number of events to handle within a + <c>burst_limit_window_time</c> time frame. After the limit is + reached, successive events are dropped until the end of the time frame.</p> + <p>Defaults to <c>500</c> events.</p> </item> - <tag><c>burst_window_time</c></tag> + <tag><c>burst_limit_window_time</c></tag> <item> - <p>The default window is <c>1000</c> milliseconds long.</p> + <p>See the previous description of <c>burst_limit_max_count</c>.</p> + <p>Defaults to <c>1000</c> milliseconds.</p> </item> </taglist> <p>A configuration example:</p> <code type="none"> logger:add_handler(my_disk_log_h, logger_disk_log_h, - #{disk_log_opts => - #{file => "./my_disk_log"}, - logger_disk_log_h => - #{burst_limit_size => 10, - burst_window_time => 500}}). + #{config => #{file => "./my_disk_log", + burst_limit_enable => true, + burst_limit_max_count => 20, + burst_limit_window_time => 500}}). </code> </section> <section> - <title>Terminating a Large Handler</title> - <p>A handler process may grow large even if it can manage peaks of high load - without crashing. The overload protection mechanism includes user configurable - levels for a maximum allowed message queue length and maximum allowed memory - usage. This feature is disabled by default, but can be switched on by means - of the following configuration parameters:</p> - + <title>Terminating an Overloaded Handler</title> + <p>It is possible that a handler, even if it can successfully manage peaks + of high load without crashing, can build up a large message queue, or use a + large amount of memory. The overload protection mechanism includes an + automatic termination and restart feature for the purpose of guaranteeing + that a handler does not grow out of bounds. The feature is configured + with the following parameters:</p> <taglist> - <tag><c>enable_kill_overloaded</c></tag> + <tag><c>overload_kill_enable</c></tag> <item> - <p>This is set to <c>false</c> by default. The value <c>true</c> - enables the feature.</p> + <p>Value <c>true</c> enables the feature and <c>false</c> disables it.</p> + <p>Defaults to <c>false</c>.</p> </item> - <tag><c>handler_overloaded_qlen</c></tag> + <tag><c>overload_kill_qlen</c></tag> <item> - <p>This is the maximum allowed queue length. If the mailbox grows larger - than this, the handler process gets terminated.</p> + <p>This is the maximum allowed queue length. If the message queue grows + larger than this, the handler process is terminated.</p> + <p>Defaults to <c>20000</c> messages.</p> </item> - <tag><c>handler_overloaded_mem</c></tag> + <tag><c>overload_kill_mem_size</c></tag> <item> - <p>This is the maximum allowed memory usage of the handler process. If - the handler grows any larger, the process gets terminated.</p> + <p>This is the maximum memory size that the handler process is allowed to use. + If the handler grows larger than this, the process is terminated.</p> + <p>Defaults to <c>3000000</c> bytes.</p> </item> - <tag><c>handler_restart_after</c></tag> + <tag><c>overload_kill_restart_after</c></tag> <item> - <p>If the handler gets terminated because of its queue length or - memory usage, it can get automatically restarted again after a - configurable delay time. The time is specified in milliseconds - and <c>5000</c> is the default value. The value <c>never</c> can - also be set, which prevents a restart.</p> + <p>If the handler is terminated, it restarts automatically after a + delay specified in milliseconds. The value <c>infinity</c> prevents + restarts.</p> + <p>Defaults to <c>5000</c> milliseconds.</p> </item> </taglist> + <p>If the handler process is terminated because of overload, it prints + information about it in the log. It also prints information about when a + restart has taken place, and the handler is back in action.</p> + <note> + <p>The sizes of the log events affect the memory needs of the handler. + For information about how to limit the size of log events, see the + <seealso marker="logger_formatter"><c>logger_formatter(3)</c></seealso> + manual page.</p> + </note> </section> </section> diff --git a/lib/kernel/doc/src/logger_disk_log_h.xml b/lib/kernel/doc/src/logger_disk_log_h.xml index 20b49b8ca0..dfe2ab3275 100644 --- a/lib/kernel/doc/src/logger_disk_log_h.xml +++ b/lib/kernel/doc/src/logger_disk_log_h.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>2017</year> + <year>2017</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -33,106 +33,112 @@ <file>logger_disk_log_h.xml</file> </header> <module>logger_disk_log_h</module> - <modulesummary>A disk_log based handler for the Logger.</modulesummary> + <modulesummary>A disk_log based handler for Logger</modulesummary> <description> <p>This is a handler for Logger that offers circular (wrapped) logs by using <seealso marker="disk_log"><c>disk_log</c></seealso>. - Multiple instances - of this handler can be added to Logger, and each instance prints to - its own disk_log file, created with the name and settings specified in - the handler configuration.</p> + Multiple instances of this handler can be added to Logger, and each instance + prints to its own disk log file, created with the name and settings specified + in the handler configuration.</p> <p>The default standard handler, <seealso marker="logger_std_h"><c>logger_std_h</c></seealso>, can be replaced by a disk_log handler at startup of the Kernel application. See an example of this below.</p> - <p>The handler has an overload protection mechanism that will keep the handler - process and the Kernel application alive during a high load of log - requests. How this feature works, and how to modify the configuration, - is described in the + <p>The handler has an overload protection mechanism that keeps the handler + process and the Kernel application alive during high loads of log + events. How overload protection works, and how to configure it, is + described in the <seealso marker="logger_chapter#overload_protection"><c>User's Guide</c> </seealso>.</p> <p>To add a new instance of the disk_log handler, use <seealso marker="logger#add_handler-3"><c>logger:add_handler/3</c> - </seealso>. The handler configuration argument is a map which may contain + </seealso>. The handler configuration argument is a map which can contain general configuration parameters, as documented in the <seealso marker="logger_chapter#handler_configuration"><c>User's Guide</c> - </seealso>, as well as handler specific parameters.</p> - <p>The settings for the disk_log log file should be specified with the - key <c>disk_log_opts</c>. These settings are a subset of the disk_log - datatype - <seealso marker="disk_log#open-1"><c>dlog_option()</c></seealso>.</p> - <p>Parameters in the <c>disk_log_opts</c> map:</p> + </seealso>, and handler specific parameters. The specific data + is stored in a sub map with the key <c>config</c>, and can contain the + following parameters:</p> <taglist> <tag><c>file</c></tag> - <item>This is the full name of the disk_log log file.</item> + <item> + <p>This is the full name of the disk log file. The option + corresponds to the <c>name</c> property in the + <seealso marker="disk_log#open-1"><c>dlog_option()</c></seealso> + datatype.</p> + </item> <tag><c>type</c></tag> - <item>This is the disk_log type, <c>wrap</c> or <c>halt</c>. The - default value is <c>wrap</c>.</item> + <item> + <p>This is the disk log type, <c>wrap</c> or <c>halt</c>. The option + corresponds to the <c>type</c> property in the + <seealso marker="disk_log#open-1"><c>dlog_option()</c></seealso> + datatype.</p> + <p>Defaults to <c>wrap</c>.</p> + </item> <tag><c>max_no_files</c></tag> - <item>This is the maximum number of files that disk_log will use - for its circular logging. The default value is <c>10</c>. (The setting - has no effect on a halt log).</item> + <item> + <p>This is the maximum number of files that disk_log uses + for its circular logging. The option + corresponds to the <c>MaxNoFiles</c> element in the <c>size</c> property in the + <seealso marker="disk_log#open-1"><c>dlog_option()</c></seealso> + datatype.</p> + <p>Defaults to <c>10</c>.</p> + <p>The setting has no effect on a halt log.</p> + </item> <tag><c>max_no_bytes</c></tag> - <item>This is the maximum number of bytes that will be written to - a log file before disk_log proceeds with the next file in order (or - generates an error in case of a full halt log). The default value for - a wrap log is <c>1048576</c> bytes, and <c>infinity</c> for a halt - log.</item> - </taglist> - <p>Specific configuration for the handler (represented as a sub map) - is specified with the key <c>logger_disk_log_h</c>. It may contain the - following parameter:</p> - <taglist> + <item> + <p>This is the maximum number of bytes that is written to + a log file before disk_log proceeds with the next file in order, or + generates an error in case of a full halt log. The option + corresponds to the <c>MaxNoBytes</c> element in the <c>size</c> property in the + <seealso marker="disk_log#open-1"><c>dlog_option()</c></seealso> + datatype.</p> + <p>Defaults to <c>1048576</c> bytes for a wrap log, and + <c>infinity</c> for a halt log.</p> + </item> <tag><c>filesync_repeat_interval</c></tag> <item> - <p>This value (in milliseconds) specifies how often the handler will - do a disk_log sync operation in order to make sure that buffered data - gets written to disk. The handler will repeatedly attempt this - operation, but only perform it if something has actually been logged - since the last sync. The default value is <c>5000</c> milliseconds. - If <c>no_repeat</c> is set as value, the repeated sync operation is - disabled. The user can also call the - <seealso marker="logger_disk_log_h#disk_log_sync-1"><c>disk_log_sync/1</c> - </seealso> function to perform a disk_log sync.</p></item> + <p>This value, in milliseconds, specifies how often the handler does + a disk_log sync operation to write buffered data to disk. The handler attempts + the operation repeatedly, but only performs a new sync if something has + actually been logged.</p> + <p>Defaults to <c>5000</c> milliseconds.</p> + <p>If <c>no_repeat</c> is set as value, the repeated sync operation + is disabled. The user can also call the + <seealso marker="logger_disk_log_h#filesync-1"><c>filesync/1</c> + </seealso> function to perform a disk_log sync.</p> + </item> </taglist> - <p>There are a number of other configuration parameters available, that are - to be used for customizing the overload protection behaviour. The same - parameters are used both in the standard handler and the disk_log handler, - and are documented in the + <p>Other configuration parameters exist, to be used for customizing + the overload protection behaviour. The same parameters are used both in the + standard handler and the disk_log handler, and are documented in the <seealso marker="logger_chapter#overload_protection"><c>User's Guide</c> </seealso>.</p> - <p>Note that when changing the configuration of the handler in runtime, by - calling - <seealso marker="logger#set_handler_config-2"><c>logger:set_handler_config/2 - or logger:set_handler_config/3</c></seealso>, the <c>disk_log_opts</c> - settings may not be modified.</p> + <p>Notice that when changing the configuration of the handler in runtime, the + disk_log options (<c>file</c>, <c>type</c>, <c>max_no_files</c>, + <c>max_no_bytes</c>) must not be modified.</p> <p>Example of adding a disk_log handler:</p> <code type="none"> logger:add_handler(my_disk_log_h, logger_disk_log_h, - #{level => error, - filter_default => log, - disk_log_opts => - #{file => "./my_disk_log", - type => wrap, - max_no_files => 4, - max_no_bytes => 10000}, - logger_disk_log_h => - #{filesync_repeat_interval => 1000}}). + #{config => #{file => "./my_disk_log", + type => wrap, + max_no_files => 4, + max_no_bytes => 10000}, + filesync_repeat_interval => 1000}}). </code> - <p>In order to use the disk_log handler instead of the default standard + <p>To use the disk_log handler instead of the default standard handler when starting an Erlang node, change the Kernel default logger to - use disk_log. Example:</p> + use <c>logger_disk_log_h</c>. Example:</p> <code type="none"> erl -kernel logger '[{handler,default,logger_disk_log_h, - #{ disk_log_opts => #{ file => "./system_disk_log"}}}]' + #{config => #{file => "./system_disk_log"}}}]' </code> </description> <funcs> <func> - <name name="disk_log_sync" arity="1" clause_i="1"/> + <name name="filesync" arity="1" clause_i="1"/> <fsummary>Writes buffered data to disk.</fsummary> <desc> <p>Write buffered data to disk.</p> @@ -143,9 +149,9 @@ erl -kernel logger '[{handler,default,logger_disk_log_h, <section> <title>See Also</title> - <p><seealso marker="logger"><c>logger(3)</c></seealso></p> - <p><seealso marker="logger_std_h"><c>logger_std_h(3)</c></seealso></p> - <p><seealso marker="disk_log"><c>disk_log(3)</c></seealso></p> + <p><seealso marker="logger"><c>logger(3)</c></seealso>, + <seealso marker="logger_std_h"><c>logger_std_h(3)</c></seealso>, + <seealso marker="disk_log"><c>disk_log(3)</c></seealso></p> </section> </erlref> diff --git a/lib/kernel/doc/src/logger_filters.xml b/lib/kernel/doc/src/logger_filters.xml index f92181ea3f..90f1fcc270 100644 --- a/lib/kernel/doc/src/logger_filters.xml +++ b/lib/kernel/doc/src/logger_filters.xml @@ -36,15 +36,15 @@ <modulesummary>Filters to use with Logger.</modulesummary> <description> - <p>All functions exported from this module can be used as logger + <p>All functions exported from this module can be used as primary or handler - filters. See <seealso marker="logger#add_logger_filter-2"> - <c>logger:add_logger_filter/2</c></seealso> + filters. See <seealso marker="logger#add_primary_filter-2"> + <c>logger:add_primary_filter/2</c></seealso> and <seealso marker="logger#add_handler_filter-3"> <c>logger:add_handler_filter/3</c></seealso> for more information about how filters are added.</p> - <p>Filters are removed with <seealso marker="logger#remove_logger_filter-1"> - <c>logger:remove_logger_filter/1</c></seealso> + <p>Filters are removed with <seealso marker="logger#remove_primary_filter-1"> + <c>logger:remove_primary_filter/1</c></seealso> and <seealso marker="logger#remove_handler_filter-2"> <c>logger:remove_handler_filter/2</c></seealso>.</p> </description> @@ -64,18 +64,18 @@ <p>A domain field must be a list of atoms, creating smaller and more specialized domains as the list grows longer. The - biggest domain is <c>[]</c>, which comprices all - possible domains.</p> + greatest domain is <c>[]</c>, which comprises all possible + domains.</p> <p>For example, consider the following domains:</p> <pre> -D1 = [beam,erlang,otp] -D2 = [beam,erlang,otp,sasl]</pre> +D1 = [otp] +D2 = [otp, sasl]</pre> - <p><c>D1</c> is the biggest of the two, and is said to be a + <p><c>D1</c> is the greatest of the two, and is said to be a super-domain of <c>D2</c>. <c>D2</c> is a sub-domain <c>D1</c>. Both <c>D1</c> and <c>D2</c> are - sub-domains of <c>[]</c></p> + sub-domains of <c>[]</c>.</p> <p>The above domains are used for logs originating from Erlang/OTP. D1 specifies that the log event comes from @@ -86,50 +86,50 @@ D2 = [beam,erlang,otp,sasl]</pre> <p>The <c><anno>Extra</anno></c> parameter to the <c>domain/2</c> function is specified when adding the - filter via <seealso marker="logger#add_logger_filter-2"> - <c>logger:add_logger_filter/2</c></seealso> + filter via <seealso marker="logger#add_primary_filter-2"> + <c>logger:add_primary_filter/2</c></seealso> or <seealso marker="logger#add_handler_filter-3"> <c>logger:add_handler_filter/3</c></seealso>.</p> - <p>The filter compares the value of the <c>domain</c> field - in the log event's metadata (<c>Domain</c>) - to <c><anno>MatchDomain</anno></c> as follows:</p> + <p>The filter compares the value of the <c>domain</c> field in + the log event's metadata (<c>Domain</c>) against + <c><anno>MatchDomain</anno></c>. The filter matches if the + value of <c>Compare</c> is:</p> <taglist> - <tag><c><anno>Compare</anno> = sub</c></tag> + <tag><c>sub</c></tag> <item> - <p>The filter matches if <c>Domain</c> is equal to or - a sub-domain of <c>MatchDomain</c>, that is, - if <c>MatchDomain</c> is a prefix of <c>Domain</c>.</p> + <p>and <c>Domain</c> is equal to or a sub-domain + of <c>MatchDomain</c>, that is, if <c>MatchDomain</c> is + a prefix of <c>Domain</c>.</p> </item> - <tag><c><anno>Compare</anno> = super</c></tag> + <tag><c>super</c></tag> <item> - <p>The filter matches if <c>Domain</c> is equal to or a - super-domain of <c>MatchDomain</c>, that is, - if <c>Domain</c> is a prefix of <c>MatchDomain</c>.</p> + <p>and <c>Domain</c> is equal to or a super-domain + of <c>MatchDomain</c>, that is, if <c>Domain</c> is a + prefix of <c>MatchDomain</c>.</p> </item> - <tag><c><anno>Compare</anno> = equal</c></tag> + <tag><c>equal</c></tag> <item> - <p>The filter matches if <c>Domain</c> is equal - to <c>MatchDomain</c>.</p> + <p>and <c>Domain</c> is equal to <c>MatchDomain</c>.</p> </item> - <tag><c><anno>Compare</anno> = not_equal</c></tag> + <tag><c>not_equal</c></tag> <item> - <p>The filter matches if <c>Domain</c> is not equal - to <c>MatchDomain</c>, or if there is no domain field in - metadata.</p> + <p>and <c>Domain</c> differs from <c>MatchDomain</c>, or + if there is no domain field in metadata.</p> </item> - <tag><c><anno>Compare</anno> = undefined</c></tag> - <item><p>The filter matches if there is no domain field in - metadata. In this case <c><anno>MatchDomain</anno></c> - must be set to <c>[]</c>.</p> + <tag><c>undefined</c></tag> + <item> + <p>and there is no domain field in metadata. In this + case <c><anno>MatchDomain</anno></c> must be set + to <c>[]</c>.</p> </item> </taglist> - <p>If the filter matches and <c><anno>Action</anno> = log</c>, - the log event is allowed. If the filter matches - and <c><anno>Action</anno> = stop</c>, the log event is - stopped.</p> + <p>If the filter matches and <c><anno>Action</anno></c> is + <c>log</c>, the log event is allowed. If the filter matches + and <c><anno>Action</anno></c> is <c>stop</c>, the log event + is stopped.</p> <p>If the filter does not match, it returns <c>ignore</c>, meaning that other filters, or the value of the @@ -137,16 +137,16 @@ D2 = [beam,erlang,otp,sasl]</pre> event is allowed or not.</p> <p>Log events that do not contain any domain field, match only - when <c><anno>Compare</anno> = undefined</c> - or <c><anno>Compare</anno> = not_equal</c>.</p> + when <c><anno>Compare</anno></c> is equal + to <c>undefined</c> or <c>not_equal</c>.</p> - <p>Example: stop all events with - domain <c>[beam,erlang,otp,sasl|_]</c></p> + <p>Example: stop all events with domain <c>[otp, + sasl | _]</c></p> <code> -logger:set_handler_config(h1,filter_default,log). % this is the default -Filter = {fun logger_filters:domain/2,{stop,sub,[beam,erlang,otp,sasl]}}. -logger:add_handler_filter(h1,no_sasl,Filter). +logger:set_handler_config(h1, filter_default, log). % this is the default +Filter = {fun logger_filters:domain/2, {stop, sub, [otp, sasl]}}. +logger:add_handler_filter(h1, no_sasl, Filter). ok</code> </desc> </func> @@ -157,34 +157,44 @@ ok</code> <desc> <p>This filter provides a way of filtering log events based on the log level. It matches log events by comparing the - log level with a predefined <c>MatchLevel</c></p> + log level with a specified <c>MatchLevel</c></p> <p>The <c><anno>Extra</anno></c> parameter is specified when adding the filter - via <seealso marker="logger#add_logger_filter-2"> - <c>logger:add_logger_filter/2</c></seealso> + via <seealso marker="logger#add_primary_filter-2"> + <c>logger:add_primary_filter/2</c></seealso> or <seealso marker="logger#add_handler_filter-3"> <c>logger:add_handler_filter/3</c></seealso>.</p> <p>The filter compares the value of the event's log level (<c>Level</c>) to <c><anno>MatchLevel</anno></c> by calling <seealso marker="logger#compare_levels-2"> - <c>logger:compare_levels(Level,MatchLevel) -> CmpRet</c></seealso>. It - matches the event if:</p> - - <list> - <item><c>CmpRet = eq</c> and <c><anno>Operator</anno> = - eq | lteq | gteq</c></item> - <item><c>CmpRet = lt</c> and <c><anno>Operator</anno> = - lt | lteq | neq</c></item> - <item><c>CmpRet = gt</c> and <c><anno>Operator</anno> = - gt | gteq | neq</c></item> - </list> - - <p>If the filter matches and <c><anno>Action</anno> = - log</c>, the log event is allowed. If the filter matches - and <c><anno>Action</anno> = stop</c>, the log event is - stopped.</p> + <c>logger:compare_levels(Level, MatchLevel)</c></seealso>. + The filter matches if the value + of <c><anno>Operator</anno></c> is:</p> + + <taglist> + <tag><c>neq</c></tag> + <item><p>and the compare function returns <c>lt</c> + or <c>gt</c>.</p></item> + <tag><c>eq</c></tag> + <item><p>and the compare function returns <c>eq</c>.</p></item> + <tag><c>lt</c></tag> + <item><p>and the compare function returns <c>lt</c>.</p></item> + <tag><c>gt</c></tag> + <item><p>and the compare function returns <c>gt</c>.</p></item> + <tag><c>lteq</c></tag> + <item><p>and the compare function returns <c>lt</c> + or <c>eq</c>.</p></item> + <tag><c>gteq</c></tag> + <item><p>and the compare function returns <c>gt</c> + or <c>eq</c>.</p></item> + </taglist> + + <p>If the filter matches and <c><anno>Action</anno></c> is + <c>log</c>, the log event is allowed. If the filter + matches and <c><anno>Action</anno></c> is <c>stop</c>, the + log event is stopped.</p> <p>If the filter does not match, it returns <c>ignore</c>, meaning that other filters, or the value of the @@ -194,9 +204,9 @@ ok</code> <p>Example: only allow debug level log events</p> <code> -logger:set_handler_config(h1,filter_default,stop). -Filter = {fun logger_filters:level/2,{log,eq,debug}}. -logger:add_handler_filter(h1,debug_only,Filter). +logger:set_handler_config(h1, filter_default, stop). +Filter = {fun logger_filters:level/2, {log, eq, debug}}. +logger:add_handler_filter(h1, debug_only, Filter). ok</code> </desc> </func> @@ -208,9 +218,9 @@ ok</code> <p>This filter matches all progress reports from <c>supervisor</c> and <c>application_controller</c>.</p> - <p>If <c><anno>Extra</anno> = log</c>, the progress reports - are allowed. If <c><anno>Extra</anno> = stop</c>, the - progress reports are stopped.</p> + <p>If <c><anno>Extra</anno></c> is <c>log</c>, the progress + reports are allowed. If <c><anno>Extra</anno></c> + is <c>stop</c>, the progress reports are stopped.</p> <p>The filter returns <c>ignore</c> for all other log events.</p> </desc> @@ -223,9 +233,9 @@ ok</code> <p>This filter matches all events originating from a process that has its group leader on a remote node.</p> - <p>If <c><anno>Extra</anno> = log</c>, the matching events - are allowed. If <c><anno>Extra</anno> = stop</c>, the - matching events are stopped.</p> + <p>If <c><anno>Extra</anno></c> is <c>log</c>, the matching + events are allowed. If <c><anno>Extra</anno></c> + is <c>stop</c>, the matching events are stopped.</p> <p>The filter returns <c>ignore</c> for all other log events.</p> </desc> @@ -233,6 +243,12 @@ ok</code> </funcs> + <section> + <title>See Also</title> + <p> + <seealso marker="logger"><c>logger(3)</c></seealso> + </p> + </section> </erlref> diff --git a/lib/kernel/doc/src/logger_formatter.xml b/lib/kernel/doc/src/logger_formatter.xml index 02f89b26be..9226d19834 100644 --- a/lib/kernel/doc/src/logger_formatter.xml +++ b/lib/kernel/doc/src/logger_formatter.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>2017</year> + <year>2017</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -36,12 +36,15 @@ <modulesummary>Default formatter for Logger.</modulesummary> <description> - <p>Each log handler has a configured formatter specified as a + <p>Each Logger handler has a configured formatter specified as a module and a configuration term. The purpose of the formatter is to translate the log events to a final printable string (<seealso marker="stdlib:unicode#type-chardata"><c>unicode:chardata()</c> - </seealso>) which can be written to the output - device of the handler.</p> + </seealso>) which can be written to the output device of the + handler. See + sections <seealso marker="logger_chapter#handlers">Handlers</seealso> + and <seealso marker="logger_chapter#formatters">Formatters</seealso> + in the Kernel User's Guide for more information.</p> <p><c>logger_formatter</c> is the default formatter used by Logger.</p> </description> @@ -55,7 +58,8 @@ <seealso marker="stdlib:maps">map</seealso>, and the following keys can be set as configuration parameters:</p> <taglist> - <tag><c>chars_limit = pos_integer() | unlimited</c></tag> + <tag><marker id="chars_limit"/> + <c>chars_limit = integer() > 0 | unlimited</c></tag> <item> <p>A positive integer representing the value of the option with the same name to be used when calling @@ -65,14 +69,8 @@ for each log event. Notice that this is a soft limit. For a hard truncation limit, see option <c>max_size</c>.</p> <p>Defaults to <c>unlimited</c>.</p> - <note> - <p><c>chars_limit</c> has no effect on log messages on - string form. These are expected to be short, but can - still be truncated by the <c>max_size</c> - parameter.</p> - </note> </item> - <tag><marker id="depth"/><c>depth = pos_integer() | unlimited</c></tag> + <tag><marker id="depth"/><c>depth = integer() > 0 | unlimited</c></tag> <item> <p>A positive integer representing the maximum depth to which terms shall be printed by this formatter. Format @@ -80,29 +78,33 @@ format controls ~p and ~w are replaced with ~P and ~W, respectively, and the value is used as the depth parameter. For details, see - <seealso marker="stdlib:io#format-2">io:format/2,3</seealso> + <seealso marker="stdlib:io#format-2"><c>io:format/2,3</c></seealso> in STDLIB.</p> <p>Defaults to <c>unlimited</c>.</p> - <note> - <p><c>depth</c> has no effect on log messages on string - form. These are expected to be short, but can still be - truncated by the <c>max_size</c> parameter.</p> - </note> + </item> + <tag><c>encoding = </c><seealso marker="stdlib:unicode#type-encoding"> + <c>unicode:encoding()</c></seealso></tag> + <item> + <p>This parameter must reflect the encoding of the device + that the handler prints to.</p> + <p>Defaults to <c>utf8</c></p> </item> <tag><c>legacy_header = boolean()</c></tag> <item> <p>If set to <c>true</c> a header field is added to logger_formatter's part of <c>Metadata</c>. The value of this field is a string similar to the header created by - the old <c>error_logger</c> event handlers. It can be - included in the log event by adding the - tuple <c>{logger_formatter,header}</c> to the + the + old <seealso marker="error_logger"><c>error_logger</c></seealso> + event handlers. It can be included in the log event by + adding the list <c>[logger_formatter,header]</c> to the template. See the description of the <seealso marker="#type-template"><c>template()</c></seealso> type for more information.</p> <p>Defaults to <c>false</c>.</p> </item> - <tag><c>max_size = pos_integer() | unlimited</c></tag> + <tag><marker id="max_size"/> + <c>max_size = integer() > 0 | unlimited</c></tag> <item> <p>A positive integer representing the absolute maximum size a string returned from this formatter can have. If the @@ -110,12 +112,13 @@ by <c>chars_limit</c> or <c>depth</c>, it is truncated.</p> <p>Defaults to <c>unlimited</c>.</p> </item> - <tag><c>report_cb = fun((</c><seealso marker="logger#type-report"><c>logger:report()</c></seealso><c>) -> {</c><seealso marker="stdlib:io#type-format"><c>io:format()</c></seealso><c>,[term()]})</c></tag> + <tag><c>report_cb = </c><seealso marker="logger#type-report_cb"> + <c>logger:report_cb()</c></seealso></tag> <item> <p>A report callback is used by the formatter to transform log messages on report form to a format string and arguments. The report callback can be specified in the - metadata for the log event. If no report callback exist + metadata for the log event. If no report callback exists in metadata, <c>logger_formatter</c> will use <seealso marker="logger#format_report-1"> <c>logger:format_report/1</c></seealso> as default @@ -124,19 +127,18 @@ both the default report callback, and any report callback found in metadata. That is, all reports are converted by this configured function.</p> - <p>The value must be a function with arity 1, - returning <c>{Format,Args}</c>, and it will be called - with a report as only argument.</p> </item> <tag><c>single_line = boolean()</c></tag> <item> <p>If set to <c>true</c>, all newlines in the message are - replaced with <c>", "</c>, and whitespaces following - directly after newlines are removed. Note that newlines + replaced with <c>", "</c>, and white spaces following + directly after newlines are removed. Notice that newlines added by the <c>template</c> parameter are not replaced.</p> <p>Defaults to <c>true</c>.</p> </item> - <tag><c>template = </c><seealso marker="#type-template"><c>template()</c></seealso></tag> + <tag><marker id="template"/> + <c>template = </c><seealso marker="#type-template"><c>template()</c> + </seealso></tag> <item> <p>The template describes how the formatted string is composed by combining different data values from the log @@ -184,31 +186,42 @@ </desc> </datatype> <datatype> + <name name="metakey"/> + <desc> + <p></p> + </desc> + </datatype> + <datatype> <name name="template"/> <desc> - <p>The template is a list of atoms, tuples and strings. The + <p>The template is a list of atoms, atom lists, tuples and strings. The atoms <c>level</c> or <c>msg</c>, are treated as placeholders for the severity level and the log message, - respectively. Other atoms or tuples are interpreted as + respectively. Other atoms or atom lists are interpreted as placeholders for metadata, where atoms are expected to match - top level keys, and tuples represent paths to sub keys when + top level keys, and atom lists represent paths to sub keys when the metadata is a nested map. For example the - tuple <c>{key1,key2}</c> is replaced by the value of + list <c>[key1,key2]</c> is replaced by the value of the <c>key2</c> field in the nested map below. The atom <c>key1</c> on its own is replaced by the complete value of the <c>key1</c> field. The values are converted to strings.</p> -<code> -#{key1=>#{key2=>my_value, - ...} + <code> +#{key1 => #{key2 => my_value, + ...} ...}</code> - <p>Strings in the template are printed literally.</p> - <p>The default template differs depending on the values - of <c>legacy_header</c> - and <c>single_line</c>:</p> + <p>Tuples in the template express if-exist tests for metadata + keys. For example, the following tuple says that + if <c>key1</c> exists in the metadata map, + print <c>"key1=Value"</c>, where <c>Value</c> is the value + that <c>key1</c> is associated with in the metadata map. If + <c>key1</c> does not exist, print nothing.</p> + <code> +{key1, ["key1=",key1], []}</code> + <p>Strings in the template are printed literally.</p> <p>The default value for the <c>template</c> configuration parameter depends on the value of the <c>single_line</c> and <c>legacy_header</c> configuration parameters as @@ -216,13 +229,13 @@ <p>The log event used in the examples is:</p> <code> -?LOG_ERROR("name: ~p~nexit_reason: ~p",[my_name,"It crashed"])</code> +?LOG_ERROR("name: ~p~nexit_reason: ~p", [my_name, "It crashed"])</code> <taglist> - <tag><c>legacy_header=true, single_line=false</c></tag> + <tag><c>legacy_header = true, single_line = false</c></tag> <item> <p>Default - template: <c>[{logger_formatter,header},"\n",msg,"\n"]</c></p> + template: <c>[[logger_formatter,header],"\n",msg,"\n"]</c></p> <p>Example log entry:</p> <code type="none"> @@ -231,15 +244,16 @@ name: my_name exit_reason: "It crashed"</code> <p>Notice that all eight levels can occur in the heading, - not only <c>ERROR</c>, <c>WARNING</c> or <c>INFO</c> as the - old <c>error_logger</c> produced. And microseconds are - added at the end of the timestamp.</p> + not only <c>ERROR</c>, <c>WARNING</c> or <c>INFO</c> as + <seealso marker="error_logger"><c>error_logger</c></seealso> + produces. And microseconds are added at the end of the + timestamp.</p> </item> - <tag><c>legacy_header=true, single_line=true</c></tag> + <tag><c>legacy_header = true, single_line = true</c></tag> <item> <p>Default - template: <c>[{logger_formatter,header},"\n",msg,"\n"]</c></p> + template: <c>[[logger_formatter,header],"\n",msg,"\n"]</c></p> <p>Notice that the template is here the same as for <c>single_line=false</c>, but the resulting log entry @@ -250,7 +264,7 @@ exit_reason: "It crashed"</code> name: my_name, exit_reason: "It crashed"</code> </item> - <tag><c>legacy_header=false, single_line=true</c></tag> + <tag><c>legacy_header = false, single_line = true</c></tag> <item> <p>Default template: <c>[time," ",level,": ",msg,"\n"]</c></p> @@ -259,7 +273,7 @@ name: my_name, exit_reason: "It crashed"</code> 2018-05-17T18:31:31.152864+02:00 error: name: my_name, exit_reason: "It crashed"</code> </item> - <tag><c>legacy_header=false, single_line=false</c></tag> + <tag><c>legacy_header = false, single_line = false</c></tag> <item> <p>Default template: <c>[time," ",level,":\n",msg,"\n"]</c></p> @@ -279,10 +293,21 @@ exit_reason: "It crashed"</code> <name name="check_config" arity="1"/> <fsummary>Validates the given formatter configuration.</fsummary> <desc> - <p>This callback function is called by Logger when the - formatter configuration for a handler is set or modified. It + <p>The function is called by Logger when the formatter + configuration for a handler is set or modified. It returns <c>ok</c> if the configuration is valid, and <c>{error,term()}</c> if it is faulty.</p> + <p>The following Logger API functions can trigger this callback:</p> + <list> + <item><seealso marker="logger#add_handler-3"> + <c>logger:add_handler/3</c></seealso></item> + <item><seealso marker="logger#set_handler_config-2"> + <c>logger:set_handler_config/2,3</c></seealso></item> + <item><seealso marker="logger#update_handler_config-2"> + <c>logger:updata_handler_config/2</c></seealso></item> + <item><seealso marker="logger#update_formatter_config-2"> + <c>logger:update_formatter_config/2</c></seealso></item> + </list> </desc> </func> <func> @@ -293,22 +318,38 @@ exit_reason: "It crashed"</code> handlers. The log event is processed as follows:</p> <list> <item>If the message is on report form, it is converted to - <c>{Format,Args}</c> by calling the report - callback.</item> - <item>The size is limited according to the values of - configuration parameters <c>chars_limit</c> - and <c>depth</c>. Notice that this does not apply to - messages on string form.</item> + <c>{Format,Args}</c> by calling the report callback. See + section <seealso marker="logger_chapter#log_message">Log + Message</seealso> in the Kernel User's Guide for more + information about report callbacks and valid forms of log + messages.</item> + <item>The message size is limited according to the values of + configuration parameters <seealso marker="#chars_limit"> + <c>chars_limit</c></seealso> + and <seealso marker="#depth"><c>depth</c></seealso>.</item> <item>The full log entry is composed according to - the <c>template</c>.</item> + the <seealso marker="#template"><c>template</c></seealso>.</item> <item>If the final string is too long, it is truncated according to the value of configuration - parameter <c>max_size</c>.</item> + parameter <seealso marker="#max_size"><c>max_size</c></seealso>.</item> </list> </desc> </func> </funcs> + <section> + <title>See Also</title> + <p> + <seealso marker="stdlib:calendar"><c>calendar(3)</c></seealso>, + <seealso marker="error_logger"><c>error_logger(3)</c></seealso>, + <seealso marker="stdlib:io"><c>io(3)</c></seealso>, + <seealso marker="stdlib:io_lib"><c>io_lib(3)</c></seealso>, + <seealso marker="logger"><c>logger(3)</c></seealso>, + <seealso marker="stdlib:maps"><c>maps(3)</c></seealso>, + <seealso marker="sasl:sasl_app"><c>sasl(6)</c></seealso>, + <seealso marker="stdlib:unicode"><c>unicode(3)</c></seealso> + </p> + </section> </erlref> diff --git a/lib/kernel/doc/src/logger_std_h.xml b/lib/kernel/doc/src/logger_std_h.xml index a4f2848037..b526ed037d 100644 --- a/lib/kernel/doc/src/logger_std_h.xml +++ b/lib/kernel/doc/src/logger_std_h.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>2017</year> + <year>2017</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -33,86 +33,78 @@ <file>logger_std_h.xml</file> </header> <module>logger_std_h</module> - <modulesummary>Default handler for Logger.</modulesummary> + <modulesummary>Standard handler for Logger.</modulesummary> <description> - <p>This is the default handler for Logger. + <p>This is the standard handler for Logger. Multiple instances of this handler can be added to - Logger, and each instance will print logs to <c>standard_io</c>, - <c>standard_error</c> or to file. The default instance that starts - with Kernel is named <c>default</c> - which is the name to be used - for reconfiguration.</p> - <p>The handler has an overload protection mechanism that will keep the handler - process and the Kernel application alive during a high load of log - requests. How this feature works, and how to modify the configuration, - is described in the + Logger, and each instance prints logs to <c>standard_io</c>, + <c>standard_error</c>, or to file.</p> + <p>The handler has an overload protection mechanism that keeps the handler + process and the Kernel application alive during high loads of log + events. How overload protection works, and how to configure it, is + described in the <seealso marker="logger_chapter#overload_protection"><c>User's Guide</c> </seealso>.</p> <p>To add a new instance of the standard handler, use <seealso marker="logger#add_handler-3"><c>logger:add_handler/3</c> - </seealso>. The handler configuration argument is a map which may contain - general configuration parameters, as documented in the + </seealso>. The handler configuration argument is a map which can contain + general configuration parameters, as documented in the <seealso marker="logger_chapter#handler_configuration"><c>User's Guide</c> - </seealso>, as well as handler specific parameters. The specific parameters - are stored in a sub map with the key <c>logger_std_h</c>. The following - keys and values may be specified:</p> + </seealso>, and handler specific parameters. The specific data + is stored in a sub map with the key <c>config</c>, and can contain the + following parameters:</p> <taglist> <tag><marker id="type"/><c>type</c></tag> <item> - <p>This will have the value <c>standard_io</c>, <c>standard_error</c>, - <c>{file,LogFileName}</c>, or <c>{file,LogFileName,LogFileOpts}</c>, - where <c>standard_io</c> is the default value for type. It's recommended - to not specify <c>LogFileOpts</c> if not absolutely necessary. The - default options used by the handler to open a file for logging are: - <c>raw</c>, <c>append</c> and <c>delayed_write</c>. The standard - handler does not have support for circular logging. Use the - <seealso marker="logger_disk_log_h"><c>logger_disk_log_h</c> - </seealso> handler for this.</p></item> + <p>This has the value <c>standard_io</c>, <c>standard_error</c>, + <c>{file,LogFileName}</c>, or <c>{file,LogFileName,LogFileOpts}</c>.</p> + <p> Defaults to <c>standard_io</c>.</p> + <p>It is recommended not to specify <c>LogFileOpts</c> unless absolutely + necessary. The default options used by the handler to open a file for logging are + <c>raw</c>, <c>append</c>, and <c>delayed_write</c>. Notice that the standard + handler does not have support for circular logging. Use the disk_log handler, + <seealso marker="logger_disk_log_h"><c>logger_disk_log_h</c></seealso>, + for this.</p> + </item> <tag><c>filesync_repeat_interval</c></tag> <item> - <p>This value (in milliseconds) specifies how often the handler will - do a file sync operation in order to make sure that buffered data gets - written to disk. The handler will repeatedly attempt this - operation, but only perform it if something has actually been logged - since the last sync. The default value is <c>5000</c> milliseconds. - If <c>no_repeat</c> is set as value, the repeated file sync operation - is disabled, and it will be the operating system settings that determine - how quickly or slowly data gets written to disk. The user can also call + <p>This value, in milliseconds, specifies how often the handler does + a file sync operation to write buffered data to disk. The handler attempts + the operation repeatedly, but only performs a new sync if something has + actually been logged.</p> + <p>Defaults to <c>5000</c> milliseconds.</p> + <p>If <c>no_repeat</c> is set as value, the repeated file sync operation + is disabled, and it is the operating system settings that determine + how quickly or slowly data is written to disk. The user can also call the <seealso marker="logger_std_h#filesync-1"><c>filesync/1</c></seealso> - function to perform a file sync.</p></item> + function to perform a file sync.</p> + </item> </taglist> - <p>There are a number of other configuration parameters available, that are - to be used for customizing the overload protection behaviour. The same - parameters are used both in the standard handler and the disk_log handler, - and are documented in the + <p>Other configuration parameters exist, to be used for customizing + the overload protection behaviour. The same parameters are used both in the + standard handler and the disk_log handler, and are documented in the <seealso marker="logger_chapter#overload_protection"><c>User's Guide</c> </seealso>.</p> - <p>Note that when changing the configuration of the handler in runtime, by - calling - <seealso marker="logger#set_handler_config-2"><c>logger:set_handler_config/2</c> - </seealso>, or - <seealso marker="logger#set_handler_config-3"><c>logger:set_handler_config/3</c> - </seealso>, - the <c>type</c> parameter may not be modified.</p> + <p>Notice that if changing the configuration of the handler in runtime, + the <c>type</c> parameter must not be modified.</p> <p>Example of adding a standard handler:</p> <code type="none"> logger:add_handler(my_standard_h, logger_std_h, - #{level => info, - filter_default => log, - logger_std_h => - #{type => {file,"./system_info.log"}, - filesync_repeat_interval => 1000}}). + #{config => #{type => {file,"./system_info.log"}, + filesync_repeat_interval => 1000}}). </code> - <p>In order to configure the default handler (that starts initially with - the Kernel application) to log to file instead of <c>standard_io</c>, - change the Kernel default logger to use a file. Example:</p> + <p>To set the default handler, that starts initially with + the Kernel application, to log to file instead of <c>standard_io</c>, + change the Kernel default logger configuration. Example:</p> <code type="none"> erl -kernel logger '[{handler,default,logger_std_h, - #{ logger_std_h => #{ type => {file,"./log.log"}}}}]' + #{config => #{type => {file,"./log.log"}}}}]' </code> <p>An example of how to replace the standard handler with a disk_log handler - at startup can be found in the manual of - <seealso marker="logger_disk_log_h"><c>logger_disk_log_h</c></seealso>.</p> + at startup is found in the + <seealso marker="logger_disk_log_h"><c>logger_disk_log_h</c></seealso> + manual.</p> </description> <funcs> @@ -129,8 +121,9 @@ erl -kernel logger '[{handler,default,logger_std_h, <section> <title>See Also</title> - <p><seealso marker="logger"><c>logger(3)</c></seealso></p> - <p><seealso marker="logger_disk_log_h"><c>logger_disk_log_h(3)</c></seealso></p> + <p><seealso marker="logger"><c>logger(3)</c></seealso>, + <seealso marker="logger_disk_log_h"> + <c>logger_disk_log_h(3)</c></seealso></p> </section> </erlref> diff --git a/lib/kernel/doc/src/net_kernel.xml b/lib/kernel/doc/src/net_kernel.xml index a30d28d55a..d3bd7365e2 100644 --- a/lib/kernel/doc/src/net_kernel.xml +++ b/lib/kernel/doc/src/net_kernel.xml @@ -102,8 +102,10 @@ $ <input>erl -sname foobar</input></pre> <fsummary>Establish a connection to a node.</fsummary> <desc> <p>Establishes a connection to <c><anno>Node</anno></c>. Returns - <c>true</c> if successful, <c>false</c> if not, and <c>ignored</c> - if the local node is not alive.</p> + <c>true</c> if a connection was established or was already + established or if <c><anno>Node</anno></c> is the local node + itself. Returns <c>false</c> if the connection attempt failed, and + <c>ignored</c> if the local node is not alive.</p> </desc> </func> diff --git a/lib/kernel/doc/src/notes.xml b/lib/kernel/doc/src/notes.xml index 09844f1502..5884f93878 100644 --- a/lib/kernel/doc/src/notes.xml +++ b/lib/kernel/doc/src/notes.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2004</year><year>2017</year> + <year>2004</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -31,6 +31,325 @@ </header> <p>This document describes the changes made to the Kernel application.</p> +<section><title>Kernel 6.0</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> Clarify the documentation of <c>rpc:multicall/5</c>. + </p> + <p> + Own Id: OTP-10551</p> + </item> + <item> + <p> + The DNS resolver when getting econnrefused from a server + retained an invalid socket so look up towards the next + server(s) also failed.</p> + <p> + Own Id: OTP-13133 Aux Id: PR-1557 </p> + </item> + <item> + <p> + No resolver backend returns V4Mapped IPv6 addresses any + more. This was inconsistent before, some did, some did + not. To facilitate working with such addresses a new + function <c>inet:ipv4_mapped_ipv6_address/1</c> has been + added.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-13761 Aux Id: ERL-503 </p> + </item> + <item> + <p> + The type specifications for <c>file:posix/0</c> and + <c>inet:posix/0</c> have been updated according to which + errors file and socket operations should be able to + return.</p> + <p> + Own Id: OTP-14019 Aux Id: ERL-550 </p> + </item> + <item> + <p> + Fix name resolving in IPv6 only environments when doing + the initial distributed connection.</p> + <p> + Own Id: OTP-14501</p> + </item> + <item> + <p> File operations used to accept <seealso + marker="kernel:file#type-name_all">filenames</seealso> + containing null characters (integer value zero). This + caused the name to be truncated and in some cases + arguments to primitive operations to be mixed up. + Filenames containing null characters inside the filename + are now <em>rejected</em> and will cause primitive file + operations to fail. </p> <p> Also environment variable + operations used to accept <seealso + marker="kernel:os#type-env_var_name">names</seealso> and + <seealso + marker="kernel:os#type-env_var_value">values</seealso> of + environment variables containing null characters (integer + value zero). This caused operations to silently produce + erroneous results. Environment variable names and values + containing null characters inside the name or value are + now <em>rejected</em> and will cause environment variable + operations to fail. </p> <p>Primitive environment + variable operations also used to accept the <c>$=</c> + character in environment variable names causing various + problems. <c>$=</c> characters in environment variable + names are now also <em>rejected</em>. </p> <p>Also + <seealso + marker="kernel:os#cmd/1"><c>os:cmd/1</c></seealso> now + reject null characters inside its <seealso + marker="kernel:os#type-os_command">command</seealso>. + </p> <p><seealso + marker="erts:erlang#open_port/2"><c>erlang:open_port/2</c></seealso> + will also reject null characters inside the port name + from now on.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-14543 Aux Id: ERL-370 </p> + </item> + <item> + <p><c>os:putenv</c> and <c>os:getenv</c> no longer access + the process environment directly and instead work on a + thread-safe emulation. The only observable difference is + that it's <em>not</em> kept in sync with libc + <c>getenv(3)</c> / <c>putenv(3)</c>, so those who relied + on that behavior in drivers or NIFs will need to add + manual synchronization.</p> <p>On Windows this means that + you can no longer resolve DLL dependencies by modifying + the <c>PATH</c> just before loading the driver/NIF. To + make this less of a problem, the emulator now adds the + target DLL's folder to the DLL search path.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-14666</p> + </item> + <item> + <p> + Fixed connection tick toward primitive hidden nodes + (erl_interface) that could cause faulty tick timeout in + rare cases when payload data is sent to hidden node but + not received.</p> + <p> + Own Id: OTP-14681</p> + </item> + <item> + <p> + Make group react immediately on an EXIT-signal from shell + in e.g ssh.</p> + <p> + Own Id: OTP-14991 Aux Id: PR1705 </p> + </item> + <item> + <p> + Calls to <c>gen_tcp:send/2</c> on closed sockets now + returns <c>{error, closed}</c> instead of + <c>{error,enotconn}</c>.</p> + <p> + Own Id: OTP-15001</p> + </item> + <item> + <p> + The <c>included_applications</c> key are no longer + duplicated as application environment variable. Earlier, + the included applications could be read both with + <c>application:get[_all]_env(...)</c> and + <c>application:get[_all]_key(...)</c> functions. Now, it + can only be read with + <c>application:get[_all]_key(...)</c>.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-15071</p> + </item> + <item> + <p>Owner and group changes through + <c>file:write_file_info</c>, <c>file:change_owner</c>, + and <c>file:change_group</c> will no longer report + success on permission errors.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-15118</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p>A new logging API is added to Erlang/OTP, see the + <seealso + marker="kernel:logger"><c>logger(3)</c></seealso> manual + page, and section <seealso + marker="kernel:logger_chapter">Logging</seealso> in the + Kernel User's Guide.</p> + <p>Calls to <c>error_logger</c> are automatically + redirected to the new API, and legacy error logger event + handlers can still be used. It is, however, recommended + to use the Logger API directly when writing new code.</p> + <p>Notice the following potential incompatibilities:</p> + <list> <item><p>Kernel configuration parameters + <c>error_logger</c> still works, but is overruled if the + default handler's output destination is configured with + Kernel configuration parameter <c>logger</c>.</p> <p>In + general, parameters for configuring error logger are + overwritten by new parameters for configuring + Logger.</p></item> <item><p>The concept of SASL error + logging is deprecated, meaning that by default the SASL + application does not affect which log events are + logged.</p> <p>By default, supervisor reports and crash + reports are logged by the default Logger handler started + by Kernel, and end up at the same destination (terminal + or file) as other standard log event from Erlang/OTP.</p> + <p>Progress reports are not logged by default, but can be + enabled by setting the primary log level to info, for + example with the Kernel configuration parameter + <c>logger_level</c>.</p> <p>To obtain backwards + compatibility with the SASL error logging functionality + from earlier releases, set Kernel configuration parameter + <c>logger_sasl_compatible</c> to <c>true</c>. This + prevents the default Logger handler from logging any + supervisor-, crash-, or progress reports. Instead, SASL + adds a separate Logger handler during application start, + which takes care of these log events. The SASL + configuration parameters <c>sasl_error_logger</c> and + <c>sasl_errlog_type</c> specify the destination (terminal + or file) and severity level to log for these + events.</p></item></list> + <p> + Since Logger is new in Erlang/OTP 21.0, we do reserve the + right to introduce changes to the Logger API and + functionality in patches following this release. These + changes might or might not be backwards compatible with + the initial version.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-13295</p> + </item> + <item> + <p> + The function <c>inet:i/0</c> has been documented.</p> + <p> + Own Id: OTP-13713 Aux Id: PR-1645 </p> + </item> + <item> + <p> + Typespecs for <c>netns</c> and <c>bind_to_device</c> + options have been added to <c>gen_tcp</c>, <c>gen_udp</c> + and <c>gen_sctp</c> functions.</p> + <p> + Own Id: OTP-14359 Aux Id: PR-1816 </p> + </item> + <item> + <p> + New functionality for implementation of alternative + carriers for the Erlang distribution has been introduced. + This mainly consists of support for usage of distribution + controller processes (previously only ports could be used + as distribution controllers). For more information see + <seealso marker="erts:alt_dist#distribution_module">ERTS + User's Guide ➜ How to implement an Alternative Carrier + for the Erlang Distribution ➜ Distribution + Module</seealso>.</p> + <p> + Own Id: OTP-14459</p> + </item> + <item> + <p><c>seq_trace</c> labels may now be any erlang + term.</p> + <p> + Own Id: OTP-14899</p> + </item> + <item> + <p> + The SSL distribution protocol <c>-proto inet_tls</c> has + stopped setting the SSL option + <c>server_name_indication</c>. New verify funs for client + and server in <c>inet_tls_dist</c> has been added, not + documented yet, that checks node name if present in peer + certificate. Usage is still also yet to be documented.</p> + <p> + Own Id: OTP-14969 Aux Id: OTP-14465, ERL-598 </p> + </item> + <item> + <p> + Changed timeout of <c>gen_server</c> calls to <c>auth</c> + server from default 5 seconds to <c>infinity</c>.</p> + <p> + Own Id: OTP-15009 Aux Id: ERL-601 </p> + </item> + <item> + <p>The callback module passed as <c>-epmd_module</c> to + erl has been expanded to be able to do name and port + resolving.</p> <p>Documentation has also been added in + the <seealso + marker="kernel:erl_epmd"><c>erl_epmd</c></seealso> + reference manual and ERTS User's Guide <seealso + marker="erts:alt_disco">How to Implement an Alternative + Service Discovery for Erlang Distribution</seealso>.</p> + <p> + Own Id: OTP-15086 Aux Id: PR-1694 </p> + </item> + <item> + <p> + Included config file specified with relative path in + sys.config are now first searched for relative to the + directory of sys.config itself. If not found, it is also + searched for relative to the current working directory. + The latter is for backwards compatibility.</p> + <p> + Own Id: OTP-15137 Aux Id: PR-1838 </p> + </item> + </list> + </section> + +</section> + +<section><title>Kernel 5.4.3.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Non semantic change in dist_util.erl to silence dialyzer + warning.</p> + <p> + Own Id: OTP-15170</p> + </item> + </list> + </section> + +</section> + +<section><title>Kernel 5.4.3.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fix some potential buggy behavior in how ticks are sent + on inter node distribution connections. Tick is now sent + to c-node even if there are unsent buffered data, as + c-nodes need ticks in order to send reply ticks. The + amount of sent data was calculated wrongly when ticks + where suppressed due to unsent buffered data.</p> + <p> + Own Id: OTP-15162 Aux Id: ERIERL-191 </p> + </item> + </list> + </section> + +</section> + <section><title>Kernel 5.4.3</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/kernel/doc/src/os.xml b/lib/kernel/doc/src/os.xml index ef416ed233..c95e615c6b 100644 --- a/lib/kernel/doc/src/os.xml +++ b/lib/kernel/doc/src/os.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1997</year><year>2017</year> + <year>1997</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/kernel/doc/src/part.xml b/lib/kernel/doc/src/part.xml index 68eb4530e2..fa7e92835f 100644 --- a/lib/kernel/doc/src/part.xml +++ b/lib/kernel/doc/src/part.xml @@ -4,7 +4,7 @@ <part xmlns:xi="http://www.w3.org/2001/XInclude"> <header> <copyright> - <year>1996</year><year>2017</year> + <year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -22,17 +22,14 @@ </legalnotice> - <title>Logger User's Guide</title> + <title>Kernel User's Guide</title> <prepared>OTP Team</prepared> <docno></docno> - <date>2017-12-01</date> - <rev>0.1</rev> + <date>2018-06-06</date> <file>part.xml</file> </header> <description> - <p>The System Architecture Support Libraries SASL application - provides support for alarm handling, release handling, and - related functions.</p> + <p></p> </description> <xi:include href="introduction_chapter.xml"/> <xi:include href="logger_chapter.xml"/> diff --git a/lib/kernel/doc/src/ref_man.xml b/lib/kernel/doc/src/ref_man.xml index b6c2714664..d3b947527f 100644 --- a/lib/kernel/doc/src/ref_man.xml +++ b/lib/kernel/doc/src/ref_man.xml @@ -4,7 +4,7 @@ <application xmlns:xi="http://www.w3.org/2001/XInclude"> <header> <copyright> - <year>1996</year><year>2016</year> + <year>1996</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/kernel/doc/src/seq_trace.xml b/lib/kernel/doc/src/seq_trace.xml index 69eb12a8a0..1a4a74419a 100644 --- a/lib/kernel/doc/src/seq_trace.xml +++ b/lib/kernel/doc/src/seq_trace.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1998</year><year>2017</year> + <year>1998</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/kernel/include/dist.hrl b/lib/kernel/include/dist.hrl index 6baaa35d72..003852f1b0 100644 --- a/lib/kernel/include/dist.hrl +++ b/lib/kernel/include/dist.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2016. All Rights Reserved. +%% Copyright Ericsson AB 1999-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/kernel/include/dist_util.hrl b/lib/kernel/include/dist_util.hrl index eeb0f8dd43..56f775f060 100644 --- a/lib/kernel/include/dist_util.hrl +++ b/lib/kernel/include/dist_util.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2016. All Rights Reserved. +%% Copyright Ericsson AB 1999-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/kernel/include/logger.hrl b/lib/kernel/include/logger.hrl index 2143ccd297..b09977e0f2 100644 --- a/lib/kernel/include/logger.hrl +++ b/lib/kernel/include/logger.hrl @@ -32,6 +32,10 @@ -define(LOG_DEBUG(A,B),?DO_LOG(debug,[A,B])). -define(LOG_DEBUG(A,B,C),?DO_LOG(debug,[A,B,C])). +-define(LOG(L,A),?DO_LOG(L,[A])). +-define(LOG(L,A,B),?DO_LOG(L,[A,B])). +-define(LOG(L,A,B,C),?DO_LOG(L,[A,B,C])). + -define(LOCATION,#{mfa=>{?MODULE,?FUNCTION_NAME,?FUNCTION_ARITY}, line=>?LINE, file=>?FILE}). diff --git a/lib/kernel/src/Makefile b/lib/kernel/src/Makefile index f265fdd272..57f17defc8 100644 --- a/lib/kernel/src/Makefile +++ b/lib/kernel/src/Makefile @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 1996-2017. All Rights Reserved. +# Copyright Ericsson AB 1996-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -112,7 +112,8 @@ MODULES = \ logger \ logger_backend \ logger_config \ - logger_std_h \ + logger_handler_watcher \ + logger_std_h \ logger_disk_log_h \ logger_h_common \ logger_filters \ diff --git a/lib/kernel/src/application_controller.erl b/lib/kernel/src/application_controller.erl index ff5df667b5..a074d2e74b 100644 --- a/lib/kernel/src/application_controller.erl +++ b/lib/kernel/src/application_controller.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2017. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -1814,8 +1814,9 @@ check_conf() -> %% Therefore read and merge contents. if BFName =:= "sys" -> + DName = filename:dirname(FName), {ok, SysEnv, Errors} = - check_conf_sys(NewEnv), + check_conf_sys(NewEnv, [], [], DName), %% Report first error, if any, and %% terminate @@ -1837,20 +1838,31 @@ check_conf() -> end. check_conf_sys(Env) -> - check_conf_sys(Env, [], []). + check_conf_sys(Env, [], [], []). -check_conf_sys([File|T], SysEnv, Errors) when is_list(File) -> +check_conf_sys([File|T], SysEnv, Errors, DName) when is_list(File),is_list(DName) -> BFName = filename:basename(File, ".config"), FName = filename:join(filename:dirname(File), BFName ++ ".config"), - case load_file(FName) of + LName = case filename:pathtype(FName) of + relative when (DName =/= []) -> + % Check if relative to sys.config dir otherwise use legacy mode, + % i.e relative to cwd. + RName = filename:join(DName, FName), + case erl_prim_loader:read_file_info(RName) of + {ok, _} -> RName ; + error -> FName + end; + _ -> FName + end, + case load_file(LName) of {ok, NewEnv} -> - check_conf_sys(T, merge_env(SysEnv, NewEnv), Errors); + check_conf_sys(T, merge_env(SysEnv, NewEnv), Errors, DName); {error, {Line, _Mod, Str}} -> - check_conf_sys(T, SysEnv, [{error, {FName, Line, Str}}|Errors]) + check_conf_sys(T, SysEnv, [{error, {LName, Line, Str}}|Errors], DName) end; -check_conf_sys([Tuple|T], SysEnv, Errors) -> - check_conf_sys(T, merge_env(SysEnv, [Tuple]), Errors); -check_conf_sys([], SysEnv, Errors) -> +check_conf_sys([Tuple|T], SysEnv, Errors, DName) -> + check_conf_sys(T, merge_env(SysEnv, [Tuple]), Errors, DName); +check_conf_sys([], SysEnv, Errors, _) -> {ok, SysEnv, lists:reverse(Errors)}. load_file(File) -> @@ -1914,19 +1926,19 @@ info_started(Name, Node) -> ?LOG_INFO(#{label=>{application_controller,progress}, report=>[{application, Name}, {started_at, Node}]}, - #{domain=>[beam,erlang,otp,sasl], + #{domain=>[otp,sasl], report_cb=>fun logger:format_otp_report/1, logger_formatter=>#{title=>"PROGRESS REPORT"}, error_logger=>#{tag=>info_report,type=>progress}}). info_exited(Name, Reason, Type) -> - ?LOG_INFO(#{label=>{application_controller,exit}, - report=>[{application, Name}, - {exited, Reason}, - {type, Type}]}, - #{domain=>[beam,erlang,otp], - report_cb=>fun logger:format_otp_report/1, - error_logger=>#{tag=>info_report,type=>std_info}}). + ?LOG_NOTICE(#{label=>{application_controller,exit}, + report=>[{application, Name}, + {exited, Reason}, + {type, Type}]}, + #{domain=>[otp], + report_cb=>fun logger:format_otp_report/1, + error_logger=>#{tag=>info_report,type=>std_info}}). %%----------------------------------------------------------------- %% Reply to all processes waiting this application to be started. @@ -2013,5 +2025,5 @@ to_string(Term) -> true -> Term; false -> - lists:flatten(io_lib:format("~134217728p", [Term])) + lists:flatten(io_lib:format("~0p", [Term])) end. diff --git a/lib/kernel/src/application_master.erl b/lib/kernel/src/application_master.erl index 5da2b0b06c..8697143dfb 100644 --- a/lib/kernel/src/application_master.erl +++ b/lib/kernel/src/application_master.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -118,6 +118,10 @@ init(Parent, Starter, ApplData, Type) -> link(Parent), process_flag(trap_exit, true), OldGleader = group_leader(), + %% We become the group leader, but forward all I/O to OldGleader. + %% This is just a way to identify processes that belong to the + %% application. Used for example to find ourselves from any + %% process, or, reciprocally, to kill them all when we terminate. group_leader(self(), self()), %% Insert ourselves as master for the process. This ensures that %% the processes in the application can use get_env/1 at startup. diff --git a/lib/kernel/src/auth.erl b/lib/kernel/src/auth.erl index a2116d8e8a..4d18daf9e4 100644 --- a/lib/kernel/src/auth.erl +++ b/lib/kernel/src/auth.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/kernel/src/code_server.erl b/lib/kernel/src/code_server.erl index 1a7677295b..1b4a67ecb7 100644 --- a/lib/kernel/src/code_server.erl +++ b/lib/kernel/src/code_server.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1998-2016. All Rights Reserved. +%% Copyright Ericsson AB 1998-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/kernel/src/disk_log_1.erl b/lib/kernel/src/disk_log_1.erl index b456b53d20..41ef33c613 100644 --- a/lib/kernel/src/disk_log_1.erl +++ b/lib/kernel/src/disk_log_1.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2017. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/kernel/src/erl_epmd.erl b/lib/kernel/src/erl_epmd.erl index 9a0939972d..b7e8868911 100644 --- a/lib/kernel/src/erl_epmd.erl +++ b/lib/kernel/src/erl_epmd.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1998-2016. All Rights Reserved. +%% Copyright Ericsson AB 1998-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/kernel/src/erl_signal_handler.erl b/lib/kernel/src/erl_signal_handler.erl index b76c2a217a..5be905d8ae 100644 --- a/lib/kernel/src/erl_signal_handler.erl +++ b/lib/kernel/src/erl_signal_handler.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2017. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in diff --git a/lib/kernel/src/error_handler.erl b/lib/kernel/src/error_handler.erl index a9582c6225..a89ef83261 100644 --- a/lib/kernel/src/error_handler.erl +++ b/lib/kernel/src/error_handler.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/kernel/src/error_logger.erl b/lib/kernel/src/error_logger.erl index b3957d0c7e..ad8c937882 100644 --- a/lib/kernel/src/error_logger.erl +++ b/lib/kernel/src/error_logger.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2017. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -74,8 +74,8 @@ start() -> type => worker, modules => dynamic}, case supervisor:start_child(logger_sup, ErrorLogger) of - {ok,_} -> - ok; + {ok,Pid} -> + ok = logger_handler_watcher:register_handler(?MODULE,Pid); Error -> Error end; @@ -95,14 +95,19 @@ start_link() -> %%% Stop the event manager -spec stop() -> ok. stop() -> - _ = supervisor:terminate_child(logger_sup,?MODULE), - _ = supervisor:delete_child(logger_sup,?MODULE), - ok. + case whereis(?MODULE) of + undefined -> + ok; + _Pid -> + _ = gen_event:stop(?MODULE,{shutdown,stopped},infinity), + _ = supervisor:delete_child(logger_sup,?MODULE), + ok + end. %%%----------------------------------------------------------------- %%% Callbacks for logger --spec adding_handler(logger:config()) -> - {ok,logger:config()} | {error,term()}. +-spec adding_handler(logger:handler_config()) -> + {ok,logger:handler_config()} | {error,term()}. adding_handler(#{id:=?MODULE}=Config) -> case start() of ok -> @@ -111,12 +116,12 @@ adding_handler(#{id:=?MODULE}=Config) -> Error end. --spec removing_handler(logger:config()) -> ok. +-spec removing_handler(logger:handler_config()) -> ok. removing_handler(#{id:=?MODULE}) -> stop(), ok. --spec log(logger:log_event(),logger:config()) -> ok. +-spec log(logger:log_event(),logger:handler_config()) -> ok. log(#{level:=Level,msg:=Msg,meta:=Meta},_Config) -> do_log(Level,Msg,Meta). @@ -169,16 +174,25 @@ do_log(_Level,_Msg,_Meta) -> -spec notify(logger:level(), msg_tag(), any(), any(), map()) -> 'ok'. notify(Level,Tag0,FormatOrType0,ArgsOrReport,#{pid:=Pid0,gl:=GL,?MODULE:=My}) -> - Tag = fix_warning_tag(Level,Tag0), + {Tag,FormatOrType} = maybe_map_warnings(Level,Tag0,FormatOrType0), Pid = case maps:get(emulator,My,false) of true -> emulator; _ -> Pid0 end, - FormatOrType = fix_warning_type(Level,FormatOrType0), gen_event:notify(?MODULE,{Tag,GL,{Pid,FormatOrType,ArgsOrReport}}). -%% This is to fix the case when the client has explicitly added the -%% error logger tag and type in metadata, and not checked the warning map. +%% For backwards compatibility with really old even handlers, check +%% the warning map and update tag and type. +maybe_map_warnings(warning,Tag,FormatOrType) -> + case error_logger:warning_map() of + warning -> + {Tag,FormatOrType}; + Level -> + {fix_warning_tag(Level,Tag),fix_warning_type(Level,FormatOrType)} + end; +maybe_map_warnings(_,Tag,FormatOrType) -> + {Tag,FormatOrType}. + fix_warning_tag(error,warning_msg) -> error; fix_warning_tag(error,warning_report) -> error_report; fix_warning_tag(info,warning_msg) -> info_msg; @@ -263,29 +277,10 @@ warning_report(Report) -> Report :: report(). warning_report(Type, Report) -> - Level = error_logger:warning_map(), - {Tag, NType} = case Level of - info -> - if - Type =:= std_warning -> - {info_report, std_info}; - true -> - {info_report, Type} - end; - warning -> - {warning_report, Type}; - error -> - if - Type =:= std_warning -> - {error_report, std_error}; - true -> - {error_report, Type} - end - end, - logger:log(Level, + logger:log(warning, #{label=>{?MODULE,warning_report}, report=>Report}, - meta(Tag,NType)). + meta(warning_report,Type)). %%----------------------------------------------------------------- %% This function provides similar functions as error_msg for @@ -304,20 +299,11 @@ warning_msg(Format) -> Data :: list(). warning_msg(Format, Args) -> - Level = error_logger:warning_map(), - Tag = case Level of - warning -> - warning_msg; - info -> - info_msg; - error -> - error - end, - logger:log(Level, + logger:log(warning, #{label=>{?MODULE,warning_msg}, format=>Format, args=>Args}, - meta(Tag)). + meta(warning_msg)). %%----------------------------------------------------------------- %% This function should be used for information reports. Events @@ -336,7 +322,7 @@ info_report(Report) -> Report :: report(). info_report(Type, Report) -> - logger:log(info, + logger:log(notice, #{label=>{?MODULE,info_report}, report=>Report}, meta(info_report,Type)). @@ -357,7 +343,7 @@ info_msg(Format) -> Data :: list(). info_msg(Format, Args) -> - logger:log(info, + logger:log(notice, #{label=>{?MODULE,info_msg}, format=>Format, args=>Args}, @@ -377,7 +363,7 @@ error_info(Error) -> false -> {"~p",[Error]} end, MyMeta = #{tag=>info,type=>Error}, - logger:log(info, Format, Args, #{?MODULE=>MyMeta,domain=>[Error]}). + logger:log(notice, Format, Args, #{?MODULE=>MyMeta,domain=>[Error]}). %%----------------------------------------------------------------- %% Create metadata @@ -532,17 +518,17 @@ tty(true) -> _ = case lists:member(error_logger_tty_h, which_report_handlers()) of false -> case logger:get_handler_config(default) of - {ok,{logger_std_h,#{logger_std_h:=#{type:=standard_io}}}} -> + {ok,#{module:=logger_std_h,config:=#{type:=standard_io}}} -> logger:remove_handler_filter(default, error_logger_tty_false); _ -> logger:add_handler(error_logger_tty_true,logger_std_h, #{filter_default=>stop, filters=>?DEFAULT_HANDLER_FILTERS( - [beam,erlang,otp]), + [otp]), formatter=>{?DEFAULT_FORMATTER, ?DEFAULT_FORMAT_CONFIG}, - logger_std_h=>#{type=>standard_io}}) + config=>#{type=>standard_io}}) end; true -> ok @@ -552,7 +538,7 @@ tty(false) -> delete_report_handler(error_logger_tty_h), _ = logger:remove_handler(error_logger_tty_true), _ = case logger:get_handler_config(default) of - {ok,{logger_std_h,#{logger_std_h:=#{type:=standard_io}}}} -> + {ok,#{module:=logger_std_h,config:=#{type:=standard_io}}} -> logger:add_handler_filter(default,error_logger_tty_false, {fun(_,_) -> stop end, ok}); _ -> @@ -575,6 +561,8 @@ get_format_depth() -> case application:get_env(kernel, error_logger_format_depth) of {ok, Depth} when is_integer(Depth) -> max(10, Depth); + {ok, unlimited} -> + unlimited; undefined -> unlimited end. diff --git a/lib/kernel/src/erts_debug.erl b/lib/kernel/src/erts_debug.erl index 1270de4144..c4d276f9e8 100644 --- a/lib/kernel/src/erts_debug.erl +++ b/lib/kernel/src/erts_debug.erl @@ -36,7 +36,8 @@ map_info/1, same/2, set_internal_state/2, size_shared/1, copy_shared/1, dirty_cpu/2, dirty_io/2, dirty/3, lcnt_control/1, lcnt_control/2, lcnt_collect/0, lcnt_clear/0, - lc_graph/0, lc_graph_to_dot/2, lc_graph_merge/2]). + lc_graph/0, lc_graph_to_dot/2, lc_graph_merge/2, + alloc_blocks_size/1]). -spec breakpoint(MFA, Flag) -> non_neg_integer() when MFA :: {Module :: module(), @@ -495,3 +496,58 @@ lcg_print_locks(Out, [LastLock]) -> lcg_print_locks(Out, [Lock | Rest]) -> io:format(Out, "~w,\n", [Lock]), lcg_print_locks(Out, Rest). + + +%% Returns the amount of memory allocated by the given allocator type. +-spec alloc_blocks_size(Type) -> non_neg_integer() | undefined when + Type :: atom(). + +alloc_blocks_size(Type) -> + Allocs = erlang:system_info(alloc_util_allocators), + Sizes = erlang:system_info({allocator_sizes, Allocs}), + alloc_blocks_size_1(Sizes, Type, 0). + +alloc_blocks_size_1([], _Type, 0) -> + undefined; +alloc_blocks_size_1([{_Type, false} | Rest], Type, Acc) -> + alloc_blocks_size_1(Rest, Type, Acc); +alloc_blocks_size_1([{Type, Instances} | Rest], Type, Acc0) -> + F = fun ({instance, _, L}, Acc) -> + MBCSPool = case lists:keyfind(mbcs_pool, 1, L) of + {_, Pool} -> Pool; + false -> [] + end, + {_,MBCS} = lists:keyfind(mbcs, 1, L), + {_,SBCS} = lists:keyfind(sbcs, 1, L), + Acc + + sum_block_sizes(MBCSPool) + + sum_block_sizes(MBCS) + + sum_block_sizes(SBCS) + end, + alloc_blocks_size_1(Rest, Type, lists:foldl(F, Acc0, Instances)); +alloc_blocks_size_1([{_Type, Instances} | Rest], Type, Acc0) -> + F = fun ({instance, _, L}, Acc) -> + Acc + sum_foreign_sizes(Type, L) + end, + alloc_blocks_size_1(Rest, Type, lists:foldl(F, Acc0, Instances)); +alloc_blocks_size_1([], _Type, Acc) -> + Acc. + +sum_foreign_sizes(Type, L) -> + case lists:keyfind(mbcs_pool, 1, L) of + {_,Pool} -> + {_,ForeignBlocks} = lists:keyfind(foreign_blocks, 1, Pool), + case lists:keyfind(Type, 1, ForeignBlocks) of + {_,TypeSizes} -> sum_block_sizes(TypeSizes); + false -> 0 + end; + _ -> + 0 + end. + +sum_block_sizes(Blocks) -> + lists:foldl( + fun({blocks_size, Sz,_,_}, Sz0) -> Sz0+Sz; + ({blocks_size, Sz}, Sz0) -> Sz0+Sz; + (_, Sz) -> Sz + end, 0, Blocks). diff --git a/lib/kernel/src/gen_sctp.erl b/lib/kernel/src/gen_sctp.erl index 3526df3600..bf795ee9c6 100644 --- a/lib/kernel/src/gen_sctp.erl +++ b/lib/kernel/src/gen_sctp.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2016. All Rights Reserved. +%% Copyright Ericsson AB 2007-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/kernel/src/gen_tcp.erl b/lib/kernel/src/gen_tcp.erl index 253c63528f..c61411e814 100644 --- a/lib/kernel/src/gen_tcp.erl +++ b/lib/kernel/src/gen_tcp.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2016. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/kernel/src/gen_udp.erl b/lib/kernel/src/gen_udp.erl index 9ab58011ec..44eef9f3c5 100644 --- a/lib/kernel/src/gen_udp.erl +++ b/lib/kernel/src/gen_udp.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2016. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/kernel/src/group.erl b/lib/kernel/src/group.erl index 2c0518ccad..5625ae6eb7 100644 --- a/lib/kernel/src/group.erl +++ b/lib/kernel/src/group.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2017. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/kernel/src/inet_config.erl b/lib/kernel/src/inet_config.erl index 9f76360b8b..e771461b65 100644 --- a/lib/kernel/src/inet_config.erl +++ b/lib/kernel/src/inet_config.erl @@ -98,7 +98,7 @@ init() -> {win32,WinType} -> win32_load_from_registry(WinType); _ -> - error("can not read win32 system registry~n", []) + error("cannot read win32 system registry~n", []) end end, CfgFiles), diff --git a/lib/kernel/src/inet_dns.erl b/lib/kernel/src/inet_dns.erl index f1f58bc872..6c98d2aab7 100644 --- a/lib/kernel/src/inet_dns.erl +++ b/lib/kernel/src/inet_dns.erl @@ -699,7 +699,7 @@ encode_labels(Bin, Comp0, Pos, [L|Ls]=Labels) none -> Comp = if Pos < (3 bsl 14) -> %% Just in case - compression - %% pointers can not reach further + %% pointers cannot reach further gb_trees:insert(Labels, Pos, Comp0); true -> Comp0 end, diff --git a/lib/kernel/src/inet_tcp_dist.erl b/lib/kernel/src/inet_tcp_dist.erl index b4b50899f7..d1701afdaa 100644 --- a/lib/kernel/src/inet_tcp_dist.erl +++ b/lib/kernel/src/inet_tcp_dist.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2017. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/kernel/src/kernel.app.src b/lib/kernel/src/kernel.app.src index d873178f55..4933eae76f 100644 --- a/lib/kernel/src/kernel.app.src +++ b/lib/kernel/src/kernel.app.src @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2017. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -67,6 +67,7 @@ logger_filters, logger_formatter, logger_h_common, + logger_handler_watcher, logger_server, logger_simple_h, logger_std_h, @@ -129,6 +130,7 @@ kernel_refc, kernel_sup, logger, + logger_handler_watcher, logger_sup, net_kernel, net_sup, @@ -140,9 +142,8 @@ inet_db, pg2]}, {applications, []}, - {env, [{logger_level, info}, - {logger_sasl_compatible, false}, - {logger_progress_reports, stop} + {env, [{logger_level, notice}, + {logger_sasl_compatible, false} ]}, {mod, {kernel, []}}, {runtime_dependencies, ["erts-10.0", "stdlib-3.5", "sasl-3.0"]} diff --git a/lib/kernel/src/kernel.erl b/lib/kernel/src/kernel.erl index b0e8c00bbf..c68d04e279 100644 --- a/lib/kernel/src/kernel.erl +++ b/lib/kernel/src/kernel.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2017. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/kernel/src/kernel_config.erl b/lib/kernel/src/kernel_config.erl index c5ff1887c2..691a266c2d 100644 --- a/lib/kernel/src/kernel_config.erl +++ b/lib/kernel/src/kernel_config.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/kernel/src/logger.erl b/lib/kernel/src/logger.erl index 3beb3102fa..ffc90f4fc5 100644 --- a/lib/kernel/src/logger.erl +++ b/lib/kernel/src/logger.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2017. All Rights Reserved. +%% Copyright Ericsson AB 2017-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -35,14 +35,17 @@ %% Configuration -export([add_handler/3, remove_handler/1, - add_logger_filter/2, add_handler_filter/3, - remove_logger_filter/1, remove_handler_filter/2, - set_module_level/2, unset_module_level/1, - set_logger_config/1, set_logger_config/2, + add_primary_filter/2, add_handler_filter/3, + remove_primary_filter/1, remove_handler_filter/2, + set_module_level/2, + unset_module_level/1, unset_module_level/0, + get_module_level/0, get_module_level/1, + set_primary_config/1, set_primary_config/2, set_handler_config/2, set_handler_config/3, - update_logger_config/1, update_handler_config/2, + update_primary_config/1, update_handler_config/2, update_formatter_config/2, update_formatter_config/3, - get_logger_config/0, get_handler_config/1, + get_primary_config/0, get_handler_config/1, + get_handler_config/0, get_handler_ids/0, get_config/0, add_handlers/1]). %% Private configuration @@ -52,7 +55,6 @@ -export([compare_levels/2]). -export([set_process_metadata/1, update_process_metadata/1, unset_process_metadata/0, get_process_metadata/0]). --export([i/0, i/1]). %% Basic report formatting -export([format_report/1, format_otp_report/1]). @@ -72,6 +74,11 @@ -type level() :: emergency | alert | critical | error | warning | notice | info | debug. -type report() :: map() | [{atom(),term()}]. +-type report_cb() :: fun((report()) -> {io:format(),[term()]}) | + fun((report(),report_cb_config()) -> unicode:chardata()). +-type report_cb_config() :: #{encoding := unicode:encoding(), + depth := pos_integer() | unlimited, + chars_limit := pos_integer() | unlimited}. -type msg_fun() :: fun((term()) -> {io:format(),[term()]} | report() | unicode:chardata()). @@ -82,7 +89,7 @@ file => file:filename(), line => non_neg_integer(), domain => [atom()], - report_cb => fun((report()) -> {io:format(),[term()]}), + report_cb => report_cb(), atom() => term()}. -type location() :: #{mfa := {module(),atom(),non_neg_integer()}, file := file:filename(), @@ -93,20 +100,37 @@ filter_return()),filter_arg()}. -type filter_arg() :: term(). -type filter_return() :: stop | ignore | log_event(). --type config() :: #{id => handler_id(), - level => level(), - filter_default => log | stop, - filters => [{filter_id(),filter()}], - formatter => {module(),formatter_config()}, - atom() => term()}. +-type primary_config() :: #{level => level() | all | none, + filter_default => log | stop, + filters => [{filter_id(),filter()}]}. +-type handler_config() :: #{id => handler_id(), + config => term(), + level => level() | all | none, + module => module(), + filter_default => log | stop, + filters => [{filter_id(),filter()}], + formatter => {module(),formatter_config()}}. -type timestamp() :: integer(). -type formatter_config() :: #{atom() => term()}. --type config_handler() :: {handler, handler_id(), module(), config()}. - --export_type([log_event/0,level/0,report/0,msg_fun/0,metadata/0,config/0, - handler_id/0,filter_id/0,filter/0,filter_arg/0,filter_return/0, - config_handler/0,formatter_config/0]). +-type config_handler() :: {handler, handler_id(), module(), handler_config()}. + +-export_type([log_event/0, + level/0, + report/0, + report_cb/0, + report_cb_config/0, + msg_fun/0, + metadata/0, + primary_config/0, + handler_config/0, + handler_id/0, + filter_id/0, + filter/0, + filter_arg/0, + filter_return/0, + config_handler/0, + formatter_config/0]). %%%----------------------------------------------------------------- %%% API @@ -308,11 +332,11 @@ internal_log(Level,Term) when is_atom(Level) -> %%%----------------------------------------------------------------- %%% Configuration --spec add_logger_filter(FilterId,Filter) -> ok | {error,term()} when +-spec add_primary_filter(FilterId,Filter) -> ok | {error,term()} when FilterId :: filter_id(), Filter :: filter(). -add_logger_filter(FilterId,Filter) -> - logger_server:add_filter(logger,{FilterId,Filter}). +add_primary_filter(FilterId,Filter) -> + logger_server:add_filter(primary,{FilterId,Filter}). -spec add_handler_filter(HandlerId,FilterId,Filter) -> ok | {error,term()} when HandlerId :: handler_id(), @@ -322,10 +346,10 @@ add_handler_filter(HandlerId,FilterId,Filter) -> logger_server:add_filter(HandlerId,{FilterId,Filter}). --spec remove_logger_filter(FilterId) -> ok | {error,term()} when +-spec remove_primary_filter(FilterId) -> ok | {error,term()} when FilterId :: filter_id(). -remove_logger_filter(FilterId) -> - logger_server:remove_filter(logger,FilterId). +remove_primary_filter(FilterId) -> + logger_server:remove_filter(primary,FilterId). -spec remove_handler_filter(HandlerId,FilterId) -> ok | {error,term()} when HandlerId :: handler_id(), @@ -336,7 +360,7 @@ remove_handler_filter(HandlerId,FilterId) -> -spec add_handler(HandlerId,Module,Config) -> ok | {error,term()} when HandlerId :: handler_id(), Module :: module(), - Config :: config(). + Config :: handler_config(). add_handler(HandlerId,Module,Config) -> logger_server:add_handler(HandlerId,Module,Config). @@ -345,79 +369,140 @@ add_handler(HandlerId,Module,Config) -> remove_handler(HandlerId) -> logger_server:remove_handler(HandlerId). --spec set_logger_config(Key,Value) -> ok | {error,term()} when - Key :: atom(), - Value :: term(). -set_logger_config(Key,Value) -> - logger_server:set_config(logger,Key,Value). - --spec set_logger_config(Config) -> ok | {error,term()} when - Config :: config(). -set_logger_config(Config) -> - logger_server:set_config(logger,Config). - --spec set_handler_config(HandlerId,Key,Value) -> ok | {error,term()} when +-spec set_primary_config(level,Level) -> ok | {error,term()} when + Level :: level() | all | none; + (filter_default,FilterDefault) -> ok | {error,term()} when + FilterDefault :: log | stop; + (filters,Filters) -> ok | {error,term()} when + Filters :: [{filter_id(),filter()}]. +set_primary_config(Key,Value) -> + logger_server:set_config(primary,Key,Value). + +-spec set_primary_config(Config) -> ok | {error,term()} when + Config :: primary_config(). +set_primary_config(Config) -> + logger_server:set_config(primary,Config). + +-spec set_handler_config(HandlerId,level,Level) -> Return when HandlerId :: handler_id(), - Key :: atom(), - Value :: term(). + Level :: level() | all | none, + Return :: ok | {error,term()}; + (HandlerId,filter_default,FilterDefault) -> Return when + HandlerId :: handler_id(), + FilterDefault :: log | stop, + Return :: ok | {error,term()}; + (HandlerId,filters,Filters) -> Return when + HandlerId :: handler_id(), + Filters :: [{filter_id(),filter()}], + Return :: ok | {error,term()}; + (HandlerId,formatter,Formatter) -> Return when + HandlerId :: handler_id(), + Formatter :: {module(), formatter_config()}, + Return :: ok | {error,term()}; + (HandlerId,config,Config) -> Return when + HandlerId :: handler_id(), + Config :: term(), + Return :: ok | {error,term()}. set_handler_config(HandlerId,Key,Value) -> logger_server:set_config(HandlerId,Key,Value). -spec set_handler_config(HandlerId,Config) -> ok | {error,term()} when HandlerId :: handler_id(), - Config :: config(). + Config :: handler_config(). set_handler_config(HandlerId,Config) -> logger_server:set_config(HandlerId,Config). --spec update_logger_config(Config) -> ok | {error,term()} when - Config :: config(). -update_logger_config(Config) -> - logger_server:update_config(logger,Config). +-spec update_primary_config(Config) -> ok | {error,term()} when + Config :: primary_config(). +update_primary_config(Config) -> + logger_server:update_config(primary,Config). -spec update_handler_config(HandlerId,Config) -> ok | {error,term()} when HandlerId :: handler_id(), - Config :: config(). + Config :: handler_config(). update_handler_config(HandlerId,Config) -> logger_server:update_config(HandlerId,Config). --spec get_logger_config() -> {ok,Config} when - Config :: config(). -get_logger_config() -> - {ok,Config} = logger_config:get(?LOGGER_TABLE,logger), - {ok,maps:remove(handlers,Config)}. +-spec get_primary_config() -> Config when + Config :: primary_config(). +get_primary_config() -> + {ok,Config} = logger_config:get(?LOGGER_TABLE,primary), + maps:remove(handlers,Config). --spec get_handler_config(HandlerId) -> {ok,{Module,Config}} | {error,term()} when +-spec get_handler_config(HandlerId) -> {ok,Config} | {error,term()} when HandlerId :: handler_id(), - Module :: module(), - Config :: config(). + Config :: handler_config(). get_handler_config(HandlerId) -> - logger_config:get(?LOGGER_TABLE,HandlerId). + case logger_config:get(?LOGGER_TABLE,HandlerId) of + {ok,{_,Config}} -> + {ok,Config}; + Error -> + Error + end. + +-spec get_handler_config() -> [Config] when + Config :: handler_config(). +get_handler_config() -> + [begin + {ok,Config} = get_handler_config(HandlerId), + Config + end || HandlerId <- get_handler_ids()]. + +-spec get_handler_ids() -> [HandlerId] when + HandlerId :: handler_id(). +get_handler_ids() -> + {ok,#{handlers:=HandlerIds}} = logger_config:get(?LOGGER_TABLE,primary), + HandlerIds. -spec update_formatter_config(HandlerId,FormatterConfig) -> ok | {error,term()} when - HandlerId :: config(), + HandlerId :: handler_id(), FormatterConfig :: formatter_config(). update_formatter_config(HandlerId,FormatterConfig) -> logger_server:update_formatter_config(HandlerId,FormatterConfig). -spec update_formatter_config(HandlerId,Key,Value) -> ok | {error,term()} when - HandlerId :: config(), + HandlerId :: handler_id(), Key :: atom(), Value :: term(). update_formatter_config(HandlerId,Key,Value) -> logger_server:update_formatter_config(HandlerId,#{Key=>Value}). --spec set_module_level(Module,Level) -> ok | {error,term()} when +-spec set_module_level(Modules,Level) -> ok | {error,term()} when + Modules :: [module()] | module(), + Level :: level() | all | none. +set_module_level(Module,Level) when is_atom(Module) -> + set_module_level([Module],Level); +set_module_level(Modules,Level) -> + logger_server:set_module_level(Modules,Level). + +-spec unset_module_level(Modules) -> ok when + Modules :: [module()] | module(). +unset_module_level(Module) when is_atom(Module) -> + unset_module_level([Module]); +unset_module_level(Modules) -> + logger_server:unset_module_level(Modules). + +-spec unset_module_level() -> ok. +unset_module_level() -> + logger_server:unset_module_level(). + +-spec get_module_level(Modules) -> [{Module,Level}] when + Modules :: [Module] | Module, Module :: module(), - Level :: level(). -set_module_level(Module,Level) -> - logger_server:set_module_level(Module,Level). - --spec unset_module_level(Module) -> ok | {error,term()} when - Module :: module(). -unset_module_level(Module) -> - logger_server:unset_module_level(Module). + Level :: level() | all | none. +get_module_level(Module) when is_atom(Module) -> + get_module_level([Module]); +get_module_level(Modules) when is_list(Modules) -> + [{M,L} || {M,L} <- get_module_level(), + lists:member(M,Modules)]. + +-spec get_module_level() -> [{Module,Level}] when + Module :: module(), + Level :: level() | all | none. +get_module_level() -> + logger_config:get_module_level(?LOGGER_TABLE). %%%----------------------------------------------------------------- %%% Misc @@ -466,82 +551,13 @@ unset_process_metadata() -> _ = erase(?LOGGER_META_KEY), ok. --spec i() -> #{logger=>config(), - handlers=>[{handler_id(),module(),config()}], - module_levels=>[{module(),level()}]}. -i() -> - i(term). - --spec i(term) -> #{logger=>config(), - handlers=>[{handler_id(),module(),config()}], - module_levels=>[{module(),level()}]}; - (print) -> ok; - (string) -> iolist(). -i(_Action = print) -> - io:put_chars(i(string)); -i(_Action = string) -> - #{logger := #{level := Level, - filters := Filters, - filter_default := FilterDefault}, - handlers := HandlerConfigs, - module_levels := Modules} = i(term), - [io_lib:format("Current logger configuration:~n", []), - io_lib:format(" Level: ~p~n",[Level]), - io_lib:format(" Filter Default: ~p~n", [FilterDefault]), - io_lib:format(" Filters: ~n", []), - print_filters(4, Filters), - io_lib:format(" Handlers: ~n", []), - print_handlers(HandlerConfigs), - io_lib:format(" Level set per module: ~n", []), - print_module_levels(Modules) - ]; -i(_Action = term) -> - {Logger, Handlers, Modules} = logger_config:get(tid()), - #{logger=>maps:remove(handlers,Logger), - handlers=>lists:keysort(1,Handlers), - module_levels=>lists:keysort(1,Modules)}. - -print_filters(Indent, {Id, {Fun, Config}}) -> - io_lib:format("~sId: ~p~n" - "~s Fun: ~p~n" - "~s Config: ~p~n",[Indent, Id, Indent, Fun, Indent, Config]); -print_filters(Indent, Filters) -> - IndentStr = io_lib:format("~.*s",[Indent, ""]), - lists:map(fun(Filter) ->print_filters(IndentStr, Filter) end, Filters). - - -print_handlers({Id,Module, - #{level := Level, - filters := Filters, filter_default := FilterDefault, - formatter := {FormatterModule,FormatterConfig}} = Config}) -> - MyKeys = [filter_default, filters, formatter, level, id], - UnhandledConfig = maps:filter(fun(Key, _) -> - not lists:member(Key, MyKeys) - end, Config), - Unhandled = lists:map(fun({Key, Value}) -> - io_lib:format(" ~p: ~p~n",[Key, Value]) - end, maps:to_list(UnhandledConfig)), - io_lib:format(" Id: ~p~n" - " Module: ~p~n" - " Level: ~p~n" - " Formatter:~n" - " Module: ~p~n" - " Config: ~p~n" - " Filter Default: ~p~n" - " Filters:~n~s" - " Handler Config:~n" - "~s" - "",[Id, Module, Level, FormatterModule, FormatterConfig, - FilterDefault, print_filters(8, Filters), Unhandled]); -print_handlers(Handlers) -> - lists:map(fun print_handlers/1, Handlers). - -print_module_levels({Module,Level}) -> - io_lib:format(" Module: ~p~n" - " Level: ~p~n", - [Module,Level]); -print_module_levels(ModuleLevels) -> - lists:map(fun print_module_levels/1, ModuleLevels). +-spec get_config() -> #{primary=>primary_config(), + handlers=>[handler_config()], + module_levels=>[{module(),level() | all | none}]}. +get_config() -> + #{primary=>get_primary_config(), + handlers=>get_handler_config(), + module_levels=>lists:keysort(1,get_module_level())}. -spec internal_init_logger() -> ok | {error,term()}. %% This function is responsible for config of the logger @@ -550,13 +566,13 @@ print_module_levels(ModuleLevels) -> %% tree is started. internal_init_logger() -> try - ok = logger:set_logger_config(level, get_logger_level()), - ok = logger:set_logger_config(filter_default, get_logger_filter_default()), + ok = logger:set_primary_config(level, get_logger_level()), + ok = logger:set_primary_config(filter_default, get_primary_filter_default()), - [case logger:add_logger_filter(Id, Filter) of + [case logger:add_primary_filter(Id, Filter) of ok -> ok; {error, Reason} -> throw(Reason) - end || {Id, Filter} <- get_logger_filters()], + end || {Id, Filter} <- get_primary_filters()], _ = [[case logger:set_module_level(Module, Level) of ok -> ok; @@ -671,13 +687,13 @@ get_logger_type() -> get_logger_level() -> case application:get_env(kernel,logger_level,info) of - Level when ?IS_LEVEL(Level) -> + Level when ?IS_LEVEL(Level); Level=:=all; Level=:=none -> Level; Level -> throw({logger_level, Level}) end. -get_logger_filter_default() -> +get_primary_filter_default() -> case lists:keyfind(filters,1,get_logger_env()) of {filters,Default,_} -> Default; @@ -685,7 +701,7 @@ get_logger_filter_default() -> log end. -get_logger_filters() -> +get_primary_filters() -> lists:foldl( fun({filters, _, Filters}, _Acc) -> Filters; @@ -700,7 +716,7 @@ init_default_config(Type) when Type==standard_io; element(1,Type)==file -> Env = get_logger_env(), DefaultFormatter = #{formatter=>{?DEFAULT_FORMATTER,?DEFAULT_FORMAT_CONFIG}}, - DefaultConfig = DefaultFormatter#{logger_std_h=>#{type=>Type}}, + DefaultConfig = DefaultFormatter#{config=>#{type=>Type}}, NewLoggerEnv = case lists:keyfind(default, 2, Env) of {handler, default, Module, Config} -> @@ -731,17 +747,9 @@ init_default_config(Type) -> get_default_handler_filters() -> case application:get_env(kernel, logger_sasl_compatible, false) of true -> - ?DEFAULT_HANDLER_FILTERS([beam,erlang,otp]); + ?DEFAULT_HANDLER_FILTERS([otp]); false -> - Extra = - case application:get_env(kernel, logger_progress_reports, stop) of - log -> - []; - stop -> - [{stop_progress, - {fun logger_filters:progress/2,stop}}] - end, - Extra ++ ?DEFAULT_HANDLER_FILTERS([beam,erlang,otp,sasl]) + ?DEFAULT_HANDLER_FILTERS([otp,sasl]) end. get_logger_env() -> @@ -749,19 +757,14 @@ get_logger_env() -> %%%----------------------------------------------------------------- %%% Internal -do_log(warning,Msg,Meta) -> - do_log_1(error_logger:warning_map(),Msg,Meta); -do_log(Level,Msg,Meta) -> - do_log_1(Level,Msg,Meta). - -do_log_1(Level,Msg,#{mfa:={Module,_,_}}=Meta) -> +do_log(Level,Msg,#{mfa:={Module,_,_}}=Meta) -> case logger_config:allow(?LOGGER_TABLE,Level,Module) of true -> log_allowed(#{},Level,Msg,Meta); false -> ok end; -do_log_1(Level,Msg,Meta) -> +do_log(Level,Msg,Meta) -> case logger_config:allow(?LOGGER_TABLE,Level) of true -> log_allowed(#{},Level,Msg,Meta); diff --git a/lib/kernel/src/logger_backend.erl b/lib/kernel/src/logger_backend.erl index b3cf7d67dd..4d7bd6b2a0 100644 --- a/lib/kernel/src/logger_backend.erl +++ b/lib/kernel/src/logger_backend.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2017. All Rights Reserved. +%% Copyright Ericsson AB 2017-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -28,9 +28,9 @@ %%%----------------------------------------------------------------- %%% The default logger backend log_allowed(Log, Tid) -> - {ok,Config} = logger_config:get(Tid,logger), + {ok,Config} = logger_config:get(Tid,primary), Filters = maps:get(filters,Config,[]), - case apply_filters(logger,Log,Filters,Config) of + case apply_filters(primary,Log,Filters,Config) of stop -> ok; Log1 -> diff --git a/lib/kernel/src/logger_config.erl b/lib/kernel/src/logger_config.erl index 1d35c2e068..55427dce5a 100644 --- a/lib/kernel/src/logger_config.erl +++ b/lib/kernel/src/logger_config.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2017. All Rights Reserved. +%% Copyright Ericsson AB 2017-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -25,7 +25,7 @@ get/2, get/3, get/1, create/3, create/4, set/3, set_module_level/3,unset_module_level/2, - cache_module_level/2, + get_module_level/1,cache_module_level/2, level_to_int/1]). -include("logger_internal.hrl"). @@ -54,7 +54,7 @@ allow(Tid,Level,Module) -> end. allow(Tid,Level) -> - GlobalLevelInt = ets:lookup_element(Tid,?LOGGER_KEY,2), + GlobalLevelInt = ets:lookup_element(Tid,?PRIMARY_KEY,2), level_to_int(Level) =< GlobalLevelInt. exist(Tid,What) -> @@ -71,7 +71,7 @@ get(Tid,What) -> end. get(Tid,What,Level) -> - MS = [{{table_key(What),'$1','$2'}, % logger config + MS = [{{table_key(What),'$1','$2'}, % primary config [{'>=','$1',level_to_int(Level)}], ['$2']}, {{table_key(What),'$1','$2','$3'}, % handler config @@ -94,7 +94,7 @@ set(Tid,What,Config) -> %% Should do this only if the level has actually changed. Possibly %% overwrite instead of delete? case What of - logger -> + primary -> _ = ets:select_delete(Tid,[{{'_',{'$1',cached}}, [{'=/=','$1',LevelInt}], [true]}]), @@ -105,27 +105,38 @@ set(Tid,What,Config) -> ets:update_element(Tid,table_key(What),[{2,LevelInt},{3,Config}]), ok. -set_module_level(Tid,Module,Level) -> - ets:insert(Tid,{Module,level_to_int(Level)}), +set_module_level(Tid,Modules,Level) -> + LevelInt = level_to_int(Level), + [ets:insert(Tid,{Module,LevelInt}) || Module <- Modules], ok. -unset_module_level(Tid,Module) -> - ets:delete(Tid,Module), % should possibley overwrite instead of delete? +%% should possibly overwrite instead of delete? +unset_module_level(Tid,all) -> + MS = [{{'$1','$2'},[{is_atom,'$1'},{is_integer,'$2'}],[true]}], + _ = ets:select_delete(Tid,MS), + ok; +unset_module_level(Tid,Modules) -> + [ets:delete(Tid,Module) || Module <- Modules], ok. +get_module_level(Tid) -> + MS = [{{'$1','$2'},[{is_atom,'$1'},{is_integer,'$2'}],[{{'$1','$2'}}]}], + Modules = ets:select(Tid,MS), + lists:sort([{M,int_to_level(L)} || {M,L} <- Modules]). + cache_module_level(Tid,Module) -> - GlobalLevelInt = ets:lookup_element(Tid,?LOGGER_KEY,2), + GlobalLevelInt = ets:lookup_element(Tid,?PRIMARY_KEY,2), ets:insert_new(Tid,{Module,{GlobalLevelInt,cached}}), ok. get(Tid) -> - {ok,Logger} = get(Tid,logger), + {ok,Primary} = get(Tid,primary), HMS = [{{table_key('$1'),'_','$2','$3'},[],[{{'$1','$3','$2'}}]}], Handlers = ets:select(Tid,HMS), - MMS = [{{'$1','$2'},[{is_atom,'$1'},{is_integer,'$2'}],[{{'$1','$2'}}]}], - Modules = ets:select(Tid,MMS), - {Logger,Handlers,[{M,int_to_level(L)} || {M,L} <- Modules]}. + Modules = get_module_level(Tid), + {Primary,Handlers,Modules}. +level_to_int(none) -> ?LOG_NONE; level_to_int(emergency) -> ?EMERGENCY; level_to_int(alert) -> ?ALERT; level_to_int(critical) -> ?CRITICAL; @@ -133,8 +144,10 @@ level_to_int(error) -> ?ERROR; level_to_int(warning) -> ?WARNING; level_to_int(notice) -> ?NOTICE; level_to_int(info) -> ?INFO; -level_to_int(debug) -> ?DEBUG. +level_to_int(debug) -> ?DEBUG; +level_to_int(all) -> ?LOG_ALL. +int_to_level(?LOG_NONE) -> none; int_to_level(?EMERGENCY) -> emergency; int_to_level(?ALERT) -> alert; int_to_level(?CRITICAL) -> critical; @@ -142,10 +155,11 @@ int_to_level(?ERROR) -> error; int_to_level(?WARNING) -> warning; int_to_level(?NOTICE) -> notice; int_to_level(?INFO) -> info; -int_to_level(?DEBUG) -> debug. +int_to_level(?DEBUG) -> debug; +int_to_level(?LOG_ALL) -> all. %%%----------------------------------------------------------------- %%% Internal -table_key(logger) -> ?LOGGER_KEY; +table_key(primary) -> ?PRIMARY_KEY; table_key(HandlerId) -> {?HANDLER_KEY,HandlerId}. diff --git a/lib/kernel/src/logger_disk_log_h.erl b/lib/kernel/src/logger_disk_log_h.erl index 773aa75bc6..e56531c3cb 100644 --- a/lib/kernel/src/logger_disk_log_h.erl +++ b/lib/kernel/src/logger_disk_log_h.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2017. All Rights Reserved. +%% Copyright Ericsson AB 2017-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -26,16 +26,17 @@ -include("logger_h_common.hrl"). %%% API --export([start_link/3, info/1, disk_log_sync/1, reset/1]). +-export([start_link/3, info/1, filesync/1, reset/1]). %% gen_server callbacks -export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]). %% logger callbacks --export([log/2, - adding_handler/1, removing_handler/1, - changing_config/2, swap_buffer/2]). +-export([log/2, adding_handler/1, removing_handler/1, changing_config/2]). + +%% handler internal +-export([log_handler_info/4]). %%%=================================================================== %%% API @@ -47,7 +48,7 @@ %%% handler process gets added (as a result of calling add/3). -spec start_link(Name, Config, HandlerState) -> {ok,Pid} | {error,Reason} when Name :: atom(), - Config :: logger:config(), + Config :: logger:handler_config(), HandlerState :: map(), Pid :: pid(), Reason :: term(). @@ -57,18 +58,19 @@ start_link(Name, Config, HandlerState) -> %%%----------------------------------------------------------------- %%% --spec disk_log_sync(Name) -> ok | {error,Reason} when +-spec filesync(Name) -> ok | {error,Reason} when Name :: atom(), Reason :: handler_busy | {badarg,term()}. -disk_log_sync(Name) when is_atom(Name) -> +filesync(Name) when is_atom(Name) -> try - gen_server:call(Name, disk_log_sync, ?DEFAULT_CALL_TIMEOUT) + gen_server:call(?name_to_reg_name(?MODULE,Name), + disk_log_sync, ?DEFAULT_CALL_TIMEOUT) catch _:{timeout,_} -> {error,handler_busy} end; -disk_log_sync(Name) -> - {error,{badarg,{disk_log_sync,[Name]}}}. +filesync(Name) -> + {error,{badarg,{filesync,[Name]}}}. %%%----------------------------------------------------------------- %%% @@ -79,7 +81,8 @@ disk_log_sync(Name) -> info(Name) when is_atom(Name) -> try - gen_server:call(Name, info, ?DEFAULT_CALL_TIMEOUT) + gen_server:call(?name_to_reg_name(?MODULE,Name), + info, ?DEFAULT_CALL_TIMEOUT) catch _:{timeout,_} -> {error,handler_busy} end; @@ -94,7 +97,8 @@ info(Name) -> reset(Name) when is_atom(Name) -> try - gen_server:call(Name, reset, ?DEFAULT_CALL_TIMEOUT) + gen_server:call(?name_to_reg_name(?MODULE,Name), + reset, ?DEFAULT_CALL_TIMEOUT) catch _:{timeout,_} -> {error,handler_busy} end; @@ -112,23 +116,16 @@ adding_handler(#{id:=Name}=Config) -> case check_config(adding, Config) of {ok, Config1} -> %% create initial handler state by merging defaults with config - HConfig = maps:get(?MODULE, Config1, #{}), + HConfig = maps:get(config, Config1, #{}), HState = maps:merge(get_init_state(), HConfig), case logger_h_common:overload_levels_ok(HState) of true -> - case start(Name, Config1, HState) of - ok -> - %% Make sure wait_for_buffer is not stored, so we - %% won't hang and wait for buffer on a restart - {ok, maps:remove(wait_for_buffer,Config1)}; - Error -> - Error - end; + start(Name, Config1, HState); false -> - #{toggle_sync_qlen := TSQL, - drop_new_reqs_qlen := DNRQL, - flush_reqs_qlen := FRQL} = HState, - {error,{invalid_levels,{TSQL,DNRQL,FRQL}}} + #{sync_mode_qlen := SMQL, + drop_mode_qlen := DMQL, + flush_qlen := FQL} = HState, + {error,{invalid_levels,{SMQL,DMQL,FQL}}} end; Error -> Error @@ -136,86 +133,89 @@ adding_handler(#{id:=Name}=Config) -> %%%----------------------------------------------------------------- %%% Updating handler config -changing_config(OldConfig=#{id:=Name, disk_log_opts:=DLOpts}, - NewConfig=#{id:=Name, disk_log_opts:=DLOpts}) -> +changing_config(OldConfig = #{id:=Name, config:=OldHConfig}, + NewConfig = #{id:=Name, config:=NewHConfig}) -> + #{type:=Type, file:=File, max_no_files:=MaxFs, + max_no_bytes:=MaxBytes} = OldHConfig, + case NewHConfig of + #{type:=Type, file:=File, max_no_files:=MaxFs, + max_no_bytes:=MaxBytes} -> + changing_config1(OldConfig, NewConfig); + _ -> + {error,{illegal_config_change,OldConfig,NewConfig}} + end; +changing_config(OldConfig, NewConfig) -> + {error,{illegal_config_change,OldConfig,NewConfig}}. + +changing_config1(OldConfig=#{config:=OldHConfig}, NewConfig) -> case check_config(changing, NewConfig) of - Result = {ok,NewConfig1} -> - try gen_server:call(Name, {change_config,OldConfig,NewConfig1}, + {ok,NewConfig1 = #{config:=NewHConfig}} -> + #{handler_pid:=HPid, + mode_tab:=ModeTab} = OldHConfig, + NewHConfig1 = NewHConfig#{handler_pid=>HPid, + mode_tab=>ModeTab}, + NewConfig2 = NewConfig1#{config=>NewHConfig1}, + try gen_server:call(HPid, {change_config,OldConfig,NewConfig2}, ?DEFAULT_CALL_TIMEOUT) of - ok -> Result; + ok -> {ok,NewConfig2}; HError -> HError catch _:{timeout,_} -> {error,handler_busy} end; Error -> Error - end; -changing_config(OldConfig, NewConfig) -> - {error,{illegal_config_change,OldConfig,NewConfig}}. + end. check_config(adding, #{id:=Name}=Config) -> - %% Merge in defaults on handler level - LogOpts0 = maps:get(disk_log_opts, Config, #{}), - LogOpts = merge_default_logopts(Name, LogOpts0), - case check_log_opts(maps:to_list(LogOpts)) of + %% merge handler specific config data + HConfig = merge_default_logopts(Name, maps:get(config, Config, #{})), + case check_h_config(maps:to_list(HConfig)) of ok -> - MyConfig = maps:get(?MODULE, Config, #{}), - case check_my_config(maps:to_list(MyConfig)) of - ok -> - {ok,Config#{disk_log_opts=>LogOpts, - ?MODULE=>MyConfig}}; - Error -> - Error - end; + {ok,Config#{config=>HConfig}}; Error -> Error end; check_config(changing, Config) -> - MyConfig = maps:get(?MODULE, Config, #{}), - case check_my_config(maps:to_list(MyConfig)) of + HConfig = maps:get(config, Config, #{}), + case check_h_config(maps:to_list(HConfig)) of ok -> {ok,Config}; Error -> Error end. -merge_default_logopts(Name, LogOpts) -> - Type = maps:get(type, LogOpts, wrap), +merge_default_logopts(Name, HConfig) -> + Type = maps:get(type, HConfig, wrap), {DefaultNoFiles,DefaultNoBytes} = case Type of halt -> {undefined,infinity}; _wrap -> {10,1048576} end, {ok,Dir} = file:get_cwd(), - Default = #{file => filename:join(Dir,Name), - max_no_files => DefaultNoFiles, - max_no_bytes => DefaultNoBytes, - type => Type}, - maps:merge(Default,LogOpts). - -check_log_opts([{file,File}|Opts]) when is_list(File) -> - check_log_opts(Opts); -check_log_opts([{max_no_files,undefined}|Opts]) -> - check_log_opts(Opts); -check_log_opts([{max_no_files,N}|Opts]) when is_integer(N), N>0 -> - check_log_opts(Opts); -check_log_opts([{max_no_bytes,infinity}|Opts]) -> - check_log_opts(Opts); -check_log_opts([{max_no_bytes,N}|Opts]) when is_integer(N), N>0 -> - check_log_opts(Opts); -check_log_opts([{type,Type}|Opts]) when Type==wrap; Type==halt -> - check_log_opts(Opts); -check_log_opts([Invalid|_]) -> - {error,{invalid_config,disk_log_opt,Invalid}}; -check_log_opts([]) -> - ok. - -check_my_config([Other | Config]) -> + Defaults = #{file => filename:join(Dir,Name), + max_no_files => DefaultNoFiles, + max_no_bytes => DefaultNoBytes, + type => Type}, + maps:merge(Defaults, HConfig). + +check_h_config([{file,File}|Config]) when is_list(File) -> + check_h_config(Config); +check_h_config([{max_no_files,undefined}|Config]) -> + check_h_config(Config); +check_h_config([{max_no_files,N}|Config]) when is_integer(N), N>0 -> + check_h_config(Config); +check_h_config([{max_no_bytes,infinity}|Config]) -> + check_h_config(Config); +check_h_config([{max_no_bytes,N}|Config]) when is_integer(N), N>0 -> + check_h_config(Config); +check_h_config([{type,Type}|Config]) when Type==wrap; Type==halt -> + check_h_config(Config); +check_h_config([Other | Config]) -> case logger_h_common:check_common_config(Other) of valid -> - check_my_config(Config); + check_h_config(Config); invalid -> {error,{invalid_config,?MODULE,Other}} end; -check_my_config([]) -> +check_h_config([]) -> ok. %%%----------------------------------------------------------------- @@ -224,86 +224,93 @@ removing_handler(#{id:=Name}) -> stop(Name). %%%----------------------------------------------------------------- -%%% Get buffer when swapping from simple handler -swap_buffer(Name,Buffer) -> - case whereis(Name) of - undefined -> - ok; - _ -> - Name ! {buffer,Buffer} - end. - -%%%----------------------------------------------------------------- %%% Log a string or report -spec log(LogEvent, Config) -> ok | dropped when LogEvent :: logger:log_event(), - Config :: logger:config(). + Config :: logger:handler_config(). -log(LogEvent,Config=#{id:=Name}) -> - %% if the handler has crashed, we must drop this request +log(LogEvent, Config = #{id := Name, + config := #{handler_pid := HPid, + mode_tab := ModeTab}}) -> + %% if the handler has crashed, we must drop this event %% and hope the handler restarts so we can try again - true = is_pid(whereis(Name)), - Bin = logger_h_common:log_to_binary(LogEvent,Config), - logger_h_common:call_cast_or_drop(Name, Bin). - + true = is_process_alive(HPid), + Bin = logger_h_common:log_to_binary(LogEvent, Config), + logger_h_common:call_cast_or_drop(Name, HPid, ModeTab, Bin). %%%=================================================================== %%% gen_server callbacks %%%=================================================================== -init([Name, Config = #{disk_log_opts := LogOpts}, +init([Name, + Config = #{config := HConfig = #{file:=File, + type:=Type, + max_no_bytes:=MNB, + max_no_files:=MNF}}, State = #{dl_sync_int := DLSyncInt}]) -> - register(Name, self()), + + RegName = ?name_to_reg_name(?MODULE,Name), + register(RegName, self()), process_flag(trap_exit, true), process_flag(message_queue_data, off_heap), ?init_test_hooks(), ?start_observation(Name), - - case open_disk_log(Name, LogOpts) of + + LogOpts = #{file=>File, type=>Type, max_no_bytes=>MNB, max_no_files=>MNF}, + case open_disk_log(Name, File, Type, MNB, MNF) of ok -> - catch ets:new(Name, [public, named_table]), - ?set_mode(Name, async), - proc_lib:init_ack({ok,self()}), - T0 = ?timestamp(), - State1 = - ?merge_with_stats(State#{id => Name, - mode => async, - dl_sync => DLSyncInt, - log_opts => LogOpts, - last_qlen => 0, - last_log_ts => T0, - burst_win_ts => T0, - burst_msg_count => 0, - last_op => sync, - prev_log_result => ok, - prev_sync_result => ok, - prev_disk_log_info => undefined}), - gen_server:cast(self(), repeated_disk_log_sync), - enter_loop(Config, State1); + try ets:new(Name, [public]) of + ModeTab -> + ?set_mode(ModeTab, async), + T0 = ?timestamp(), + State1 = + ?merge_with_stats(State#{ + id => Name, + mode_tab => ModeTab, + mode => async, + dl_sync => DLSyncInt, + log_opts => LogOpts, + last_qlen => 0, + last_log_ts => T0, + burst_win_ts => T0, + burst_msg_count => 0, + last_op => sync, + prev_log_result => ok, + prev_sync_result => ok, + prev_disk_log_info => undefined}), + Config1 = + Config#{config => HConfig#{handler_pid => self(), + mode_tab => ModeTab}}, + proc_lib:init_ack({ok,self(),Config1}), + gen_server:cast(self(), repeated_disk_log_sync), + case logger_h_common:unset_restart_flag(Name, ?MODULE) of + true -> + %% inform about restart + gen_server:cast(self(), {log_handler_info, + "Handler ~p restarted", + [Name]}); + false -> + %% initial start + ok + end, + enter_loop(Config1, State1) + catch + _:Error -> + unregister(RegName), + logger_h_common:error_notify({open_disk_log,Name,Error}), + proc_lib:init_ack(Error) + end; Error -> + unregister(RegName), logger_h_common:error_notify({open_disk_log,Name,Error}), proc_lib:init_ack(Error) end. -enter_loop(#{wait_for_buffer:=true}=Config,State) -> - State1 = - receive - {buffer,Buffer} -> - lists:foldl( - fun(Log,S) -> - Bin = logger_h_common:log_to_binary(Log,Config), - {_,S1} = do_log(Bin,cast,S), - S1 - end, - State, - Buffer) - end, - gen_server:enter_loop(?MODULE,[],State1); enter_loop(_Config,State) -> gen_server:enter_loop(?MODULE,[],State). -%% This is the synchronous log request. +%% This is the synchronous log event. handle_call({log, Bin}, _From, State) -> {Result,State1} = do_log(Bin, call, State), %% Result == ok | dropped @@ -315,10 +322,10 @@ handle_call(disk_log_sync, _From, State = #{id := Name}) -> handle_call({change_config,_OldConfig,NewConfig}, _From, State = #{filesync_repeat_interval := FSyncInt0}) -> - HConfig = maps:get(?MODULE, NewConfig, #{}), - State1 = #{toggle_sync_qlen := TSQL, - drop_new_reqs_qlen := DNRQL, - flush_reqs_qlen := FRQL} = maps:merge(State, HConfig), + HConfig = maps:get(config, NewConfig, #{}), + State1 = #{sync_mode_qlen := SMQL, + drop_mode_qlen := DMQL, + flush_qlen := FQL} = maps:merge(State, HConfig), case logger_h_common:overload_levels_ok(State1) of true -> _ = @@ -339,7 +346,7 @@ handle_call({change_config,_OldConfig,NewConfig}, _From, end, {reply, ok, State1}; false -> - {reply, {error,{invalid_levels,{TSQL,DNRQL,FRQL}}}, State} + {reply, {error,{invalid_levels,{SMQL,DMQL,FQL}}}, State} end; handle_call(info, _From, State) -> @@ -357,15 +364,19 @@ handle_call(stop, _From, State) -> {stop, {shutdown,stopped}, ok, State}. -%% This is the asynchronous log request. +%% This is the asynchronous log event. handle_cast({log, Bin}, State) -> {_,State1} = do_log(Bin, cast, State), {noreply, State1}; +handle_cast({log_handler_info, Format, Args}, State = #{id:=Name}) -> + log_handler_info(Name, Format, Args, State), + {noreply, State}; + %% If FILESYNC_REPEAT_INTERVAL is set to a millisec value, this %% clause gets called repeatedly by the handler. In order to %% guarantee that a filesync *always* happens after the last log -%% request, the repeat operation must be active! +%% event, the repeat operation must be active! handle_cast(repeated_disk_log_sync, State = #{id := Name, filesync_repeat_interval := FSyncInt, @@ -418,6 +429,7 @@ terminate(Reason, State = #{id := Name}) -> _ = logger_h_common:cancel_timer(maps:get(rep_sync_tref, State, undefined)), _ = close_disk_log(Name, normal), + unregister(?name_to_reg_name(?MODULE, Name)), logger_h_common:stop_or_restart(Name, Reason, State). code_change(_OldVsn, State, _Extra) -> @@ -429,19 +441,19 @@ code_change(_OldVsn, State, _Extra) -> %%%----------------------------------------------------------------- %%% get_init_state() -> - #{toggle_sync_qlen => ?TOGGLE_SYNC_QLEN, - drop_new_reqs_qlen => ?DROP_NEW_REQS_QLEN, - flush_reqs_qlen => ?FLUSH_REQS_QLEN, - enable_burst_limit => ?ENABLE_BURST_LIMIT, - burst_limit_size => ?BURST_LIMIT_SIZE, - burst_window_time => ?BURST_WINDOW_TIME, - enable_kill_overloaded => ?ENABLE_KILL_OVERLOADED, - handler_overloaded_qlen => ?HANDLER_OVERLOADED_QLEN, - handler_overloaded_mem => ?HANDLER_OVERLOADED_MEM, - handler_restart_after => ?HANDLER_RESTART_AFTER, - dl_sync_int => ?CONTROLLER_SYNC_INTERVAL, - filesync_ok_qlen => ?FILESYNC_OK_QLEN, - filesync_repeat_interval => ?FILESYNC_REPEAT_INTERVAL}. + #{sync_mode_qlen => ?SYNC_MODE_QLEN, + drop_mode_qlen => ?DROP_MODE_QLEN, + flush_qlen => ?FLUSH_QLEN, + burst_limit_enable => ?BURST_LIMIT_ENABLE, + burst_limit_max_count => ?BURST_LIMIT_MAX_COUNT, + burst_limit_window_time => ?BURST_LIMIT_WINDOW_TIME, + overload_kill_enable => ?OVERLOAD_KILL_ENABLE, + overload_kill_qlen => ?OVERLOAD_KILL_QLEN, + overload_kill_mem_size => ?OVERLOAD_KILL_MEM_SIZE, + overload_kill_restart_after => ?OVERLOAD_KILL_RESTART_AFTER, + dl_sync_int => ?CONTROLLER_SYNC_INTERVAL, + filesync_ok_qlen => ?FILESYNC_OK_QLEN, + filesync_repeat_interval => ?FILESYNC_REPEAT_INTERVAL}. %%%----------------------------------------------------------------- %%% Add a disk_log handler to the logger. @@ -449,22 +461,21 @@ get_init_state() -> %%% exist if the handler is registered with logger (and should not %%% exist if the handler is not registered). %%% -%%% Config is the logger:config() map containing a sub map with any of -%%% the following associations: +%%% Config is the logger:handler_config() map. Handler specific parameters +%%% should be provided with a sub map associated with a key named +%%% 'config', e.g: %%% -%%% Config = #{disk_log_opts => #{file => file:filename(), -%%% max_no_bytes => integer(), -%%% max_no_files => integer(), -%%% type => wrap | halt}}. +%%% Config = #{config => #{sync_mode_qlen => 50} %%% -%%% This map will be merged with the logger configuration data for -%%% the disk_log LogName. If type == halt, then max_no_files is -%%% ignored. +%%% The 'config' sub map will also contain parameters for configuring +%%% the disk_log: %%% -%%% Handler specific config should be provided with a sub map associated -%%% with a key named the same as this module, e.g: +%%% Config = #{config => #{file => file:filename(), +%%% max_no_bytes => integer(), +%%% max_no_files => integer(), +%%% type => wrap | halt}}. %%% -%%% Config = #{logger_disk_log_h => #{toggle_sync_qlen => 50} +%%% If type == halt, then max_no_files is ignored. %%% %%% The disk_log handler process is linked to logger_sup, which is %%% part of the kernel application's supervision tree. @@ -477,8 +488,9 @@ start(Name, Config, HandlerState) -> type => worker, modules => [?MODULE]}, case supervisor:start_child(logger_sup, LoggerDLH) of - {ok,_} -> - ok; + {ok,Pid,Config1} -> + ok = logger_handler_watcher:register_handler(Name,Pid), + {ok,Config1}; Error -> Error end. @@ -486,17 +498,20 @@ start(Name, Config, HandlerState) -> %%%----------------------------------------------------------------- %%% Stop and remove the handler. stop(Name) -> - case whereis(Name) of + case whereis(?name_to_reg_name(?MODULE,Name)) of undefined -> ok; - _ -> + Pid -> %% We don't want to do supervisor:terminate_child here %% since we need to distinguish this explicit stop from a %% system termination in order to avoid circular attempts %% at removing the handler (implying deadlocks and %% timeouts). - _ = gen_server:call(Name,stop), - _ = supervisor:delete_child(logger_sup, Name), + %% And we don't need to do supervisor:delete_child, since + %% the restart type is temporary, which means that the + %% child specification is automatically removed from the + %% supervisor when the process dies. + _ = gen_server:call(Pid, stop), ok end. @@ -506,21 +521,31 @@ stop(Name) -> if C == 0 -> Interval; true -> C-1 end). -%% check for overload between every request (and set Mode to async, +%% check for overload between every event (and set Mode to async, %% sync or drop accordingly), but never flush the whole mailbox -%% before LogWindowSize requests have been handled -do_log(Bin, CallOrCast, State = #{id:=Name, mode := _Mode0}) -> +%% before LogWindowSize events have been handled +do_log(Bin, CallOrCast, State = #{id:=Name, mode := Mode0}) -> T1 = ?timestamp(), %% check if the handler is getting overloaded, or if it's %% recovering from overload (the check must be done for each - %% request to react quickly to large bursts of requests and + %% event to react quickly to large bursts of events and %% to ensure that the handler can never end up in drop mode %% with an empty mailbox, which would stop operation) {Mode1,QLen,Mem,State1} = logger_h_common:check_load(State), + if (Mode1 == drop) andalso (Mode0 =/= drop) -> + log_handler_info(Name, "Handler ~p switched to drop mode", + [Name], State); + (Mode0 == drop) andalso ((Mode1 == async) orelse (Mode1 == sync)) -> + log_handler_info(Name, "Handler ~p switched to ~w mode", + [Name,Mode1], State); + true -> + ok + end, + %% kill the handler if it can't keep up with the load - logger_h_common:kill_if_choked(Name, QLen, Mem, State), + logger_h_common:kill_if_choked(Name, QLen, Mem, ?MODULE, State), if Mode1 == flush -> flush(Name, QLen, T1, State1); @@ -530,10 +555,14 @@ do_log(Bin, CallOrCast, State = #{id:=Name, mode := _Mode0}) -> %% this function is called by do_log/3 after an overload check %% has been performed, where QLen > FlushQLen -flush(Name, _QLen0, T1, State=#{last_log_ts := _T0}) -> +flush(Name, _QLen0, T1, State=#{last_log_ts := _T0, mode_tab := ModeTab}) -> %% flush messages in the mailbox (a limited number in %% order to not cause long delays) - _NewFlushed = logger_h_common:flush_log_requests(?FLUSH_MAX_N), + NewFlushed = logger_h_common:flush_log_events(?FLUSH_MAX_N), + + %% write info in log about flushed messages + log_handler_info(Name, "Handler ~p flushed ~w log events", + [Name,NewFlushed], State), %% because of the receive loop when flushing messages, the %% handler will be scheduled out often and the mailbox could @@ -541,26 +570,27 @@ flush(Name, _QLen0, T1, State=#{last_log_ts := _T0}) -> {_,_QLen1} = process_info(self(), message_queue_len), ?observe(Name,{max_qlen,_QLen1}), - %% Add 1 for the current log request - ?observe(Name,{flushed,_NewFlushed+1}), + %% Add 1 for the current log event + ?observe(Name,{flushed,NewFlushed+1}), State1 = ?update_max_time(?diff_time(T1,_T0),State), - {dropped,?update_other(flushed,FLUSHED,_NewFlushed, - State1#{mode => ?set_mode(Name,async), + {dropped,?update_other(flushed,FLUSHED,NewFlushed, + State1#{mode => ?set_mode(ModeTab,async), last_qlen => 0, last_log_ts => T1})}. %% this function is called to write to disk_log write(Name, Mode, T1, Bin, _CallOrCast, - State = #{dl_sync := DLSync, + State = #{mode_tab := ModeTab, + dl_sync := DLSync, dl_sync_int := DLSyncInt, last_qlen := LastQLen, last_log_ts := T0}) -> %% check if we need to limit the number of writes - %% during a burst of log requests + %% during a burst of log events {DoWrite,BurstWinT,BurstMsgCount} = logger_h_common:limit_burst(State), - %% only send a synhrounous request to the disk_log process + %% only send a synhrounous event to the disk_log process %% every DLSyncInt time, to give the handler time between %% writes so it can keep up with incoming messages {Status,LastQLen1,State1} = @@ -578,18 +608,18 @@ write(Name, Mode, T1, Bin, _CallOrCast, {dropped, LastQLen, State} end, - %% Check if the time since the previous log request is long enough - + %% Check if the time since the previous log event is long enough - %% and the queue length small enough - to assume the mailbox has %% been emptied, and if so, do filesync operation and reset mode to %% async. Note that this is the best we can do to detect an idle %% handler without setting a timer after each log call/cast. If the - %% time between two consecutive log requests is fast and no new - %% request comes in after the last one, idle state won't be detected! + %% time between two consecutive log events is fast and no new + %% event comes in after the last one, idle state won't be detected! Time = ?diff_time(T1,T0), {Mode1,BurstMsgCount1,State2} = if (LastQLen1 < ?FILESYNC_OK_QLEN) andalso (Time > ?IDLE_DETECT_TIME_USEC) -> - {?change_mode(Name,Mode,async), 0, disk_log_sync(Name,State1)}; + {?change_mode(ModeTab,Mode,async), 0, disk_log_sync(Name,State1)}; true -> {Mode, BurstMsgCount,State1} end, @@ -607,11 +637,21 @@ write(Name, Mode, T1, Bin, _CallOrCast, {Status,State4}. -open_disk_log(Name, LogOpts) -> - #{file := File, - max_no_bytes := MaxNoBytes, - max_no_files := MaxNoFiles, - type := Type} = LogOpts, +log_handler_info(Name, Format, Args, State) -> + Config = + case logger:get_handler_config(Name) of + {ok,Conf} -> Conf; + _ -> #{formatter=>{?DEFAULT_FORMATTER,?DEFAULT_FORMAT_CONFIG}} + end, + Meta = #{time=>erlang:system_time(microsecond)}, + Bin = logger_h_common:log_to_binary(#{level => notice, + msg => {Format,Args}, + meta => Meta}, Config), + _ = disk_log_write(Name, Bin, State), + ok. + + +open_disk_log(Name, File, Type, MaxNoBytes, MaxNoFiles) -> case filelib:ensure_dir(File) of ok -> Size = @@ -672,7 +712,7 @@ disk_log_sync(Name, State) -> ok; _ -> LogOpts = maps:get(log_opts, State), - logger_h_common:error_notify({Name,sync, + logger_h_common:error_notify({Name,filesync, LogOpts, SyncError}) end, diff --git a/lib/kernel/src/logger_filters.erl b/lib/kernel/src/logger_filters.erl index 7359b3b4b7..0664c598e1 100644 --- a/lib/kernel/src/logger_filters.erl +++ b/lib/kernel/src/logger_filters.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2017. All Rights Reserved. +%% Copyright Ericsson AB 2017-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/kernel/src/logger_formatter.erl b/lib/kernel/src/logger_formatter.erl index 4d727b3da0..a5c6984bc6 100644 --- a/lib/kernel/src/logger_formatter.erl +++ b/lib/kernel/src/logger_formatter.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2017. All Rights Reserved. +%% Copyright Ericsson AB 2017-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -26,16 +26,18 @@ %%%----------------------------------------------------------------- %%% Types --type config() :: #{chars_limit=>pos_integer()| unlimited, - depth=>pos_integer() | unlimited, - legacy_header=>boolean(), - max_size=>pos_integer() | unlimited, - report_cb=>fun((logger:report()) -> {io:format(),[term()]}), - single_line=>boolean(), - template=>template(), - time_designator=>byte(), - time_offset=>integer()|[byte()]}. --type template() :: [atom()|tuple()|string()]. +-type config() :: #{chars_limit => pos_integer() | unlimited, + depth => pos_integer() | unlimited, + encoding => unicode:encoding(), + legacy_header => boolean(), + max_size => pos_integer() | unlimited, + report_cb => logger:report_cb(), + single_line => boolean(), + template => template(), + time_designator => byte(), + time_offset => integer() | [byte()]}. +-type template() :: [metakey() | {metakey(),template(),template()} | string()]. +-type metakey() :: atom() | [atom()]. %%%----------------------------------------------------------------- %%% API @@ -53,8 +55,8 @@ format(#{level:=Level,msg:=Msg0,meta:=Meta},Config0) [msg|Rest] -> {true,Rest}; _ ->{false,AT0} end, - B = do_format(Level,"",Meta1,BT,Config), - A = do_format(Level,"",Meta1,AT,Config), + B = do_format(Level,Meta1,BT,Config), + A = do_format(Level,Meta1,AT,Config), MsgStr = if DoMsg -> Config1 = @@ -75,7 +77,7 @@ format(#{level:=Level,msg:=Msg0,meta:=Meta},Config0) %% Trim leading and trailing whitespaces, and replace %% newlines with ", " re:replace(string:trim(MsgStr0),",?\r?\n\s*",", ", - [{return,list},global]); + [{return,list},global,unicode]); _false -> MsgStr0 end; @@ -84,88 +86,130 @@ format(#{level:=Level,msg:=Msg0,meta:=Meta},Config0) end, truncate(B ++ MsgStr ++ A,maps:get(max_size,Config)). -do_format(Level,Msg,Data,[level|Format],Config) -> - [to_string(level,Level,Config)|do_format(Level,Msg,Data,Format,Config)]; -do_format(Level,Msg,Data,[Key|Format],Config) when is_atom(Key); is_tuple(Key) -> - Value = value(Key,Data), - [to_string(Key,Value,Config)|do_format(Level,Msg,Data,Format,Config)]; -do_format(Level,Msg,Data,[Str|Format],Config) -> - [Str|do_format(Level,Msg,Data,Format,Config)]; -do_format(_Level,_Msg,_Data,[],_Config) -> +do_format(Level,Data,[level|Format],Config) -> + [to_string(level,Level,Config)|do_format(Level,Data,Format,Config)]; +do_format(Level,Data,[{Key,IfExist,Else}|Format],Config) -> + String = + case value(Key,Data) of + {ok,Value} -> do_format(Level,Data#{Key=>Value},IfExist,Config); + error -> do_format(Level,Data,Else,Config) + end, + [String|do_format(Level,Data,Format,Config)]; +do_format(Level,Data,[Key|Format],Config) + when is_atom(Key) orelse + (is_list(Key) andalso is_atom(hd(Key))) -> + String = + case value(Key,Data) of + {ok,Value} -> to_string(Key,Value,Config); + error -> "" + end, + [String|do_format(Level,Data,Format,Config)]; +do_format(Level,Data,[Str|Format],Config) -> + [Str|do_format(Level,Data,Format,Config)]; +do_format(_Level,_Data,[],_Config) -> []. -value(Key,Meta) when is_atom(Key), is_map(Meta) -> - maps:get(Key,Meta,""); -value(Key,_) when is_atom(Key) -> - ""; -value(Keys,Meta) when is_tuple(Keys) -> - value(tuple_to_list(Keys),Meta); -value([Key|Keys],Meta) -> - value(Keys,value(Key,Meta)); +value(Key,Meta) when is_map_key(Key,Meta) -> + {ok,maps:get(Key,Meta)}; +value([Key|Keys],Meta) when is_map_key(Key,Meta) -> + value(Keys,maps:get(Key,Meta)); value([],Value) -> - Value. + {ok,Value}; +value(_,_) -> + error. to_string(time,Time,Config) -> format_time(Time,Config); -to_string(mfa,MFA,_Config) -> - format_mfa(MFA); -to_string(_,Value,_Config) -> - to_string(Value). +to_string(mfa,MFA,Config) -> + format_mfa(MFA,Config); +to_string(_,Value,Config) -> + to_string(Value,Config). -to_string(X) when is_atom(X) -> +to_string(X,_) when is_atom(X) -> atom_to_list(X); -to_string(X) when is_integer(X) -> +to_string(X,_) when is_integer(X) -> integer_to_list(X); -to_string(X) when is_pid(X) -> +to_string(X,_) when is_pid(X) -> pid_to_list(X); -to_string(X) when is_reference(X) -> +to_string(X,_) when is_reference(X) -> ref_to_list(X); -to_string(X) when is_list(X) -> - case io_lib:printable_unicode_list(lists:flatten(X)) of +to_string(X,Config) when is_list(X) -> + case printable_list(lists:flatten(X)) of true -> X; - _ -> io_lib:format("~tp",[X]) + _ -> io_lib:format(p(Config),[X]) end; -to_string(X) -> - io_lib:format("~tp",[X]). +to_string(X,Config) -> + io_lib:format(p(Config),[X]). + +printable_list([]) -> + false; +printable_list(X) -> + io_lib:printable_list(X). format_msg({string,Chardata},Meta,Config) -> - format_msg({"~ts",[Chardata]},Meta,Config); -format_msg({report,_}=Msg,Meta,#{report_cb:=Fun}=Config) when is_function(Fun,1) -> + format_msg({s(Config),[Chardata]},Meta,Config); +format_msg({report,_}=Msg,Meta,#{report_cb:=Fun}=Config) + when is_function(Fun,1); is_function(Fun,2) -> format_msg(Msg,Meta#{report_cb=>Fun},maps:remove(report_cb,Config)); format_msg({report,Report},#{report_cb:=Fun}=Meta,Config) when is_function(Fun,1) -> try Fun(Report) of {Format,Args} when is_list(Format), is_list(Args) -> format_msg({Format,Args},maps:remove(report_cb,Meta),Config); Other -> - format_msg({"REPORT_CB ERROR: ~tp; Returned: ~tp", + P = p(Config), + format_msg({"REPORT_CB/1 ERROR: "++P++"; Returned: "++P, [Report,Other]},Meta,Config) - catch C:R -> - format_msg({"REPORT_CB CRASH: ~tp; Reason: ~tp", - [Report,{C,R}]},Meta,Config) + catch C:R:S -> + P = p(Config), + format_msg({"REPORT_CB/1 CRASH: "++P++"; Reason: "++P, + [Report,{C,R,logger:filter_stacktrace(?MODULE,S)}]}, + Meta,Config) + end; +format_msg({report,Report},#{report_cb:=Fun}=Meta,Config) when is_function(Fun,2) -> + try Fun(Report,maps:with([encoding,depth,chars_limit],Config)) of + String when ?IS_STRING(String) -> + try unicode:characters_to_list(String) + catch _:_ -> + P = p(Config), + format_msg({"REPORT_CB/2 ERROR: "++P++"; Returned: "++P, + [Report,String]},Meta,Config) + end; + Other -> + P = p(Config), + format_msg({"REPORT_CB/2 ERROR: "++P++"; Returned: "++P, + [Report,Other]},Meta,Config) + catch C:R:S -> + P = p(Config), + format_msg({"REPORT_CB/2 CRASH: "++P++"; Reason: "++P, + [Report,{C,R,logger:filter_stacktrace(?MODULE,S)}]}, + Meta,Config) end; format_msg({report,Report},Meta,Config) -> format_msg({report,Report}, Meta#{report_cb=>fun logger:format_report/1}, Config); -format_msg(Msg,_Meta,#{depth:=Depth,chars_limit:=CharsLimit}) -> - limit_size(Msg, Depth, CharsLimit). - -limit_size(Msg,Depth,unlimited) -> - limit_size(Msg,Depth,[]); -limit_size(Msg,Depth,CharsLimit) when is_integer(CharsLimit) -> - limit_size(Msg,Depth,[{chars_limit,CharsLimit}]); -limit_size({Format,Args},unlimited,Opts) when is_list(Opts) -> +format_msg(Msg,_Meta,#{depth:=Depth,chars_limit:=CharsLimit,encoding:=Enc}) -> + limit_size(Msg, Depth, CharsLimit, Enc). + +limit_size(Msg,Depth,unlimited,Enc) -> + limit_size(Msg,Depth,[],Enc); +limit_size(Msg,Depth,CharsLimit,Enc) when is_integer(CharsLimit) -> + limit_size(Msg,Depth,[{chars_limit,CharsLimit}],Enc); +limit_size({Format,Args},unlimited,Opts,Enc) when is_list(Opts) -> try io_lib:format(Format,Args,Opts) catch _:_ -> - io_lib:format("FORMAT ERROR: ~tp - ~tp",[Format,Args],Opts) + P = p(Enc), + io_lib:format("FORMAT ERROR: "++P++" - "++P,[Format,Args],Opts) end; -limit_size({Format0,Args},Depth,Opts) when is_integer(Depth) -> +limit_size({Format0,Args},Depth,Opts,Enc) when is_integer(Depth) -> try Format1 = io_lib:scan_format(Format0, Args), Format = limit_format(Format1, Depth), io_lib:build_text(Format,Opts) catch _:_ -> - limit_size({"FORMAT ERROR: ~tp - ~tp",[Format0,Args]},Depth,Opts) + P = p(Enc), + limit_size({"FORMAT ERROR: "++P++" - "++P,[Format0,Args]}, + Depth,Opts,Enc) end. limit_format([#{control_char:=C0}=M0|T], Depth) when C0 =:= $p; @@ -213,31 +257,44 @@ timestamp_to_datetimemicro(SysTime,Config) when is_integer(SysTime) -> end, {Date,Time,Micro,UtcStr}. -format_mfa({M,F,A}) when is_atom(M), is_atom(F), is_integer(A) -> +format_mfa({M,F,A},_) when is_atom(M), is_atom(F), is_integer(A) -> atom_to_list(M)++":"++atom_to_list(F)++"/"++integer_to_list(A); -format_mfa({M,F,A}) when is_atom(M), is_atom(F), is_list(A) -> - format_mfa({M,F,length(A)}); -format_mfa(MFA) -> - to_string(MFA). +format_mfa({M,F,A},Config) when is_atom(M), is_atom(F), is_list(A) -> + format_mfa({M,F,length(A)},Config); +format_mfa(MFA,Config) -> + to_string(MFA,Config). maybe_add_legacy_header(Level, #{time:=Timestamp}=Meta, #{legacy_header:=true}=Config) -> - #{title:=Title}=MyMeta = add_legacy_title(Level,maps:get(?MODULE,Meta,#{})), - {{Y,Mo,D},{H,Mi,S},Micro,UtcStr} = + #{title:=Title}=MyMeta = add_legacy_title(Level,Meta,Config), + {{Y,Mo,D},{H,Mi,Sec},Micro,UtcStr} = timestamp_to_datetimemicro(Timestamp,Config), + S = s(Config), Header = - io_lib:format("=~ts==== ~w-~s-~4w::~2..0w:~2..0w:~2..0w.~6..0w ~s===", - [Title,D,month(Mo),Y,H,Mi,S,Micro,UtcStr]), + io_lib:format("="++S++"==== ~w-~s-~4w::~2..0w:~2..0w:~2..0w.~6..0w ~s===", + [Title,D,month(Mo),Y,H,Mi,Sec,Micro,UtcStr]), Meta#{?MODULE=>MyMeta#{header=>Header}}; maybe_add_legacy_header(_,Meta,_) -> Meta. -add_legacy_title(_Level,#{title:=_}=MyMeta) -> +add_legacy_title(_Level,#{?MODULE:=#{title:=_}=MyMeta},_) -> MyMeta; -add_legacy_title(Level,MyMeta) -> - Title = string:uppercase(atom_to_list(Level)) ++ " REPORT", - MyMeta#{title=>Title}. +add_legacy_title(Level,Meta,Config) -> + case maps:get(?MODULE,Meta,#{}) of + #{title:=_}=MyMeta -> + MyMeta; + MyMeta -> + TitleLevel = + case (Level=:=notice andalso maps:find(error_logger,Meta)) of + {ok,_} -> + maps:get(error_logger_notice_header,Config); + _ -> + Level + end, + Title = string:uppercase(atom_to_list(TitleLevel)) ++ " REPORT", + MyMeta#{title=>Title} + end. month(1) -> "Jan"; month(2) -> "Feb"; @@ -256,9 +313,11 @@ month(12) -> "Dec". %% configuration map add_default_config(Config0) -> Default = - #{legacy_header=>false, + #{chars_limit=>unlimited, + encoding=>utf8, + error_logger_notice_header=>info, + legacy_header=>false, single_line=>true, - chars_limit=>unlimited, time_designator=>$T}, MaxSize = get_max_size(maps:get(max_size,Config0,undefined)), Depth = get_depth(maps:get(depth,Config0,undefined)), @@ -341,19 +400,16 @@ do_check_config([{single_line,SL}|Config]) when is_boolean(SL) -> do_check_config(Config); do_check_config([{legacy_header,LH}|Config]) when is_boolean(LH) -> do_check_config(Config); -do_check_config([{report_cb,RCB}|Config]) when is_function(RCB,1) -> +do_check_config([{error_logger_notice_header,ELNH}|Config]) when ELNH == info; + ELNH == notice -> do_check_config(Config); -do_check_config([{template,T}|Config]) when is_list(T) -> - case lists:all(fun(X) when is_atom(X) -> true; - (X) when is_tuple(X), is_atom(element(1,X)) -> true; - (X) when is_list(X) -> io_lib:printable_unicode_list(X); - (_) -> false - end, - T) of - true -> - do_check_config(Config); - false -> - {error,{invalid_formatter_template,?MODULE,T}} +do_check_config([{report_cb,RCB}|Config]) when is_function(RCB,1); + is_function(RCB,2) -> + do_check_config(Config); +do_check_config([{template,T}|Config]) -> + case check_template(T) of + ok -> do_check_config(Config); + error -> {error,{invalid_formatter_template,?MODULE,T}} end; do_check_config([{time_offset,Offset}|Config]) -> case check_offset(Offset) of @@ -381,6 +437,42 @@ check_limit(unlimited) -> check_limit(_) -> error. +check_template([Key|T]) when is_atom(Key) -> + check_template(T); +check_template([Key|T]) when is_list(Key), is_atom(hd(Key)) -> + case lists:all(fun(X) when is_atom(X) -> true; + (_) -> false + end, + Key) of + true -> + check_template(T); + false -> + error + end; +check_template([{Key,IfExist,Else}|T]) + when is_atom(Key) orelse + (is_list(Key) andalso is_atom(hd(Key))) -> + case check_template(IfExist) of + ok -> + case check_template(Else) of + ok -> + check_template(T); + error -> + error + end; + error -> + error + end; +check_template([Str|T]) when is_list(Str) -> + case io_lib:printable_unicode_list(Str) of + true -> check_template(T); + false -> error + end; +check_template([]) -> + ok; +check_template(_) -> + error. + check_offset(I) when is_integer(I) -> ok; check_offset(Tz) when Tz=:=""; Tz=:="Z"; Tz=:="z" -> @@ -399,3 +491,17 @@ check_timezone(Tz) -> catch _:_ -> error end. + +p(#{encoding:=Enc}) -> + p(Enc); +p(latin1) -> + "~p"; +p(_) -> + "~tp". + +s(#{encoding:=Enc}) -> + s(Enc); +s(latin1) -> + "~s"; +s(_) -> + "~ts". diff --git a/lib/kernel/src/logger_h_common.erl b/lib/kernel/src/logger_h_common.erl index 336398cd4a..854e5479b9 100644 --- a/lib/kernel/src/logger_h_common.erl +++ b/lib/kernel/src/logger_h_common.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2017. All Rights Reserved. +%% Copyright Ericsson AB 2017-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -24,13 +24,15 @@ -export([log_to_binary/2, check_common_config/1, - call_cast_or_drop/2, + call_cast_or_drop/4, check_load/1, limit_burst/1, - kill_if_choked/4, - flush_log_requests/0, - flush_log_requests/1, + kill_if_choked/5, + flush_log_events/0, + flush_log_events/1, handler_exit/2, + set_restart_flag/2, + unset_restart_flag/2, cancel_timer/1, stop_or_restart/3, overload_levels_ok/1, @@ -41,7 +43,7 @@ %%% Covert log data on any form to binary -spec log_to_binary(LogEvent,Config) -> LogString when LogEvent :: logger:log_event(), - Config :: logger:config(), + Config :: logger:handler_config(), LogString :: binary(). log_to_binary(#{msg:={report,_},meta:=#{report_cb:=_}}=Log,Config) -> do_log_to_binary(Log,Config); @@ -52,7 +54,8 @@ log_to_binary(Log,Config) -> do_log_to_binary(Log,Config). do_log_to_binary(Log,Config) -> - {Formatter,FormatterConfig} = maps:get(formatter,Config), + {Formatter,FormatterConfig} = + maps:get(formatter,Config,{?DEFAULT_FORMATTER,?DEFAULT_FORMAT_CONFIG}), String = try_format(Log,Formatter,FormatterConfig), try unicode:characters_to_binary(String) catch _:_ -> @@ -84,29 +87,37 @@ try_format(Log,Formatter,FormatterConfig) -> %%%----------------------------------------------------------------- %%% Check that the configuration term is valid -check_common_config({toggle_sync_qlen,N}) when is_integer(N) -> +check_common_config({mode_tab,_Tid}) -> valid; -check_common_config({drop_new_reqs_qlen,N}) when is_integer(N) -> +check_common_config({handler_pid,Pid}) when is_pid(Pid) -> valid; -check_common_config({flush_reqs_qlen,N}) when is_integer(N) -> + +check_common_config({sync_mode_qlen,N}) when is_integer(N) -> + valid; +check_common_config({drop_mode_qlen,N}) when is_integer(N) -> valid; -check_common_config({enable_burst_limit,Bool}) when Bool == true; +check_common_config({flush_qlen,N}) when is_integer(N) -> + valid; + +check_common_config({burst_limit_enable,Bool}) when Bool == true; Bool == false -> valid; -check_common_config({burst_limit_size,N}) when is_integer(N) -> +check_common_config({burst_limit_max_count,N}) when is_integer(N) -> valid; -check_common_config({burst_window_time,N}) when is_integer(N) -> +check_common_config({burst_limit_window_time,N}) when is_integer(N) -> valid; -check_common_config({enable_kill_overloaded,Bool}) when Bool == true; - Bool == false -> + +check_common_config({overload_kill_enable,Bool}) when Bool == true; + Bool == false -> valid; -check_common_config({handler_overloaded_qlen,N}) when is_integer(N) -> +check_common_config({overload_kill_qlen,N}) when is_integer(N) -> valid; -check_common_config({handler_overloaded_mem,N}) when is_integer(N) -> +check_common_config({overload_kill_mem_size,N}) when is_integer(N) -> valid; -check_common_config({handler_restart_after,NorA}) when is_integer(NorA); - NorA == never -> +check_common_config({overload_kill_restart_after,NorA}) when is_integer(NorA); + NorA == infinity -> valid; + check_common_config({filesync_repeat_interval,NorA}) when is_integer(NorA); NorA == no_repeat -> valid; @@ -116,16 +127,16 @@ check_common_config(_) -> %%%----------------------------------------------------------------- %%% Overload Protection -call_cast_or_drop(Name, Bin) -> - %% If the handler process is getting overloaded, the log request +call_cast_or_drop(_Name, HandlerPid, ModeTab, Bin) -> + %% If the handler process is getting overloaded, the log event %% will be synchronous instead of asynchronous (slows down the %% logging tempo of a process doing lots of logging. If the - %% handler is choked, drop mode is set and no request will be sent. - try ?get_mode(Name) of + %% handler is choked, drop mode is set and no event will be sent. + try ?get_mode(ModeTab) of async -> - gen_server:cast(Name, {log,Bin}); + gen_server:cast(HandlerPid, {log,Bin}); sync -> - try gen_server:call(Name, {log,Bin}, ?DEFAULT_CALL_TIMEOUT) of + try gen_server:call(HandlerPid, {log,Bin}, ?DEFAULT_CALL_TIMEOUT) of %% if return value from call == dropped, the %% message has been flushed by handler and should %% therefore not be counted as dropped in stats @@ -133,32 +144,50 @@ call_cast_or_drop(Name, Bin) -> dropped -> ok catch _:{timeout,_} -> - ?observe(Name,{dropped,1}) + ?observe(_Name,{dropped,1}) end; drop -> - ?observe(Name,{dropped,1}) + ?observe(_Name,{dropped,1}) catch %% if the ETS table doesn't exist (maybe because of a - %% handler restart), we can only drop the request - _:_ -> ?observe(Name,{dropped,1}) + %% handler restart), we can only drop the event + _:_ -> ?observe(_Name,{dropped,1}) end, ok. handler_exit(_Name, Reason) -> exit(Reason). -check_load(State = #{id:=Name, mode := Mode, - toggle_sync_qlen := ToggleSyncQLen, - drop_new_reqs_qlen := DropNewQLen, - flush_reqs_qlen := FlushQLen}) -> +set_restart_flag(Name, Module) -> + Flag = list_to_atom(lists:concat([Module,"_",Name,"_restarting"])), + spawn(fun() -> + register(Flag, self()), + timer:sleep(infinity) + end), + ok. + +unset_restart_flag(Name, Module) -> + Flag = list_to_atom(lists:concat([Module,"_",Name,"_restarting"])), + case whereis(Flag) of + undefined -> + false; + Pid -> + exit(Pid, kill), + true + end. + +check_load(State = #{id:=_Name, mode_tab := ModeTab, mode := Mode, + sync_mode_qlen := SyncModeQLen, + drop_mode_qlen := DropModeQLen, + flush_qlen := FlushQLen}) -> {_,Mem} = process_info(self(), memory), - ?observe(Name,{max_mem,Mem}), + ?observe(_Name,{max_mem,Mem}), {_,QLen} = process_info(self(), message_queue_len), - ?observe(Name,{max_qlen,QLen}), + ?observe(_Name,{max_qlen,QLen}), %% When the handler process gets scheduled in, it's impossible %% to predict the QLen. We could jump "up" arbitrarily from say %% async to sync, async to drop, sync to flush, etc. However, when - %% the handler process manages the log requests (without flushing), + %% the handler process manages the log events (without flushing), %% one after the other, we will move "down" from drop to sync and %% from sync to async. This way we don't risk getting stuck in %% drop or sync mode with an empty mailbox. @@ -166,33 +195,33 @@ check_load(State = #{id:=Name, mode := Mode, if QLen >= FlushQLen -> {flush, 0,1}; - QLen >= DropNewQLen -> - %% Note that drop mode will force log requests to + QLen >= DropModeQLen -> + %% Note that drop mode will force log events to %% be dropped on the client side (never sent get to %% the handler). IncDrops = if Mode == drop -> 0; true -> 1 end, - {?change_mode(Name, Mode, drop), IncDrops,0}; - QLen >= ToggleSyncQLen -> - {?change_mode(Name, Mode, sync), 0,0}; + {?change_mode(ModeTab, Mode, drop), IncDrops,0}; + QLen >= SyncModeQLen -> + {?change_mode(ModeTab, Mode, sync), 0,0}; true -> - {?change_mode(Name, Mode, async), 0,0} + {?change_mode(ModeTab, Mode, async), 0,0} end, State1 = ?update_other(drops,DROPS,_NewDrops,State), {Mode1, QLen, Mem, ?update_other(flushes,FLUSHES,_NewFlushes, State1#{last_qlen => QLen})}. -limit_burst(#{enable_burst_limit := false}) -> +limit_burst(#{burst_limit_enable := false}) -> {true,0,0}; limit_burst(#{burst_win_ts := BurstWinT0, burst_msg_count := BurstMsgCount, - burst_window_time := BurstWinTime, - burst_limit_size := BurstLimitSz}) -> - if (BurstMsgCount >= BurstLimitSz) -> + burst_limit_window_time := BurstLimitWinTime, + burst_limit_max_count := BurstLimitMaxCnt}) -> + if (BurstMsgCount >= BurstLimitMaxCnt) -> %% the limit for allowed messages has been reached BurstWinT1 = ?timestamp(), case ?diff_time(BurstWinT1,BurstWinT0) of - BurstCheckTime when BurstCheckTime < (BurstWinTime*1000) -> + BurstCheckTime when BurstCheckTime < (BurstLimitWinTime*1000) -> %% we're still within the burst time frame {false,BurstWinT0,BurstMsgCount}; _BurstCheckTime -> @@ -204,38 +233,42 @@ limit_burst(#{burst_win_ts := BurstWinT0, {true,BurstWinT0,BurstMsgCount+1} end. -kill_if_choked(Name, QLen, Mem, - #{enable_kill_overloaded := KillIfOL, - handler_overloaded_qlen := HOLQLen, - handler_overloaded_mem := HOLMem}) -> +kill_if_choked(Name, QLen, Mem, HandlerMod, + State = #{overload_kill_enable := KillIfOL, + overload_kill_qlen := OLKillQLen, + overload_kill_mem_size := OLKillMem}) -> if KillIfOL andalso - ((QLen > HOLQLen) orelse (Mem > HOLMem)) -> + ((QLen > OLKillQLen) orelse (Mem > OLKillMem)) -> + HandlerMod:log_handler_info(Name, + "Handler ~p overloaded and stopping", + [Name], State), + set_restart_flag(Name, HandlerMod), handler_exit(Name, {shutdown,{overloaded,Name,QLen,Mem}}); true -> ok end. -flush_log_requests() -> - flush_log_requests(-1). +flush_log_events() -> + flush_log_events(-1). -flush_log_requests(Limit) -> +flush_log_events(Limit) -> process_flag(priority, high), - Flushed = flush_log_requests(0, Limit), + Flushed = flush_log_events(0, Limit), process_flag(priority, normal), Flushed. -flush_log_requests(Limit, Limit) -> +flush_log_events(Limit, Limit) -> Limit; -flush_log_requests(N, Limit) -> - %% flush log requests but leave other requests, such as - %% file/disk_log_sync, info and change_config, so that these +flush_log_events(N, Limit) -> + %% flush log events but leave other events, such as + %% filesync, info and change_config, so that these %% have a chance to be processed even under heavy load receive {'$gen_cast',{log,_}} -> - flush_log_requests(N+1, Limit); + flush_log_events(N+1, Limit); {'$gen_call',{Pid,MRef},{log,_}} -> Pid ! {MRef, dropped}, - flush_log_requests(N+1, Limit) + flush_log_events(N+1, Limit) after 0 -> N end. @@ -245,7 +278,7 @@ cancel_timer(TRef) -> timer:cancel(TRef). stop_or_restart(Name, {shutdown,Reason={overloaded,_Name,_QLen,_Mem}}, - #{handler_restart_after := RestartAfter}) -> + #{overload_kill_restart_after := RestartAfter}) -> %% If we're terminating because of an overload situation (see %% logger_h_common:kill_if_choked/4), we need to remove the handler %% and set a restart timer. A separate process must perform this @@ -262,7 +295,7 @@ stop_or_restart(Name, {shutdown,Reason={overloaded,_Name,_QLen,_Mem}}, exit(HandlerPid, kill) end, case logger:get_handler_config(Name) of - {ok,{HMod,HConfig}} when is_integer(RestartAfter) -> + {ok,#{module:=HMod}=HConfig} when is_integer(RestartAfter) -> _ = logger:remove_handler(Name), _ = timer:apply_after(RestartAfter, logger, add_handler, [Name,HMod,HConfig]); @@ -276,27 +309,14 @@ stop_or_restart(Name, {shutdown,Reason={overloaded,_Name,_QLen,_Mem}}, end, spawn(RemoveAndRestart), ok; - -stop_or_restart(Name, shutdown, _State) -> - %% Probably terminated by supervisor. Remove the handler to avoid - %% error printouts due to failing handler. - _ = case logger:get_handler_config(Name) of - {ok,_} -> - %% Spawning to avoid deadlock - spawn(logger,remove_handler,[Name]); - _ -> - ok - end, - ok; - stop_or_restart(_Name, _Reason, _State) -> ok. overload_levels_ok(HandlerConfig) -> - TSQL = maps:get(toggle_sync_qlen, HandlerConfig, ?TOGGLE_SYNC_QLEN), - DNRQL = maps:get(drop_new_reqs_qlen, HandlerConfig, ?DROP_NEW_REQS_QLEN), - FRQL = maps:get(flush_reqs_qlen, HandlerConfig, ?FLUSH_REQS_QLEN), - (DNRQL > 1) andalso (TSQL =< DNRQL) andalso (DNRQL =< FRQL). + SMQL = maps:get(sync_mode_qlen, HandlerConfig, ?SYNC_MODE_QLEN), + DMQL = maps:get(drop_mode_qlen, HandlerConfig, ?DROP_MODE_QLEN), + FQL = maps:get(flush_qlen, HandlerConfig, ?FLUSH_QLEN), + (DMQL > 1) andalso (SMQL =< DMQL) andalso (DMQL =< FQL). error_notify(Term) -> ?internal_log(error, Term). diff --git a/lib/kernel/src/logger_h_common.hrl b/lib/kernel/src/logger_h_common.hrl index ed365ce6eb..e0a7b6e3ca 100644 --- a/lib/kernel/src/logger_h_common.hrl +++ b/lib/kernel/src/logger_h_common.hrl @@ -4,47 +4,47 @@ %%! *** NOTE *** %%! It's important that: -%%! TOGGLE_SYNC_QLEN < DROP_NEW_REQS_QLEN < FLUSH_REQS_QLEN -%%! and that DROP_NEW_REQS_QLEN >= 2. +%%! SYNC_MODE_QLEN =< DROP_MODE_QLEN =< FLUSH_QLEN +%%! and that DROP_MODE_QLEN >= 2. %%! Otherwise the handler could end up in drop mode with no new %%! log requests to process. This would cause all future requests %%! to be dropped (no switch to async mode would ever take place). %% This specifies the message_queue_len value where the log %% requests switch from asynchronous casts to synchronous calls. --define(TOGGLE_SYNC_QLEN, 10). +-define(SYNC_MODE_QLEN, 10). %% Above this message_queue_len, log requests will be dropped, %% i.e. no log requests get sent to the handler process. --define(DROP_NEW_REQS_QLEN, 200). +-define(DROP_MODE_QLEN, 200). %% Above this message_queue_len, the handler process will flush %% its mailbox and only leave this number of messages in it. --define(FLUSH_REQS_QLEN, 1000). +-define(FLUSH_QLEN, 1000). %% Never flush more than this number of messages in one go, %% or the handler will be unresponsive for seconds (keep this %% number as large as possible or the mailbox could grow large). -define(FLUSH_MAX_N, 5000). -%% BURST_LIMIT is the max number of log requests allowed to be -%% written within a BURST_WINDOW_TIME time frame. --define(ENABLE_BURST_LIMIT, true). --define(BURST_LIMIT_SIZE, 500). --define(BURST_WINDOW_TIME, 1000). +%% BURST_LIMIT_MAX_COUNT is the max number of log requests allowed +%% to be written within a BURST_LIMIT_WINDOW_TIME time frame. +-define(BURST_LIMIT_ENABLE, true). +-define(BURST_LIMIT_MAX_COUNT, 500). +-define(BURST_LIMIT_WINDOW_TIME, 1000). %% This enables/disables the feature to automatically get the %% handler terminated if it gets too loaded (and can't keep up). --define(ENABLE_KILL_OVERLOADED, false). +-define(OVERLOAD_KILL_ENABLE, false). %% If the message_queue_len goes above this size even after %% flushing has been performed, the handler is terminated. --define(HANDLER_OVERLOADED_QLEN, 20000). +-define(OVERLOAD_KILL_QLEN, 20000). %% If the memory usage exceeds this level --define(HANDLER_OVERLOADED_MEM, 3000000). +-define(OVERLOAD_KILL_MEM_SIZE, 3000000). %% This is the default time that the handler will wait before -%% restarting and accepting new requests. The value 'never' +%% restarting and accepting new requests. The value 'infinity' %% disables restarts. --define(HANDLER_RESTART_AFTER, 5000). -%%-define(HANDLER_RESTART_AFTER, never). +-define(OVERLOAD_KILL_RESTART_AFTER, 5000). +%%-define(OVERLOAD_KILL_RESTART_AFTER, infinity). %% The handler sends asynchronous write requests to the process %% controlling the i/o device, but every once in this interval @@ -77,29 +77,30 @@ -define(DISK_LOG_MAX_NO_BYTES, 1048576). %%%----------------------------------------------------------------- +%%% Utility macros + +-define(name_to_reg_name(MODULE,Name), + list_to_atom(lists:concat([MODULE,"_",Name]))). + +%%%----------------------------------------------------------------- %%% Overload protection macros -define(timestamp(), erlang:monotonic_time(microsecond)). --define(get_mode(HandlerName), - case ets:lookup(HandlerName, mode) of - [{mode,sync}] -> - case whereis(HandlerName)==self() of - true -> async; - _ -> sync - end; +-define(get_mode(Tid), + case ets:lookup(Tid, mode) of [{mode,M}] -> M; _ -> async end). --define(set_mode(HandlerName, M), - begin ets:insert(HandlerName, {mode,M}), M end). +-define(set_mode(Tid, M), + begin ets:insert(Tid, {mode,M}), M end). --define(change_mode(HandlerName, M0, M1), +-define(change_mode(Tid, M0, M1), if M0 == M1 -> M0; true -> - ets:insert(HandlerName, {mode,M1}), + ets:insert(Tid, {mode,M1}), M1 end). @@ -124,7 +125,7 @@ %%% slow down execution and therefore should not be include in code %%% to be officially released. --define(TEST_HOOKS, true). +%%-define(TEST_HOOKS, true). -ifdef(TEST_HOOKS). -define(TEST_HOOKS_TAB, logger_h_test_hooks). diff --git a/lib/kernel/src/logger_handler_watcher.erl b/lib/kernel/src/logger_handler_watcher.erl new file mode 100644 index 0000000000..b75c74c643 --- /dev/null +++ b/lib/kernel/src/logger_handler_watcher.erl @@ -0,0 +1,113 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2018. All Rights Reserved. +%% +%% Licensed under the Apache License, Version 2.0 (the "License"); +%% you may not use this file except in compliance with the License. +%% You may obtain a copy of the License at +%% +%% http://www.apache.org/licenses/LICENSE-2.0 +%% +%% Unless required by applicable law or agreed to in writing, software +%% distributed under the License is distributed on an "AS IS" BASIS, +%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +%% See the License for the specific language governing permissions and +%% limitations under the License. +%% +%% %CopyrightEnd% +%% +-module(logger_handler_watcher). + +-behaviour(gen_server). + +%% API +-export([start_link/0]). +-export([register_handler/2]). + +%% gen_server callbacks +-export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2]). + +-define(SERVER, ?MODULE). + +-record(state, {handlers}). + +%%%=================================================================== +%%% API +%%%=================================================================== + +-spec start_link() -> {ok, Pid :: pid()} | + {error, Error :: {already_started, pid()}} | + {error, Error :: term()} | + ignore. +start_link() -> + gen_server:start_link({local, ?SERVER}, ?MODULE, [], []). + +-spec register_handler(Id::logger:handler_id(),Pid::pid()) -> ok. +register_handler(Id,Pid) -> + gen_server:call(?SERVER,{register,Id,Pid}). + +%%%=================================================================== +%%% gen_server callbacks +%%%=================================================================== + +-spec init(Args :: term()) -> {ok, State :: term()} | + {ok, State :: term(), Timeout :: timeout()} | + {ok, State :: term(), hibernate} | + {stop, Reason :: term()} | + ignore. +init([]) -> + process_flag(trap_exit, true), + {ok, #state{handlers=[]}}. + +-spec handle_call(Request :: term(), From :: {pid(), term()}, State :: term()) -> + {reply, Reply :: term(), NewState :: term()} | + {reply, Reply :: term(), NewState :: term(), Timeout :: timeout()} | + {reply, Reply :: term(), NewState :: term(), hibernate} | + {noreply, NewState :: term()} | + {noreply, NewState :: term(), Timeout :: timeout()} | + {noreply, NewState :: term(), hibernate} | + {stop, Reason :: term(), Reply :: term(), NewState :: term()} | + {stop, Reason :: term(), NewState :: term()}. +handle_call({register,Id,Pid}, _From, #state{handlers=Hs}=State) -> + Ref = erlang:monitor(process,Pid), + Hs1 = lists:keystore(Id,1,Hs,{Id,Ref}), + {reply, ok, State#state{handlers=Hs1}}. + +-spec handle_cast(Request :: term(), State :: term()) -> + {noreply, NewState :: term()} | + {noreply, NewState :: term(), Timeout :: timeout()} | + {noreply, NewState :: term(), hibernate} | + {stop, Reason :: term(), NewState :: term()}. +handle_cast(_Request, State) -> + {noreply, State}. + +-spec handle_info(Info :: timeout() | term(), State :: term()) -> + {noreply, NewState :: term()} | + {noreply, NewState :: term(), Timeout :: timeout()} | + {noreply, NewState :: term(), hibernate} | + {stop, Reason :: normal | term(), NewState :: term()}. +handle_info({'DOWN',Ref,process,_,shutdown}, #state{handlers=Hs}=State) -> + case lists:keytake(Ref,2,Hs) of + {value,{Id,Ref},Hs1} -> + %% Probably terminated by supervisor. Remove the handler to avoid + %% error printouts due to failing handler. + _ = case logger:get_handler_config(Id) of + {ok,_} -> + logger:remove_handler(Id); + _ -> + ok + end, + {noreply,State#state{handlers=Hs1}}; + false -> + {noreply, State} + end; +handle_info({'DOWN',Ref,process,_,_OtherReason}, #state{handlers=Hs}=State) -> + {noreply,State#state{handlers=lists:keydelete(Ref,2,Hs)}}; +handle_info(_Other,State) -> + {noreply,State}. + +-spec terminate(Reason :: normal | shutdown | {shutdown, term()} | term(), + State :: term()) -> any(). +terminate(_Reason, _State) -> + ok. diff --git a/lib/kernel/src/logger_internal.hrl b/lib/kernel/src/logger_internal.hrl index fedd6db370..d96a4ac78b 100644 --- a/lib/kernel/src/logger_internal.hrl +++ b/lib/kernel/src/logger_internal.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2017. All Rights Reserved. +%% Copyright Ericsson AB 2017-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -19,12 +19,11 @@ %% -include_lib("kernel/include/logger.hrl"). -define(LOGGER_TABLE,logger). --define(LOGGER_KEY,'$logger_config$'). +-define(PRIMARY_KEY,'$primary_config$'). -define(HANDLER_KEY,'$handler_config$'). -define(LOGGER_META_KEY,'$logger_metadata$'). -define(STANDARD_HANDLER, default). --define(DEFAULT_HANDLER_FILTERS, - ?DEFAULT_HANDLER_FILTERS([beam,erlang,otp])). +-define(DEFAULT_HANDLER_FILTERS,?DEFAULT_HANDLER_FILTERS([otp])). -define(DEFAULT_HANDLER_FILTERS(Domain), [{remote_gl,{fun logger_filters:remote_gl/2,stop}}, {domain,{fun logger_filters:domain/2,{log,super,Domain}}}, @@ -33,7 +32,7 @@ -define(DEFAULT_FORMAT_CONFIG,#{legacy_header=>true, single_line=>false}). -define(DEFAULT_FORMAT_TEMPLATE_HEADER, - [{logger_formatter,header},"\n",msg,"\n"]). + [[logger_formatter,header],"\n",msg,"\n"]). -define(DEFAULT_FORMAT_TEMPLATE_SINGLE, [time," ",level,": ",msg,"\n"]). -define(DEFAULT_FORMAT_TEMPLATE, @@ -55,14 +54,17 @@ %%%----------------------------------------------------------------- %%% Levels %%% Using same as syslog --define(LEVELS,[emergency, +-define(LEVELS,[none, + emergency, alert, critical, error, warning, notice, info, - debug]). + debug, + all]). +-define(LOG_NONE,-1). -define(EMERGENCY,0). -define(ALERT,1). -define(CRITICAL,2). @@ -71,6 +73,7 @@ -define(NOTICE,5). -define(INFO,6). -define(DEBUG,7). +-define(LOG_ALL,10). -define(IS_LEVEL(L), (L=:=emergency orelse diff --git a/lib/kernel/src/logger_server.erl b/lib/kernel/src/logger_server.erl index 47010c9fa5..28e31d46ea 100644 --- a/lib/kernel/src/logger_server.erl +++ b/lib/kernel/src/logger_server.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2017. All Rights Reserved. +%% Copyright Ericsson AB 2017-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -25,8 +25,8 @@ -export([start_link/0, add_handler/3, remove_handler/1, add_filter/2, remove_filter/2, - set_module_level/2, unset_module_level/1, - cache_module_level/1, + set_module_level/2, unset_module_level/0, + unset_module_level/1, cache_module_level/1, set_config/2, set_config/3, update_config/2, update_formatter_config/2]). @@ -53,7 +53,7 @@ add_handler(Id,Module,Config0) -> {ok,ok} -> case sanity_check(Id,Config0) of ok -> - Default = default_config(Id), + Default = default_config(Id,Module), Config = maps:merge(Default,Config0), call({add_handler,Id,Module,Config}); Error -> @@ -75,30 +75,41 @@ add_filter(Owner,Filter) -> remove_filter(Owner,FilterId) -> call({remove_filter,Owner,FilterId}). -set_module_level(Module,Level) when is_atom(Module) -> - case sanity_check(logger,level,Level) of - ok -> call({set_module_level,Module,Level}); - Error -> Error +set_module_level(Modules,Level) when is_list(Modules) -> + case lists:all(fun(M) -> is_atom(M) end,Modules) of + true -> + case sanity_check(primary,level,Level) of + ok -> call({set_module_level,Modules,Level}); + Error -> Error + end; + false -> + {error,{not_a_list_of_modles,Modules}} end; -set_module_level(Module,_) -> - {error,{not_a_module,Module}}. +set_module_level(Modules,_) -> + {error,{not_a_list_of_modules,Modules}}. + +unset_module_level() -> + call({unset_module_level,all}). -unset_module_level(Module) when is_atom(Module) -> - call({unset_module_level,Module}); -unset_module_level(Module) -> - {error,{not_a_module,Module}}. +unset_module_level(Modules) when is_list(Modules) -> + case lists:all(fun(M) -> is_atom(M) end,Modules) of + true -> + call({unset_module_level,Modules}); + false -> + {error,{not_a_list_of_modles,Modules}} + end; +unset_module_level(Modules) -> + {error,{not_a_list_of_modules,Modules}}. cache_module_level(Module) -> gen_server:cast(?SERVER,{cache_module_level,Module}). - set_config(Owner,Key,Value) -> update_config(Owner,#{Key=>Value}). -set_config(Owner,Config0) -> - case sanity_check(Owner,Config0) of +set_config(Owner,Config) -> + case sanity_check(Owner,Config) of ok -> - Config = maps:merge(default_config(Owner),Config0), call({set_config,Owner,Config}); Error -> Error @@ -125,11 +136,12 @@ update_formatter_config(_HandlerId, FormatterConfig) -> init([]) -> process_flag(trap_exit, true), + put(?LOGGER_SERVER_TAG,true), Tid = logger_config:new(?LOGGER_TABLE), - LoggerConfig = maps:merge(default_config(logger), + PrimaryConfig = maps:merge(default_config(primary), #{handlers=>[simple]}), - logger_config:create(Tid,logger,LoggerConfig), - SimpleConfig0 = maps:merge(default_config(simple), + logger_config:create(Tid,primary,PrimaryConfig), + SimpleConfig0 = maps:merge(default_config(simple,logger_simple_h), #{filter_default=>stop, filters=>?DEFAULT_HANDLER_FILTERS}), %% If this fails, then the node should crash @@ -154,9 +166,9 @@ handle_call({add_handler,Id,Module,HConfig}, From, #state{tid=Tid}=State) -> case erlang:function_exported(Module, log, 2) of true -> logger_config:create(Tid,Id,Module,HConfig1), - {ok,Config} = do_get_config(Tid,logger), + {ok,Config} = do_get_config(Tid,primary), Handlers = maps:get(handlers,Config,[]), - do_set_config(Tid,logger, + do_set_config(Tid,primary, Config#{handlers=>[Id|Handlers]}); false -> {error,{invalid_handler, @@ -170,7 +182,7 @@ handle_call({add_handler,Id,Module,HConfig}, From, #state{tid=Tid}=State) -> handle_call({remove_handler,HandlerId}, From, #state{tid=Tid}=State) -> case logger_config:get(Tid,HandlerId) of {ok,{Module,HConfig}} -> - {ok,Config} = do_get_config(Tid,logger), + {ok,Config} = do_get_config(Tid,primary), Handlers0 = maps:get(handlers,Config,[]), Handlers = lists:delete(HandlerId,Handlers0), call_h_async( @@ -179,7 +191,7 @@ handle_call({remove_handler,HandlerId}, From, #state{tid=Tid}=State) -> call_h(Module,removing_handler,[HConfig],ok) end, fun(_Res) -> - do_set_config(Tid,logger,Config#{handlers=>Handlers}), + do_set_config(Tid,primary,Config#{handlers=>Handlers}), logger_config:delete(Tid,HandlerId), ok end,From,State); @@ -203,13 +215,15 @@ handle_call({update_config,Id,NewConfig}, From, #state{tid=Tid}=State) -> Error -> {reply,Error,State} end; -handle_call({set_config,logger,Config}, _From, #state{tid=Tid}=State) -> - {ok,#{handlers:=Handlers}} = logger_config:get(Tid,logger), - Reply = do_set_config(Tid,logger,Config#{handlers=>Handlers}), +handle_call({set_config,primary,Config0}, _From, #state{tid=Tid}=State) -> + Config = maps:merge(default_config(primary),Config0), + {ok,#{handlers:=Handlers}} = logger_config:get(Tid,primary), + Reply = do_set_config(Tid,primary,Config#{handlers=>Handlers}), {reply,Reply,State}; -handle_call({set_config,HandlerId,Config}, From, #state{tid=Tid}=State) -> +handle_call({set_config,HandlerId,Config0}, From, #state{tid=Tid}=State) -> case logger_config:get(Tid,HandlerId) of {ok,{Module,OldConfig}} -> + Config = maps:merge(default_config(HandlerId,Module),Config0), call_h_async( fun() -> call_h(Module,changing_config,[OldConfig,Config], @@ -239,11 +253,11 @@ handle_call({update_formatter_config,HandlerId,NewFConfig},_From, {error,{not_found,HandlerId}} end, {reply,Reply,State}; -handle_call({set_module_level,Module,Level}, _From, #state{tid=Tid}=State) -> - Reply = logger_config:set_module_level(Tid,Module,Level), +handle_call({set_module_level,Modules,Level}, _From, #state{tid=Tid}=State) -> + Reply = logger_config:set_module_level(Tid,Modules,Level), {reply,Reply,State}; -handle_call({unset_module_level,Module}, _From, #state{tid=Tid}=State) -> - Reply = logger_config:unset_module_level(Tid,Module), +handle_call({unset_module_level,Modules}, _From, #state{tid=Tid}=State) -> + Reply = logger_config:unset_module_level(Tid,Modules), {reply,Reply,State}. handle_cast({async_req_reply,_Ref,_Reply} = Reply,State) -> @@ -270,7 +284,7 @@ handle_info({Ref,_Reply},State) when is_reference(Ref) -> handle_info({'DOWN',_Ref,_Proc,_Pid,_Reason} = Down,State) -> call_h_reply(Down,State); handle_info(Unexpected,State) when element(1,Unexpected) == 'EXIT' -> - %% The simple logger will send an 'EXIT' message when it is replaced + %% The simple handler will send an 'EXIT' message when it is replaced %% We may as well ignore all 'EXIT' messages that we get ?LOG_INTERNAL(debug, [{logger,got_unexpected_message}, @@ -343,16 +357,18 @@ do_set_config(Tid,Id,Config) -> logger_config:set(Tid,Id,Config), ok. -default_config(logger) -> - #{level=>info, +default_config(primary) -> + #{level=>notice, filters=>[], filter_default=>log}; default_config(Id) -> #{id=>Id, - level=>info, + level=>all, filters=>[], filter_default=>log, formatter=>{?DEFAULT_FORMATTER,#{}}}. +default_config(Id,Module) -> + (default_config(Id))#{module=>Module}. sanity_check(Owner,Key,Value) -> sanity_check_1(Owner,[{Key,Value}]). @@ -369,8 +385,8 @@ sanity_check_1(Owner,Config) when is_list(Config) -> catch throw:Error -> {error,Error} end. -get_type(logger) -> - logger; +get_type(primary) -> + primary; get_type(Id) -> check_id(Id), handler. @@ -387,8 +403,8 @@ check_config(Owner,[{filter_default,FD}|Config]) -> check_config(handler,[{formatter,Formatter}|Config]) -> check_formatter(Formatter), check_config(handler,Config); -check_config(logger,[C|_]) -> - throw({invalid_logger_config,C}); +check_config(primary,[C|_]) -> + throw({invalid_primary_config,C}); check_config(handler,[{_,_}|Config]) -> %% Arbitrary config elements are allowed for handlers check_config(handler,Config); diff --git a/lib/kernel/src/logger_simple_h.erl b/lib/kernel/src/logger_simple_h.erl index 19fb3b54ba..8b51dd8569 100644 --- a/lib/kernel/src/logger_simple_h.erl +++ b/lib/kernel/src/logger_simple_h.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2017. All Rights Reserved. +%% Copyright Ericsson AB 2017-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/kernel/src/logger_std_h.erl b/lib/kernel/src/logger_std_h.erl index 63c3ab2dac..9a2a1443b3 100644 --- a/lib/kernel/src/logger_std_h.erl +++ b/lib/kernel/src/logger_std_h.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2017. All Rights Reserved. +%% Copyright Ericsson AB 2017-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -35,8 +35,10 @@ terminate/2, code_change/3]). %% logger callbacks --export([log/2, adding_handler/1, removing_handler/1, - changing_config/2, swap_buffer/2]). +-export([log/2, adding_handler/1, removing_handler/1, changing_config/2]). + +%% handler internal +-export([log_handler_info/4]). %%%=================================================================== %%% API @@ -48,7 +50,7 @@ %%% handler process gets added -spec start_link(Name, Config, HandlerState) -> {ok,Pid} | {error,Reason} when Name :: atom(), - Config :: logger:config(), + Config :: logger:handler_config(), HandlerState :: map(), Pid :: pid(), Reason :: term(). @@ -64,7 +66,8 @@ start_link(Name, Config, HandlerState) -> filesync(Name) when is_atom(Name) -> try - gen_server:call(Name, filesync, ?DEFAULT_CALL_TIMEOUT) + gen_server:call(?name_to_reg_name(?MODULE,Name), + filesync, ?DEFAULT_CALL_TIMEOUT) catch _:{timeout,_} -> {error,handler_busy} end; @@ -80,7 +83,8 @@ filesync(Name) -> info(Name) when is_atom(Name) -> try - gen_server:call(Name, info, ?DEFAULT_CALL_TIMEOUT) + gen_server:call(?name_to_reg_name(?MODULE,Name), + info, ?DEFAULT_CALL_TIMEOUT) catch _:{timeout,_} -> {error,handler_busy} end; @@ -95,7 +99,8 @@ info(Name) -> reset(Name) when is_atom(Name) -> try - gen_server:call(Name, reset, ?DEFAULT_CALL_TIMEOUT) + gen_server:call(?name_to_reg_name(?MODULE,Name), + reset, ?DEFAULT_CALL_TIMEOUT) catch _:{timeout,_} -> {error,handler_busy} end; @@ -113,23 +118,16 @@ adding_handler(#{id:=Name}=Config) -> case check_config(adding, Config) of {ok, Config1} -> %% create initial handler state by merging defaults with config - HConfig = maps:get(?MODULE, Config1, #{}), + HConfig = maps:get(config, Config1, #{}), HState = maps:merge(get_init_state(), HConfig), case logger_h_common:overload_levels_ok(HState) of true -> - case start(Name, Config1, HState) of - ok -> - %% Make sure wait_for_buffer is not stored, so we - %% won't hang and wait for buffer on a restart - {ok, maps:remove(wait_for_buffer,Config1)}; - Error -> - Error - end; + start(Name, Config1, HState); false -> - #{toggle_sync_qlen := TSQL, - drop_new_reqs_qlen := DNRQL, - flush_reqs_qlen := FRQL} = HState, - {error,{invalid_levels,{TSQL,DNRQL,FRQL}}} + #{sync_mode_qlen := SMQL, + drop_mode_qlen := DMQL, + flush_qlen := FQL} = HState, + {error,{invalid_levels,{SMQL,DMQL,FQL}}} end; Error -> Error @@ -137,24 +135,27 @@ adding_handler(#{id:=Name}=Config) -> %%%----------------------------------------------------------------- %%% Updating handler config -changing_config(OldConfig=#{id:=Name, ?MODULE:=#{type:=Type}}, +changing_config(OldConfig=#{id:=Name, config:=OldHConfig}, NewConfig=#{id:=Name}) -> - MyConfig = maps:get(?MODULE, NewConfig, #{}), - case maps:get(type, MyConfig, Type) of + #{type:=Type, handler_pid:=HPid, mode_tab:=ModeTab} = OldHConfig, + NewHConfig = maps:get(config, NewConfig, #{}), + case maps:get(type, NewHConfig, Type) of Type -> - MyConfig1 = MyConfig#{type=>Type}, - changing_config1(Name, OldConfig, - NewConfig#{?MODULE=>MyConfig1}); + NewHConfig1 = NewHConfig#{type=>Type, + handler_pid=>HPid, + mode_tab=>ModeTab}, + changing_config1(HPid, OldConfig, + NewConfig#{config=>NewHConfig1}); _ -> {error,{illegal_config_change,OldConfig,NewConfig}} end; changing_config(OldConfig, NewConfig) -> {error,{illegal_config_change,OldConfig,NewConfig}}. -changing_config1(Name, OldConfig, NewConfig) -> +changing_config1(HPid, OldConfig, NewConfig) -> case check_config(changing, NewConfig) of Result = {ok,NewConfig1} -> - try gen_server:call(Name, {change_config,OldConfig,NewConfig1}, + try gen_server:call(HPid, {change_config,OldConfig,NewConfig1}, ?DEFAULT_CALL_TIMEOUT) of ok -> Result; HError -> HError @@ -167,38 +168,38 @@ changing_config1(Name, OldConfig, NewConfig) -> check_config(adding, Config) -> %% Merge in defaults on handler level - MyConfig0 = maps:get(?MODULE, Config, #{}), - MyConfig = maps:merge(#{type => standard_io}, - MyConfig0), - case check_my_config(maps:to_list(MyConfig)) of + HConfig0 = maps:get(config, Config, #{}), + HConfig = maps:merge(#{type => standard_io}, + HConfig0), + case check_h_config(maps:to_list(HConfig)) of ok -> - {ok,Config#{?MODULE=>MyConfig}}; + {ok,Config#{config=>HConfig}}; Error -> Error end; check_config(changing, Config) -> - MyConfig = maps:get(?MODULE, Config, #{}), - case check_my_config(maps:to_list(MyConfig)) of + HConfig = maps:get(config, Config, #{}), + case check_h_config(maps:to_list(HConfig)) of ok -> {ok,Config}; Error -> Error end. -check_my_config([{type,Type} | Config]) when Type == standard_io; - Type == standard_error -> - check_my_config(Config); -check_my_config([{type,{file,File}} | Config]) when is_list(File) -> - check_my_config(Config); -check_my_config([{type,{file,File,Modes}} | Config]) when is_list(File), - is_list(Modes) -> - check_my_config(Config); -check_my_config([Other | Config]) -> +check_h_config([{type,Type} | Config]) when Type == standard_io; + Type == standard_error -> + check_h_config(Config); +check_h_config([{type,{file,File}} | Config]) when is_list(File) -> + check_h_config(Config); +check_h_config([{type,{file,File,Modes}} | Config]) when is_list(File), + is_list(Modes) -> + check_h_config(Config); +check_h_config([Other | Config]) -> case logger_h_common:check_common_config(Other) of valid -> - check_my_config(Config); + check_h_config(Config); invalid -> {error,{invalid_config,?MODULE,Other}} end; -check_my_config([]) -> +check_h_config([]) -> ok. @@ -208,35 +209,28 @@ removing_handler(#{id:=Name}) -> stop(Name). %%%----------------------------------------------------------------- -%%% Get buffer when swapping from simple handler -swap_buffer(Name,Buffer) -> - case whereis(Name) of - undefined -> - ok; - _ -> - Name ! {buffer,Buffer} - end. - -%%%----------------------------------------------------------------- %%% Log a string or report -spec log(LogEvent, Config) -> ok | dropped when LogEvent :: logger:log_event(), - Config :: logger:config(). + Config :: logger:handler_config(). -log(LogEvent,Config=#{id:=Name}) -> - %% if the handler has crashed, we must drop this request +log(LogEvent, Config = #{id := Name, + config := #{handler_pid := HPid, + mode_tab := ModeTab}}) -> + %% if the handler has crashed, we must drop this event %% and hope the handler restarts so we can try again - true = is_pid(whereis(Name)), - Bin = logger_h_common:log_to_binary(LogEvent,Config), - logger_h_common:call_cast_or_drop(Name, Bin). + true = is_process_alive(HPid), + Bin = logger_h_common:log_to_binary(LogEvent, Config), + logger_h_common:call_cast_or_drop(Name, HPid, ModeTab, Bin). %%%=================================================================== %%% gen_server callbacks %%%=================================================================== -init([Name, Config, - State0 = #{type := Type, file_ctrl_sync_int := FileCtrlSyncInt}]) -> - register(Name, self()), +init([Name, Config = #{config := HConfig}, + State0 = #{type := Type, file_ctrl_sync_int := FileCtrlSyncInt}]) -> + RegName = ?name_to_reg_name(?MODULE,Name), + register(RegName, self()), process_flag(trap_exit, true), process_flag(message_queue_data, off_heap), @@ -245,59 +239,61 @@ init([Name, Config, case do_init(Name, Type) of {ok,InitState} -> - catch ets:new(Name, [public, named_table]), - ?set_mode(Name, async), - State = maps:merge(State0, InitState), - T0 = ?timestamp(), - State1 = - ?merge_with_stats(State#{mode => async, - file_ctrl_sync => FileCtrlSyncInt, - last_qlen => 0, - last_log_ts => T0, - last_op => sync, - burst_win_ts => T0, - burst_msg_count => 0}), - proc_lib:init_ack({ok,self()}), - gen_server:cast(self(), repeated_filesync), - enter_loop(Config, State1); + try ets:new(Name, [public]) of + ModeTab -> + ?set_mode(ModeTab, async), + State = maps:merge(State0, InitState), + T0 = ?timestamp(), + State1 = + ?merge_with_stats(State#{ + mode_tab => ModeTab, + mode => async, + file_ctrl_sync => FileCtrlSyncInt, + last_qlen => 0, + last_log_ts => T0, + last_op => sync, + burst_win_ts => T0, + burst_msg_count => 0}), + Config1 = + Config#{config => HConfig#{handler_pid => self(), + mode_tab => ModeTab}}, + proc_lib:init_ack({ok,self(),Config1}), + gen_server:cast(self(), repeated_filesync), + enter_loop(Config1, State1) + catch + _:Error -> + unregister(RegName), + logger_h_common:error_notify({init_handler,Name,Error}), + proc_lib:init_ack(Error) + end; Error -> + unregister(RegName), logger_h_common:error_notify({init_handler,Name,Error}), proc_lib:init_ack(Error) end. -do_init(Name, Std) when Std=:=standard_io; Std=:=standard_error -> - case open_log_file(Name, Std) of - {ok,FileCtrlPid} -> - {ok,#{id=>Name,type=>Std,file_ctrl_pid=>FileCtrlPid}}; - Error -> - Error - end; -do_init(Name, FileInfo) when is_tuple(FileInfo) -> - case open_log_file(Name, FileInfo) of +do_init(Name, Type) -> + case open_log_file(Name, Type) of {ok,FileCtrlPid} -> - {ok,#{id=>Name,type=>FileInfo,file_ctrl_pid=>FileCtrlPid}}; + case logger_h_common:unset_restart_flag(Name, ?MODULE) of + true -> + %% inform about restart + gen_server:cast(self(), {log_handler_info, + "Handler ~p restarted", + [Name]}); + false -> + %% initial start + ok + end, + {ok,#{id=>Name,type=>Type,file_ctrl_pid=>FileCtrlPid}}; Error -> Error end. -enter_loop(#{wait_for_buffer:=true}=Config,State) -> - State1 = - receive - {buffer,Buffer} -> - lists:foldl( - fun(Log,S) -> - Bin = logger_h_common:log_to_binary(Log,Config), - {_,S1} = do_log(Bin,cast,S), - S1 - end, - State, - Buffer) - end, - gen_server:enter_loop(?MODULE,[],State1); enter_loop(_Config,State) -> gen_server:enter_loop(?MODULE,[],State). -%% This is the synchronous log request. +%% This is the synchronous log event. handle_call({log, Bin}, _From, State) -> {Result,State1} = do_log(Bin, call, State), %% Result == ok | dropped @@ -313,7 +309,7 @@ handle_call(filesync, _From, State = #{type := Type, handle_call({change_config,_OldConfig,NewConfig}, _From, State = #{filesync_repeat_interval := FSyncInt0}) -> - HConfig = maps:get(?MODULE, NewConfig, #{}), + HConfig = maps:get(config, NewConfig, #{}), State1 = maps:merge(State, HConfig), case logger_h_common:overload_levels_ok(State1) of true -> @@ -335,10 +331,10 @@ handle_call({change_config,_OldConfig,NewConfig}, _From, end, {reply, ok, State1}; false -> - #{toggle_sync_qlen := TSQL, - drop_new_reqs_qlen := DNRQL, - flush_reqs_qlen := FRQL} = State1, - {reply, {error,{invalid_levels,{TSQL,DNRQL,FRQL}}}, State} + #{sync_mode_qlen := SMQL, + drop_mode_qlen := DMQL, + flush_qlen := FQL} = State1, + {reply, {error,{invalid_levels,{SMQL,DMQL,FQL}}}, State} end; handle_call(info, _From, State) -> @@ -352,15 +348,19 @@ handle_call(reset, _From, State) -> handle_call(stop, _From, State) -> {stop, {shutdown,stopped}, ok, State}. -%% This is the asynchronous log request. +%% This is the asynchronous log event. handle_cast({log, Bin}, State) -> {_,State1} = do_log(Bin, cast, State), {noreply, State1}; +handle_cast({log_handler_info, Format, Args}, State = #{id:=Name}) -> + log_handler_info(Name, Format, Args, State), + {noreply, State}; + %% If FILESYNC_REPEAT_INTERVAL is set to a millisec value, this %% clause gets called repeatedly by the handler. In order to %% guarantee that a filesync *always* happens after the last log -%% request, the repeat operation must be active! +%% event, the repeat operation must be active! handle_cast(repeated_filesync, State = #{type := Type, file_ctrl_pid := FileCtrlPid, @@ -418,6 +418,7 @@ terminate(Reason, State = #{id:=Name, file_ctrl_pid:=FWPid, false -> ok end, + unregister(?name_to_reg_name(?MODULE, Name)), logger_h_common:stop_or_restart(Name, Reason, State). code_change(_OldVsn, State, _Extra) -> @@ -430,19 +431,19 @@ code_change(_OldVsn, State, _Extra) -> %%%----------------------------------------------------------------- %%% get_init_state() -> - #{toggle_sync_qlen => ?TOGGLE_SYNC_QLEN, - drop_new_reqs_qlen => ?DROP_NEW_REQS_QLEN, - flush_reqs_qlen => ?FLUSH_REQS_QLEN, - enable_burst_limit => ?ENABLE_BURST_LIMIT, - burst_limit_size => ?BURST_LIMIT_SIZE, - burst_window_time => ?BURST_WINDOW_TIME, - enable_kill_overloaded => ?ENABLE_KILL_OVERLOADED, - handler_overloaded_qlen => ?HANDLER_OVERLOADED_QLEN, - handler_overloaded_mem => ?HANDLER_OVERLOADED_MEM, - handler_restart_after => ?HANDLER_RESTART_AFTER, - file_ctrl_sync_int => ?CONTROLLER_SYNC_INTERVAL, - filesync_ok_qlen => ?FILESYNC_OK_QLEN, - filesync_repeat_interval => ?FILESYNC_REPEAT_INTERVAL}. + #{sync_mode_qlen => ?SYNC_MODE_QLEN, + drop_mode_qlen => ?DROP_MODE_QLEN, + flush_qlen => ?FLUSH_QLEN, + burst_limit_enable => ?BURST_LIMIT_ENABLE, + burst_limit_max_count => ?BURST_LIMIT_MAX_COUNT, + burst_limit_window_time => ?BURST_LIMIT_WINDOW_TIME, + overload_kill_enable => ?OVERLOAD_KILL_ENABLE, + overload_kill_qlen => ?OVERLOAD_KILL_QLEN, + overload_kill_mem_size => ?OVERLOAD_KILL_MEM_SIZE, + overload_kill_restart_after => ?OVERLOAD_KILL_RESTART_AFTER, + file_ctrl_sync_int => ?CONTROLLER_SYNC_INTERVAL, + filesync_ok_qlen => ?FILESYNC_OK_QLEN, + filesync_repeat_interval => ?FILESYNC_REPEAT_INTERVAL}. %%%----------------------------------------------------------------- %%% Add a standard handler to the logger. @@ -451,9 +452,9 @@ get_init_state() -> %%% exist if the handler is not registered). %%% %%% Handler specific config should be provided with a sub map associated -%%% with a key named the same as this module, e.g: +%%% with a key named 'config', e.g: %%% -%%% Config = #{logger_std_h => #{toggle_sync_qlen => 50} +%%% Config = #{config => #{sync_mode_qlen => 50} %%% %%% The standard handler process is linked to logger_sup, which is %%% part of the kernel application's supervision tree. @@ -466,8 +467,9 @@ start(Name, Config, HandlerState) -> type => worker, modules => [?MODULE]}, case supervisor:start_child(logger_sup, LoggerStdH) of - {ok,_Pid} -> - ok; + {ok,Pid,Config1} -> + ok = logger_handler_watcher:register_handler(Name,Pid), + {ok,Config1}; Error -> Error end. @@ -475,17 +477,20 @@ start(Name, Config, HandlerState) -> %%%----------------------------------------------------------------- %%% Stop and remove the handler. stop(Name) -> - case whereis(Name) of + case whereis(?name_to_reg_name(?MODULE,Name)) of undefined -> ok; - _ -> + Pid -> %% We don't want to do supervisor:terminate_child here %% since we need to distinguish this explicit stop from a %% system termination in order to avoid circular attempts %% at removing the handler (implying deadlocks and %% timeouts). - _ = gen_server:call(Name,stop), - _ = supervisor:delete_child(logger_sup, Name), + %% And we don't need to do supervisor:delete_child, since + %% the restart type is temporary, which means that the + %% child specification is automatically removed from the + %% supervisor when the process dies. + _ = gen_server:call(Pid, stop), ok end. @@ -495,21 +500,31 @@ stop(Name) -> if C == 0 -> Interval; true -> C-1 end). -%% check for overload between every request (and set Mode to async, +%% check for overload between every event (and set Mode to async, %% sync or drop accordingly), but never flush the whole mailbox -%% before LogWindowSize requests have been handled -do_log(Bin, CallOrCast, State = #{id:=Name}) -> +%% before LogWindowSize events have been handled +do_log(Bin, CallOrCast, State = #{id:=Name, mode:=Mode0}) -> T1 = ?timestamp(), %% check if the handler is getting overloaded, or if it's %% recovering from overload (the check must be done for each - %% request to react quickly to large bursts of requests and + %% event to react quickly to large bursts of events and %% to ensure that the handler can never end up in drop mode %% with an empty mailbox, which would stop operation) {Mode1,QLen,Mem,State1} = logger_h_common:check_load(State), + if (Mode1 == drop) andalso (Mode0 =/= drop) -> + log_handler_info(Name, "Handler ~p switched to drop mode", + [Name], State); + (Mode0 == drop) andalso ((Mode1 == async) orelse (Mode1 == sync)) -> + log_handler_info(Name, "Handler ~p switched to ~w mode", + [Name,Mode1], State); + true -> + ok + end, + %% kill the handler if it can't keep up with the load - logger_h_common:kill_if_choked(Name, QLen, Mem, State), + logger_h_common:kill_if_choked(Name, QLen, Mem, ?MODULE, State), if Mode1 == flush -> flush(Name, QLen, T1, State1); @@ -519,10 +534,14 @@ do_log(Bin, CallOrCast, State = #{id:=Name}) -> %% this clause is called by do_log/3 after an overload check %% has been performed, where QLen > FlushQLen -flush(Name, _QLen0, T1, State=#{last_log_ts := _T0}) -> +flush(Name, _QLen0, T1, State=#{last_log_ts := _T0, mode_tab := ModeTab}) -> %% flush messages in the mailbox (a limited number in %% order to not cause long delays) - _NewFlushed = logger_h_common:flush_log_requests(?FLUSH_MAX_N), + NewFlushed = logger_h_common:flush_log_events(?FLUSH_MAX_N), + + %% write info in log about flushed messages + log_handler_info(Name, "Handler ~p flushed ~w log events", + [Name,NewFlushed], State), %% because of the receive loop when flushing messages, the %% handler will be scheduled out often and the mailbox could @@ -530,50 +549,51 @@ flush(Name, _QLen0, T1, State=#{last_log_ts := _T0}) -> {_,_QLen1} = process_info(self(), message_queue_len), ?observe(Name,{max_qlen,_QLen1}), - %% Add 1 for the current log request - ?observe(Name,{flushed,_NewFlushed+1}), + %% Add 1 for the current log event + ?observe(Name,{flushed,NewFlushed+1}), State1 = ?update_max_time(?diff_time(T1,_T0),State), - {dropped,?update_other(flushed,FLUSHED,_NewFlushed, - State1#{mode => ?set_mode(Name,async), + {dropped,?update_other(flushed,FLUSHED,NewFlushed, + State1#{mode => ?set_mode(ModeTab,async), last_qlen => 0, last_log_ts => T1})}. %% this clause is called to write to file -write(Name, Mode, T1, Bin, _CallOrCast, - State = #{file_ctrl_pid := FileCtrlPid, +write(_Name, Mode, T1, Bin, _CallOrCast, + State = #{mode_tab := ModeTab, + file_ctrl_pid := FileCtrlPid, file_ctrl_sync := FileCtrlSync, last_qlen := LastQLen, last_log_ts := T0, file_ctrl_sync_int := FileCtrlSyncInt}) -> %% check if we need to limit the number of writes - %% during a burst of log requests + %% during a burst of log events {DoWrite,BurstWinT,BurstMsgCount} = logger_h_common:limit_burst(State), - %% only send a synhrounous request to the file controller process + %% only send a synhrounous event to the file controller process %% every FileCtrlSyncInt time, to give the handler time between %% file writes so it can keep up with incoming messages {Result,LastQLen1} = if DoWrite, FileCtrlSync == 0 -> - ?observe(Name,{_CallOrCast,1}), + ?observe(_Name,{_CallOrCast,1}), file_write_sync(FileCtrlPid, Bin, false), {ok,element(2, process_info(self(), message_queue_len))}; DoWrite -> - ?observe(Name,{_CallOrCast,1}), + ?observe(_Name,{_CallOrCast,1}), file_write_async(FileCtrlPid, Bin), {ok,LastQLen}; not DoWrite -> - ?observe(Name,{flushed,1}), + ?observe(_Name,{flushed,1}), {dropped,LastQLen} end, - %% Check if the time since the previous log request is long enough - + %% Check if the time since the previous log event is long enough - %% and the queue length small enough - to assume the mailbox has %% been emptied, and if so, do filesync operation and reset mode to %% async. Note that this is the best we can do to detect an idle %% handler without setting a timer after each log call/cast. If the - %% time between two consecutive log requests is fast and no new - %% request comes in after the last one, idle state won't be detected! + %% time between two consecutive log events is fast and no new + %% event comes in after the last one, idle state won't be detected! Time = ?diff_time(T1,T0), {Mode1,BurstMsgCount1} = if (LastQLen1 < ?FILESYNC_OK_QLEN) andalso @@ -585,7 +605,7 @@ write(Name, Mode, T1, Bin, _CallOrCast, _File -> file_ctrl_filesync_async(FileCtrlPid) end, - {?change_mode(Name, Mode, async),0}; + {?change_mode(ModeTab, Mode, async),0}; true -> {Mode,BurstMsgCount} end, @@ -634,6 +654,20 @@ close_log_file(Fd) -> _ = file:datasync(Fd), _ = file:close(Fd). + +log_handler_info(Name, Format, Args, #{file_ctrl_pid := FileCtrlPid}) -> + Config = + case logger:get_handler_config(Name) of + {ok,Conf} -> Conf; + _ -> #{formatter=>{?DEFAULT_FORMATTER,?DEFAULT_FORMAT_CONFIG}} + end, + Meta = #{time=>erlang:system_time(microsecond)}, + Bin = logger_h_common:log_to_binary(#{level => notice, + msg => {Format,Args}, + meta => Meta}, Config), + _ = file_write_async(FileCtrlPid, Bin), + ok. + %%%----------------------------------------------------------------- %%% File control process @@ -706,7 +740,7 @@ file_ctrl_init(HandlerName, StdDev, Starter) -> file_ctrl_loop(Fd, Type, DevName, Synced, PrevWriteResult, PrevSyncResult, HandlerName) -> receive - %% asynchronous request + %% asynchronous event {log,Bin} -> Result = if Type == file -> write_to_dev(Fd, Bin, DevName, @@ -717,7 +751,7 @@ file_ctrl_loop(Fd, Type, DevName, Synced, file_ctrl_loop(Fd, Type, DevName, false, Result, PrevSyncResult, HandlerName); - %% synchronous request + %% synchronous event {{log,From,Bin,FileSync},MRef} -> if Type == file -> %% check that file hasn't been deleted diff --git a/lib/kernel/src/logger_sup.erl b/lib/kernel/src/logger_sup.erl index 4e4de94d5c..3d6f482e20 100644 --- a/lib/kernel/src/logger_sup.erl +++ b/lib/kernel/src/logger_sup.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2017. All Rights Reserved. +%% Copyright Ericsson AB 2017-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -46,7 +46,11 @@ init([]) -> intensity => 1, period => 5}, - {ok, {SupFlags, []}}. + Watcher = #{id => logger_handler_watcher, + start => {logger_handler_watcher, start_link, []}, + shutdown => brutal_kill}, + + {ok, {SupFlags, [Watcher]}}. %%%=================================================================== %%% Internal functions diff --git a/lib/kernel/src/net_kernel.erl b/lib/kernel/src/net_kernel.erl index c4e1a0ce1e..bea08242d8 100644 --- a/lib/kernel/src/net_kernel.erl +++ b/lib/kernel/src/net_kernel.erl @@ -472,7 +472,7 @@ handle_call({passive_cnct, Node}, From, State) -> %% Explicit connect %% The response is delayed until the connection is up and running. %% -handle_call({connect, _, Node, _, _}, From, State) when Node =:= node() -> +handle_call({connect, _, Node}, From, State) when Node =:= node() -> async_reply({reply, true, State}, From); handle_call({connect, Type, Node}, From, State) -> verbose({connect, Type, Node}, 1, State), diff --git a/lib/kernel/src/os.erl b/lib/kernel/src/os.erl index 77c883f57f..29a26674ba 100644 --- a/lib/kernel/src/os.erl +++ b/lib/kernel/src/os.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2017. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/kernel/src/seq_trace.erl b/lib/kernel/src/seq_trace.erl index 8d7aba0f27..14fe21e9de 100644 --- a/lib/kernel/src/seq_trace.erl +++ b/lib/kernel/src/seq_trace.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1998-2016. All Rights Reserved. +%% Copyright Ericsson AB 1998-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/kernel/src/user_drv.erl b/lib/kernel/src/user_drv.erl index 99ea4210bd..9f914aa222 100644 --- a/lib/kernel/src/user_drv.erl +++ b/lib/kernel/src/user_drv.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/kernel/test/Makefile b/lib/kernel/test/Makefile index 2ad1e3107c..4a86265a4a 100644 --- a/lib/kernel/test/Makefile +++ b/lib/kernel/test/Makefile @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 1997-2016. All Rights Reserved. +# Copyright Ericsson AB 1997-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -71,7 +71,6 @@ MODULES= \ init_SUITE \ kernel_config_SUITE \ logger_SUITE \ - logger_bench_SUITE \ logger_disk_log_h_SUITE \ logger_env_var_SUITE \ logger_filters_SUITE \ diff --git a/lib/kernel/test/application_SUITE.erl b/lib/kernel/test/application_SUITE.erl index 988f26280f..5c35b82207 100644 --- a/lib/kernel/test/application_SUITE.erl +++ b/lib/kernel/test/application_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2017. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -37,7 +37,8 @@ -export([config_change/1, persistent_env/1, distr_changed_tc1/1, distr_changed_tc2/1, ensure_started/1, ensure_all_started/1, - shutdown_func/1, do_shutdown/1, shutdown_timeout/1, shutdown_deadlock/1]). + shutdown_func/1, do_shutdown/1, shutdown_timeout/1, shutdown_deadlock/1, + config_relative_paths/1]). -define(TESTCASE, testcase_name). -define(testcase, proplists:get_value(?TESTCASE, Config)). @@ -55,7 +56,7 @@ all() -> script_start, nodedown_start, permit_false_start_local, permit_false_start_dist, get_key, get_env, ensure_all_started, {group, distr_changed}, config_change, shutdown_func, shutdown_timeout, - shutdown_deadlock, + shutdown_deadlock, config_relative_paths, persistent_env]. groups() -> @@ -2075,6 +2076,42 @@ shutdown_deadlock(Config) when is_list(Config) -> %%----------------------------------------------------------------- +%% Relative paths in sys.config +%%----------------------------------------------------------------- +config_relative_paths(Config) -> + Dir = ?config(priv_dir,Config), + SubDir = filename:join(Dir,"subdir"), + Sys = filename:join(SubDir,"sys.config"), + ok = filelib:ensure_dir(Sys), + ok = file:write_file(Sys,"[\"../up.config\",\"current\"].\n"), + + Up = filename:join(Dir,"up.config"), + ok = file:write_file(Up,"[{app1,[{key1,value}]}].\n"), + + {ok,Cwd} = file:get_cwd(), + Current1 = filename:join(Cwd,"current.config"), + ok = file:write_file(Current1,"[{app1,[{key2,value1}]}].\n"), + + N1 = list_to_atom(lists:concat([?FUNCTION_NAME,"_1"])), + {ok,Node1} = start_node(N1,filename:rootname(Sys)), + ok = rpc:call(Node1, application, load, [app1()]), + {ok, value} = rpc:call(Node1, application, get_env,[app1,key1]), + {ok, value1} = rpc:call(Node1, application, get_env,[app1,key2]), + + Current2 = filename:join(SubDir,"current.config"), + ok = file:write_file(Current2,"[{app1,[{key2,value2}]}].\n"), + + N2 = list_to_atom(lists:concat([?FUNCTION_NAME,"_2"])), + {ok, Node2} = start_node(N2,filename:rootname(Sys)), + ok = rpc:call(Node2, application, load, [app1()]), + {ok, value} = rpc:call(Node2, application, get_env,[app1,key1]), + {ok, value2} = rpc:call(Node2, application, get_env,[app1,key2]), + + stop_node_nice([Node1,Node2]), + + ok. + +%%----------------------------------------------------------------- %% Utility functions %%----------------------------------------------------------------- app0() -> diff --git a/lib/kernel/test/code_SUITE.erl b/lib/kernel/test/code_SUITE.erl index 902196def2..1314316c13 100644 --- a/lib/kernel/test/code_SUITE.erl +++ b/lib/kernel/test/code_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2017. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/kernel/test/erl_distribution_SUITE.erl b/lib/kernel/test/erl_distribution_SUITE.erl index 9c6712ad74..0e13f0383e 100644 --- a/lib/kernel/test/erl_distribution_SUITE.erl +++ b/lib/kernel/test/erl_distribution_SUITE.erl @@ -25,6 +25,7 @@ init_per_group/2,end_per_group/2]). -export([tick/1, tick_change/1, + connect_node/1, nodenames/1, hostnames/1, illegal_nodenames/1, hidden_node/1, setopts/1, @@ -70,6 +71,7 @@ suite() -> all() -> [tick, tick_change, nodenames, hostnames, illegal_nodenames, + connect_node, hidden_node, setopts, table_waste, net_setuptime, inet_dist_options_options, {group, monitor_nodes}]. @@ -106,6 +108,12 @@ init_per_testcase(Func, Config) when is_atom(Func), is_list(Config) -> end_per_testcase(_Func, _Config) -> ok. +connect_node(Config) when is_list(Config) -> + Connected = nodes(connected), + true = net_kernel:connect_node(node()), + Connected = nodes(connected), + ok. + tick(Config) when is_list(Config) -> PaDir = filename:dirname(code:which(erl_distribution_SUITE)), diff --git a/lib/kernel/test/erl_prim_loader_SUITE.erl b/lib/kernel/test/erl_prim_loader_SUITE.erl index 3502a4ad08..16a127aa3e 100644 --- a/lib/kernel/test/erl_prim_loader_SUITE.erl +++ b/lib/kernel/test/erl_prim_loader_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/kernel/test/error_logger_SUITE.erl b/lib/kernel/test/error_logger_SUITE.erl index 6c4526d0cf..eab72e58a7 100644 --- a/lib/kernel/test/error_logger_SUITE.erl +++ b/lib/kernel/test/error_logger_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2017. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -33,7 +33,7 @@ off_heap/1, error_report/1, info_report/1, error/1, info/1, emulator/1, via_logger_process/1, other_node/1, - tty/1, logfile/1, add/1, delete/1]). + tty/1, logfile/1, add/1, delete/1, format_depth/1]). -export([generate_error/2]). @@ -48,7 +48,8 @@ suite() -> all() -> [off_heap, error_report, info_report, error, info, emulator, - via_logger_process, other_node, tty, logfile, add, delete]. + via_logger_process, other_node, tty, logfile, add, delete, + format_depth]. groups() -> []. @@ -309,6 +310,21 @@ delete(Config) when is_list(Config) -> ok. %%----------------------------------------------------------------- + +format_depth(_Config) -> + ok = application:set_env(kernel,error_logger_format_depth,30), + 30 = error_logger:get_format_depth(), + ok = application:set_env(kernel,error_logger_format_depth,3), + 10 = error_logger:get_format_depth(), + ok = application:set_env(kernel,error_logger_format_depth,11), + 11 = error_logger:get_format_depth(), + ok = application:set_env(kernel,error_logger_format_depth,unlimited), + unlimited = error_logger:get_format_depth(), + ok = application:unset_env(kernel,error_logger_format_depth), + unlimited = error_logger:get_format_depth(), + ok. + +%%----------------------------------------------------------------- %% Check that the report has been received. %%----------------------------------------------------------------- reported(Tag, Type, Report) -> diff --git a/lib/kernel/test/error_logger_warn_SUITE.erl b/lib/kernel/test/error_logger_warn_SUITE.erl index ef55a2d339..8f1eb2ba0a 100644 --- a/lib/kernel/test/error_logger_warn_SUITE.erl +++ b/lib/kernel/test/error_logger_warn_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2003-2016. All Rights Reserved. +%% Copyright Ericsson AB 2003-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/kernel/test/file_SUITE.erl b/lib/kernel/test/file_SUITE.erl index ff93f25e25..e784c06865 100644 --- a/lib/kernel/test/file_SUITE.erl +++ b/lib/kernel/test/file_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2017. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -1367,6 +1367,10 @@ file_info_basic_file(Config) when is_list(Config) -> io:put_chars(Fd1, "foo bar"), ok = ?FILE_MODULE:close(Fd1), + %% Don't crash the file server when passing incorrect arguments. + {error,badarg} = ?FILE_MODULE:read_file_info(Name, [{time, gurka}]), + {error,badarg} = ?FILE_MODULE:read_file_info([#{} | gaffel]), + %% Test that the file has the expected attributes. %% The times are tricky, so we will save them to a separate test case. {ok,FileInfo} = ?FILE_MODULE:read_file_info(Name), diff --git a/lib/kernel/test/gen_sctp_SUITE.erl b/lib/kernel/test/gen_sctp_SUITE.erl index 9dde00652c..a0ae792ba9 100644 --- a/lib/kernel/test/gen_sctp_SUITE.erl +++ b/lib/kernel/test/gen_sctp_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2016. All Rights Reserved. +%% Copyright Ericsson AB 2007-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/kernel/test/gen_tcp_api_SUITE.erl b/lib/kernel/test/gen_tcp_api_SUITE.erl index 0fe44e8a88..1be016444f 100644 --- a/lib/kernel/test/gen_tcp_api_SUITE.erl +++ b/lib/kernel/test/gen_tcp_api_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1998-2017. All Rights Reserved. +%% Copyright Ericsson AB 1998-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/kernel/test/gen_tcp_misc_SUITE.erl b/lib/kernel/test/gen_tcp_misc_SUITE.erl index e47023d201..950f5bea6f 100644 --- a/lib/kernel/test/gen_tcp_misc_SUITE.erl +++ b/lib/kernel/test/gen_tcp_misc_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1998-2016. All Rights Reserved. +%% Copyright Ericsson AB 1998-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/kernel/test/gen_udp_SUITE.erl b/lib/kernel/test/gen_udp_SUITE.erl index 6a50239c2a..3acfff929e 100644 --- a/lib/kernel/test/gen_udp_SUITE.erl +++ b/lib/kernel/test/gen_udp_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1998-2017. All Rights Reserved. +%% Copyright Ericsson AB 1998-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -34,7 +34,7 @@ -export([init_per_testcase/2, end_per_testcase/2]). -export([send_to_closed/1, active_n/1, - buffer_size/1, binary_passive_recv/1, bad_address/1, + buffer_size/1, binary_passive_recv/1, max_buffer_size/1, bad_address/1, read_packets/1, open_fd/1, connect/1, implicit_inet6/1, local_basic/1, local_unbound/1, local_fdopen/1, local_fdopen_unbound/1, local_abstract/1]). @@ -44,7 +44,7 @@ suite() -> {timetrap,{minutes,1}}]. all() -> - [send_to_closed, buffer_size, binary_passive_recv, + [send_to_closed, buffer_size, binary_passive_recv, max_buffer_size, bad_address, read_packets, open_fd, connect, implicit_inet6, active_n, {group, local}]. @@ -237,6 +237,14 @@ buffer_size_server_recv(Socket, IP, Port, Cnt) -> end. +%%------------------------------------------------------------- +%% OTP-15206: Keep buffer small for udp +%%------------------------------------------------------------- +max_buffer_size(Config) when is_list(Config) -> + {ok, Socket} = gen_udp:open(0, [binary]), + ok = inet:setopts(Socket,[{recbuf, 1 bsl 20}]), + {ok, [{buffer, 65536}]} = inet:getopts(Socket,[buffer]), + gen_udp:close(Socket). %%------------------------------------------------------------- %% OTP-3823 gen_udp:recv does not return address in binary mode diff --git a/lib/kernel/test/heart_SUITE.erl b/lib/kernel/test/heart_SUITE.erl index e95635b800..f5ca6d0e1d 100644 --- a/lib/kernel/test/heart_SUITE.erl +++ b/lib/kernel/test/heart_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/kernel/test/inet_SUITE.erl b/lib/kernel/test/inet_SUITE.erl index 2e5f8c7d2c..713de8c9a8 100644 --- a/lib/kernel/test/inet_SUITE.erl +++ b/lib/kernel/test/inet_SUITE.erl @@ -97,7 +97,7 @@ end_per_group(_GroupName, Config) -> init_per_testcase(lookup_bad_search_option, Config) -> Db = inet_db, Key = res_lookup, - %% The bad option can not enter through inet_db:set_lookup/1, + %% The bad option cannot enter through inet_db:set_lookup/1, %% but through e.g .inetrc. Prev = ets:lookup(Db, Key), ets:delete(Db, Key), diff --git a/lib/kernel/test/inet_res_SUITE.erl b/lib/kernel/test/inet_res_SUITE.erl index 2a5b8d0044..cbec8d430c 100644 --- a/lib/kernel/test/inet_res_SUITE.erl +++ b/lib/kernel/test/inet_res_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009-2016. All Rights Reserved. +%% Copyright Ericsson AB 2009-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -531,7 +531,7 @@ edns0(Config) when is_list(Config) -> case os:version() of {M,V,_} when M < 5; M == 5, V =< 8 -> %% In our test park only known platform - %% with an DNS resolver that can not do + %% with an DNS resolver that cannot do %% EDNS0. {comment,"No EDNS0"} end diff --git a/lib/kernel/test/kernel.spec b/lib/kernel/test/kernel.spec index 86d2155828..62afc9f97b 100644 --- a/lib/kernel/test/kernel.spec +++ b/lib/kernel/test/kernel.spec @@ -2,4 +2,3 @@ {config, "../test_server/ts.unix.config"}. {suites,"../kernel_test", all}. -{skip_suites,"../kernel_test",[logger_bench_SUITE],"Not ready"}. diff --git a/lib/kernel/test/kernel_SUITE.erl b/lib/kernel/test/kernel_SUITE.erl index 7898988dbe..3e5ed855b5 100644 --- a/lib/kernel/test/kernel_SUITE.erl +++ b/lib/kernel/test/kernel_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2016. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/kernel/test/kernel_config_SUITE.erl b/lib/kernel/test/kernel_config_SUITE.erl index a21020ff97..9207025a2c 100644 --- a/lib/kernel/test/kernel_config_SUITE.erl +++ b/lib/kernel/test/kernel_config_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/kernel/test/logger_SUITE.erl b/lib/kernel/test/logger_SUITE.erl index f7ec59a7b7..8f74ebdc47 100644 --- a/lib/kernel/test/logger_SUITE.erl +++ b/lib/kernel/test/logger_SUITE.erl @@ -50,7 +50,7 @@ init_per_suite(Config) -> end_per_suite(Config) -> case ?config(default_handler,Config) of - {HMod,HConfig} -> + #{module:=HMod} = HConfig -> ok = logger:add_handler(?STANDARD_HANDLER,HMod,HConfig); _ -> ok @@ -63,8 +63,8 @@ end_per_group(_Group, _Config) -> ok. init_per_testcase(_TestCase, Config) -> - {ok,LC} = logger:get_logger_config(), - [{logger_config,LC}|Config]. + PC = logger:get_primary_config(), + [{logger_config,PC}|Config]. end_per_testcase(Case, Config) -> try apply(?MODULE,Case,[cleanup,Config]) @@ -82,11 +82,12 @@ all() -> add_remove_filter, change_config, set_formatter, + log_no_levels, log_all_levels_api, macros, set_level, - set_level_module, - cache_level_module, + set_module_level, + cache_module_level, format_report, filter_failed, handler_failed, @@ -105,29 +106,29 @@ start_stop(_Config) -> add_remove_handler(_Config) -> register(callback_receiver,self()), - #{handlers:=Hs0} = logger:i(), + Hs0 = logger:get_handler_config(), {error,{not_found,h1}} = logger:get_handler_config(h1), ok = logger:add_handler(h1,?MODULE,#{}), [add] = test_server:messages_get(), - #{handlers:=Hs} = logger:i(), - {value,_,Hs0} = lists:keytake(h1,1,Hs), - {ok,{?MODULE,#{level:=info,filters:=[],filter_default:=log}}} = % defaults + Hs = logger:get_handler_config(), + Hs0 = lists:filter(fun(#{id:=h1}) -> false; (_) -> true end, Hs), + {ok,#{module:=?MODULE,level:=all,filters:=[],filter_default:=log}} = %defaults logger:get_handler_config(h1), ok = logger:set_handler_config(h1,filter_default,stop), [changing_config] = test_server:messages_get(), - ?LOG_INFO("hello",[]), + ?LOG_NOTICE("hello",[]), ok = check_no_log(), ok = logger:set_handler_config(h1,filter_default,log), [changing_config] = test_server:messages_get(), - {ok,{?MODULE,#{filter_default:=log}}} = logger:get_handler_config(h1), - ?LOG_INFO("hello",[]), - ok = check_logged(info,"hello",[],?MY_LOC(1)), + {ok,#{filter_default:=log}} = logger:get_handler_config(h1), + ?LOG_NOTICE("hello",[]), + ok = check_logged(notice,"hello",[],?MY_LOC(1)), ok = logger:remove_handler(h1), [remove] = test_server:messages_get(), - #{handlers:=Hs0} = logger:i(), + Hs0 = logger:get_handler_config(), {error,{not_found,h1}} = logger:get_handler_config(h1), {error,{not_found,h1}} = logger:remove_handler(h1), - logger:info("hello",[]), + logger:notice("hello",[]), ok = check_no_log(), ok. @@ -136,13 +137,13 @@ add_remove_handler(cleanup,_Config) -> ok. multiple_handlers(_Config) -> - ok = logger:add_handler(h1,?MODULE,#{level=>info,filter_default=>log}), + ok = logger:add_handler(h1,?MODULE,#{level=>notice,filter_default=>log}), ok = logger:add_handler(h2,?MODULE,#{level=>error,filter_default=>log}), ?LOG_ERROR("hello",[]), ok = check_logged(error,"hello",[],?MY_LOC(1)), ok = check_logged(error,"hello",[],?MY_LOC(2)), - ?LOG_INFO("hello",[]), - ok = check_logged(info,"hello",[],?MY_LOC(1)), + ?LOG_NOTICE("hello",[]), + ok = check_logged(notice,"hello",[],?MY_LOC(1)), ok = check_no_log(), ok. @@ -152,18 +153,18 @@ multiple_handlers(cleanup,_Config) -> ok. add_remove_filter(_Config) -> - ok = logger:add_handler(h1,?MODULE,#{level=>info,filter_default=>log}), + ok = logger:add_handler(h1,?MODULE,#{level=>notice,filter_default=>log}), LF = {fun(Log,_) -> Log#{level=>error} end, []}, - ok = logger:add_logger_filter(lf,LF), - {error,{already_exist,lf}} = logger:add_logger_filter(lf,LF), - {error,{already_exist,lf}} = logger:add_logger_filter(lf,{fun(Log,_) -> - Log - end, []}), - ?LOG_INFO("hello",[]), + ok = logger:add_primary_filter(lf,LF), + {error,{already_exist,lf}} = logger:add_primary_filter(lf,LF), + {error,{already_exist,lf}} = logger:add_primary_filter(lf,{fun(Log,_) -> + Log + end, []}), + ?LOG_NOTICE("hello",[]), ok = check_logged(error,"hello",[],?MY_LOC(1)), ok = check_no_log(), - ok = logger:add_handler(h2,?MODULE,#{level=>info,filter_default=>log}), + ok = logger:add_handler(h2,?MODULE,#{level=>notice,filter_default=>log}), HF = {fun(#{level:=error}=Log,_) -> Log#{level=>mylevel}; (_,_) -> @@ -175,16 +176,16 @@ add_remove_filter(_Config) -> {error,{already_exist,hf}} = logger:add_handler_filter(h1,hf,{fun(Log,_) -> Log end, []}), - ?LOG_INFO("hello",[]), + ?LOG_NOTICE("hello",[]), ok = check_logged(mylevel,"hello",[],?MY_LOC(1)), ok = check_logged(error,"hello",[],?MY_LOC(2)), - ok = logger:remove_logger_filter(lf), - {error,{not_found,lf}} = logger:remove_logger_filter(lf), + ok = logger:remove_primary_filter(lf), + {error,{not_found,lf}} = logger:remove_primary_filter(lf), - ?LOG_INFO("hello",[]), - ok = check_logged(info,"hello",[],?MY_LOC(1)), - ok = check_logged(info,"hello",[],?MY_LOC(2)), + ?LOG_NOTICE("hello",[]), + ok = check_logged(notice,"hello",[],?MY_LOC(1)), + ok = check_logged(notice,"hello",[],?MY_LOC(2)), ?LOG_ERROR("hello",[]), ok = check_logged(mylevel,"hello",[],?MY_LOC(1)), @@ -192,9 +193,9 @@ add_remove_filter(_Config) -> ok = logger:remove_handler_filter(h1,hf), {error,{not_found,hf}} = logger:remove_handler_filter(h1,hf), - ?LOG_INFO("hello",[]), - ok = check_logged(info,"hello",[],?MY_LOC(1)), - ok = check_logged(info,"hello",[],?MY_LOC(2)), + ?LOG_NOTICE("hello",[]), + ok = check_logged(notice,"hello",[],?MY_LOC(1)), + ok = check_logged(notice,"hello",[],?MY_LOC(2)), ?LOG_ERROR("hello",[]), ok = check_logged(error,"hello",[],?MY_LOC(1)), @@ -202,20 +203,20 @@ add_remove_filter(_Config) -> ok. add_remove_filter(cleanup,_Config) -> - logger:remove_logger_filter(lf), + logger:remove_primary_filter(lf), logger:remove_handler(h1), logger:remove_handler(h2), ok. change_config(_Config) -> %% Overwrite handler config - check that defaults are added - ok = logger:add_handler(h1,?MODULE,#{level=>debug,custom=>custom}), - {ok,{?MODULE,#{level:=debug,filter_default:=log,custom:=custom}}} = + ok = logger:add_handler(h1,?MODULE,#{level=>notice,custom=>custom}), + {ok,#{module:=?MODULE,level:=notice,filter_default:=log,custom:=custom}} = logger:get_handler_config(h1), register(callback_receiver,self()), ok = logger:set_handler_config(h1,#{filter_default=>stop}), [changing_config] = test_server:messages_get(), - {ok,{?MODULE,#{level:=info,filter_default:=stop}=C2}} = + {ok,#{module:=?MODULE,level:=all,filter_default:=stop}=C2} = logger:get_handler_config(h1), false = maps:is_key(custom,C2), {error,fail} = logger:set_handler_config(h1,#{conf_call=>fun() -> {error,fail} end}), @@ -225,60 +226,60 @@ change_config(_Config) -> ok = logger:set_handler_config( h1,#{conf_call=>fun() -> logger:set_module_level(?MODULE,debug) end}), - {ok,{?MODULE,C2}} = logger:get_handler_config(h1), + {ok,C2} = logger:get_handler_config(h1), %% Change handler config: Single key {error,fail} = logger:set_handler_config(h1,conf_call,fun() -> {error,fail} end), ok = logger:set_handler_config(h1,custom,custom), [changing_config] = test_server:messages_get(), - {ok,{?MODULE,#{custom:=custom}=C3}} = logger:get_handler_config(h1), + {ok,#{custom:=custom}=C3} = logger:get_handler_config(h1), C2 = maps:remove(custom,C3), %% Change handler config: Map ok = logger:update_handler_config(h1,#{custom=>new_custom}), [changing_config] = test_server:messages_get(), - {ok,{_,C4}} = logger:get_handler_config(h1), + {ok,C4} = logger:get_handler_config(h1), C4 = C3#{custom:=new_custom}, - %% Change logger config: Single key - {ok,LConfig0} = logger:get_logger_config(), - ok = logger:set_logger_config(level,warning), - {ok,LConfig1} = logger:get_logger_config(), - LConfig1 = LConfig0#{level:=warning}, - - %% Change logger config: Map - ok = logger:update_logger_config(#{level=>error}), - {ok,LConfig2} = logger:get_logger_config(), - LConfig2 = LConfig1#{level:=error}, - - %% Overwrite logger config - check that defaults are added - ok = logger:set_logger_config(#{filter_default=>stop}), - {ok,#{level:=info,filters:=[],filter_default:=stop}=LC1} = - logger:get_logger_config(), - 3 = maps:size(LC1), + %% Change primary config: Single key + PConfig0 = logger:get_primary_config(), + ok = logger:set_primary_config(level,warning), + PConfig1 = logger:get_primary_config(), + PConfig1 = PConfig0#{level:=warning}, + + %% Change primary config: Map + ok = logger:update_primary_config(#{level=>error}), + PConfig2 = logger:get_primary_config(), + PConfig2 = PConfig1#{level:=error}, + + %% Overwrite primary config - check that defaults are added + ok = logger:set_primary_config(#{filter_default=>stop}), + #{level:=notice,filters:=[],filter_default:=stop}=PC1 = + logger:get_primary_config(), + 3 = maps:size(PC1), %% Check that internal 'handlers' field has not been changed - #{handlers:=HCs} = logger:i(), - HIds1 = [Id || {Id,_,_} <- HCs], - {ok,#{handlers:=HIds2}} = logger_config:get(?LOGGER_TABLE,logger), - HIds1 = lists:sort(HIds2), + MS = [{{{?HANDLER_KEY,'$1'},'_','_','_'},[],['$1']}], + HIds1 = lists:sort(ets:select(?LOGGER_TABLE,MS)), % dirty, internal data + HIds2 = lists:sort(logger:get_handler_ids()), + HIds1 = HIds2, %% Cleanup - ok = logger:set_logger_config(LConfig0), + ok = logger:set_primary_config(PConfig0), [] = test_server:messages_get(), ok. change_config(cleanup,Config) -> logger:remove_handler(h1), - LC = ?config(logger_config,Config), - logger:set_logger_config(LC), + PC = ?config(logger_config,Config), + logger:set_primary_config(PC), ok. set_formatter(_Config) -> {error,{not_found,h1}}=logger:set_handler_config(h1,formatter,{?MODULE,[]}), - ok = logger:add_handler(h1,?MODULE,#{level=>info,filter_default=>log}), + ok = logger:add_handler(h1,?MODULE,#{level=>notice,filter_default=>log}), ok = logger:set_handler_config(h1,formatter,{?MODULE,[]}), - logger:info("hello",[]), + logger:notice("hello",[]), receive {_Log,#{formatter:={?MODULE,[]}}} -> ok @@ -291,9 +292,50 @@ set_formatter(cleanup,_Config) -> logger:remove_handler(h1), ok. +log_no_levels(_Config) -> + ok = logger:add_handler(h1,?MODULE,#{level=>all,filter_default=>log}), + logger:notice(M1=?map_rep), + ok = check_logged(notice,M1,#{}), + + Levels = [emergency,alert,critical,error,warning,notice,info,debug], + ok = logger:set_primary_config(level,none), + [logger:Level(#{Level=>rep}) || Level <- Levels], + ok = check_no_log(), + + ok = logger:set_primary_config(level,all), + M2 = ?map_rep, + ?LOG_NOTICE(M2), + ok = check_logged(notice,M2,#{}), + + ok = logger:set_module_level(?MODULE,none), + ?LOG_EMERGENCY(?map_rep), + ?LOG_ALERT(?map_rep), + ?LOG_CRITICAL(?map_rep), + ?LOG_ERROR(?map_rep), + ?LOG_WARNING(?map_rep), + ?LOG_NOTICE(?map_rep), + ?LOG_INFO(?map_rep), + ?LOG_DEBUG(?map_rep), + ok = check_no_log(), + + ok = logger:unset_module_level(?MODULE), + logger:notice(M3=?map_rep), + ok = check_logged(notice,M3,#{}), + + ok = logger:set_handler_config(h1,level,none), + [logger:Level(#{Level=>rep}) || Level <- Levels], + ok = check_no_log(), + + ok. +log_no_levels(cleanup,_Config) -> + logger:remove_handler(h1), + logger:set_primary_config(level,notice), + logger:unset_module_level(?MODULE), + ok. + log_all_levels_api(_Config) -> - ok = logger:set_logger_config(level,debug), - ok = logger:add_handler(h1,?MODULE,#{level=>debug,filter_default=>log}), + ok = logger:set_primary_config(level,all), + ok = logger:add_handler(h1,?MODULE,#{level=>all,filter_default=>log}), test_api(emergency), test_api(alert), test_api(critical), @@ -307,13 +349,13 @@ log_all_levels_api(_Config) -> log_all_levels_api(cleanup,_Config) -> logger:remove_handler(h1), - logger:set_logger_config(level,info), + logger:set_primary_config(level,notice), ok. macros(_Config) -> - ok = logger:set_module_level(?MODULE,debug), - ok = logger:add_handler(h1,?MODULE,#{level=>debug,filter_default=>log}), + ok = logger:add_handler(h1,?MODULE,#{level=>all,filter_default=>log}), test_macros(emergency), + test_log_macro(alert), ok. macros(cleanup,_Config) -> @@ -322,56 +364,77 @@ macros(cleanup,_Config) -> ok. set_level(_Config) -> - ok = logger:add_handler(h1,?MODULE,#{level=>debug,filter_default=>log}), + ok = logger:add_handler(h1,?MODULE,#{level=>all,filter_default=>log}), logger:debug(?map_rep), ok = check_no_log(), - logger:info(M1=?map_rep), - ok = check_logged(info,M1,#{}), - ok = logger:set_logger_config(level,debug), + logger:notice(M1=?map_rep), + ok = check_logged(notice,M1,#{}), + ok = logger:set_primary_config(level,debug), logger:debug(M2=?map_rep), ok = check_logged(debug,M2,#{}), ok. set_level(cleanup,_Config) -> logger:remove_handler(h1), - logger:set_logger_config(level,info), + logger:set_primary_config(level,notice), ok. -set_level_module(_Config) -> - ok = logger:add_handler(h1,?MODULE,#{level=>info,filter_default=>log}), +set_module_level(_Config) -> + [] = logger:get_module_level([?MODULE,other]), + [] = logger:get_module_level(?MODULE), + [] = logger:get_module_level(), + + ok = logger:add_handler(h1,?MODULE,#{level=>notice,filter_default=>log}), {error,{invalid_level,bad}} = logger:set_module_level(?MODULE,bad), - {error,{not_a_module,{bad}}} = logger:set_module_level({bad},warning), + {error,{not_a_list_of_modules,{bad}}} = + logger:set_module_level({bad},warning), ok = logger:set_module_level(?MODULE,warning), - logger:info(?map_rep,?MY_LOC(0)), + [{?MODULE,warning}] = logger:get_module_level([?MODULE,other]), + [{?MODULE,warning}] = logger:get_module_level(?MODULE), + [{?MODULE,warning}] = logger:get_module_level(), + logger:notice(?map_rep,?MY_LOC(0)), ok = check_no_log(), logger:warning(M1=?map_rep,?MY_LOC(0)), ok = check_logged(warning,M1,?MY_LOC(1)), - ok = logger:set_module_level(?MODULE,info), - logger:info(M2=?map_rep,?MY_LOC(0)), - ok = check_logged(info,M2,?MY_LOC(1)), - - {error,{not_a_module,{bad}}} = logger:unset_module_level({bad}), + ok = logger:set_module_level(?MODULE,notice), + [{?MODULE,notice}] = logger:get_module_level([?MODULE,other]), + [{?MODULE,notice}] = logger:get_module_level(?MODULE), + [{?MODULE,notice}] = logger:get_module_level(), + logger:notice(M2=?map_rep,?MY_LOC(0)), + ok = check_logged(notice,M2,?MY_LOC(1)), + + {error,{not_a_list_of_modules,{bad}}} = logger:unset_module_level({bad}), ok = logger:unset_module_level(?MODULE), + [] = logger:get_module_level([?MODULE,other]), + [] = logger:get_module_level(?MODULE), + [] = logger:get_module_level(), + + ok = logger:set_module_level([m1,m2,m3],notice), + [{m1,notice},{m2,notice},{m3,notice}] = logger:get_module_level(), + ok = logger:unset_module_level(m2), + [{m1,notice},{m3,notice}] = logger:get_module_level(), + ok = logger:unset_module_level(), + [] = logger:get_module_level(), ok. -set_level_module(cleanup,_Config) -> +set_module_level(cleanup,_Config) -> logger:remove_handler(h1), logger:unset_module_level(?MODULE), ok. -cache_level_module(_Config) -> +cache_module_level(_Config) -> ok = logger:unset_module_level(?MODULE), - [] = ets:lookup(logger,?MODULE), %dirty - add API in logger_config? - ?LOG_INFO(?map_rep), + [] = ets:lookup(?LOGGER_TABLE,?MODULE), %dirty - add API in logger_config? + ?LOG_NOTICE(?map_rep), %% Caching is done asynchronously, so wait a bit for the update timer:sleep(100), - [_] = ets:lookup(logger,?MODULE), %dirty - add API in logger_config? + [_] = ets:lookup(?LOGGER_TABLE,?MODULE), %dirty - add API in logger_config? ok = logger:unset_module_level(?MODULE), - [] = ets:lookup(logger,?MODULE), %dirty - add API in logger_config? + [] = ets:lookup(?LOGGER_TABLE,?MODULE), %dirty - add API in logger_config? ok. -cache_level_module(cleanup,_Config) -> +cache_module_level(cleanup,_Config) -> logger:unset_module_level(?MODULE), ok. @@ -402,22 +465,26 @@ format_report(_Config) -> ok. filter_failed(_Config) -> - ok = logger:add_handler(h1,?MODULE,#{level=>info,filter_default=>log}), + ok = logger:add_handler(h1,?MODULE,#{level=>notice,filter_default=>log}), %% Logger filters {error,{invalid_filter,_}} = - logger:add_logger_filter(lf,{fun(_) -> ok end,args}), - ok = logger:add_logger_filter(lf,{fun(_,_) -> a=b end,args}), - {ok,#{filters:=[_]}} = logger:get_logger_config(), - ok = logger:info(M1=?map_rep), - ok = check_logged(info,M1,#{}), - {error,{not_found,lf}} = logger:remove_logger_filter(lf), - - ok = logger:add_logger_filter(lf,{fun(_,_) -> faulty_return end,args}), - {ok,#{filters:=[_]}} = logger:get_logger_config(), - ok = logger:info(M2=?map_rep), - ok = check_logged(info,M2,#{}), - {error,{not_found,lf}} = logger:remove_logger_filter(lf), + logger:add_primary_filter(lf,{fun(_) -> ok end,args}), + ok = logger:add_primary_filter(lf, + {fun(_,_) -> + erlang:error({badmatch,b}) + end, + args}), + #{filters:=[_]} = logger:get_primary_config(), + ok = logger:notice(M1=?map_rep), + ok = check_logged(notice,M1,#{}), + {error,{not_found,lf}} = logger:remove_primary_filter(lf), + + ok = logger:add_primary_filter(lf,{fun(_,_) -> faulty_return end,args}), + #{filters:=[_]} = logger:get_primary_config(), + ok = logger:notice(M2=?map_rep), + ok = check_logged(notice,M2,#{}), + {error,{not_found,lf}} = logger:remove_primary_filter(lf), %% Handler filters {error,{not_found,h0}} = @@ -425,16 +492,20 @@ filter_failed(_Config) -> {error,{not_found,h0}} = logger:remove_handler_filter(h0,hf), {error,{invalid_filter,_}} = logger:add_handler_filter(h1,hf,{fun(_) -> ok end,args}), - ok = logger:add_handler_filter(h1,hf,{fun(_,_) -> a=b end,args}), - {ok,{?MODULE,#{filters:=[_]}}} = logger:get_handler_config(h1), - ok = logger:info(M3=?map_rep), - ok = check_logged(info,M3,#{}), + ok = logger:add_handler_filter(h1,hf, + {fun(_,_) -> + erlang:error({badmatch,b}) + end, + args}), + {ok,#{filters:=[_]}} = logger:get_handler_config(h1), + ok = logger:notice(M3=?map_rep), + ok = check_logged(notice,M3,#{}), {error,{not_found,hf}} = logger:remove_handler_filter(h1,hf), ok = logger:add_handler_filter(h1,hf,{fun(_,_) -> faulty_return end,args}), - {ok,{?MODULE,#{filters:=[_]}}} = logger:get_handler_config(h1), - ok = logger:info(M4=?map_rep), - ok = check_logged(info,M4,#{}), + {ok,#{filters:=[_]}} = logger:get_handler_config(h1), + ok = logger:notice(M4=?map_rep), + ok = check_logged(notice,M4,#{}), {error,{not_found,hf}} = logger:remove_handler_filter(h1,hf), ok. @@ -455,24 +526,28 @@ handler_failed(_Config) -> {error,{invalid_formatter,[]}} = logger:add_handler(h1,?MODULE,#{formatter=>[]}), {error,{invalid_handler,_}} = logger:add_handler(h1,nomodule,#{filter_default=>log}), - logger:info(?map_rep), + logger:notice(?map_rep), check_no_log(), - #{handlers:=H1} = logger:i(), - false = lists:keymember(h1,1,H1), + H1 = logger:get_handler_config(), + false = lists:search(fun(#{id:=h1}) -> true; (_) -> false end,H1), {error,{not_found,h1}} = logger:remove_handler(h1), - ok = logger:add_handler(h2,?MODULE,#{filter_default=>log,log_call=>fun() -> a = b end}), + ok = logger:add_handler(h2,?MODULE, + #{filter_default => log, + log_call => fun() -> + erlang:error({badmatch,b}) + end}), {error,{already_exist,h2}} = logger:add_handler(h2,othermodule,#{}), [add] = test_server:messages_get(), - logger:info(?map_rep), + logger:notice(?map_rep), [remove] = test_server:messages_get(), - #{handlers:=H2} = logger:i(), - false = lists:keymember(h2,1,H2), + H2 = logger:get_handler_config(), + false = lists:search(fun(#{id:=h2}) -> true; (_) -> false end,H2), {error,{not_found,h2}} = logger:remove_handler(h2), CallAddHandler = fun() -> logger:add_handler(h2,?MODULE,#{}) end, - CrashHandler = fun() -> a = b end, + CrashHandler = fun() -> erlang:error({badmatch,b}) end, KillHandler = fun() -> exit(self(), die) end, {error,{handler_not_added,{attempting_syncronous_call_to_self,_}}} = @@ -520,18 +595,18 @@ handler_failed(cleanup,_Config) -> config_sanity_check(_Config) -> %% Logger config {error,{invalid_filter_default,bad}} = - logger:set_logger_config(filter_default,bad), - {error,{invalid_level,bad}} = logger:set_logger_config(level,bad), - {error,{invalid_filters,bad}} = logger:set_logger_config(filters,bad), - {error,{invalid_filter,bad}} = logger:set_logger_config(filters,[bad]), + logger:set_primary_config(filter_default,bad), + {error,{invalid_level,bad}} = logger:set_primary_config(level,bad), + {error,{invalid_filters,bad}} = logger:set_primary_config(filters,bad), + {error,{invalid_filter,bad}} = logger:set_primary_config(filters,[bad]), {error,{invalid_filter,{_,_}}} = - logger:set_logger_config(filters,[{id,bad}]), + logger:set_primary_config(filters,[{id,bad}]), {error,{invalid_filter,{_,{_,_}}}} = - logger:set_logger_config(filters,[{id,{bad,args}}]), + logger:set_primary_config(filters,[{id,{bad,args}}]), {error,{invalid_filter,{_,{_,_}}}} = - logger:set_logger_config(filters,[{id,{fun() -> ok end,args}}]), - {error,{invalid_logger_config,{bad,bad}}} = - logger:set_logger_config(bad,bad), + logger:set_primary_config(filters,[{id,{fun() -> ok end,args}}]), + {error,{invalid_primary_config,{bad,bad}}} = + logger:set_primary_config(bad,bad), %% Handler config {error,{not_found,h1}} = logger:set_handler_config(h1,a,b), @@ -555,7 +630,7 @@ config_sanity_check(_Config) -> logger:set_handler_config(h1,formatter,{logger_formatter,bad}), {error,{invalid_formatter_config,logger_formatter,{bad,bad}}} = logger:set_handler_config(h1,formatter,{logger_formatter,#{bad=>bad}}), - {error,{invalid_formatter_config,logger_formatter,{template,bad}}} = + {error,{invalid_formatter_template,logger_formatter,bad}} = logger:set_handler_config(h1,formatter,{logger_formatter, #{template=>bad}}), {error,{invalid_formatter_template,logger_formatter,[1]}} = @@ -648,30 +723,30 @@ config_sanity_check(cleanup,_Config) -> ok. log_failed(_Config) -> - ok = logger:add_handler(h1,?MODULE,#{level=>info,filter_default=>log}), + ok = logger:add_handler(h1,?MODULE,#{level=>notice,filter_default=>log}), {error,function_clause} = ?TRY(logger:log(bad,?map_rep)), - {error,function_clause} = ?TRY(logger:log(info,?map_rep,bad)), - {error,function_clause} = ?TRY(logger:log(info,fun() -> ?map_rep end,bad)), - {error,function_clause} = ?TRY(logger:log(info,fun() -> ?map_rep end,bad,#{})), - {error,function_clause} = ?TRY(logger:log(info,bad,bad,bad)), - {error,function_clause} = ?TRY(logger:log(info,bad,bad,#{})), + {error,function_clause} = ?TRY(logger:log(notice,?map_rep,bad)), + {error,function_clause} = ?TRY(logger:log(notice,fun() -> ?map_rep end,bad)), + {error,function_clause} = ?TRY(logger:log(notice,fun() -> ?map_rep end,bad,#{})), + {error,function_clause} = ?TRY(logger:log(notice,bad,bad,bad)), + {error,function_clause} = ?TRY(logger:log(notice,bad,bad,#{})), check_no_log(), - ok = logger:log(info,M1=?str,#{}), - check_logged(info,M1,#{}), - ok = logger:log(info,M2=?map_rep,#{}), - check_logged(info,M2,#{}), - ok = logger:log(info,M3=?keyval_rep,#{}), - check_logged(info,M3,#{}), + ok = logger:log(notice,M1=?str,#{}), + check_logged(notice,M1,#{}), + ok = logger:log(notice,M2=?map_rep,#{}), + check_logged(notice,M2,#{}), + ok = logger:log(notice,M3=?keyval_rep,#{}), + check_logged(notice,M3,#{}), %% Should we check report input more thoroughly? - ok = logger:log(info,M4=?keyval_rep++[other,stuff,in,list],#{}), - check_logged(info,M4,#{}), + ok = logger:log(notice,M4=?keyval_rep++[other,stuff,in,list],#{}), + check_logged(notice,M4,#{}), %% This might break a handler since it is assumed to be a format %% string and args, so it depends how the handler protects itself %% against something like io_lib:format("ok","ok") - ok = logger:log(info,"ok","ok",#{}), - check_logged(info,"ok","ok",#{}), + ok = logger:log(notice,"ok","ok",#{}), + check_logged(notice,"ok","ok",#{}), ok. @@ -680,7 +755,7 @@ log_failed(cleanup,_Config) -> ok. emulator(_Config) -> - ok = logger:add_handler(h1,?MODULE,#{level=>info,filter_default=>log, + ok = logger:add_handler(h1,?MODULE,#{level=>notice,filter_default=>log, tc_proc=>self()}), Msg = "Error in process ~p on node ~p with exit value:~n~p~n", Error = {badmatch,4}, @@ -699,7 +774,7 @@ generate_error(Error, Stack) -> erlang:raise(error, Error, Stack). via_logger_process(Config) -> - ok = logger:add_handler(h1,?MODULE,#{level=>info,filter_default=>log, + ok = logger:add_handler(h1,?MODULE,#{level=>notice,filter_default=>log, tc_proc=>self()}), %% Explicitly send a message to the logger process @@ -736,7 +811,7 @@ via_logger_process(cleanup, Config) -> ok. other_node(_Config) -> - ok = logger:add_handler(h1,?MODULE,#{level=>info,filter_default=>log, + ok = logger:add_handler(h1,?MODULE,#{level=>notice,filter_default=>log, tc_proc=>self()}), {ok,Node} = test_server:start_node(?FUNCTION_NAME,slave,[]), rpc:call(Node,logger,error,[Msg=?str,#{}]), @@ -753,8 +828,8 @@ compare_levels(_Config) -> Levels = [emergency,alert,critical,error,warning,notice,info,debug], ok = compare(Levels), {error,badarg} = ?TRY(logger:compare_levels(bad,bad)), - {error,badarg} = ?TRY(logger:compare_levels({bad},info)), - {error,badarg} = ?TRY(logger:compare_levels(info,"bad")), + {error,badarg} = ?TRY(logger:compare_levels({bad},notice)), + {error,badarg} = ?TRY(logger:compare_levels(notice,"bad")), ok. compare([L|Rest]) -> @@ -768,21 +843,21 @@ compare([]) -> process_metadata(_Config) -> undefined = logger:get_process_metadata(), {error,badarg} = ?TRY(logger:set_process_metadata(bad)), - ok = logger:add_handler(h1,?MODULE,#{level=>info,filter_default=>log}), + ok = logger:add_handler(h1,?MODULE,#{level=>notice,filter_default=>log}), Time = erlang:system_time(microsecond), ProcMeta = #{time=>Time,line=>0,custom=>proc}, ok = logger:set_process_metadata(ProcMeta), S1 = ?str, - ?LOG_INFO(S1,#{custom=>macro}), - check_logged(info,S1,#{time=>Time,line=>0,custom=>macro}), + ?LOG_NOTICE(S1,#{custom=>macro}), + check_logged(notice,S1,#{time=>Time,line=>0,custom=>macro}), Time2 = erlang:system_time(microsecond), S2 = ?str, - ?LOG_INFO(S2,#{time=>Time2,line=>1,custom=>macro}), - check_logged(info,S2,#{time=>Time2,line=>1,custom=>macro}), + ?LOG_NOTICE(S2,#{time=>Time2,line=>1,custom=>macro}), + check_logged(notice,S2,#{time=>Time2,line=>1,custom=>macro}), - logger:info(S3=?str,#{custom=>func}), - check_logged(info,S3,#{time=>Time,line=>0,custom=>func}), + logger:notice(S3=?str,#{custom=>func}), + check_logged(notice,S3,#{time=>Time,line=>0,custom=>func}), ProcMeta = logger:get_process_metadata(), ok = logger:update_process_metadata(#{custom=>changed,custom2=>added}), @@ -947,6 +1022,34 @@ test_macros(emergency=Level) -> [{F2,x},{error,fun_that_crashes}],#{}), ok. +test_log_macro(Level) -> + ?LOG(Level,#{Level=>rep}), + ok = check_logged(Level,#{Level=>rep},?MY_LOC(1)), + ?LOG(Level,#{Level=>rep},#{my=>meta}), + ok = check_logged(Level,#{Level=>rep},(?MY_LOC(1))#{my=>meta}), + ?LOG(Level,"~w: ~w",[Level,fa]), + ok = check_logged(Level,"~w: ~w",[Level,fa],?MY_LOC(1)), + ?LOG(Level,"~w: ~w ~w",[Level,fa,meta],#{my=>meta}), + ok = check_logged(Level,"~w: ~w ~w",[Level,fa,meta],(?MY_LOC(1))#{my=>meta}), + ?LOG(Level,fun(x) -> {"~w: ~w ~w",[Level,fun_to_fa,meta]} end, + x, #{my=>meta}), + ok = check_logged(Level,"~w: ~w ~w",[Level,fun_to_fa,meta], + (?MY_LOC(3))#{my=>meta}), + ?LOG(Level,fun(x) -> #{Level=>fun_to_r,meta=>true} end, x, #{my=>meta}), + ok = check_logged(Level,#{Level=>fun_to_r,meta=>true}, + (?MY_LOC(2))#{my=>meta}), + ?LOG(Level,fun(x) -> <<"fun_to_s">> end,x,#{}), + ok = check_logged(Level,<<"fun_to_s">>,?MY_LOC(1)), + F1=fun(x) -> {fun_to_bad} end, + ?LOG(Level,F1,x,#{}), + ok = check_logged(Level,"LAZY_FUN ERROR: ~tp; Returned: ~tp", + [{F1,x},{fun_to_bad}],#{}), + F2=fun(x) -> erlang:error(fun_that_crashes) end, + ?LOG(Level,F2,x,#{}), + ok = check_logged(Level,"LAZY_FUN CRASH: ~tp; Reason: ~tp", + [{F2,x},{error,fun_that_crashes}],#{}), + ok. + %%%----------------------------------------------------------------- %%% Called by macro ?TRY(X) my_try(Fun) -> diff --git a/lib/kernel/test/logger_bench_SUITE.erl b/lib/kernel/test/logger_bench_SUITE.erl deleted file mode 100644 index d47122ea9d..0000000000 --- a/lib/kernel/test/logger_bench_SUITE.erl +++ /dev/null @@ -1,500 +0,0 @@ -%% -%% %CopyrightBegin% -%% -%% Copyright Ericsson AB 2018. All Rights Reserved. -%% -%% Licensed under the Apache License, Version 2.0 (the "License"); -%% you may not use this file except in compliance with the License. -%% You may obtain a copy of the License at -%% -%% http://www.apache.org/licenses/LICENSE-2.0 -%% -%% Unless required by applicable law or agreed to in writing, software -%% distributed under the License is distributed on an "AS IS" BASIS, -%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -%% See the License for the specific language governing permissions and -%% limitations under the License. -%% -%% %CopyrightEnd% -%% --module(logger_bench_SUITE). - --compile(export_all). - -%%%----------------------------------------------------------------- -%%% To include lager tests, add paths to lager and goldrush -%%% (goldrush is a dependency inside the lager repo) -%%% -%%% To print data to .csv files, add the following to a config file: -%%% {print_csv,[{console_handler,[{path,"/some/dir/"}]}]}. -%%% -%%%----------------------------------------------------------------- - --include_lib("common_test/include/ct.hrl"). --include_lib("common_test/include/ct_event.hrl"). --include_lib("kernel/include/logger.hrl"). --include_lib("kernel/src/logger_internal.hrl"). - --define(msg,lists:flatten(string:pad("Log from "++atom_to_list(?FUNCTION_NAME)++ - ":"++integer_to_list(?LINE), - 80,trailing,$*))). --define(meta,#{mfa=>{?MODULE,?FUNCTION_NAME,?FUNCTION_ARITY}, - pid=>self()}). - --define(NO_COMPARE,[profile]). - --define(TIMES,100000). - -suite() -> - [{timetrap,{seconds,120}}]. - -init_per_suite(Config) -> - DataDir = ?config(data_dir,Config), - have_lager() andalso make(DataDir), - Config. - -end_per_suite(_Config) -> - ok. - -init_per_group(Group, Config) -> - H = remove_all_handlers(), - do_init_per_group(Group), - [{handlers,H}|Config]. - -do_init_per_group(minimal_handler) -> - ok = logger:add_handler(?MODULE,?MODULE,#{level=>error,filter_default=>log}); -do_init_per_group(console_handler) -> - ok = logger:add_handler(?MODULE,logger_std_h, - #{filter_default=>stop, - filters=>?DEFAULT_HANDLER_FILTERS, - logger_std_h=>#{type=>standard_io, - toggle_sync_qlen => ?TIMES+1, - drop_new_reqs_qlen => ?TIMES+2, - flush_reqs_qlen => ?TIMES+3, - enable_burst_limit => false}}), - have_lager() andalso lager_helper:start(), - ok. - -end_per_group(Group, Config) -> - case ?config(saved_config,Config) of - {_,[{bench,Bench}]} -> - print_compare_chart(Group,Bench); - _ -> - ok - end, - add_all_handlers(?config(handlers,Config)), - do_end_per_group(Group). - -do_end_per_group(minimal_handler) -> - ok = logger:remove_handler(?MODULE); -do_end_per_group(console_handler) -> - ok = logger:remove_handler(?MODULE), - have_lager() andalso lager_helper:stop(), - ok. - -init_per_testcase(_TestCase, Config) -> - Config. - -end_per_testcase(Case, Config) -> - try apply(?MODULE,Case,[cleanup,Config]) - catch error:undef -> ok - end, - wait_for_handlers(), - ok. - -wait_for_handlers() -> - wait_for_handler(?MODULE), - wait_for_handler(lager_event). - -wait_for_handler(Handler) -> - case whereis(Handler) of - undefined -> - io:format("~p: noproc1",[Handler]), - ok; - Pid -> - case process_info(Pid,message_queue_len) of - {_,0} -> - io:format("~p: queue=~p",[Handler,0]), - ok; - {_,Q} -> - io:format("~p: queue=~p",[Handler,Q]), - timer:sleep(2000), - wait_for_handler(Handler); - undefined -> - io:format("~p: noproc2",[Handler]), - ok - end - end. - -groups() -> - [{minimal_handler,[],[log, - log_drop, - log_drop_by_handler, - macro, - macro_drop, - macro_drop_by_handler, - error_logger, - error_logger_drop, - error_logger_drop_by_handler - ]}, - {console_handler,[],[%profile, - log, - log_drop, - log_drop_by_handler, - %% log_handler_complete, - macro, - macro_drop, - macro_drop_by_handler, - %% macro_handler_complete, - error_logger, - error_logger_drop, - error_logger_drop_by_handler%% , - %% error_logger_handler_complete - ] ++ lager_cases()} - ]. - -lager_cases() -> - case have_lager() of - true -> - [lager_log, - lager_log_drop, - lager_log_drop_by_handler, - %% lager_log_handler_complete, - lager_parsetrans, - lager_parsetrans_drop, - lager_parsetrans_drop_by_handler%% , - %% lager_parsetrans_handler_complete - ]; - false -> - [] - end. - - -all() -> - [{group,minimal_handler}, - {group,console_handler} - ]. - -log(Config) -> - Times = ?TIMES, - run_benchmark(Config,?FUNCTION_NAME,fun do_log_func/2, [error,?msg], Times). - -log_drop(Config) -> - Times = ?TIMES*100, - ok = logger:set_logger_config(level,error), - run_benchmark(Config,?FUNCTION_NAME,fun do_log_func/2, [info,?msg], Times). - -log_drop(cleanup,_Config) -> - ok = logger:set_logger_config(level,info). - -log_drop_by_handler(Config) -> - Times = ?TIMES, - %% just ensure correct levels - ok = logger:set_logger_config(level,info), - ok = logger:set_handler_config(?MODULE,level,error), - run_benchmark(Config,?FUNCTION_NAME,fun do_log_func/2, [info,?msg], Times). - -log_handler_complete(Config) -> - ok = logger:set_handler_config(?MODULE,formatter, - {?MODULE,?DEFAULT_FORMAT_CONFIG}), - handler_complete(Config, ?FUNCTION_NAME, fun do_log_func/2, [error,?msg]). - -log_handler_complete(cleanup,_Config) -> - ok=logger:set_handler_config(?MODULE,formatter, - {?DEFAULT_FORMATTER,?DEFAULT_FORMAT_CONFIG}). - -macro(Config) -> - Times = ?TIMES, - run_benchmark(Config,?FUNCTION_NAME,fun do_log_macro/2,[error,?msg], Times). - -macro_drop(Config) -> - Times = ?TIMES*100, - ok = logger:set_logger_config(level,error), - run_benchmark(Config,?FUNCTION_NAME,fun do_log_macro/2,[info,?msg], Times). - -macro_drop(cleanup,_Config) -> - ok = logger:set_logger_config(level,info). - -macro_drop_by_handler(Config) -> - Times = ?TIMES, - %% just ensure correct levels - ok = logger:set_logger_config(level,info), - ok = logger:set_handler_config(?MODULE,level,error), - run_benchmark(Config,?FUNCTION_NAME,fun do_log_macro/2, [info,?msg], Times). - -macro_handler_complete(Config) -> - ok = logger:set_handler_config(?MODULE,formatter, - {?MODULE,?DEFAULT_FORMAT_CONFIG}), - handler_complete(Config, ?FUNCTION_NAME, fun do_log_macro/2, [error,?msg]). - -macro_handler_complete(cleanup,_Config) -> - ok=logger:set_handler_config(?MODULE,formatter, - {?DEFAULT_FORMATTER,?DEFAULT_FORMAT_CONFIG}). - -error_logger(Config) -> - Times = ?TIMES, - run_benchmark(Config,?FUNCTION_NAME,fun do_error_logger/2, [error,?msg], Times). - -error_logger_drop(Config) -> - Times = ?TIMES*100, - ok = logger:set_logger_config(level,error), - run_benchmark(Config,?FUNCTION_NAME,fun do_error_logger/2, [info,?msg], Times). - -error_logger_drop(cleanup,_Config) -> - ok = logger:set_logger_config(level,info). - -error_logger_drop_by_handler(Config) -> - Times = ?TIMES, - %% just ensure correct levels - ok = logger:set_logger_config(level,info), - ok = logger:set_handler_config(?MODULE,level,error), - run_benchmark(Config,?FUNCTION_NAME,fun do_log_func/2, [info,?msg], Times). - -error_logger_handler_complete(Config) -> - ok = logger:set_handler_config(?MODULE,formatter, - {?MODULE,?DEFAULT_FORMAT_CONFIG}), - handler_complete(Config, ?FUNCTION_NAME, fun do_error_logger/2, [error,?msg]). - -error_logger_handler_complete(cleanup,_Config) -> - ok=logger:set_handler_config(?MODULE,formatter, - {?DEFAULT_FORMATTER,?DEFAULT_FORMAT_CONFIG}). - -lager_log(Config) -> - Times = ?TIMES, - run_benchmark(Config,?FUNCTION_NAME,fun lager_helper:do_func/2, [error,?msg], Times). - -lager_log_drop(Config) -> - Times = ?TIMES*100, - run_benchmark(Config,?FUNCTION_NAME,fun lager_helper:do_func/2, [info,?msg], Times). - -lager_log_drop_by_handler(Config) -> - %% This concept does not exist, so doing same as lager_log_drop/1 - Times = ?TIMES, - run_benchmark(Config,?FUNCTION_NAME,fun lager_helper:do_func/2, [info,?msg], Times). - -lager_log_handler_complete(Config) -> - handler_complete(Config, ?FUNCTION_NAME, fun lager_helper:do_func/2, [error,?msg]). - -lager_parsetrans(Config) -> - Times = ?TIMES, - run_benchmark(Config,?FUNCTION_NAME,fun lager_helper:do_parsetrans/2, [error,?msg], Times). - -lager_parsetrans_drop(Config) -> - Times = ?TIMES*100, - run_benchmark(Config,?FUNCTION_NAME,fun lager_helper:do_parsetrans/2, [info,?msg], Times). - -lager_parsetrans_drop_by_handler(Config) -> - %% This concept does not exist, so doing same as lager_parsetrans_drop/1 - Times = ?TIMES, - run_benchmark(Config,?FUNCTION_NAME,fun lager_helper:do_parsetrans/2, [info,?msg], Times). - -lager_parsetrans_handler_complete(Config) -> - handler_complete(Config, ?FUNCTION_NAME, fun lager_helper:do_parsetrans/2, [error,?msg]). - - -profile(Config) -> - Times = ?TIMES, - %% fprof:apply(fun repeated_apply/3,[fun lager_helper:do_func/2,[error,?msg],Times]), - fprof:apply(fun repeated_apply/3,[fun do_log_func/2,[error,?msg],Times]), - ok = fprof:profile(), - ok = fprof:analyse(dest,"../fprof.analyse"), - ok. - -%%%----------------------------------------------------------------- -%%% Internal -%% Handler -log(_Log,_Config) -> - ok. - -format(Log=#{meta:=#{pid:=Pid}},Config) when is_pid(Pid) -> - String = ?DEFAULT_FORMATTER:format(Log,Config), - Pid ! done, - String; -format(Log=#{meta:=#{pid:=PidStr}},Config) when is_list(PidStr) -> - String = ?DEFAULT_FORMATTER:format(Log,Config), - list_to_pid(PidStr) ! done, - String. - -handler_complete(Config, TC, Fun, Args) -> - Times = ?TIMES, - Start = os:perf_counter(microsecond), - repeated_apply(Fun, Args, Times), - MSecs = wait_for_done(Start,Times), - calc_and_report(Config,TC,MSecs,Times). - -wait_for_done(Start,0) -> - os:perf_counter(microsecond) - Start; -wait_for_done(Start,N) -> - receive - done -> - wait_for_done(Start,N-1) - after 20000 -> - ct:fail("missing " ++ integer_to_list(N) ++ " replys") - end. - -%%%----------------------------------------------------------------- -%%% Benchmark stuff -run_benchmark(Config,Tag,Fun,Args,Times) -> - _ = erlang:apply(Fun, Args), % apply once to ensure level is cached - MSecs = measure_repeated_op(Fun, Args, Times), - %% fprof:profile(), - %% fprof:analyse(dest,"../"++atom_to_list(Tag)++".prof"), - calc_and_report(Config,Tag,MSecs,Times). - -measure_repeated_op(Fun, Args, Times) -> - Start = os:perf_counter(microsecond), - %% fprof:apply(fun repeated_apply/3, [Fun, Args, Times]), - repeated_apply(Fun, Args, Times), - os:perf_counter(microsecond) - Start. - -repeated_apply(_F, _Args, Times) when Times =< 0 -> - ok; -repeated_apply(F, Args, Times) -> - erlang:apply(F, Args), - repeated_apply(F, Args, Times - 1). - -calc_and_report(Config,Tag,MSecs,Times) -> - IOPS = trunc(Times * (1000000 / MSecs)), - ct_event:notify(#event{ name = benchmark_data, data = [{value,IOPS}] }), - ct:print("~p:~n~p IOPS, ~p us", [Tag, IOPS, MSecs]), - ct:comment("~p IOPS, ~p us", [IOPS, MSecs]), - Bench = case ?config(saved_config,Config) of - {_,[{bench,B}]} -> B; - undefined -> [] - end, - {save_config,[{bench,[{Tag,IOPS,MSecs}|Bench]}]}. - -remove_all_handlers() -> - #{handlers:=Hs} = logger:i(), - [logger:remove_handler(Id) || {Id,_,_} <- Hs], - Hs. - -add_all_handlers(Hs) -> - [logger:add_handler(Id,Mod,Config) || {Id,Mod,Config} <- Hs], - ok. - -%%%----------------------------------------------------------------- -%%% Call logger in different ways -do_log_func(Level,Msg) -> - logger:Level(Msg,[],?meta). - -do_log_macro(error,Msg) -> - ?LOG_ERROR(Msg,[]); -do_log_macro(info,Msg) -> - ?LOG_INFO(Msg,[]); -do_log_macro(debug,Msg) -> - ?LOG_DEBUG(Msg,[]). - -do_error_logger(error,Msg) -> - error_logger:error_msg(Msg,[]); -do_error_logger(info,Msg) -> - error_logger:info_msg(Msg,[]). - -%%%----------------------------------------------------------------- -%%% -print_compare_chart(Group,Bench) -> - io:format("~-20s~12s~12s~12s~12s", - ["Microseconds:","Log","Drop","HDrop","Complete"]), - io:format(user,"~-20s~12s~12s~12s~12s~n", - ["Microseconds:","Log","Drop","HDrop","Complete"]), - {Log,Drop,HDrop,Comp} = sort_bench(Bench,[],[],[],[]), - print_compare_chart(Log,Drop,HDrop,Comp), - io:format(user,"~n",[]), - maybe_print_csv_files(Group, - [{log,Log},{drop,Drop},{hdrop,HDrop},{comp,Comp}]). - -print_compare_chart([{What,LIOPS,LMSecs}|Log], - [{What,DIOPS,DMSecs}|Drop], - [{What,HIOPS,HMSecs}|HDrop], - [{What,CIOPS,CMSecs}|Comp]) -> - io:format("~-20w~12w~12w~12w~12w",[What,LMSecs,DMSecs,HMSecs,CMSecs]), - io:format(user,"~-20w~12w~12w~12w~12w~n",[What,LMSecs,DMSecs,HMSecs,CMSecs]), - print_compare_chart(Log,Drop,HDrop,Comp); -print_compare_chart([{What,LIOPS,LMSecs}|Log], - [{What,DIOPS,DMSecs}|Drop], - [{What,HIOPS,HMSecs}|HDrop], - []=Comp) -> - io:format("~-20w~12w~12w~12w",[What,LMSecs,DMSecs,HMSecs]), - io:format(user,"~-20w~12w~12w~12w~n",[What,LMSecs,DMSecs,HMSecs]), - print_compare_chart(Log,Drop,HDrop,Comp); -print_compare_chart([],[],[],[]) -> - ok; -print_compare_chart(Log,Drop,HDrop,Comp) -> - ct:fail({Log,Drop,HDrop,Comp}). - -sort_bench([{TC,IOPS,MSecs}|Bench],Log,Drop,HDrop,Comp) -> - case lists:member(TC,?NO_COMPARE) of - true -> - sort_bench(Bench,Log,Drop,HDrop,Comp); - false -> - TCStr = atom_to_list(TC), - {What,Type} = - case re:run(TCStr,"(.*)_(drop.*)", - [{capture,all_but_first,list}]) of - {match,[WhatStr,TypeStr]} -> - {list_to_atom(WhatStr),list_to_atom(TypeStr)}; - nomatch -> - case re:run(TCStr,"(.*)_(handler_complete.*)", - [{capture,all_but_first,list}]) of - {match,[WhatStr,TypeStr]} -> - {list_to_atom(WhatStr),list_to_atom(TypeStr)}; - nomatch -> - {TC,log} - end - end, - case Type of - log -> - sort_bench(Bench,[{What,IOPS,MSecs}|Log],Drop,HDrop,Comp); - drop -> - sort_bench(Bench,Log,[{What,IOPS,MSecs}|Drop],HDrop,Comp); - drop_by_handler -> - sort_bench(Bench,Log,Drop,[{What,IOPS,MSecs}|HDrop],Comp); - handler_complete -> - sort_bench(Bench,Log,Drop,HDrop,[{What,IOPS,MSecs}|Comp]) - end - end; -sort_bench([],Log,Drop,HDrop,Comp) -> - {lists:keysort(1,Log), - lists:keysort(1,Drop), - lists:keysort(1,HDrop), - lists:keysort(1,Comp)}. - -maybe_print_csv_files(Group,Data) -> - case ct:get_config({print_csv,Group}) of - undefined -> - ok; - Cfg -> - Path = proplists:get_value(path,Cfg,".."), - Files = [begin - File = filename:join(Path,F)++".csv", - case filelib:is_regular(File) of - true -> - {ok,Fd} = file:open(File,[append]), - Fd; - false -> - {ok,Fd} = file:open(File,[write]), - ok = file:write(Fd, - "error_logger,lager_log," - "lager_parsetrans,logger_log," - "logger_macro\n"), - Fd - end - end || {F,_} <- Data], - [print_csv_file(F,D) || {F,D} <- lists:zip(Files,Data)], - [file:close(Fd) || Fd <- Files], - ok - end. - -print_csv_file(Fd,{_,Data}) -> - AllIOPS = [integer_to_list(IOPS) || {_,IOPS,_} <- Data], - ok = file:write(Fd,lists:join(",",AllIOPS)++"\n"). - -have_lager() -> - code:ensure_loaded(lager) == {module,lager}. - -make(Dir) -> - {ok,Cwd} = file:get_cwd(), - ok = file:set_cwd(Dir), - up_to_date = make:all([load]), - ok = file:set_cwd(Cwd), - code:add_path(Dir). diff --git a/lib/kernel/test/logger_bench_SUITE_data/Emakefile b/lib/kernel/test/logger_bench_SUITE_data/Emakefile deleted file mode 100644 index 85c82bdaab..0000000000 --- a/lib/kernel/test/logger_bench_SUITE_data/Emakefile +++ /dev/null @@ -1 +0,0 @@ -{['lager_helper'],[{outdir,"."},debug_info,{i,"/home/uabshan/Work/git/otp/lib/kernel/src"},{i,"/home/uabshan/Work/git/otp/lib/kernel/include"}]}. diff --git a/lib/kernel/test/logger_bench_SUITE_data/lager_helper.erl b/lib/kernel/test/logger_bench_SUITE_data/lager_helper.erl deleted file mode 100644 index 296ced4276..0000000000 --- a/lib/kernel/test/logger_bench_SUITE_data/lager_helper.erl +++ /dev/null @@ -1,73 +0,0 @@ --module(lager_helper). - --compile(export_all). --compile({parse_transform,lager_transform}). - --include_lib("kernel/src/logger_internal.hrl"). - -start() -> - application:load(lager), - application:set_env(lager, error_logger_redirect, false), - application:set_env(lager, async_threshold, 100010), - application:set_env(lager, async_threshold_window, 100), - application:set_env(lager,handlers,[{?MODULE,[{level,error}]}]), - lager:start(). - -stop() -> - application:stop(lager). - -do_func(Level,Msg) -> - lager:log(Level,[{pid,self()}],Msg,[]). - -do_parsetrans(error,Msg) -> - lager:error(Msg,[]); -do_parsetrans(info,Msg) -> - lager:info(Msg,[]). - -%%%----------------------------------------------------------------- -%%% Dummy handler for lager --record(state, {level :: {'mask', integer()}, - formatter :: atom(), - format_config :: any()}). -init(Opts) -> - Level = proplists:get_value(level,Opts,info), - Formatter = proplists:get_value(formatter,Opts,logger_bench_SUITE), - FormatConfig = proplists:get_value(format_config,Opts,?DEFAULT_FORMAT_CONFIG), - {ok,#state{level=lager_util:config_to_mask(Level), - formatter=Formatter, - format_config=FormatConfig}}. - -handle_call(get_loglevel, #state{level=Level} = State) -> - {ok, Level, State}; -handle_call({set_loglevel, Level}, State) -> - try lager_util:config_to_mask(Level) of - Levels -> - {ok, ok, State#state{level=Levels}} - catch - _:_ -> - {ok, {error, bad_log_level}, State} - end; -handle_call(_Request, State) -> - {ok, ok, State}. - -handle_event({log, Message}, - #state{level=L,formatter=Formatter,format_config=FormatConfig} = State) -> - case lager_util:is_loggable(Message, L, ?MODULE) of - true -> - Metadata = - case maps:from_list(lager_msg:metadata(Message)) of - Meta = #{pid:=Pid} when is_pid(Pid) -> - Meta; - Meta = #{pid:=PidStr} when is_list(PidStr) -> - Meta - end, - Log = #{level=>lager_msg:severity(Message), - msg=>{report,lager_msg:message(Message)}, - meta=>Metadata}, - io:put_chars(user, Formatter:format(Log,FormatConfig)), - {ok, State}; - false -> - {ok, State} - end; -handle_event(_Event, State) -> - {ok, State}. diff --git a/lib/kernel/test/logger_disk_log_h_SUITE.erl b/lib/kernel/test/logger_disk_log_h_SUITE.erl index 7a1736c814..a4b15c841b 100644 --- a/lib/kernel/test/logger_disk_log_h_SUITE.erl +++ b/lib/kernel/test/logger_disk_log_h_SUITE.erl @@ -1,3 +1,22 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2018. All Rights Reserved. +%% +%% Licensed under the Apache License, Version 2.0 (the "License"); +%% you may not use this file except in compliance with the License. +%% You may obtain a copy of the License at +%% +%% http://www.apache.org/licenses/LICENSE-2.0 +%% +%% Unless required by applicable law or agreed to in writing, software +%% distributed under the License is distributed on an "AS IS" BASIS, +%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +%% See the License for the specific language governing permissions and +%% limitations under the License. +%% +%% %CopyrightEnd% +%% -module(logger_disk_log_h_SUITE). -compile(export_all). @@ -26,10 +45,6 @@ -define(log_no(File,N), lists:concat([File,".",N])). -define(domain,#{domain=>[?MODULE]}). --define(SYNC_REP_INT, if is_atom(?FILESYNC_REPEAT_INTERVAL) -> 5500; - true -> ?FILESYNC_REPEAT_INTERVAL + 500 - end). - suite() -> [{timetrap,{seconds,30}}, {ct_hooks,[logger_test_lib]}]. @@ -50,9 +65,10 @@ end_per_group(_Group, _Config) -> init_per_testcase(TestHooksCase, Config) when TestHooksCase == write_failure; TestHooksCase == sync_failure -> - if ?TEST_HOOKS_TAB == undefined -> + case (fun() -> ?TEST_HOOKS_TAB == undefined end)() of + true -> {skip,"Define the TEST_HOOKS macro to run this test"}; - true -> + false -> ct:print("********** ~w **********", [TestHooksCase]), Config end; @@ -82,7 +98,7 @@ all() -> bad_input, info_and_reset, reconfig, - disk_log_sync, + sync, disk_log_full, disk_log_wrap, disk_log_events, @@ -107,10 +123,10 @@ start_stop_handler(_Config) -> ok = logger:add_handler(?MODULE, logger_disk_log_h, #{}), {error,{already_exist,?MODULE}} = logger:add_handler(?MODULE, logger_disk_log_h, #{}), - true = is_pid(whereis(?MODULE)), + true = is_pid(whereis(h_proc_name())), ok = logger:remove_handler(?MODULE), timer:sleep(500), - undefined = whereis(?MODULE). + undefined = whereis(h_proc_name()). start_stop_handler(cleanup, _Config) -> logger:remove_handler(?MODULE). @@ -123,8 +139,8 @@ create_log(Config) -> filters=>?DEFAULT_HANDLER_FILTERS([?MODULE]), formatter=>{?MODULE,self()}}, #{file=>LogFile1}), - logger:info("hello", ?domain), - logger_disk_log_h:disk_log_sync(Name1), + logger:notice("hello", ?domain), + logger_disk_log_h:filesync(Name1), ct:pal("Checking contents of ~p", [?log_no(LogFile1,1)]), try_read_file(?log_no(LogFile1,1), {ok,<<"hello\n">>}, 5000), @@ -136,8 +152,8 @@ create_log(Config) -> filters=>?DEFAULT_HANDLER_FILTERS([?MODULE]), formatter=>{?MODULE,self()}}, #{file=>LogFile2}), - logger:info("dummy", ?domain), - logger_disk_log_h:disk_log_sync(Name2), + logger:notice("dummy", ?domain), + logger_disk_log_h:filesync(Name2), ct:pal("Checking contents of ~p", [?log_no(LogFile2,1)]), try_read_file(?log_no(LogFile2,1), {ok,<<"dummy\n">>}, 5000), @@ -157,22 +173,22 @@ open_existing_log(Config) -> filters=>?DEFAULT_HANDLER_FILTERS([?MODULE]), formatter=>{?MODULE,self()}}, #{file=>LogFile1}), - logger:info("one", ?domain), - logger_disk_log_h:disk_log_sync(HName), + logger:notice("one", ?domain), + logger_disk_log_h:filesync(HName), ct:pal("Checking contents of ~p", [?log_no(LogFile1,1)]), try_read_file(?log_no(LogFile1,1), {ok,<<"one\n">>}, 5000), - logger:info("two", ?domain), + logger:notice("two", ?domain), ok = remove_and_stop(HName), try_read_file(?log_no(LogFile1,1), {ok,<<"one\ntwo\n">>}, 5000), - logger:info("two and a half", ?domain), + logger:notice("two and a half", ?domain), ok = start_and_add(HName, #{filter_default=>stop, filters=>?DEFAULT_HANDLER_FILTERS([?MODULE]), formatter=>{?MODULE,self()}}, #{file=>LogFile1}), - logger:info("three", ?domain), - logger_disk_log_h:disk_log_sync(HName), + logger:notice("three", ?domain), + logger_disk_log_h:filesync(HName), try_read_file(?log_no(LogFile1,1), {ok,<<"one\ntwo\nthree\n">>}, 5000), remove_and_stop(HName), try_read_file(?log_no(LogFile1,1), {ok,<<"one\ntwo\nthree\n">>}, 5000). @@ -196,23 +212,23 @@ disk_log_opts(Config) -> ct:log("Fullname = ~s", [WFileFull]), {WFileFull,wrap,{Size,2},1} = {Get(file,WInfo1),Get(type,WInfo1), Get(size,WInfo1),Get(current_file,WInfo1)}, - logger:info("123", ?domain), - logger_disk_log_h:disk_log_sync(WName), + logger:notice("123", ?domain), + logger_disk_log_h:filesync(WName), timer:sleep(500), 1 = Get(current_file, disk_log:info(WName)), - logger:info("45", ?domain), - logger_disk_log_h:disk_log_sync(WName), + logger:notice("45", ?domain), + logger_disk_log_h:filesync(WName), timer:sleep(500), 1 = Get(current_file, disk_log:info(WName)), - logger:info("6", ?domain), - logger_disk_log_h:disk_log_sync(WName), + logger:notice("6", ?domain), + logger_disk_log_h:filesync(WName), timer:sleep(500), 2 = Get(current_file, disk_log:info(WName)), - logger:info("7890", ?domain), - logger_disk_log_h:disk_log_sync(WName), + logger:notice("7890", ?domain), + logger_disk_log_h:filesync(WName), timer:sleep(500), 2 = Get(current_file, disk_log:info(WName)), @@ -229,8 +245,8 @@ disk_log_opts(Config) -> ct:log("Fullname = ~s", [HFile1Full]), {HFile1Full,halt,infinity} = {Get(file,HInfo1),Get(type,HInfo1), Get(size,HInfo1)}, - logger:info("12345", ?domain), - logger_disk_log_h:disk_log_sync(HName1), + logger:notice("12345", ?domain), + logger_disk_log_h:filesync(HName1), timer:sleep(500), 1 = Get(no_written_items, disk_log:info(HName1)), @@ -253,17 +269,17 @@ disk_log_opts(Config) -> default_formatter(Config) -> PrivDir = ?config(priv_dir,Config), LogFile = filename:join(PrivDir,atom_to_list(?FUNCTION_NAME)), - HConfig = #{disk_log_opts => #{file=>LogFile}, - filter_default=>log}, + HandlerConfig = #{config => #{file=>LogFile}, + filter_default=>log}, ct:pal("Log: ~p", [LogFile]), - ok = logger:add_handler(?MODULE, logger_disk_log_h, HConfig), + ok = logger:add_handler(?MODULE, logger_disk_log_h, HandlerConfig), ok = logger:set_handler_config(?MODULE,formatter, {?DEFAULT_FORMATTER,?DEFAULT_FORMAT_CONFIG}), LogName = lists:concat([LogFile, ".1"]), - logger:info("dummy"), + logger:notice("dummy"), wait_until_written(LogName), {ok,Bin} = file:read_file(LogName), - match = re:run(Bin, "=INFO REPORT====.*\ndummy", [{capture,none}]), + match = re:run(Bin, "=NOTICE REPORT====.*\ndummy", [{capture,none}]), ok. default_formatter(cleanup, _Config) -> logger:remove_handler(?MODULE). @@ -277,8 +293,8 @@ logging(Config) -> formatter=>{?MODULE,self()}}, #{file => LogFile}), MsgFormatter = fun(Term) -> {io_lib:format("Term:~p",[Term]),[]} end, - logger:info([{x,y}], #{report_cb => MsgFormatter}), - logger:info([{x,y}], #{}), + logger:notice([{x,y}], #{report_cb => MsgFormatter}), + logger:notice([{x,y}], #{}), ct:pal("Checking contents of ~p", [?log_no(LogFile,1)]), try_read_file(?log_no(LogFile,1), {ok,<<"Term:[{x,y}]\n x: y\n">>}, 5000). @@ -290,10 +306,10 @@ errors(Config) -> PrivDir = ?config(priv_dir,Config), Name1 = list_to_atom(lists:concat([?FUNCTION_NAME,"_1"])), LogFile1 = filename:join(PrivDir,Name1), - HConfig = #{disk_log_opts=>#{file=>LogFile1}, - filter_default=>log, - formatter=>{?MODULE,self()}}, - ok = logger:add_handler(Name1, logger_disk_log_h, HConfig), + HandlerConfig = #{config=>#{file=>LogFile1}, + filter_default=>log, + formatter=>{?MODULE,self()}}, + ok = logger:add_handler(Name1, logger_disk_log_h, HandlerConfig), {error,{already_exist,Name1}} = logger:add_handler(Name1, logger_disk_log_h, #{}), @@ -302,7 +318,7 @@ errors(Config) -> {error,{illegal_config_change,_,_}} = logger:set_handler_config(Name1, - disk_log_opts, + config, #{file=>LogFile1, type=>halt}), {error,{illegal_config_change,_,_}} = @@ -321,45 +337,42 @@ formatter_fail(Config) -> Name = ?FUNCTION_NAME, LogFile = filename:join(PrivDir,Name), ct:pal("Log = ~p", [LogFile]), - HConfig = #{disk_log_opts => #{file=>LogFile}, - filter_default=>stop, - filters=>?DEFAULT_HANDLER_FILTERS([?MODULE])}, + HandlerConfig = #{config => #{file=>LogFile}, + filter_default=>stop, + filters=>?DEFAULT_HANDLER_FILTERS([?MODULE])}, %% no formatter! - logger:add_handler(Name, logger_disk_log_h, HConfig), - Pid = whereis(Name), + logger:add_handler(Name, logger_disk_log_h, HandlerConfig), + Pid = whereis(h_proc_name(Name)), true = is_pid(Pid), - #{handlers:=HC1} = logger:i(), - H = [Id || {Id,_,_} <- HC1], + H = logger:get_handler_ids(), true = lists:member(Name,H), %% Formatter is added automatically - {ok,{_,#{formatter:={logger_formatter,_}}}} = - logger:get_handler_config(Name), - logger:info(M1=?msg,?domain), - Got1 = try_match_file(?log_no(LogFile,1),"[0-9\\+\\-T:\\.]* info: "++M1,5000), + {ok,#{formatter:={logger_formatter,_}}} = logger:get_handler_config(Name), + logger:notice(M1=?msg,?domain), + Got1 = try_match_file(?log_no(LogFile,1),"[0-9\\+\\-T:\\.]* notice: "++M1,5000), ok = logger:set_handler_config(Name,formatter,{nonexistingmodule,#{}}), - logger:info(M2=?msg,?domain), + logger:notice(M2=?msg,?domain), Got2 = try_match_file(?log_no(LogFile,1), - escape(Got1)++"[0-9\\+\\-T:\\.]* info: FORMATTER CRASH: .*"++M2, + escape(Got1)++"[0-9\\+\\-T:\\.]* notice: FORMATTER CRASH: .*"++M2, 5000), ok = logger:set_handler_config(Name,formatter,{?MODULE,crash}), - logger:info(M3=?msg,?domain), + logger:notice(M3=?msg,?domain), Got3 = try_match_file(?log_no(LogFile,1), - escape(Got2)++"[0-9\\+\\-T:\\.]* info: FORMATTER CRASH: .*"++M3, + escape(Got2)++"[0-9\\+\\-T:\\.]* notice: FORMATTER CRASH: .*"++M3, 5000), ok = logger:set_handler_config(Name,formatter,{?MODULE,bad_return}), - logger:info(?msg,?domain), + logger:notice(?msg,?domain), try_match_file(?log_no(LogFile,1), escape(Got3)++"FORMATTER ERROR: bad_return_value", 5000), %% Check that handler is still alive and was never dead - Pid = whereis(Name), - #{handlers:=HC2} = logger:i(), - H = [Id || {Id,_,_} <- HC2], + Pid = whereis(h_proc_name(Name)), + H = logger:get_handler_ids(), ok. formatter_fail(cleanup,_Config) -> @@ -369,48 +382,49 @@ formatter_fail(cleanup,_Config) -> config_fail(_Config) -> {error,{handler_not_added,{invalid_config,logger_disk_log_h,{bad,bad}}}} = logger:add_handler(?MODULE,logger_disk_log_h, - #{logger_disk_log_h => #{bad => bad}, + #{config => #{bad => bad}, filter_default=>log, formatter=>{?MODULE,self()}}), {error,{handler_not_added,{invalid_levels,{_,1,_}}}} = logger:add_handler(?MODULE,logger_disk_log_h, - #{logger_disk_log_h => #{drop_new_reqs_qlen=>1}}), + #{config => #{drop_mode_qlen=>1}}), {error,{handler_not_added,{invalid_levels,{43,42,_}}}} = logger:add_handler(?MODULE,logger_disk_log_h, - #{logger_disk_log_h => #{toggle_sync_qlen=>43, - drop_new_reqs_qlen=>42}}), + #{config => #{sync_mode_qlen=>43, + drop_mode_qlen=>42}}), {error,{handler_not_added,{invalid_levels,{_,43,42}}}} = logger:add_handler(?MODULE,logger_disk_log_h, - #{logger_disk_log_h => #{drop_new_reqs_qlen=>43, - flush_reqs_qlen=>42}}), + #{config => #{drop_mode_qlen=>43, + flush_qlen=>42}}), ok = logger:add_handler(?MODULE,logger_disk_log_h, #{filter_default=>log, formatter=>{?MODULE,self()}}), %% can't change the disk log options for a log already in use {error,{illegal_config_change,_,_}} = - logger:set_handler_config(?MODULE,disk_log_opts, + logger:set_handler_config(?MODULE,config, #{max_no_files=>2}), %% can't change name of an existing handler {error,{illegal_config_change,_,_}} = logger:set_handler_config(?MODULE,id,bad), - %% incorrect values of OP params + %% incorrect values of OP params + {ok,#{config := HConfig}} = logger:get_handler_config(?MODULE), {error,{invalid_levels,_}} = - logger:set_handler_config(?MODULE,logger_disk_log_h, - #{toggle_sync_qlen=>100, - flush_reqs_qlen=>99}), + logger:set_handler_config(?MODULE,config, + HConfig#{sync_mode_qlen=>100, + flush_qlen=>99}), %% invalid name of config parameter {error,{invalid_config,logger_disk_log_h,{filesync_rep_int,2000}}} = - logger:set_handler_config(?MODULE, logger_disk_log_h, - #{filesync_rep_int => 2000}), + logger:set_handler_config(?MODULE, config, + HConfig#{filesync_rep_int => 2000}), ok. config_fail(cleanup,_Config) -> logger:remove_handler(?MODULE). bad_input(_Config) -> - {error,{badarg,{disk_log_sync,["BadType"]}}} = - logger_disk_log_h:disk_log_sync("BadType"), + {error,{badarg,{filesync,["BadType"]}}} = + logger_disk_log_h:filesync("BadType"), {error,{badarg,{info,["BadType"]}}} = logger_disk_log_h:info("BadType"), {error,{badarg,{reset,["BadType"]}}} = logger_disk_log_h:reset("BadType"). @@ -431,16 +445,16 @@ reconfig(Config) -> filters=>?DEFAULT_HANDLER_FILTERS([?MODULE]), formatter=>{?MODULE,self()}}), #{id := ?MODULE, - toggle_sync_qlen := ?TOGGLE_SYNC_QLEN, - drop_new_reqs_qlen := ?DROP_NEW_REQS_QLEN, - flush_reqs_qlen := ?FLUSH_REQS_QLEN, - enable_burst_limit := ?ENABLE_BURST_LIMIT, - burst_limit_size := ?BURST_LIMIT_SIZE, - burst_window_time := ?BURST_WINDOW_TIME, - enable_kill_overloaded := ?ENABLE_KILL_OVERLOADED, - handler_overloaded_qlen := ?HANDLER_OVERLOADED_QLEN, - handler_overloaded_mem := ?HANDLER_OVERLOADED_MEM, - handler_restart_after := ?HANDLER_RESTART_AFTER, + sync_mode_qlen := ?SYNC_MODE_QLEN, + drop_mode_qlen := ?DROP_MODE_QLEN, + flush_qlen := ?FLUSH_QLEN, + burst_limit_enable := ?BURST_LIMIT_ENABLE, + burst_limit_max_count := ?BURST_LIMIT_MAX_COUNT, + burst_limit_window_time := ?BURST_LIMIT_WINDOW_TIME, + overload_kill_enable := ?OVERLOAD_KILL_ENABLE, + overload_kill_qlen := ?OVERLOAD_KILL_QLEN, + overload_kill_mem_size := ?OVERLOAD_KILL_MEM_SIZE, + overload_kill_restart_after := ?OVERLOAD_KILL_RESTART_AFTER, filesync_repeat_interval := ?FILESYNC_REPEAT_INTERVAL, log_opts := #{type := ?DISK_LOG_TYPE, max_no_files := ?DISK_LOG_MAX_NO_FILES, @@ -448,29 +462,30 @@ reconfig(Config) -> file := _DiskLogFile}} = logger_disk_log_h:info(?MODULE), - ok = logger:set_handler_config(?MODULE, logger_disk_log_h, - #{toggle_sync_qlen => 1, - drop_new_reqs_qlen => 2, - flush_reqs_qlen => 3, - enable_burst_limit => false, - burst_limit_size => 10, - burst_window_time => 10, - enable_kill_overloaded => true, - handler_overloaded_qlen => 100000, - handler_overloaded_mem => 10000000, - handler_restart_after => never, - filesync_repeat_interval => no_repeat}), + {ok,#{config := HConfig0}} = logger:get_handler_config(?MODULE), + HConfig1 = HConfig0#{sync_mode_qlen => 1, + drop_mode_qlen => 2, + flush_qlen => 3, + burst_limit_enable => false, + burst_limit_max_count => 10, + burst_limit_window_time => 10, + overload_kill_enable => true, + overload_kill_qlen => 100000, + overload_kill_mem_size => 10000000, + overload_kill_restart_after => infinity, + filesync_repeat_interval => no_repeat}, + ok = logger:set_handler_config(?MODULE, config, HConfig1), #{id := ?MODULE, - toggle_sync_qlen := 1, - drop_new_reqs_qlen := 2, - flush_reqs_qlen := 3, - enable_burst_limit := false, - burst_limit_size := 10, - burst_window_time := 10, - enable_kill_overloaded := true, - handler_overloaded_qlen := 100000, - handler_overloaded_mem := 10000000, - handler_restart_after := never, + sync_mode_qlen := 1, + drop_mode_qlen := 2, + flush_qlen := 3, + burst_limit_enable := false, + burst_limit_max_count := 10, + burst_limit_window_time := 10, + overload_kill_enable := true, + overload_kill_qlen := 100000, + overload_kill_mem_size := 10000000, + overload_kill_restart_after := infinity, filesync_repeat_interval := no_repeat} = logger_disk_log_h:info(?MODULE), @@ -482,7 +497,7 @@ reconfig(Config) -> #{filter_default=>log, filters=>?DEFAULT_HANDLER_FILTERS([?MODULE]), formatter=>{?MODULE,self()}, - disk_log_opts=> + config=> #{type => halt, max_no_files => 1, max_no_bytes => 1024, @@ -497,13 +512,13 @@ reconfig(Config) -> reconfig(cleanup, _Config) -> logger:remove_handler(?MODULE). -disk_log_sync(Config) -> +sync(Config) -> Dir = ?config(priv_dir,Config), File = filename:join(Dir, ?FUNCTION_NAME), Log = lists:concat([File,".1"]), ok = logger:add_handler(?MODULE, logger_disk_log_h, - #{disk_log_opts => #{file => File}, + #{config => #{file => File}, filter_default=>log, filters=>?DEFAULT_HANDLER_FILTERS([?MODULE]), formatter=>{?MODULE,nl}}), @@ -513,7 +528,7 @@ disk_log_sync(Config) -> [{disk_log,blog,<<"first\n">>}, {disk_log,sync}]), - logger:info("first", ?domain), + logger:notice("first", ?domain), %% wait for automatic disk_log_sync check_tracer(?FILESYNC_REPEAT_INTERVAL*2), @@ -524,16 +539,18 @@ disk_log_sync(Config) -> {disk_log,sync}]), %% two log requests in fast succession will make the handler skip %% an automatic disk log sync - logger:info("second", ?domain), - logger:info("third", ?domain), - %% do explicit disk_log_sync - logger_disk_log_h:disk_log_sync(?MODULE), + logger:notice("second", ?domain), + logger:notice("third", ?domain), + %% do explicit sync + logger_disk_log_h:filesync(?MODULE), check_tracer(100), %% check that if there's no repeated disk_log_sync active, %% a disk_log_sync is still performed when handler goes idle - logger:set_handler_config(?MODULE, logger_disk_log_h, - #{filesync_repeat_interval => no_repeat}), + {ok,#{config := HConfig}} = logger:get_handler_config(?MODULE), + HConfig1 = HConfig#{filesync_repeat_interval => no_repeat}, + ok = logger:set_handler_config(?MODULE, config, HConfig1), + no_repeat = maps:get(filesync_repeat_interval, logger_disk_log_h:info(?MODULE)), @@ -543,9 +560,9 @@ disk_log_sync(Config) -> {disk_log,blog,<<"fifth\n">>}, {disk_log,sync}]), - logger:info("fourth", ?domain), + logger:notice("fourth", ?domain), timer:sleep(?IDLE_DETECT_TIME_MSEC*2), - logger:info("fifth", ?domain), + logger:notice("fifth", ?domain), %% wait for automatic disk_log_sync check_tracer(?IDLE_DETECT_TIME_MSEC*2), @@ -559,16 +576,17 @@ disk_log_sync(Config) -> start_tracer([{logger_disk_log_h,handle_cast,2}], [OneSync || _ <- lists:seq(1, 1 + trunc(WaitT/SyncInt))]), - logger:set_handler_config(?MODULE, logger_disk_log_h, - #{filesync_repeat_interval => SyncInt}), + HConfig2 = HConfig#{filesync_repeat_interval => SyncInt}, + ok = logger:set_handler_config(?MODULE, config, HConfig2), + SyncInt = maps:get(filesync_repeat_interval, logger_disk_log_h:info(?MODULE)), timer:sleep(WaitT), - logger:set_handler_config(?MODULE, logger_disk_log_h, - #{filesync_repeat_interval => no_repeat}), + HConfig3 = HConfig#{filesync_repeat_interval => no_repeat}, + ok = logger:set_handler_config(?MODULE, config, HConfig3), check_tracer(100), ok. -disk_log_sync(cleanup,_Config) -> +sync(cleanup,_Config) -> dbg:stop_clear(), logger:remove_handler(?MODULE). @@ -584,7 +602,7 @@ disk_log_wrap(Config) -> #{filter_default=>log, filters=>?DEFAULT_HANDLER_FILTERS([?MODULE]), formatter=>{?MODULE,self()}, - disk_log_opts=> + config=> #{type => wrap, max_no_files => MaxFiles, max_no_bytes => MaxBytes, @@ -598,7 +616,7 @@ disk_log_wrap(Config) -> Pid end, {ok,_} = dbg:tracer(process, {TraceFun, Tester}), - {ok,_} = dbg:p(whereis(?MODULE), [c]), + {ok,_} = dbg:p(whereis(h_proc_name()), [c]), {ok,_} = dbg:tp(logger_disk_log_h, handle_info, 2, []), Text = [34 + rand:uniform(126-34) || _ <- lists:seq(1,MaxBytes)], @@ -606,7 +624,7 @@ disk_log_wrap(Config) -> %% fill first file lists:foreach(fun(N) -> Log = lists:concat([File,".",N]), - logger:info(Text, ?domain), + logger:notice(Text, ?domain), wait_until_written(Log), ct:pal("N = ~w", [N = Get(current_file, @@ -640,7 +658,7 @@ disk_log_full(Config) -> #{filter_default=>log, filters=>?DEFAULT_HANDLER_FILTERS([?MODULE]), formatter=>{?MODULE,self()}, - disk_log_opts=> + config=> #{type => halt, max_no_files => 1, max_no_bytes => MaxBytes, @@ -652,12 +670,12 @@ disk_log_full(Config) -> Pid end, {ok,_} = dbg:tracer(process, {TraceFun, Tester}), - {ok,_} = dbg:p(whereis(?MODULE), [c]), + {ok,_} = dbg:p(whereis(h_proc_name()), [c]), {ok,_} = dbg:tp(logger_disk_log_h, handle_info, 2, []), NoOfChars = 5, Text = [34 + rand:uniform(126-34) || _ <- lists:seq(1,NoOfChars)], - [logger:info(Text, ?domain) || _ <- lists:seq(1,trunc(MaxBytes/NoOfChars)+1)], + [logger:notice(Text, ?domain) || _ <- lists:seq(1,trunc(MaxBytes/NoOfChars)+1)], %% wait for trace messages timer:sleep(2000), @@ -701,10 +719,10 @@ disk_log_events(Config) -> Pid end, {ok,_} = dbg:tracer(process, {TraceFun, Tester}), - {ok,_} = dbg:p(whereis(?MODULE), [c]), + {ok,_} = dbg:p(whereis(h_proc_name()), [c]), {ok,_} = dbg:tp(logger_disk_log_h, handle_info, 2, []), - [whereis(?MODULE) ! E || E <- Events], + [whereis(h_proc_name()) ! E || E <- Events], %% wait for trace messages timer:sleep(2000), dbg:stop_clear(), @@ -737,9 +755,15 @@ write_failure(Config) -> ct:pal("LogOpts = ~p", [LogOpts = maps:get(log_opts, HState)]), ok = log_on_remote_node(Node, "Logged1"), - rpc:call(Node, logger_disk_log_h, disk_log_sync, [?STANDARD_HANDLER]), + rpc:call(Node, logger_disk_log_h, filesync, [?STANDARD_HANDLER]), ?check_no_log, - try_read_file(Log, {ok,<<"Logged1\n">>}, ?SYNC_REP_INT), + + SyncRepInt = case (fun() -> is_atom(?FILESYNC_REPEAT_INTERVAL) end)() of + true -> 5500; + false -> ?FILESYNC_REPEAT_INTERVAL + 500 + end, + + try_read_file(Log, {ok,<<"Logged1\n">>}, SyncRepInt), rpc:call(Node, ?MODULE, set_result, [disk_log_blog,{error,no_such_log}]), ok = log_on_remote_node(Node, "Cause simple error printout"), @@ -757,9 +781,9 @@ write_failure(Config) -> rpc:call(Node, ?MODULE, set_result, [disk_log_blog,ok]), ok = log_on_remote_node(Node, "Logged2"), - rpc:call(Node, logger_disk_log_h, disk_log_sync, [?STANDARD_HANDLER]), + rpc:call(Node, logger_disk_log_h, filesync, [?STANDARD_HANDLER]), ?check_no_log, - try_read_file(Log, {ok,<<"Logged1\nLogged2\n">>}, ?SYNC_REP_INT), + try_read_file(Log, {ok,<<"Logged1\nLogged2\n">>}, SyncRepInt), ok. write_failure(cleanup, _Config) -> Nodes = nodes(), @@ -782,7 +806,7 @@ sync_failure(Config) -> SyncInt = 500, ok = rpc:call(Node, logger, set_handler_config, - [?STANDARD_HANDLER, logger_disk_log_h, + [?STANDARD_HANDLER, config, #{filesync_repeat_interval => SyncInt}]), Info = rpc:call(Node, logger_disk_log_h, info, [?STANDARD_HANDLER]), SyncInt = maps:get(filesync_repeat_interval, Info), @@ -793,7 +817,7 @@ sync_failure(Config) -> rpc:call(Node, ?MODULE, set_result, [disk_log_sync,{error,no_such_log}]), ok = log_on_remote_node(Node, "Cause simple error printout"), - ?check({error,{?STANDARD_HANDLER,sync,LogOpts,{error,no_such_log}}}), + ?check({error,{?STANDARD_HANDLER,filesync,LogOpts,{error,no_such_log}}}), ok = log_on_remote_node(Node, "No second error printout"), ?check_no_log, @@ -801,7 +825,7 @@ sync_failure(Config) -> rpc:call(Node, ?MODULE, set_result, [disk_log_sync,{error,{blocked_log,?STANDARD_HANDLER}}]), ok = log_on_remote_node(Node, "Cause simple error printout"), - ?check({error,{?STANDARD_HANDLER,sync,LogOpts, + ?check({error,{?STANDARD_HANDLER,filesync,LogOpts, {error,{blocked_log,?STANDARD_HANDLER}}}}), rpc:call(Node, ?MODULE, set_result, [disk_log_sync,ok]), @@ -817,7 +841,7 @@ start_h_on_new_node(Config, File) -> logger_test_lib:setup( Config, [{logger,[{handler,default,logger_disk_log_h, - #{ disk_log_opts => #{ file => File }}}]}]), + #{ config => #{ file => File }}}]}]), ok = rpc:call(Node,logger,set_handler_config,[?STANDARD_HANDLER,formatter, {?MODULE,nl}]), Node. @@ -825,7 +849,7 @@ start_h_on_new_node(Config, File) -> log_on_remote_node(Node,Msg) -> _ = spawn_link(Node, fun() -> erlang:group_leader(whereis(user),self()), - logger:info(Msg) + logger:notice(Msg) end), ok. @@ -852,15 +876,15 @@ op_switch_to_sync(Config) -> {Log,HConfig,DLHConfig} = start_handler(?MODULE, ?FUNCTION_NAME, Config), NumOfReqs = 500, NewHConfig = - HConfig#{logger_disk_log_h => DLHConfig#{toggle_sync_qlen => 2, - drop_new_reqs_qlen => NumOfReqs+1, - flush_reqs_qlen => 2*NumOfReqs, - enable_burst_limit => false}}, + HConfig#{config => DLHConfig#{sync_mode_qlen => 2, + drop_mode_qlen => NumOfReqs+1, + flush_qlen => 2*NumOfReqs, + burst_limit_enable => false}}, ok = logger:set_handler_config(?MODULE, NewHConfig), - send_burst({n,NumOfReqs}, seq, {chars,79}, info), + send_burst({n,NumOfReqs}, seq, {chars,79}, notice), Lines = count_lines(Log), - ok = file:delete(Log), NumOfReqs = Lines, + ok = file_delete(Log), ok. op_switch_to_sync(cleanup, _Config) -> ok = stop_handler(?MODULE). @@ -876,25 +900,25 @@ op_switch_to_drop(Config) -> Procs = 2, Bursts = 10, NewHConfig = - HConfig#{logger_disk_log_h => - DLHConfig#{toggle_sync_qlen => 1, - drop_new_reqs_qlen => 2, - flush_reqs_qlen => Procs*NumOfReqs*Bursts, - enable_burst_limit => false}}, + HConfig#{config => + DLHConfig#{sync_mode_qlen => 1, + drop_mode_qlen => 2, + flush_qlen => Procs*NumOfReqs*Bursts, + burst_limit_enable => false}}, ok = logger:set_handler_config(?MODULE, NewHConfig), %% It sometimes happens that the handler either gets %% the requests in a slow enough pace so that dropping %% never occurs. Therefore, lets generate a number of %% bursts to increase the chance of message buildup. - [send_burst({n,NumOfReqs}, {spawn,Procs,0}, {chars,79}, info) || + [send_burst({n,NumOfReqs}, {spawn,Procs,0}, {chars,79}, notice) || _ <- lists:seq(1, Bursts)], Logged = count_lines(Log), - ok= stop_handler(?MODULE), - _ = file:delete(Log), + ok = stop_handler(?MODULE), ct:pal("Number of messages dropped = ~w (~w)", [Procs*NumOfReqs*Bursts-Logged,Procs*NumOfReqs*Bursts]), true = (Logged < (Procs*NumOfReqs*Bursts)), true = (Logged > 0), + _ = file_delete(Log), ok end, %% As it's tricky to get the timing right in only one go, we perform the @@ -921,12 +945,12 @@ op_switch_to_flush(Config) -> %% (verify with coverage of flush_log_requests/2) NewHConfig = - HConfig#{logger_disk_log_h => - DLHConfig#{toggle_sync_qlen => 2, + HConfig#{config => + DLHConfig#{sync_mode_qlen => 2, %% disable drop mode - drop_new_reqs_qlen => 300, - flush_reqs_qlen => 300, - enable_burst_limit => false}}, + drop_mode_qlen => 300, + flush_qlen => 300, + burst_limit_enable => false}}, ok = logger:set_handler_config(?MODULE, NewHConfig), NumOfReqs = 1500, Procs = 10, @@ -938,15 +962,15 @@ op_switch_to_flush(Config) -> %% sync messages gets tested). Therefore, lets %% generate a number of bursts to increase the chance %% of message buildup in some random fashion. - [send_burst({n,NumOfReqs}, {spawn,Procs,0}, {chars,79}, info) || + [send_burst({n,NumOfReqs}, {spawn,Procs,0}, {chars,79}, notice) || _ <- lists:seq(1,Bursts)], Logged = count_lines(Log), ok= stop_handler(?MODULE), - _ = file:delete(Log), ct:pal("Number of messages flushed/dropped = ~w (~w)", [NumOfReqs*Procs*Bursts-Logged,NumOfReqs*Procs*Bursts]), true = (Logged < (NumOfReqs*Procs*Bursts)), true = (Logged > 0), + _ = file_delete(Log), ok end, %% As it's tricky to get the timing right in only one go, we perform the @@ -964,18 +988,19 @@ op_switch_to_flush(cleanup, _Config) -> limit_burst_disabled(Config) -> {Log,HConfig,DLHConfig} = start_handler(?MODULE, ?FUNCTION_NAME, Config), NewHConfig = - HConfig#{logger_disk_log_h => DLHConfig#{enable_burst_limit => false, - burst_limit_size => 10, - burst_window_time => 2000, - drop_new_reqs_qlen => 200, - flush_reqs_qlen => 300}}, + HConfig#{config => DLHConfig#{burst_limit_enable => false, + burst_limit_max_count => 10, + burst_limit_window_time => 2000, + drop_mode_qlen => 200, + flush_qlen => 300}}, ok = logger:set_handler_config(?MODULE, NewHConfig), NumOfReqs = 100, - send_burst({n,NumOfReqs}, seq, {chars,79}, info), + send_burst({n,NumOfReqs}, seq, {chars,79}, notice), Logged = count_lines(Log), ct:pal("Number of messages logged = ~w", [Logged]), - ok = file:delete(Log), - NumOfReqs = Logged. + NumOfReqs = Logged, + ok = file_delete(Log), + ok. limit_burst_disabled(cleanup, _Config) -> ok = stop_handler(?MODULE). @@ -983,18 +1008,19 @@ limit_burst_enabled_one(Config) -> {Log,HConfig,DLHConfig} = start_handler(?MODULE, ?FUNCTION_NAME, Config), ReqLimit = 10, NewHConfig = - HConfig#{logger_disk_log_h => DLHConfig#{enable_burst_limit => true, - burst_limit_size => ReqLimit, - burst_window_time => 2000, - drop_new_reqs_qlen => 200, - flush_reqs_qlen => 300}}, + HConfig#{config => DLHConfig#{burst_limit_enable => true, + burst_limit_max_count => ReqLimit, + burst_limit_window_time => 2000, + drop_mode_qlen => 200, + flush_qlen => 300}}, ok = logger:set_handler_config(?MODULE, NewHConfig), NumOfReqs = 100, - send_burst({n,NumOfReqs}, seq, {chars,79}, info), + send_burst({n,NumOfReqs}, seq, {chars,79}, notice), Logged = count_lines(Log), ct:pal("Number of messages logged = ~w", [Logged]), - ok = file:delete(Log), - ReqLimit = Logged. + ReqLimit = Logged, + ok = file_delete(Log), + ok. limit_burst_enabled_one(cleanup, _Config) -> ok = stop_handler(?MODULE). @@ -1003,58 +1029,59 @@ limit_burst_enabled_period(Config) -> ReqLimit = 10, BurstTWin = 1000, NewHConfig = - HConfig#{logger_disk_log_h => DLHConfig#{enable_burst_limit => true, - burst_limit_size => ReqLimit, - burst_window_time => BurstTWin, - drop_new_reqs_qlen => 20000, - flush_reqs_qlen => 20001}}, + HConfig#{config => DLHConfig#{burst_limit_enable => true, + burst_limit_max_count => ReqLimit, + burst_limit_window_time => BurstTWin, + drop_mode_qlen => 20000, + flush_qlen => 20001}}, ok = logger:set_handler_config(?MODULE, NewHConfig), Windows = 3, - Sent = send_burst({t,BurstTWin*Windows}, seq, {chars,79}, info), + Sent = send_burst({t,BurstTWin*Windows}, seq, {chars,79}, notice), Logged = count_lines(Log), ct:pal("Number of messages sent = ~w~nNumber of messages logged = ~w", [Sent,Logged]), - ok = file:delete(Log), true = (Logged > (ReqLimit*Windows)) andalso - (Logged < (ReqLimit*(Windows+2))). + (Logged < (ReqLimit*(Windows+2))), + ok = file_delete(Log), + ok. limit_burst_enabled_period(cleanup, _Config) -> ok = stop_handler(?MODULE). kill_disabled(Config) -> {Log,HConfig,DLHConfig} = start_handler(?MODULE, ?FUNCTION_NAME, Config), NewHConfig = - HConfig#{logger_disk_log_h=>DLHConfig#{enable_kill_overloaded=>false, - handler_overloaded_qlen=>10, - handler_overloaded_mem=>100}}, + HConfig#{config=>DLHConfig#{overload_kill_enable=>false, + overload_kill_qlen=>10, + overload_kill_mem_size=>100}}, ok = logger:set_handler_config(?MODULE, NewHConfig), NumOfReqs = 100, - send_burst({n,NumOfReqs}, seq, {chars,79}, info), + send_burst({n,NumOfReqs}, seq, {chars,79}, notice), Logged = count_lines(Log), ct:pal("Number of messages logged = ~w", [Logged]), - ok = file:delete(Log), - true = is_pid(whereis(?MODULE)), + ok = file_delete(Log), + true = is_pid(whereis(h_proc_name())), ok. kill_disabled(cleanup, _Config) -> ok = stop_handler(?MODULE). qlen_kill_new(Config) -> - {_Log,HConfig,DLHConfig} = start_handler(?MODULE, ?FUNCTION_NAME, Config), - Pid0 = whereis(?MODULE), + {Log,HConfig,DLHConfig} = start_handler(?MODULE, ?FUNCTION_NAME, Config), + Pid0 = whereis(h_proc_name()), {_,Mem0} = process_info(Pid0, memory), - RestartAfter = ?HANDLER_RESTART_AFTER, + RestartAfter = ?OVERLOAD_KILL_RESTART_AFTER, NewHConfig = - HConfig#{logger_disk_log_h => - DLHConfig#{enable_kill_overloaded=>true, - handler_overloaded_qlen=>10, - handler_overloaded_mem=>Mem0+50000, - handler_restart_after=>RestartAfter}}, + HConfig#{config => + DLHConfig#{overload_kill_enable=>true, + overload_kill_qlen=>10, + overload_kill_mem_size=>Mem0+50000, + overload_kill_restart_after=>RestartAfter}}, ok = logger:set_handler_config(?MODULE, NewHConfig), MRef = erlang:monitor(process, Pid0), NumOfReqs = 100, - Procs = 2, - send_burst({n,NumOfReqs}, {spawn,Procs,0}, {chars,79}, info), - %% send_burst({n,NumOfReqs}, seq, {chars,79}, info), + Procs = 4, + send_burst({n,NumOfReqs}, {spawn,Procs,0}, {chars,79}, notice), + %% send_burst({n,NumOfReqs}, seq, {chars,79}, notice), receive {'DOWN', MRef, _, _, Info} -> case Info of @@ -1063,8 +1090,8 @@ qlen_kill_new(Config) -> killed -> ct:pal("Slow shutdown, handler process was killed!", []) end, - timer:sleep(RestartAfter + 2000), - true = is_pid(whereis(?MODULE)), + file_delete(Log), + {ok,_} = wait_for_process_up(RestartAfter * 3), ok after 5000 -> @@ -1076,22 +1103,22 @@ qlen_kill_new(cleanup, _Config) -> ok = stop_handler(?MODULE). mem_kill_new(Config) -> - {_Log,HConfig,DLHConfig} = start_handler(?MODULE, ?FUNCTION_NAME, Config), - Pid0 = whereis(?MODULE), + {Log,HConfig,DLHConfig} = start_handler(?MODULE, ?FUNCTION_NAME, Config), + Pid0 = whereis(h_proc_name()), {_,Mem0} = process_info(Pid0, memory), - RestartAfter = ?HANDLER_RESTART_AFTER, + RestartAfter = ?OVERLOAD_KILL_RESTART_AFTER, NewHConfig = - HConfig#{logger_disk_log_h => - DLHConfig#{enable_kill_overloaded=>true, - handler_overloaded_qlen=>50000, - handler_overloaded_mem=>Mem0+500, - handler_restart_after=>RestartAfter}}, + HConfig#{config => + DLHConfig#{overload_kill_enable=>true, + overload_kill_qlen=>50000, + overload_kill_mem_size=>Mem0+500, + overload_kill_restart_after=>RestartAfter}}, ok = logger:set_handler_config(?MODULE, NewHConfig), MRef = erlang:monitor(process, Pid0), NumOfReqs = 100, - Procs = 2, - send_burst({n,NumOfReqs}, {spawn,Procs,0}, {chars,79}, info), - %% send_burst({n,NumOfReqs}, seq, {chars,79}, info), + Procs = 4, + send_burst({n,NumOfReqs}, {spawn,Procs,0}, {chars,79}, notice), + %% send_burst({n,NumOfReqs}, seq, {chars,79}, notice), receive {'DOWN', MRef, _, _, Info} -> case Info of @@ -1100,8 +1127,8 @@ mem_kill_new(Config) -> killed -> ct:pal("Slow shutdown, handler process was killed!", []) end, - timer:sleep(RestartAfter + 2000), - true = is_pid(whereis(?MODULE)), + file_delete(Log), + {ok,_} = wait_for_process_up(RestartAfter * 3), ok after 5000 -> @@ -1112,53 +1139,58 @@ mem_kill_new(Config) -> mem_kill_new(cleanup, _Config) -> ok = stop_handler(?MODULE). +restart_after() -> + [{timetrap,{minutes,2}}]. restart_after(Config) -> {Log,HConfig,DLHConfig} = start_handler(?MODULE, ?FUNCTION_NAME, Config), NewHConfig1 = - HConfig#{logger_disk_log_h=>DLHConfig#{enable_kill_overloaded=>true, - handler_overloaded_qlen=>10, - handler_restart_after=>never}}, + HConfig#{config=>DLHConfig#{overload_kill_enable=>true, + overload_kill_qlen=>10, + overload_kill_restart_after=>infinity}}, ok = logger:set_handler_config(?MODULE, NewHConfig1), - MRef1 = erlang:monitor(process, whereis(?MODULE)), + MRef1 = erlang:monitor(process, whereis(h_proc_name())), %% kill handler - send_burst({n,100}, {spawn,2,0}, {chars,79}, info), + send_burst({n,100}, {spawn,4,0}, {chars,79}, notice), receive - {'DOWN', MRef1, _, _, _Info1} -> - timer:sleep(?HANDLER_RESTART_AFTER + 1000), - undefined = whereis(?MODULE), + {'DOWN', MRef1, _, _, _Reason1} -> + file_delete(Log), + error = wait_for_process_up(?OVERLOAD_KILL_RESTART_AFTER * 3), ok after 5000 -> + Info1 = logger_std_h:info(?MODULE), + ct:pal("Handler state = ~p", [Info1]), ct:fail("Handler not dead! It should not have survived this!") end, {Log,_,_} = start_handler(?MODULE, ?FUNCTION_NAME, Config), - RestartAfter = ?HANDLER_RESTART_AFTER, + RestartAfter = ?OVERLOAD_KILL_RESTART_AFTER, NewHConfig2 = - HConfig#{logger_disk_log_h=>DLHConfig#{enable_kill_overloaded=>true, - handler_overloaded_qlen=>10, - handler_restart_after=>RestartAfter}}, + HConfig#{config=>DLHConfig#{overload_kill_enable=>true, + overload_kill_qlen=>10, + overload_kill_restart_after=>RestartAfter}}, ok = logger:set_handler_config(?MODULE, NewHConfig2), - Pid0 = whereis(?MODULE), + Pid0 = whereis(h_proc_name()), MRef2 = erlang:monitor(process, Pid0), %% kill handler - send_burst({n,100}, {spawn,2,0}, {chars,79}, info), + send_burst({n,100}, {spawn,4,0}, {chars,79}, notice), receive - {'DOWN', MRef2, _, _, _Info2} -> - timer:sleep(RestartAfter + 2000), - Pid1 = whereis(?MODULE), - true = is_pid(Pid1), + {'DOWN', MRef2, _, _, _Reason2} -> + file_delete(Log), + {ok,Pid1} = wait_for_process_up(RestartAfter * 3), false = (Pid1 == Pid0), ok after 5000 -> + Info2 = logger_std_h:info(?MODULE), + ct:pal("Handler state = ~p", [Info2]), ct:fail("Handler not dead! It should not have survived this!") end, ok. restart_after(cleanup, _Config) -> ok = stop_handler(?MODULE). -%% send handler requests (filesync, info, reset, change_config) +%% send handler requests (sync, info, reset, change_config) %% during high load to verify that sync, dropping and flushing is %% handled correctly. handler_requests_under_load() -> @@ -1166,18 +1198,18 @@ handler_requests_under_load() -> handler_requests_under_load(Config) -> {Log,HConfig,DLHConfig} = start_handler(?MODULE, ?FUNCTION_NAME, Config), NewHConfig = - HConfig#{logger_disk_log_h => DLHConfig#{toggle_sync_qlen => 2, - drop_new_reqs_qlen => 1000, - flush_reqs_qlen => 2000, - enable_burst_limit => false}}, + HConfig#{config => DLHConfig#{sync_mode_qlen => 2, + drop_mode_qlen => 1000, + flush_qlen => 2000, + burst_limit_enable => false}}, ok = logger:set_handler_config(?MODULE, NewHConfig), - Pid = spawn_link(fun() -> send_requests(?MODULE, 1, [{disk_log_sync,[]}, + Pid = spawn_link(fun() -> send_requests(?MODULE, 1, [{filesync,[]}, {info,[]}, {reset,[]}, {change_config,[]}]) end), Procs = 100, - Sent = Procs * send_burst({n,5000}, {spawn,Procs,10}, {chars,79}, info), + Sent = Procs * send_burst({n,5000}, {spawn,Procs,10}, {chars,79}, notice), Pid ! {self(),finish}, ReqResult = receive {Pid,Result} -> Result end, Logged = count_lines(Log), @@ -1190,7 +1222,7 @@ handler_requests_under_load(Config) -> Errors = [{Req,FindError(Res)} || {Req,Res} <- ReqResult], NoOfReqs = lists:foldl(fun({_,Res}, N) -> N + length(Res) end, 0, ReqResult), ct:pal("~w requests made. Errors: ~n~p", [NoOfReqs,Errors]), - ok = file:delete(Log). + ok = file_delete(Log). handler_requests_under_load(cleanup, _Config) -> ok = stop_handler(?MODULE). @@ -1204,7 +1236,7 @@ send_requests(HName, TO, Reqs = [{Req,Res}|Rs]) -> case Req of change_config -> logger:set_handler_config(HName, logger_disk_log_h, - #{enable_kill_overloaded => + #{overload_kill_enable => false}); Func -> logger_disk_log_h:Func(HName) @@ -1218,21 +1250,22 @@ start_handler(Name, FuncName, Config) -> Dir = ?config(priv_dir,Config), File = filename:join(Dir, FuncName), ct:pal("Logging to ~tp", [File]), + FullFile = lists:concat([File,".1"]), + _ = file_delete(FullFile), ok = logger:add_handler(Name, logger_disk_log_h, - #{disk_log_opts=>#{file => File, - max_no_files => 1, - max_no_bytes => 100000000}, + #{config=>#{file => File, + max_no_files => 1, + max_no_bytes => 100000000}, filter_default=>log, filters=>?DEFAULT_HANDLER_FILTERS([Name]), formatter=>{?MODULE,op}}), - {ok,{_,HConfig = #{logger_disk_log_h := DLHConfig}}} = - logger:get_handler_config(Name), - {lists:concat([File,".1"]),HConfig,DLHConfig}. + {ok,HConfig = #{config := DLHConfig}} = logger:get_handler_config(Name), + {FullFile,HConfig,DLHConfig}. stop_handler(Name) -> - ok = logger:remove_handler(Name), - ct:pal("Handler ~p stopped!", [Name]). + ct:pal("Stopping handler ~p!", [Name]), + logger:remove_handler(Name). send_burst(NorT, Type, {chars,Sz}, Class) -> Text = [34 + rand:uniform(126-34) || _ <- lists:seq(1,Sz)], @@ -1255,12 +1288,9 @@ send_n_burst(N, seq, Text, Class) -> send_n_burst(N-1, seq, Text, Class); send_n_burst(N, {spawn,Ps,TO}, Text, Class) -> ct:pal("~w processes each sending ~w messages", [Ps,N]), - PerProc = fun() -> - send_n_burst(N, seq, Text, Class) - end, MRefs = [begin if TO == 0 -> ok; true -> timer:sleep(TO) end, - monitor(process,spawn_link(PerProc)) end || - _ <- lists:seq(1,Ps)], + monitor(process,spawn_link(per_proc_fun(N,Text,Class,X))) + end || X <- lists:seq(1,Ps)], lists:foreach(fun(MRef) -> receive {'DOWN', MRef, _, _, _} -> @@ -1279,6 +1309,16 @@ send_t_burst(T0, T, Text, Class, N) -> send_t_burst(T0, T, Text, Class, N+1) end. +per_proc_fun(N,Text,Class,X) when X rem 2 == 0 -> + fun() -> + process_flag(priority,high), + send_n_burst(N, seq, Text, Class) + end; +per_proc_fun(N,Text,Class,_) -> + fun() -> + send_n_burst(N, seq, Text, Class) + end. + %%%----------------------------------------------------------------- %%% Formatter callback %%% Using this to send the formatted string back to the test case @@ -1304,9 +1344,11 @@ format(#{msg:={report,#{label:={gen_server,terminate}}}},op) -> ""; format(#{msg:={report,#{label:={proc_lib,crash}}}},op) -> ""; -format(#{msg:={F,A}},Pid) when is_list(F), is_list(A) -> +format(#{msg:={F,A}},OpOrPid) when is_list(F), is_list(A) -> String = lists:flatten(io_lib:format(F,A)), - Pid ! {log,String}, + if is_pid(OpOrPid) -> OpOrPid ! {log,String}; + true -> ok + end, String++"\n"; format(#{msg:={string,String0}},Pid) -> String = unicode:characters_to_list(String0), @@ -1322,16 +1364,17 @@ remove(Handler, LogName) -> HState = #{log_names := Logs} = logger_disk_log_h:info(), false = maps:is_key(LogName, HState), false = lists:member(LogName, Logs), - false = logger_config:exist(logger, LogName), + false = logger_config:exist(?LOGGER_TABLE, LogName), {error,no_such_log} = disk_log:info(LogName), ok. start_and_add(Name, Config, LogOpts) -> - ct:pal("Adding handler ~w with: ~p", - [Name,Config#{disk_log_opts=>LogOpts}]), - ok = logger:add_handler(Name, logger_disk_log_h, - Config#{disk_log_opts=>LogOpts}), - Pid = whereis(Name), + HConfig = maps:get(config, Config, #{}), + HConfig1 = maps:merge(HConfig, LogOpts), + Config1 = Config#{config=>HConfig1}, + ct:pal("Adding handler ~w with: ~p", [Name,Config1]), + ok = logger:add_handler(Name, logger_disk_log_h, Config1), + Pid = whereis(h_proc_name(Name)), true = is_pid(Pid), Name = proplists:get_value(name, disk_log:info(Name)), ok. @@ -1339,7 +1382,7 @@ start_and_add(Name, Config, LogOpts) -> remove_and_stop(Handler) -> ok = logger:remove_handler(Handler), timer:sleep(500), - undefined = whereis(Handler), + undefined = whereis(h_proc_name(Handler)), ok. try_read_file(FileName, Expected, Time) -> @@ -1395,27 +1438,31 @@ wait_until_written(File, Sz) -> {ok,#file_info{size = Sz}} -> timer:sleep(1000), case file:read_file_info(File) of - {ok,#file_info{size = Sz1}} -> + {ok,#file_info{size = Sz}} -> ok; - {ok,#file_info{size = Sz2}} -> - wait_until_written(File, Sz2) + {ok,#file_info{size = Sz1}} -> + wait_until_written(File, Sz1) end; {ok,#file_info{size = Sz1}} -> wait_until_written(File, Sz1) end. count_lines1(File) -> - Counter = fun Cnt(Dev,LC) -> - case file:read_line(Dev) of - eof -> LC; - _ -> Cnt(Dev,LC+1) - end - end, {_,Dev} = file:open(File, [read]), - Lines = Counter(Dev, 0), + Lines = count_lines2(Dev, 0), file:close(Dev), Lines. +count_lines2(Dev, LC) -> + case file:read_line(Dev) of + {ok,"Handler logger_disk_log_h_SUITE " ++_} -> + %% Not counting handler info + count_lines2(Dev,LC); + {ok,_} -> + count_lines2(Dev,LC+1); + eof -> LC + end. + repeat_until_ok(Fun, N) -> repeat_until_ok(Fun, 0, N, undefined). @@ -1442,7 +1489,7 @@ repeat_until_ok(Fun, C, Stop, FirstReason) -> start_tracer(Trace,Expected) -> Pid = self(), dbg:tracer(process,{fun tracer/2,{Pid,Expected}}), - dbg:p(whereis(?MODULE),[c]), + dbg:p(h_proc_name(),[c]), tpl(Trace), ok. @@ -1498,3 +1545,36 @@ escape([H|T]) -> [H|escape(T)]; escape([]) -> []. + +h_proc_name() -> + h_proc_name(?MODULE). +h_proc_name(Name) -> + list_to_atom(lists:concat([logger_disk_log_h,"_",Name])). + +wait_for_process_up(T) -> + wait_for_process_up(?MODULE, h_proc_name(), T). + +wait_for_process_up(Name, RegName, T) -> + N = (T div 500) + 1, + wait_for_process_up1(Name, RegName, N). + +wait_for_process_up1(_Name, _RegName, 0) -> + error; +wait_for_process_up1(Name, RegName, N) -> + timer:sleep(500), + case whereis(RegName) of + Pid when is_pid(Pid) -> + case logger:get_handler_config(Name) of + {ok,_} -> + %% ct:pal("Process ~p up (~p tries left)",[Name,N]), + {ok,Pid}; + _ -> + wait_for_process_up1(Name, RegName, N-1) + end; + undefined -> + %% ct:pal("Waiting for process ~p (~p tries left)",[Name,N]), + wait_for_process_up1(Name, RegName, N-1) + end. + +file_delete(Log) -> + file:delete(Log). diff --git a/lib/kernel/test/logger_env_var_SUITE.erl b/lib/kernel/test/logger_env_var_SUITE.erl index 601d331fb0..e8d1a313dc 100644 --- a/lib/kernel/test/logger_env_var_SUITE.erl +++ b/lib/kernel/test/logger_env_var_SUITE.erl @@ -63,8 +63,7 @@ groups() -> ]}, {bad,[],[bad_error_logger, bad_level, - bad_sasl_compatibility, - bad_progress]}]. + bad_sasl_compatibility]}]. all() -> [default, @@ -72,134 +71,133 @@ all() -> sasl_compatible_false, sasl_compatible_false_no_progress, sasl_compatible, + all_logger_level, {group,bad}, {group,error_logger}, {group,logger} ]. default(Config) -> - {ok,#{handlers:=Hs},_Node} = setup(Config,[]), - {?STANDARD_HANDLER,logger_std_h,StdC} = lists:keyfind(?STANDARD_HANDLER,1,Hs), - info = maps:get(level,StdC), + {ok,#{primary:=P,handlers:=Hs,module_levels:=ML},_Node} = setup(Config,[]), + notice = maps:get(level,P), + #{module:=logger_std_h} = StdC = find(?STANDARD_HANDLER,Hs), + all = maps:get(level,StdC), StdFilters = maps:get(filters,StdC), - {domain,{_,{log,super,[beam,erlang,otp,sasl]}}} = - lists:keyfind(domain,1,StdFilters), - true = lists:keymember(stop_progress,1,StdFilters), - false = lists:keymember(simple,1,Hs), - false = lists:keymember(sasl,1,Hs), + {domain,{_,{log,super,[otp,sasl]}}} = lists:keyfind(domain,1,StdFilters), + false = exists(simple,Hs), + false = exists(sasl,Hs), + [] = ML, ok. default_sasl_compatible(Config) -> - {ok,#{handlers:=Hs},_Node} = setup(Config, - [{logger_sasl_compatible,true}]), - {?STANDARD_HANDLER,logger_std_h,StdC} = lists:keyfind(?STANDARD_HANDLER,1,Hs), - info = maps:get(level,StdC), + {ok,#{primary:=P,handlers:=Hs,module_levels:=ML},_Node} = + setup(Config,[{logger_sasl_compatible,true}]), + info = maps:get(level,P), + #{module:=logger_std_h} = StdC = find(?STANDARD_HANDLER,Hs), + all = maps:get(level,StdC), StdFilters = maps:get(filters,StdC), - {domain,{_,{log,super,[beam,erlang,otp]}}} = - lists:keyfind(domain,1,StdFilters), - false = lists:keymember(stop_progress,1,StdFilters), - false = lists:keymember(simple,1,Hs), - true = lists:keymember(sasl,1,Hs), + {domain,{_,{log,super,[otp]}}} = lists:keyfind(domain,1,StdFilters), + false = exists(simple,Hs), + true = exists(sasl,Hs), + [] = ML, ok. error_logger_tty(Config) -> - {ok,#{handlers:=Hs},_Node} = setup(Config,[{error_logger,tty}]), - {?STANDARD_HANDLER,logger_std_h,StdC} = lists:keyfind(?STANDARD_HANDLER,1,Hs), - info = maps:get(level,StdC), + {ok,#{primary:=P,handlers:=Hs,module_levels:=ML},_Node} = + setup(Config,[{error_logger,tty}]), + notice = maps:get(level,P), + #{module:=logger_std_h} = StdC = find(?STANDARD_HANDLER,Hs), + all = maps:get(level,StdC), StdFilters = maps:get(filters,StdC), - {domain,{_,{log,super,[beam,erlang,otp,sasl]}}} = - lists:keyfind(domain,1,StdFilters), - true = lists:keymember(stop_progress,1,StdFilters), - false = lists:keymember(simple,1,Hs), - false = lists:keymember(sasl,1,Hs), + {domain,{_,{log,super,[otp,sasl]}}} = lists:keyfind(domain,1,StdFilters), + false = exists(simple,Hs), + false = exists(sasl,Hs), + [] = ML, ok. error_logger_tty_sasl_compatible(Config) -> - {ok,#{handlers:=Hs},_Node} = setup(Config, - [{error_logger,tty}, - {logger_sasl_compatible,true}]), - {?STANDARD_HANDLER,logger_std_h,StdC} = lists:keyfind(?STANDARD_HANDLER,1,Hs), - info = maps:get(level,StdC), + {ok,#{primary:=P,handlers:=Hs,module_levels:=ML},_Node} = + setup(Config, + [{error_logger,tty}, + {logger_sasl_compatible,true}]), + info = maps:get(level,P), + #{module:=logger_std_h} = StdC = find(?STANDARD_HANDLER,Hs), + all = maps:get(level,StdC), StdFilters = maps:get(filters,StdC), - {domain,{_,{log,super,[beam,erlang,otp]}}} = - lists:keyfind(domain,1,StdFilters), - false = lists:keymember(stop_progress,1,StdFilters), - false = lists:keymember(simple,1,Hs), - true = lists:keymember(sasl,1,Hs), + {domain,{_,{log,super,[otp]}}} = lists:keyfind(domain,1,StdFilters), + false = exists(simple,Hs), + true = exists(sasl,Hs), + [] = ML, ok. error_logger_false(Config) -> - {ok,#{handlers:=Hs,logger:=L},_Node} = + {ok,#{handlers:=Hs,primary:=P,module_levels:=ML},_Node} = setup(Config, [{error_logger,false}, {logger_level,notice}]), - false = lists:keymember(?STANDARD_HANDLER,1,Hs), - {simple,logger_simple_h,SimpleC} = lists:keyfind(simple,1,Hs), - info = maps:get(level,SimpleC), - notice = maps:get(level,L), + false = exists(?STANDARD_HANDLER,Hs), + #{module:=logger_simple_h} = SimpleC = find(simple,Hs), + all = maps:get(level,SimpleC), + notice = maps:get(level,P), SimpleFilters = maps:get(filters,SimpleC), - {domain,{_,{log,super,[beam,erlang,otp,sasl]}}} = - lists:keyfind(domain,1,SimpleFilters), - true = lists:keymember(stop_progress,1,SimpleFilters), - false = lists:keymember(sasl,1,Hs), + {domain,{_,{log,super,[otp,sasl]}}} = lists:keyfind(domain,1,SimpleFilters), + false = exists(sasl,Hs), + [] = ML, ok. error_logger_false_progress(Config) -> - {ok,#{handlers:=Hs,logger:=L},_Node} = + {ok,#{handlers:=Hs,primary:=P,module_levels:=ML},_Node} = setup(Config, [{error_logger,false}, - {logger_level,notice}, - {logger_progress_reports,log}]), - false = lists:keymember(?STANDARD_HANDLER,1,Hs), - {simple,logger_simple_h,SimpleC} = lists:keyfind(simple,1,Hs), - info = maps:get(level,SimpleC), - notice = maps:get(level,L), + {logger_level,notice}]), + false = exists(?STANDARD_HANDLER,Hs), + #{module:=logger_simple_h} = SimpleC = find(simple,Hs), + all = maps:get(level,SimpleC), + notice = maps:get(level,P), SimpleFilters = maps:get(filters,SimpleC), - {domain,{_,{log,super,[beam,erlang,otp,sasl]}}} = - lists:keyfind(domain,1,SimpleFilters), - false = lists:keymember(stop_progress,1,SimpleFilters), - false = lists:keymember(sasl,1,Hs), + {domain,{_,{log,super,[otp,sasl]}}} = lists:keyfind(domain,1,SimpleFilters), + false = exists(sasl,Hs), + [] = ML, ok. error_logger_false_sasl_compatible(Config) -> - {ok,#{handlers:=Hs,logger:=L},_Node} = + {ok,#{handlers:=Hs,primary:=P,module_levels:=ML},_Node} = setup(Config, [{error_logger,false}, {logger_level,notice}, {logger_sasl_compatible,true}]), - false = lists:keymember(?STANDARD_HANDLER,1,Hs), - {simple,logger_simple_h,SimpleC} = lists:keyfind(simple,1,Hs), - info = maps:get(level,SimpleC), - notice = maps:get(level,L), + false = exists(?STANDARD_HANDLER,Hs), + #{module:=logger_simple_h} = SimpleC = find(simple,Hs), + all = maps:get(level,SimpleC), + info = maps:get(level,P), SimpleFilters = maps:get(filters,SimpleC), - {domain,{_,{log,super,[beam,erlang,otp]}}} = - lists:keyfind(domain,1,SimpleFilters), - false = lists:keymember(stop_progress,1,SimpleFilters), - true = lists:keymember(sasl,1,Hs), + {domain,{_,{log,super,[otp]}}} = lists:keyfind(domain,1,SimpleFilters), + true = exists(sasl,Hs), + [] = ML, ok. error_logger_silent(Config) -> {ok,#{handlers:=Hs},_Node} = setup(Config, [{error_logger,silent}]), - false = lists:keymember(?STANDARD_HANDLER,1,Hs), - false = lists:keymember(simple,1,Hs), - false = lists:keymember(sasl,1,Hs), + false = exists(?STANDARD_HANDLER,Hs), + false = exists(simple,Hs), + false = exists(sasl,Hs), ok. error_logger_silent_sasl_compatible(Config) -> {ok,#{handlers:=Hs},_Node} = setup(Config, [{error_logger,silent}, {logger_sasl_compatible,true}]), - false = lists:keymember(?STANDARD_HANDLER,1,Hs), - false = lists:keymember(simple,1,Hs), - true = lists:keymember(sasl,1,Hs), + false = exists(?STANDARD_HANDLER,Hs), + false = exists(simple,Hs), + true = exists(sasl,Hs), ok. error_logger_file(Config) -> Log = file(Config,?FUNCTION_NAME), - {ok,_Hs,Node} = setup(Config, - [{error_logger,{file,Log}}]), + {ok,_,Node} = setup(Config, + [{error_logger,{file,Log}}]), check_default_log(Node,Log, file,% dest 0),% progress in std logger @@ -208,70 +206,68 @@ error_logger_file(Config) -> logger_file(Config) -> Log = file(Config,?FUNCTION_NAME), - {ok,#{handlers:=Hs},Node} + {ok,#{primary:=P,handlers:=Hs,module_levels:=ML},Node} = setup(Config, [{logger, [{handler,?STANDARD_HANDLER,logger_std_h, - #{logger_std_h=>#{type=>{file,Log}}}}]}]), + #{config=>#{type=>{file,Log}}}}]}]), check_default_log(Node,Log, file,% dest 0),% progress in std logger - {?STANDARD_HANDLER,logger_std_h,StdC} = lists:keyfind(?STANDARD_HANDLER,1,Hs), - info = maps:get(level,StdC), + notice = maps:get(level,P), + #{module:=logger_std_h} = StdC = find(?STANDARD_HANDLER,Hs), + all = maps:get(level,StdC), StdFilters = maps:get(filters,StdC), - {domain,{_,{log,super,[beam,erlang,otp,sasl]}}} = - lists:keyfind(domain,1,StdFilters), - true = lists:keymember(stop_progress,1,StdFilters), - false = lists:keymember(simple,1,Hs), - false = lists:keymember(sasl,1,Hs), - + {domain,{_,{log,super,[otp,sasl]}}} = lists:keyfind(domain,1,StdFilters), + false = exists(simple,Hs), + false = exists(sasl,Hs), + [] = ML, ok. logger_file_sasl_compatible(Config) -> Log = file(Config,?FUNCTION_NAME), - {ok,#{handlers:=Hs},Node} + {ok,#{primary:=P,handlers:=Hs,module_levels:=ML},Node} = setup(Config, [{logger_sasl_compatible,true}, {logger, [{handler,?STANDARD_HANDLER,logger_std_h, - #{logger_std_h=>#{type=>{file,Log}}}}]}]), + #{config=>#{type=>{file,Log}}}}]}]), check_default_log(Node,Log, file,% dest 0),% progress in std logger - {?STANDARD_HANDLER,logger_std_h,StdC} = lists:keyfind(?STANDARD_HANDLER,1,Hs), - info = maps:get(level,StdC), + info = maps:get(level,P), + #{module:=logger_std_h} = StdC = find(?STANDARD_HANDLER,Hs), + all = maps:get(level,StdC), StdFilters = maps:get(filters,StdC), - {domain,{_,{log,super,[beam,erlang,otp]}}} = - lists:keyfind(domain,1,StdFilters), - false = lists:keymember(stop_progress,1,StdFilters), - false = lists:keymember(simple,1,Hs), - true = lists:keymember(sasl,1,Hs), - + {domain,{_,{log,super,[otp]}}} = lists:keyfind(domain,1,StdFilters), + false = exists(simple,Hs), + true = exists(sasl,Hs), + [] = ML, ok. logger_file_log_progress(Config) -> Log = file(Config,?FUNCTION_NAME), - {ok,#{handlers:=Hs},Node} + {ok,#{primary:=P,handlers:=Hs,module_levels:=ML},Node} = setup(Config, - [{logger_progress_reports,log}, + [{logger_level,info}, {logger, [{handler,?STANDARD_HANDLER,logger_std_h, - #{logger_std_h=>#{type=>{file,Log}}}}]}]), + #{config=>#{type=>{file,Log}}}}]}]), check_default_log(Node,Log, file,% dest - 6),% progress in std logger + 6,% progress in std logger + info), - {?STANDARD_HANDLER,logger_std_h,StdC} = lists:keyfind(?STANDARD_HANDLER,1,Hs), - info = maps:get(level,StdC), + info = maps:get(level,P), + #{module:=logger_std_h} = StdC = find(?STANDARD_HANDLER,Hs), + all = maps:get(level,StdC), StdFilters = maps:get(filters,StdC), - {domain,{_,{log,super,[beam,erlang,otp,sasl]}}} = - lists:keyfind(domain,1,StdFilters), - false = lists:keymember(stop_progress,1,StdFilters), - false = lists:keymember(simple,1,Hs), - false = lists:keymember(sasl,1,Hs), - + {domain,{_,{log,super,[otp,sasl]}}} = lists:keyfind(domain,1,StdFilters), + false = exists(simple,Hs), + false = exists(sasl,Hs), + [] = ML, ok. logger_file_no_filter(Config) -> @@ -281,16 +277,16 @@ logger_file_no_filter(Config) -> [{logger, [{handler,?STANDARD_HANDLER,logger_std_h, #{filter_default=>log,filters=>[], - logger_std_h=>#{type=>{file,Log}}}}]}]), + config=>#{type=>{file,Log}}}}]}]), check_default_log(Node,Log, file,% dest 6),% progress in std logger - {?STANDARD_HANDLER,logger_std_h,StdC} = lists:keyfind(?STANDARD_HANDLER,1,Hs), - info = maps:get(level,StdC), + #{module:=logger_std_h} = StdC = find(?STANDARD_HANDLER,Hs), + all = maps:get(level,StdC), [] = maps:get(filters,StdC), - false = lists:keymember(simple,1,Hs), - false = lists:keymember(sasl,1,Hs), + false = exists(simple,Hs), + false = exists(sasl,Hs), ok. @@ -301,17 +297,17 @@ logger_file_no_filter_level(Config) -> [{logger, [{handler,?STANDARD_HANDLER,logger_std_h, #{filters=>[],level=>error, - logger_std_h=>#{type=>{file,Log}}}}]}]), + config=>#{type=>{file,Log}}}}]}]), check_default_log(Node,Log, file,% dest 0,% progress in std logger error),% level - {?STANDARD_HANDLER,logger_std_h,StdC} = lists:keyfind(?STANDARD_HANDLER,1,Hs), + #{module:=logger_std_h} = StdC = find(?STANDARD_HANDLER,Hs), error = maps:get(level,StdC), [] = maps:get(filters,StdC), - false = lists:keymember(simple,1,Hs), - false = lists:keymember(sasl,1,Hs), + false = exists(simple,Hs), + false = exists(sasl,Hs), ok. @@ -323,68 +319,67 @@ logger_file_formatter(Config) -> [{handler,?STANDARD_HANDLER,logger_std_h, #{filters=>[], formatter=>{logger_formatter,#{}}, - logger_std_h=>#{type=>{file,Log}}}}]}]), + config=>#{type=>{file,Log}}}}]}]), check_single_log(Node,Log, file,% dest 6),% progress in std logger - {?STANDARD_HANDLER,logger_std_h,StdC} = lists:keyfind(?STANDARD_HANDLER,1,Hs), - info = maps:get(level,StdC), + #{module:=logger_std_h} = StdC = find(?STANDARD_HANDLER,Hs), + all = maps:get(level,StdC), [] = maps:get(filters,StdC), - false = lists:keymember(simple,1,Hs), - false = lists:keymember(sasl,1,Hs), + false = exists(simple,Hs), + false = exists(sasl,Hs), ok. logger_filters(Config) -> Log = file(Config,?FUNCTION_NAME), - {ok,#{handlers:=Hs,logger:=Logger},Node} + {ok,#{handlers:=Hs,primary:=P},Node} = setup(Config, - [{logger_progress_reports,log}, + [{logger_level,info}, {logger, [{handler,?STANDARD_HANDLER,logger_std_h, - #{logger_std_h=>#{type=>{file,Log}}}}, + #{config=>#{type=>{file,Log}}}}, {filters,log,[{stop_progress,{fun logger_filters:progress/2,stop}}]} ]}]), check_default_log(Node,Log, file,% dest - 0),% progress in std logger + 0,% progress in std logger + info), - {?STANDARD_HANDLER,logger_std_h,StdC} = lists:keyfind(?STANDARD_HANDLER,1,Hs), - info = maps:get(level,StdC), + #{module:=logger_std_h} = StdC = find(?STANDARD_HANDLER,Hs), + all = maps:get(level,StdC), StdFilters = maps:get(filters,StdC), - {domain,{_,{log,super,[beam,erlang,otp,sasl]}}} = - lists:keyfind(domain,1,StdFilters), - false = lists:keymember(stop_progress,1,StdFilters), - false = lists:keymember(simple,1,Hs), - false = lists:keymember(sasl,1,Hs), - LoggerFilters = maps:get(filters,Logger), + {domain,{_,{log,super,[otp,sasl]}}} = lists:keyfind(domain,1,StdFilters), + false = exists(simple,Hs), + false = exists(sasl,Hs), + LoggerFilters = maps:get(filters,P), true = lists:keymember(stop_progress,1,LoggerFilters), ok. logger_filters_stop(Config) -> Log = file(Config,?FUNCTION_NAME), - {ok,#{handlers:=Hs,logger:=Logger},Node} + {ok,#{handlers:=Hs,primary:=P},Node} = setup(Config, - [{logger_progress_reports,log}, + [{logger_level,info}, {logger, [{handler,?STANDARD_HANDLER,logger_std_h, #{filters=>[], - logger_std_h=>#{type=>{file,Log}}}}, + config=>#{type=>{file,Log}}}}, {filters,stop,[{log_error,{fun logger_filters:level/2,{log,gt,info}}}]} ]}]), check_default_log(Node,Log, file,% dest - 0, - notice),% progress in std logger + 0,% progress in std logger + info), - {?STANDARD_HANDLER,logger_std_h,StdC} = lists:keyfind(?STANDARD_HANDLER,1,Hs), - info = maps:get(level,StdC), + #{module:=logger_std_h} = StdC = find(?STANDARD_HANDLER,Hs), + all = maps:get(level,StdC), [] = maps:get(filters,StdC), - false = lists:keymember(simple,1,Hs), - false = lists:keymember(sasl,1,Hs), - LoggerFilters = maps:get(filters,Logger), + false = exists(simple,Hs), + false = exists(sasl,Hs), + LoggerFilters = maps:get(filters,P), true = lists:keymember(log_error,1,LoggerFilters), ok. @@ -393,24 +388,23 @@ logger_module_level(Config) -> Log = file(Config,?FUNCTION_NAME), {ok,#{handlers:=Hs,module_levels:=ModuleLevels},Node} = setup(Config, - [{logger_progress_reports,log}, + [{logger_level,info}, {logger, [{handler,?STANDARD_HANDLER,logger_std_h, - #{logger_std_h=>#{type=>{file,Log}}}}, + #{config=>#{type=>{file,Log}}}}, {module_level,error,[supervisor]} ]}]), check_default_log(Node,Log, file,% dest - 3),% progress in std logger + 3,% progress in std logger + info), - {?STANDARD_HANDLER,logger_std_h,StdC} = lists:keyfind(?STANDARD_HANDLER,1,Hs), - info = maps:get(level,StdC), + #{module:=logger_std_h} = StdC = find(?STANDARD_HANDLER,Hs), + all = maps:get(level,StdC), StdFilters = maps:get(filters,StdC), - {domain,{_,{log,super,[beam,erlang,otp,sasl]}}} = - lists:keyfind(domain,1,StdFilters), - false = lists:keymember(stop_progress,1,StdFilters), - false = lists:keymember(simple,1,Hs), - false = lists:keymember(sasl,1,Hs), + {domain,{_,{log,super,[otp,sasl]}}} = lists:keyfind(domain,1,StdFilters), + false = exists(simple,Hs), + false = exists(sasl,Hs), [{supervisor,error}] = ModuleLevels, ok. @@ -420,19 +414,17 @@ logger_disk_log(Config) -> = setup(Config, [{logger, [{handler,?STANDARD_HANDLER,logger_disk_log_h, - #{disk_log_opts=>#{file=>Log}}}]}]), + #{config=>#{file=>Log}}}]}]), check_default_log(Node,Log, disk_log,% dest 0),% progress in std logger - {?STANDARD_HANDLER,logger_disk_log_h,StdC} = lists:keyfind(?STANDARD_HANDLER,1,Hs), - info = maps:get(level,StdC), + #{module:=logger_disk_log_h} = StdC = find(?STANDARD_HANDLER,Hs), + all = maps:get(level,StdC), StdFilters = maps:get(filters,StdC), - {domain,{_,{log,super,[beam,erlang,otp,sasl]}}} = - lists:keyfind(domain,1,StdFilters), - true = lists:keymember(stop_progress,1,StdFilters), - false = lists:keymember(simple,1,Hs), - false = lists:keymember(sasl,1,Hs), + {domain,{_,{log,super,[otp,sasl]}}} = lists:keyfind(domain,1,StdFilters), + false = exists(simple,Hs), + false = exists(sasl,Hs), ok. @@ -444,31 +436,29 @@ logger_disk_log_formatter(Config) -> [{handler,?STANDARD_HANDLER,logger_disk_log_h, #{filters=>[], formatter=>{logger_formatter,#{}}, - disk_log_opts=>#{file=>Log}}}]}]), + config=>#{file=>Log}}}]}]), check_single_log(Node,Log, disk_log,% dest 6),% progress in std logger - {?STANDARD_HANDLER,logger_disk_log_h,StdC} = lists:keyfind(?STANDARD_HANDLER,1,Hs), - info = maps:get(level,StdC), + #{module:=logger_disk_log_h} = StdC = find(?STANDARD_HANDLER,Hs), + all = maps:get(level,StdC), [] = maps:get(filters,StdC), - false = lists:keymember(simple,1,Hs), - false = lists:keymember(sasl,1,Hs), + false = exists(simple,Hs), + false = exists(sasl,Hs), ok. logger_undefined(Config) -> - {ok,#{handlers:=Hs,logger:=L},_Node} = + {ok,#{handlers:=Hs,primary:=P},_Node} = setup(Config,[{logger,[{handler,?STANDARD_HANDLER,undefined}]}]), - false = lists:keymember(?STANDARD_HANDLER,1,Hs), - {simple,logger_simple_h,SimpleC} = lists:keyfind(simple,1,Hs), - info = maps:get(level,SimpleC), - info = maps:get(level,L), + false = exists(?STANDARD_HANDLER,Hs), + #{module:=logger_simple_h} = SimpleC = find(simple,Hs), + all = maps:get(level,SimpleC), + notice = maps:get(level,P), SimpleFilters = maps:get(filters,SimpleC), - {domain,{_,{log,super,[beam,erlang,otp,sasl]}}} = - lists:keyfind(domain,1,SimpleFilters), - true = lists:keymember(stop_progress,1,SimpleFilters), - false = lists:keymember(sasl,1,Hs), + {domain,{_,{log,super,[otp,sasl]}}} = lists:keyfind(domain,1,SimpleFilters), + false = exists(sasl,Hs), ok. @@ -483,14 +473,15 @@ logger_many_handlers_default_first(Config) -> #{level=>error, filters=>[], formatter=>{logger_formatter,#{}}, - logger_std_h=>#{type=>{file,LogErr}}} + config=>#{type=>{file,LogErr}}} }, {handler,info,logger_std_h, #{level=>info, filters=>[{level,{fun logger_filters:level/2,{stop,gteq,error}}}], - logger_std_h=>#{type=>{file,LogInfo}}} + config=>#{type=>{file,LogInfo}}} } - ]}], LogErr, LogInfo, 6). + ]}, + {logger_level,info}], LogErr, LogInfo, 6). %% Test that we can add multiple handlers with the default last logger_many_handlers_default_last(Config) -> @@ -501,15 +492,16 @@ logger_many_handlers_default_last(Config) -> [{handler,info,logger_std_h, #{level=>info, filters=>[{level,{fun logger_filters:level/2,{stop,gteq,error}}}], - logger_std_h=>#{type=>{file,LogInfo}}} + config=>#{type=>{file,LogInfo}}} }, {handler,?STANDARD_HANDLER,logger_std_h, #{level=>error, filters=>[], formatter=>{logger_formatter,#{}}, - logger_std_h=>#{type=>{file,LogErr}}} + config=>#{type=>{file,LogErr}}} } - ]}], LogErr, LogInfo, 7). + ]}, + {logger_level,info}], LogErr, LogInfo, 7). %% Check that we can handle that an added logger has a broken filter %% This used to cause a deadlock. @@ -523,18 +515,19 @@ logger_many_handlers_default_last_broken_filter(Config) -> #{level=>info, filters=>[{broken,{fun logger_filters:level/2,broken_state}}, {level,{fun logger_filters:level/2,{stop,gteq,error}}}], - logger_std_h=>#{type=>{file,LogInfo}}} + config=>#{type=>{file,LogInfo}}} }, {handler,?STANDARD_HANDLER,logger_std_h, #{level=>error, filters=>[], formatter=>{logger_formatter,#{}}, - logger_std_h=>#{type=>{file,LogErr}}} + config=>#{type=>{file,LogErr}}} } - ]}], LogErr, LogInfo, 7). + ]}, + {logger_level,info}], LogErr, LogInfo, 7). logger_many_handlers(Config, Env, LogErr, LogInfo, NumProgress) -> - {ok,#{handlers:=Hs},Node} = setup(Config,Env), + {ok,_,Node} = setup(Config,Env), check_single_log(Node,LogErr, file,% dest 0,% progress in std logger @@ -542,28 +535,29 @@ logger_many_handlers(Config, Env, LogErr, LogInfo, NumProgress) -> ok = rpc:call(Node,logger_std_h,filesync,[info]), {ok, Bin} = file:read_file(LogInfo), ct:log("Log content:~n~s",[Bin]), - match(Bin,<<"info:">>,NumProgress+1,info,info), + match(Bin,<<"info:">>,NumProgress,info,info), + match(Bin,<<"notice:">>,1,notice,info), match(Bin,<<"alert:">>,0,alert,info), ok. sasl_compatible_false(Config) -> Log = file(Config,?FUNCTION_NAME), - {ok,_Hs,Node} = setup(Config, - [{error_logger,{file,Log}}, - {logger_sasl_compatible,false}, - {logger_progress_reports,log}]), + {ok,_,Node} = setup(Config, + [{error_logger,{file,Log}}, + {logger_sasl_compatible,false}, + {logger_level,info}]), % to get progress check_default_log(Node,Log, file,% dest - 6),% progress in std logger + 6,% progress in std logger + info), ok. sasl_compatible_false_no_progress(Config) -> Log = file(Config,?FUNCTION_NAME), - {ok,_Hs,Node} = setup(Config, - [{error_logger,{file,Log}}, - {logger_sasl_compatible,false}, - {logger_progress_reports,stop}]), + {ok,_,Node} = setup(Config, + [{error_logger,{file,Log}}, + {logger_sasl_compatible,false}]), check_default_log(Node,Log, file,% dest 0),% progress in std logger @@ -571,14 +565,32 @@ sasl_compatible_false_no_progress(Config) -> sasl_compatible(Config) -> Log = file(Config,?FUNCTION_NAME), - {ok,_Hs,Node} = setup(Config, - [{error_logger,{file,Log}}, - {sasl_compatible,true}]), + {ok,_,Node} = setup(Config, + [{error_logger,{file,Log}}, + {sasl_compatible,true}]), check_default_log(Node,Log, file,% dest 0),% progress in std logger ok. +all_logger_level(Config) -> + [all_logger_level(Config,Level) || Level <- [none, + emergency, + alert, + critical, + error, + warning, + notice, + info, + debug, + all]], + ok. + +all_logger_level(Config,Level) -> + {ok,#{primary:=#{level:=Level}},Node} = setup(Config,[{logger_level,Level}]), + true = test_server:stop_node(Node), + ok. + bad_error_logger(Config) -> error = setup(Config,[{error_logger,baddest}]). @@ -588,9 +600,6 @@ bad_level(Config) -> bad_sasl_compatibility(Config) -> error = setup(Config,[{logger_sasl_compatible,badcomp}]). -bad_progress(Config) -> - error = setup(Config,[{logger_progress_reports,badprogress}]). - %%%----------------------------------------------------------------- %%% Internal file(Config,Func) -> @@ -598,22 +607,22 @@ file(Config,Func) -> lists:concat([Func,".log"])). check_default_log(Node,Log,Dest,NumProgress) -> - check_default_log(Node,Log,Dest,NumProgress,info). + check_default_log(Node,Log,Dest,NumProgress,notice). check_default_log(Node,Log,Dest,NumProgress,Level) -> {ok,Bin1,Bin2} = check_log(Node,Log,Dest), match(Bin1,<<"PROGRESS REPORT">>,NumProgress,info,Level), match(Bin1,<<"ALERT REPORT">>,1,alert,Level), - match(Bin1,<<"INFO REPORT">>,0,info,Level), + match(Bin1,<<"INFO REPORT">>,0,notice,Level), match(Bin1,<<"DEBUG REPORT">>,0,debug,Level), - match(Bin2,<<"INFO REPORT">>,1,info,Level), + match(Bin2,<<"INFO REPORT">>,1,notice,Level), match(Bin2,<<"DEBUG REPORT">>,0,debug,Level), ok. check_single_log(Node,Log,Dest,NumProgress) -> - check_single_log(Node,Log,Dest,NumProgress,info). + check_single_log(Node,Log,Dest,NumProgress,notice). check_single_log(Node,Log,Dest,NumProgress,Level) -> {ok,Bin1,Bin2} = check_log(Node,Log,Dest), @@ -654,3 +663,21 @@ match(Bin,Pattern,N,LogLevel,ConfLevel) -> {match,M} = re:run(Bin,Pattern,[{capture,all},global]), N = length(M) end. + +find(Id,Handlers) -> + case lists:search(fun(#{id:=Id0}) when Id0=:=Id-> true; + (_) -> false end, + Handlers) of + {value,Config} -> + Config; + false -> + false + end. + +exists(Id,Handlers) -> + case find(Id,Handlers) of + false -> + false; + _ -> + true + end. diff --git a/lib/kernel/test/logger_formatter_SUITE.erl b/lib/kernel/test/logger_formatter_SUITE.erl index 7a93f2ca79..aa8dc42691 100644 --- a/lib/kernel/test/logger_formatter_SUITE.erl +++ b/lib/kernel/test/logger_formatter_SUITE.erl @@ -56,6 +56,7 @@ groups() -> all() -> [default, legacy_header, + error_logger_notice_header, single_line, template, format_msg, @@ -69,6 +70,7 @@ all() -> faulty_log, faulty_config, faulty_msg, + check_config, update_config]. default(_Config) -> @@ -124,6 +126,34 @@ legacy_header(_Config) -> "=INFO REPORT==== "++_ = String5, ok. +error_logger_notice_header(_Config) -> + Meta1 = #{error_logger=>#{tag => info_report,type => std_info}}, + String1 = format(notice,{"~p",[term]},Meta1, + #{legacy_header=>true, + error_logger_notice_header=>notice}), + ct:log(String1), + "=NOTICE REPORT==== "++_ = String1, + + String2 = format(notice,{"~p",[term]},Meta1, + #{legacy_header=>true, + error_logger_notice_header=>info}), + ct:log(String2), + "=INFO REPORT==== "++_ = String2, + + String3 = format(notice,{"~p",[term]},#{}, + #{legacy_header=>true, + error_logger_notice_header=>notice}), + ct:log(String3), + "=NOTICE REPORT==== "++_ = String3, + + String4 = format(notice,{"~p",[term]},#{}, + #{legacy_header=>true, + error_logger_notice_header=>info}), + ct:log(String4), + "=NOTICE REPORT==== "++_ = String4, + + ok. + single_line(_Config) -> Time = timestamp(), ExpectedTimestamp = default_time_format(Time), @@ -158,7 +188,7 @@ template(_Config) -> Template4 = ["string\nnewline"], String4 = format(info,{"~p",[term]},#{time=>Time},#{template=>Template4, - single_line=>true}), + single_line=>true}), ct:log(String4), "string\nnewline" = String4, @@ -178,8 +208,8 @@ template(_Config) -> time=>Time, tuple=>{1,atom,"list"}, nested=>#{subkey=>subvalue}}, - Template6 = lists:join(";",maps:keys(maps:remove(nested,Meta6)) ++ - [{nested,subkey}]), + Template6 = lists:join(";",lists:sort(maps:keys(maps:remove(nested,Meta6))) ++ + [[nested,subkey]]), String6 = format(info,{"~p",[term]},Meta6,#{template=>Template6, single_line=>true}), ct:log(String6), @@ -202,16 +232,16 @@ template(_Config) -> nested=>#{key1=>#{subkey1=>value1}, key2=>value2}}, Template7 = lists:join(";",[nested, - {nested,key1}, - {nested,key1,subkey1}, - {nested,key2}, - {nested,key2,subkey2}, - {nested,key3}, - {nested,key3,subkey3}]), + [nested,key1], + [nested,key1,subkey1], + [nested,key2], + [nested,key2,subkey2], + [nested,key3], + [nested,key3,subkey3]]), String7 = format(info,{"~p",[term]},Meta7,#{template=>Template7, single_line=>true}), ct:log(String7), - [MultipleKeysStr, + [MultipleKeysStr7, "#{subkey1 => value1}", "value1", "value2", @@ -219,11 +249,42 @@ template(_Config) -> "", ""] = string:split(String7,";",all), %% Order of keys is not fixed - case MultipleKeysStr of + case MultipleKeysStr7 of "#{key2 => value2,key1 => #{subkey1 => value1}}" -> ok; "#{key1 => #{subkey1 => value1},key2 => value2}" -> ok; - _ -> ct:fail({full_nested_map_unexpected,MultipleKeysStr}) + _ -> ct:fail({full_nested_map_unexpected,MultipleKeysStr7}) end, + + Meta8 = #{time=>Time, + nested=>#{key1=>#{subkey1=>value1}, + key2=>value2}}, + Template8 = + lists:join( + ";", + [{nested,["exist:",nested],["noexist"]}, + {[nested,key1],["exist:",[nested,key1]],["noexist"]}, + {[nested,key1,subkey1],["exist:",[nested,key1,subkey1]],["noexist"]}, + {[nested,key2],["exist:",[nested,key2]],["noexist"]}, + {[nested,key2,subkey2],["exist:",[nested,key2,subkey2]],["noexist"]}, + {[nested,key3],["exist:",[nested,key3]],["noexist"]}, + {[nested,key3,subkey3],["exist:",[nested,key3,subkey3]],["noexist"]}]), + String8 = format(info,{"~p",[term]},Meta8,#{template=>Template8, + single_line=>true}), + ct:log(String8), + [MultipleKeysStr8, + "exist:#{subkey1 => value1}", + "exist:value1", + "exist:value2", + "noexist", + "noexist", + "noexist"] = string:split(String8,";",all), + %% Order of keys is not fixed + case MultipleKeysStr8 of + "exist:#{key2 => value2,key1 => #{subkey1 => value1}}" -> ok; + "exist:#{key1 => #{subkey1 => value1},key2 => value2}" -> ok; + _ -> ct:fail({full_nested_map_unexpected,MultipleKeysStr8}) + end, + ok. format_msg(_Config) -> @@ -251,30 +312,48 @@ format_msg(_Config) -> #{report_cb=>fun(_)-> faulty_return end}, #{template=>Template}), ct:log(String5), - "REPORT_CB ERROR: term; Returned: faulty_return" = String5, + "REPORT_CB/1 ERROR: term; Returned: faulty_return" = String5, String6 = format(info,{report,term}, #{report_cb=>fun(_)-> erlang:error(fun_crashed) end}, #{template=>Template}), ct:log(String6), - "REPORT_CB CRASH: term; Reason: {error,fun_crashed}" = String6, + "REPORT_CB/1 CRASH: term; Reason: {error,fun_crashed,"++_ = String6, + + String7 = format(info,{report,term}, + #{report_cb=>fun(_,_)-> ['not',a,string] end}, + #{template=>Template}), + ct:log(String7), + "REPORT_CB/2 ERROR: term; Returned: ['not',a,string]" = String7, + + String8 = format(info,{report,term}, + #{report_cb=>fun(_,_)-> faulty_return end}, + #{template=>Template}), + ct:log(String8), + "REPORT_CB/2 ERROR: term; Returned: faulty_return" = String8, + + String9 = format(info,{report,term}, + #{report_cb=>fun(_,_)-> erlang:error(fun_crashed) end}, + #{template=>Template}), + ct:log(String9), + "REPORT_CB/2 CRASH: term; Reason: {error,fun_crashed,"++_ = String9, %% strings are not formatted - String7 = format(info,{string,"string"}, + String10 = format(info,{string,"string"}, #{report_cb=>fun(_)-> {"formatted",[]} end}, #{template=>Template}), - ct:log(String7), - "string" = String7, + ct:log(String10), + "string" = String10, - String8 = format(info,{string,['not',printable,list]}, + String11 = format(info,{string,['not',printable,list]}, #{report_cb=>fun(_)-> {"formatted",[]} end}, #{template=>Template}), - ct:log("~ts",[String8]), % avoiding ct_log crash - "FORMAT ERROR: \"~ts\" - [['not',printable,list]]" = String8, + ct:log("~ts",[String11]), % avoiding ct_log crash + "FORMAT ERROR: \"~ts\" - [['not',printable,list]]" = String11, - String9 = format(info,{string,"string"},#{},#{template=>Template}), - ct:log(String9), - "string" = String9, + String12 = format(info,{string,"string"},#{},#{template=>Template}), + ct:log(String12), + "string" = String12, ok. @@ -541,23 +620,115 @@ faulty_msg(_Config) -> #{})), ok. +-define(cfgerr(X), {error,{invalid_formatter_config,logger_formatter,X}}). +check_config(_Config) -> + ok = logger_formatter:check_config(#{}), + ?cfgerr(bad) = logger_formatter:check_config(bad), + + C1 = #{chars_limit => 1, + depth => 1, + legacy_header => true, + error_logger_notice_header => info, + max_size => 1, + report_cb => fun(R) -> {"~p",[R]} end, + single_line => false, + template => [], + time_designator => $T, + time_offset => 0}, + ok = logger_formatter:check_config(C1), + + ok = logger_formatter:check_config(#{chars_limit => unlimited}), + ?cfgerr({chars_limit,bad}) = + logger_formatter:check_config(#{chars_limit => bad}), + + ok = logger_formatter:check_config(#{depth => unlimited}), + ?cfgerr({depth,bad}) = + logger_formatter:check_config(#{depth => bad}), + + ok = logger_formatter:check_config(#{legacy_header => false}), + ?cfgerr({legacy_header,bad}) = + logger_formatter:check_config(#{legacy_header => bad}), + + ok = logger_formatter:check_config(#{error_logger_notice_header => notice}), + ?cfgerr({error_logger_notice_header,bad}) = + logger_formatter:check_config(#{error_logger_notice_header => bad}), + + ok = logger_formatter:check_config(#{max_size => unlimited}), + ?cfgerr({max_size,bad}) = + logger_formatter:check_config(#{max_size => bad}), + + ok = + logger_formatter:check_config(#{report_cb => fun(_,_) -> "" end}), + ?cfgerr({report_cb,F}) = + logger_formatter:check_config(#{report_cb => F=fun(_,_,_) -> {"",[]} end}), + ?cfgerr({report_cb,bad}) = + logger_formatter:check_config(#{report_cb => bad}), + + ok = logger_formatter:check_config(#{single_line => true}), + ?cfgerr({single_line,bad}) = + logger_formatter:check_config(#{single_line => bad}), + + Ts = [[key], + [[key1,key2]], + [{key,[key],[]}], + [{[key1,key2],[[key1,key2]],["noexist"]}], + ["string"]], + [begin + ct:log("check template: ~p",[T]), + ok = logger_formatter:check_config(#{template => T}) + end + || T <- Ts], + + ETs = [bad, + [{key,bad}], + [{key,[key],bad}], + [{key,[key],"bad"}], + "bad", + [[key,$a,$b,$c]], + [[$a,$b,$c,key]]], + [begin + ct:log("check template: ~p",[T]), + {error,{invalid_formatter_template,logger_formatter,T}} = + logger_formatter:check_config(#{template => T}) + end + || T <- ETs], + + ?cfgerr({time_designator,bad}) = + logger_formatter:check_config(#{time_designator => bad}), + ?cfgerr({time_designator,"b"}) = + logger_formatter:check_config(#{time_designator => "b"}), + + ok = logger_formatter:check_config(#{time_offset => -1}), + ok = logger_formatter:check_config(#{time_offset => "+02:00"}), + ok = logger_formatter:check_config(#{time_offset => "-23:59"}), + ok = logger_formatter:check_config(#{time_offset => "+24:00"}), + ok = logger_formatter:check_config(#{time_offset => "-25:00"}), + ?cfgerr({time_offset,bad}) = + logger_formatter:check_config(#{time_offset => bad}), + ?cfgerr({time_offset,"02:00"}) = + logger_formatter:check_config(#{time_offset => "02:00"}), + ?cfgerr({time_offset,"+02"}) = + logger_formatter:check_config(#{time_offset => "+02"}), + + ok. + %% Test that formatter config can be changed, and that the default %% template is updated accordingly update_config(_Config) -> logger:add_handler_filter(default,silence,{fun(_,_) -> stop end,ok}), ok = logger:add_handler(?MODULE,?MODULE,#{}), D = lists:seq(1,1000), - logger:info("~p~n",[D]), + logger:notice("~p~n",[D]), {Lines1,C1} = check_log(), [ct:log(L) || L <- Lines1], ct:log("~p",[C1]), [Line1] = Lines1, - [_Time,"info: "++D1] = string:split(Line1," "), + [_Time,"notice: "++D1] = string:split(Line1," "), true = length(D1)>3000, true = #{}==C1, ok = logger:update_formatter_config(?MODULE,single_line,false), - logger:info("~p~n",[D]), + logger:notice("~p~n",[D]), {Lines2,C2} = check_log(), [ct:log(L) || L <- Lines2], ct:log("~p",[C2]), @@ -565,23 +736,37 @@ update_config(_Config) -> true = #{single_line=>false}==C2, ok = logger:update_formatter_config(?MODULE,#{legacy_header=>true}), - logger:info("~p~n",[D]), + logger:notice("~p~n",[D]), {Lines3,C3} = check_log(), [ct:log(L) || L <- Lines3], ct:log("~p",[C3]), - ["=INFO REPORT==== "++_|D3] = Lines3, + ["=NOTICE REPORT==== "++_|D3] = Lines3, true = length(D3)>50, true = #{legacy_header=>true,single_line=>false}==C3, ok = logger:update_formatter_config(?MODULE,single_line,true), - logger:info("~p~n",[D]), + logger:notice("~p~n",[D]), {Lines4,C4} = check_log(), [ct:log(L) || L <- Lines4], ct:log("~p",[C4]), - ["=INFO REPORT==== "++_,D4] = Lines4, + ["=NOTICE REPORT==== "++_,D4] = Lines4, true = length(D4)>3000, true = #{legacy_header=>true,single_line=>true}==C4, + %% Finally, check that error_logger_notice_header works, default=info + error_logger:info_msg("~p",[D]), + {Lines5,C5} = check_log(), + [ct:log(L) || L <- Lines5], + ct:log("~p",[C5]), + ["=INFO REPORT==== "++_,_D5] = Lines5, + + ok=logger:update_formatter_config(?MODULE,error_logger_notice_header,notice), + error_logger:info_msg("~p",[D]), + {Lines6,C6} = check_log(), + [ct:log(L) || L <- Lines6], + ct:log("~p",[C6]), + ["=NOTICE REPORT==== "++_,_D6] = Lines6, + ok. update_config(cleanup,_Config) -> diff --git a/lib/kernel/test/logger_legacy_SUITE.erl b/lib/kernel/test/logger_legacy_SUITE.erl index cfba35e43f..c3cab07d81 100644 --- a/lib/kernel/test/logger_legacy_SUITE.erl +++ b/lib/kernel/test/logger_legacy_SUITE.erl @@ -20,6 +20,8 @@ -module(logger_legacy_SUITE). -compile(export_all). +-compile({nowarn_deprecated_function,[{gen_fsm,start,3}, + {gen_fsm,send_all_state_event,2}]}). -include_lib("common_test/include/ct.hrl"). -include_lib("kernel/include/logger.hrl"). @@ -67,25 +69,29 @@ end_per_suite(_Config) -> init_per_group(std, Config) -> ok = logger:set_handler_config( error_logger,filters, - [{domain,{fun logger_filters:domain/2, - {log,super,[beam,erlang,otp]}}}]), + [{domain,{fun logger_filters:domain/2,{log,super,[otp]}}}]), Config; init_per_group(sasl, Config) -> + %% Since default level is notice, and progress reports are info, + %% we need to raise the global logger level to info in order to + %% receive these. + ok = logger:set_primary_config(level,info), ok = logger:set_handler_config( error_logger,filters, - [{domain,{fun logger_filters:domain/2, - {log,super,[beam,erlang,otp,sasl]}}}]), + [{domain,{fun logger_filters:domain/2,{log,super,[otp,sasl]}}}]), %% cth_log_redirect checks if sasl is started before displaying %% any sasl reports - so just to see the real sasl reports in tc %% log: - application:start(sasl), - Config; + {ok,Apps} = application:ensure_all_started(sasl), + [{stop_apps,Apps}|Config]; init_per_group(_Group, Config) -> Config. -end_per_group(sasl, _Config) -> - application:stop(sasl), +end_per_group(sasl, Config) -> + Apps = ?config(stop_apps,Config), + [application:stop(App) || App <- Apps], + ok = logger:set_primary_config(level,notice), ok; end_per_group(_Group, _Config) -> ok. @@ -119,7 +125,7 @@ all() -> gen_server(_Config) -> {ok,Pid} = gen_server:start(?MODULE,gen_server,[]), - Msg = fun() -> a=b end, + Msg = fun() -> erlang:error({badmatch,b}) end, Pid ! Msg, ?check({warning_msg,"** Undefined handle_info in ~p"++_,[?MODULE,Msg]}), ok = gen_server:cast(Pid,Msg), @@ -129,7 +135,7 @@ gen_server(_Config) -> gen_event(_Config) -> {ok,Pid} = gen_event:start(), ok = gen_event:add_handler(Pid,?MODULE,gen_event), - Msg = fun() -> a=b end, + Msg = fun() -> erlang:error({badmatch,b}) end, Pid ! Msg, ?check({warning_msg,"** Undefined handle_info in ~tp"++_,[?MODULE,Msg]}), gen_event:notify(Pid,Msg), @@ -138,7 +144,7 @@ gen_event(_Config) -> gen_fsm(_Config) -> {ok,Pid} = gen_fsm:start(?MODULE,gen_fsm,[]), - Msg = fun() -> a=b end, + Msg = fun() -> erlang:error({badmatch,b}) end, Pid ! Msg, ?check({warning_msg,"** Undefined handle_info in ~p"++_,[?MODULE,Msg]}), gen_fsm:send_all_state_event(Pid,Msg), @@ -147,7 +153,7 @@ gen_fsm(_Config) -> gen_statem(_Config) -> {ok,Pid} = gen_statem:start(?MODULE,gen_statem,[]), - Msg = fun() -> a=b end, + Msg = fun() -> erlang:error({badmatch,b}) end, Pid ! Msg, ?check({error,"** State machine ~tp terminating"++_, [Pid,{info,Msg},{mystate,gen_statem},error,{badmatch,b}|_]}). @@ -176,7 +182,7 @@ sasl_reports(Config) -> ok = gen_server:cast(ChPid, fun() -> spawn_link(fun() -> receive x->ok end end) end), - Msg = fun() -> a=b end, + Msg = fun() -> erlang:error({badmatch,b}) end, ok = gen_server:cast(ChPid,Msg), ?check_no_flush({error,"** Generic server ~tp terminating"++_, [ChPid,{'$gen_cast',Msg},gen_server,{{badmatch,b},_}]}), diff --git a/lib/kernel/test/logger_simple_h_SUITE.erl b/lib/kernel/test/logger_simple_h_SUITE.erl index 271a2126de..79e5c057ad 100644 --- a/lib/kernel/test/logger_simple_h_SUITE.erl +++ b/lib/kernel/test/logger_simple_h_SUITE.erl @@ -48,7 +48,7 @@ suite() -> {ct_hooks, [logger_test_lib]}]. init_per_suite(Config) -> - #{handlers:=Hs0} = logger:i(), + Hs0 = logger:get_handler_config(), Hs = lists:keydelete(cth_log_redirect,1,Hs0), [ok = logger:remove_handler(Id) || {Id,_,_} <- Hs], Env = [{App,Key,application:get_env(App,Key)} || @@ -107,15 +107,15 @@ start_stop(cleanup,_Config) -> replace_default(Config) -> {ok, _, Node} = logger_test_lib:setup(Config, [{logger, [{handler, default, undefined}]}]), - log(Node, emergency, [M1=?str]), - log(Node, alert, [M2=?str,[]]), - log(Node, error, [M3=?map_rep]), - log(Node, info, [M4=?keyval_rep]), - log(Node, info, [M41=?keyval_rep++[not_key_val]]), - rpc:call(Node, error_logger, error_report, [some_type,M5=?map_rep]), - rpc:call(Node, error_logger, warning_report, ["some_type",M6=?map_rep]), - log(Node, critical, [M7=?str,[A7=?keyval_rep]]), - log(Node, notice, [M8=["fake",string,"line:",?LINE]]), + log(Node, emergency, [?str]), + log(Node, alert, [?str,[]]), + log(Node, error, [?map_rep]), + log(Node, info, [?keyval_rep]), + log(Node, info, [?keyval_rep++[not_key_val]]), + rpc:call(Node, error_logger, error_report, [some_type,?map_rep]), + rpc:call(Node, error_logger, warning_report, ["some_type",?map_rep]), + log(Node, critical, [?str,[?keyval_rep]]), + log(Node, notice, [["fake",string,"line:",?LINE]]), Env = rpc:call(Node, application, get_env, [kernel, logger, []]), ok = rpc:call(Node, logger, add_handlers, [Env]), @@ -127,18 +127,18 @@ replace_file(Config) -> {ok, _, Node} = logger_test_lib:setup(Config, [{logger, [{handler, default, undefined}]}]), log(Node, emergency, [M1=?str]), log(Node, alert, [M2=?str,[]]), - log(Node, error, [M3=?map_rep]), - log(Node, info, [M4=?keyval_rep]), - log(Node, info, [M41=?keyval_rep++[not_key_val]]), - log(Node, critical, [M7=?str,[A7=?keyval_rep]]), - log(Node, notice, [M8=["fake",string,"line:",?LINE]]), + log(Node, error, [?map_rep]), + log(Node, warning, [?keyval_rep]), + log(Node, warning, [?keyval_rep++[not_key_val]]), + log(Node, critical, [?str,[?keyval_rep]]), + log(Node, notice, [["fake",string,"line:",?LINE]]), File = filename:join(proplists:get_value(priv_dir,Config), atom_to_list(?FUNCTION_NAME)++".log"), ok = rpc:call(Node, logger, add_handlers, [[{handler, default, logger_std_h, - #{ logger_std_h => #{ type => {file, File} }, + #{ config => #{ type => {file, File} }, formatter => {?DEFAULT_FORMATTER,?DEFAULT_FORMAT_CONFIG}}}]]), {ok,Bin} = sync_and_read(Node, file, File), @@ -151,10 +151,10 @@ replace_file(Config) -> "=ERROR REPORT===="++_, _, _, - "=INFO REPORT===="++_, + "=WARNING REPORT===="++_, _, _, - "=INFO REPORT===="++_, + "=WARNING REPORT===="++_, _, _, _, @@ -171,18 +171,18 @@ replace_disk_log(Config) -> {ok, _, Node} = logger_test_lib:setup(Config, [{logger, [{handler, default, undefined}]}]), log(Node, emergency, [M1=?str]), log(Node, alert, [M2=?str,[]]), - log(Node, error, [M3=?map_rep]), - log(Node, info, [M4=?keyval_rep]), - log(Node, info, [M41=?keyval_rep++[not_key_val]]), - log(Node, critical, [M7=?str,[A7=?keyval_rep]]), - log(Node, notice, [M8=["fake",string,"line:",?LINE]]), + log(Node, error, [?map_rep]), + log(Node, warning, [?keyval_rep]), + log(Node, warning, [?keyval_rep++[not_key_val]]), + log(Node, critical, [?str,[?keyval_rep]]), + log(Node, notice, [["fake",string,"line:",?LINE]]), File = filename:join(proplists:get_value(priv_dir,Config), atom_to_list(?FUNCTION_NAME)++".log"), ok = rpc:call(Node, logger, add_handlers, [[{handler, default, logger_disk_log_h, - #{ disk_log_opts => #{ file => File }, + #{ config => #{ file => File }, formatter => {?DEFAULT_FORMATTER,?DEFAULT_FORMAT_CONFIG}}}]]), {ok,Bin} = sync_and_read(Node, disk_log, File), Lines = [unicode:characters_to_list(L) || @@ -194,10 +194,10 @@ replace_disk_log(Config) -> "=ERROR REPORT===="++_, _, _, - "=INFO REPORT===="++_, + "=WARNING REPORT===="++_, _, _, - "=INFO REPORT===="++_, + "=WARNING REPORT===="++_, _, _, _, diff --git a/lib/kernel/test/logger_std_h_SUITE.erl b/lib/kernel/test/logger_std_h_SUITE.erl index 5764abd063..0930cd4211 100644 --- a/lib/kernel/test/logger_std_h_SUITE.erl +++ b/lib/kernel/test/logger_std_h_SUITE.erl @@ -45,17 +45,13 @@ -define(bin(Msg), list_to_binary(Msg++"\n")). -define(domain,#{domain=>[?MODULE]}). --define(FILESYNC_REP_INT, if is_atom(?FILESYNC_REPEAT_INTERVAL) -> 5500; - true -> ?FILESYNC_REPEAT_INTERVAL + 500 - end). - suite() -> [{timetrap,{seconds,30}}, {ct_hooks,[logger_test_lib]}]. init_per_suite(Config) -> timer:start(), % to avoid progress report - {ok,{logger_std_h,#{formatter:=OrigFormatter}}} = + {ok,#{formatter:=OrigFormatter}} = logger:get_handler_config(?STANDARD_HANDLER), [{formatter,OrigFormatter}|Config]. @@ -73,12 +69,25 @@ end_per_group(_Group, _Config) -> init_per_testcase(TestHooksCase, Config) when TestHooksCase == write_failure; TestHooksCase == sync_failure -> - if ?TEST_HOOKS_TAB == undefined -> + case (fun() -> ?TEST_HOOKS_TAB == undefined end)() of + true -> {skip,"Define the TEST_HOOKS macro to run this test"}; - true -> + false -> ct:print("********** ~w **********", [TestHooksCase]), Config end; +init_per_testcase(OPCase, Config) when + OPCase == qlen_kill_new; + OPCase == restart_after -> + case re:run(erlang:system_info(system_version), + "dirty-schedulers-TEST", + [{capture,none}]) of + match -> + {skip,"Overload protection test skipped on dirty-schedulers-TEST"}; + nomatch -> + ct:print("********** ~w **********", [OPCase]), + Config + end; init_per_testcase(TestCase, Config) -> ct:print("********** ~w **********", [TestCase]), Config. @@ -108,7 +117,7 @@ all() -> info_and_reset, reconfig, file_opts, - filesync, + sync, write_failure, sync_failure, op_switch_to_sync_file, @@ -132,7 +141,7 @@ all() -> add_remove_instance_tty(_Config) -> {error,{handler_not_added,{invalid_config,logger_std_h,{type,tty}}}} = logger:add_handler(?MODULE,logger_std_h, - #{logger_std_h => #{type => tty}, + #{config => #{type => tty}, filter_default=>log, formatter=>{?MODULE,self()}}), ok. @@ -166,33 +175,33 @@ add_remove_instance_file2(cleanup,_Config) -> add_remove_instance_file(Log, Type) -> ok = logger:add_handler(?MODULE, logger_std_h, - #{logger_std_h => #{type => Type}, + #{config => #{type => Type}, filter_default=>stop, filters=>?DEFAULT_HANDLER_FILTERS([?MODULE]), formatter=>{?MODULE,self()}}), - Pid = whereis(?MODULE), + Pid = whereis(h_proc_name()), true = is_pid(Pid), - logger:info(M1=?msg,?domain), + logger:notice(M1=?msg,?domain), ?check(M1), B1 = ?bin(M1), - try_read_file(Log, {ok,B1}, ?FILESYNC_REP_INT), + try_read_file(Log, {ok,B1}, filesync_rep_int()), ok = logger:remove_handler(?MODULE), timer:sleep(500), - undefined = whereis(?MODULE), - logger:info(?msg,?domain), + undefined = whereis(h_proc_name()), + logger:notice(?msg,?domain), ?check_no_log, - try_read_file(Log, {ok,B1}, ?FILESYNC_REP_INT), + try_read_file(Log, {ok,B1}, filesync_rep_int()), ok. default_formatter(_Config) -> ok = logger:set_handler_config(?STANDARD_HANDLER,formatter, {?DEFAULT_FORMATTER,?DEFAULT_FORMAT_CONFIG}), ct:capture_start(), - logger:info(M1=?msg), + logger:notice(M1=?msg), timer:sleep(100), ct:capture_stop(), [Msg] = ct:capture_get(), - match = re:run(Msg,"=INFO REPORT====.*\n"++M1,[{capture,none}]), + match = re:run(Msg,"=NOTICE REPORT====.*\n"++M1,[{capture,none}]), ok. errors(Config) -> @@ -212,20 +221,26 @@ errors(Config) -> {handler_not_added, {invalid_config,logger_std_h,{type,faulty_type}}}} = logger:add_handler(?MODULE,logger_std_h, - #{logger_std_h => #{type => faulty_type}}), + #{config => #{type => faulty_type}}), - NoDir = lists:concat(["/",?MODULE,"_dir"]), - {error, - {handler_not_added,{{open_failed,NoDir,eacces},_}}} = - logger:add_handler(myh2,logger_std_h, - #{logger_std_h=>#{type=>{file,NoDir}}}), + case os:type() of + {win32,_} -> + %% No use in testing file access on windows + ok; + _ -> + NoDir = lists:concat(["/",?MODULE,"_dir"]), + {error, + {handler_not_added,{{open_failed,NoDir,eacces},_}}} = + logger:add_handler(myh2,logger_std_h, + #{config=>#{type=>{file,NoDir}}}) + end, {error, {handler_not_added,{{open_failed,Log,_},_}}} = logger:add_handler(myh3,logger_std_h, - #{logger_std_h=>#{type=>{file,Log,[bad_file_opt]}}}), + #{config=>#{type=>{file,Log,[bad_file_opt]}}}), - ok = logger:info(?msg). + ok = logger:notice(?msg). errors(cleanup,_Config) -> logger:remove_handler(?MODULE). @@ -237,43 +252,40 @@ formatter_fail(Config) -> %% no formatter ok = logger:add_handler(?MODULE, logger_std_h, - #{logger_std_h => #{type => {file,Log}}, + #{config => #{type => {file,Log}}, filter_default=>stop, filters=>?DEFAULT_HANDLER_FILTERS([?MODULE])}), - Pid = whereis(?MODULE), + Pid = whereis(h_proc_name()), true = is_pid(Pid), - #{handlers:=HC1} = logger:i(), - H = [Id || {Id,_,_} <- HC1], + H = logger:get_handler_ids(), true = lists:member(?MODULE,H), %% Formatter is added automatically - {ok,{_,#{formatter:={logger_formatter,_}}}} = - logger:get_handler_config(?MODULE), - logger:info(M1=?msg,?domain), - Got1 = try_match_file(Log,"[0-9\\+\\-T:\\.]* info: "++M1,5000), + {ok,#{formatter:={logger_formatter,_}}} = logger:get_handler_config(?MODULE), + logger:notice(M1=?msg,?domain), + Got1 = try_match_file(Log,"[0-9\\+\\-T:\\.]* notice: "++M1,5000), ok = logger:set_handler_config(?MODULE,formatter,{nonexistingmodule,#{}}), - logger:info(M2=?msg,?domain), + logger:notice(M2=?msg,?domain), Got2 = try_match_file(Log, - escape(Got1)++"[0-9\\+\\-T:\\.]* info: FORMATTER CRASH: .*"++M2, + escape(Got1)++"[0-9\\+\\-T:\\.]* notice: FORMATTER CRASH: .*"++M2, 5000), ok = logger:set_handler_config(?MODULE,formatter,{?MODULE,crash}), - logger:info(M3=?msg,?domain), + logger:notice(M3=?msg,?domain), Got3 = try_match_file(Log, - escape(Got2)++"[0-9\\+\\-T:\\.]* info: FORMATTER CRASH: .*"++M3, + escape(Got2)++"[0-9\\+\\-T:\\.]* notice: FORMATTER CRASH: .*"++M3, 5000), ok = logger:set_handler_config(?MODULE,formatter,{?MODULE,bad_return}), - logger:info(?msg,?domain), + logger:notice(?msg,?domain), try_match_file(Log, escape(Got3)++"FORMATTER ERROR: bad_return_value", 5000), %% Check that handler is still alive and was never dead - Pid = whereis(?MODULE), - #{handlers:=HC2} = logger:i(), - H = [Id || {Id,_,_} <- HC2], + Pid = whereis(h_proc_name()), + H = logger:get_handler_ids(), ok. @@ -283,41 +295,41 @@ formatter_fail(cleanup,_Config) -> config_fail(_Config) -> {error,{handler_not_added,{invalid_config,logger_std_h,{bad,bad}}}} = logger:add_handler(?MODULE,logger_std_h, - #{logger_std_h => #{bad => bad}, + #{config => #{bad => bad}, filter_default=>log, formatter=>{?MODULE,self()}}), {error,{handler_not_added,{invalid_config,logger_std_h, {restart_type,bad}}}} = logger:add_handler(?MODULE,logger_std_h, - #{logger_std_h => #{restart_type => bad}, + #{config => #{restart_type => bad}, filter_default=>log, formatter=>{?MODULE,self()}}), {error,{handler_not_added,{invalid_levels,{_,1,_}}}} = logger:add_handler(?MODULE,logger_std_h, - #{logger_std_h => #{drop_new_reqs_qlen=>1}}), + #{config => #{drop_mode_qlen=>1}}), {error,{handler_not_added,{invalid_levels,{43,42,_}}}} = logger:add_handler(?MODULE,logger_std_h, - #{logger_std_h => #{toggle_sync_qlen=>43, - drop_new_reqs_qlen=>42}}), + #{config => #{sync_mode_qlen=>43, + drop_mode_qlen=>42}}), {error,{handler_not_added,{invalid_levels,{_,43,42}}}} = logger:add_handler(?MODULE,logger_std_h, - #{logger_std_h => #{drop_new_reqs_qlen=>43, - flush_reqs_qlen=>42}}), + #{config => #{drop_mode_qlen=>43, + flush_qlen=>42}}), ok = logger:add_handler(?MODULE,logger_std_h, #{filter_default=>log, formatter=>{?MODULE,self()}}), {error,{illegal_config_change,_,_}} = - logger:set_handler_config(?MODULE,logger_std_h, + logger:set_handler_config(?MODULE,config, #{type=>{file,"file"}}), {error,{illegal_config_change,_,_}} = logger:set_handler_config(?MODULE,id,bad), {error,{invalid_levels,_}} = - logger:set_handler_config(?MODULE,logger_std_h, - #{toggle_sync_qlen=>100, - flush_reqs_qlen=>99}), + logger:set_handler_config(?MODULE,config, + #{sync_mode_qlen=>100, + flush_qlen=>99}), {error,{invalid_config,logger_std_h,{filesync_rep_int,2000}}} = - logger:set_handler_config(?MODULE, logger_std_h, + logger:set_handler_config(?MODULE, config, #{filesync_rep_int => 2000}), ok. @@ -329,7 +341,7 @@ crash_std_h_to_file(Config) -> Log = filename:join(Dir,lists:concat([?MODULE,"_",?FUNCTION_NAME,".log"])), crash_std_h(Config,?FUNCTION_NAME, [{handler,default,logger_std_h, - #{ logger_std_h => #{ type => {file, Log} }}}], + #{ config => #{ type => {file, Log} }}}], file, Log). crash_std_h_to_file(cleanup,_Config) -> crash_std_h(cleanup). @@ -339,7 +351,7 @@ crash_std_h_to_disk_log(Config) -> Log = filename:join(Dir,lists:concat([?MODULE,"_",?FUNCTION_NAME,".log"])), crash_std_h(Config,?FUNCTION_NAME, [{handler,default,logger_disk_log_h, - #{ disk_log_opts => #{ file => Log }}}], + #{ config => #{ file => Log }}}], disk_log,Log). crash_std_h_to_disk_log(cleanup,_Config) -> crash_std_h(cleanup). @@ -354,7 +366,12 @@ crash_std_h(Config,Func,Var,Type,Log) -> ct:pal("Starting ~p with ~tp", [Name,Args]), %% Start a node which prints kernel logs to the destination specified by Type {ok,Node} = test_server:start_node(Name, peer, [{args, Args}]), - Pid = rpc:call(Node,erlang,whereis,[?STANDARD_HANDLER]), + HProcName = + case Type of + file -> ?name_to_reg_name(logger_std_h,?STANDARD_HANDLER); + disk_log -> ?name_to_reg_name(logger_disk_log_h,?STANDARD_HANDLER) + end, + Pid = rpc:call(Node,erlang,whereis,[HProcName]), ok = rpc:call(Node,logger,set_handler_config,[?STANDARD_HANDLER,formatter, {?MODULE,self()}]), ok = log_on_remote_node(Node,"dummy1"), @@ -367,7 +384,7 @@ crash_std_h(Config,Func,Var,Type,Log) -> %% Wait a bit, then check that it is gone timer:sleep(2000), - undefined = rpc:call(Node,erlang,whereis,[?STANDARD_HANDLER]), + undefined = rpc:call(Node,erlang,whereis,[HProcName]), %% Check that file is not empty {ok,Bin2} = sync_and_read(Node,Type,Log), @@ -381,7 +398,7 @@ crash_std_h(Config,Func,Var,Type,Log) -> log_on_remote_node(Node,Msg) -> _ = spawn_link(Node, fun() -> erlang:group_leader(whereis(user),self()), - logger:info(Msg) + logger:notice(Msg) end), ok. @@ -391,7 +408,7 @@ crash_std_h(cleanup) -> [test_server:stop_node(Node) || Node <- Nodes]. sync_and_read(Node,disk_log,Log) -> - rpc:call(Node,logger_disk_log_h,disk_log_sync,[?STANDARD_HANDLER]), + rpc:call(Node,logger_disk_log_h,filesync,[?STANDARD_HANDLER]), case file:read_file(Log ++ ".1") of {ok,<<>>} -> timer:sleep(5000), @@ -423,51 +440,51 @@ reconfig(Config) -> Dir = ?config(priv_dir,Config), ok = logger:add_handler(?MODULE, logger_std_h, - #{logger_std_h => #{type => standard_io}, + #{config => #{type => standard_io}, filter_default=>log, filters=>?DEFAULT_HANDLER_FILTERS([?MODULE]), formatter=>{?MODULE,self()}}), #{id := ?MODULE, type := standard_io, file_ctrl_pid := FileCtrlPid, - toggle_sync_qlen := ?TOGGLE_SYNC_QLEN, - drop_new_reqs_qlen := ?DROP_NEW_REQS_QLEN, - flush_reqs_qlen := ?FLUSH_REQS_QLEN, - enable_burst_limit := ?ENABLE_BURST_LIMIT, - burst_limit_size := ?BURST_LIMIT_SIZE, - burst_window_time := ?BURST_WINDOW_TIME, - enable_kill_overloaded := ?ENABLE_KILL_OVERLOADED, - handler_overloaded_qlen := ?HANDLER_OVERLOADED_QLEN, - handler_overloaded_mem := ?HANDLER_OVERLOADED_MEM, - handler_restart_after := ?HANDLER_RESTART_AFTER, + sync_mode_qlen := ?SYNC_MODE_QLEN, + drop_mode_qlen := ?DROP_MODE_QLEN, + flush_qlen := ?FLUSH_QLEN, + burst_limit_enable := ?BURST_LIMIT_ENABLE, + burst_limit_max_count := ?BURST_LIMIT_MAX_COUNT, + burst_limit_window_time := ?BURST_LIMIT_WINDOW_TIME, + overload_kill_enable := ?OVERLOAD_KILL_ENABLE, + overload_kill_qlen := ?OVERLOAD_KILL_QLEN, + overload_kill_mem_size := ?OVERLOAD_KILL_MEM_SIZE, + overload_kill_restart_after := ?OVERLOAD_KILL_RESTART_AFTER, filesync_repeat_interval := ?FILESYNC_REPEAT_INTERVAL} = logger_std_h:info(?MODULE), - ok = logger:set_handler_config(?MODULE, logger_std_h, - #{toggle_sync_qlen => 1, - drop_new_reqs_qlen => 2, - flush_reqs_qlen => 3, - enable_burst_limit => false, - burst_limit_size => 10, - burst_window_time => 10, - enable_kill_overloaded => true, - handler_overloaded_qlen => 100000, - handler_overloaded_mem => 10000000, - handler_restart_after => never, + ok = logger:set_handler_config(?MODULE, config, + #{sync_mode_qlen => 1, + drop_mode_qlen => 2, + flush_qlen => 3, + burst_limit_enable => false, + burst_limit_max_count => 10, + burst_limit_window_time => 10, + overload_kill_enable => true, + overload_kill_qlen => 100000, + overload_kill_mem_size => 10000000, + overload_kill_restart_after => infinity, filesync_repeat_interval => no_repeat}), #{id := ?MODULE, type := standard_io, file_ctrl_pid := FileCtrlPid, - toggle_sync_qlen := 1, - drop_new_reqs_qlen := 2, - flush_reqs_qlen := 3, - enable_burst_limit := false, - burst_limit_size := 10, - burst_window_time := 10, - enable_kill_overloaded := true, - handler_overloaded_qlen := 100000, - handler_overloaded_mem := 10000000, - handler_restart_after := never, + sync_mode_qlen := 1, + drop_mode_qlen := 2, + flush_qlen := 3, + burst_limit_enable := false, + burst_limit_max_count := 10, + burst_limit_window_time := 10, + overload_kill_enable := true, + overload_kill_qlen := 100000, + overload_kill_mem_size := 10000000, + overload_kill_restart_after := infinity, filesync_repeat_interval := no_repeat} = logger_std_h:info(?MODULE), ok. @@ -482,34 +499,34 @@ file_opts(Config) -> BadType = {file,Log,BadFileOpts}, {error,{handler_not_added,{{open_failed,Log,enoent},_}}} = logger:add_handler(?MODULE, logger_std_h, - #{logger_std_h => #{type => BadType}}), + #{config => #{type => BadType}}), OkFileOpts = [raw,append], OkType = {file,Log,OkFileOpts}, ok = logger:add_handler(?MODULE, logger_std_h, - #{logger_std_h => #{type => OkType}, + #{config => #{type => OkType}, filter_default=>log, filters=>?DEFAULT_HANDLER_FILTERS([?MODULE]), formatter=>{?MODULE,self()}}), #{type := OkType} = logger_std_h:info(?MODULE), - logger:info(M1=?msg,?domain), + logger:notice(M1=?msg,?domain), ?check(M1), B1 = ?bin(M1), - try_read_file(Log, {ok,B1}, ?FILESYNC_REP_INT), + try_read_file(Log, {ok,B1}, filesync_rep_int()), ok. file_opts(cleanup, _Config) -> logger:remove_handler(?MODULE). -filesync(Config) -> +sync(Config) -> Dir = ?config(priv_dir,Config), Log = filename:join(Dir, lists:concat([?FUNCTION_NAME,".log"])), Type = {file,Log}, ok = logger:add_handler(?MODULE, logger_std_h, - #{logger_std_h => #{type => Type}, + #{config => #{type => Type}, filter_default=>log, filters=>?DEFAULT_HANDLER_FILTERS([?MODULE]), formatter=>{?MODULE,nl}}), @@ -522,9 +539,9 @@ filesync(Config) -> {logger_std_h, sync_dev}, {file,datasync}]), - logger:info("first", ?domain), + logger:notice("first", ?domain), %% wait for automatic filesync - check_tracer(?FILESYNC_REP_INT*2), + check_tracer(filesync_rep_int()*2), %% check that explicit filesync is only done once start_tracer([{logger_std_h, write_to_dev, 5}, @@ -535,16 +552,16 @@ filesync(Config) -> {file,datasync}, {no_more,500} ]), - logger:info("second", ?domain), - %% do explicit filesync + logger:notice("second", ?domain), + %% do explicit sync logger_std_h:filesync(?MODULE), - %% a second filesync should be ignored + %% a second sync should be ignored logger_std_h:filesync(?MODULE), check_tracer(100), %% check that if there's no repeated filesync active, %% a filesync is still performed when handler goes idle - logger:set_handler_config(?MODULE, logger_std_h, + logger:set_handler_config(?MODULE, config, #{filesync_repeat_interval => no_repeat}), no_repeat = maps:get(filesync_repeat_interval, logger_std_h:info(?MODULE)), %% The following timer is to make sure the time from last log @@ -560,10 +577,10 @@ filesync(Config) -> {logger_std_h, write_to_dev, <<"fourth\n">>}, {logger_std_h, sync_dev}, {file,datasync}]), - logger:info("third", ?domain), + logger:notice("third", ?domain), %% wait for automatic filesync timer:sleep(?IDLE_DETECT_TIME_MSEC*2), - logger:info("fourth", ?domain), + logger:notice("fourth", ?domain), %% wait for automatic filesync check_tracer(?IDLE_DETECT_TIME_MSEC*2), @@ -575,15 +592,15 @@ filesync(Config) -> start_tracer([{logger_std_h,handle_cast,2}], [OneSync || _ <- lists:seq(1, 1 + trunc(WaitT/SyncInt))]), - logger:set_handler_config(?MODULE, logger_std_h, + logger:set_handler_config(?MODULE, config, #{filesync_repeat_interval => SyncInt}), SyncInt = maps:get(filesync_repeat_interval, logger_std_h:info(?MODULE)), timer:sleep(WaitT), - logger:set_handler_config(?MODULE, logger_std_h, + logger:set_handler_config(?MODULE, config, #{filesync_repeat_interval => no_repeat}), check_tracer(100), ok. -filesync(cleanup, _Config) -> +sync(cleanup, _Config) -> dbg:stop_clear(), logger:remove_handler(?MODULE). @@ -600,7 +617,7 @@ write_failure(Config) -> ok = log_on_remote_node(Node, "Logged1"), rpc:call(Node, logger_std_h, filesync, [?STANDARD_HANDLER]), ?check_no_log, - try_read_file(Log, {ok,<<"Logged1\n">>}, ?FILESYNC_REP_INT), + try_read_file(Log, {ok,<<"Logged1\n">>}, filesync_rep_int()), rpc:call(Node, ?MODULE, set_result, [file_write,{error,terminated}]), ok = log_on_remote_node(Node, "Cause simple error printout"), @@ -618,7 +635,7 @@ write_failure(Config) -> ok = log_on_remote_node(Node, "Logged2"), rpc:call(Node, logger_std_h, filesync, [?STANDARD_HANDLER]), ?check_no_log, - try_read_file(Log, {ok,<<"Logged1\nLogged2\n">>}, ?FILESYNC_REP_INT), + try_read_file(Log, {ok,<<"Logged1\nLogged2\n">>}, filesync_rep_int()), ok. write_failure(cleanup, _Config) -> Nodes = nodes(), @@ -636,7 +653,7 @@ sync_failure(Config) -> SyncInt = 500, ok = rpc:call(Node, logger, set_handler_config, - [?STANDARD_HANDLER, logger_std_h, + [?STANDARD_HANDLER, config, #{filesync_repeat_interval => SyncInt}]), Info = rpc:call(Node, logger_std_h, info, [?STANDARD_HANDLER]), SyncInt = maps:get(filesync_repeat_interval, Info), @@ -669,16 +686,16 @@ start_std_h_on_new_node(Config, Log) -> logger_test_lib:setup( Config, [{logger,[{handler,default,logger_std_h, - #{ logger_std_h => #{ type => {file,Log}}}}]}]), + #{ config => #{ type => {file,Log}}}}]}]), ok = rpc:call(Node,logger,set_handler_config,[?STANDARD_HANDLER,formatter, {?MODULE,nl}]), Node. %% functions for test hook macros to be called by rpc -set_internal_log(Mod, Func) -> - ?set_internal_log({Mod,Func}). -set_result(Op, Result) -> - ?set_result(Op, Result). +set_internal_log(_Mod, _Func) -> + ?set_internal_log({_Mod,_Func}). +set_result(_Op, _Result) -> + ?set_result(_Op, _Result). set_defaults() -> ?set_defaults(). @@ -697,15 +714,14 @@ op_switch_to_sync_file(Config) -> {Log,HConfig,StdHConfig} = start_handler(?MODULE, ?FUNCTION_NAME, Config), NumOfReqs = 500, NewHConfig = - HConfig#{logger_std_h => StdHConfig#{toggle_sync_qlen => 2, - drop_new_reqs_qlen => NumOfReqs+1, - flush_reqs_qlen => 2*NumOfReqs, - enable_burst_limit => false}}, + HConfig#{config => StdHConfig#{sync_mode_qlen => 2, + drop_mode_qlen => NumOfReqs+1, + flush_qlen => 2*NumOfReqs, + burst_limit_enable => false}}, ok = logger:set_handler_config(?MODULE, NewHConfig), %% TRecvPid = start_op_trace(), - send_burst({n,NumOfReqs}, seq, {chars,79}, info), + send_burst({n,NumOfReqs}, seq, {chars,79}, notice), Lines = count_lines(Log), - ok = file:delete(Log), %% true = analyse_trace(TRecvPid, %% fun(Events) -> find_mode(async,Events) end), %% true = analyse_trace(TRecvPid, @@ -718,6 +734,7 @@ op_switch_to_sync_file(Config) -> %% fun(Events) -> find_mode(flush,Events) end), %% stop_op_trace(TRecvPid), NumOfReqs = Lines, + ok = file_delete(Log), ok. op_switch_to_sync_file(cleanup, _Config) -> ok = stop_handler(?MODULE). @@ -726,12 +743,12 @@ op_switch_to_sync_tty(Config) -> {HConfig,StdHConfig} = start_handler(?MODULE, standard_io, Config), NumOfReqs = 500, NewHConfig = - HConfig#{logger_std_h => StdHConfig#{toggle_sync_qlen => 3, - drop_new_reqs_qlen => NumOfReqs+1, - flush_reqs_qlen => 2*NumOfReqs, - enable_burst_limit => false}}, + HConfig#{config => StdHConfig#{sync_mode_qlen => 3, + drop_mode_qlen => NumOfReqs+1, + flush_qlen => 2*NumOfReqs, + burst_limit_enable => false}}, ok = logger:set_handler_config(?MODULE, NewHConfig), - send_burst({n,NumOfReqs}, seq, {chars,79}, info), + send_burst({n,NumOfReqs}, seq, {chars,79}, notice), ok. op_switch_to_sync_tty(cleanup, _Config) -> ok = stop_handler(?MODULE). @@ -747,26 +764,26 @@ op_switch_to_drop_file(Config) -> Procs = 2, Bursts = 10, NewHConfig = - HConfig#{logger_std_h => - StdHConfig#{toggle_sync_qlen => 1, - drop_new_reqs_qlen => 2, - flush_reqs_qlen => + HConfig#{config => + StdHConfig#{sync_mode_qlen => 1, + drop_mode_qlen => 2, + flush_qlen => Procs*NumOfReqs*Bursts, - enable_burst_limit => false}}, + burst_limit_enable => false}}, ok = logger:set_handler_config(?MODULE, NewHConfig), %% It sometimes happens that the handler gets the %% requests in a slow enough pace so that dropping %% never occurs. Therefore, lets generate a number of %% bursts to increase the chance of message buildup. - [send_burst({n,NumOfReqs}, {spawn,Procs,0}, {chars,79}, info) || + [send_burst({n,NumOfReqs}, {spawn,Procs,0}, {chars,79}, notice) || _ <- lists:seq(1, Bursts)], Logged = count_lines(Log), ok = stop_handler(?MODULE), - _ = file:delete(Log), ct:pal("Number of messages dropped = ~w (~w)", [Procs*NumOfReqs*Bursts-Logged,Procs*NumOfReqs*Bursts]), true = (Logged < (Procs*NumOfReqs*Bursts)), true = (Logged > 0), + _ = file_delete(Log), ok end, %% As it's tricky to get the timing right in only one go, we perform the @@ -785,19 +802,19 @@ op_switch_to_drop_tty(Config) -> NumOfReqs = 300, Procs = 2, NewHConfig = - HConfig#{logger_std_h => StdHConfig#{toggle_sync_qlen => 1, - drop_new_reqs_qlen => 2, - flush_reqs_qlen => - Procs*NumOfReqs+1, - enable_burst_limit => false}}, + HConfig#{config => StdHConfig#{sync_mode_qlen => 1, + drop_mode_qlen => 2, + flush_qlen => + Procs*NumOfReqs+1, + burst_limit_enable => false}}, ok = logger:set_handler_config(?MODULE, NewHConfig), - send_burst({n,NumOfReqs}, {spawn,Procs,0}, {chars,79}, info), + send_burst({n,NumOfReqs}, {spawn,Procs,0}, {chars,79}, notice), ok. op_switch_to_drop_tty(cleanup, _Config) -> ok = stop_handler(?MODULE). op_switch_to_flush_file() -> - [{timetrap,{minutes,3}}]. + [{timetrap,{minutes,5}}]. op_switch_to_flush_file(Config) -> Test = fun() -> @@ -809,12 +826,12 @@ op_switch_to_flush_file(Config) -> %% (verify with coverage of flush_log_requests/2) NewHConfig = - HConfig#{logger_std_h => - StdHConfig#{toggle_sync_qlen => 2, + HConfig#{config => + StdHConfig#{sync_mode_qlen => 2, %% disable drop mode - drop_new_reqs_qlen => 300, - flush_reqs_qlen => 300, - enable_burst_limit => false}}, + drop_mode_qlen => 300, + flush_qlen => 300, + burst_limit_enable => false}}, ok = logger:set_handler_config(?MODULE, NewHConfig), NumOfReqs = 1500, Procs = 10, @@ -826,15 +843,15 @@ op_switch_to_flush_file(Config) -> %% sync messages gets tested). Therefore, lets %% generate a number of bursts to increase the chance %% of message buildup in some random fashion. - [send_burst({n,NumOfReqs}, {spawn,Procs,0}, {chars,79}, info) || + [send_burst({n,NumOfReqs}, {spawn,Procs,0}, {chars,79}, notice) || _ <- lists:seq(1,Bursts)], Logged = count_lines(Log), ok = stop_handler(?MODULE), - _ = file:delete(Log), ct:pal("Number of messages flushed/dropped = ~w (~w)", [NumOfReqs*Procs*Bursts-Logged,NumOfReqs*Procs*Bursts]), true = (Logged < (NumOfReqs*Procs*Bursts)), true = (Logged > 0), + _ = file_delete(Log), ok end, %% As it's tricky to get the timing right in only one go, we perform the @@ -848,6 +865,8 @@ op_switch_to_flush_file(Config) -> op_switch_to_flush_file(cleanup, _Config) -> _ = stop_handler(?MODULE). +op_switch_to_flush_tty() -> + [{timetrap,{minutes,5}}]. op_switch_to_flush_tty(Config) -> {HConfig,StdHConfig} = start_handler(?MODULE, standard_io, Config), @@ -855,15 +874,15 @@ op_switch_to_flush_tty(Config) -> %% when the flush happens (verify with coverage of flush_log_requests/2) NewHConfig = - HConfig#{logger_std_h => StdHConfig#{toggle_sync_qlen => 2, - %% disable drop mode - drop_new_reqs_qlen => 100, - flush_reqs_qlen => 100, - enable_burst_limit => false}}, + HConfig#{config => StdHConfig#{sync_mode_qlen => 2, + %% disable drop mode + drop_mode_qlen => 100, + flush_qlen => 100, + burst_limit_enable => false}}, ok = logger:set_handler_config(?MODULE, NewHConfig), NumOfReqs = 1000, Procs = 100, - send_burst({n,NumOfReqs}, {spawn,Procs,0}, {chars,79}, info), + send_burst({n,NumOfReqs}, {spawn,Procs,0}, {chars,79}, notice), ok. op_switch_to_flush_tty(cleanup, _Config) -> ok = stop_handler(?MODULE). @@ -871,18 +890,19 @@ op_switch_to_flush_tty(cleanup, _Config) -> limit_burst_disabled(Config) -> {Log,HConfig,StdHConfig} = start_handler(?MODULE, ?FUNCTION_NAME, Config), NewHConfig = - HConfig#{logger_std_h => StdHConfig#{enable_burst_limit => false, - burst_limit_size => 10, - burst_window_time => 2000, - drop_new_reqs_qlen => 200, - flush_reqs_qlen => 300}}, + HConfig#{config => StdHConfig#{burst_limit_enable => false, + burst_limit_max_count => 10, + burst_limit_window_time => 2000, + drop_mode_qlen => 200, + flush_qlen => 300}}, ok = logger:set_handler_config(?MODULE, NewHConfig), NumOfReqs = 100, - send_burst({n,NumOfReqs}, seq, {chars,79}, info), + send_burst({n,NumOfReqs}, seq, {chars,79}, notice), Logged = count_lines(Log), ct:pal("Number of messages logged = ~w", [Logged]), - ok = file:delete(Log), - NumOfReqs = Logged. + NumOfReqs = Logged, + ok = file_delete(Log), + ok. limit_burst_disabled(cleanup, _Config) -> ok = stop_handler(?MODULE). @@ -890,18 +910,19 @@ limit_burst_enabled_one(Config) -> {Log,HConfig,StdHConfig} = start_handler(?MODULE, ?FUNCTION_NAME, Config), ReqLimit = 10, NewHConfig = - HConfig#{logger_std_h => StdHConfig#{enable_burst_limit => true, - burst_limit_size => ReqLimit, - burst_window_time => 2000, - drop_new_reqs_qlen => 200, - flush_reqs_qlen => 300}}, + HConfig#{config => StdHConfig#{burst_limit_enable => true, + burst_limit_max_count => ReqLimit, + burst_limit_window_time => 2000, + drop_mode_qlen => 200, + flush_qlen => 300}}, ok = logger:set_handler_config(?MODULE, NewHConfig), NumOfReqs = 100, - send_burst({n,NumOfReqs}, seq, {chars,79}, info), + send_burst({n,NumOfReqs}, seq, {chars,79}, notice), Logged = count_lines(Log), ct:pal("Number of messages logged = ~w", [Logged]), - ok = file:delete(Log), - ReqLimit = Logged. + ReqLimit = Logged, + ok = file_delete(Log), + ok. limit_burst_enabled_one(cleanup, _Config) -> ok = stop_handler(?MODULE). @@ -910,57 +931,58 @@ limit_burst_enabled_period(Config) -> ReqLimit = 10, BurstTWin = 1000, NewHConfig = - HConfig#{logger_std_h => StdHConfig#{enable_burst_limit => true, - burst_limit_size => ReqLimit, - burst_window_time => BurstTWin, - drop_new_reqs_qlen => 20000, - flush_reqs_qlen => 20001}}, + HConfig#{config => StdHConfig#{burst_limit_enable => true, + burst_limit_max_count => ReqLimit, + burst_limit_window_time => BurstTWin, + drop_mode_qlen => 20000, + flush_qlen => 20001}}, ok = logger:set_handler_config(?MODULE, NewHConfig), Windows = 3, - Sent = send_burst({t,BurstTWin*Windows}, seq, {chars,79}, info), + Sent = send_burst({t,BurstTWin*Windows}, seq, {chars,79}, notice), Logged = count_lines(Log), ct:pal("Number of messages sent = ~w~nNumber of messages logged = ~w", [Sent,Logged]), - ok = file:delete(Log), true = (Logged > (ReqLimit*Windows)) andalso - (Logged < (ReqLimit*(Windows+2))). + (Logged < (ReqLimit*(Windows+2))), + ok = file_delete(Log), + ok. limit_burst_enabled_period(cleanup, _Config) -> ok = stop_handler(?MODULE). kill_disabled(Config) -> {Log,HConfig,StdHConfig} = start_handler(?MODULE, ?FUNCTION_NAME, Config), NewHConfig = - HConfig#{logger_std_h=>StdHConfig#{enable_kill_overloaded=>false, - handler_overloaded_qlen=>10, - handler_overloaded_mem=>100}}, + HConfig#{config=>StdHConfig#{overload_kill_enable=>false, + overload_kill_qlen=>10, + overload_kill_mem_size=>100}}, ok = logger:set_handler_config(?MODULE, NewHConfig), NumOfReqs = 100, - send_burst({n,NumOfReqs}, seq, {chars,79}, info), + send_burst({n,NumOfReqs}, seq, {chars,79}, notice), Logged = count_lines(Log), ct:pal("Number of messages logged = ~w", [Logged]), - ok = file:delete(Log), - true = is_pid(whereis(?MODULE)), + ok = file_delete(Log), + true = is_pid(whereis(h_proc_name())), ok. kill_disabled(cleanup, _Config) -> ok = stop_handler(?MODULE). qlen_kill_new(Config) -> - {_Log,HConfig,StdHConfig} = start_handler(?MODULE, ?FUNCTION_NAME, Config), - Pid0 = whereis(?MODULE), + {Log,HConfig,StdHConfig} = start_handler(?MODULE, ?FUNCTION_NAME, Config), + Pid0 = whereis(h_proc_name()), {_,Mem0} = process_info(Pid0, memory), - RestartAfter = ?HANDLER_RESTART_AFTER, + RestartAfter = ?OVERLOAD_KILL_RESTART_AFTER, NewHConfig = - HConfig#{logger_std_h=>StdHConfig#{enable_kill_overloaded=>true, - handler_overloaded_qlen=>10, - handler_overloaded_mem=>Mem0+50000, - handler_restart_after=>RestartAfter}}, + HConfig#{config=>StdHConfig#{overload_kill_enable=>true, + overload_kill_qlen=>10, + overload_kill_mem_size=>Mem0+50000, + overload_kill_restart_after=>RestartAfter}}, ok = logger:set_handler_config(?MODULE, NewHConfig), MRef = erlang:monitor(process, Pid0), NumOfReqs = 100, - Procs = 2, - send_burst({n,NumOfReqs}, {spawn,Procs,0}, {chars,79}, info), - %% send_burst({n,NumOfReqs}, seq, {chars,79}, info), + Procs = 4, + send_burst({n,NumOfReqs}, {spawn,Procs,0}, {chars,79}, notice), + %% send_burst({n,NumOfReqs}, seq, {chars,79}, notice), receive {'DOWN', MRef, _, _, Info} -> case Info of @@ -969,8 +991,8 @@ qlen_kill_new(Config) -> killed -> ct:pal("Slow shutdown, handler process was killed!", []) end, - timer:sleep(RestartAfter + 2000), - true = is_pid(whereis(?MODULE)), + file_delete(Log), + {ok,_} = wait_for_process_up(RestartAfter * 3), ok after 5000 -> @@ -990,28 +1012,28 @@ qlen_kill_std(_Config) -> %% Log = filename:join(Dir, File), %% Node = start_std_h_on_new_node(Config, ?FUNCTION_NAME, Log), %% ok = rpc:call(Node, logger, set_handler_config, - %% [?STANDARD_HANDLER, logger_std_h, - %% #{enable_kill_overloaded=>true, - %% handler_overloaded_qlen=>10, - %% handler_overloaded_mem=>100000}]), + %% [?STANDARD_HANDLER, config, + %% #{overload_kill_enable=>true, + %% overload_kill_qlen=>10, + %% overload_kill_mem_size=>100000}]), {skip,"Not done yet"}. mem_kill_new(Config) -> - {_Log,HConfig,StdHConfig} = start_handler(?MODULE, ?FUNCTION_NAME, Config), - Pid0 = whereis(?MODULE), + {Log,HConfig,StdHConfig} = start_handler(?MODULE, ?FUNCTION_NAME, Config), + Pid0 = whereis(h_proc_name()), {_,Mem0} = process_info(Pid0, memory), - RestartAfter = ?HANDLER_RESTART_AFTER, + RestartAfter = ?OVERLOAD_KILL_RESTART_AFTER, NewHConfig = - HConfig#{logger_std_h=>StdHConfig#{enable_kill_overloaded=>true, - handler_overloaded_qlen=>50000, - handler_overloaded_mem=>Mem0+500, - handler_restart_after=>RestartAfter}}, + HConfig#{config=>StdHConfig#{overload_kill_enable=>true, + overload_kill_qlen=>50000, + overload_kill_mem_size=>Mem0+500, + overload_kill_restart_after=>RestartAfter}}, ok = logger:set_handler_config(?MODULE, NewHConfig), MRef = erlang:monitor(process, Pid0), NumOfReqs = 100, - Procs = 2, - send_burst({n,NumOfReqs}, {spawn,Procs,0}, {chars,79}, info), - %% send_burst({n,NumOfReqs}, seq, {chars,79}, info), + Procs = 4, + send_burst({n,NumOfReqs}, {spawn,Procs,0}, {chars,79}, notice), + %% send_burst({n,NumOfReqs}, seq, {chars,79}, notice), receive {'DOWN', MRef, _, _, Info} -> case Info of @@ -1020,8 +1042,8 @@ mem_kill_new(Config) -> killed -> ct:pal("Slow shutdown, handler process was killed!", []) end, - timer:sleep(RestartAfter * 3), - true = is_pid(whereis(?MODULE)), + file_delete(Log), + {ok,_} = wait_for_process_up(RestartAfter * 3), ok after 5000 -> @@ -1037,53 +1059,58 @@ mem_kill_new(cleanup, _Config) -> mem_kill_std(_Config) -> {skip,"Not done yet"}. +restart_after() -> + [{timetrap,{minutes,2}}]. restart_after(Config) -> {Log,HConfig,StdHConfig} = start_handler(?MODULE, ?FUNCTION_NAME, Config), NewHConfig1 = - HConfig#{logger_std_h=>StdHConfig#{enable_kill_overloaded=>true, - handler_overloaded_qlen=>10, - handler_restart_after=>never}}, + HConfig#{config=>StdHConfig#{overload_kill_enable=>true, + overload_kill_qlen=>10, + overload_kill_restart_after=>infinity}}, ok = logger:set_handler_config(?MODULE, NewHConfig1), - MRef1 = erlang:monitor(process, whereis(?MODULE)), + MRef1 = erlang:monitor(process, whereis(h_proc_name())), %% kill handler - send_burst({n,100}, {spawn,2,0}, {chars,79}, info), + send_burst({n,100}, {spawn,4,0}, {chars,79}, notice), receive - {'DOWN', MRef1, _, _, _Info1} -> - timer:sleep(?HANDLER_RESTART_AFTER + 1000), - undefined = whereis(?MODULE), + {'DOWN', MRef1, _, _, _Reason1} -> + file_delete(Log), + error = wait_for_process_up(?OVERLOAD_KILL_RESTART_AFTER * 3), ok after 5000 -> + Info1 = logger_std_h:info(?MODULE), + ct:pal("Handler state = ~p", [Info1]), ct:fail("Handler not dead! It should not have survived this!") end, {Log,_,_} = start_handler(?MODULE, ?FUNCTION_NAME, Config), - RestartAfter = ?HANDLER_RESTART_AFTER, + RestartAfter = ?OVERLOAD_KILL_RESTART_AFTER, NewHConfig2 = - HConfig#{logger_std_h=>StdHConfig#{enable_kill_overloaded=>true, - handler_overloaded_qlen=>10, - handler_restart_after=>RestartAfter}}, + HConfig#{config=>StdHConfig#{overload_kill_enable=>true, + overload_kill_qlen=>10, + overload_kill_restart_after=>RestartAfter}}, ok = logger:set_handler_config(?MODULE, NewHConfig2), - Pid0 = whereis(?MODULE), + Pid0 = whereis(h_proc_name()), MRef2 = erlang:monitor(process, Pid0), %% kill handler - send_burst({n,100}, {spawn,2,0}, {chars,79}, info), + send_burst({n,100}, {spawn,4,0}, {chars,79}, notice), receive - {'DOWN', MRef2, _, _, _Info2} -> - timer:sleep(RestartAfter + 2000), - Pid1 = whereis(?MODULE), - true = is_pid(Pid1), + {'DOWN', MRef2, _, _, _Reason2} -> + file_delete(Log), + {ok,Pid1} = wait_for_process_up(RestartAfter * 3), false = (Pid1 == Pid0), ok after 5000 -> + Info2 = logger_std_h:info(?MODULE), + ct:pal("Handler state = ~p", [Info2]), ct:fail("Handler not dead! It should not have survived this!") end, ok. restart_after(cleanup, _Config) -> ok = stop_handler(?MODULE). -%% send handler requests (filesync, info, reset, change_config) +%% send handler requests (sync, info, reset, change_config) %% during high load to verify that sync, dropping and flushing is %% handled correctly. handler_requests_under_load() -> @@ -1092,17 +1119,17 @@ handler_requests_under_load(Config) -> {Log,HConfig,StdHConfig} = start_handler(?MODULE, ?FUNCTION_NAME, Config), NewHConfig = - HConfig#{logger_std_h => StdHConfig#{toggle_sync_qlen => 2, - drop_new_reqs_qlen => 1000, - flush_reqs_qlen => 2000, - enable_burst_limit => false}}, + HConfig#{config => StdHConfig#{sync_mode_qlen => 2, + drop_mode_qlen => 1000, + flush_qlen => 2000, + burst_limit_enable => false}}, ok = logger:set_handler_config(?MODULE, NewHConfig), Pid = spawn_link(fun() -> send_requests(?MODULE, 1, [{filesync,[]}, {info,[]}, {reset,[]}, {change_config,[]}]) end), - Sent = send_burst({t,10000}, seq, {chars,79}, info), + Sent = send_burst({t,10000}, seq, {chars,79}, notice), Pid ! {self(),finish}, ReqResult = receive {Pid,Result} -> Result end, Logged = count_lines(Log), @@ -1115,7 +1142,7 @@ handler_requests_under_load(Config) -> Errors = [{Req,FindError(Res)} || {Req,Res} <- ReqResult], NoOfReqs = lists:foldl(fun({_,Res}, N) -> N + length(Res) end, 0, ReqResult), ct:pal("~w requests made. Errors: ~n~p", [NoOfReqs,Errors]), - ok = file:delete(Log). + ok = file_delete(Log). handler_requests_under_load(cleanup, _Config) -> ok = stop_handler(?MODULE). @@ -1128,8 +1155,8 @@ send_requests(HName, TO, Reqs = [{Req,Res}|Rs]) -> Result = case Req of change_config -> - logger:set_handler_config(HName, logger_std_h, - #{enable_kill_overloaded => + logger:set_handler_config(HName, config, + #{overload_kill_enable => false}); Func -> logger_std_h:Func(HName) @@ -1144,12 +1171,11 @@ start_handler(Name, TTY, Config) when TTY == standard_io; TTY == standard_error-> ok = logger:add_handler(Name, logger_std_h, - #{logger_std_h => #{type => TTY}, + #{config => #{type => TTY}, filter_default=>log, filters=>?DEFAULT_HANDLER_FILTERS([Name]), formatter=>{?MODULE,op}}), - {ok,{_,HConfig = #{logger_std_h := StdHConfig}}} = - logger:get_handler_config(Name), + {ok,HConfig = #{config := StdHConfig}} = logger:get_handler_config(Name), {HConfig,StdHConfig}; start_handler(Name, FuncName, Config) -> @@ -1157,14 +1183,14 @@ start_handler(Name, FuncName, Config) -> Log = filename:join(Dir, lists:concat([FuncName,".log"])), ct:pal("Logging to ~tp", [Log]), Type = {file,Log}, + _ = file_delete(Log), ok = logger:add_handler(Name, logger_std_h, - #{logger_std_h => #{type => Type}, + #{config => #{type => Type}, filter_default=>log, filters=>?DEFAULT_HANDLER_FILTERS([Name]), formatter=>{?MODULE,op}}), - {ok,{_,HConfig = #{logger_std_h := StdHConfig}}} = - logger:get_handler_config(Name), + {ok,HConfig = #{config := StdHConfig}} = logger:get_handler_config(Name), {Log,HConfig,StdHConfig}. stop_handler(Name) -> @@ -1182,27 +1208,31 @@ wait_until_written(File, Sz) -> {ok,#file_info{size = Sz}} -> timer:sleep(1000), case file:read_file_info(File) of - {ok,#file_info{size = Sz1}} -> + {ok,#file_info{size = Sz}} -> ok; - {ok,#file_info{size = Sz2}} -> - wait_until_written(File, Sz2) + {ok,#file_info{size = Sz1}} -> + wait_until_written(File, Sz1) end; {ok,#file_info{size = Sz1}} -> wait_until_written(File, Sz1) end. count_lines1(File) -> - Counter = fun Cnt(Dev,LC) -> - case file:read_line(Dev) of - eof -> LC; - _ -> Cnt(Dev,LC+1) - end - end, {_,Dev} = file:open(File, [read]), - Lines = Counter(Dev, 0), + Lines = count_lines2(Dev, 0), file:close(Dev), Lines. +count_lines2(Dev, LC) -> + case file:read_line(Dev) of + {ok,"Handler logger_std_h_SUITE " ++_} -> + %% Not counting handler info + count_lines2(Dev,LC); + {ok,_} -> + count_lines2(Dev,LC+1); + eof -> LC + end. + send_burst(NorT, Type, {chars,Sz}, Class) -> Text = [34 + rand:uniform(126-34) || _ <- lists:seq(1,Sz)], case NorT of @@ -1224,12 +1254,9 @@ send_n_burst(N, seq, Text, Class) -> send_n_burst(N-1, seq, Text, Class); send_n_burst(N, {spawn,Ps,TO}, Text, Class) -> ct:pal("~w processes each sending ~w messages", [Ps,N]), - PerProc = fun() -> - send_n_burst(N, seq, Text, Class) - end, MRefs = [begin if TO == 0 -> ok; true -> timer:sleep(TO) end, - monitor(process,spawn_link(PerProc)) end || - _ <- lists:seq(1,Ps)], + monitor(process,spawn_link(per_proc_fun(N,Text,Class,X))) + end || X <- lists:seq(1,Ps)], lists:foreach(fun(MRef) -> receive {'DOWN', MRef, _, _, _} -> @@ -1248,6 +1275,16 @@ send_t_burst(T0, T, Text, Class, N) -> send_t_burst(T0, T, Text, Class, N+1) end. +per_proc_fun(N,Text,Class,X) when X rem 2 == 0 -> + fun() -> + process_flag(priority,high), + send_n_burst(N, seq, Text, Class) + end; +per_proc_fun(N,Text,Class,_) -> + fun() -> + send_n_burst(N, seq, Text, Class) + end. + %%%----------------------------------------------------------------- %%% Formatter callback %%% Using this to send the formatted string back to the test case @@ -1271,9 +1308,11 @@ format(#{msg:={report,#{label:={gen_server,terminate}}}},op) -> ""; format(#{msg:={report,#{label:={proc_lib,crash}}}},op) -> ""; -format(#{msg:={F,A}},Pid) when is_list(F), is_list(A) -> +format(#{msg:={F,A}},OpOrPid) when is_list(F), is_list(A) -> String = lists:flatten(io_lib:format(F,A)), - Pid ! {log,String}, + if is_pid(OpOrPid) -> OpOrPid ! {log,String}; + true -> ok + end, String++"\n"; format(#{msg:={string,String0}},Pid) -> String = unicode:characters_to_list(String0), @@ -1282,21 +1321,21 @@ format(#{msg:={string,String0}},Pid) -> add_remove_instance_nofile(Type) -> ok = logger:add_handler(?MODULE,logger_std_h, - #{logger_std_h => #{type => Type}, + #{config => #{type => Type}, filter_default=>stop, filters=>?DEFAULT_HANDLER_FILTERS([?MODULE]), formatter=>{?MODULE,self()}}), - Pid = whereis(?MODULE), + Pid = whereis(h_proc_name()), true = is_pid(Pid), group_leader(group_leader(),Pid), % to get printouts in test log - logger:info(M1=?msg,?domain), + logger:notice(M1=?msg,?domain), ?check(M1), - %% check that filesync doesn't do damage even if not relevant + %% check that sync doesn't do damage even if not relevant ok = logger_std_h:filesync(?MODULE), ok = logger:remove_handler(?MODULE), timer:sleep(500), - undefined = whereis(?MODULE), - logger:info(?msg,?domain), + undefined = whereis(h_proc_name()), + logger:notice(?msg,?domain), ?check_no_log, ok. @@ -1379,7 +1418,7 @@ start_op_trace() -> TRecvPid = spawn_link(fun() -> trace_receiver(5000) end), {ok,_} = dbg:tracer(process, {TraceFun, TRecvPid}), - {ok,_} = dbg:p(whereis(?MODULE), [c]), + {ok,_} = dbg:p(whereis(h_proc_name()), [c]), {ok,_} = dbg:p(self(), [c]), MS1 = dbg:fun2ms(fun([_]) -> return_trace() end), @@ -1459,7 +1498,7 @@ start_tracer(Trace,Expected) -> Pid = self(), FileCtrlPid = maps:get(file_ctrl_pid, logger_std_h:info(?MODULE)), dbg:tracer(process,{fun tracer/2,{Pid,Expected}}), - dbg:p(whereis(?MODULE),[c]), + dbg:p(whereis(h_proc_name()),[c]), dbg:p(FileCtrlPid,[c]), tpl(Trace), ok. @@ -1525,3 +1564,44 @@ escape([H|T]) -> [H|escape(T)]; escape([]) -> []. + +h_proc_name() -> + h_proc_name(?MODULE). +h_proc_name(Name) -> + ?name_to_reg_name(logger_std_h,Name). + +wait_for_process_up(T) -> + wait_for_process_up(?MODULE, h_proc_name(), T). + +wait_for_process_up(Name, RegName, T) -> + N = (T div 500) + 1, + wait_for_process_up1(Name, RegName, N). + +wait_for_process_up1(_Name, _RegName, 0) -> + error; +wait_for_process_up1(Name, RegName, N) -> + timer:sleep(500), + case whereis(RegName) of + Pid when is_pid(Pid) -> + case logger:get_handler_config(Name) of + {ok,_} -> + %% ct:pal("Process ~p up (~p tries left)",[Name,N]), + {ok,Pid}; + _ -> + wait_for_process_up1(Name, RegName, N-1) + end; + undefined -> + %% ct:pal("Waiting for process ~p (~p tries left)",[Name,N]), + wait_for_process_up1(Name, RegName, N-1) + end. + +filesync_rep_int() -> + case (fun() -> is_atom(?FILESYNC_REPEAT_INTERVAL) end)() of + true -> 5500; + false -> ?FILESYNC_REPEAT_INTERVAL + 500 + end. + + +file_delete(Log) -> + file:delete(Log). + diff --git a/lib/kernel/test/logger_test_lib.erl b/lib/kernel/test/logger_test_lib.erl index 4ac05e6480..81eb9ce5eb 100644 --- a/lib/kernel/test/logger_test_lib.erl +++ b/lib/kernel/test/logger_test_lib.erl @@ -37,7 +37,7 @@ setup(Config,Vars) -> " -boot start_sasl -kernel start_timer true " "-config ",ConfigFileName]}]) of {ok, Node} -> - L = rpc:call(Node, logger, i, []), + L = rpc:call(Node, logger, get_config, []), ct:log("~p",[L]), {ok, L, Node}; {error, Reason} -> @@ -52,7 +52,7 @@ log(Node, M, F, A) -> rpc:call(Node, M, F, A ++ [MD]). sync_and_read(Node,disk_log,Log) -> - rpc:call(Node,logger_disk_log_h,disk_log_sync,[?STANDARD_HANDLER]), + rpc:call(Node,logger_disk_log_h,filesync,[?STANDARD_HANDLER]), file:read_file(Log ++ ".1"); sync_and_read(Node, file,Log) -> ok = rpc:call(Node,logger_std_h,filesync,[?STANDARD_HANDLER]), diff --git a/lib/kernel/test/os_SUITE.erl b/lib/kernel/test/os_SUITE.erl index abbc301360..710b9b115c 100644 --- a/lib/kernel/test/os_SUITE.erl +++ b/lib/kernel/test/os_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2017. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/kernel/test/pdict_SUITE.erl b/lib/kernel/test/pdict_SUITE.erl index a891451c82..3685e51c10 100644 --- a/lib/kernel/test/pdict_SUITE.erl +++ b/lib/kernel/test/pdict_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2016. All Rights Reserved. +%% Copyright Ericsson AB 1999-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/kernel/test/prim_file_SUITE.erl b/lib/kernel/test/prim_file_SUITE.erl index 5bb230d1c4..a02b5f87d1 100644 --- a/lib/kernel/test/prim_file_SUITE.erl +++ b/lib/kernel/test/prim_file_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2000-2017. All Rights Reserved. +%% Copyright Ericsson AB 2000-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/kernel/test/seq_trace_SUITE.erl b/lib/kernel/test/seq_trace_SUITE.erl index aae8a83304..cf4bf11328 100644 --- a/lib/kernel/test/seq_trace_SUITE.erl +++ b/lib/kernel/test/seq_trace_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1998-2016. All Rights Reserved. +%% Copyright Ericsson AB 1998-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -783,6 +783,24 @@ do_shrink(N) -> erlang:garbage_collect(), do_shrink(N-1). +%% Test that messages from a port does not clear the token +port_clean_token(Config) when is_list(Config) -> + seq_trace:reset_trace(), + Label = make_ref(), + seq_trace:set_token(label, Label), + {label,Label} = seq_trace:get_token(label), + + %% Create a port and get messages from it + %% We use os:cmd as a convenience as it does + %% open_port, port_command, port_close and receives replies. + %% Maybe it is not ideal to rely on the internal implementation + %% of os:cmd but it will have to do. + os:cmd("ls"), + + %% Make sure that the seq_trace token is still there + {label,Label} = seq_trace:get_token(label), + + ok. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% diff --git a/lib/kernel/test/wrap_log_reader_SUITE.erl b/lib/kernel/test/wrap_log_reader_SUITE.erl index b1ee29a11f..59b088ca73 100644 --- a/lib/kernel/test/wrap_log_reader_SUITE.erl +++ b/lib/kernel/test/wrap_log_reader_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1998-2016. All Rights Reserved. +%% Copyright Ericsson AB 1998-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/kernel/test/wrap_log_reader_SUITE_data/wrap_log_test.erl b/lib/kernel/test/wrap_log_reader_SUITE_data/wrap_log_test.erl index 2b24ccc66f..d2bac40192 100644 --- a/lib/kernel/test/wrap_log_reader_SUITE_data/wrap_log_test.erl +++ b/lib/kernel/test/wrap_log_reader_SUITE_data/wrap_log_test.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1998-2016. All Rights Reserved. +%% Copyright Ericsson AB 1998-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/kernel/test/zlib_SUITE.erl b/lib/kernel/test/zlib_SUITE.erl index f203ef878f..52ae1b3ae6 100644 --- a/lib/kernel/test/zlib_SUITE.erl +++ b/lib/kernel/test/zlib_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2016. All Rights Reserved. +%% Copyright Ericsson AB 2005-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/kernel/vsn.mk b/lib/kernel/vsn.mk index 60a1b0bff8..aa8e4dc119 100644 --- a/lib/kernel/vsn.mk +++ b/lib/kernel/vsn.mk @@ -1 +1 @@ -KERNEL_VSN = 5.4.3 +KERNEL_VSN = 6.0 diff --git a/lib/megaco/doc/src/Makefile b/lib/megaco/doc/src/Makefile index 43704cddf4..72feb9d2b3 100644 --- a/lib/megaco/doc/src/Makefile +++ b/lib/megaco/doc/src/Makefile @@ -114,6 +114,7 @@ clean_man: clean_html: rm -rf $(HTMLDIR)/* + rm -rf $(XMLDIR) imgs: $(IMG_FILES:%=$(HTMLDIR)/%) diff --git a/lib/mnesia/doc/src/Makefile b/lib/mnesia/doc/src/Makefile index aed46d50db..d9647fc081 100644 --- a/lib/mnesia/doc/src/Makefile +++ b/lib/mnesia/doc/src/Makefile @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 1997-2017. All Rights Reserved. +# Copyright Ericsson AB 1997-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -118,6 +118,7 @@ html: gifs $(HTML_REF_MAN_FILE) clean clean_docs: rm -rf $(HTMLDIR)/* + rm -rf $(XMLDIR) rm -f $(MAN3DIR)/* rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) rm -f errs core *~ diff --git a/lib/mnesia/doc/src/Mnesia_chap2.xmlsrc b/lib/mnesia/doc/src/Mnesia_chap2.xmlsrc index 914e0e509c..8135e14301 100644 --- a/lib/mnesia/doc/src/Mnesia_chap2.xmlsrc +++ b/lib/mnesia/doc/src/Mnesia_chap2.xmlsrc @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>1997</year><year>2016</year> + <year>1997</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/mnesia/doc/src/Mnesia_chap5.xmlsrc b/lib/mnesia/doc/src/Mnesia_chap5.xmlsrc index 0265e0efa0..481e6651e6 100644 --- a/lib/mnesia/doc/src/Mnesia_chap5.xmlsrc +++ b/lib/mnesia/doc/src/Mnesia_chap5.xmlsrc @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>1997</year><year>2016</year> + <year>1997</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/mnesia/doc/src/company.erl b/lib/mnesia/doc/src/company.erl index 5a3b122394..fc04aa77bf 100644 --- a/lib/mnesia/doc/src/company.erl +++ b/lib/mnesia/doc/src/company.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2016. All Rights Reserved. +%% Copyright Ericsson AB 1999-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/mnesia/doc/src/company_o.erl b/lib/mnesia/doc/src/company_o.erl index 650b6cdeca..b4b3638596 100644 --- a/lib/mnesia/doc/src/company_o.erl +++ b/lib/mnesia/doc/src/company_o.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2016. All Rights Reserved. +%% Copyright Ericsson AB 1999-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/mnesia/doc/src/notes.xml b/lib/mnesia/doc/src/notes.xml index ba94e913f5..8a7dab739e 100644 --- a/lib/mnesia/doc/src/notes.xml +++ b/lib/mnesia/doc/src/notes.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>1996</year><year>2017</year> + <year>1996</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -39,7 +39,48 @@ thus constitutes one section in this document. The title of each section is the version number of Mnesia.</p> - <section><title>Mnesia 4.15.3</title> + <section><title>Mnesia 4.15.4</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> Calls to <c>erlang:get_stacktrace()</c> are removed. + </p> + <p> + Own Id: OTP-14861</p> + </item> + </list> + </section> + +</section> + +<section><title>Mnesia 4.15.3.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + When master node is set do not force a load from + ram_copies replica when there are no available + disc_copies, since that would load an empty table. Wait + until a disk replica is available or until user + explicitly force_loads the table.</p> + <p> + Own Id: OTP-15221 Aux Id: ERIERL-217 </p> + </item> + <item> + <p> + Allow to add replicas even if all other replicas are down + when the other replicase are not stored on disk.</p> + <p> + Own Id: OTP-15226 Aux Id: ERIERL-221 </p> + </item> + </list> + </section> + +</section> + +<section><title>Mnesia 4.15.3</title> <section><title>Fixed Bugs and Malfunctions</title> <list> @@ -648,7 +689,7 @@ <p> Returns the same value for mnesia_loader:disc_load_table/2 as - mnesia_loader:net_load_table/4 if a table copy can not be + mnesia_loader:net_load_table/4 if a table copy cannot be found. (Thanks to Uwe Dauernheim)</p> <p> Own Id: OTP-10015</p> diff --git a/lib/mnesia/src/mnesia.erl b/lib/mnesia/src/mnesia.erl index 6d87544bd2..62202401ed 100644 --- a/lib/mnesia/src/mnesia.erl +++ b/lib/mnesia/src/mnesia.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2017. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -168,6 +168,9 @@ -type snmp_struct() :: [{atom(), snmp_type() | tuple_of(snmp_type())}]. -type snmp_type() :: 'fix_string' | 'string' | 'integer'. -type tuple_of(_T) :: tuple(). +-type config_key() :: extra_db_nodes | dc_dump_limit. +-type config_value() :: [node()] | number(). +-type config_result() :: {ok, config_value()} | {error, term()}. -define(DEFAULT_ACCESS, ?MODULE). @@ -278,7 +281,8 @@ stop() -> Other -> Other end. --spec change_config(Config::atom(), Value::_) -> ok | {error, term()}. +-spec change_config(Config::config_key(), Value::config_value()) -> + config_result(). change_config(extra_db_nodes, Ns) when is_list(Ns) -> mnesia_controller:connect_nodes(Ns); change_config(dc_dump_limit, N) when is_number(N), N > 0 -> diff --git a/lib/mnesia/src/mnesia.hrl b/lib/mnesia/src/mnesia.hrl index e36222d098..fe48a6fe3d 100644 --- a/lib/mnesia/src/mnesia.hrl +++ b/lib/mnesia/src/mnesia.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/mnesia/src/mnesia_bup.erl b/lib/mnesia/src/mnesia_bup.erl index 9dda340333..e57fc5199d 100644 --- a/lib/mnesia/src/mnesia_bup.erl +++ b/lib/mnesia/src/mnesia_bup.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/mnesia/src/mnesia_checkpoint.erl b/lib/mnesia/src/mnesia_checkpoint.erl index 8290237908..3273d3d27e 100644 --- a/lib/mnesia/src/mnesia_checkpoint.erl +++ b/lib/mnesia/src/mnesia_checkpoint.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2017 +%% Copyright Ericsson AB 1996-2018 %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/mnesia/src/mnesia_controller.erl b/lib/mnesia/src/mnesia_controller.erl index 77e67a59db..882de0d613 100644 --- a/lib/mnesia/src/mnesia_controller.erl +++ b/lib/mnesia/src/mnesia_controller.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2017. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -772,22 +772,6 @@ handle_call({unannounce_add_table_copy, [Tab, Node], From}, ReplyTo, State) -> noreply(State#state{early_msgs = [{call, Msg, undefined} | Msgs]}) end; -handle_call({net_load, Tab, Cs}, From, State) -> - State2 = - case State#state.schema_is_merged of - true -> - Worker = #net_load{table = Tab, - opt_reply_to = From, - reason = {dumper,{add_table_copy, unknown}}, - cstruct = Cs - }, - add_worker(Worker, State); - false -> - reply(From, {not_loaded, schema_not_merged}), - State - end, - noreply(State2); - handle_call(Msg, From, State) when State#state.schema_is_merged /= true -> %% Buffer early messages Msgs = State#state.early_msgs, @@ -1457,7 +1441,8 @@ orphan_tables([Tab | Tabs], Node, Ns, Local, Remote) -> L = [Tab | Local], orphan_tables(Tabs, Node, Ns, L, Remote); Masters -> - R = [{Tab, Masters} | Remote], + %% Do not disc_load table from RamCopyHolders + R = [{Tab, Masters -- RamCopyHolders} | Remote], orphan_tables(Tabs, Node, Ns, Local, R) end; _ -> @@ -2162,6 +2147,15 @@ load_table_fun(#net_load{cstruct=Cs, table=Tab, reason=Reason, opt_reply_to=Repl {dumper,{add_table_copy,_}} -> true; _ -> false end, + + OnlyRamCopies = case Cs of + #cstruct{disc_copies = DC, + disc_only_copies = DOC, + external_copies = Ext} -> + [] =:= (DC ++ (DOC ++ Ext)) -- [node()]; + _ -> + false + end, if ReadNode == node() -> %% Already loaded locally @@ -2173,6 +2167,8 @@ load_table_fun(#net_load{cstruct=Cs, table=Tab, reason=Reason, opt_reply_to=Repl end; AccessMode == read_only, not AddTableCopy -> fun() -> disc_load_table(Tab, Reason, ReplyTo) end; + Active =:= [], AddTableCopy, OnlyRamCopies -> + fun() -> disc_load_table(Tab, Reason, ReplyTo) end; true -> fun() -> %% Either we cannot read the table yet diff --git a/lib/mnesia/src/mnesia_dumper.erl b/lib/mnesia/src/mnesia_dumper.erl index 31bcc1451f..a2880d6cf4 100644 --- a/lib/mnesia/src/mnesia_dumper.erl +++ b/lib/mnesia/src/mnesia_dumper.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/mnesia/src/mnesia_frag.erl b/lib/mnesia/src/mnesia_frag.erl index 63fb4981fe..8f7dd321b0 100644 --- a/lib/mnesia/src/mnesia_frag.erl +++ b/lib/mnesia/src/mnesia_frag.erl @@ -1,7 +1,7 @@ %%% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1998-2016. All Rights Reserved. +%% Copyright Ericsson AB 1998-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/mnesia/src/mnesia_index.erl b/lib/mnesia/src/mnesia_index.erl index d121bd01e9..917ada65df 100644 --- a/lib/mnesia/src/mnesia_index.erl +++ b/lib/mnesia/src/mnesia_index.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/mnesia/src/mnesia_late_loader.erl b/lib/mnesia/src/mnesia_late_loader.erl index e4f8dcf2b9..45afda6e4b 100644 --- a/lib/mnesia/src/mnesia_late_loader.erl +++ b/lib/mnesia/src/mnesia_late_loader.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1998-2016. All Rights Reserved. +%% Copyright Ericsson AB 1998-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/mnesia/src/mnesia_lib.erl b/lib/mnesia/src/mnesia_lib.erl index 56f6926df0..6abc05fade 100644 --- a/lib/mnesia/src/mnesia_lib.erl +++ b/lib/mnesia/src/mnesia_lib.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -929,7 +929,7 @@ error_desc(no_transaction) -> "Operation not allowed outside transactions"; error_desc(combine_error) -> "Table options were ilegally combined"; error_desc(bad_index) -> "Index already exists or was out of bounds"; error_desc(already_exists) -> "Some schema option we try to set is already on"; -error_desc(index_exists)-> "Some ops can not be performed on tabs with index"; +error_desc(index_exists)-> "Some ops cannot be performed on tabs with index"; error_desc(no_exists)-> "Tried to perform op on non-existing (non alive) item"; error_desc(system_limit) -> "Some system_limit was exhausted"; error_desc(mnesia_down) -> "A transaction involving objects at some remote " diff --git a/lib/mnesia/src/mnesia_loader.erl b/lib/mnesia/src/mnesia_loader.erl index bbceb9ba66..2cdae0c906 100644 --- a/lib/mnesia/src/mnesia_loader.erl +++ b/lib/mnesia/src/mnesia_loader.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1998-2016. All Rights Reserved. +%% Copyright Ericsson AB 1998-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -67,7 +67,7 @@ do_get_disc_copy2(Tab, Reason, Storage, Type) when Storage == disc_copies -> EtsOpts = proplists:get_value(ets, StorageProps, []), Args = [{keypos, 2}, public, named_table, Type | EtsOpts], case Reason of - {dumper, _} -> %% Resources already allocated + {dumper, DR} when is_atom(DR) -> %% Resources already allocated ignore; _ -> mnesia_monitor:mktab(Tab, Args), @@ -91,8 +91,8 @@ do_get_disc_copy2(Tab, Reason, Storage, Type) when Storage == ram_copies -> EtsOpts = proplists:get_value(ets, StorageProps, []), Args = [{keypos, 2}, public, named_table, Type | EtsOpts], case Reason of - {dumper, _} -> %% Resources allready allocated - ignore; + {dumper, DR} when is_atom(DR) -> + ignore; %% Resources already allocated _ -> mnesia_monitor:mktab(Tab, Args), Fname = mnesia_lib:tab2dcd(Tab), @@ -131,7 +131,7 @@ do_get_disc_copy2(Tab, Reason, Storage, Type) when Storage == disc_only_copies - {repair, mnesia_monitor:get_env(auto_repair)} | DetsOpts], case Reason of - {dumper, _} -> + {dumper, DR} when is_atom(DR) -> mnesia_index:init_index(Tab, Storage), snmpify(Tab, Storage), set({Tab, load_node}, node()), diff --git a/lib/mnesia/src/mnesia_locker.erl b/lib/mnesia/src/mnesia_locker.erl index 710bf38cb5..f68626413e 100644 --- a/lib/mnesia/src/mnesia_locker.erl +++ b/lib/mnesia/src/mnesia_locker.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/mnesia/src/mnesia_log.erl b/lib/mnesia/src/mnesia_log.erl index a2de23a2a3..03411ace41 100644 --- a/lib/mnesia/src/mnesia_log.erl +++ b/lib/mnesia/src/mnesia_log.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/mnesia/src/mnesia_recover.erl b/lib/mnesia/src/mnesia_recover.erl index 475bf3c327..2ccea1ea6d 100644 --- a/lib/mnesia/src/mnesia_recover.erl +++ b/lib/mnesia/src/mnesia_recover.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2016. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/mnesia/src/mnesia_schema.erl b/lib/mnesia/src/mnesia_schema.erl index de6c3fb4ee..ef38adca1e 100644 --- a/lib/mnesia/src/mnesia_schema.erl +++ b/lib/mnesia/src/mnesia_schema.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2017. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/mnesia/src/mnesia_subscr.erl b/lib/mnesia/src/mnesia_subscr.erl index dfaa20d2d3..21a308cfb6 100644 --- a/lib/mnesia/src/mnesia_subscr.erl +++ b/lib/mnesia/src/mnesia_subscr.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2016. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/mnesia/src/mnesia_text.erl b/lib/mnesia/src/mnesia_text.erl index 7d24d09472..cc21621ff4 100644 --- a/lib/mnesia/src/mnesia_text.erl +++ b/lib/mnesia/src/mnesia_text.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/mnesia/src/mnesia_tm.erl b/lib/mnesia/src/mnesia_tm.erl index 3f6f6c98d8..cbf7db28f0 100644 --- a/lib/mnesia/src/mnesia_tm.erl +++ b/lib/mnesia/src/mnesia_tm.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -1661,7 +1661,7 @@ commit_participant(Coord, Tid, Bin, C0, DiscNs, _RamNs) -> ?eval_debug_fun({?MODULE, commit_participant, pre}, [{tid, Tid}]), try mnesia_schema:prepare_commit(Tid, C0, {part, Coord}) of {Modified, C = #commit{}, DumperMode} -> - %% If we can not find any local unclear decision + %% If we cannot find any local unclear decision %% we should presume abort at startup recovery case lists:member(node(), DiscNs) of false -> diff --git a/lib/mnesia/test/mnesia_SUITE.erl b/lib/mnesia/test/mnesia_SUITE.erl index 279744dbb0..24c1def6da 100644 --- a/lib/mnesia/test/mnesia_SUITE.erl +++ b/lib/mnesia/test/mnesia_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2016. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/mnesia/test/mnesia_atomicity_test.erl b/lib/mnesia/test/mnesia_atomicity_test.erl index 58a5dc1d40..4764f9e7c0 100644 --- a/lib/mnesia/test/mnesia_atomicity_test.erl +++ b/lib/mnesia/test/mnesia_atomicity_test.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2016. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/mnesia/test/mnesia_bench_SUITE.erl b/lib/mnesia/test/mnesia_bench_SUITE.erl index 5cc01867c4..8a225629e6 100644 --- a/lib/mnesia/test/mnesia_bench_SUITE.erl +++ b/lib/mnesia/test/mnesia_bench_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2012-2016. All Rights Reserved. +%% Copyright Ericsson AB 2012-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/mnesia/test/mnesia_consistency_test.erl b/lib/mnesia/test/mnesia_consistency_test.erl index 7a2678cee3..46bafaf65c 100644 --- a/lib/mnesia/test/mnesia_consistency_test.erl +++ b/lib/mnesia/test/mnesia_consistency_test.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2016. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/mnesia/test/mnesia_cost.erl b/lib/mnesia/test/mnesia_cost.erl index 4d0dd7b0ee..b5d5253147 100644 --- a/lib/mnesia/test/mnesia_cost.erl +++ b/lib/mnesia/test/mnesia_cost.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/mnesia/test/mnesia_dirty_access_test.erl b/lib/mnesia/test/mnesia_dirty_access_test.erl index 92124ebc65..67ef1fe901 100644 --- a/lib/mnesia/test/mnesia_dirty_access_test.erl +++ b/lib/mnesia/test/mnesia_dirty_access_test.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/mnesia/test/mnesia_durability_test.erl b/lib/mnesia/test/mnesia_durability_test.erl index 2fac5cac82..ccbfdc9738 100644 --- a/lib/mnesia/test/mnesia_durability_test.erl +++ b/lib/mnesia/test/mnesia_durability_test.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2016. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -46,6 +46,7 @@ master_nodes/1, starting_master_nodes/1, master_on_non_local_tables/1, remote_force_load_with_local_master_node/1, + master_node_with_ram_copy_2/1, master_node_with_ram_copy_3/1, dump_ram_copies/1, dump_disc_copies/1, dump_disc_only/1]). -include("mnesia_test_lib.hrl"). @@ -91,7 +92,8 @@ groups() -> {load_tables_with_master_tables, [], [master_nodes, starting_master_nodes, master_on_non_local_tables, - remote_force_load_with_local_master_node]}, + remote_force_load_with_local_master_node, + master_node_with_ram_copy_2, master_node_with_ram_copy_3]}, {durability_of_dump_tables, [], [dump_ram_copies, dump_disc_copies, dump_disc_only]}]. @@ -1165,6 +1167,107 @@ remote_force_load_with_local_master_node(Config) when is_list(Config) -> ?verify_mnesia(Nodes, []). +master_node_with_ram_copy_2(Config) when is_list(Config) -> + [A, B] = Nodes = ?acquire_nodes(2, Config), + Tab = ?FUNCTION_NAME, + ?match({atomic,ok}, mnesia:create_table(Tab, [{disc_copies, [A]}, {ram_copies, [B]}])), + ?match({atomic,ok}, mnesia:sync_transaction(?SDwrite({Tab, 1, init}))), + + %% Test that we don't load from ram_copies + ?match(stopped, rpc:call(A, mnesia, stop, [])), + ?match(stopped, rpc:call(B, mnesia, stop, [])), + ?match(ok, rpc:call(B, mnesia, start, [])), + ?match({timeout, [Tab]}, rpc:call(B, mnesia, wait_for_tables, [[Tab], 1000])), + ?match(ok, rpc:call(A, mnesia, start, [])), + ?match(ok, rpc:call(B, mnesia, wait_for_tables, [[Tab], 3000])), + ?match([{Tab, 1, init}], rpc:call(A, mnesia, dirty_read, [{Tab, 1}])), + ?match([{Tab, 1, init}], rpc:call(B, mnesia, dirty_read, [{Tab, 1}])), + + %% Test that master_nodes set to ram_copy node require force_load + ?match(ok, rpc:call(A, mnesia, set_master_nodes, [[B]])), + ?match(stopped, rpc:call(A, mnesia, stop, [])), + ?match(stopped, rpc:call(B, mnesia, stop, [])), + ?match(ok, rpc:call(B, mnesia, start, [])), + ?match({timeout, [Tab]}, rpc:call(B, mnesia, wait_for_tables, [[Tab], 1000])), + ?match(ok, rpc:call(A, mnesia, start, [])), + ?match({timeout, [Tab]}, rpc:call(B, mnesia, wait_for_tables, [[Tab], 1000])), + + ?match(yes, rpc:call(A, mnesia, force_load_table, [Tab])), + ?match(ok, rpc:call(A, mnesia, wait_for_tables, [[Tab], 1000])), + ?match(ok, rpc:call(B, mnesia, wait_for_tables, [[Tab], 1000])), + ?match([{Tab, 1, init}], rpc:call(A, mnesia, dirty_read, [{Tab, 1}])), + ?match([{Tab, 1, init}], rpc:call(B, mnesia, dirty_read, [{Tab, 1}])), + + ?verify_mnesia(Nodes, []). + + +master_node_with_ram_copy_3(Config) when is_list(Config) -> + [A, B, C] = Nodes = ?acquire_nodes(3, Config), + Tab = ?FUNCTION_NAME, + ?match({atomic,ok}, mnesia:create_table(Tab, [{disc_copies, [A,C]}, {ram_copies, [B]}])), + ?match({atomic,ok}, mnesia:sync_transaction(?SDwrite({Tab, 1, init}))), + + %% Test that we don't load from ram_copies + ?match(stopped, rpc:call(A, mnesia, stop, [])), + ?match(stopped, rpc:call(C, mnesia, stop, [])), + ?match(stopped, rpc:call(B, mnesia, stop, [])), + ?match(ok, rpc:call(B, mnesia, start, [])), + ?match({timeout, [Tab]}, rpc:call(B, mnesia, wait_for_tables, [[Tab], 1000])), + ?match(ok, rpc:call(A, mnesia, start, [])), + ?match(ok, rpc:call(C, mnesia, start, [])), + ?match(ok, rpc:call(B, mnesia, wait_for_tables, [[Tab], 3000])), + ?match(ok, rpc:call(A, mnesia, wait_for_tables, [[Tab], 3000])), + ?match([{Tab, 1, init}], rpc:call(A, mnesia, dirty_read, [{Tab, 1}])), + ?match([{Tab, 1, init}], rpc:call(B, mnesia, dirty_read, [{Tab, 1}])), + ?match([{Tab, 1, init}], rpc:call(C, mnesia, dirty_read, [{Tab, 1}])), + + %% Test that master_nodes set to ram_copy node will wait until loaded + ?match(ok, rpc:call(A, mnesia, set_master_nodes, [[B]])), + ?match(stopped, rpc:call(A, mnesia, stop, [])), + ?match({atomic,ok}, rpc:call(B, mnesia, sync_transaction, [?SDwrite({Tab, 1, update})])), + ?match(stopped, rpc:call(C, mnesia, stop, [])), + ?match({atomic,ok}, rpc:call(B, mnesia, sync_transaction, [?SDwrite({Tab, 1, ram_copies})])), + ?match(stopped, rpc:call(B, mnesia, stop, [])), + ?match(ok, rpc:call(B, mnesia, start, [])), + ?match({timeout, [Tab]}, rpc:call(B, mnesia, wait_for_tables, [[Tab], 500])), + ?match(ok, rpc:call(A, mnesia, start, [])), + ?match({timeout, [Tab]}, rpc:call(A, mnesia, wait_for_tables, [[Tab], 500])), + ?match(ok, rpc:call(C, mnesia, start, [])), + ?match(ok, rpc:call(B, mnesia, wait_for_tables, [[Tab], 3000])), + ?match(ok, rpc:call(A, mnesia, wait_for_tables, [[Tab], 3000])), + ?match([{Tab, 1, update}], rpc:call(A, mnesia, dirty_read, [{Tab, 1}])), + ?match([{Tab, 1, update}], rpc:call(B, mnesia, dirty_read, [{Tab, 1}])), + ?match([{Tab, 1, update}], rpc:call(C, mnesia, dirty_read, [{Tab, 1}])), + + %% Test that master_nodes set to ram_copy node requires force load + ?match({atomic,ok}, mnesia:sync_transaction(?SDwrite({Tab, 1, init}))), + ?match(ok, rpc:call(A, mnesia, set_master_nodes, [[B]])), + ?match(ok, rpc:call(C, mnesia, set_master_nodes, [[B]])), + + ?match(stopped, rpc:call(A, mnesia, stop, [])), + ?match({atomic,ok}, rpc:call(B, mnesia, sync_transaction, [?SDwrite({Tab, 1, update})])), + ?match(stopped, rpc:call(C, mnesia, stop, [])), + ?match({atomic,ok}, rpc:call(B, mnesia, sync_transaction, [?SDwrite({Tab, 1, ram_copies})])), + ?match(stopped, rpc:call(B, mnesia, stop, [])), + ?match(ok, rpc:call(B, mnesia, start, [])), + ?match({timeout, [Tab]}, rpc:call(B, mnesia, wait_for_tables, [[Tab], 500])), + ?match(ok, rpc:call(A, mnesia, start, [])), + ?match({timeout, [Tab]}, rpc:call(A, mnesia, wait_for_tables, [[Tab], 500])), + ?match(ok, rpc:call(C, mnesia, start, [])), + ?match({timeout, [Tab]}, rpc:call(A, mnesia, wait_for_tables, [[Tab], 500])), + ?match({timeout, [Tab]}, rpc:call(B, mnesia, wait_for_tables, [[Tab], 500])), + ?match({timeout, [Tab]}, rpc:call(B, mnesia, wait_for_tables, [[Tab], 500])), + ?match(yes, rpc:call(C, mnesia, force_load_table, [Tab])), + + ?match(ok, rpc:call(A, mnesia, wait_for_tables, [[Tab], 3000])), + ?match(ok, rpc:call(B, mnesia, wait_for_tables, [[Tab], 3000])), + ?match(ok, rpc:call(C, mnesia, wait_for_tables, [[Tab], 3000])), + ?match([{Tab, 1, update}], rpc:call(A, mnesia, dirty_read, [{Tab, 1}])), + ?match([{Tab, 1, update}], rpc:call(B, mnesia, dirty_read, [{Tab, 1}])), + ?match([{Tab, 1, update}], rpc:call(C, mnesia, dirty_read, [{Tab, 1}])), + + ?verify_mnesia(Nodes, []). + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -1415,7 +1518,7 @@ do_disc_durability(Config,CopyType) -> [{Tab_bag, 22, a_2222}], [{Tab_bag, 33, a_3333}], [{Tab_set, counter, 10}]]), - timer:sleep(1000), %% Debugging strange msgs.. + timer:sleep(500), %% Debugging strange msgs.. ?log("Flushed ~p ~n", [mnesia_test_lib:flush()]), ?verify_mnesia(Nodes, []). diff --git a/lib/mnesia/test/mnesia_evil_coverage_test.erl b/lib/mnesia/test/mnesia_evil_coverage_test.erl index 074967469b..77236940d4 100644 --- a/lib/mnesia/test/mnesia_evil_coverage_test.erl +++ b/lib/mnesia/test/mnesia_evil_coverage_test.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -31,6 +31,7 @@ db_node_lifecycle/1, evil_delete_db_node/1, start_and_stop/1, checkpoint/1, table_lifecycle/1, storage_options/1, add_copy_conflict/1, add_copy_when_going_down/1, + add_copy_with_down/1, replica_management/1, clear_table_during_load/1, schema_availability/1, local_content/1, replica_location/1, user_properties/1, unsupp_user_props/1, @@ -65,7 +66,8 @@ all() -> db_node_lifecycle, evil_delete_db_node, start_and_stop, checkpoint, table_lifecycle, storage_options, add_copy_conflict, - add_copy_when_going_down, replica_management, clear_table_during_load, + add_copy_when_going_down, add_copy_with_down, replica_management, + clear_table_during_load, schema_availability, local_content, {group, table_access_modifications}, replica_location, {group, table_sync}, user_properties, unsupp_user_props, @@ -732,6 +734,49 @@ add_copy_when_going_down(Config) -> ?match_receive({test,{aborted,_}}), ?verify_mnesia([Node2], []). +add_copy_with_down(suite) -> []; +add_copy_with_down(Config) -> + %% Allow add_table_copy() with ram_copies even all other replicas are down + Nodes = [Node1, Node2, Node3] = ?acquire_nodes(3, Config), + ?match({atomic, ok}, mnesia:create_table(a, [{ram_copies, [Node3]}, {disc_copies, [Node2]}])), + stopped = rpc:call(Node2, mnesia, stop, []), + stopped = rpc:call(Node3, mnesia, stop, []), + ?match({aborted, _}, mnesia:add_table_copy(a, Node1, ram_copies)), + ?match({aborted, _}, mnesia:del_table_copy(a, Node2)), + ok = rpc:call(Node3, mnesia, start, []), + ?match({aborted, _}, mnesia:add_table_copy(a, Node1, ram_copies)), + ?match([], mnesia_test_lib:start_mnesia([Node2], [a])), + ?match({atomic, ok}, mnesia:change_table_copy_type(a, Node2, ram_copies)), + stopped = rpc:call(Node2, mnesia, stop, []), + stopped = rpc:call(Node3, mnesia, stop, []), + ?match({atomic, ok}, mnesia:add_table_copy(a, Node1, ram_copies)), + ?match(ok, mnesia:dirty_write({a,1,1})), + ?match([], mnesia_test_lib:start_mnesia([Node2,Node3], [a])), + ?match([{a,1,1}], rpc:call(Node1, mnesia, dirty_read, [{a,1}])), + ?match([{a,1,1}], rpc:call(Node2, mnesia, dirty_read, [{a,1}])), + ?match([{a,1,1}], rpc:call(Node3, mnesia, dirty_read, [{a,1}])), + + ?match({atomic, ok}, mnesia:del_table_copy(a, Node1)), + stopped = rpc:call(Node2, mnesia, stop, []), + stopped = rpc:call(Node3, mnesia, stop, []), + ?match({atomic, ok}, mnesia:add_table_copy(a, Node1, disc_copies)), + ?match(ok, mnesia:dirty_write({a,1,1})), + ?match([], mnesia_test_lib:start_mnesia([Node2,Node3], [a])), + ?match([{a,1,1}], rpc:call(Node1, mnesia, dirty_read, [{a,1}])), + ?match([{a,1,1}], rpc:call(Node2, mnesia, dirty_read, [{a,1}])), + ?match([{a,1,1}], rpc:call(Node3, mnesia, dirty_read, [{a,1}])), + + ?match({atomic, ok}, mnesia:del_table_copy(a, Node1)), + stopped = rpc:call(Node2, mnesia, stop, []), + stopped = rpc:call(Node3, mnesia, stop, []), + ?match({atomic, ok}, mnesia:add_table_copy(a, Node1, disc_only_copies)), + ?match(ok, mnesia:dirty_write({a,1,1})), + ?match([], mnesia_test_lib:start_mnesia([Node2,Node3], [a])), + ?match([{a,1,1}], rpc:call(Node1, mnesia, dirty_read, [{a,1}])), + ?match([{a,1,1}], rpc:call(Node2, mnesia, dirty_read, [{a,1}])), + ?match([{a,1,1}], rpc:call(Node3, mnesia, dirty_read, [{a,1}])), + + ?verify_mnesia(Nodes, []). %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% Add, drop and move replicas, change storage types diff --git a/lib/mnesia/test/mnesia_examples_test.erl b/lib/mnesia/test/mnesia_examples_test.erl index f1259abf90..3bbb6e4d77 100644 --- a/lib/mnesia/test/mnesia_examples_test.erl +++ b/lib/mnesia/test/mnesia_examples_test.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2016. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/mnesia/test/mnesia_frag_test.erl b/lib/mnesia/test/mnesia_frag_test.erl index 7371792fda..7b37fcb684 100644 --- a/lib/mnesia/test/mnesia_frag_test.erl +++ b/lib/mnesia/test/mnesia_frag_test.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2016. All Rights Reserved. +%% Copyright Ericsson AB 1999-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/mnesia/test/mnesia_install_test.erl b/lib/mnesia/test/mnesia_install_test.erl index 3f67396eb0..2aee5137c3 100644 --- a/lib/mnesia/test/mnesia_install_test.erl +++ b/lib/mnesia/test/mnesia_install_test.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/mnesia/test/mnesia_isolation_test.erl b/lib/mnesia/test/mnesia_isolation_test.erl index 1c3ea5ec92..b2eea2390b 100644 --- a/lib/mnesia/test/mnesia_isolation_test.erl +++ b/lib/mnesia/test/mnesia_isolation_test.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2016. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/mnesia/test/mnesia_majority_test.erl b/lib/mnesia/test/mnesia_majority_test.erl index eb82617b60..aae27e069e 100644 --- a/lib/mnesia/test/mnesia_majority_test.erl +++ b/lib/mnesia/test/mnesia_majority_test.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2016. All Rights Reserved. +%% Copyright Ericsson AB 2010-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/mnesia/test/mnesia_measure_test.erl b/lib/mnesia/test/mnesia_measure_test.erl index 4e63eaee22..8eb3590168 100644 --- a/lib/mnesia/test/mnesia_measure_test.erl +++ b/lib/mnesia/test/mnesia_measure_test.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/mnesia/test/mnesia_nice_coverage_test.erl b/lib/mnesia/test/mnesia_nice_coverage_test.erl index 7c96d6e6a0..f8c6b2ce20 100644 --- a/lib/mnesia/test/mnesia_nice_coverage_test.erl +++ b/lib/mnesia/test/mnesia_nice_coverage_test.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/mnesia/test/mnesia_qlc_test.erl b/lib/mnesia/test/mnesia_qlc_test.erl index 262a6b4abc..e66fd84995 100644 --- a/lib/mnesia/test/mnesia_qlc_test.erl +++ b/lib/mnesia/test/mnesia_qlc_test.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2016. All Rights Reserved. +%% Copyright Ericsson AB 2004-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/mnesia/test/mnesia_recovery_test.erl b/lib/mnesia/test/mnesia_recovery_test.erl index 1783d2ae94..b5749408f8 100644 --- a/lib/mnesia/test/mnesia_recovery_test.erl +++ b/lib/mnesia/test/mnesia_recovery_test.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/mnesia/test/mnesia_registry_test.erl b/lib/mnesia/test/mnesia_registry_test.erl index 08157f1be3..c15b8e97af 100644 --- a/lib/mnesia/test/mnesia_registry_test.erl +++ b/lib/mnesia/test/mnesia_registry_test.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1998-2016. All Rights Reserved. +%% Copyright Ericsson AB 1998-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/mnesia/test/mnesia_schema_recovery_test.erl b/lib/mnesia/test/mnesia_schema_recovery_test.erl index 5e7627ca47..e4199758c1 100644 --- a/lib/mnesia/test/mnesia_schema_recovery_test.erl +++ b/lib/mnesia/test/mnesia_schema_recovery_test.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1998-2016. All Rights Reserved. +%% Copyright Ericsson AB 1998-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/mnesia/test/mnesia_test_lib.erl b/lib/mnesia/test/mnesia_test_lib.erl index db0e7beac6..1cdac3cde6 100644 --- a/lib/mnesia/test/mnesia_test_lib.erl +++ b/lib/mnesia/test/mnesia_test_lib.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/mnesia/test/mnesia_test_lib.hrl b/lib/mnesia/test/mnesia_test_lib.hrl index 90a3c686b0..b8eeb5783f 100644 --- a/lib/mnesia/test/mnesia_test_lib.hrl +++ b/lib/mnesia/test/mnesia_test_lib.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/mnesia/test/mnesia_trans_access_test.erl b/lib/mnesia/test/mnesia_trans_access_test.erl index c00a1ed51f..723a85fd2c 100644 --- a/lib/mnesia/test/mnesia_trans_access_test.erl +++ b/lib/mnesia/test/mnesia_trans_access_test.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/mnesia/vsn.mk b/lib/mnesia/vsn.mk index 45f811846d..aaa1c3006f 100644 --- a/lib/mnesia/vsn.mk +++ b/lib/mnesia/vsn.mk @@ -1 +1 @@ -MNESIA_VSN = 4.15.3 +MNESIA_VSN = 4.15.4 diff --git a/lib/observer/doc/src/Makefile b/lib/observer/doc/src/Makefile index 11bfee1bdb..e843772f0b 100644 --- a/lib/observer/doc/src/Makefile +++ b/lib/observer/doc/src/Makefile @@ -105,6 +105,7 @@ html: gifs $(HTML_REF_MAN_FILE) $(ONLY_HTML_FILE:%=$(HTMLDIR)/%) clean clean_docs: rm -rf $(HTMLDIR)/* + rm -rf $(XMLDIR) rm -f $(MAN1DIR)/* rm -f $(MAN3DIR)/* rm -f $(MAN6DIR)/* diff --git a/lib/observer/doc/src/notes.xml b/lib/observer/doc/src/notes.xml index c0b8309af6..67bba37e39 100644 --- a/lib/observer/doc/src/notes.xml +++ b/lib/observer/doc/src/notes.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2004</year><year>2017</year> + <year>2004</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -32,6 +32,35 @@ <p>This document describes the changes made to the Observer application.</p> +<section><title>Observer 2.8</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Added possibility to garbage collect selected processes + and fixed a crash when the saved config file contained + bad data.</p> + <p> + Own Id: OTP-14993 Aux Id: PR-1666 </p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Use uri_string module instead of http_uri.</p> + <p> + Own Id: OTP-14902</p> + </item> + </list> + </section> + +</section> + <section><title>Observer 2.7</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/observer/include/etop.hrl b/lib/observer/include/etop.hrl index f8d370450b..cda68422ab 100644 --- a/lib/observer/include/etop.hrl +++ b/lib/observer/include/etop.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2002-2016. All Rights Reserved. +%% Copyright Ericsson AB 2002-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/observer/src/cdv_atom_cb.erl b/lib/observer/src/cdv_atom_cb.erl index 86cdf2fd6d..87f613124c 100644 --- a/lib/observer/src/cdv_atom_cb.erl +++ b/lib/observer/src/cdv_atom_cb.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2013-2016. All Rights Reserved. +%% Copyright Ericsson AB 2013-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/observer/src/cdv_bin_cb.erl b/lib/observer/src/cdv_bin_cb.erl index a4a542297c..91d33474c8 100644 --- a/lib/observer/src/cdv_bin_cb.erl +++ b/lib/observer/src/cdv_bin_cb.erl @@ -49,7 +49,7 @@ format_bin_fun(Format,Bin) -> try io_lib:format(Format,[Bin]) of Str -> plain_html(lists:flatten(Str)) catch error:badarg -> - Warning = "This binary can not be formatted with " ++ Format, + Warning = "This binary cannot be formatted with " ++ Format, observer_html_lib:warning(Warning) end end. @@ -59,7 +59,7 @@ binary_to_term_fun(Bin) -> try binary_to_term(Bin) of Term -> plain_html(io_lib:format("~tp",[Term])) catch error:badarg -> - Warning = "This binary can not be converted to an Erlang term", + Warning = "This binary cannot be converted to an Erlang term", observer_html_lib:warning(Warning) end end. diff --git a/lib/observer/src/cdv_detail_wx.erl b/lib/observer/src/cdv_detail_wx.erl index 6c4739042b..4b1984c394 100644 --- a/lib/observer/src/cdv_detail_wx.erl +++ b/lib/observer/src/cdv_detail_wx.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2013-2017. All Rights Reserved. +%% Copyright Ericsson AB 2013-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/observer/src/cdv_dist_cb.erl b/lib/observer/src/cdv_dist_cb.erl index aeb34e5baf..ad1eb6e73c 100644 --- a/lib/observer/src/cdv_dist_cb.erl +++ b/lib/observer/src/cdv_dist_cb.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2013-2016. All Rights Reserved. +%% Copyright Ericsson AB 2013-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/observer/src/cdv_html_wx.erl b/lib/observer/src/cdv_html_wx.erl index 77a9d3207b..d9efa7fc2f 100644 --- a/lib/observer/src/cdv_html_wx.erl +++ b/lib/observer/src/cdv_html_wx.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2013-2016. All Rights Reserved. +%% Copyright Ericsson AB 2013-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/observer/src/cdv_info_wx.erl b/lib/observer/src/cdv_info_wx.erl index 07c28610e2..0a23d4ae3f 100644 --- a/lib/observer/src/cdv_info_wx.erl +++ b/lib/observer/src/cdv_info_wx.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2011-2016. All Rights Reserved. +%% Copyright Ericsson AB 2011-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/observer/src/cdv_mem_cb.erl b/lib/observer/src/cdv_mem_cb.erl index 925487786c..99ffdda765 100644 --- a/lib/observer/src/cdv_mem_cb.erl +++ b/lib/observer/src/cdv_mem_cb.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2011-2016. All Rights Reserved. +%% Copyright Ericsson AB 2011-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/observer/src/cdv_multi_wx.erl b/lib/observer/src/cdv_multi_wx.erl index 93f045b1da..79a44245aa 100644 --- a/lib/observer/src/cdv_multi_wx.erl +++ b/lib/observer/src/cdv_multi_wx.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2011-2016. All Rights Reserved. +%% Copyright Ericsson AB 2011-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/observer/src/cdv_port_cb.erl b/lib/observer/src/cdv_port_cb.erl index 6bb8f07a74..cbd5f696a6 100644 --- a/lib/observer/src/cdv_port_cb.erl +++ b/lib/observer/src/cdv_port_cb.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2013-2016. All Rights Reserved. +%% Copyright Ericsson AB 2013-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/observer/src/cdv_proc_cb.erl b/lib/observer/src/cdv_proc_cb.erl index 0ea23dd7cb..2497b4889e 100644 --- a/lib/observer/src/cdv_proc_cb.erl +++ b/lib/observer/src/cdv_proc_cb.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2013-2016. All Rights Reserved. +%% Copyright Ericsson AB 2013-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/observer/src/cdv_sched_cb.erl b/lib/observer/src/cdv_sched_cb.erl index d2696a276f..a3695a9418 100644 --- a/lib/observer/src/cdv_sched_cb.erl +++ b/lib/observer/src/cdv_sched_cb.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2013-2016. All Rights Reserved. +%% Copyright Ericsson AB 2013-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/observer/src/cdv_table_wx.erl b/lib/observer/src/cdv_table_wx.erl index ba23758ea6..0f28a51017 100644 --- a/lib/observer/src/cdv_table_wx.erl +++ b/lib/observer/src/cdv_table_wx.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2011-2016. All Rights Reserved. +%% Copyright Ericsson AB 2011-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/observer/src/cdv_term_cb.erl b/lib/observer/src/cdv_term_cb.erl index bdcb13f22d..85da1d227a 100644 --- a/lib/observer/src/cdv_term_cb.erl +++ b/lib/observer/src/cdv_term_cb.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2013-2016. All Rights Reserved. +%% Copyright Ericsson AB 2013-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -51,7 +51,7 @@ format_term_fun(Format,Term,Tab) -> try io_lib:format(Format,[Term]) of Str -> {expand, plain_html(Str), Tab} catch error:badarg -> - Warning = "This term can not be formatted with " ++ Format, + Warning = "This term cannot be formatted with " ++ Format, observer_html_lib:warning(Warning) after observer_lib:report_progress({ok,stop_pulse}) diff --git a/lib/observer/src/cdv_virtual_list_wx.erl b/lib/observer/src/cdv_virtual_list_wx.erl index 33c0c880b1..2702301021 100644 --- a/lib/observer/src/cdv_virtual_list_wx.erl +++ b/lib/observer/src/cdv_virtual_list_wx.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2013-2016. All Rights Reserved. +%% Copyright Ericsson AB 2013-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/observer/src/cdv_wx.erl b/lib/observer/src/cdv_wx.erl index 0e33c9e618..78a897111c 100644 --- a/lib/observer/src/cdv_wx.erl +++ b/lib/observer/src/cdv_wx.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2013-2016. All Rights Reserved. +%% Copyright Ericsson AB 2013-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/observer/src/crashdump_viewer.erl b/lib/observer/src/crashdump_viewer.erl index d0c14db486..14b086ff58 100644 --- a/lib/observer/src/crashdump_viewer.erl +++ b/lib/observer/src/crashdump_viewer.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2003-2017. All Rights Reserved. +%% Copyright Ericsson AB 2003-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/observer/src/crashdump_viewer.hrl b/lib/observer/src/crashdump_viewer.hrl index 252e19379d..856e558e6c 100644 --- a/lib/observer/src/crashdump_viewer.hrl +++ b/lib/observer/src/crashdump_viewer.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2003-2017. All Rights Reserved. +%% Copyright Ericsson AB 2003-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/observer/src/etop_tr.erl b/lib/observer/src/etop_tr.erl index 1e48fefca4..c7e62e8332 100644 --- a/lib/observer/src/etop_tr.erl +++ b/lib/observer/src/etop_tr.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2002-2016. All Rights Reserved. +%% Copyright Ericsson AB 2002-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/observer/src/multitrace.erl b/lib/observer/src/multitrace.erl index 82aec05e0d..35b70c63a3 100644 --- a/lib/observer/src/multitrace.erl +++ b/lib/observer/src/multitrace.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2002-2016. All Rights Reserved. +%% Copyright Ericsson AB 2002-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/observer/src/observer.app.src b/lib/observer/src/observer.app.src index cd4aa2e299..d73293a5f9 100644 --- a/lib/observer/src/observer.app.src +++ b/lib/observer/src/observer.app.src @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2002-2016. All Rights Reserved. +%% Copyright Ericsson AB 2002-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/observer/src/observer_alloc_wx.erl b/lib/observer/src/observer_alloc_wx.erl index e0f319b6cc..54e246f247 100644 --- a/lib/observer/src/observer_alloc_wx.erl +++ b/lib/observer/src/observer_alloc_wx.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2015-2017. All Rights Reserved. +%% Copyright Ericsson AB 2015-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/observer/src/observer_html_lib.erl b/lib/observer/src/observer_html_lib.erl index 8202f91030..0c4e32af49 100644 --- a/lib/observer/src/observer_html_lib.erl +++ b/lib/observer/src/observer_html_lib.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2003-2016. All Rights Reserved. +%% Copyright Ericsson AB 2003-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/observer/src/observer_lib.erl b/lib/observer/src/observer_lib.erl index 718ef91942..7c68b0ebb6 100644 --- a/lib/observer/src/observer_lib.erl +++ b/lib/observer/src/observer_lib.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2011-2017. All Rights Reserved. +%% Copyright Ericsson AB 2011-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/observer/src/observer_perf_wx.erl b/lib/observer/src/observer_perf_wx.erl index eddb9327f3..21c6d26f49 100644 --- a/lib/observer/src/observer_perf_wx.erl +++ b/lib/observer/src/observer_perf_wx.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2012-2017. All Rights Reserved. +%% Copyright Ericsson AB 2012-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/observer/src/observer_port_wx.erl b/lib/observer/src/observer_port_wx.erl index f7ae07fb85..445f3dd6b1 100644 --- a/lib/observer/src/observer_port_wx.erl +++ b/lib/observer/src/observer_port_wx.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2011-2017. All Rights Reserved. +%% Copyright Ericsson AB 2011-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/observer/src/observer_pro_wx.erl b/lib/observer/src/observer_pro_wx.erl index e2372cde33..04e654a37e 100644 --- a/lib/observer/src/observer_pro_wx.erl +++ b/lib/observer/src/observer_pro_wx.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2011-2017. All Rights Reserved. +%% Copyright Ericsson AB 2011-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/observer/src/observer_procinfo.erl b/lib/observer/src/observer_procinfo.erl index 6a85d29c1e..f436886735 100644 --- a/lib/observer/src/observer_procinfo.erl +++ b/lib/observer/src/observer_procinfo.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2011-2017. All Rights Reserved. +%% Copyright Ericsson AB 2011-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/observer/src/observer_traceoptions_wx.erl b/lib/observer/src/observer_traceoptions_wx.erl index ff8c40ce77..ea292b92af 100644 --- a/lib/observer/src/observer_traceoptions_wx.erl +++ b/lib/observer/src/observer_traceoptions_wx.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2011-2016. All Rights Reserved. +%% Copyright Ericsson AB 2011-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/observer/src/observer_tv_wx.erl b/lib/observer/src/observer_tv_wx.erl index 2e387f7e74..dfd19569fd 100644 --- a/lib/observer/src/observer_tv_wx.erl +++ b/lib/observer/src/observer_tv_wx.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2011-2017. All Rights Reserved. +%% Copyright Ericsson AB 2011-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -147,7 +147,7 @@ handle_event(#wx{event=#wxList{type=command_list_item_activated, State=#state{holder=Holder, node=Node, opts=#opts{type=Type}, grid=Grid}) -> case get_table(Holder, Index) of #tab{protection=private} -> - self() ! {error, "Table has 'private' protection and can not be read"}; + self() ! {error, "Table has 'private' protection and cannot be read"}; #tab{}=Table -> observer_tv_table:start_link(Grid, [{node,Node}, {type,Type}, {table,Table}]); _ -> ignore @@ -187,7 +187,7 @@ handle_event(#wx{id=?ID_SHOW_TABLE}, R when is_integer(R) -> case get_table(Holder, R) of #tab{protection=private} -> - self() ! {error, "Table has 'private' protection and can not be read"}; + self() ! {error, "Table has 'private' protection and cannot be read"}; #tab{}=Table -> observer_tv_table:start_link(Grid, [{node,Node}, {type,Type}, {table,Table}]); _ -> ignore diff --git a/lib/observer/src/observer_wx.erl b/lib/observer/src/observer_wx.erl index 453e3bdc2d..1f748cb8d2 100644 --- a/lib/observer/src/observer_wx.erl +++ b/lib/observer/src/observer_wx.erl @@ -806,7 +806,7 @@ is_rb_compatible(Node) -> is_rb_server_running(Node, LogState) -> %% If already started, somebody else may use it. - %% We can not use it too, as far log file would be overriden. Not fair. + %% We cannot use it too, as far log file would be overriden. Not fair. case rpc:block_call(Node, erlang, whereis, [rb_server]) of Pid when is_pid(Pid), (LogState == false) -> throw("Error: rb_server is already started and maybe used by someone."); diff --git a/lib/observer/src/ttb_et.erl b/lib/observer/src/ttb_et.erl index 1b828eebc0..f90a7f6dcf 100644 --- a/lib/observer/src/ttb_et.erl +++ b/lib/observer/src/ttb_et.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2002-2016. All Rights Reserved. +%% Copyright Ericsson AB 2002-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/observer/test/crashdump_helper.erl b/lib/observer/test/crashdump_helper.erl index b5e94a893a..145ff56b71 100644 --- a/lib/observer/test/crashdump_helper.erl +++ b/lib/observer/test/crashdump_helper.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2017. All Rights Reserved. +%% Copyright Ericsson AB 2007-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/observer/test/crashdump_viewer_SUITE.erl b/lib/observer/test/crashdump_viewer_SUITE.erl index 8bd7ad387b..864454cdff 100644 --- a/lib/observer/test/crashdump_viewer_SUITE.erl +++ b/lib/observer/test/crashdump_viewer_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2003-2016. All Rights Reserved. +%% Copyright Ericsson AB 2003-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/observer/test/crashdump_viewer_SUITE_data/old_format.dump b/lib/observer/test/crashdump_viewer_SUITE_data/old_format.dump index 2c8944fa9d..2737757bfa 100644 --- a/lib/observer/test/crashdump_viewer_SUITE_data/old_format.dump +++ b/lib/observer/test/crashdump_viewer_SUITE_data/old_format.dump @@ -1285,7 +1285,7 @@ sleep start_prim_loader erl_prim_loader set_path -'can not start loader' +'cannot start loader' add_to_kernel prim_load_flags '-loader' @@ -1304,7 +1304,7 @@ path_flags '-pz' get_boot not_found -'can not get bootfile' +'cannot get bootfile' 'bootfile format error' get_file script @@ -1349,7 +1349,7 @@ extension '-boot/1-fun-0-' '-bs2ss/1-fun-0-' '-bs2as/1-fun-0-' -'can not load' +'cannot load' 'unexpected command in bootfile' prim_inet open @@ -3071,7 +3071,7 @@ change_group change_time master start_slave -'can not get remote filer ' +'cannot get remote filer ' start_relay relay 'Port controlling ~w terminated in file_server' diff --git a/lib/observer/test/observer_SUITE.erl b/lib/observer/test/observer_SUITE.erl index fd4f93f662..75336cedcc 100644 --- a/lib/observer/test/observer_SUITE.erl +++ b/lib/observer/test/observer_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2006-2017. All Rights Reserved. +%% Copyright Ericsson AB 2006-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -301,7 +301,7 @@ table_win(Config) when is_list(Config) -> Notebook = setup_whitebox_testing(), Parent = get_top_level_parent(Notebook), TObj = observer_tv_table:start_link(Parent, [{node,node()}, {type,ets}, {table,#tab{name=foo, id=Table}}]), - %% Modal can not test edit.. + %% Modal cannot test edit.. %% TPid = wx_object:get_pid(TObj), %% TPid ! #wx{event=#wxList{type=command_list_item_activated, itemIndex=12}}, timer:sleep(3000), diff --git a/lib/observer/test/ttb_SUITE.erl b/lib/observer/test/ttb_SUITE.erl index ed62efbb58..33133dd78d 100644 --- a/lib/observer/test/ttb_SUITE.erl +++ b/lib/observer/test/ttb_SUITE.erl @@ -2,7 +2,7 @@ %% %CopyrightBegin% %% %% -%% Copyright Ericsson AB 2002-2016. All Rights Reserved. +%% Copyright Ericsson AB 2002-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/observer/vsn.mk b/lib/observer/vsn.mk index 74a6db768e..bf6fd2a421 100644 --- a/lib/observer/vsn.mk +++ b/lib/observer/vsn.mk @@ -1 +1 @@ -OBSERVER_VSN = 2.7 +OBSERVER_VSN = 2.8 diff --git a/lib/odbc/doc/src/Makefile b/lib/odbc/doc/src/Makefile index b3f93a7e9c..553db8a9a4 100644 --- a/lib/odbc/doc/src/Makefile +++ b/lib/odbc/doc/src/Makefile @@ -101,6 +101,7 @@ html: gifs $(HTML_REF_MAN_FILE) clean clean_docs: rm -rf $(HTMLDIR)/* + rm -rf $(XMLDIR) rm -f $(MAN3DIR)/* rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) rm -f errs core *~ diff --git a/lib/odbc/doc/src/databases.xml b/lib/odbc/doc/src/databases.xml index a060f87e46..614c327a46 100644 --- a/lib/odbc/doc/src/databases.xml +++ b/lib/odbc/doc/src/databases.xml @@ -65,7 +65,7 @@ <p>Another obstacle is that some drivers do not support scrollable cursors which has the effect that the only way to traverse the result set is sequentially, with next, from the first row to the - last, and once you pass a row you can not go back. This means + last, and once you pass a row you cannot go back. This means that some functions in the interface will not work together with certain drivers. A similar problem is that not all drivers support "row count" for select queries, hence resulting in that diff --git a/lib/odbc/doc/src/error_handling.xml b/lib/odbc/doc/src/error_handling.xml index 4c80cff12f..8d794d2c99 100644 --- a/lib/odbc/doc/src/error_handling.xml +++ b/lib/odbc/doc/src/error_handling.xml @@ -138,7 +138,7 @@ should be run through the erlang port was not compiled for your platform.</item> <item>Errors discovered by the ODBC driver - If calls to the - ODBC-driver fails due to circumstances that can not be + ODBC-driver fails due to circumstances that cannot be controlled by the Erlang ODBC application programmer, an error string will be dug up from the driver. This string will be the <c>Reason</c> in the <c>{error, Reason} </c> diff --git a/lib/odbc/doc/src/getting_started.xml b/lib/odbc/doc/src/getting_started.xml index f2ff8b8993..b27754ff22 100644 --- a/lib/odbc/doc/src/getting_started.xml +++ b/lib/odbc/doc/src/getting_started.xml @@ -50,7 +50,7 @@ and paths to appropriate values. This may differ a lot between different os's, databases and ODBC drivers. This is a configuration problem related to the third party product - and hence we can not give you a standard solution in this guide.</item> + and hence we cannot give you a standard solution in this guide.</item> <item>The Erlang ODBC application consists of both <c>Erlang</c> and <c>C</c> code. The <c>C</c> code is delivered as a precompiled executable for windows, solaris and linux (SLES10) in the commercial diff --git a/lib/odbc/doc/src/notes_history.xml b/lib/odbc/doc/src/notes_history.xml index 22a92f67cd..d0c0a472e7 100644 --- a/lib/odbc/doc/src/notes_history.xml +++ b/lib/odbc/doc/src/notes_history.xml @@ -63,7 +63,7 @@ <list type="bulleted"> <item> <p>The erlang odbc process will now die normally if a - connection can not be established. No connection no + connection cannot be established. No connection no process it is expected. And as the client has already received the error message that would be the reason with which the erlang process would be stopped, the supervisor @@ -193,7 +193,7 @@ </item> <item> <p>The erlang odbc process will now die normally if a - connection can not be established. No connection no + connection cannot be established. No connection no process it is expected. And as the client has already received the error message that would be the reason with which the erlang process would be stopped, the supervisor diff --git a/lib/odbc/doc/src/odbc.xml b/lib/odbc/doc/src/odbc.xml index 4bb1f035f9..9760908855 100644 --- a/lib/odbc/doc/src/odbc.xml +++ b/lib/odbc/doc/src/odbc.xml @@ -246,7 +246,7 @@ <p>Closes a connection to a database. This will also terminate all processes that may have been spawned when the connection was opened. This call will always succeed. - If the connection can not be disconnected gracefully it will + If the connection cannot be disconnected gracefully it will be brutally killed. However you may receive an error message as result if you try to disconnect a connection started by another process. diff --git a/lib/odbc/test/odbc_connect_SUITE.erl b/lib/odbc/test/odbc_connect_SUITE.erl index 261dfc6f20..94ca62b3fb 100644 --- a/lib/odbc/test/odbc_connect_SUITE.erl +++ b/lib/odbc/test/odbc_connect_SUITE.erl @@ -259,7 +259,7 @@ not_exist_db(_Config) -> %%------------------------------------------------------------------------- no_c_executable() -> - [{doc,"Test what happens if the port-program can not be found"}]. + [{doc,"Test what happens if the port-program cannot be found"}]. no_c_executable(_Config) -> process_flag(trap_exit, true), Dir = filename:nativename(filename:join(code:priv_dir(odbc), diff --git a/lib/os_mon/doc/src/Makefile b/lib/os_mon/doc/src/Makefile index 4aa8879a91..354f8ed26b 100644 --- a/lib/os_mon/doc/src/Makefile +++ b/lib/os_mon/doc/src/Makefile @@ -98,6 +98,7 @@ debug opt: clean clean_docs: rm -rf $(HTMLDIR)/* + rm -rf $(XMLDIR) rm -f $(MAN3DIR)/* rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) rm -f errs core *~ diff --git a/lib/os_mon/doc/src/notes.xml b/lib/os_mon/doc/src/notes.xml index cec0856a8b..4a878dd704 100644 --- a/lib/os_mon/doc/src/notes.xml +++ b/lib/os_mon/doc/src/notes.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2004</year><year>2016</year> + <year>2004</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -31,6 +31,22 @@ </header> <p>This document describes the changes made to the OS_Mon application.</p> +<section><title>Os_Mon 2.4.5</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fix <c>disksup</c> to handle mount paths with spaces in + them.</p> + <p> + Own Id: OTP-14513</p> + </item> + </list> + </section> + +</section> + <section><title>Os_Mon 2.4.4</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/os_mon/doc/src/os_mon_mib.xml b/lib/os_mon/doc/src/os_mon_mib.xml index dcf3649876..e995bf3de1 100644 --- a/lib/os_mon/doc/src/os_mon_mib.xml +++ b/lib/os_mon/doc/src/os_mon_mib.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>2004</year><year>2016</year> + <year>2004</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -35,7 +35,10 @@ <p>Functions for loading and unloading the OTP-OS-MON-MIB into/from an SNMP agent. The instrumentation of the OTP-OS-MON-MIB uses Mnesia, hence Mnesia must be started prior to loading - the OTP-OS-MON-MIB.</p> + the OTP-OS-MON-MIB.</p> + <warning> + <p>This module has been deprecated and will be removed in a furture release.</p> + </warning> </description> <funcs> <func> diff --git a/lib/os_mon/src/cpu_sup.erl b/lib/os_mon/src/cpu_sup.erl index c1c972b5b1..81e049ef22 100644 --- a/lib/os_mon/src/cpu_sup.erl +++ b/lib/os_mon/src/cpu_sup.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2016. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/os_mon/src/memsup.erl b/lib/os_mon/src/memsup.erl index a30d962ad4..9d6447430d 100644 --- a/lib/os_mon/src/memsup.erl +++ b/lib/os_mon/src/memsup.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/os_mon/test/cpu_sup_SUITE.erl b/lib/os_mon/test/cpu_sup_SUITE.erl index 7122d23503..ba28f31f26 100644 --- a/lib/os_mon/test/cpu_sup_SUITE.erl +++ b/lib/os_mon/test/cpu_sup_SUITE.erl @@ -122,19 +122,19 @@ util_api(Config) when is_list(Config) -> %% util([]) {all, Busy1, NonBusy1, []} = cpu_sup:util([]), - 100.00 = Busy1 + NonBusy1, + true = tiny_diff(100.00, Busy1 + NonBusy1), %% util([detailed]) {Cpus2, Busy2, NonBusy2, []} = cpu_sup:util([detailed]), true = lists:all(fun(X) -> is_integer(X) end, Cpus2), true = lists:all(BusyP, Busy2), true = lists:all(NonBusyP, NonBusy2), - 100.00 = lists:foldl(Sum,0,Busy2)+lists:foldl(Sum,0,NonBusy2), + true = tiny_diff(100.00, lists:foldl(Sum,0,Busy2)+lists:foldl(Sum,0,NonBusy2)), %% util([per_cpu]) [{Cpu3, Busy3, NonBusy3, []}|_] = cpu_sup:util([per_cpu]), true = is_integer(Cpu3), - 100.00 = Busy3 + NonBusy3, + true = tiny_diff(100.00, Busy3 + NonBusy3), %% util([detailed, per_cpu]) [{Cpu4, Busy4, NonBusy4, []}|_] = @@ -142,7 +142,7 @@ util_api(Config) when is_list(Config) -> true = is_integer(Cpu4), true = lists:all(BusyP, Busy2), true = lists:all(NonBusyP, NonBusy2), - 100.00 = lists:foldl(Sum,0,Busy4)+lists:foldl(Sum,0,NonBusy4), + true = tiny_diff(100.00, lists:foldl(Sum,0,Busy4)+lists:foldl(Sum,0,NonBusy4)), %% bad util/1 calls {'EXIT',{badarg,_}} = (catch cpu_sup:util(detailed)), @@ -150,6 +150,9 @@ util_api(Config) when is_list(Config) -> ok. +tiny_diff(A, B) -> + (abs(A - B) < 1.0e-11). + -define(SPIN_TIME, 1000). %% Test utilization values diff --git a/lib/os_mon/test/disksup_SUITE.erl b/lib/os_mon/test/disksup_SUITE.erl index d7f2626160..fe27ea9046 100644 --- a/lib/os_mon/test/disksup_SUITE.erl +++ b/lib/os_mon/test/disksup_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/os_mon/vsn.mk b/lib/os_mon/vsn.mk index eb4f13ea9e..4a327e5506 100644 --- a/lib/os_mon/vsn.mk +++ b/lib/os_mon/vsn.mk @@ -1 +1 @@ -OS_MON_VSN = 2.4.4 +OS_MON_VSN = 2.4.5 diff --git a/lib/otp_mibs/doc/src/Makefile b/lib/otp_mibs/doc/src/Makefile index c65e2a8e3c..143d9befb8 100644 --- a/lib/otp_mibs/doc/src/Makefile +++ b/lib/otp_mibs/doc/src/Makefile @@ -93,6 +93,7 @@ debug opt: clean clean_docs: rm -rf $(HTMLDIR)/* + rm -rf $(XMLDIR) rm -f $(MAN3DIR)/* rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) rm -f errs core *~ diff --git a/lib/otp_mibs/doc/src/notes.xml b/lib/otp_mibs/doc/src/notes.xml index c99148a904..ac0777d225 100644 --- a/lib/otp_mibs/doc/src/notes.xml +++ b/lib/otp_mibs/doc/src/notes.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2004</year><year>2016</year> + <year>2004</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -32,6 +32,22 @@ <p>This document describes the changes made to the OTP_Mibs application.</p> +<section><title>Otp_Mibs 1.2</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + The otp_mibs application has been deprecated and will be + removed in a future release.</p> + <p> + Own Id: OTP-15141</p> + </item> + </list> + </section> + +</section> + <section><title>Otp_Mibs 1.1.2</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/otp_mibs/doc/src/otp_mib.xml b/lib/otp_mibs/doc/src/otp_mib.xml index 681ce9fa2d..530c529c69 100644 --- a/lib/otp_mibs/doc/src/otp_mib.xml +++ b/lib/otp_mibs/doc/src/otp_mib.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>2003</year><year>2016</year> + <year>2003</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -34,7 +34,10 @@ <p>The SNMP application should be used to start an SNMP agent. Then the API functions below can be used to load/unload the OTP-MIB into/from the agent. The instrumentation of the OTP-MIB uses - Mnesia, hence Mnesia must be started prior to loading the OTP-MIB.</p> + Mnesia, hence Mnesia must be started prior to loading the OTP-MIB.</p> + <warning> + <p>This application has been deprecated and will be removed in a furture release.</p> + </warning> </description> <funcs> <func> diff --git a/lib/otp_mibs/src/otp_mib.erl b/lib/otp_mibs/src/otp_mib.erl index 1431818be4..ca868f2817 100644 --- a/lib/otp_mibs/src/otp_mib.erl +++ b/lib/otp_mibs/src/otp_mib.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -36,6 +36,9 @@ %% Exported for internal use via rpc -export([get_erl_node/1, get_appls/1]). +-deprecated([{load,1,eventually}, + {unload,1,eventually}]). + %% Shadow tables -record(erlNodeTable, {erlNodeId, erlNodeName, erlNodeMachine, erlNodeVersion, diff --git a/lib/otp_mibs/vsn.mk b/lib/otp_mibs/vsn.mk index 13406cb5b1..2e9e367d98 100644 --- a/lib/otp_mibs/vsn.mk +++ b/lib/otp_mibs/vsn.mk @@ -1,4 +1,4 @@ -OTP_MIBS_VSN = 1.1.2 +OTP_MIBS_VSN = 1.2 # Note: The branch 'otp_mibs' is defunct as of otp_mibs-1.0.4 and # should NOT be used again. diff --git a/lib/parsetools/doc/src/Makefile b/lib/parsetools/doc/src/Makefile index e4cd2c0a76..a40ccd0fe7 100644 --- a/lib/parsetools/doc/src/Makefile +++ b/lib/parsetools/doc/src/Makefile @@ -89,6 +89,7 @@ html: gifs $(HTML_REF_MAN_FILE) clean clean_docs: rm -rf $(HTMLDIR)/* + rm -rf $(XMLDIR) rm -f $(MAN3DIR)/* rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) rm -f errs core *~ diff --git a/lib/parsetools/doc/src/notes.xml b/lib/parsetools/doc/src/notes.xml index b3370a06ab..3e999c8cd0 100644 --- a/lib/parsetools/doc/src/notes.xml +++ b/lib/parsetools/doc/src/notes.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>1997</year><year>2017</year> + <year>1997</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -31,6 +31,21 @@ </header> <p>This document describes the changes made to the Parsetools application.</p> +<section><title>Parsetools 2.1.7</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> Calls to <c>erlang:get_stacktrace()</c> are removed. + </p> + <p> + Own Id: OTP-14861</p> + </item> + </list> + </section> + +</section> + <section><title>Parsetools 2.1.6</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/parsetools/include/yeccpre.hrl b/lib/parsetools/include/yeccpre.hrl index 53e8124aaa..562f17c19e 100644 --- a/lib/parsetools/include/yeccpre.hrl +++ b/lib/parsetools/include/yeccpre.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2017. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/parsetools/src/yecc.erl b/lib/parsetools/src/yecc.erl index ce1b9468fd..3343a4282b 100644 --- a/lib/parsetools/src/yecc.erl +++ b/lib/parsetools/src/yecc.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2017. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/parsetools/src/yeccparser.erl b/lib/parsetools/src/yeccparser.erl index 45d91a83a3..0deecc7879 100644 --- a/lib/parsetools/src/yeccparser.erl +++ b/lib/parsetools/src/yeccparser.erl @@ -17,7 +17,7 @@ anno_of(Token) -> %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2017. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/parsetools/test/yecc_SUITE.erl b/lib/parsetools/test/yecc_SUITE.erl index 2d3b15326b..715e50c301 100644 --- a/lib/parsetools/test/yecc_SUITE.erl +++ b/lib/parsetools/test/yecc_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2017. All Rights Reserved. +%% Copyright Ericsson AB 2005-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/parsetools/vsn.mk b/lib/parsetools/vsn.mk index b6d2ce0cd4..210723e449 100644 --- a/lib/parsetools/vsn.mk +++ b/lib/parsetools/vsn.mk @@ -1 +1 @@ -PARSETOOLS_VSN = 2.1.6 +PARSETOOLS_VSN = 2.1.7 diff --git a/lib/public_key/doc/src/Makefile b/lib/public_key/doc/src/Makefile index f5157fe87a..03467e9783 100644 --- a/lib/public_key/doc/src/Makefile +++ b/lib/public_key/doc/src/Makefile @@ -99,6 +99,7 @@ html: gifs $(HTML_REF_MAN_FILE) clean clean_docs: rm -rf $(HTMLDIR)/* + rm -rf $(XMLDIR) rm -f $(MAN3DIR)/* rm -f $(MAN6DIR)/* rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) diff --git a/lib/public_key/doc/src/notes.xml b/lib/public_key/doc/src/notes.xml index 11012ee9e5..204520473a 100644 --- a/lib/public_key/doc/src/notes.xml +++ b/lib/public_key/doc/src/notes.xml @@ -5,7 +5,7 @@ <header> <copyright> <year>2008</year> - <year>2017</year> + <year>2018</year> <holder>Ericsson AB, All Rights Reserved</holder> </copyright> <legalnotice> @@ -35,6 +35,83 @@ <file>notes.xml</file> </header> +<section><title>Public_Key 1.6.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Some of the keylengths in the newly generated moduli file + in public_key are not universally supported. This could + cause the SSH key exchange + diffie-hellman-group-exchange-sha* to fail.</p> + <p> + Those keylengths are now removed.</p> + <p> + Own Id: OTP-15151 Aux Id: OTP-15113 </p> + </item> + </list> + </section> + +</section> + +<section><title>Public_Key 1.6</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Update calls to the base64 module to conform to that + module's type specifications.</p> + <p> + Own Id: OTP-14788 Aux Id: OTP-14624 </p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Use uri_string module instead of http_uri.</p> + <p> + Own Id: OTP-14902</p> + </item> + <item> + <p> + A new function - + <c>public_key:pkix_verify_hostname_match_fun/1</c> - + returns a fun to be given as option <c>match_fun</c> to + <c>public_key:pkix_verify_hostname/3</c> or via ssl.</p> + <p> + The fun makes the verify hostname matching according to + the specific rules for the protocol in the argument. + Presently only <c>https</c> is supported.</p> + <p> + Own Id: OTP-14962 Aux Id: ERL-542, OTP-15102 </p> + </item> + <item> + <p> + Compleate PKCS-8 encoding support and enhance the + decoding of 'PrivateKeyInfo' to conform to the rest of + Erlang public_key API.</p> + <p> + Own Id: OTP-15093</p> + </item> + <item> + <p> + A new moduli file is generated. This file is used for the + recommended <c>diffie-hellman-group-exchange-sha256</c> + key exchange algorithm in SSH.</p> + <p> + Own Id: OTP-15113</p> + </item> + </list> + </section> + +</section> + <section><title>Public_Key 1.5.2</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/public_key/doc/src/public_key.xml b/lib/public_key/doc/src/public_key.xml index 5d57109140..aaccb5088b 100644 --- a/lib/public_key/doc/src/public_key.xml +++ b/lib/public_key/doc/src/public_key.xml @@ -5,7 +5,7 @@ <header> <copyright> <year>2008</year> - <year>2017</year> + <year>2018</year> <holder>Ericsson AB, All Rights Reserved</holder> </copyright> <legalnotice> @@ -682,7 +682,7 @@ fun(#'DistributionPoint'{}, #'CertificateList'{}, <tag>{undetermined_details, boolean()}</tag> <item> - <p>Defaults to false. When revocation status can not be + <p>Defaults to false. When revocation status cannot be determined, and this option is set to true, details of why no CRLs where accepted are included in the return value.</p> </item> @@ -902,7 +902,7 @@ fun(#'DistributionPoint'{}, #'CertificateList'{}, <note><p> Note that the generated certificates and keys does not provide a formally correct PKIX-trust-chain - and they can not be used to achieve real security. This function is provided for testing purposes only. + and they cannot be used to achieve real security. This function is provided for testing purposes only. </p></note> </desc> </func> @@ -969,35 +969,76 @@ fun(#'DistributionPoint'{}, #'CertificateList'{}, </type> <desc> <p>This function checks that the <i>Presented Identifier</i> (e.g hostname) in a peer certificate - is in agreement with the <i>Reference Identifier</i> that the client expects to be connected to. + is in agreement with at least one of the <i>Reference Identifier</i> that the client expects to be connected to. The function is intended to be added as an extra client check of the peer certificate when performing <seealso marker="public_key:public_key#pkix_path_validation-3">public_key:pkix_path_validation/3</seealso> </p> <p>See <url href="https://tools.ietf.org/html/rfc6125">RFC 6125</url> for detailed information about hostname verification. - The <seealso marker="using_public_key#verify_hostname">User's Manual</seealso> + The <seealso marker="using_public_key#verify_hostname">User's Guide</seealso> and <seealso marker="using_public_key#verify_hostname_examples">code examples</seealso> describes this function more detailed. </p> <p>The <c>{OtherRefId,term()}</c> is defined by the user and is passed to the <c>match_fun</c>, if defined. - If that term is a binary, it will be converted to a string. + If the term in <c>OtherRefId</c> is a binary, it will be converted to a string. </p> <p>The <c>ip</c> Reference ID takes an <seealso marker="inet:inet#type-ip_address">inet:ip_address()</seealso> or an ip address in string format (E.g "10.0.1.1" or "1234::5678:9012") as second element. </p> - <p>See <seealso marker="#pkix_verify_hostname_match_fun-1">pkix_verify_hostname_match_fun/1</seealso> for a - function that return a fun suitable for this option. - </p> + <p>The options are:</p> + <taglist> + <tag><c>match_fun</c></tag> + <item> + The <c>fun/2</c> in this option replaces the default host name matching rules. The fun should return a + boolean to tell if the Reference ID and Presented ID matches or not. The fun can also return a third + value, the atom <c>default</c>, if the default matching rules shall apply. + This makes it possible to augment the tests with a special case: + <code> +fun(....) -> true; % My special case + (_, _) -> default % all others falls back to the inherit tests +end + </code> + <br/>See <seealso marker="#pkix_verify_hostname_match_fun-1">pkix_verify_hostname_match_fun/1</seealso> for a + function that takes a protocol name as argument and returns a <c>fun/2</c> suitable for this option and + <seealso marker="using_public_key#redefining_match_op">Re-defining the match operation</seealso> + in the User's Guide for an example. + </item> + + <tag><c>fail_callback</c></tag> + <item>If a matching fails, there could be circumstances when the certificate should be accepted anyway. Think for + example of a web browser where you choose to accept an outdated certificate. This option enables implementation + of such a function. This <c>fun/1</c> is called when no <c>ReferenceID</c> matches. The return value of the fun + (a <c>boolean()</c>) decides the outcome. If <c>true</c> the the certificate is accepted otherwise + it is rejected. See + <seealso marker="using_public_key#-pinning--a-certificate">"Pinning" a Certificate</seealso> + in the User's Guide. + </item> + + <tag><c>fqdn_fun</c></tag> + <item>This option augments the host name extraction from URIs and other Reference IDs. It could for example be + a very special URI that is not standardised. The fun takes a Reference ID as argument and returns one of: + <list> + <item>the hostname</item> + <item>the atom <c>default</c>: the default host name extract function will be used</item> + <item>the atom <c>undefined</c>: a host name could not be extracted. The pkix_verify_hostname/3 + will return <c>false</c>.</item> + </list> + <br/>For an example, see + <seealso marker="using_public_key#hostname_extraction">Hostname extraction</seealso> + in the User's Guide. + </item> + </taglist> + </desc> </func> <func> - <name>pkix_verify_hostname_match_fun(Alg) -> fun(RefId | FQDN::string(), PresentedID) -> boolean() | default</name> + <name>pkix_verify_hostname_match_fun(Protcol) -> fun(RefId | FQDN::string(), PresentedID) -> boolean() | default</name> <fsummary>Returns a fun that is intendended as argument to the match_fun option in pkix_verify_hostname/3. </fsummary> <type> - <v>Alg = https</v> + <v>Protocol = https</v> <d>The algorithm for wich the fun should implement the special matching rules</d> <v>RefId</v> <d>See <seealso marker="#pkix_verify_hostname-3">pkix_verify_hostname/3</seealso>.</d> diff --git a/lib/public_key/doc/src/public_key_records.xml b/lib/public_key/doc/src/public_key_records.xml index 9ebdbb244d..d26867c12f 100644 --- a/lib/public_key/doc/src/public_key_records.xml +++ b/lib/public_key/doc/src/public_key_records.xml @@ -5,7 +5,7 @@ <header> <copyright> <year>2008</year> - <year>2015</year> + <year>2018</year> <holder>Ericsson AB, All Rights Reserved</holder> </copyright> <legalnotice> diff --git a/lib/public_key/doc/src/using_public_key.xml b/lib/public_key/doc/src/using_public_key.xml index 417d479da3..de0a6596c3 100644 --- a/lib/public_key/doc/src/using_public_key.xml +++ b/lib/public_key/doc/src/using_public_key.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2011</year><year>2016</year> + <year>2011</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -570,6 +570,7 @@ true = public_key:verify(Digest, none, Signature, PublicKey),</code> <c>fqdn_fun</c> and <c>match_fun</c>. </p> <section> + <marker id="hostname_extraction"></marker> <title>Hostname extraction</title> <p>The <c>fqdn_fun</c> extracts hostnames (Fully Qualified Domain Names) from uri_id or other ReferenceIDs that are not pre-defined in the public_key function. @@ -595,7 +596,8 @@ true = public_key:verify(Digest, none, Signature, PublicKey),</code> </code> </section> <section> - <title>Re-defining the match operations</title> + <marker id="redefining_match_op"></marker> + <title>Re-defining the match operation</title> <p>The default matching handles dns_id and uri_id. In an uri_id the value is tested for equality with a value from the <c>Subject Alternate Name</c>. If som other kind of matching is needed, use the <c>match_fun</c> option. diff --git a/lib/public_key/include/public_key.hrl b/lib/public_key/include/public_key.hrl index 663e1856ac..98d032bfdd 100644 --- a/lib/public_key/include/public_key.hrl +++ b/lib/public_key/include/public_key.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2016. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/public_key/priv/moduli b/lib/public_key/priv/moduli index 0c44d525ba..83e8767a5e 100644 --- a/lib/public_key/priv/moduli +++ b/lib/public_key/priv/moduli @@ -172,40 +172,6 @@ 20180605201507 2 6 100 4095 5 C99ACF60A371F11C9A02EA6920EA7F8F897742FD159713DE275C9C05059C86D7433F57043C70CC9D33F1E3CE222BB1991703F5F9688AAAF21305A0C263C82CAC5174B2621A318E14B1D7888B5F7ECDD4737101DCEC368A64AB38430AE56C3A6300E36FE8FB57C3619327C62ADC80671F67DC772A9C44EFB266D1120F9B9E7DEE5E3D092D6560D2794BFE19391FF4D11D5860DA7D110F3E13F6503765AA4060394E92F6E4301AEB248BFEA34182444B09732F95AA9B0FAAE1DC295022F1267099DE7E23B50D85585BED875C529FA59D642A09743222FDA04031281D729E9C2E9FC3E22FAE8C1AEA0A0F2FF7EC0C32E9B6B190B71917FFB8949A3ADB771843AE1DD39A3DAE9521CE2D10D2D12584E9289D849DBB9A52C7D990FA52EF786A0CA3D33EEDDD1F7350F501EC0ADAFBCA1B825FE995BF373853AC3FE33E48E728D918FA9A68B0E60F283EC8232B8F00984DB33C8800F9800183709C71F7CD5DF5837799837B473BFDB77E579DD76A2A239BFE09D0E32401B7D784391EEC064DF984E29F5C67F7F039BC8F156407AABD94879B4ABD773DA5506743A2E1AC7A5250ED51DD3D30E2CCC63971F1C72E7F424009884416F869A701C5257E9A6A1580324971575BF2999D37088AFB1B57B6F582007B7927CDE06166F2E500B5AC2C5143E6BE0C1A9AF4FEC2C9F688C86EB7DCB6A2C0F9CF6E4901361CAFC40EC640F737264DFF 20180605201935 2 6 100 4095 2 C99ACF60A371F11C9A02EA6920EA7F8F897742FD159713DE275C9C05059C86D7433F57043C70CC9D33F1E3CE222BB1991703F5F9688AAAF21305A0C263C82CAC5174B2621A318E14B1D7888B5F7ECDD4737101DCEC368A64AB38430AE56C3A6300E36FE8FB57C3619327C62ADC80671F67DC772A9C44EFB266D1120F9B9E7DEE5E3D092D6560D2794BFE19391FF4D11D5860DA7D110F3E13F6503765AA4060394E92F6E4301AEB248BFEA34182444B09732F95AA9B0FAAE1DC295022F1267099DE7E23B50D85585BED875C529FA59D642A09743222FDA04031281D729E9C2E9FC3E22FAE8C1AEA0A0F2FF7EC0C32E9B6B190B71917FFB8949A3ADB771843AE1DD39A3DAE9521CE2D10D2D12584E9289D849DBB9A52C7D990FA52EF786A0CA3D33EEDDD1F7350F501EC0ADAFBCA1B825FE995BF373853AC3FE33E48E728D918FA9A68B0E60F283EC8232B8F00984DB33C8800F9800183709C71F7CD5DF5837799837B473BFDB77E579DD76A2A239BFE09D0E32401B7D784391EEC064DF984E29F5C67F7F039BC8F156407AABD94879B4ABD773DA5506743A2E1AC7A5250ED51DD3D30E2CCC63971F1C72E7F424009884416F869A701C5257E9A6A1580324971575BF2999D37088AFB1B57B6F582007B7927CDE06166F2E500B5AC2C5143E6BE0C1A9AF4FEC2C9F688C86EB7DCB6A2C0F9CF6E4901361CAFC40EC640F737F47113 20180605202538 2 6 100 4095 2 C99ACF60A371F11C9A02EA6920EA7F8F897742FD159713DE275C9C05059C86D7433F57043C70CC9D33F1E3CE222BB1991703F5F9688AAAF21305A0C263C82CAC5174B2621A318E14B1D7888B5F7ECDD4737101DCEC368A64AB38430AE56C3A6300E36FE8FB57C3619327C62ADC80671F67DC772A9C44EFB266D1120F9B9E7DEE5E3D092D6560D2794BFE19391FF4D11D5860DA7D110F3E13F6503765AA4060394E92F6E4301AEB248BFEA34182444B09732F95AA9B0FAAE1DC295022F1267099DE7E23B50D85585BED875C529FA59D642A09743222FDA04031281D729E9C2E9FC3E22FAE8C1AEA0A0F2FF7EC0C32E9B6B190B71917FFB8949A3ADB771843AE1DD39A3DAE9521CE2D10D2D12584E9289D849DBB9A52C7D990FA52EF786A0CA3D33EEDDD1F7350F501EC0ADAFBCA1B825FE995BF373853AC3FE33E48E728D918FA9A68B0E60F283EC8232B8F00984DB33C8800F9800183709C71F7CD5DF5837799837B473BFDB77E579DD76A2A239BFE09D0E32401B7D784391EEC064DF984E29F5C67F7F039BC8F156407AABD94879B4ABD773DA5506743A2E1AC7A5250ED51DD3D30E2CCC63971F1C72E7F424009884416F869A701C5257E9A6A1580324971575BF2999D37088AFB1B57B6F582007B7927CDE06166F2E500B5AC2C5143E6BE0C1A9AF4FEC2C9F688C86EB7DCB6A2C0F9CF6E4901361CAFC40EC640F7390F0743 -20180605210627 2 6 100 5121 2 37585C4D659694D0B23500E18769F1EF93A776FF9FB92171358D9FE60A517E2CB91A4984A7E923B213CA5ED889D7CBD176B7B475315CB90EBD9920E41043A6C0BD37A8E2679AA7944ED273FDB1CBCAED3B2950BC73132CDE38789421CDDFAEC1C7662F3F9C3EA0C56758D7D3D0257499CE832FEF537F1BC17FFCB2360465A663AB629EF222B7C1E2ACF6368963B6A3D89E5617765B01D483C3BEB0F40D2BC4F4E0512CE41796C411A1EF20D70ED4F071EC56A24C52161D82B9222A52D56B0E48E118DDD1A5C01042E401B6724C80E939B3140A471CEA8C28E33883903283153A3EED07A506818930D61B420B0FF8D1B5F33A2E112B363D2B85A9FDE12AAA3DAE0E08945F4CE4E038A1B61624DD76CADFE172B0FE3C332F7B014634E7F4F80268EB6F1261F0667BF1FC4D9F0BA87C4CC42AC1261F7DA7C4508E3A70485F3D41B5B7AFA8DF0CA0CE61AF7A696B769FF69AB808AED3AE33F40E5E6DA7F8849349FB4A262965160C155B86AA0E13A8FB5EF1BD477790FD8949970B71DD508C971A88781DADEF4F67C1BF8E6BE1AC75B9E6CD74C471FB2D94B27269A9E30B671A52ABE673DAD632C1CC9C139468B852C06844D0D05A40D3A50D410DC4B8DD5B118626128CC5A2DA0093BE486F5D30E621DDB217B03AB97097733BD6E8D1CCFC7515D8761CF7EB0525483678785E29093865A60DA525D0A1D282501C27B1F6C7FEC64BED44F5409889BE78E049E22C5095E7EF4C9B7E88A49DF5CDBE6016A13072A28500AC00E64DB3CE3EBFC668CA0CB94CDDE8381F342E5CE4E2B1FC3067A295771DD0A8FFF308D5A47A91F1A67D8F77C3D20B88EAF17915867EE181DBC2DE8DF43B7179396A563292DB2966E79672BF8B7D0D0A3398A2E374C0E4A1C91DFBDE9C663 -20180605211229 2 6 100 5121 5 37585C4D659694D0B23500E18769F1EF93A776FF9FB92171358D9FE60A517E2CB91A4984A7E923B213CA5ED889D7CBD176B7B475315CB90EBD9920E41043A6C0BD37A8E2679AA7944ED273FDB1CBCAED3B2950BC73132CDE38789421CDDFAEC1C7662F3F9C3EA0C56758D7D3D0257499CE832FEF537F1BC17FFCB2360465A663AB629EF222B7C1E2ACF6368963B6A3D89E5617765B01D483C3BEB0F40D2BC4F4E0512CE41796C411A1EF20D70ED4F071EC56A24C52161D82B9222A52D56B0E48E118DDD1A5C01042E401B6724C80E939B3140A471CEA8C28E33883903283153A3EED07A506818930D61B420B0FF8D1B5F33A2E112B363D2B85A9FDE12AAA3DAE0E08945F4CE4E038A1B61624DD76CADFE172B0FE3C332F7B014634E7F4F80268EB6F1261F0667BF1FC4D9F0BA87C4CC42AC1261F7DA7C4508E3A70485F3D41B5B7AFA8DF0CA0CE61AF7A696B769FF69AB808AED3AE33F40E5E6DA7F8849349FB4A262965160C155B86AA0E13A8FB5EF1BD477790FD8949970B71DD508C971A88781DADEF4F67C1BF8E6BE1AC75B9E6CD74C471FB2D94B27269A9E30B671A52ABE673DAD632C1CC9C139468B852C06844D0D05A40D3A50D410DC4B8DD5B118626128CC5A2DA0093BE486F5D30E621DDB217B03AB97097733BD6E8D1CCFC7515D8761CF7EB0525483678785E29093865A60DA525D0A1D282501C27B1F6C7FEC64BED44F5409889BE78E049E22C5095E7EF4C9B7E88A49DF5CDBE6016A13072A28500AC00E64DB3CE3EBFC668CA0CB94CDDE8381F342E5CE4E2B1FC3067A295771DD0A8FFF308D5A47A91F1A67D8F77C3D20B88EAF17915867EE181DBC2DE8DF43B7179396A563292DB2966E79672BF8B7D0D0A3398A2E374C0E4A1C91DFBE5D31FF -20180605212615 2 6 100 5121 5 37585C4D659694D0B23500E18769F1EF93A776FF9FB92171358D9FE60A517E2CB91A4984A7E923B213CA5ED889D7CBD176B7B475315CB90EBD9920E41043A6C0BD37A8E2679AA7944ED273FDB1CBCAED3B2950BC73132CDE38789421CDDFAEC1C7662F3F9C3EA0C56758D7D3D0257499CE832FEF537F1BC17FFCB2360465A663AB629EF222B7C1E2ACF6368963B6A3D89E5617765B01D483C3BEB0F40D2BC4F4E0512CE41796C411A1EF20D70ED4F071EC56A24C52161D82B9222A52D56B0E48E118DDD1A5C01042E401B6724C80E939B3140A471CEA8C28E33883903283153A3EED07A506818930D61B420B0FF8D1B5F33A2E112B363D2B85A9FDE12AAA3DAE0E08945F4CE4E038A1B61624DD76CADFE172B0FE3C332F7B014634E7F4F80268EB6F1261F0667BF1FC4D9F0BA87C4CC42AC1261F7DA7C4508E3A70485F3D41B5B7AFA8DF0CA0CE61AF7A696B769FF69AB808AED3AE33F40E5E6DA7F8849349FB4A262965160C155B86AA0E13A8FB5EF1BD477790FD8949970B71DD508C971A88781DADEF4F67C1BF8E6BE1AC75B9E6CD74C471FB2D94B27269A9E30B671A52ABE673DAD632C1CC9C139468B852C06844D0D05A40D3A50D410DC4B8DD5B118626128CC5A2DA0093BE486F5D30E621DDB217B03AB97097733BD6E8D1CCFC7515D8761CF7EB0525483678785E29093865A60DA525D0A1D282501C27B1F6C7FEC64BED44F5409889BE78E049E22C5095E7EF4C9B7E88A49DF5CDBE6016A13072A28500AC00E64DB3CE3EBFC668CA0CB94CDDE8381F342E5CE4E2B1FC3067A295771DD0A8FFF308D5A47A91F1A67D8F77C3D20B88EAF17915867EE181DBC2DE8DF43B7179396A563292DB2966E79672BF8B7D0D0A3398A2E374C0E4A1C91DFBF726B37 -20180605213758 2 6 100 5121 5 37585C4D659694D0B23500E18769F1EF93A776FF9FB92171358D9FE60A517E2CB91A4984A7E923B213CA5ED889D7CBD176B7B475315CB90EBD9920E41043A6C0BD37A8E2679AA7944ED273FDB1CBCAED3B2950BC73132CDE38789421CDDFAEC1C7662F3F9C3EA0C56758D7D3D0257499CE832FEF537F1BC17FFCB2360465A663AB629EF222B7C1E2ACF6368963B6A3D89E5617765B01D483C3BEB0F40D2BC4F4E0512CE41796C411A1EF20D70ED4F071EC56A24C52161D82B9222A52D56B0E48E118DDD1A5C01042E401B6724C80E939B3140A471CEA8C28E33883903283153A3EED07A506818930D61B420B0FF8D1B5F33A2E112B363D2B85A9FDE12AAA3DAE0E08945F4CE4E038A1B61624DD76CADFE172B0FE3C332F7B014634E7F4F80268EB6F1261F0667BF1FC4D9F0BA87C4CC42AC1261F7DA7C4508E3A70485F3D41B5B7AFA8DF0CA0CE61AF7A696B769FF69AB808AED3AE33F40E5E6DA7F8849349FB4A262965160C155B86AA0E13A8FB5EF1BD477790FD8949970B71DD508C971A88781DADEF4F67C1BF8E6BE1AC75B9E6CD74C471FB2D94B27269A9E30B671A52ABE673DAD632C1CC9C139468B852C06844D0D05A40D3A50D410DC4B8DD5B118626128CC5A2DA0093BE486F5D30E621DDB217B03AB97097733BD6E8D1CCFC7515D8761CF7EB0525483678785E29093865A60DA525D0A1D282501C27B1F6C7FEC64BED44F5409889BE78E049E22C5095E7EF4C9B7E88A49DF5CDBE6016A13072A28500AC00E64DB3CE3EBFC668CA0CB94CDDE8381F342E5CE4E2B1FC3067A295771DD0A8FFF308D5A47A91F1A67D8F77C3D20B88EAF17915867EE181DBC2DE8DF43B7179396A563292DB2966E79672BF8B7D0D0A3398A2E374C0E4A1C91DFC05E3C97 -20180605214634 2 6 100 5121 5 37585C4D659694D0B23500E18769F1EF93A776FF9FB92171358D9FE60A517E2CB91A4984A7E923B213CA5ED889D7CBD176B7B475315CB90EBD9920E41043A6C0BD37A8E2679AA7944ED273FDB1CBCAED3B2950BC73132CDE38789421CDDFAEC1C7662F3F9C3EA0C56758D7D3D0257499CE832FEF537F1BC17FFCB2360465A663AB629EF222B7C1E2ACF6368963B6A3D89E5617765B01D483C3BEB0F40D2BC4F4E0512CE41796C411A1EF20D70ED4F071EC56A24C52161D82B9222A52D56B0E48E118DDD1A5C01042E401B6724C80E939B3140A471CEA8C28E33883903283153A3EED07A506818930D61B420B0FF8D1B5F33A2E112B363D2B85A9FDE12AAA3DAE0E08945F4CE4E038A1B61624DD76CADFE172B0FE3C332F7B014634E7F4F80268EB6F1261F0667BF1FC4D9F0BA87C4CC42AC1261F7DA7C4508E3A70485F3D41B5B7AFA8DF0CA0CE61AF7A696B769FF69AB808AED3AE33F40E5E6DA7F8849349FB4A262965160C155B86AA0E13A8FB5EF1BD477790FD8949970B71DD508C971A88781DADEF4F67C1BF8E6BE1AC75B9E6CD74C471FB2D94B27269A9E30B671A52ABE673DAD632C1CC9C139468B852C06844D0D05A40D3A50D410DC4B8DD5B118626128CC5A2DA0093BE486F5D30E621DDB217B03AB97097733BD6E8D1CCFC7515D8761CF7EB0525483678785E29093865A60DA525D0A1D282501C27B1F6C7FEC64BED44F5409889BE78E049E22C5095E7EF4C9B7E88A49DF5CDBE6016A13072A28500AC00E64DB3CE3EBFC668CA0CB94CDDE8381F342E5CE4E2B1FC3067A295771DD0A8FFF308D5A47A91F1A67D8F77C3D20B88EAF17915867EE181DBC2DE8DF43B7179396A563292DB2966E79672BF8B7D0D0A3398A2E374C0E4A1C91DFC108344F -20180605221135 2 6 100 5121 5 37585C4D659694D0B23500E18769F1EF93A776FF9FB92171358D9FE60A517E2CB91A4984A7E923B213CA5ED889D7CBD176B7B475315CB90EBD9920E41043A6C0BD37A8E2679AA7944ED273FDB1CBCAED3B2950BC73132CDE38789421CDDFAEC1C7662F3F9C3EA0C56758D7D3D0257499CE832FEF537F1BC17FFCB2360465A663AB629EF222B7C1E2ACF6368963B6A3D89E5617765B01D483C3BEB0F40D2BC4F4E0512CE41796C411A1EF20D70ED4F071EC56A24C52161D82B9222A52D56B0E48E118DDD1A5C01042E401B6724C80E939B3140A471CEA8C28E33883903283153A3EED07A506818930D61B420B0FF8D1B5F33A2E112B363D2B85A9FDE12AAA3DAE0E08945F4CE4E038A1B61624DD76CADFE172B0FE3C332F7B014634E7F4F80268EB6F1261F0667BF1FC4D9F0BA87C4CC42AC1261F7DA7C4508E3A70485F3D41B5B7AFA8DF0CA0CE61AF7A696B769FF69AB808AED3AE33F40E5E6DA7F8849349FB4A262965160C155B86AA0E13A8FB5EF1BD477790FD8949970B71DD508C971A88781DADEF4F67C1BF8E6BE1AC75B9E6CD74C471FB2D94B27269A9E30B671A52ABE673DAD632C1CC9C139468B852C06844D0D05A40D3A50D410DC4B8DD5B118626128CC5A2DA0093BE486F5D30E621DDB217B03AB97097733BD6E8D1CCFC7515D8761CF7EB0525483678785E29093865A60DA525D0A1D282501C27B1F6C7FEC64BED44F5409889BE78E049E22C5095E7EF4C9B7E88A49DF5CDBE6016A13072A28500AC00E64DB3CE3EBFC668CA0CB94CDDE8381F342E5CE4E2B1FC3067A295771DD0A8FFF308D5A47A91F1A67D8F77C3D20B88EAF17915867EE181DBC2DE8DF43B7179396A563292DB2966E79672BF8B7D0D0A3398A2E374C0E4A1C91DFC2F6C7EF -20180605221612 2 6 100 5121 2 37585C4D659694D0B23500E18769F1EF93A776FF9FB92171358D9FE60A517E2CB91A4984A7E923B213CA5ED889D7CBD176B7B475315CB90EBD9920E41043A6C0BD37A8E2679AA7944ED273FDB1CBCAED3B2950BC73132CDE38789421CDDFAEC1C7662F3F9C3EA0C56758D7D3D0257499CE832FEF537F1BC17FFCB2360465A663AB629EF222B7C1E2ACF6368963B6A3D89E5617765B01D483C3BEB0F40D2BC4F4E0512CE41796C411A1EF20D70ED4F071EC56A24C52161D82B9222A52D56B0E48E118DDD1A5C01042E401B6724C80E939B3140A471CEA8C28E33883903283153A3EED07A506818930D61B420B0FF8D1B5F33A2E112B363D2B85A9FDE12AAA3DAE0E08945F4CE4E038A1B61624DD76CADFE172B0FE3C332F7B014634E7F4F80268EB6F1261F0667BF1FC4D9F0BA87C4CC42AC1261F7DA7C4508E3A70485F3D41B5B7AFA8DF0CA0CE61AF7A696B769FF69AB808AED3AE33F40E5E6DA7F8849349FB4A262965160C155B86AA0E13A8FB5EF1BD477790FD8949970B71DD508C971A88781DADEF4F67C1BF8E6BE1AC75B9E6CD74C471FB2D94B27269A9E30B671A52ABE673DAD632C1CC9C139468B852C06844D0D05A40D3A50D410DC4B8DD5B118626128CC5A2DA0093BE486F5D30E621DDB217B03AB97097733BD6E8D1CCFC7515D8761CF7EB0525483678785E29093865A60DA525D0A1D282501C27B1F6C7FEC64BED44F5409889BE78E049E22C5095E7EF4C9B7E88A49DF5CDBE6016A13072A28500AC00E64DB3CE3EBFC668CA0CB94CDDE8381F342E5CE4E2B1FC3067A295771DD0A8FFF308D5A47A91F1A67D8F77C3D20B88EAF17915867EE181DBC2DE8DF43B7179396A563292DB2966E79672BF8B7D0D0A3398A2E374C0E4A1C91DFC34A2373 -20180605222719 2 6 100 5121 5 37585C4D659694D0B23500E18769F1EF93A776FF9FB92171358D9FE60A517E2CB91A4984A7E923B213CA5ED889D7CBD176B7B475315CB90EBD9920E41043A6C0BD37A8E2679AA7944ED273FDB1CBCAED3B2950BC73132CDE38789421CDDFAEC1C7662F3F9C3EA0C56758D7D3D0257499CE832FEF537F1BC17FFCB2360465A663AB629EF222B7C1E2ACF6368963B6A3D89E5617765B01D483C3BEB0F40D2BC4F4E0512CE41796C411A1EF20D70ED4F071EC56A24C52161D82B9222A52D56B0E48E118DDD1A5C01042E401B6724C80E939B3140A471CEA8C28E33883903283153A3EED07A506818930D61B420B0FF8D1B5F33A2E112B363D2B85A9FDE12AAA3DAE0E08945F4CE4E038A1B61624DD76CADFE172B0FE3C332F7B014634E7F4F80268EB6F1261F0667BF1FC4D9F0BA87C4CC42AC1261F7DA7C4508E3A70485F3D41B5B7AFA8DF0CA0CE61AF7A696B769FF69AB808AED3AE33F40E5E6DA7F8849349FB4A262965160C155B86AA0E13A8FB5EF1BD477790FD8949970B71DD508C971A88781DADEF4F67C1BF8E6BE1AC75B9E6CD74C471FB2D94B27269A9E30B671A52ABE673DAD632C1CC9C139468B852C06844D0D05A40D3A50D410DC4B8DD5B118626128CC5A2DA0093BE486F5D30E621DDB217B03AB97097733BD6E8D1CCFC7515D8761CF7EB0525483678785E29093865A60DA525D0A1D282501C27B1F6C7FEC64BED44F5409889BE78E049E22C5095E7EF4C9B7E88A49DF5CDBE6016A13072A28500AC00E64DB3CE3EBFC668CA0CB94CDDE8381F342E5CE4E2B1FC3067A295771DD0A8FFF308D5A47A91F1A67D8F77C3D20B88EAF17915867EE181DBC2DE8DF43B7179396A563292DB2966E79672BF8B7D0D0A3398A2E374C0E4A1C91DFC426CF47 -20180605225054 2 6 100 5121 2 37585C4D659694D0B23500E18769F1EF93A776FF9FB92171358D9FE60A517E2CB91A4984A7E923B213CA5ED889D7CBD176B7B475315CB90EBD9920E41043A6C0BD37A8E2679AA7944ED273FDB1CBCAED3B2950BC73132CDE38789421CDDFAEC1C7662F3F9C3EA0C56758D7D3D0257499CE832FEF537F1BC17FFCB2360465A663AB629EF222B7C1E2ACF6368963B6A3D89E5617765B01D483C3BEB0F40D2BC4F4E0512CE41796C411A1EF20D70ED4F071EC56A24C52161D82B9222A52D56B0E48E118DDD1A5C01042E401B6724C80E939B3140A471CEA8C28E33883903283153A3EED07A506818930D61B420B0FF8D1B5F33A2E112B363D2B85A9FDE12AAA3DAE0E08945F4CE4E038A1B61624DD76CADFE172B0FE3C332F7B014634E7F4F80268EB6F1261F0667BF1FC4D9F0BA87C4CC42AC1261F7DA7C4508E3A70485F3D41B5B7AFA8DF0CA0CE61AF7A696B769FF69AB808AED3AE33F40E5E6DA7F8849349FB4A262965160C155B86AA0E13A8FB5EF1BD477790FD8949970B71DD508C971A88781DADEF4F67C1BF8E6BE1AC75B9E6CD74C471FB2D94B27269A9E30B671A52ABE673DAD632C1CC9C139468B852C06844D0D05A40D3A50D410DC4B8DD5B118626128CC5A2DA0093BE486F5D30E621DDB217B03AB97097733BD6E8D1CCFC7515D8761CF7EB0525483678785E29093865A60DA525D0A1D282501C27B1F6C7FEC64BED44F5409889BE78E049E22C5095E7EF4C9B7E88A49DF5CDBE6016A13072A28500AC00E64DB3CE3EBFC668CA0CB94CDDE8381F342E5CE4E2B1FC3067A295771DD0A8FFF308D5A47A91F1A67D8F77C3D20B88EAF17915867EE181DBC2DE8DF43B7179396A563292DB2966E79672BF8B7D0D0A3398A2E374C0E4A1C91DFC5FF2A63 -20180605230438 2 6 100 5121 5 37585C4D659694D0B23500E18769F1EF93A776FF9FB92171358D9FE60A517E2CB91A4984A7E923B213CA5ED889D7CBD176B7B475315CB90EBD9920E41043A6C0BD37A8E2679AA7944ED273FDB1CBCAED3B2950BC73132CDE38789421CDDFAEC1C7662F3F9C3EA0C56758D7D3D0257499CE832FEF537F1BC17FFCB2360465A663AB629EF222B7C1E2ACF6368963B6A3D89E5617765B01D483C3BEB0F40D2BC4F4E0512CE41796C411A1EF20D70ED4F071EC56A24C52161D82B9222A52D56B0E48E118DDD1A5C01042E401B6724C80E939B3140A471CEA8C28E33883903283153A3EED07A506818930D61B420B0FF8D1B5F33A2E112B363D2B85A9FDE12AAA3DAE0E08945F4CE4E038A1B61624DD76CADFE172B0FE3C332F7B014634E7F4F80268EB6F1261F0667BF1FC4D9F0BA87C4CC42AC1261F7DA7C4508E3A70485F3D41B5B7AFA8DF0CA0CE61AF7A696B769FF69AB808AED3AE33F40E5E6DA7F8849349FB4A262965160C155B86AA0E13A8FB5EF1BD477790FD8949970B71DD508C971A88781DADEF4F67C1BF8E6BE1AC75B9E6CD74C471FB2D94B27269A9E30B671A52ABE673DAD632C1CC9C139468B852C06844D0D05A40D3A50D410DC4B8DD5B118626128CC5A2DA0093BE486F5D30E621DDB217B03AB97097733BD6E8D1CCFC7515D8761CF7EB0525483678785E29093865A60DA525D0A1D282501C27B1F6C7FEC64BED44F5409889BE78E049E22C5095E7EF4C9B7E88A49DF5CDBE6016A13072A28500AC00E64DB3CE3EBFC668CA0CB94CDDE8381F342E5CE4E2B1FC3067A295771DD0A8FFF308D5A47A91F1A67D8F77C3D20B88EAF17915867EE181DBC2DE8DF43B7179396A563292DB2966E79672BF8B7D0D0A3398A2E374C0E4A1C91DFC70EA13F -20180605232516 2 6 100 5121 5 37585C4D659694D0B23500E18769F1EF93A776FF9FB92171358D9FE60A517E2CB91A4984A7E923B213CA5ED889D7CBD176B7B475315CB90EBD9920E41043A6C0BD37A8E2679AA7944ED273FDB1CBCAED3B2950BC73132CDE38789421CDDFAEC1C7662F3F9C3EA0C56758D7D3D0257499CE832FEF537F1BC17FFCB2360465A663AB629EF222B7C1E2ACF6368963B6A3D89E5617765B01D483C3BEB0F40D2BC4F4E0512CE41796C411A1EF20D70ED4F071EC56A24C52161D82B9222A52D56B0E48E118DDD1A5C01042E401B6724C80E939B3140A471CEA8C28E33883903283153A3EED07A506818930D61B420B0FF8D1B5F33A2E112B363D2B85A9FDE12AAA3DAE0E08945F4CE4E038A1B61624DD76CADFE172B0FE3C332F7B014634E7F4F80268EB6F1261F0667BF1FC4D9F0BA87C4CC42AC1261F7DA7C4508E3A70485F3D41B5B7AFA8DF0CA0CE61AF7A696B769FF69AB808AED3AE33F40E5E6DA7F8849349FB4A262965160C155B86AA0E13A8FB5EF1BD477790FD8949970B71DD508C971A88781DADEF4F67C1BF8E6BE1AC75B9E6CD74C471FB2D94B27269A9E30B671A52ABE673DAD632C1CC9C139468B852C06844D0D05A40D3A50D410DC4B8DD5B118626128CC5A2DA0093BE486F5D30E621DDB217B03AB97097733BD6E8D1CCFC7515D8761CF7EB0525483678785E29093865A60DA525D0A1D282501C27B1F6C7FEC64BED44F5409889BE78E049E22C5095E7EF4C9B7E88A49DF5CDBE6016A13072A28500AC00E64DB3CE3EBFC668CA0CB94CDDE8381F342E5CE4E2B1FC3067A295771DD0A8FFF308D5A47A91F1A67D8F77C3D20B88EAF17915867EE181DBC2DE8DF43B7179396A563292DB2966E79672BF8B7D0D0A3398A2E374C0E4A1C91DFC8A507BF -20180606000440 2 6 100 5121 2 37585C4D659694D0B23500E18769F1EF93A776FF9FB92171358D9FE60A517E2CB91A4984A7E923B213CA5ED889D7CBD176B7B475315CB90EBD9920E41043A6C0BD37A8E2679AA7944ED273FDB1CBCAED3B2950BC73132CDE38789421CDDFAEC1C7662F3F9C3EA0C56758D7D3D0257499CE832FEF537F1BC17FFCB2360465A663AB629EF222B7C1E2ACF6368963B6A3D89E5617765B01D483C3BEB0F40D2BC4F4E0512CE41796C411A1EF20D70ED4F071EC56A24C52161D82B9222A52D56B0E48E118DDD1A5C01042E401B6724C80E939B3140A471CEA8C28E33883903283153A3EED07A506818930D61B420B0FF8D1B5F33A2E112B363D2B85A9FDE12AAA3DAE0E08945F4CE4E038A1B61624DD76CADFE172B0FE3C332F7B014634E7F4F80268EB6F1261F0667BF1FC4D9F0BA87C4CC42AC1261F7DA7C4508E3A70485F3D41B5B7AFA8DF0CA0CE61AF7A696B769FF69AB808AED3AE33F40E5E6DA7F8849349FB4A262965160C155B86AA0E13A8FB5EF1BD477790FD8949970B71DD508C971A88781DADEF4F67C1BF8E6BE1AC75B9E6CD74C471FB2D94B27269A9E30B671A52ABE673DAD632C1CC9C139468B852C06844D0D05A40D3A50D410DC4B8DD5B118626128CC5A2DA0093BE486F5D30E621DDB217B03AB97097733BD6E8D1CCFC7515D8761CF7EB0525483678785E29093865A60DA525D0A1D282501C27B1F6C7FEC64BED44F5409889BE78E049E22C5095E7EF4C9B7E88A49DF5CDBE6016A13072A28500AC00E64DB3CE3EBFC668CA0CB94CDDE8381F342E5CE4E2B1FC3067A295771DD0A8FFF308D5A47A91F1A67D8F77C3D20B88EAF17915867EE181DBC2DE8DF43B7179396A563292DB2966E79672BF8B7D0D0A3398A2E374C0E4A1C91DFCBB897D3 -20180606003911 2 6 100 5121 5 37585C4D659694D0B23500E18769F1EF93A776FF9FB92171358D9FE60A517E2CB91A4984A7E923B213CA5ED889D7CBD176B7B475315CB90EBD9920E41043A6C0BD37A8E2679AA7944ED273FDB1CBCAED3B2950BC73132CDE38789421CDDFAEC1C7662F3F9C3EA0C56758D7D3D0257499CE832FEF537F1BC17FFCB2360465A663AB629EF222B7C1E2ACF6368963B6A3D89E5617765B01D483C3BEB0F40D2BC4F4E0512CE41796C411A1EF20D70ED4F071EC56A24C52161D82B9222A52D56B0E48E118DDD1A5C01042E401B6724C80E939B3140A471CEA8C28E33883903283153A3EED07A506818930D61B420B0FF8D1B5F33A2E112B363D2B85A9FDE12AAA3DAE0E08945F4CE4E038A1B61624DD76CADFE172B0FE3C332F7B014634E7F4F80268EB6F1261F0667BF1FC4D9F0BA87C4CC42AC1261F7DA7C4508E3A70485F3D41B5B7AFA8DF0CA0CE61AF7A696B769FF69AB808AED3AE33F40E5E6DA7F8849349FB4A262965160C155B86AA0E13A8FB5EF1BD477790FD8949970B71DD508C971A88781DADEF4F67C1BF8E6BE1AC75B9E6CD74C471FB2D94B27269A9E30B671A52ABE673DAD632C1CC9C139468B852C06844D0D05A40D3A50D410DC4B8DD5B118626128CC5A2DA0093BE486F5D30E621DDB217B03AB97097733BD6E8D1CCFC7515D8761CF7EB0525483678785E29093865A60DA525D0A1D282501C27B1F6C7FEC64BED44F5409889BE78E049E22C5095E7EF4C9B7E88A49DF5CDBE6016A13072A28500AC00E64DB3CE3EBFC668CA0CB94CDDE8381F342E5CE4E2B1FC3067A295771DD0A8FFF308D5A47A91F1A67D8F77C3D20B88EAF17915867EE181DBC2DE8DF43B7179396A563292DB2966E79672BF8B7D0D0A3398A2E374C0E4A1C91DFCE5E6DDF -20180606004021 2 6 100 5121 2 37585C4D659694D0B23500E18769F1EF93A776FF9FB92171358D9FE60A517E2CB91A4984A7E923B213CA5ED889D7CBD176B7B475315CB90EBD9920E41043A6C0BD37A8E2679AA7944ED273FDB1CBCAED3B2950BC73132CDE38789421CDDFAEC1C7662F3F9C3EA0C56758D7D3D0257499CE832FEF537F1BC17FFCB2360465A663AB629EF222B7C1E2ACF6368963B6A3D89E5617765B01D483C3BEB0F40D2BC4F4E0512CE41796C411A1EF20D70ED4F071EC56A24C52161D82B9222A52D56B0E48E118DDD1A5C01042E401B6724C80E939B3140A471CEA8C28E33883903283153A3EED07A506818930D61B420B0FF8D1B5F33A2E112B363D2B85A9FDE12AAA3DAE0E08945F4CE4E038A1B61624DD76CADFE172B0FE3C332F7B014634E7F4F80268EB6F1261F0667BF1FC4D9F0BA87C4CC42AC1261F7DA7C4508E3A70485F3D41B5B7AFA8DF0CA0CE61AF7A696B769FF69AB808AED3AE33F40E5E6DA7F8849349FB4A262965160C155B86AA0E13A8FB5EF1BD477790FD8949970B71DD508C971A88781DADEF4F67C1BF8E6BE1AC75B9E6CD74C471FB2D94B27269A9E30B671A52ABE673DAD632C1CC9C139468B852C06844D0D05A40D3A50D410DC4B8DD5B118626128CC5A2DA0093BE486F5D30E621DDB217B03AB97097733BD6E8D1CCFC7515D8761CF7EB0525483678785E29093865A60DA525D0A1D282501C27B1F6C7FEC64BED44F5409889BE78E049E22C5095E7EF4C9B7E88A49DF5CDBE6016A13072A28500AC00E64DB3CE3EBFC668CA0CB94CDDE8381F342E5CE4E2B1FC3067A295771DD0A8FFF308D5A47A91F1A67D8F77C3D20B88EAF17915867EE181DBC2DE8DF43B7179396A563292DB2966E79672BF8B7D0D0A3398A2E374C0E4A1C91DFCE6D003B -20180606004054 2 6 100 5121 2 37585C4D659694D0B23500E18769F1EF93A776FF9FB92171358D9FE60A517E2CB91A4984A7E923B213CA5ED889D7CBD176B7B475315CB90EBD9920E41043A6C0BD37A8E2679AA7944ED273FDB1CBCAED3B2950BC73132CDE38789421CDDFAEC1C7662F3F9C3EA0C56758D7D3D0257499CE832FEF537F1BC17FFCB2360465A663AB629EF222B7C1E2ACF6368963B6A3D89E5617765B01D483C3BEB0F40D2BC4F4E0512CE41796C411A1EF20D70ED4F071EC56A24C52161D82B9222A52D56B0E48E118DDD1A5C01042E401B6724C80E939B3140A471CEA8C28E33883903283153A3EED07A506818930D61B420B0FF8D1B5F33A2E112B363D2B85A9FDE12AAA3DAE0E08945F4CE4E038A1B61624DD76CADFE172B0FE3C332F7B014634E7F4F80268EB6F1261F0667BF1FC4D9F0BA87C4CC42AC1261F7DA7C4508E3A70485F3D41B5B7AFA8DF0CA0CE61AF7A696B769FF69AB808AED3AE33F40E5E6DA7F8849349FB4A262965160C155B86AA0E13A8FB5EF1BD477790FD8949970B71DD508C971A88781DADEF4F67C1BF8E6BE1AC75B9E6CD74C471FB2D94B27269A9E30B671A52ABE673DAD632C1CC9C139468B852C06844D0D05A40D3A50D410DC4B8DD5B118626128CC5A2DA0093BE486F5D30E621DDB217B03AB97097733BD6E8D1CCFC7515D8761CF7EB0525483678785E29093865A60DA525D0A1D282501C27B1F6C7FEC64BED44F5409889BE78E049E22C5095E7EF4C9B7E88A49DF5CDBE6016A13072A28500AC00E64DB3CE3EBFC668CA0CB94CDDE8381F342E5CE4E2B1FC3067A295771DD0A8FFF308D5A47A91F1A67D8F77C3D20B88EAF17915867EE181DBC2DE8DF43B7179396A563292DB2966E79672BF8B7D0D0A3398A2E374C0E4A1C91DFCE6EF7C3 -20180606011249 2 6 100 5121 2 37585C4D659694D0B23500E18769F1EF93A776FF9FB92171358D9FE60A517E2CB91A4984A7E923B213CA5ED889D7CBD176B7B475315CB90EBD9920E41043A6C0BD37A8E2679AA7944ED273FDB1CBCAED3B2950BC73132CDE38789421CDDFAEC1C7662F3F9C3EA0C56758D7D3D0257499CE832FEF537F1BC17FFCB2360465A663AB629EF222B7C1E2ACF6368963B6A3D89E5617765B01D483C3BEB0F40D2BC4F4E0512CE41796C411A1EF20D70ED4F071EC56A24C52161D82B9222A52D56B0E48E118DDD1A5C01042E401B6724C80E939B3140A471CEA8C28E33883903283153A3EED07A506818930D61B420B0FF8D1B5F33A2E112B363D2B85A9FDE12AAA3DAE0E08945F4CE4E038A1B61624DD76CADFE172B0FE3C332F7B014634E7F4F80268EB6F1261F0667BF1FC4D9F0BA87C4CC42AC1261F7DA7C4508E3A70485F3D41B5B7AFA8DF0CA0CE61AF7A696B769FF69AB808AED3AE33F40E5E6DA7F8849349FB4A262965160C155B86AA0E13A8FB5EF1BD477790FD8949970B71DD508C971A88781DADEF4F67C1BF8E6BE1AC75B9E6CD74C471FB2D94B27269A9E30B671A52ABE673DAD632C1CC9C139468B852C06844D0D05A40D3A50D410DC4B8DD5B118626128CC5A2DA0093BE486F5D30E621DDB217B03AB97097733BD6E8D1CCFC7515D8761CF7EB0525483678785E29093865A60DA525D0A1D282501C27B1F6C7FEC64BED44F5409889BE78E049E22C5095E7EF4C9B7E88A49DF5CDBE6016A13072A28500AC00E64DB3CE3EBFC668CA0CB94CDDE8381F342E5CE4E2B1FC3067A295771DD0A8FFF308D5A47A91F1A67D8F77C3D20B88EAF17915867EE181DBC2DE8DF43B7179396A563292DB2966E79672BF8B7D0D0A3398A2E374C0E4A1C91DFD0E6DF43 -20180606013848 2 6 100 5121 2 37585C4D659694D0B23500E18769F1EF93A776FF9FB92171358D9FE60A517E2CB91A4984A7E923B213CA5ED889D7CBD176B7B475315CB90EBD9920E41043A6C0BD37A8E2679AA7944ED273FDB1CBCAED3B2950BC73132CDE38789421CDDFAEC1C7662F3F9C3EA0C56758D7D3D0257499CE832FEF537F1BC17FFCB2360465A663AB629EF222B7C1E2ACF6368963B6A3D89E5617765B01D483C3BEB0F40D2BC4F4E0512CE41796C411A1EF20D70ED4F071EC56A24C52161D82B9222A52D56B0E48E118DDD1A5C01042E401B6724C80E939B3140A471CEA8C28E33883903283153A3EED07A506818930D61B420B0FF8D1B5F33A2E112B363D2B85A9FDE12AAA3DAE0E08945F4CE4E038A1B61624DD76CADFE172B0FE3C332F7B014634E7F4F80268EB6F1261F0667BF1FC4D9F0BA87C4CC42AC1261F7DA7C4508E3A70485F3D41B5B7AFA8DF0CA0CE61AF7A696B769FF69AB808AED3AE33F40E5E6DA7F8849349FB4A262965160C155B86AA0E13A8FB5EF1BD477790FD8949970B71DD508C971A88781DADEF4F67C1BF8E6BE1AC75B9E6CD74C471FB2D94B27269A9E30B671A52ABE673DAD632C1CC9C139468B852C06844D0D05A40D3A50D410DC4B8DD5B118626128CC5A2DA0093BE486F5D30E621DDB217B03AB97097733BD6E8D1CCFC7515D8761CF7EB0525483678785E29093865A60DA525D0A1D282501C27B1F6C7FEC64BED44F5409889BE78E049E22C5095E7EF4C9B7E88A49DF5CDBE6016A13072A28500AC00E64DB3CE3EBFC668CA0CB94CDDE8381F342E5CE4E2B1FC3067A295771DD0A8FFF308D5A47A91F1A67D8F77C3D20B88EAF17915867EE181DBC2DE8DF43B7179396A563292DB2966E79672BF8B7D0D0A3398A2E374C0E4A1C91DFD2E87873 -20180606020718 2 6 100 5121 5 37585C4D659694D0B23500E18769F1EF93A776FF9FB92171358D9FE60A517E2CB91A4984A7E923B213CA5ED889D7CBD176B7B475315CB90EBD9920E41043A6C0BD37A8E2679AA7944ED273FDB1CBCAED3B2950BC73132CDE38789421CDDFAEC1C7662F3F9C3EA0C56758D7D3D0257499CE832FEF537F1BC17FFCB2360465A663AB629EF222B7C1E2ACF6368963B6A3D89E5617765B01D483C3BEB0F40D2BC4F4E0512CE41796C411A1EF20D70ED4F071EC56A24C52161D82B9222A52D56B0E48E118DDD1A5C01042E401B6724C80E939B3140A471CEA8C28E33883903283153A3EED07A506818930D61B420B0FF8D1B5F33A2E112B363D2B85A9FDE12AAA3DAE0E08945F4CE4E038A1B61624DD76CADFE172B0FE3C332F7B014634E7F4F80268EB6F1261F0667BF1FC4D9F0BA87C4CC42AC1261F7DA7C4508E3A70485F3D41B5B7AFA8DF0CA0CE61AF7A696B769FF69AB808AED3AE33F40E5E6DA7F8849349FB4A262965160C155B86AA0E13A8FB5EF1BD477790FD8949970B71DD508C971A88781DADEF4F67C1BF8E6BE1AC75B9E6CD74C471FB2D94B27269A9E30B671A52ABE673DAD632C1CC9C139468B852C06844D0D05A40D3A50D410DC4B8DD5B118626128CC5A2DA0093BE486F5D30E621DDB217B03AB97097733BD6E8D1CCFC7515D8761CF7EB0525483678785E29093865A60DA525D0A1D282501C27B1F6C7FEC64BED44F5409889BE78E049E22C5095E7EF4C9B7E88A49DF5CDBE6016A13072A28500AC00E64DB3CE3EBFC668CA0CB94CDDE8381F342E5CE4E2B1FC3067A295771DD0A8FFF308D5A47A91F1A67D8F77C3D20B88EAF17915867EE181DBC2DE8DF43B7179396A563292DB2966E79672BF8B7D0D0A3398A2E374C0E4A1C91DFD5166B47 -20180606021121 2 6 100 5121 2 37585C4D659694D0B23500E18769F1EF93A776FF9FB92171358D9FE60A517E2CB91A4984A7E923B213CA5ED889D7CBD176B7B475315CB90EBD9920E41043A6C0BD37A8E2679AA7944ED273FDB1CBCAED3B2950BC73132CDE38789421CDDFAEC1C7662F3F9C3EA0C56758D7D3D0257499CE832FEF537F1BC17FFCB2360465A663AB629EF222B7C1E2ACF6368963B6A3D89E5617765B01D483C3BEB0F40D2BC4F4E0512CE41796C411A1EF20D70ED4F071EC56A24C52161D82B9222A52D56B0E48E118DDD1A5C01042E401B6724C80E939B3140A471CEA8C28E33883903283153A3EED07A506818930D61B420B0FF8D1B5F33A2E112B363D2B85A9FDE12AAA3DAE0E08945F4CE4E038A1B61624DD76CADFE172B0FE3C332F7B014634E7F4F80268EB6F1261F0667BF1FC4D9F0BA87C4CC42AC1261F7DA7C4508E3A70485F3D41B5B7AFA8DF0CA0CE61AF7A696B769FF69AB808AED3AE33F40E5E6DA7F8849349FB4A262965160C155B86AA0E13A8FB5EF1BD477790FD8949970B71DD508C971A88781DADEF4F67C1BF8E6BE1AC75B9E6CD74C471FB2D94B27269A9E30B671A52ABE673DAD632C1CC9C139468B852C06844D0D05A40D3A50D410DC4B8DD5B118626128CC5A2DA0093BE486F5D30E621DDB217B03AB97097733BD6E8D1CCFC7515D8761CF7EB0525483678785E29093865A60DA525D0A1D282501C27B1F6C7FEC64BED44F5409889BE78E049E22C5095E7EF4C9B7E88A49DF5CDBE6016A13072A28500AC00E64DB3CE3EBFC668CA0CB94CDDE8381F342E5CE4E2B1FC3067A295771DD0A8FFF308D5A47A91F1A67D8F77C3D20B88EAF17915867EE181DBC2DE8DF43B7179396A563292DB2966E79672BF8B7D0D0A3398A2E374C0E4A1C91DFD55F3E2B -20180606030823 2 6 100 5121 2 37585C4D659694D0B23500E18769F1EF93A776FF9FB92171358D9FE60A517E2CB91A4984A7E923B213CA5ED889D7CBD176B7B475315CB90EBD9920E41043A6C0BD37A8E2679AA7944ED273FDB1CBCAED3B2950BC73132CDE38789421CDDFAEC1C7662F3F9C3EA0C56758D7D3D0257499CE832FEF537F1BC17FFCB2360465A663AB629EF222B7C1E2ACF6368963B6A3D89E5617765B01D483C3BEB0F40D2BC4F4E0512CE41796C411A1EF20D70ED4F071EC56A24C52161D82B9222A52D56B0E48E118DDD1A5C01042E401B6724C80E939B3140A471CEA8C28E33883903283153A3EED07A506818930D61B420B0FF8D1B5F33A2E112B363D2B85A9FDE12AAA3DAE0E08945F4CE4E038A1B61624DD76CADFE172B0FE3C332F7B014634E7F4F80268EB6F1261F0667BF1FC4D9F0BA87C4CC42AC1261F7DA7C4508E3A70485F3D41B5B7AFA8DF0CA0CE61AF7A696B769FF69AB808AED3AE33F40E5E6DA7F8849349FB4A262965160C155B86AA0E13A8FB5EF1BD477790FD8949970B71DD508C971A88781DADEF4F67C1BF8E6BE1AC75B9E6CD74C471FB2D94B27269A9E30B671A52ABE673DAD632C1CC9C139468B852C06844D0D05A40D3A50D410DC4B8DD5B118626128CC5A2DA0093BE486F5D30E621DDB217B03AB97097733BD6E8D1CCFC7515D8761CF7EB0525483678785E29093865A60DA525D0A1D282501C27B1F6C7FEC64BED44F5409889BE78E049E22C5095E7EF4C9B7E88A49DF5CDBE6016A13072A28500AC00E64DB3CE3EBFC668CA0CB94CDDE8381F342E5CE4E2B1FC3067A295771DD0A8FFF308D5A47A91F1A67D8F77C3D20B88EAF17915867EE181DBC2DE8DF43B7179396A563292DB2966E79672BF8B7D0D0A3398A2E374C0E4A1C91DFD9C61CC3 -20180606033204 2 6 100 5121 5 37585C4D659694D0B23500E18769F1EF93A776FF9FB92171358D9FE60A517E2CB91A4984A7E923B213CA5ED889D7CBD176B7B475315CB90EBD9920E41043A6C0BD37A8E2679AA7944ED273FDB1CBCAED3B2950BC73132CDE38789421CDDFAEC1C7662F3F9C3EA0C56758D7D3D0257499CE832FEF537F1BC17FFCB2360465A663AB629EF222B7C1E2ACF6368963B6A3D89E5617765B01D483C3BEB0F40D2BC4F4E0512CE41796C411A1EF20D70ED4F071EC56A24C52161D82B9222A52D56B0E48E118DDD1A5C01042E401B6724C80E939B3140A471CEA8C28E33883903283153A3EED07A506818930D61B420B0FF8D1B5F33A2E112B363D2B85A9FDE12AAA3DAE0E08945F4CE4E038A1B61624DD76CADFE172B0FE3C332F7B014634E7F4F80268EB6F1261F0667BF1FC4D9F0BA87C4CC42AC1261F7DA7C4508E3A70485F3D41B5B7AFA8DF0CA0CE61AF7A696B769FF69AB808AED3AE33F40E5E6DA7F8849349FB4A262965160C155B86AA0E13A8FB5EF1BD477790FD8949970B71DD508C971A88781DADEF4F67C1BF8E6BE1AC75B9E6CD74C471FB2D94B27269A9E30B671A52ABE673DAD632C1CC9C139468B852C06844D0D05A40D3A50D410DC4B8DD5B118626128CC5A2DA0093BE486F5D30E621DDB217B03AB97097733BD6E8D1CCFC7515D8761CF7EB0525483678785E29093865A60DA525D0A1D282501C27B1F6C7FEC64BED44F5409889BE78E049E22C5095E7EF4C9B7E88A49DF5CDBE6016A13072A28500AC00E64DB3CE3EBFC668CA0CB94CDDE8381F342E5CE4E2B1FC3067A295771DD0A8FFF308D5A47A91F1A67D8F77C3D20B88EAF17915867EE181DBC2DE8DF43B7179396A563292DB2966E79672BF8B7D0D0A3398A2E374C0E4A1C91DFDB8F4F47 -20180606033802 2 6 100 5121 2 37585C4D659694D0B23500E18769F1EF93A776FF9FB92171358D9FE60A517E2CB91A4984A7E923B213CA5ED889D7CBD176B7B475315CB90EBD9920E41043A6C0BD37A8E2679AA7944ED273FDB1CBCAED3B2950BC73132CDE38789421CDDFAEC1C7662F3F9C3EA0C56758D7D3D0257499CE832FEF537F1BC17FFCB2360465A663AB629EF222B7C1E2ACF6368963B6A3D89E5617765B01D483C3BEB0F40D2BC4F4E0512CE41796C411A1EF20D70ED4F071EC56A24C52161D82B9222A52D56B0E48E118DDD1A5C01042E401B6724C80E939B3140A471CEA8C28E33883903283153A3EED07A506818930D61B420B0FF8D1B5F33A2E112B363D2B85A9FDE12AAA3DAE0E08945F4CE4E038A1B61624DD76CADFE172B0FE3C332F7B014634E7F4F80268EB6F1261F0667BF1FC4D9F0BA87C4CC42AC1261F7DA7C4508E3A70485F3D41B5B7AFA8DF0CA0CE61AF7A696B769FF69AB808AED3AE33F40E5E6DA7F8849349FB4A262965160C155B86AA0E13A8FB5EF1BD477790FD8949970B71DD508C971A88781DADEF4F67C1BF8E6BE1AC75B9E6CD74C471FB2D94B27269A9E30B671A52ABE673DAD632C1CC9C139468B852C06844D0D05A40D3A50D410DC4B8DD5B118626128CC5A2DA0093BE486F5D30E621DDB217B03AB97097733BD6E8D1CCFC7515D8761CF7EB0525483678785E29093865A60DA525D0A1D282501C27B1F6C7FEC64BED44F5409889BE78E049E22C5095E7EF4C9B7E88A49DF5CDBE6016A13072A28500AC00E64DB3CE3EBFC668CA0CB94CDDE8381F342E5CE4E2B1FC3067A295771DD0A8FFF308D5A47A91F1A67D8F77C3D20B88EAF17915867EE181DBC2DE8DF43B7179396A563292DB2966E79672BF8B7D0D0A3398A2E374C0E4A1C91DFDBF98B5B -20180606035103 2 6 100 5121 5 37585C4D659694D0B23500E18769F1EF93A776FF9FB92171358D9FE60A517E2CB91A4984A7E923B213CA5ED889D7CBD176B7B475315CB90EBD9920E41043A6C0BD37A8E2679AA7944ED273FDB1CBCAED3B2950BC73132CDE38789421CDDFAEC1C7662F3F9C3EA0C56758D7D3D0257499CE832FEF537F1BC17FFCB2360465A663AB629EF222B7C1E2ACF6368963B6A3D89E5617765B01D483C3BEB0F40D2BC4F4E0512CE41796C411A1EF20D70ED4F071EC56A24C52161D82B9222A52D56B0E48E118DDD1A5C01042E401B6724C80E939B3140A471CEA8C28E33883903283153A3EED07A506818930D61B420B0FF8D1B5F33A2E112B363D2B85A9FDE12AAA3DAE0E08945F4CE4E038A1B61624DD76CADFE172B0FE3C332F7B014634E7F4F80268EB6F1261F0667BF1FC4D9F0BA87C4CC42AC1261F7DA7C4508E3A70485F3D41B5B7AFA8DF0CA0CE61AF7A696B769FF69AB808AED3AE33F40E5E6DA7F8849349FB4A262965160C155B86AA0E13A8FB5EF1BD477790FD8949970B71DD508C971A88781DADEF4F67C1BF8E6BE1AC75B9E6CD74C471FB2D94B27269A9E30B671A52ABE673DAD632C1CC9C139468B852C06844D0D05A40D3A50D410DC4B8DD5B118626128CC5A2DA0093BE486F5D30E621DDB217B03AB97097733BD6E8D1CCFC7515D8761CF7EB0525483678785E29093865A60DA525D0A1D282501C27B1F6C7FEC64BED44F5409889BE78E049E22C5095E7EF4C9B7E88A49DF5CDBE6016A13072A28500AC00E64DB3CE3EBFC668CA0CB94CDDE8381F342E5CE4E2B1FC3067A295771DD0A8FFF308D5A47A91F1A67D8F77C3D20B88EAF17915867EE181DBC2DE8DF43B7179396A563292DB2966E79672BF8B7D0D0A3398A2E374C0E4A1C91DFDCF1AF47 -20180606041154 2 6 100 5121 5 37585C4D659694D0B23500E18769F1EF93A776FF9FB92171358D9FE60A517E2CB91A4984A7E923B213CA5ED889D7CBD176B7B475315CB90EBD9920E41043A6C0BD37A8E2679AA7944ED273FDB1CBCAED3B2950BC73132CDE38789421CDDFAEC1C7662F3F9C3EA0C56758D7D3D0257499CE832FEF537F1BC17FFCB2360465A663AB629EF222B7C1E2ACF6368963B6A3D89E5617765B01D483C3BEB0F40D2BC4F4E0512CE41796C411A1EF20D70ED4F071EC56A24C52161D82B9222A52D56B0E48E118DDD1A5C01042E401B6724C80E939B3140A471CEA8C28E33883903283153A3EED07A506818930D61B420B0FF8D1B5F33A2E112B363D2B85A9FDE12AAA3DAE0E08945F4CE4E038A1B61624DD76CADFE172B0FE3C332F7B014634E7F4F80268EB6F1261F0667BF1FC4D9F0BA87C4CC42AC1261F7DA7C4508E3A70485F3D41B5B7AFA8DF0CA0CE61AF7A696B769FF69AB808AED3AE33F40E5E6DA7F8849349FB4A262965160C155B86AA0E13A8FB5EF1BD477790FD8949970B71DD508C971A88781DADEF4F67C1BF8E6BE1AC75B9E6CD74C471FB2D94B27269A9E30B671A52ABE673DAD632C1CC9C139468B852C06844D0D05A40D3A50D410DC4B8DD5B118626128CC5A2DA0093BE486F5D30E621DDB217B03AB97097733BD6E8D1CCFC7515D8761CF7EB0525483678785E29093865A60DA525D0A1D282501C27B1F6C7FEC64BED44F5409889BE78E049E22C5095E7EF4C9B7E88A49DF5CDBE6016A13072A28500AC00E64DB3CE3EBFC668CA0CB94CDDE8381F342E5CE4E2B1FC3067A295771DD0A8FFF308D5A47A91F1A67D8F77C3D20B88EAF17915867EE181DBC2DE8DF43B7179396A563292DB2966E79672BF8B7D0D0A3398A2E374C0E4A1C91DFDE894FEF -20180606041520 2 6 100 5121 2 37585C4D659694D0B23500E18769F1EF93A776FF9FB92171358D9FE60A517E2CB91A4984A7E923B213CA5ED889D7CBD176B7B475315CB90EBD9920E41043A6C0BD37A8E2679AA7944ED273FDB1CBCAED3B2950BC73132CDE38789421CDDFAEC1C7662F3F9C3EA0C56758D7D3D0257499CE832FEF537F1BC17FFCB2360465A663AB629EF222B7C1E2ACF6368963B6A3D89E5617765B01D483C3BEB0F40D2BC4F4E0512CE41796C411A1EF20D70ED4F071EC56A24C52161D82B9222A52D56B0E48E118DDD1A5C01042E401B6724C80E939B3140A471CEA8C28E33883903283153A3EED07A506818930D61B420B0FF8D1B5F33A2E112B363D2B85A9FDE12AAA3DAE0E08945F4CE4E038A1B61624DD76CADFE172B0FE3C332F7B014634E7F4F80268EB6F1261F0667BF1FC4D9F0BA87C4CC42AC1261F7DA7C4508E3A70485F3D41B5B7AFA8DF0CA0CE61AF7A696B769FF69AB808AED3AE33F40E5E6DA7F8849349FB4A262965160C155B86AA0E13A8FB5EF1BD477790FD8949970B71DD508C971A88781DADEF4F67C1BF8E6BE1AC75B9E6CD74C471FB2D94B27269A9E30B671A52ABE673DAD632C1CC9C139468B852C06844D0D05A40D3A50D410DC4B8DD5B118626128CC5A2DA0093BE486F5D30E621DDB217B03AB97097733BD6E8D1CCFC7515D8761CF7EB0525483678785E29093865A60DA525D0A1D282501C27B1F6C7FEC64BED44F5409889BE78E049E22C5095E7EF4C9B7E88A49DF5CDBE6016A13072A28500AC00E64DB3CE3EBFC668CA0CB94CDDE8381F342E5CE4E2B1FC3067A295771DD0A8FFF308D5A47A91F1A67D8F77C3D20B88EAF17915867EE181DBC2DE8DF43B7179396A563292DB2966E79672BF8B7D0D0A3398A2E374C0E4A1C91DFDEC4EE7B -20180606043713 2 6 100 5121 2 37585C4D659694D0B23500E18769F1EF93A776FF9FB92171358D9FE60A517E2CB91A4984A7E923B213CA5ED889D7CBD176B7B475315CB90EBD9920E41043A6C0BD37A8E2679AA7944ED273FDB1CBCAED3B2950BC73132CDE38789421CDDFAEC1C7662F3F9C3EA0C56758D7D3D0257499CE832FEF537F1BC17FFCB2360465A663AB629EF222B7C1E2ACF6368963B6A3D89E5617765B01D483C3BEB0F40D2BC4F4E0512CE41796C411A1EF20D70ED4F071EC56A24C52161D82B9222A52D56B0E48E118DDD1A5C01042E401B6724C80E939B3140A471CEA8C28E33883903283153A3EED07A506818930D61B420B0FF8D1B5F33A2E112B363D2B85A9FDE12AAA3DAE0E08945F4CE4E038A1B61624DD76CADFE172B0FE3C332F7B014634E7F4F80268EB6F1261F0667BF1FC4D9F0BA87C4CC42AC1261F7DA7C4508E3A70485F3D41B5B7AFA8DF0CA0CE61AF7A696B769FF69AB808AED3AE33F40E5E6DA7F8849349FB4A262965160C155B86AA0E13A8FB5EF1BD477790FD8949970B71DD508C971A88781DADEF4F67C1BF8E6BE1AC75B9E6CD74C471FB2D94B27269A9E30B671A52ABE673DAD632C1CC9C139468B852C06844D0D05A40D3A50D410DC4B8DD5B118626128CC5A2DA0093BE486F5D30E621DDB217B03AB97097733BD6E8D1CCFC7515D8761CF7EB0525483678785E29093865A60DA525D0A1D282501C27B1F6C7FEC64BED44F5409889BE78E049E22C5095E7EF4C9B7E88A49DF5CDBE6016A13072A28500AC00E64DB3CE3EBFC668CA0CB94CDDE8381F342E5CE4E2B1FC3067A295771DD0A8FFF308D5A47A91F1A67D8F77C3D20B88EAF17915867EE181DBC2DE8DF43B7179396A563292DB2966E79672BF8B7D0D0A3398A2E374C0E4A1C91DFE06AD163 -20180606053417 2 6 100 5121 2 37585C4D659694D0B23500E18769F1EF93A776FF9FB92171358D9FE60A517E2CB91A4984A7E923B213CA5ED889D7CBD176B7B475315CB90EBD9920E41043A6C0BD37A8E2679AA7944ED273FDB1CBCAED3B2950BC73132CDE38789421CDDFAEC1C7662F3F9C3EA0C56758D7D3D0257499CE832FEF537F1BC17FFCB2360465A663AB629EF222B7C1E2ACF6368963B6A3D89E5617765B01D483C3BEB0F40D2BC4F4E0512CE41796C411A1EF20D70ED4F071EC56A24C52161D82B9222A52D56B0E48E118DDD1A5C01042E401B6724C80E939B3140A471CEA8C28E33883903283153A3EED07A506818930D61B420B0FF8D1B5F33A2E112B363D2B85A9FDE12AAA3DAE0E08945F4CE4E038A1B61624DD76CADFE172B0FE3C332F7B014634E7F4F80268EB6F1261F0667BF1FC4D9F0BA87C4CC42AC1261F7DA7C4508E3A70485F3D41B5B7AFA8DF0CA0CE61AF7A696B769FF69AB808AED3AE33F40E5E6DA7F8849349FB4A262965160C155B86AA0E13A8FB5EF1BD477790FD8949970B71DD508C971A88781DADEF4F67C1BF8E6BE1AC75B9E6CD74C471FB2D94B27269A9E30B671A52ABE673DAD632C1CC9C139468B852C06844D0D05A40D3A50D410DC4B8DD5B118626128CC5A2DA0093BE486F5D30E621DDB217B03AB97097733BD6E8D1CCFC7515D8761CF7EB0525483678785E29093865A60DA525D0A1D282501C27B1F6C7FEC64BED44F5409889BE78E049E22C5095E7EF4C9B7E88A49DF5CDBE6016A13072A28500AC00E64DB3CE3EBFC668CA0CB94CDDE8381F342E5CE4E2B1FC3067A295771DD0A8FFF308D5A47A91F1A67D8F77C3D20B88EAF17915867EE181DBC2DE8DF43B7179396A563292DB2966E79672BF8B7D0D0A3398A2E374C0E4A1C91DFE4C5AD13 -20180606060033 2 6 100 5121 5 37585C4D659694D0B23500E18769F1EF93A776FF9FB92171358D9FE60A517E2CB91A4984A7E923B213CA5ED889D7CBD176B7B475315CB90EBD9920E41043A6C0BD37A8E2679AA7944ED273FDB1CBCAED3B2950BC73132CDE38789421CDDFAEC1C7662F3F9C3EA0C56758D7D3D0257499CE832FEF537F1BC17FFCB2360465A663AB629EF222B7C1E2ACF6368963B6A3D89E5617765B01D483C3BEB0F40D2BC4F4E0512CE41796C411A1EF20D70ED4F071EC56A24C52161D82B9222A52D56B0E48E118DDD1A5C01042E401B6724C80E939B3140A471CEA8C28E33883903283153A3EED07A506818930D61B420B0FF8D1B5F33A2E112B363D2B85A9FDE12AAA3DAE0E08945F4CE4E038A1B61624DD76CADFE172B0FE3C332F7B014634E7F4F80268EB6F1261F0667BF1FC4D9F0BA87C4CC42AC1261F7DA7C4508E3A70485F3D41B5B7AFA8DF0CA0CE61AF7A696B769FF69AB808AED3AE33F40E5E6DA7F8849349FB4A262965160C155B86AA0E13A8FB5EF1BD477790FD8949970B71DD508C971A88781DADEF4F67C1BF8E6BE1AC75B9E6CD74C471FB2D94B27269A9E30B671A52ABE673DAD632C1CC9C139468B852C06844D0D05A40D3A50D410DC4B8DD5B118626128CC5A2DA0093BE486F5D30E621DDB217B03AB97097733BD6E8D1CCFC7515D8761CF7EB0525483678785E29093865A60DA525D0A1D282501C27B1F6C7FEC64BED44F5409889BE78E049E22C5095E7EF4C9B7E88A49DF5CDBE6016A13072A28500AC00E64DB3CE3EBFC668CA0CB94CDDE8381F342E5CE4E2B1FC3067A295771DD0A8FFF308D5A47A91F1A67D8F77C3D20B88EAF17915867EE181DBC2DE8DF43B7179396A563292DB2966E79672BF8B7D0D0A3398A2E374C0E4A1C91DFE6C71C97 -20180606061040 2 6 100 5121 2 37585C4D659694D0B23500E18769F1EF93A776FF9FB92171358D9FE60A517E2CB91A4984A7E923B213CA5ED889D7CBD176B7B475315CB90EBD9920E41043A6C0BD37A8E2679AA7944ED273FDB1CBCAED3B2950BC73132CDE38789421CDDFAEC1C7662F3F9C3EA0C56758D7D3D0257499CE832FEF537F1BC17FFCB2360465A663AB629EF222B7C1E2ACF6368963B6A3D89E5617765B01D483C3BEB0F40D2BC4F4E0512CE41796C411A1EF20D70ED4F071EC56A24C52161D82B9222A52D56B0E48E118DDD1A5C01042E401B6724C80E939B3140A471CEA8C28E33883903283153A3EED07A506818930D61B420B0FF8D1B5F33A2E112B363D2B85A9FDE12AAA3DAE0E08945F4CE4E038A1B61624DD76CADFE172B0FE3C332F7B014634E7F4F80268EB6F1261F0667BF1FC4D9F0BA87C4CC42AC1261F7DA7C4508E3A70485F3D41B5B7AFA8DF0CA0CE61AF7A696B769FF69AB808AED3AE33F40E5E6DA7F8849349FB4A262965160C155B86AA0E13A8FB5EF1BD477790FD8949970B71DD508C971A88781DADEF4F67C1BF8E6BE1AC75B9E6CD74C471FB2D94B27269A9E30B671A52ABE673DAD632C1CC9C139468B852C06844D0D05A40D3A50D410DC4B8DD5B118626128CC5A2DA0093BE486F5D30E621DDB217B03AB97097733BD6E8D1CCFC7515D8761CF7EB0525483678785E29093865A60DA525D0A1D282501C27B1F6C7FEC64BED44F5409889BE78E049E22C5095E7EF4C9B7E88A49DF5CDBE6016A13072A28500AC00E64DB3CE3EBFC668CA0CB94CDDE8381F342E5CE4E2B1FC3067A295771DD0A8FFF308D5A47A91F1A67D8F77C3D20B88EAF17915867EE181DBC2DE8DF43B7179396A563292DB2966E79672BF8B7D0D0A3398A2E374C0E4A1C91DFE77EB58B -20180606062720 2 6 100 5121 5 37585C4D659694D0B23500E18769F1EF93A776FF9FB92171358D9FE60A517E2CB91A4984A7E923B213CA5ED889D7CBD176B7B475315CB90EBD9920E41043A6C0BD37A8E2679AA7944ED273FDB1CBCAED3B2950BC73132CDE38789421CDDFAEC1C7662F3F9C3EA0C56758D7D3D0257499CE832FEF537F1BC17FFCB2360465A663AB629EF222B7C1E2ACF6368963B6A3D89E5617765B01D483C3BEB0F40D2BC4F4E0512CE41796C411A1EF20D70ED4F071EC56A24C52161D82B9222A52D56B0E48E118DDD1A5C01042E401B6724C80E939B3140A471CEA8C28E33883903283153A3EED07A506818930D61B420B0FF8D1B5F33A2E112B363D2B85A9FDE12AAA3DAE0E08945F4CE4E038A1B61624DD76CADFE172B0FE3C332F7B014634E7F4F80268EB6F1261F0667BF1FC4D9F0BA87C4CC42AC1261F7DA7C4508E3A70485F3D41B5B7AFA8DF0CA0CE61AF7A696B769FF69AB808AED3AE33F40E5E6DA7F8849349FB4A262965160C155B86AA0E13A8FB5EF1BD477790FD8949970B71DD508C971A88781DADEF4F67C1BF8E6BE1AC75B9E6CD74C471FB2D94B27269A9E30B671A52ABE673DAD632C1CC9C139468B852C06844D0D05A40D3A50D410DC4B8DD5B118626128CC5A2DA0093BE486F5D30E621DDB217B03AB97097733BD6E8D1CCFC7515D8761CF7EB0525483678785E29093865A60DA525D0A1D282501C27B1F6C7FEC64BED44F5409889BE78E049E22C5095E7EF4C9B7E88A49DF5CDBE6016A13072A28500AC00E64DB3CE3EBFC668CA0CB94CDDE8381F342E5CE4E2B1FC3067A295771DD0A8FFF308D5A47A91F1A67D8F77C3D20B88EAF17915867EE181DBC2DE8DF43B7179396A563292DB2966E79672BF8B7D0D0A3398A2E374C0E4A1C91DFE8BC530F -20180606065112 2 6 100 5121 2 37585C4D659694D0B23500E18769F1EF93A776FF9FB92171358D9FE60A517E2CB91A4984A7E923B213CA5ED889D7CBD176B7B475315CB90EBD9920E41043A6C0BD37A8E2679AA7944ED273FDB1CBCAED3B2950BC73132CDE38789421CDDFAEC1C7662F3F9C3EA0C56758D7D3D0257499CE832FEF537F1BC17FFCB2360465A663AB629EF222B7C1E2ACF6368963B6A3D89E5617765B01D483C3BEB0F40D2BC4F4E0512CE41796C411A1EF20D70ED4F071EC56A24C52161D82B9222A52D56B0E48E118DDD1A5C01042E401B6724C80E939B3140A471CEA8C28E33883903283153A3EED07A506818930D61B420B0FF8D1B5F33A2E112B363D2B85A9FDE12AAA3DAE0E08945F4CE4E038A1B61624DD76CADFE172B0FE3C332F7B014634E7F4F80268EB6F1261F0667BF1FC4D9F0BA87C4CC42AC1261F7DA7C4508E3A70485F3D41B5B7AFA8DF0CA0CE61AF7A696B769FF69AB808AED3AE33F40E5E6DA7F8849349FB4A262965160C155B86AA0E13A8FB5EF1BD477790FD8949970B71DD508C971A88781DADEF4F67C1BF8E6BE1AC75B9E6CD74C471FB2D94B27269A9E30B671A52ABE673DAD632C1CC9C139468B852C06844D0D05A40D3A50D410DC4B8DD5B118626128CC5A2DA0093BE486F5D30E621DDB217B03AB97097733BD6E8D1CCFC7515D8761CF7EB0525483678785E29093865A60DA525D0A1D282501C27B1F6C7FEC64BED44F5409889BE78E049E22C5095E7EF4C9B7E88A49DF5CDBE6016A13072A28500AC00E64DB3CE3EBFC668CA0CB94CDDE8381F342E5CE4E2B1FC3067A295771DD0A8FFF308D5A47A91F1A67D8F77C3D20B88EAF17915867EE181DBC2DE8DF43B7179396A563292DB2966E79672BF8B7D0D0A3398A2E374C0E4A1C91DFEA7F6853 -20180606065226 2 6 100 5121 2 37585C4D659694D0B23500E18769F1EF93A776FF9FB92171358D9FE60A517E2CB91A4984A7E923B213CA5ED889D7CBD176B7B475315CB90EBD9920E41043A6C0BD37A8E2679AA7944ED273FDB1CBCAED3B2950BC73132CDE38789421CDDFAEC1C7662F3F9C3EA0C56758D7D3D0257499CE832FEF537F1BC17FFCB2360465A663AB629EF222B7C1E2ACF6368963B6A3D89E5617765B01D483C3BEB0F40D2BC4F4E0512CE41796C411A1EF20D70ED4F071EC56A24C52161D82B9222A52D56B0E48E118DDD1A5C01042E401B6724C80E939B3140A471CEA8C28E33883903283153A3EED07A506818930D61B420B0FF8D1B5F33A2E112B363D2B85A9FDE12AAA3DAE0E08945F4CE4E038A1B61624DD76CADFE172B0FE3C332F7B014634E7F4F80268EB6F1261F0667BF1FC4D9F0BA87C4CC42AC1261F7DA7C4508E3A70485F3D41B5B7AFA8DF0CA0CE61AF7A696B769FF69AB808AED3AE33F40E5E6DA7F8849349FB4A262965160C155B86AA0E13A8FB5EF1BD477790FD8949970B71DD508C971A88781DADEF4F67C1BF8E6BE1AC75B9E6CD74C471FB2D94B27269A9E30B671A52ABE673DAD632C1CC9C139468B852C06844D0D05A40D3A50D410DC4B8DD5B118626128CC5A2DA0093BE486F5D30E621DDB217B03AB97097733BD6E8D1CCFC7515D8761CF7EB0525483678785E29093865A60DA525D0A1D282501C27B1F6C7FEC64BED44F5409889BE78E049E22C5095E7EF4C9B7E88A49DF5CDBE6016A13072A28500AC00E64DB3CE3EBFC668CA0CB94CDDE8381F342E5CE4E2B1FC3067A295771DD0A8FFF308D5A47A91F1A67D8F77C3D20B88EAF17915867EE181DBC2DE8DF43B7179396A563292DB2966E79672BF8B7D0D0A3398A2E374C0E4A1C91DFEA8EAA53 -20180606071446 2 6 100 5121 5 37585C4D659694D0B23500E18769F1EF93A776FF9FB92171358D9FE60A517E2CB91A4984A7E923B213CA5ED889D7CBD176B7B475315CB90EBD9920E41043A6C0BD37A8E2679AA7944ED273FDB1CBCAED3B2950BC73132CDE38789421CDDFAEC1C7662F3F9C3EA0C56758D7D3D0257499CE832FEF537F1BC17FFCB2360465A663AB629EF222B7C1E2ACF6368963B6A3D89E5617765B01D483C3BEB0F40D2BC4F4E0512CE41796C411A1EF20D70ED4F071EC56A24C52161D82B9222A52D56B0E48E118DDD1A5C01042E401B6724C80E939B3140A471CEA8C28E33883903283153A3EED07A506818930D61B420B0FF8D1B5F33A2E112B363D2B85A9FDE12AAA3DAE0E08945F4CE4E038A1B61624DD76CADFE172B0FE3C332F7B014634E7F4F80268EB6F1261F0667BF1FC4D9F0BA87C4CC42AC1261F7DA7C4508E3A70485F3D41B5B7AFA8DF0CA0CE61AF7A696B769FF69AB808AED3AE33F40E5E6DA7F8849349FB4A262965160C155B86AA0E13A8FB5EF1BD477790FD8949970B71DD508C971A88781DADEF4F67C1BF8E6BE1AC75B9E6CD74C471FB2D94B27269A9E30B671A52ABE673DAD632C1CC9C139468B852C06844D0D05A40D3A50D410DC4B8DD5B118626128CC5A2DA0093BE486F5D30E621DDB217B03AB97097733BD6E8D1CCFC7515D8761CF7EB0525483678785E29093865A60DA525D0A1D282501C27B1F6C7FEC64BED44F5409889BE78E049E22C5095E7EF4C9B7E88A49DF5CDBE6016A13072A28500AC00E64DB3CE3EBFC668CA0CB94CDDE8381F342E5CE4E2B1FC3067A295771DD0A8FFF308D5A47A91F1A67D8F77C3D20B88EAF17915867EE181DBC2DE8DF43B7179396A563292DB2966E79672BF8B7D0D0A3398A2E374C0E4A1C91DFEC3498CF -20180606073822 2 6 100 5121 2 37585C4D659694D0B23500E18769F1EF93A776FF9FB92171358D9FE60A517E2CB91A4984A7E923B213CA5ED889D7CBD176B7B475315CB90EBD9920E41043A6C0BD37A8E2679AA7944ED273FDB1CBCAED3B2950BC73132CDE38789421CDDFAEC1C7662F3F9C3EA0C56758D7D3D0257499CE832FEF537F1BC17FFCB2360465A663AB629EF222B7C1E2ACF6368963B6A3D89E5617765B01D483C3BEB0F40D2BC4F4E0512CE41796C411A1EF20D70ED4F071EC56A24C52161D82B9222A52D56B0E48E118DDD1A5C01042E401B6724C80E939B3140A471CEA8C28E33883903283153A3EED07A506818930D61B420B0FF8D1B5F33A2E112B363D2B85A9FDE12AAA3DAE0E08945F4CE4E038A1B61624DD76CADFE172B0FE3C332F7B014634E7F4F80268EB6F1261F0667BF1FC4D9F0BA87C4CC42AC1261F7DA7C4508E3A70485F3D41B5B7AFA8DF0CA0CE61AF7A696B769FF69AB808AED3AE33F40E5E6DA7F8849349FB4A262965160C155B86AA0E13A8FB5EF1BD477790FD8949970B71DD508C971A88781DADEF4F67C1BF8E6BE1AC75B9E6CD74C471FB2D94B27269A9E30B671A52ABE673DAD632C1CC9C139468B852C06844D0D05A40D3A50D410DC4B8DD5B118626128CC5A2DA0093BE486F5D30E621DDB217B03AB97097733BD6E8D1CCFC7515D8761CF7EB0525483678785E29093865A60DA525D0A1D282501C27B1F6C7FEC64BED44F5409889BE78E049E22C5095E7EF4C9B7E88A49DF5CDBE6016A13072A28500AC00E64DB3CE3EBFC668CA0CB94CDDE8381F342E5CE4E2B1FC3067A295771DD0A8FFF308D5A47A91F1A67D8F77C3D20B88EAF17915867EE181DBC2DE8DF43B7179396A563292DB2966E79672BF8B7D0D0A3398A2E374C0E4A1C91DFEDF3BF93 20180606081635 2 6 100 6143 5 E52412D53049FA1E04E2773DFEC7889C30A4F3D4C3B3F7097EE2AF5389338B19F1A6F05E376AACA05A03716100BA338DA622A68FBE05566BBE6A32410BFAAF78177F712BEEE39DC8282F5E395907F7B7BB12E2A4C5C5E960E3C130C6F47F5A4D3E568D82E2D3A2BCB2BFDBDEA6739C8BB88F694A66EBF24F1AE57EB8C932B4DD9E83E0B400B70670CC4956B5B6E8F386D597FBDD1B7C7835024F002C2D3786B6BB82877959B16A78ED6192602832F6931E0A822D4ED88EB2BFA40234061518AC7E18EA0DF2398E27ECA306532B6155E18477E5A74B005A7A3903F762930F0EAA569B7C50B7502E3D4E4EDB115627DE49BBDED6528FFBBEEBF16D7D3EE29D61B0D5859ABDD8665345D05F7E5DD8FBB170203113333A87502E3E299E90F2D83AF447D67ADA23E77475E0E7416860385EDF12DD59F2576551D9E82457396064EDE8D13BDDF3F40AE591B4A0D3CB0527454EFABC40E629AF3B5A02BDCA06E194A195B13AD319E81915AB1375BB952A0B779040C46C23184962602D70BB0CBA2ED70AB24E2EB08B1BAEFE96D14154FCF1D00461A4D0B7291EC5010678482A136E4B3A4588C80C7AD89189FAA17F29FF4E5D318079EE5C181CC2CF586A6F73114BD85117EB97FE65CC2D2E3441B688C0DBDD11909783227150DF0FEE0946F39E6565285A62CCBF5464D5A518A7D217C6142A31134C0E8A8F401930F3A6979BC35C24275247444FBAAC542AB1280E7B455888F3AD8FE770FF3782DA1A5D76B3C633C40EC2E6A1B0615AF15931D3DD11F98907E0B836B16D5A2E450A97747509EFEB9963BC306A650B3215E1E0FCCEF9F323B56B75D579A8B280D757E2D28D7ABD079E2C0A7FAE800035BD7386E4EB901AA3BE80644720E625195CA36048A7C3514BD0247C37AD92AA3265B496C539B7DE84E39909E982A6D10D177803B0D38E2B80323C1DE7AD104FD5C1D1D631DA6309651C987255308DBE2DF7F4ED233BF6E6C07AD563E6F6FA23E3E9D248135EF83CBC35DB87F39E666F238B8DB06200DCF63EFA84117C2FDF365125604E09D972A32CA03490FD7D81C947C0B278BB2A3819B8FC5F 20180606083850 2 6 100 6143 2 E52412D53049FA1E04E2773DFEC7889C30A4F3D4C3B3F7097EE2AF5389338B19F1A6F05E376AACA05A03716100BA338DA622A68FBE05566BBE6A32410BFAAF78177F712BEEE39DC8282F5E395907F7B7BB12E2A4C5C5E960E3C130C6F47F5A4D3E568D82E2D3A2BCB2BFDBDEA6739C8BB88F694A66EBF24F1AE57EB8C932B4DD9E83E0B400B70670CC4956B5B6E8F386D597FBDD1B7C7835024F002C2D3786B6BB82877959B16A78ED6192602832F6931E0A822D4ED88EB2BFA40234061518AC7E18EA0DF2398E27ECA306532B6155E18477E5A74B005A7A3903F762930F0EAA569B7C50B7502E3D4E4EDB115627DE49BBDED6528FFBBEEBF16D7D3EE29D61B0D5859ABDD8665345D05F7E5DD8FBB170203113333A87502E3E299E90F2D83AF447D67ADA23E77475E0E7416860385EDF12DD59F2576551D9E82457396064EDE8D13BDDF3F40AE591B4A0D3CB0527454EFABC40E629AF3B5A02BDCA06E194A195B13AD319E81915AB1375BB952A0B779040C46C23184962602D70BB0CBA2ED70AB24E2EB08B1BAEFE96D14154FCF1D00461A4D0B7291EC5010678482A136E4B3A4588C80C7AD89189FAA17F29FF4E5D318079EE5C181CC2CF586A6F73114BD85117EB97FE65CC2D2E3441B688C0DBDD11909783227150DF0FEE0946F39E6565285A62CCBF5464D5A518A7D217C6142A31134C0E8A8F401930F3A6979BC35C24275247444FBAAC542AB1280E7B455888F3AD8FE770FF3782DA1A5D76B3C633C40EC2E6A1B0615AF15931D3DD11F98907E0B836B16D5A2E450A97747509EFEB9963BC306A650B3215E1E0FCCEF9F323B56B75D579A8B280D757E2D28D7ABD079E2C0A7FAE800035BD7386E4EB901AA3BE80644720E625195CA36048A7C3514BD0247C37AD92AA3265B496C539B7DE84E39909E982A6D10D177803B0D38E2B80323C1DE7AD104FD5C1D1D631DA6309651C987255308DBE2DF7F4ED233BF6E6C07AD563E6F6FA23E3E9D248135EF83CBC35DB87F39E666F238B8DB06200DCF63EFA84117C2FDF365125604E09D972A32CA03490FD7D81C947C0B278BB2A381B1D13EB 20180606084353 2 6 100 6143 5 E52412D53049FA1E04E2773DFEC7889C30A4F3D4C3B3F7097EE2AF5389338B19F1A6F05E376AACA05A03716100BA338DA622A68FBE05566BBE6A32410BFAAF78177F712BEEE39DC8282F5E395907F7B7BB12E2A4C5C5E960E3C130C6F47F5A4D3E568D82E2D3A2BCB2BFDBDEA6739C8BB88F694A66EBF24F1AE57EB8C932B4DD9E83E0B400B70670CC4956B5B6E8F386D597FBDD1B7C7835024F002C2D3786B6BB82877959B16A78ED6192602832F6931E0A822D4ED88EB2BFA40234061518AC7E18EA0DF2398E27ECA306532B6155E18477E5A74B005A7A3903F762930F0EAA569B7C50B7502E3D4E4EDB115627DE49BBDED6528FFBBEEBF16D7D3EE29D61B0D5859ABDD8665345D05F7E5DD8FBB170203113333A87502E3E299E90F2D83AF447D67ADA23E77475E0E7416860385EDF12DD59F2576551D9E82457396064EDE8D13BDDF3F40AE591B4A0D3CB0527454EFABC40E629AF3B5A02BDCA06E194A195B13AD319E81915AB1375BB952A0B779040C46C23184962602D70BB0CBA2ED70AB24E2EB08B1BAEFE96D14154FCF1D00461A4D0B7291EC5010678482A136E4B3A4588C80C7AD89189FAA17F29FF4E5D318079EE5C181CC2CF586A6F73114BD85117EB97FE65CC2D2E3441B688C0DBDD11909783227150DF0FEE0946F39E6565285A62CCBF5464D5A518A7D217C6142A31134C0E8A8F401930F3A6979BC35C24275247444FBAAC542AB1280E7B455888F3AD8FE770FF3782DA1A5D76B3C633C40EC2E6A1B0615AF15931D3DD11F98907E0B836B16D5A2E450A97747509EFEB9963BC306A650B3215E1E0FCCEF9F323B56B75D579A8B280D757E2D28D7ABD079E2C0A7FAE800035BD7386E4EB901AA3BE80644720E625195CA36048A7C3514BD0247C37AD92AA3265B496C539B7DE84E39909E982A6D10D177803B0D38E2B80323C1DE7AD104FD5C1D1D631DA6309651C987255308DBE2DF7F4ED233BF6E6C07AD563E6F6FA23E3E9D248135EF83CBC35DB87F39E666F238B8DB06200DCF63EFA84117C2FDF365125604E09D972A32CA03490FD7D81C947C0B278BB2A381B65B5F7 @@ -235,38 +201,6 @@ 20180607010449 2 6 100 6143 5 E52412D53049FA1E04E2773DFEC7889C30A4F3D4C3B3F7097EE2AF5389338B19F1A6F05E376AACA05A03716100BA338DA622A68FBE05566BBE6A32410BFAAF78177F712BEEE39DC8282F5E395907F7B7BB12E2A4C5C5E960E3C130C6F47F5A4D3E568D82E2D3A2BCB2BFDBDEA6739C8BB88F694A66EBF24F1AE57EB8C932B4DD9E83E0B400B70670CC4956B5B6E8F386D597FBDD1B7C7835024F002C2D3786B6BB82877959B16A78ED6192602832F6931E0A822D4ED88EB2BFA40234061518AC7E18EA0DF2398E27ECA306532B6155E18477E5A74B005A7A3903F762930F0EAA569B7C50B7502E3D4E4EDB115627DE49BBDED6528FFBBEEBF16D7D3EE29D61B0D5859ABDD8665345D05F7E5DD8FBB170203113333A87502E3E299E90F2D83AF447D67ADA23E77475E0E7416860385EDF12DD59F2576551D9E82457396064EDE8D13BDDF3F40AE591B4A0D3CB0527454EFABC40E629AF3B5A02BDCA06E194A195B13AD319E81915AB1375BB952A0B779040C46C23184962602D70BB0CBA2ED70AB24E2EB08B1BAEFE96D14154FCF1D00461A4D0B7291EC5010678482A136E4B3A4588C80C7AD89189FAA17F29FF4E5D318079EE5C181CC2CF586A6F73114BD85117EB97FE65CC2D2E3441B688C0DBDD11909783227150DF0FEE0946F39E6565285A62CCBF5464D5A518A7D217C6142A31134C0E8A8F401930F3A6979BC35C24275247444FBAAC542AB1280E7B455888F3AD8FE770FF3782DA1A5D76B3C633C40EC2E6A1B0615AF15931D3DD11F98907E0B836B16D5A2E450A97747509EFEB9963BC306A650B3215E1E0FCCEF9F323B56B75D579A8B280D757E2D28D7ABD079E2C0A7FAE800035BD7386E4EB901AA3BE80644720E625195CA36048A7C3514BD0247C37AD92AA3265B496C539B7DE84E39909E982A6D10D177803B0D38E2B80323C1DE7AD104FD5C1D1D631DA6309651C987255308DBE2DF7F4ED233BF6E6C07AD563E6F6FA23E3E9D248135EF83CBC35DB87F39E666F238B8DB06200DCF63EFA84117C2FDF365125604E09D972A32CA03490FD7D81C947C0B278BB2A385542D20F 20180607022006 2 6 100 6143 2 E52412D53049FA1E04E2773DFEC7889C30A4F3D4C3B3F7097EE2AF5389338B19F1A6F05E376AACA05A03716100BA338DA622A68FBE05566BBE6A32410BFAAF78177F712BEEE39DC8282F5E395907F7B7BB12E2A4C5C5E960E3C130C6F47F5A4D3E568D82E2D3A2BCB2BFDBDEA6739C8BB88F694A66EBF24F1AE57EB8C932B4DD9E83E0B400B70670CC4956B5B6E8F386D597FBDD1B7C7835024F002C2D3786B6BB82877959B16A78ED6192602832F6931E0A822D4ED88EB2BFA40234061518AC7E18EA0DF2398E27ECA306532B6155E18477E5A74B005A7A3903F762930F0EAA569B7C50B7502E3D4E4EDB115627DE49BBDED6528FFBBEEBF16D7D3EE29D61B0D5859ABDD8665345D05F7E5DD8FBB170203113333A87502E3E299E90F2D83AF447D67ADA23E77475E0E7416860385EDF12DD59F2576551D9E82457396064EDE8D13BDDF3F40AE591B4A0D3CB0527454EFABC40E629AF3B5A02BDCA06E194A195B13AD319E81915AB1375BB952A0B779040C46C23184962602D70BB0CBA2ED70AB24E2EB08B1BAEFE96D14154FCF1D00461A4D0B7291EC5010678482A136E4B3A4588C80C7AD89189FAA17F29FF4E5D318079EE5C181CC2CF586A6F73114BD85117EB97FE65CC2D2E3441B688C0DBDD11909783227150DF0FEE0946F39E6565285A62CCBF5464D5A518A7D217C6142A31134C0E8A8F401930F3A6979BC35C24275247444FBAAC542AB1280E7B455888F3AD8FE770FF3782DA1A5D76B3C633C40EC2E6A1B0615AF15931D3DD11F98907E0B836B16D5A2E450A97747509EFEB9963BC306A650B3215E1E0FCCEF9F323B56B75D579A8B280D757E2D28D7ABD079E2C0A7FAE800035BD7386E4EB901AA3BE80644720E625195CA36048A7C3514BD0247C37AD92AA3265B496C539B7DE84E39909E982A6D10D177803B0D38E2B80323C1DE7AD104FD5C1D1D631DA6309651C987255308DBE2DF7F4ED233BF6E6C07AD563E6F6FA23E3E9D248135EF83CBC35DB87F39E666F238B8DB06200DCF63EFA84117C2FDF365125604E09D972A32CA03490FD7D81C947C0B278BB2A3859892A7B 20180607022905 2 6 100 6143 2 E52412D53049FA1E04E2773DFEC7889C30A4F3D4C3B3F7097EE2AF5389338B19F1A6F05E376AACA05A03716100BA338DA622A68FBE05566BBE6A32410BFAAF78177F712BEEE39DC8282F5E395907F7B7BB12E2A4C5C5E960E3C130C6F47F5A4D3E568D82E2D3A2BCB2BFDBDEA6739C8BB88F694A66EBF24F1AE57EB8C932B4DD9E83E0B400B70670CC4956B5B6E8F386D597FBDD1B7C7835024F002C2D3786B6BB82877959B16A78ED6192602832F6931E0A822D4ED88EB2BFA40234061518AC7E18EA0DF2398E27ECA306532B6155E18477E5A74B005A7A3903F762930F0EAA569B7C50B7502E3D4E4EDB115627DE49BBDED6528FFBBEEBF16D7D3EE29D61B0D5859ABDD8665345D05F7E5DD8FBB170203113333A87502E3E299E90F2D83AF447D67ADA23E77475E0E7416860385EDF12DD59F2576551D9E82457396064EDE8D13BDDF3F40AE591B4A0D3CB0527454EFABC40E629AF3B5A02BDCA06E194A195B13AD319E81915AB1375BB952A0B779040C46C23184962602D70BB0CBA2ED70AB24E2EB08B1BAEFE96D14154FCF1D00461A4D0B7291EC5010678482A136E4B3A4588C80C7AD89189FAA17F29FF4E5D318079EE5C181CC2CF586A6F73114BD85117EB97FE65CC2D2E3441B688C0DBDD11909783227150DF0FEE0946F39E6565285A62CCBF5464D5A518A7D217C6142A31134C0E8A8F401930F3A6979BC35C24275247444FBAAC542AB1280E7B455888F3AD8FE770FF3782DA1A5D76B3C633C40EC2E6A1B0615AF15931D3DD11F98907E0B836B16D5A2E450A97747509EFEB9963BC306A650B3215E1E0FCCEF9F323B56B75D579A8B280D757E2D28D7ABD079E2C0A7FAE800035BD7386E4EB901AA3BE80644720E625195CA36048A7C3514BD0247C37AD92AA3265B496C539B7DE84E39909E982A6D10D177803B0D38E2B80323C1DE7AD104FD5C1D1D631DA6309651C987255308DBE2DF7F4ED233BF6E6C07AD563E6F6FA23E3E9D248135EF83CBC35DB87F39E666F238B8DB06200DCF63EFA84117C2FDF365125604E09D972A32CA03490FD7D81C947C0B278BB2A385A0321EB -20180607045001 2 6 100 7167 2 E76114725DD41FA35A7DA1CF2A6C5701F689FC40E96BDAB540431472B92D5E38711F596734C76D1407BC11D5EECD10C52AE09D6EB46C7A119FE7BF2CE718363E5D9D06D972A80D1AEBE550FEF87C596C7B2FAB2FA6A27BE1D6CBFBD7A8B55FFBEFC78E6028AB4F83093162C4D8F9C4E72D45AEE3F788A1AA98A93B2E92DCC69739C3311002388723ED096B0FECB0698AB631108A2F5922453FA81DD178DDFB7A8E651E19E086F36684EF4958EF57E18393C173D07B594AEA0EAF5FE770F72550528C1637E341715CC80C74A2024CDEB2F949200E29BBFC0519F029CA2D45C4A66602553EF9A9086183018A055DFE19698B779908B119CB4C9E162680CA5E3EA471F4AF46F77DA5FF2A2C23C18509B1E8D0591B868A76163079461A83FBDAC95A5E8E4EB9152EAFCDFAD9CE5CDA7CE7846BEE134D52242152C4B671D1B2A05B927F6546A272DDEC50DDAE2D66B866C393F7CA3E2FF505FCC5187394BEEA2BC94F840B800884A8853A6A9AF1FB62837DDE066CC17EDDAD6921F0D96A7A9C2C1705CC024050201E93AF587FE1A480EDA8DCC257B0F7FF6A3006613250098090436BC3C63039846C8A2FD4180BB744D8584184C18F7B05BB110FC4D1C8C1D019EB072FC5C55DDBAE8272CAC153810A6D5A828B7B86350F656D350299264A111C8C876D7502D634C6014F324773E64FFE56DA602736355061CAF39DD29706111ED10F00BD80EE4FE076F6BA743F210C9DC502493A8974FEC1E2706C094A20C29EA7DB29B37D3BD709CD6942FB8ED5EE3D1C521AFFA3A242A13A5A705C458027118C950E656794F671373CE3A7A2D53C0C4BCDDF539D530ABE4A554ED76F7D0D43C1DFFEA320D1CD14A2BC5A5B2C04D84C17399C590D94D86F3E85EC66693D364379149FB7E5709C45B7077D8784114D14B640EB8F50DECCE1A83B041CD217155918BFE937BC0C4E8BBA30452A2F00A4D5F2091DD12C8849EA4989EAC9BCB7040C0049C3986D4F5BF0F3D55FD05A847D3FC4A58F4F5DB860333B1C60F8AE4646919D339B0E4342FEE167A214B25D17D25BDDB7F81BCC37551435C033A66BCD3775FCAE15CA9F311E90FBFF2A082AC2A6B8B0C7F4F5865BFB3B18DEE1E184587C3BA746869D70402BCE1AFAC1AA45BF4D8A6A390F076CA1D6381F7581E2A5718BF44EFB75E9864EBF5EB011FD7C0892C8500BFA01B422E4B67222C259A263863E66CC26C533D0796C0C17D4945D9E0ADDD961EAC0FC3E594B14C13B9A586B0EA3B1267B -20180607051849 2 6 100 7167 2 E76114725DD41FA35A7DA1CF2A6C5701F689FC40E96BDAB540431472B92D5E38711F596734C76D1407BC11D5EECD10C52AE09D6EB46C7A119FE7BF2CE718363E5D9D06D972A80D1AEBE550FEF87C596C7B2FAB2FA6A27BE1D6CBFBD7A8B55FFBEFC78E6028AB4F83093162C4D8F9C4E72D45AEE3F788A1AA98A93B2E92DCC69739C3311002388723ED096B0FECB0698AB631108A2F5922453FA81DD178DDFB7A8E651E19E086F36684EF4958EF57E18393C173D07B594AEA0EAF5FE770F72550528C1637E341715CC80C74A2024CDEB2F949200E29BBFC0519F029CA2D45C4A66602553EF9A9086183018A055DFE19698B779908B119CB4C9E162680CA5E3EA471F4AF46F77DA5FF2A2C23C18509B1E8D0591B868A76163079461A83FBDAC95A5E8E4EB9152EAFCDFAD9CE5CDA7CE7846BEE134D52242152C4B671D1B2A05B927F6546A272DDEC50DDAE2D66B866C393F7CA3E2FF505FCC5187394BEEA2BC94F840B800884A8853A6A9AF1FB62837DDE066CC17EDDAD6921F0D96A7A9C2C1705CC024050201E93AF587FE1A480EDA8DCC257B0F7FF6A3006613250098090436BC3C63039846C8A2FD4180BB744D8584184C18F7B05BB110FC4D1C8C1D019EB072FC5C55DDBAE8272CAC153810A6D5A828B7B86350F656D350299264A111C8C876D7502D634C6014F324773E64FFE56DA602736355061CAF39DD29706111ED10F00BD80EE4FE076F6BA743F210C9DC502493A8974FEC1E2706C094A20C29EA7DB29B37D3BD709CD6942FB8ED5EE3D1C521AFFA3A242A13A5A705C458027118C950E656794F671373CE3A7A2D53C0C4BCDDF539D530ABE4A554ED76F7D0D43C1DFFEA320D1CD14A2BC5A5B2C04D84C17399C590D94D86F3E85EC66693D364379149FB7E5709C45B7077D8784114D14B640EB8F50DECCE1A83B041CD217155918BFE937BC0C4E8BBA30452A2F00A4D5F2091DD12C8849EA4989EAC9BCB7040C0049C3986D4F5BF0F3D55FD05A847D3FC4A58F4F5DB860333B1C60F8AE4646919D339B0E4342FEE167A214B25D17D25BDDB7F81BCC37551435C033A66BCD3775FCAE15CA9F311E90FBFF2A082AC2A6B8B0C7F4F5865BFB3B18DEE1E184587C3BA746869D70402BCE1AFAC1AA45BF4D8A6A390F076CA1D6381F7581E2A5718BF44EFB75E9864EBF5EB011FD7C0892C8500BFA01B422E4B67222C259A263863E66CC26C533D0796C0C17D4945D9E0ADDD961EAC0FC3E594B14C13B9A586B0EA4D3FCDB -20180607052645 2 6 100 7167 5 E76114725DD41FA35A7DA1CF2A6C5701F689FC40E96BDAB540431472B92D5E38711F596734C76D1407BC11D5EECD10C52AE09D6EB46C7A119FE7BF2CE718363E5D9D06D972A80D1AEBE550FEF87C596C7B2FAB2FA6A27BE1D6CBFBD7A8B55FFBEFC78E6028AB4F83093162C4D8F9C4E72D45AEE3F788A1AA98A93B2E92DCC69739C3311002388723ED096B0FECB0698AB631108A2F5922453FA81DD178DDFB7A8E651E19E086F36684EF4958EF57E18393C173D07B594AEA0EAF5FE770F72550528C1637E341715CC80C74A2024CDEB2F949200E29BBFC0519F029CA2D45C4A66602553EF9A9086183018A055DFE19698B779908B119CB4C9E162680CA5E3EA471F4AF46F77DA5FF2A2C23C18509B1E8D0591B868A76163079461A83FBDAC95A5E8E4EB9152EAFCDFAD9CE5CDA7CE7846BEE134D52242152C4B671D1B2A05B927F6546A272DDEC50DDAE2D66B866C393F7CA3E2FF505FCC5187394BEEA2BC94F840B800884A8853A6A9AF1FB62837DDE066CC17EDDAD6921F0D96A7A9C2C1705CC024050201E93AF587FE1A480EDA8DCC257B0F7FF6A3006613250098090436BC3C63039846C8A2FD4180BB744D8584184C18F7B05BB110FC4D1C8C1D019EB072FC5C55DDBAE8272CAC153810A6D5A828B7B86350F656D350299264A111C8C876D7502D634C6014F324773E64FFE56DA602736355061CAF39DD29706111ED10F00BD80EE4FE076F6BA743F210C9DC502493A8974FEC1E2706C094A20C29EA7DB29B37D3BD709CD6942FB8ED5EE3D1C521AFFA3A242A13A5A705C458027118C950E656794F671373CE3A7A2D53C0C4BCDDF539D530ABE4A554ED76F7D0D43C1DFFEA320D1CD14A2BC5A5B2C04D84C17399C590D94D86F3E85EC66693D364379149FB7E5709C45B7077D8784114D14B640EB8F50DECCE1A83B041CD217155918BFE937BC0C4E8BBA30452A2F00A4D5F2091DD12C8849EA4989EAC9BCB7040C0049C3986D4F5BF0F3D55FD05A847D3FC4A58F4F5DB860333B1C60F8AE4646919D339B0E4342FEE167A214B25D17D25BDDB7F81BCC37551435C033A66BCD3775FCAE15CA9F311E90FBFF2A082AC2A6B8B0C7F4F5865BFB3B18DEE1E184587C3BA746869D70402BCE1AFAC1AA45BF4D8A6A390F076CA1D6381F7581E2A5718BF44EFB75E9864EBF5EB011FD7C0892C8500BFA01B422E4B67222C259A263863E66CC26C533D0796C0C17D4945D9E0ADDD961EAC0FC3E594B14C13B9A586B0EA51AE857 -20180607074455 2 6 100 7167 5 E76114725DD41FA35A7DA1CF2A6C5701F689FC40E96BDAB540431472B92D5E38711F596734C76D1407BC11D5EECD10C52AE09D6EB46C7A119FE7BF2CE718363E5D9D06D972A80D1AEBE550FEF87C596C7B2FAB2FA6A27BE1D6CBFBD7A8B55FFBEFC78E6028AB4F83093162C4D8F9C4E72D45AEE3F788A1AA98A93B2E92DCC69739C3311002388723ED096B0FECB0698AB631108A2F5922453FA81DD178DDFB7A8E651E19E086F36684EF4958EF57E18393C173D07B594AEA0EAF5FE770F72550528C1637E341715CC80C74A2024CDEB2F949200E29BBFC0519F029CA2D45C4A66602553EF9A9086183018A055DFE19698B779908B119CB4C9E162680CA5E3EA471F4AF46F77DA5FF2A2C23C18509B1E8D0591B868A76163079461A83FBDAC95A5E8E4EB9152EAFCDFAD9CE5CDA7CE7846BEE134D52242152C4B671D1B2A05B927F6546A272DDEC50DDAE2D66B866C393F7CA3E2FF505FCC5187394BEEA2BC94F840B800884A8853A6A9AF1FB62837DDE066CC17EDDAD6921F0D96A7A9C2C1705CC024050201E93AF587FE1A480EDA8DCC257B0F7FF6A3006613250098090436BC3C63039846C8A2FD4180BB744D8584184C18F7B05BB110FC4D1C8C1D019EB072FC5C55DDBAE8272CAC153810A6D5A828B7B86350F656D350299264A111C8C876D7502D634C6014F324773E64FFE56DA602736355061CAF39DD29706111ED10F00BD80EE4FE076F6BA743F210C9DC502493A8974FEC1E2706C094A20C29EA7DB29B37D3BD709CD6942FB8ED5EE3D1C521AFFA3A242A13A5A705C458027118C950E656794F671373CE3A7A2D53C0C4BCDDF539D530ABE4A554ED76F7D0D43C1DFFEA320D1CD14A2BC5A5B2C04D84C17399C590D94D86F3E85EC66693D364379149FB7E5709C45B7077D8784114D14B640EB8F50DECCE1A83B041CD217155918BFE937BC0C4E8BBA30452A2F00A4D5F2091DD12C8849EA4989EAC9BCB7040C0049C3986D4F5BF0F3D55FD05A847D3FC4A58F4F5DB860333B1C60F8AE4646919D339B0E4342FEE167A214B25D17D25BDDB7F81BCC37551435C033A66BCD3775FCAE15CA9F311E90FBFF2A082AC2A6B8B0C7F4F5865BFB3B18DEE1E184587C3BA746869D70402BCE1AFAC1AA45BF4D8A6A390F076CA1D6381F7581E2A5718BF44EFB75E9864EBF5EB011FD7C0892C8500BFA01B422E4B67222C259A263863E66CC26C533D0796C0C17D4945D9E0ADDD961EAC0FC3E594B14C13B9A586B0EAABB372F -20180607085512 2 6 100 7167 2 E76114725DD41FA35A7DA1CF2A6C5701F689FC40E96BDAB540431472B92D5E38711F596734C76D1407BC11D5EECD10C52AE09D6EB46C7A119FE7BF2CE718363E5D9D06D972A80D1AEBE550FEF87C596C7B2FAB2FA6A27BE1D6CBFBD7A8B55FFBEFC78E6028AB4F83093162C4D8F9C4E72D45AEE3F788A1AA98A93B2E92DCC69739C3311002388723ED096B0FECB0698AB631108A2F5922453FA81DD178DDFB7A8E651E19E086F36684EF4958EF57E18393C173D07B594AEA0EAF5FE770F72550528C1637E341715CC80C74A2024CDEB2F949200E29BBFC0519F029CA2D45C4A66602553EF9A9086183018A055DFE19698B779908B119CB4C9E162680CA5E3EA471F4AF46F77DA5FF2A2C23C18509B1E8D0591B868A76163079461A83FBDAC95A5E8E4EB9152EAFCDFAD9CE5CDA7CE7846BEE134D52242152C4B671D1B2A05B927F6546A272DDEC50DDAE2D66B866C393F7CA3E2FF505FCC5187394BEEA2BC94F840B800884A8853A6A9AF1FB62837DDE066CC17EDDAD6921F0D96A7A9C2C1705CC024050201E93AF587FE1A480EDA8DCC257B0F7FF6A3006613250098090436BC3C63039846C8A2FD4180BB744D8584184C18F7B05BB110FC4D1C8C1D019EB072FC5C55DDBAE8272CAC153810A6D5A828B7B86350F656D350299264A111C8C876D7502D634C6014F324773E64FFE56DA602736355061CAF39DD29706111ED10F00BD80EE4FE076F6BA743F210C9DC502493A8974FEC1E2706C094A20C29EA7DB29B37D3BD709CD6942FB8ED5EE3D1C521AFFA3A242A13A5A705C458027118C950E656794F671373CE3A7A2D53C0C4BCDDF539D530ABE4A554ED76F7D0D43C1DFFEA320D1CD14A2BC5A5B2C04D84C17399C590D94D86F3E85EC66693D364379149FB7E5709C45B7077D8784114D14B640EB8F50DECCE1A83B041CD217155918BFE937BC0C4E8BBA30452A2F00A4D5F2091DD12C8849EA4989EAC9BCB7040C0049C3986D4F5BF0F3D55FD05A847D3FC4A58F4F5DB860333B1C60F8AE4646919D339B0E4342FEE167A214B25D17D25BDDB7F81BCC37551435C033A66BCD3775FCAE15CA9F311E90FBFF2A082AC2A6B8B0C7F4F5865BFB3B18DEE1E184587C3BA746869D70402BCE1AFAC1AA45BF4D8A6A390F076CA1D6381F7581E2A5718BF44EFB75E9864EBF5EB011FD7C0892C8500BFA01B422E4B67222C259A263863E66CC26C533D0796C0C17D4945D9E0ADDD961EAC0FC3E594B14C13B9A586B0EAD8764F3 -20180607101435 2 6 100 7167 5 E76114725DD41FA35A7DA1CF2A6C5701F689FC40E96BDAB540431472B92D5E38711F596734C76D1407BC11D5EECD10C52AE09D6EB46C7A119FE7BF2CE718363E5D9D06D972A80D1AEBE550FEF87C596C7B2FAB2FA6A27BE1D6CBFBD7A8B55FFBEFC78E6028AB4F83093162C4D8F9C4E72D45AEE3F788A1AA98A93B2E92DCC69739C3311002388723ED096B0FECB0698AB631108A2F5922453FA81DD178DDFB7A8E651E19E086F36684EF4958EF57E18393C173D07B594AEA0EAF5FE770F72550528C1637E341715CC80C74A2024CDEB2F949200E29BBFC0519F029CA2D45C4A66602553EF9A9086183018A055DFE19698B779908B119CB4C9E162680CA5E3EA471F4AF46F77DA5FF2A2C23C18509B1E8D0591B868A76163079461A83FBDAC95A5E8E4EB9152EAFCDFAD9CE5CDA7CE7846BEE134D52242152C4B671D1B2A05B927F6546A272DDEC50DDAE2D66B866C393F7CA3E2FF505FCC5187394BEEA2BC94F840B800884A8853A6A9AF1FB62837DDE066CC17EDDAD6921F0D96A7A9C2C1705CC024050201E93AF587FE1A480EDA8DCC257B0F7FF6A3006613250098090436BC3C63039846C8A2FD4180BB744D8584184C18F7B05BB110FC4D1C8C1D019EB072FC5C55DDBAE8272CAC153810A6D5A828B7B86350F656D350299264A111C8C876D7502D634C6014F324773E64FFE56DA602736355061CAF39DD29706111ED10F00BD80EE4FE076F6BA743F210C9DC502493A8974FEC1E2706C094A20C29EA7DB29B37D3BD709CD6942FB8ED5EE3D1C521AFFA3A242A13A5A705C458027118C950E656794F671373CE3A7A2D53C0C4BCDDF539D530ABE4A554ED76F7D0D43C1DFFEA320D1CD14A2BC5A5B2C04D84C17399C590D94D86F3E85EC66693D364379149FB7E5709C45B7077D8784114D14B640EB8F50DECCE1A83B041CD217155918BFE937BC0C4E8BBA30452A2F00A4D5F2091DD12C8849EA4989EAC9BCB7040C0049C3986D4F5BF0F3D55FD05A847D3FC4A58F4F5DB860333B1C60F8AE4646919D339B0E4342FEE167A214B25D17D25BDDB7F81BCC37551435C033A66BCD3775FCAE15CA9F311E90FBFF2A082AC2A6B8B0C7F4F5865BFB3B18DEE1E184587C3BA746869D70402BCE1AFAC1AA45BF4D8A6A390F076CA1D6381F7581E2A5718BF44EFB75E9864EBF5EB011FD7C0892C8500BFA01B422E4B67222C259A263863E66CC26C533D0796C0C17D4945D9E0ADDD961EAC0FC3E594B14C13B9A586B0EB0B93D7F -20180607122858 2 6 100 7167 5 E76114725DD41FA35A7DA1CF2A6C5701F689FC40E96BDAB540431472B92D5E38711F596734C76D1407BC11D5EECD10C52AE09D6EB46C7A119FE7BF2CE718363E5D9D06D972A80D1AEBE550FEF87C596C7B2FAB2FA6A27BE1D6CBFBD7A8B55FFBEFC78E6028AB4F83093162C4D8F9C4E72D45AEE3F788A1AA98A93B2E92DCC69739C3311002388723ED096B0FECB0698AB631108A2F5922453FA81DD178DDFB7A8E651E19E086F36684EF4958EF57E18393C173D07B594AEA0EAF5FE770F72550528C1637E341715CC80C74A2024CDEB2F949200E29BBFC0519F029CA2D45C4A66602553EF9A9086183018A055DFE19698B779908B119CB4C9E162680CA5E3EA471F4AF46F77DA5FF2A2C23C18509B1E8D0591B868A76163079461A83FBDAC95A5E8E4EB9152EAFCDFAD9CE5CDA7CE7846BEE134D52242152C4B671D1B2A05B927F6546A272DDEC50DDAE2D66B866C393F7CA3E2FF505FCC5187394BEEA2BC94F840B800884A8853A6A9AF1FB62837DDE066CC17EDDAD6921F0D96A7A9C2C1705CC024050201E93AF587FE1A480EDA8DCC257B0F7FF6A3006613250098090436BC3C63039846C8A2FD4180BB744D8584184C18F7B05BB110FC4D1C8C1D019EB072FC5C55DDBAE8272CAC153810A6D5A828B7B86350F656D350299264A111C8C876D7502D634C6014F324773E64FFE56DA602736355061CAF39DD29706111ED10F00BD80EE4FE076F6BA743F210C9DC502493A8974FEC1E2706C094A20C29EA7DB29B37D3BD709CD6942FB8ED5EE3D1C521AFFA3A242A13A5A705C458027118C950E656794F671373CE3A7A2D53C0C4BCDDF539D530ABE4A554ED76F7D0D43C1DFFEA320D1CD14A2BC5A5B2C04D84C17399C590D94D86F3E85EC66693D364379149FB7E5709C45B7077D8784114D14B640EB8F50DECCE1A83B041CD217155918BFE937BC0C4E8BBA30452A2F00A4D5F2091DD12C8849EA4989EAC9BCB7040C0049C3986D4F5BF0F3D55FD05A847D3FC4A58F4F5DB860333B1C60F8AE4646919D339B0E4342FEE167A214B25D17D25BDDB7F81BCC37551435C033A66BCD3775FCAE15CA9F311E90FBFF2A082AC2A6B8B0C7F4F5865BFB3B18DEE1E184587C3BA746869D70402BCE1AFAC1AA45BF4D8A6A390F076CA1D6381F7581E2A5718BF44EFB75E9864EBF5EB011FD7C0892C8500BFA01B422E4B67222C259A263863E66CC26C533D0796C0C17D4945D9E0ADDD961EAC0FC3E594B14C13B9A586B0EB613F7A7 -20180607163002 2 6 100 7167 5 E76114725DD41FA35A7DA1CF2A6C5701F689FC40E96BDAB540431472B92D5E38711F596734C76D1407BC11D5EECD10C52AE09D6EB46C7A119FE7BF2CE718363E5D9D06D972A80D1AEBE550FEF87C596C7B2FAB2FA6A27BE1D6CBFBD7A8B55FFBEFC78E6028AB4F83093162C4D8F9C4E72D45AEE3F788A1AA98A93B2E92DCC69739C3311002388723ED096B0FECB0698AB631108A2F5922453FA81DD178DDFB7A8E651E19E086F36684EF4958EF57E18393C173D07B594AEA0EAF5FE770F72550528C1637E341715CC80C74A2024CDEB2F949200E29BBFC0519F029CA2D45C4A66602553EF9A9086183018A055DFE19698B779908B119CB4C9E162680CA5E3EA471F4AF46F77DA5FF2A2C23C18509B1E8D0591B868A76163079461A83FBDAC95A5E8E4EB9152EAFCDFAD9CE5CDA7CE7846BEE134D52242152C4B671D1B2A05B927F6546A272DDEC50DDAE2D66B866C393F7CA3E2FF505FCC5187394BEEA2BC94F840B800884A8853A6A9AF1FB62837DDE066CC17EDDAD6921F0D96A7A9C2C1705CC024050201E93AF587FE1A480EDA8DCC257B0F7FF6A3006613250098090436BC3C63039846C8A2FD4180BB744D8584184C18F7B05BB110FC4D1C8C1D019EB072FC5C55DDBAE8272CAC153810A6D5A828B7B86350F656D350299264A111C8C876D7502D634C6014F324773E64FFE56DA602736355061CAF39DD29706111ED10F00BD80EE4FE076F6BA743F210C9DC502493A8974FEC1E2706C094A20C29EA7DB29B37D3BD709CD6942FB8ED5EE3D1C521AFFA3A242A13A5A705C458027118C950E656794F671373CE3A7A2D53C0C4BCDDF539D530ABE4A554ED76F7D0D43C1DFFEA320D1CD14A2BC5A5B2C04D84C17399C590D94D86F3E85EC66693D364379149FB7E5709C45B7077D8784114D14B640EB8F50DECCE1A83B041CD217155918BFE937BC0C4E8BBA30452A2F00A4D5F2091DD12C8849EA4989EAC9BCB7040C0049C3986D4F5BF0F3D55FD05A847D3FC4A58F4F5DB860333B1C60F8AE4646919D339B0E4342FEE167A214B25D17D25BDDB7F81BCC37551435C033A66BCD3775FCAE15CA9F311E90FBFF2A082AC2A6B8B0C7F4F5865BFB3B18DEE1E184587C3BA746869D70402BCE1AFAC1AA45BF4D8A6A390F076CA1D6381F7581E2A5718BF44EFB75E9864EBF5EB011FD7C0892C8500BFA01B422E4B67222C259A263863E66CC26C533D0796C0C17D4945D9E0ADDD961EAC0FC3E594B14C13B9A586B0EBF961A17 -20180607164733 2 6 100 7167 5 E76114725DD41FA35A7DA1CF2A6C5701F689FC40E96BDAB540431472B92D5E38711F596734C76D1407BC11D5EECD10C52AE09D6EB46C7A119FE7BF2CE718363E5D9D06D972A80D1AEBE550FEF87C596C7B2FAB2FA6A27BE1D6CBFBD7A8B55FFBEFC78E6028AB4F83093162C4D8F9C4E72D45AEE3F788A1AA98A93B2E92DCC69739C3311002388723ED096B0FECB0698AB631108A2F5922453FA81DD178DDFB7A8E651E19E086F36684EF4958EF57E18393C173D07B594AEA0EAF5FE770F72550528C1637E341715CC80C74A2024CDEB2F949200E29BBFC0519F029CA2D45C4A66602553EF9A9086183018A055DFE19698B779908B119CB4C9E162680CA5E3EA471F4AF46F77DA5FF2A2C23C18509B1E8D0591B868A76163079461A83FBDAC95A5E8E4EB9152EAFCDFAD9CE5CDA7CE7846BEE134D52242152C4B671D1B2A05B927F6546A272DDEC50DDAE2D66B866C393F7CA3E2FF505FCC5187394BEEA2BC94F840B800884A8853A6A9AF1FB62837DDE066CC17EDDAD6921F0D96A7A9C2C1705CC024050201E93AF587FE1A480EDA8DCC257B0F7FF6A3006613250098090436BC3C63039846C8A2FD4180BB744D8584184C18F7B05BB110FC4D1C8C1D019EB072FC5C55DDBAE8272CAC153810A6D5A828B7B86350F656D350299264A111C8C876D7502D634C6014F324773E64FFE56DA602736355061CAF39DD29706111ED10F00BD80EE4FE076F6BA743F210C9DC502493A8974FEC1E2706C094A20C29EA7DB29B37D3BD709CD6942FB8ED5EE3D1C521AFFA3A242A13A5A705C458027118C950E656794F671373CE3A7A2D53C0C4BCDDF539D530ABE4A554ED76F7D0D43C1DFFEA320D1CD14A2BC5A5B2C04D84C17399C590D94D86F3E85EC66693D364379149FB7E5709C45B7077D8784114D14B640EB8F50DECCE1A83B041CD217155918BFE937BC0C4E8BBA30452A2F00A4D5F2091DD12C8849EA4989EAC9BCB7040C0049C3986D4F5BF0F3D55FD05A847D3FC4A58F4F5DB860333B1C60F8AE4646919D339B0E4342FEE167A214B25D17D25BDDB7F81BCC37551435C033A66BCD3775FCAE15CA9F311E90FBFF2A082AC2A6B8B0C7F4F5865BFB3B18DEE1E184587C3BA746869D70402BCE1AFAC1AA45BF4D8A6A390F076CA1D6381F7581E2A5718BF44EFB75E9864EBF5EB011FD7C0892C8500BFA01B422E4B67222C259A263863E66CC26C533D0796C0C17D4945D9E0ADDD961EAC0FC3E594B14C13B9A586B0EC03FE5D7 -20180607170431 2 6 100 7167 2 E76114725DD41FA35A7DA1CF2A6C5701F689FC40E96BDAB540431472B92D5E38711F596734C76D1407BC11D5EECD10C52AE09D6EB46C7A119FE7BF2CE718363E5D9D06D972A80D1AEBE550FEF87C596C7B2FAB2FA6A27BE1D6CBFBD7A8B55FFBEFC78E6028AB4F83093162C4D8F9C4E72D45AEE3F788A1AA98A93B2E92DCC69739C3311002388723ED096B0FECB0698AB631108A2F5922453FA81DD178DDFB7A8E651E19E086F36684EF4958EF57E18393C173D07B594AEA0EAF5FE770F72550528C1637E341715CC80C74A2024CDEB2F949200E29BBFC0519F029CA2D45C4A66602553EF9A9086183018A055DFE19698B779908B119CB4C9E162680CA5E3EA471F4AF46F77DA5FF2A2C23C18509B1E8D0591B868A76163079461A83FBDAC95A5E8E4EB9152EAFCDFAD9CE5CDA7CE7846BEE134D52242152C4B671D1B2A05B927F6546A272DDEC50DDAE2D66B866C393F7CA3E2FF505FCC5187394BEEA2BC94F840B800884A8853A6A9AF1FB62837DDE066CC17EDDAD6921F0D96A7A9C2C1705CC024050201E93AF587FE1A480EDA8DCC257B0F7FF6A3006613250098090436BC3C63039846C8A2FD4180BB744D8584184C18F7B05BB110FC4D1C8C1D019EB072FC5C55DDBAE8272CAC153810A6D5A828B7B86350F656D350299264A111C8C876D7502D634C6014F324773E64FFE56DA602736355061CAF39DD29706111ED10F00BD80EE4FE076F6BA743F210C9DC502493A8974FEC1E2706C094A20C29EA7DB29B37D3BD709CD6942FB8ED5EE3D1C521AFFA3A242A13A5A705C458027118C950E656794F671373CE3A7A2D53C0C4BCDDF539D530ABE4A554ED76F7D0D43C1DFFEA320D1CD14A2BC5A5B2C04D84C17399C590D94D86F3E85EC66693D364379149FB7E5709C45B7077D8784114D14B640EB8F50DECCE1A83B041CD217155918BFE937BC0C4E8BBA30452A2F00A4D5F2091DD12C8849EA4989EAC9BCB7040C0049C3986D4F5BF0F3D55FD05A847D3FC4A58F4F5DB860333B1C60F8AE4646919D339B0E4342FEE167A214B25D17D25BDDB7F81BCC37551435C033A66BCD3775FCAE15CA9F311E90FBFF2A082AC2A6B8B0C7F4F5865BFB3B18DEE1E184587C3BA746869D70402BCE1AFAC1AA45BF4D8A6A390F076CA1D6381F7581E2A5718BF44EFB75E9864EBF5EB011FD7C0892C8500BFA01B422E4B67222C259A263863E66CC26C533D0796C0C17D4945D9E0ADDD961EAC0FC3E594B14C13B9A586B0EC0E41B93 -20180607210523 2 6 100 7167 2 E76114725DD41FA35A7DA1CF2A6C5701F689FC40E96BDAB540431472B92D5E38711F596734C76D1407BC11D5EECD10C52AE09D6EB46C7A119FE7BF2CE718363E5D9D06D972A80D1AEBE550FEF87C596C7B2FAB2FA6A27BE1D6CBFBD7A8B55FFBEFC78E6028AB4F83093162C4D8F9C4E72D45AEE3F788A1AA98A93B2E92DCC69739C3311002388723ED096B0FECB0698AB631108A2F5922453FA81DD178DDFB7A8E651E19E086F36684EF4958EF57E18393C173D07B594AEA0EAF5FE770F72550528C1637E341715CC80C74A2024CDEB2F949200E29BBFC0519F029CA2D45C4A66602553EF9A9086183018A055DFE19698B779908B119CB4C9E162680CA5E3EA471F4AF46F77DA5FF2A2C23C18509B1E8D0591B868A76163079461A83FBDAC95A5E8E4EB9152EAFCDFAD9CE5CDA7CE7846BEE134D52242152C4B671D1B2A05B927F6546A272DDEC50DDAE2D66B866C393F7CA3E2FF505FCC5187394BEEA2BC94F840B800884A8853A6A9AF1FB62837DDE066CC17EDDAD6921F0D96A7A9C2C1705CC024050201E93AF587FE1A480EDA8DCC257B0F7FF6A3006613250098090436BC3C63039846C8A2FD4180BB744D8584184C18F7B05BB110FC4D1C8C1D019EB072FC5C55DDBAE8272CAC153810A6D5A828B7B86350F656D350299264A111C8C876D7502D634C6014F324773E64FFE56DA602736355061CAF39DD29706111ED10F00BD80EE4FE076F6BA743F210C9DC502493A8974FEC1E2706C094A20C29EA7DB29B37D3BD709CD6942FB8ED5EE3D1C521AFFA3A242A13A5A705C458027118C950E656794F671373CE3A7A2D53C0C4BCDDF539D530ABE4A554ED76F7D0D43C1DFFEA320D1CD14A2BC5A5B2C04D84C17399C590D94D86F3E85EC66693D364379149FB7E5709C45B7077D8784114D14B640EB8F50DECCE1A83B041CD217155918BFE937BC0C4E8BBA30452A2F00A4D5F2091DD12C8849EA4989EAC9BCB7040C0049C3986D4F5BF0F3D55FD05A847D3FC4A58F4F5DB860333B1C60F8AE4646919D339B0E4342FEE167A214B25D17D25BDDB7F81BCC37551435C033A66BCD3775FCAE15CA9F311E90FBFF2A082AC2A6B8B0C7F4F5865BFB3B18DEE1E184587C3BA746869D70402BCE1AFAC1AA45BF4D8A6A390F076CA1D6381F7581E2A5718BF44EFB75E9864EBF5EB011FD7C0892C8500BFA01B422E4B67222C259A263863E66CC26C533D0796C0C17D4945D9E0ADDD961EAC0FC3E594B14C13B9A586B0ECA453803 -20180608000228 2 6 100 7167 2 E76114725DD41FA35A7DA1CF2A6C5701F689FC40E96BDAB540431472B92D5E38711F596734C76D1407BC11D5EECD10C52AE09D6EB46C7A119FE7BF2CE718363E5D9D06D972A80D1AEBE550FEF87C596C7B2FAB2FA6A27BE1D6CBFBD7A8B55FFBEFC78E6028AB4F83093162C4D8F9C4E72D45AEE3F788A1AA98A93B2E92DCC69739C3311002388723ED096B0FECB0698AB631108A2F5922453FA81DD178DDFB7A8E651E19E086F36684EF4958EF57E18393C173D07B594AEA0EAF5FE770F72550528C1637E341715CC80C74A2024CDEB2F949200E29BBFC0519F029CA2D45C4A66602553EF9A9086183018A055DFE19698B779908B119CB4C9E162680CA5E3EA471F4AF46F77DA5FF2A2C23C18509B1E8D0591B868A76163079461A83FBDAC95A5E8E4EB9152EAFCDFAD9CE5CDA7CE7846BEE134D52242152C4B671D1B2A05B927F6546A272DDEC50DDAE2D66B866C393F7CA3E2FF505FCC5187394BEEA2BC94F840B800884A8853A6A9AF1FB62837DDE066CC17EDDAD6921F0D96A7A9C2C1705CC024050201E93AF587FE1A480EDA8DCC257B0F7FF6A3006613250098090436BC3C63039846C8A2FD4180BB744D8584184C18F7B05BB110FC4D1C8C1D019EB072FC5C55DDBAE8272CAC153810A6D5A828B7B86350F656D350299264A111C8C876D7502D634C6014F324773E64FFE56DA602736355061CAF39DD29706111ED10F00BD80EE4FE076F6BA743F210C9DC502493A8974FEC1E2706C094A20C29EA7DB29B37D3BD709CD6942FB8ED5EE3D1C521AFFA3A242A13A5A705C458027118C950E656794F671373CE3A7A2D53C0C4BCDDF539D530ABE4A554ED76F7D0D43C1DFFEA320D1CD14A2BC5A5B2C04D84C17399C590D94D86F3E85EC66693D364379149FB7E5709C45B7077D8784114D14B640EB8F50DECCE1A83B041CD217155918BFE937BC0C4E8BBA30452A2F00A4D5F2091DD12C8849EA4989EAC9BCB7040C0049C3986D4F5BF0F3D55FD05A847D3FC4A58F4F5DB860333B1C60F8AE4646919D339B0E4342FEE167A214B25D17D25BDDB7F81BCC37551435C033A66BCD3775FCAE15CA9F311E90FBFF2A082AC2A6B8B0C7F4F5865BFB3B18DEE1E184587C3BA746869D70402BCE1AFAC1AA45BF4D8A6A390F076CA1D6381F7581E2A5718BF44EFB75E9864EBF5EB011FD7C0892C8500BFA01B422E4B67222C259A263863E66CC26C533D0796C0C17D4945D9E0ADDD961EAC0FC3E594B14C13B9A586B0ED1125EF3 -20180608010403 2 6 100 7167 2 E76114725DD41FA35A7DA1CF2A6C5701F689FC40E96BDAB540431472B92D5E38711F596734C76D1407BC11D5EECD10C52AE09D6EB46C7A119FE7BF2CE718363E5D9D06D972A80D1AEBE550FEF87C596C7B2FAB2FA6A27BE1D6CBFBD7A8B55FFBEFC78E6028AB4F83093162C4D8F9C4E72D45AEE3F788A1AA98A93B2E92DCC69739C3311002388723ED096B0FECB0698AB631108A2F5922453FA81DD178DDFB7A8E651E19E086F36684EF4958EF57E18393C173D07B594AEA0EAF5FE770F72550528C1637E341715CC80C74A2024CDEB2F949200E29BBFC0519F029CA2D45C4A66602553EF9A9086183018A055DFE19698B779908B119CB4C9E162680CA5E3EA471F4AF46F77DA5FF2A2C23C18509B1E8D0591B868A76163079461A83FBDAC95A5E8E4EB9152EAFCDFAD9CE5CDA7CE7846BEE134D52242152C4B671D1B2A05B927F6546A272DDEC50DDAE2D66B866C393F7CA3E2FF505FCC5187394BEEA2BC94F840B800884A8853A6A9AF1FB62837DDE066CC17EDDAD6921F0D96A7A9C2C1705CC024050201E93AF587FE1A480EDA8DCC257B0F7FF6A3006613250098090436BC3C63039846C8A2FD4180BB744D8584184C18F7B05BB110FC4D1C8C1D019EB072FC5C55DDBAE8272CAC153810A6D5A828B7B86350F656D350299264A111C8C876D7502D634C6014F324773E64FFE56DA602736355061CAF39DD29706111ED10F00BD80EE4FE076F6BA743F210C9DC502493A8974FEC1E2706C094A20C29EA7DB29B37D3BD709CD6942FB8ED5EE3D1C521AFFA3A242A13A5A705C458027118C950E656794F671373CE3A7A2D53C0C4BCDDF539D530ABE4A554ED76F7D0D43C1DFFEA320D1CD14A2BC5A5B2C04D84C17399C590D94D86F3E85EC66693D364379149FB7E5709C45B7077D8784114D14B640EB8F50DECCE1A83B041CD217155918BFE937BC0C4E8BBA30452A2F00A4D5F2091DD12C8849EA4989EAC9BCB7040C0049C3986D4F5BF0F3D55FD05A847D3FC4A58F4F5DB860333B1C60F8AE4646919D339B0E4342FEE167A214B25D17D25BDDB7F81BCC37551435C033A66BCD3775FCAE15CA9F311E90FBFF2A082AC2A6B8B0C7F4F5865BFB3B18DEE1E184587C3BA746869D70402BCE1AFAC1AA45BF4D8A6A390F076CA1D6381F7581E2A5718BF44EFB75E9864EBF5EB011FD7C0892C8500BFA01B422E4B67222C259A263863E66CC26C533D0796C0C17D4945D9E0ADDD961EAC0FC3E594B14C13B9A586B0ED3657E33 -20180608015201 2 6 100 7167 2 E76114725DD41FA35A7DA1CF2A6C5701F689FC40E96BDAB540431472B92D5E38711F596734C76D1407BC11D5EECD10C52AE09D6EB46C7A119FE7BF2CE718363E5D9D06D972A80D1AEBE550FEF87C596C7B2FAB2FA6A27BE1D6CBFBD7A8B55FFBEFC78E6028AB4F83093162C4D8F9C4E72D45AEE3F788A1AA98A93B2E92DCC69739C3311002388723ED096B0FECB0698AB631108A2F5922453FA81DD178DDFB7A8E651E19E086F36684EF4958EF57E18393C173D07B594AEA0EAF5FE770F72550528C1637E341715CC80C74A2024CDEB2F949200E29BBFC0519F029CA2D45C4A66602553EF9A9086183018A055DFE19698B779908B119CB4C9E162680CA5E3EA471F4AF46F77DA5FF2A2C23C18509B1E8D0591B868A76163079461A83FBDAC95A5E8E4EB9152EAFCDFAD9CE5CDA7CE7846BEE134D52242152C4B671D1B2A05B927F6546A272DDEC50DDAE2D66B866C393F7CA3E2FF505FCC5187394BEEA2BC94F840B800884A8853A6A9AF1FB62837DDE066CC17EDDAD6921F0D96A7A9C2C1705CC024050201E93AF587FE1A480EDA8DCC257B0F7FF6A3006613250098090436BC3C63039846C8A2FD4180BB744D8584184C18F7B05BB110FC4D1C8C1D019EB072FC5C55DDBAE8272CAC153810A6D5A828B7B86350F656D350299264A111C8C876D7502D634C6014F324773E64FFE56DA602736355061CAF39DD29706111ED10F00BD80EE4FE076F6BA743F210C9DC502493A8974FEC1E2706C094A20C29EA7DB29B37D3BD709CD6942FB8ED5EE3D1C521AFFA3A242A13A5A705C458027118C950E656794F671373CE3A7A2D53C0C4BCDDF539D530ABE4A554ED76F7D0D43C1DFFEA320D1CD14A2BC5A5B2C04D84C17399C590D94D86F3E85EC66693D364379149FB7E5709C45B7077D8784114D14B640EB8F50DECCE1A83B041CD217155918BFE937BC0C4E8BBA30452A2F00A4D5F2091DD12C8849EA4989EAC9BCB7040C0049C3986D4F5BF0F3D55FD05A847D3FC4A58F4F5DB860333B1C60F8AE4646919D339B0E4342FEE167A214B25D17D25BDDB7F81BCC37551435C033A66BCD3775FCAE15CA9F311E90FBFF2A082AC2A6B8B0C7F4F5865BFB3B18DEE1E184587C3BA746869D70402BCE1AFAC1AA45BF4D8A6A390F076CA1D6381F7581E2A5718BF44EFB75E9864EBF5EB011FD7C0892C8500BFA01B422E4B67222C259A263863E66CC26C533D0796C0C17D4945D9E0ADDD961EAC0FC3E594B14C13B9A586B0ED537E7FB -20180608023333 2 6 100 7167 2 E76114725DD41FA35A7DA1CF2A6C5701F689FC40E96BDAB540431472B92D5E38711F596734C76D1407BC11D5EECD10C52AE09D6EB46C7A119FE7BF2CE718363E5D9D06D972A80D1AEBE550FEF87C596C7B2FAB2FA6A27BE1D6CBFBD7A8B55FFBEFC78E6028AB4F83093162C4D8F9C4E72D45AEE3F788A1AA98A93B2E92DCC69739C3311002388723ED096B0FECB0698AB631108A2F5922453FA81DD178DDFB7A8E651E19E086F36684EF4958EF57E18393C173D07B594AEA0EAF5FE770F72550528C1637E341715CC80C74A2024CDEB2F949200E29BBFC0519F029CA2D45C4A66602553EF9A9086183018A055DFE19698B779908B119CB4C9E162680CA5E3EA471F4AF46F77DA5FF2A2C23C18509B1E8D0591B868A76163079461A83FBDAC95A5E8E4EB9152EAFCDFAD9CE5CDA7CE7846BEE134D52242152C4B671D1B2A05B927F6546A272DDEC50DDAE2D66B866C393F7CA3E2FF505FCC5187394BEEA2BC94F840B800884A8853A6A9AF1FB62837DDE066CC17EDDAD6921F0D96A7A9C2C1705CC024050201E93AF587FE1A480EDA8DCC257B0F7FF6A3006613250098090436BC3C63039846C8A2FD4180BB744D8584184C18F7B05BB110FC4D1C8C1D019EB072FC5C55DDBAE8272CAC153810A6D5A828B7B86350F656D350299264A111C8C876D7502D634C6014F324773E64FFE56DA602736355061CAF39DD29706111ED10F00BD80EE4FE076F6BA743F210C9DC502493A8974FEC1E2706C094A20C29EA7DB29B37D3BD709CD6942FB8ED5EE3D1C521AFFA3A242A13A5A705C458027118C950E656794F671373CE3A7A2D53C0C4BCDDF539D530ABE4A554ED76F7D0D43C1DFFEA320D1CD14A2BC5A5B2C04D84C17399C590D94D86F3E85EC66693D364379149FB7E5709C45B7077D8784114D14B640EB8F50DECCE1A83B041CD217155918BFE937BC0C4E8BBA30452A2F00A4D5F2091DD12C8849EA4989EAC9BCB7040C0049C3986D4F5BF0F3D55FD05A847D3FC4A58F4F5DB860333B1C60F8AE4646919D339B0E4342FEE167A214B25D17D25BDDB7F81BCC37551435C033A66BCD3775FCAE15CA9F311E90FBFF2A082AC2A6B8B0C7F4F5865BFB3B18DEE1E184587C3BA746869D70402BCE1AFAC1AA45BF4D8A6A390F076CA1D6381F7581E2A5718BF44EFB75E9864EBF5EB011FD7C0892C8500BFA01B422E4B67222C259A263863E66CC26C533D0796C0C17D4945D9E0ADDD961EAC0FC3E594B14C13B9A586B0ED6C3867B -20180608023523 2 6 100 7167 5 E76114725DD41FA35A7DA1CF2A6C5701F689FC40E96BDAB540431472B92D5E38711F596734C76D1407BC11D5EECD10C52AE09D6EB46C7A119FE7BF2CE718363E5D9D06D972A80D1AEBE550FEF87C596C7B2FAB2FA6A27BE1D6CBFBD7A8B55FFBEFC78E6028AB4F83093162C4D8F9C4E72D45AEE3F788A1AA98A93B2E92DCC69739C3311002388723ED096B0FECB0698AB631108A2F5922453FA81DD178DDFB7A8E651E19E086F36684EF4958EF57E18393C173D07B594AEA0EAF5FE770F72550528C1637E341715CC80C74A2024CDEB2F949200E29BBFC0519F029CA2D45C4A66602553EF9A9086183018A055DFE19698B779908B119CB4C9E162680CA5E3EA471F4AF46F77DA5FF2A2C23C18509B1E8D0591B868A76163079461A83FBDAC95A5E8E4EB9152EAFCDFAD9CE5CDA7CE7846BEE134D52242152C4B671D1B2A05B927F6546A272DDEC50DDAE2D66B866C393F7CA3E2FF505FCC5187394BEEA2BC94F840B800884A8853A6A9AF1FB62837DDE066CC17EDDAD6921F0D96A7A9C2C1705CC024050201E93AF587FE1A480EDA8DCC257B0F7FF6A3006613250098090436BC3C63039846C8A2FD4180BB744D8584184C18F7B05BB110FC4D1C8C1D019EB072FC5C55DDBAE8272CAC153810A6D5A828B7B86350F656D350299264A111C8C876D7502D634C6014F324773E64FFE56DA602736355061CAF39DD29706111ED10F00BD80EE4FE076F6BA743F210C9DC502493A8974FEC1E2706C094A20C29EA7DB29B37D3BD709CD6942FB8ED5EE3D1C521AFFA3A242A13A5A705C458027118C950E656794F671373CE3A7A2D53C0C4BCDDF539D530ABE4A554ED76F7D0D43C1DFFEA320D1CD14A2BC5A5B2C04D84C17399C590D94D86F3E85EC66693D364379149FB7E5709C45B7077D8784114D14B640EB8F50DECCE1A83B041CD217155918BFE937BC0C4E8BBA30452A2F00A4D5F2091DD12C8849EA4989EAC9BCB7040C0049C3986D4F5BF0F3D55FD05A847D3FC4A58F4F5DB860333B1C60F8AE4646919D339B0E4342FEE167A214B25D17D25BDDB7F81BCC37551435C033A66BCD3775FCAE15CA9F311E90FBFF2A082AC2A6B8B0C7F4F5865BFB3B18DEE1E184587C3BA746869D70402BCE1AFAC1AA45BF4D8A6A390F076CA1D6381F7581E2A5718BF44EFB75E9864EBF5EB011FD7C0892C8500BFA01B422E4B67222C259A263863E66CC26C533D0796C0C17D4945D9E0ADDD961EAC0FC3E594B14C13B9A586B0ED6CC5597 -20180608025418 2 6 100 7167 2 E76114725DD41FA35A7DA1CF2A6C5701F689FC40E96BDAB540431472B92D5E38711F596734C76D1407BC11D5EECD10C52AE09D6EB46C7A119FE7BF2CE718363E5D9D06D972A80D1AEBE550FEF87C596C7B2FAB2FA6A27BE1D6CBFBD7A8B55FFBEFC78E6028AB4F83093162C4D8F9C4E72D45AEE3F788A1AA98A93B2E92DCC69739C3311002388723ED096B0FECB0698AB631108A2F5922453FA81DD178DDFB7A8E651E19E086F36684EF4958EF57E18393C173D07B594AEA0EAF5FE770F72550528C1637E341715CC80C74A2024CDEB2F949200E29BBFC0519F029CA2D45C4A66602553EF9A9086183018A055DFE19698B779908B119CB4C9E162680CA5E3EA471F4AF46F77DA5FF2A2C23C18509B1E8D0591B868A76163079461A83FBDAC95A5E8E4EB9152EAFCDFAD9CE5CDA7CE7846BEE134D52242152C4B671D1B2A05B927F6546A272DDEC50DDAE2D66B866C393F7CA3E2FF505FCC5187394BEEA2BC94F840B800884A8853A6A9AF1FB62837DDE066CC17EDDAD6921F0D96A7A9C2C1705CC024050201E93AF587FE1A480EDA8DCC257B0F7FF6A3006613250098090436BC3C63039846C8A2FD4180BB744D8584184C18F7B05BB110FC4D1C8C1D019EB072FC5C55DDBAE8272CAC153810A6D5A828B7B86350F656D350299264A111C8C876D7502D634C6014F324773E64FFE56DA602736355061CAF39DD29706111ED10F00BD80EE4FE076F6BA743F210C9DC502493A8974FEC1E2706C094A20C29EA7DB29B37D3BD709CD6942FB8ED5EE3D1C521AFFA3A242A13A5A705C458027118C950E656794F671373CE3A7A2D53C0C4BCDDF539D530ABE4A554ED76F7D0D43C1DFFEA320D1CD14A2BC5A5B2C04D84C17399C590D94D86F3E85EC66693D364379149FB7E5709C45B7077D8784114D14B640EB8F50DECCE1A83B041CD217155918BFE937BC0C4E8BBA30452A2F00A4D5F2091DD12C8849EA4989EAC9BCB7040C0049C3986D4F5BF0F3D55FD05A847D3FC4A58F4F5DB860333B1C60F8AE4646919D339B0E4342FEE167A214B25D17D25BDDB7F81BCC37551435C033A66BCD3775FCAE15CA9F311E90FBFF2A082AC2A6B8B0C7F4F5865BFB3B18DEE1E184587C3BA746869D70402BCE1AFAC1AA45BF4D8A6A390F076CA1D6381F7581E2A5718BF44EFB75E9864EBF5EB011FD7C0892C8500BFA01B422E4B67222C259A263863E66CC26C533D0796C0C17D4945D9E0ADDD961EAC0FC3E594B14C13B9A586B0ED77890BB -20180608052209 2 6 100 7167 2 E76114725DD41FA35A7DA1CF2A6C5701F689FC40E96BDAB540431472B92D5E38711F596734C76D1407BC11D5EECD10C52AE09D6EB46C7A119FE7BF2CE718363E5D9D06D972A80D1AEBE550FEF87C596C7B2FAB2FA6A27BE1D6CBFBD7A8B55FFBEFC78E6028AB4F83093162C4D8F9C4E72D45AEE3F788A1AA98A93B2E92DCC69739C3311002388723ED096B0FECB0698AB631108A2F5922453FA81DD178DDFB7A8E651E19E086F36684EF4958EF57E18393C173D07B594AEA0EAF5FE770F72550528C1637E341715CC80C74A2024CDEB2F949200E29BBFC0519F029CA2D45C4A66602553EF9A9086183018A055DFE19698B779908B119CB4C9E162680CA5E3EA471F4AF46F77DA5FF2A2C23C18509B1E8D0591B868A76163079461A83FBDAC95A5E8E4EB9152EAFCDFAD9CE5CDA7CE7846BEE134D52242152C4B671D1B2A05B927F6546A272DDEC50DDAE2D66B866C393F7CA3E2FF505FCC5187394BEEA2BC94F840B800884A8853A6A9AF1FB62837DDE066CC17EDDAD6921F0D96A7A9C2C1705CC024050201E93AF587FE1A480EDA8DCC257B0F7FF6A3006613250098090436BC3C63039846C8A2FD4180BB744D8584184C18F7B05BB110FC4D1C8C1D019EB072FC5C55DDBAE8272CAC153810A6D5A828B7B86350F656D350299264A111C8C876D7502D634C6014F324773E64FFE56DA602736355061CAF39DD29706111ED10F00BD80EE4FE076F6BA743F210C9DC502493A8974FEC1E2706C094A20C29EA7DB29B37D3BD709CD6942FB8ED5EE3D1C521AFFA3A242A13A5A705C458027118C950E656794F671373CE3A7A2D53C0C4BCDDF539D530ABE4A554ED76F7D0D43C1DFFEA320D1CD14A2BC5A5B2C04D84C17399C590D94D86F3E85EC66693D364379149FB7E5709C45B7077D8784114D14B640EB8F50DECCE1A83B041CD217155918BFE937BC0C4E8BBA30452A2F00A4D5F2091DD12C8849EA4989EAC9BCB7040C0049C3986D4F5BF0F3D55FD05A847D3FC4A58F4F5DB860333B1C60F8AE4646919D339B0E4342FEE167A214B25D17D25BDDB7F81BCC37551435C033A66BCD3775FCAE15CA9F311E90FBFF2A082AC2A6B8B0C7F4F5865BFB3B18DEE1E184587C3BA746869D70402BCE1AFAC1AA45BF4D8A6A390F076CA1D6381F7581E2A5718BF44EFB75E9864EBF5EB011FD7C0892C8500BFA01B422E4B67222C259A263863E66CC26C533D0796C0C17D4945D9E0ADDD961EAC0FC3E594B14C13B9A586B0EDD073603 -20180608064909 2 6 100 7167 2 E76114725DD41FA35A7DA1CF2A6C5701F689FC40E96BDAB540431472B92D5E38711F596734C76D1407BC11D5EECD10C52AE09D6EB46C7A119FE7BF2CE718363E5D9D06D972A80D1AEBE550FEF87C596C7B2FAB2FA6A27BE1D6CBFBD7A8B55FFBEFC78E6028AB4F83093162C4D8F9C4E72D45AEE3F788A1AA98A93B2E92DCC69739C3311002388723ED096B0FECB0698AB631108A2F5922453FA81DD178DDFB7A8E651E19E086F36684EF4958EF57E18393C173D07B594AEA0EAF5FE770F72550528C1637E341715CC80C74A2024CDEB2F949200E29BBFC0519F029CA2D45C4A66602553EF9A9086183018A055DFE19698B779908B119CB4C9E162680CA5E3EA471F4AF46F77DA5FF2A2C23C18509B1E8D0591B868A76163079461A83FBDAC95A5E8E4EB9152EAFCDFAD9CE5CDA7CE7846BEE134D52242152C4B671D1B2A05B927F6546A272DDEC50DDAE2D66B866C393F7CA3E2FF505FCC5187394BEEA2BC94F840B800884A8853A6A9AF1FB62837DDE066CC17EDDAD6921F0D96A7A9C2C1705CC024050201E93AF587FE1A480EDA8DCC257B0F7FF6A3006613250098090436BC3C63039846C8A2FD4180BB744D8584184C18F7B05BB110FC4D1C8C1D019EB072FC5C55DDBAE8272CAC153810A6D5A828B7B86350F656D350299264A111C8C876D7502D634C6014F324773E64FFE56DA602736355061CAF39DD29706111ED10F00BD80EE4FE076F6BA743F210C9DC502493A8974FEC1E2706C094A20C29EA7DB29B37D3BD709CD6942FB8ED5EE3D1C521AFFA3A242A13A5A705C458027118C950E656794F671373CE3A7A2D53C0C4BCDDF539D530ABE4A554ED76F7D0D43C1DFFEA320D1CD14A2BC5A5B2C04D84C17399C590D94D86F3E85EC66693D364379149FB7E5709C45B7077D8784114D14B640EB8F50DECCE1A83B041CD217155918BFE937BC0C4E8BBA30452A2F00A4D5F2091DD12C8849EA4989EAC9BCB7040C0049C3986D4F5BF0F3D55FD05A847D3FC4A58F4F5DB860333B1C60F8AE4646919D339B0E4342FEE167A214B25D17D25BDDB7F81BCC37551435C033A66BCD3775FCAE15CA9F311E90FBFF2A082AC2A6B8B0C7F4F5865BFB3B18DEE1E184587C3BA746869D70402BCE1AFAC1AA45BF4D8A6A390F076CA1D6381F7581E2A5718BF44EFB75E9864EBF5EB011FD7C0892C8500BFA01B422E4B67222C259A263863E66CC26C533D0796C0C17D4945D9E0ADDD961EAC0FC3E594B14C13B9A586B0EE0434A13 -20180608083049 2 6 100 7167 2 E76114725DD41FA35A7DA1CF2A6C5701F689FC40E96BDAB540431472B92D5E38711F596734C76D1407BC11D5EECD10C52AE09D6EB46C7A119FE7BF2CE718363E5D9D06D972A80D1AEBE550FEF87C596C7B2FAB2FA6A27BE1D6CBFBD7A8B55FFBEFC78E6028AB4F83093162C4D8F9C4E72D45AEE3F788A1AA98A93B2E92DCC69739C3311002388723ED096B0FECB0698AB631108A2F5922453FA81DD178DDFB7A8E651E19E086F36684EF4958EF57E18393C173D07B594AEA0EAF5FE770F72550528C1637E341715CC80C74A2024CDEB2F949200E29BBFC0519F029CA2D45C4A66602553EF9A9086183018A055DFE19698B779908B119CB4C9E162680CA5E3EA471F4AF46F77DA5FF2A2C23C18509B1E8D0591B868A76163079461A83FBDAC95A5E8E4EB9152EAFCDFAD9CE5CDA7CE7846BEE134D52242152C4B671D1B2A05B927F6546A272DDEC50DDAE2D66B866C393F7CA3E2FF505FCC5187394BEEA2BC94F840B800884A8853A6A9AF1FB62837DDE066CC17EDDAD6921F0D96A7A9C2C1705CC024050201E93AF587FE1A480EDA8DCC257B0F7FF6A3006613250098090436BC3C63039846C8A2FD4180BB744D8584184C18F7B05BB110FC4D1C8C1D019EB072FC5C55DDBAE8272CAC153810A6D5A828B7B86350F656D350299264A111C8C876D7502D634C6014F324773E64FFE56DA602736355061CAF39DD29706111ED10F00BD80EE4FE076F6BA743F210C9DC502493A8974FEC1E2706C094A20C29EA7DB29B37D3BD709CD6942FB8ED5EE3D1C521AFFA3A242A13A5A705C458027118C950E656794F671373CE3A7A2D53C0C4BCDDF539D530ABE4A554ED76F7D0D43C1DFFEA320D1CD14A2BC5A5B2C04D84C17399C590D94D86F3E85EC66693D364379149FB7E5709C45B7077D8784114D14B640EB8F50DECCE1A83B041CD217155918BFE937BC0C4E8BBA30452A2F00A4D5F2091DD12C8849EA4989EAC9BCB7040C0049C3986D4F5BF0F3D55FD05A847D3FC4A58F4F5DB860333B1C60F8AE4646919D339B0E4342FEE167A214B25D17D25BDDB7F81BCC37551435C033A66BCD3775FCAE15CA9F311E90FBFF2A082AC2A6B8B0C7F4F5865BFB3B18DEE1E184587C3BA746869D70402BCE1AFAC1AA45BF4D8A6A390F076CA1D6381F7581E2A5718BF44EFB75E9864EBF5EB011FD7C0892C8500BFA01B422E4B67222C259A263863E66CC26C533D0796C0C17D4945D9E0ADDD961EAC0FC3E594B14C13B9A586B0EE410EA0B -20180608084134 2 6 100 7167 2 E76114725DD41FA35A7DA1CF2A6C5701F689FC40E96BDAB540431472B92D5E38711F596734C76D1407BC11D5EECD10C52AE09D6EB46C7A119FE7BF2CE718363E5D9D06D972A80D1AEBE550FEF87C596C7B2FAB2FA6A27BE1D6CBFBD7A8B55FFBEFC78E6028AB4F83093162C4D8F9C4E72D45AEE3F788A1AA98A93B2E92DCC69739C3311002388723ED096B0FECB0698AB631108A2F5922453FA81DD178DDFB7A8E651E19E086F36684EF4958EF57E18393C173D07B594AEA0EAF5FE770F72550528C1637E341715CC80C74A2024CDEB2F949200E29BBFC0519F029CA2D45C4A66602553EF9A9086183018A055DFE19698B779908B119CB4C9E162680CA5E3EA471F4AF46F77DA5FF2A2C23C18509B1E8D0591B868A76163079461A83FBDAC95A5E8E4EB9152EAFCDFAD9CE5CDA7CE7846BEE134D52242152C4B671D1B2A05B927F6546A272DDEC50DDAE2D66B866C393F7CA3E2FF505FCC5187394BEEA2BC94F840B800884A8853A6A9AF1FB62837DDE066CC17EDDAD6921F0D96A7A9C2C1705CC024050201E93AF587FE1A480EDA8DCC257B0F7FF6A3006613250098090436BC3C63039846C8A2FD4180BB744D8584184C18F7B05BB110FC4D1C8C1D019EB072FC5C55DDBAE8272CAC153810A6D5A828B7B86350F656D350299264A111C8C876D7502D634C6014F324773E64FFE56DA602736355061CAF39DD29706111ED10F00BD80EE4FE076F6BA743F210C9DC502493A8974FEC1E2706C094A20C29EA7DB29B37D3BD709CD6942FB8ED5EE3D1C521AFFA3A242A13A5A705C458027118C950E656794F671373CE3A7A2D53C0C4BCDDF539D530ABE4A554ED76F7D0D43C1DFFEA320D1CD14A2BC5A5B2C04D84C17399C590D94D86F3E85EC66693D364379149FB7E5709C45B7077D8784114D14B640EB8F50DECCE1A83B041CD217155918BFE937BC0C4E8BBA30452A2F00A4D5F2091DD12C8849EA4989EAC9BCB7040C0049C3986D4F5BF0F3D55FD05A847D3FC4A58F4F5DB860333B1C60F8AE4646919D339B0E4342FEE167A214B25D17D25BDDB7F81BCC37551435C033A66BCD3775FCAE15CA9F311E90FBFF2A082AC2A6B8B0C7F4F5865BFB3B18DEE1E184587C3BA746869D70402BCE1AFAC1AA45BF4D8A6A390F076CA1D6381F7581E2A5718BF44EFB75E9864EBF5EB011FD7C0892C8500BFA01B422E4B67222C259A263863E66CC26C533D0796C0C17D4945D9E0ADDD961EAC0FC3E594B14C13B9A586B0EE46EFA03 -20180608113212 2 6 100 7167 2 E76114725DD41FA35A7DA1CF2A6C5701F689FC40E96BDAB540431472B92D5E38711F596734C76D1407BC11D5EECD10C52AE09D6EB46C7A119FE7BF2CE718363E5D9D06D972A80D1AEBE550FEF87C596C7B2FAB2FA6A27BE1D6CBFBD7A8B55FFBEFC78E6028AB4F83093162C4D8F9C4E72D45AEE3F788A1AA98A93B2E92DCC69739C3311002388723ED096B0FECB0698AB631108A2F5922453FA81DD178DDFB7A8E651E19E086F36684EF4958EF57E18393C173D07B594AEA0EAF5FE770F72550528C1637E341715CC80C74A2024CDEB2F949200E29BBFC0519F029CA2D45C4A66602553EF9A9086183018A055DFE19698B779908B119CB4C9E162680CA5E3EA471F4AF46F77DA5FF2A2C23C18509B1E8D0591B868A76163079461A83FBDAC95A5E8E4EB9152EAFCDFAD9CE5CDA7CE7846BEE134D52242152C4B671D1B2A05B927F6546A272DDEC50DDAE2D66B866C393F7CA3E2FF505FCC5187394BEEA2BC94F840B800884A8853A6A9AF1FB62837DDE066CC17EDDAD6921F0D96A7A9C2C1705CC024050201E93AF587FE1A480EDA8DCC257B0F7FF6A3006613250098090436BC3C63039846C8A2FD4180BB744D8584184C18F7B05BB110FC4D1C8C1D019EB072FC5C55DDBAE8272CAC153810A6D5A828B7B86350F656D350299264A111C8C876D7502D634C6014F324773E64FFE56DA602736355061CAF39DD29706111ED10F00BD80EE4FE076F6BA743F210C9DC502493A8974FEC1E2706C094A20C29EA7DB29B37D3BD709CD6942FB8ED5EE3D1C521AFFA3A242A13A5A705C458027118C950E656794F671373CE3A7A2D53C0C4BCDDF539D530ABE4A554ED76F7D0D43C1DFFEA320D1CD14A2BC5A5B2C04D84C17399C590D94D86F3E85EC66693D364379149FB7E5709C45B7077D8784114D14B640EB8F50DECCE1A83B041CD217155918BFE937BC0C4E8BBA30452A2F00A4D5F2091DD12C8849EA4989EAC9BCB7040C0049C3986D4F5BF0F3D55FD05A847D3FC4A58F4F5DB860333B1C60F8AE4646919D339B0E4342FEE167A214B25D17D25BDDB7F81BCC37551435C033A66BCD3775FCAE15CA9F311E90FBFF2A082AC2A6B8B0C7F4F5865BFB3B18DEE1E184587C3BA746869D70402BCE1AFAC1AA45BF4D8A6A390F076CA1D6381F7581E2A5718BF44EFB75E9864EBF5EB011FD7C0892C8500BFA01B422E4B67222C259A263863E66CC26C533D0796C0C17D4945D9E0ADDD961EAC0FC3E594B14C13B9A586B0EEAB7B3A3 -20180608123538 2 6 100 7167 5 E76114725DD41FA35A7DA1CF2A6C5701F689FC40E96BDAB540431472B92D5E38711F596734C76D1407BC11D5EECD10C52AE09D6EB46C7A119FE7BF2CE718363E5D9D06D972A80D1AEBE550FEF87C596C7B2FAB2FA6A27BE1D6CBFBD7A8B55FFBEFC78E6028AB4F83093162C4D8F9C4E72D45AEE3F788A1AA98A93B2E92DCC69739C3311002388723ED096B0FECB0698AB631108A2F5922453FA81DD178DDFB7A8E651E19E086F36684EF4958EF57E18393C173D07B594AEA0EAF5FE770F72550528C1637E341715CC80C74A2024CDEB2F949200E29BBFC0519F029CA2D45C4A66602553EF9A9086183018A055DFE19698B779908B119CB4C9E162680CA5E3EA471F4AF46F77DA5FF2A2C23C18509B1E8D0591B868A76163079461A83FBDAC95A5E8E4EB9152EAFCDFAD9CE5CDA7CE7846BEE134D52242152C4B671D1B2A05B927F6546A272DDEC50DDAE2D66B866C393F7CA3E2FF505FCC5187394BEEA2BC94F840B800884A8853A6A9AF1FB62837DDE066CC17EDDAD6921F0D96A7A9C2C1705CC024050201E93AF587FE1A480EDA8DCC257B0F7FF6A3006613250098090436BC3C63039846C8A2FD4180BB744D8584184C18F7B05BB110FC4D1C8C1D019EB072FC5C55DDBAE8272CAC153810A6D5A828B7B86350F656D350299264A111C8C876D7502D634C6014F324773E64FFE56DA602736355061CAF39DD29706111ED10F00BD80EE4FE076F6BA743F210C9DC502493A8974FEC1E2706C094A20C29EA7DB29B37D3BD709CD6942FB8ED5EE3D1C521AFFA3A242A13A5A705C458027118C950E656794F671373CE3A7A2D53C0C4BCDDF539D530ABE4A554ED76F7D0D43C1DFFEA320D1CD14A2BC5A5B2C04D84C17399C590D94D86F3E85EC66693D364379149FB7E5709C45B7077D8784114D14B640EB8F50DECCE1A83B041CD217155918BFE937BC0C4E8BBA30452A2F00A4D5F2091DD12C8849EA4989EAC9BCB7040C0049C3986D4F5BF0F3D55FD05A847D3FC4A58F4F5DB860333B1C60F8AE4646919D339B0E4342FEE167A214B25D17D25BDDB7F81BCC37551435C033A66BCD3775FCAE15CA9F311E90FBFF2A082AC2A6B8B0C7F4F5865BFB3B18DEE1E184587C3BA746869D70402BCE1AFAC1AA45BF4D8A6A390F076CA1D6381F7581E2A5718BF44EFB75E9864EBF5EB011FD7C0892C8500BFA01B422E4B67222C259A263863E66CC26C533D0796C0C17D4945D9E0ADDD961EAC0FC3E594B14C13B9A586B0EED06D17F -20180608133606 2 6 100 7167 2 E76114725DD41FA35A7DA1CF2A6C5701F689FC40E96BDAB540431472B92D5E38711F596734C76D1407BC11D5EECD10C52AE09D6EB46C7A119FE7BF2CE718363E5D9D06D972A80D1AEBE550FEF87C596C7B2FAB2FA6A27BE1D6CBFBD7A8B55FFBEFC78E6028AB4F83093162C4D8F9C4E72D45AEE3F788A1AA98A93B2E92DCC69739C3311002388723ED096B0FECB0698AB631108A2F5922453FA81DD178DDFB7A8E651E19E086F36684EF4958EF57E18393C173D07B594AEA0EAF5FE770F72550528C1637E341715CC80C74A2024CDEB2F949200E29BBFC0519F029CA2D45C4A66602553EF9A9086183018A055DFE19698B779908B119CB4C9E162680CA5E3EA471F4AF46F77DA5FF2A2C23C18509B1E8D0591B868A76163079461A83FBDAC95A5E8E4EB9152EAFCDFAD9CE5CDA7CE7846BEE134D52242152C4B671D1B2A05B927F6546A272DDEC50DDAE2D66B866C393F7CA3E2FF505FCC5187394BEEA2BC94F840B800884A8853A6A9AF1FB62837DDE066CC17EDDAD6921F0D96A7A9C2C1705CC024050201E93AF587FE1A480EDA8DCC257B0F7FF6A3006613250098090436BC3C63039846C8A2FD4180BB744D8584184C18F7B05BB110FC4D1C8C1D019EB072FC5C55DDBAE8272CAC153810A6D5A828B7B86350F656D350299264A111C8C876D7502D634C6014F324773E64FFE56DA602736355061CAF39DD29706111ED10F00BD80EE4FE076F6BA743F210C9DC502493A8974FEC1E2706C094A20C29EA7DB29B37D3BD709CD6942FB8ED5EE3D1C521AFFA3A242A13A5A705C458027118C950E656794F671373CE3A7A2D53C0C4BCDDF539D530ABE4A554ED76F7D0D43C1DFFEA320D1CD14A2BC5A5B2C04D84C17399C590D94D86F3E85EC66693D364379149FB7E5709C45B7077D8784114D14B640EB8F50DECCE1A83B041CD217155918BFE937BC0C4E8BBA30452A2F00A4D5F2091DD12C8849EA4989EAC9BCB7040C0049C3986D4F5BF0F3D55FD05A847D3FC4A58F4F5DB860333B1C60F8AE4646919D339B0E4342FEE167A214B25D17D25BDDB7F81BCC37551435C033A66BCD3775FCAE15CA9F311E90FBFF2A082AC2A6B8B0C7F4F5865BFB3B18DEE1E184587C3BA746869D70402BCE1AFAC1AA45BF4D8A6A390F076CA1D6381F7581E2A5718BF44EFB75E9864EBF5EB011FD7C0892C8500BFA01B422E4B67222C259A263863E66CC26C533D0796C0C17D4945D9E0ADDD961EAC0FC3E594B14C13B9A586B0EEF40EF2B -20180608134723 2 6 100 7167 2 E76114725DD41FA35A7DA1CF2A6C5701F689FC40E96BDAB540431472B92D5E38711F596734C76D1407BC11D5EECD10C52AE09D6EB46C7A119FE7BF2CE718363E5D9D06D972A80D1AEBE550FEF87C596C7B2FAB2FA6A27BE1D6CBFBD7A8B55FFBEFC78E6028AB4F83093162C4D8F9C4E72D45AEE3F788A1AA98A93B2E92DCC69739C3311002388723ED096B0FECB0698AB631108A2F5922453FA81DD178DDFB7A8E651E19E086F36684EF4958EF57E18393C173D07B594AEA0EAF5FE770F72550528C1637E341715CC80C74A2024CDEB2F949200E29BBFC0519F029CA2D45C4A66602553EF9A9086183018A055DFE19698B779908B119CB4C9E162680CA5E3EA471F4AF46F77DA5FF2A2C23C18509B1E8D0591B868A76163079461A83FBDAC95A5E8E4EB9152EAFCDFAD9CE5CDA7CE7846BEE134D52242152C4B671D1B2A05B927F6546A272DDEC50DDAE2D66B866C393F7CA3E2FF505FCC5187394BEEA2BC94F840B800884A8853A6A9AF1FB62837DDE066CC17EDDAD6921F0D96A7A9C2C1705CC024050201E93AF587FE1A480EDA8DCC257B0F7FF6A3006613250098090436BC3C63039846C8A2FD4180BB744D8584184C18F7B05BB110FC4D1C8C1D019EB072FC5C55DDBAE8272CAC153810A6D5A828B7B86350F656D350299264A111C8C876D7502D634C6014F324773E64FFE56DA602736355061CAF39DD29706111ED10F00BD80EE4FE076F6BA743F210C9DC502493A8974FEC1E2706C094A20C29EA7DB29B37D3BD709CD6942FB8ED5EE3D1C521AFFA3A242A13A5A705C458027118C950E656794F671373CE3A7A2D53C0C4BCDDF539D530ABE4A554ED76F7D0D43C1DFFEA320D1CD14A2BC5A5B2C04D84C17399C590D94D86F3E85EC66693D364379149FB7E5709C45B7077D8784114D14B640EB8F50DECCE1A83B041CD217155918BFE937BC0C4E8BBA30452A2F00A4D5F2091DD12C8849EA4989EAC9BCB7040C0049C3986D4F5BF0F3D55FD05A847D3FC4A58F4F5DB860333B1C60F8AE4646919D339B0E4342FEE167A214B25D17D25BDDB7F81BCC37551435C033A66BCD3775FCAE15CA9F311E90FBFF2A082AC2A6B8B0C7F4F5865BFB3B18DEE1E184587C3BA746869D70402BCE1AFAC1AA45BF4D8A6A390F076CA1D6381F7581E2A5718BF44EFB75E9864EBF5EB011FD7C0892C8500BFA01B422E4B67222C259A263863E66CC26C533D0796C0C17D4945D9E0ADDD961EAC0FC3E594B14C13B9A586B0EEFA3A923 -20180608141153 2 6 100 7167 2 E76114725DD41FA35A7DA1CF2A6C5701F689FC40E96BDAB540431472B92D5E38711F596734C76D1407BC11D5EECD10C52AE09D6EB46C7A119FE7BF2CE718363E5D9D06D972A80D1AEBE550FEF87C596C7B2FAB2FA6A27BE1D6CBFBD7A8B55FFBEFC78E6028AB4F83093162C4D8F9C4E72D45AEE3F788A1AA98A93B2E92DCC69739C3311002388723ED096B0FECB0698AB631108A2F5922453FA81DD178DDFB7A8E651E19E086F36684EF4958EF57E18393C173D07B594AEA0EAF5FE770F72550528C1637E341715CC80C74A2024CDEB2F949200E29BBFC0519F029CA2D45C4A66602553EF9A9086183018A055DFE19698B779908B119CB4C9E162680CA5E3EA471F4AF46F77DA5FF2A2C23C18509B1E8D0591B868A76163079461A83FBDAC95A5E8E4EB9152EAFCDFAD9CE5CDA7CE7846BEE134D52242152C4B671D1B2A05B927F6546A272DDEC50DDAE2D66B866C393F7CA3E2FF505FCC5187394BEEA2BC94F840B800884A8853A6A9AF1FB62837DDE066CC17EDDAD6921F0D96A7A9C2C1705CC024050201E93AF587FE1A480EDA8DCC257B0F7FF6A3006613250098090436BC3C63039846C8A2FD4180BB744D8584184C18F7B05BB110FC4D1C8C1D019EB072FC5C55DDBAE8272CAC153810A6D5A828B7B86350F656D350299264A111C8C876D7502D634C6014F324773E64FFE56DA602736355061CAF39DD29706111ED10F00BD80EE4FE076F6BA743F210C9DC502493A8974FEC1E2706C094A20C29EA7DB29B37D3BD709CD6942FB8ED5EE3D1C521AFFA3A242A13A5A705C458027118C950E656794F671373CE3A7A2D53C0C4BCDDF539D530ABE4A554ED76F7D0D43C1DFFEA320D1CD14A2BC5A5B2C04D84C17399C590D94D86F3E85EC66693D364379149FB7E5709C45B7077D8784114D14B640EB8F50DECCE1A83B041CD217155918BFE937BC0C4E8BBA30452A2F00A4D5F2091DD12C8849EA4989EAC9BCB7040C0049C3986D4F5BF0F3D55FD05A847D3FC4A58F4F5DB860333B1C60F8AE4646919D339B0E4342FEE167A214B25D17D25BDDB7F81BCC37551435C033A66BCD3775FCAE15CA9F311E90FBFF2A082AC2A6B8B0C7F4F5865BFB3B18DEE1E184587C3BA746869D70402BCE1AFAC1AA45BF4D8A6A390F076CA1D6381F7581E2A5718BF44EFB75E9864EBF5EB011FD7C0892C8500BFA01B422E4B67222C259A263863E66CC26C533D0796C0C17D4945D9E0ADDD961EAC0FC3E594B14C13B9A586B0EF0866973 -20180608154036 2 6 100 7167 5 E76114725DD41FA35A7DA1CF2A6C5701F689FC40E96BDAB540431472B92D5E38711F596734C76D1407BC11D5EECD10C52AE09D6EB46C7A119FE7BF2CE718363E5D9D06D972A80D1AEBE550FEF87C596C7B2FAB2FA6A27BE1D6CBFBD7A8B55FFBEFC78E6028AB4F83093162C4D8F9C4E72D45AEE3F788A1AA98A93B2E92DCC69739C3311002388723ED096B0FECB0698AB631108A2F5922453FA81DD178DDFB7A8E651E19E086F36684EF4958EF57E18393C173D07B594AEA0EAF5FE770F72550528C1637E341715CC80C74A2024CDEB2F949200E29BBFC0519F029CA2D45C4A66602553EF9A9086183018A055DFE19698B779908B119CB4C9E162680CA5E3EA471F4AF46F77DA5FF2A2C23C18509B1E8D0591B868A76163079461A83FBDAC95A5E8E4EB9152EAFCDFAD9CE5CDA7CE7846BEE134D52242152C4B671D1B2A05B927F6546A272DDEC50DDAE2D66B866C393F7CA3E2FF505FCC5187394BEEA2BC94F840B800884A8853A6A9AF1FB62837DDE066CC17EDDAD6921F0D96A7A9C2C1705CC024050201E93AF587FE1A480EDA8DCC257B0F7FF6A3006613250098090436BC3C63039846C8A2FD4180BB744D8584184C18F7B05BB110FC4D1C8C1D019EB072FC5C55DDBAE8272CAC153810A6D5A828B7B86350F656D350299264A111C8C876D7502D634C6014F324773E64FFE56DA602736355061CAF39DD29706111ED10F00BD80EE4FE076F6BA743F210C9DC502493A8974FEC1E2706C094A20C29EA7DB29B37D3BD709CD6942FB8ED5EE3D1C521AFFA3A242A13A5A705C458027118C950E656794F671373CE3A7A2D53C0C4BCDDF539D530ABE4A554ED76F7D0D43C1DFFEA320D1CD14A2BC5A5B2C04D84C17399C590D94D86F3E85EC66693D364379149FB7E5709C45B7077D8784114D14B640EB8F50DECCE1A83B041CD217155918BFE937BC0C4E8BBA30452A2F00A4D5F2091DD12C8849EA4989EAC9BCB7040C0049C3986D4F5BF0F3D55FD05A847D3FC4A58F4F5DB860333B1C60F8AE4646919D339B0E4342FEE167A214B25D17D25BDDB7F81BCC37551435C033A66BCD3775FCAE15CA9F311E90FBFF2A082AC2A6B8B0C7F4F5865BFB3B18DEE1E184587C3BA746869D70402BCE1AFAC1AA45BF4D8A6A390F076CA1D6381F7581E2A5718BF44EFB75E9864EBF5EB011FD7C0892C8500BFA01B422E4B67222C259A263863E66CC26C533D0796C0C17D4945D9E0ADDD961EAC0FC3E594B14C13B9A586B0EF3C931BF -20180608155132 2 6 100 7167 2 E76114725DD41FA35A7DA1CF2A6C5701F689FC40E96BDAB540431472B92D5E38711F596734C76D1407BC11D5EECD10C52AE09D6EB46C7A119FE7BF2CE718363E5D9D06D972A80D1AEBE550FEF87C596C7B2FAB2FA6A27BE1D6CBFBD7A8B55FFBEFC78E6028AB4F83093162C4D8F9C4E72D45AEE3F788A1AA98A93B2E92DCC69739C3311002388723ED096B0FECB0698AB631108A2F5922453FA81DD178DDFB7A8E651E19E086F36684EF4958EF57E18393C173D07B594AEA0EAF5FE770F72550528C1637E341715CC80C74A2024CDEB2F949200E29BBFC0519F029CA2D45C4A66602553EF9A9086183018A055DFE19698B779908B119CB4C9E162680CA5E3EA471F4AF46F77DA5FF2A2C23C18509B1E8D0591B868A76163079461A83FBDAC95A5E8E4EB9152EAFCDFAD9CE5CDA7CE7846BEE134D52242152C4B671D1B2A05B927F6546A272DDEC50DDAE2D66B866C393F7CA3E2FF505FCC5187394BEEA2BC94F840B800884A8853A6A9AF1FB62837DDE066CC17EDDAD6921F0D96A7A9C2C1705CC024050201E93AF587FE1A480EDA8DCC257B0F7FF6A3006613250098090436BC3C63039846C8A2FD4180BB744D8584184C18F7B05BB110FC4D1C8C1D019EB072FC5C55DDBAE8272CAC153810A6D5A828B7B86350F656D350299264A111C8C876D7502D634C6014F324773E64FFE56DA602736355061CAF39DD29706111ED10F00BD80EE4FE076F6BA743F210C9DC502493A8974FEC1E2706C094A20C29EA7DB29B37D3BD709CD6942FB8ED5EE3D1C521AFFA3A242A13A5A705C458027118C950E656794F671373CE3A7A2D53C0C4BCDDF539D530ABE4A554ED76F7D0D43C1DFFEA320D1CD14A2BC5A5B2C04D84C17399C590D94D86F3E85EC66693D364379149FB7E5709C45B7077D8784114D14B640EB8F50DECCE1A83B041CD217155918BFE937BC0C4E8BBA30452A2F00A4D5F2091DD12C8849EA4989EAC9BCB7040C0049C3986D4F5BF0F3D55FD05A847D3FC4A58F4F5DB860333B1C60F8AE4646919D339B0E4342FEE167A214B25D17D25BDDB7F81BCC37551435C033A66BCD3775FCAE15CA9F311E90FBFF2A082AC2A6B8B0C7F4F5865BFB3B18DEE1E184587C3BA746869D70402BCE1AFAC1AA45BF4D8A6A390F076CA1D6381F7581E2A5718BF44EFB75E9864EBF5EB011FD7C0892C8500BFA01B422E4B67222C259A263863E66CC26C533D0796C0C17D4945D9E0ADDD961EAC0FC3E594B14C13B9A586B0EF42B6B9B -20180608192033 2 6 100 7167 5 E76114725DD41FA35A7DA1CF2A6C5701F689FC40E96BDAB540431472B92D5E38711F596734C76D1407BC11D5EECD10C52AE09D6EB46C7A119FE7BF2CE718363E5D9D06D972A80D1AEBE550FEF87C596C7B2FAB2FA6A27BE1D6CBFBD7A8B55FFBEFC78E6028AB4F83093162C4D8F9C4E72D45AEE3F788A1AA98A93B2E92DCC69739C3311002388723ED096B0FECB0698AB631108A2F5922453FA81DD178DDFB7A8E651E19E086F36684EF4958EF57E18393C173D07B594AEA0EAF5FE770F72550528C1637E341715CC80C74A2024CDEB2F949200E29BBFC0519F029CA2D45C4A66602553EF9A9086183018A055DFE19698B779908B119CB4C9E162680CA5E3EA471F4AF46F77DA5FF2A2C23C18509B1E8D0591B868A76163079461A83FBDAC95A5E8E4EB9152EAFCDFAD9CE5CDA7CE7846BEE134D52242152C4B671D1B2A05B927F6546A272DDEC50DDAE2D66B866C393F7CA3E2FF505FCC5187394BEEA2BC94F840B800884A8853A6A9AF1FB62837DDE066CC17EDDAD6921F0D96A7A9C2C1705CC024050201E93AF587FE1A480EDA8DCC257B0F7FF6A3006613250098090436BC3C63039846C8A2FD4180BB744D8584184C18F7B05BB110FC4D1C8C1D019EB072FC5C55DDBAE8272CAC153810A6D5A828B7B86350F656D350299264A111C8C876D7502D634C6014F324773E64FFE56DA602736355061CAF39DD29706111ED10F00BD80EE4FE076F6BA743F210C9DC502493A8974FEC1E2706C094A20C29EA7DB29B37D3BD709CD6942FB8ED5EE3D1C521AFFA3A242A13A5A705C458027118C950E656794F671373CE3A7A2D53C0C4BCDDF539D530ABE4A554ED76F7D0D43C1DFFEA320D1CD14A2BC5A5B2C04D84C17399C590D94D86F3E85EC66693D364379149FB7E5709C45B7077D8784114D14B640EB8F50DECCE1A83B041CD217155918BFE937BC0C4E8BBA30452A2F00A4D5F2091DD12C8849EA4989EAC9BCB7040C0049C3986D4F5BF0F3D55FD05A847D3FC4A58F4F5DB860333B1C60F8AE4646919D339B0E4342FEE167A214B25D17D25BDDB7F81BCC37551435C033A66BCD3775FCAE15CA9F311E90FBFF2A082AC2A6B8B0C7F4F5865BFB3B18DEE1E184587C3BA746869D70402BCE1AFAC1AA45BF4D8A6A390F076CA1D6381F7581E2A5718BF44EFB75E9864EBF5EB011FD7C0892C8500BFA01B422E4B67222C259A263863E66CC26C533D0796C0C17D4945D9E0ADDD961EAC0FC3E594B14C13B9A586B0EFBB82C37 -20180608203618 2 6 100 7167 5 E76114725DD41FA35A7DA1CF2A6C5701F689FC40E96BDAB540431472B92D5E38711F596734C76D1407BC11D5EECD10C52AE09D6EB46C7A119FE7BF2CE718363E5D9D06D972A80D1AEBE550FEF87C596C7B2FAB2FA6A27BE1D6CBFBD7A8B55FFBEFC78E6028AB4F83093162C4D8F9C4E72D45AEE3F788A1AA98A93B2E92DCC69739C3311002388723ED096B0FECB0698AB631108A2F5922453FA81DD178DDFB7A8E651E19E086F36684EF4958EF57E18393C173D07B594AEA0EAF5FE770F72550528C1637E341715CC80C74A2024CDEB2F949200E29BBFC0519F029CA2D45C4A66602553EF9A9086183018A055DFE19698B779908B119CB4C9E162680CA5E3EA471F4AF46F77DA5FF2A2C23C18509B1E8D0591B868A76163079461A83FBDAC95A5E8E4EB9152EAFCDFAD9CE5CDA7CE7846BEE134D52242152C4B671D1B2A05B927F6546A272DDEC50DDAE2D66B866C393F7CA3E2FF505FCC5187394BEEA2BC94F840B800884A8853A6A9AF1FB62837DDE066CC17EDDAD6921F0D96A7A9C2C1705CC024050201E93AF587FE1A480EDA8DCC257B0F7FF6A3006613250098090436BC3C63039846C8A2FD4180BB744D8584184C18F7B05BB110FC4D1C8C1D019EB072FC5C55DDBAE8272CAC153810A6D5A828B7B86350F656D350299264A111C8C876D7502D634C6014F324773E64FFE56DA602736355061CAF39DD29706111ED10F00BD80EE4FE076F6BA743F210C9DC502493A8974FEC1E2706C094A20C29EA7DB29B37D3BD709CD6942FB8ED5EE3D1C521AFFA3A242A13A5A705C458027118C950E656794F671373CE3A7A2D53C0C4BCDDF539D530ABE4A554ED76F7D0D43C1DFFEA320D1CD14A2BC5A5B2C04D84C17399C590D94D86F3E85EC66693D364379149FB7E5709C45B7077D8784114D14B640EB8F50DECCE1A83B041CD217155918BFE937BC0C4E8BBA30452A2F00A4D5F2091DD12C8849EA4989EAC9BCB7040C0049C3986D4F5BF0F3D55FD05A847D3FC4A58F4F5DB860333B1C60F8AE4646919D339B0E4342FEE167A214B25D17D25BDDB7F81BCC37551435C033A66BCD3775FCAE15CA9F311E90FBFF2A082AC2A6B8B0C7F4F5865BFB3B18DEE1E184587C3BA746869D70402BCE1AFAC1AA45BF4D8A6A390F076CA1D6381F7581E2A5718BF44EFB75E9864EBF5EB011FD7C0892C8500BFA01B422E4B67222C259A263863E66CC26C533D0796C0C17D4945D9E0ADDD961EAC0FC3E594B14C13B9A586B0EFE7167C7 -20180608203803 2 6 100 7167 2 E76114725DD41FA35A7DA1CF2A6C5701F689FC40E96BDAB540431472B92D5E38711F596734C76D1407BC11D5EECD10C52AE09D6EB46C7A119FE7BF2CE718363E5D9D06D972A80D1AEBE550FEF87C596C7B2FAB2FA6A27BE1D6CBFBD7A8B55FFBEFC78E6028AB4F83093162C4D8F9C4E72D45AEE3F788A1AA98A93B2E92DCC69739C3311002388723ED096B0FECB0698AB631108A2F5922453FA81DD178DDFB7A8E651E19E086F36684EF4958EF57E18393C173D07B594AEA0EAF5FE770F72550528C1637E341715CC80C74A2024CDEB2F949200E29BBFC0519F029CA2D45C4A66602553EF9A9086183018A055DFE19698B779908B119CB4C9E162680CA5E3EA471F4AF46F77DA5FF2A2C23C18509B1E8D0591B868A76163079461A83FBDAC95A5E8E4EB9152EAFCDFAD9CE5CDA7CE7846BEE134D52242152C4B671D1B2A05B927F6546A272DDEC50DDAE2D66B866C393F7CA3E2FF505FCC5187394BEEA2BC94F840B800884A8853A6A9AF1FB62837DDE066CC17EDDAD6921F0D96A7A9C2C1705CC024050201E93AF587FE1A480EDA8DCC257B0F7FF6A3006613250098090436BC3C63039846C8A2FD4180BB744D8584184C18F7B05BB110FC4D1C8C1D019EB072FC5C55DDBAE8272CAC153810A6D5A828B7B86350F656D350299264A111C8C876D7502D634C6014F324773E64FFE56DA602736355061CAF39DD29706111ED10F00BD80EE4FE076F6BA743F210C9DC502493A8974FEC1E2706C094A20C29EA7DB29B37D3BD709CD6942FB8ED5EE3D1C521AFFA3A242A13A5A705C458027118C950E656794F671373CE3A7A2D53C0C4BCDDF539D530ABE4A554ED76F7D0D43C1DFFEA320D1CD14A2BC5A5B2C04D84C17399C590D94D86F3E85EC66693D364379149FB7E5709C45B7077D8784114D14B640EB8F50DECCE1A83B041CD217155918BFE937BC0C4E8BBA30452A2F00A4D5F2091DD12C8849EA4989EAC9BCB7040C0049C3986D4F5BF0F3D55FD05A847D3FC4A58F4F5DB860333B1C60F8AE4646919D339B0E4342FEE167A214B25D17D25BDDB7F81BCC37551435C033A66BCD3775FCAE15CA9F311E90FBFF2A082AC2A6B8B0C7F4F5865BFB3B18DEE1E184587C3BA746869D70402BCE1AFAC1AA45BF4D8A6A390F076CA1D6381F7581E2A5718BF44EFB75E9864EBF5EB011FD7C0892C8500BFA01B422E4B67222C259A263863E66CC26C533D0796C0C17D4945D9E0ADDD961EAC0FC3E594B14C13B9A586B0EFE777583 -20180608231830 2 6 100 7167 2 E76114725DD41FA35A7DA1CF2A6C5701F689FC40E96BDAB540431472B92D5E38711F596734C76D1407BC11D5EECD10C52AE09D6EB46C7A119FE7BF2CE718363E5D9D06D972A80D1AEBE550FEF87C596C7B2FAB2FA6A27BE1D6CBFBD7A8B55FFBEFC78E6028AB4F83093162C4D8F9C4E72D45AEE3F788A1AA98A93B2E92DCC69739C3311002388723ED096B0FECB0698AB631108A2F5922453FA81DD178DDFB7A8E651E19E086F36684EF4958EF57E18393C173D07B594AEA0EAF5FE770F72550528C1637E341715CC80C74A2024CDEB2F949200E29BBFC0519F029CA2D45C4A66602553EF9A9086183018A055DFE19698B779908B119CB4C9E162680CA5E3EA471F4AF46F77DA5FF2A2C23C18509B1E8D0591B868A76163079461A83FBDAC95A5E8E4EB9152EAFCDFAD9CE5CDA7CE7846BEE134D52242152C4B671D1B2A05B927F6546A272DDEC50DDAE2D66B866C393F7CA3E2FF505FCC5187394BEEA2BC94F840B800884A8853A6A9AF1FB62837DDE066CC17EDDAD6921F0D96A7A9C2C1705CC024050201E93AF587FE1A480EDA8DCC257B0F7FF6A3006613250098090436BC3C63039846C8A2FD4180BB744D8584184C18F7B05BB110FC4D1C8C1D019EB072FC5C55DDBAE8272CAC153810A6D5A828B7B86350F656D350299264A111C8C876D7502D634C6014F324773E64FFE56DA602736355061CAF39DD29706111ED10F00BD80EE4FE076F6BA743F210C9DC502493A8974FEC1E2706C094A20C29EA7DB29B37D3BD709CD6942FB8ED5EE3D1C521AFFA3A242A13A5A705C458027118C950E656794F671373CE3A7A2D53C0C4BCDDF539D530ABE4A554ED76F7D0D43C1DFFEA320D1CD14A2BC5A5B2C04D84C17399C590D94D86F3E85EC66693D364379149FB7E5709C45B7077D8784114D14B640EB8F50DECCE1A83B041CD217155918BFE937BC0C4E8BBA30452A2F00A4D5F2091DD12C8849EA4989EAC9BCB7040C0049C3986D4F5BF0F3D55FD05A847D3FC4A58F4F5DB860333B1C60F8AE4646919D339B0E4342FEE167A214B25D17D25BDDB7F81BCC37551435C033A66BCD3775FCAE15CA9F311E90FBFF2A082AC2A6B8B0C7F4F5865BFB3B18DEE1E184587C3BA746869D70402BCE1AFAC1AA45BF4D8A6A390F076CA1D6381F7581E2A5718BF44EFB75E9864EBF5EB011FD7C0892C8500BFA01B422E4B67222C259A263863E66CC26C533D0796C0C17D4945D9E0ADDD961EAC0FC3E594B14C13B9A586B0F04486A93 20180607113212 2 6 100 8191 5 CEC852A0F60D2D7ECA1964A06B30C9C74CED6097183E357235647B360695981BFCA2F828FF22BB22CBCFF24E5B421D02E4169AA216FA977C33943011749067AF3996452BEB9C466CB71DB44802D60450F77A488C2C653115C2DD322D781CB1F4F70CBA60F5030B2569B1147F1EA25872E40326FF2F21D60DDBCC2350883EDF246F9F60D2BED0887D8CD9629A2317C4342785BED83D75765C8CCB9770B156256AB3A3D2F1E5C213BF51DA215802212A3EB18ECA6AFB944D26382E2F314FFB6F83AA2BB057E17BF7BE2604BEFF9BEAF8C72594B5097B97CDE8B4724B7A41816481154886EA99878F7D1570F1365971A237B18B0C048B4E0B550F0E4B84B0AEA8111991ADDB782A2585632E85100B8E6B04048565A70F699A04DF650C7FD138F489FC7B6D5D306FD86902666DA4C2F5259795BAEFA426F391265C2840232DC57DF50093EB6A6BDA2884BE777701953FBA502A233D811A49EBF59241B5931C6DAFC1CF3CFD688E182D8D27FF2CE1CA1D5AB06A44ACF0BAC4A35BF5212955D52AF7FE1F0EED58205C7A08EDF536AB22727C670B56D3F134485D4D4CE3D93883EA07AB900E4B9E1B39B6C19FFACFF664224038B5DE4C97DF0008282E6DDFCB9CAD35CD18FD9EE5EDCE40800E741F5CACAC25CEA9EF53C3B43008FF371EF9453A4E4AB69112C5761E903B5656D7FCA40090A1B26C8670217A2DF0AB05F4E530E1445FB243F2E8376BA169BD46A57EED72BE4395D77F5C595E422DECC125DC9ED0445CC15BC977F7DB6F683A419D862606E8EEC600CB206CDB958E9C98DC23E42BCDAE6560561177C4A119E0C65B22121B127B0A3E4A1D92CB42C297E59EDCD056F32DF3B8BF436785A842414BE3203395703B08EA6DCE4ECFE7831A882C0E0644C9D8C259CBFB3DB05AFCA2669D61D2E12EC3EB7BB105BC96627D239F3009F246CB5312C35420251A92F49C0B4C4B51B5526919C6767A13068A220497DFDEA49FC181F4EA2BAF496AE58E2A924F27F658FE6F7F7E21E26F5BCE5D469ABD9F3F19766051E9352CDB1324388C5DB78A6BD1DC861BE588FFDE7DDFE87F1CB0E9F422BD2B95A1EDC2D65EF9A88F36465053F4611E2B425CA697D4ACB99B5DE6B699495EA2F8F840D4720E40F475D3761432EBAAB46C15F2C251AB9C0AA4FAADEFD40B3C707F7B2A6729E55342F8728D21D263FF204B95407976147B9482170466B718232AA3E82A0C53049C6B2C0884830CEDCD8B9992309BB49BFD6BFD4695B0060540DE388DABC39D732B392F38D433EE0C02BE154DD8A48DB89BD98CB161A63AEEB0D25CC9A29D7C6F3780EA87808E8463E9EE46CBD669FAD39B6BDF75B054951784154BAB9C0BC554F30258AC6B39D72A492C027EE8955704E44414DF957398A9A0DA9005E6B152930F7A71AEEAAF2543947A773545E7A8B0B9C13F0CCAD9285F38A617 20180607125146 2 6 100 8191 2 CEC852A0F60D2D7ECA1964A06B30C9C74CED6097183E357235647B360695981BFCA2F828FF22BB22CBCFF24E5B421D02E4169AA216FA977C33943011749067AF3996452BEB9C466CB71DB44802D60450F77A488C2C653115C2DD322D781CB1F4F70CBA60F5030B2569B1147F1EA25872E40326FF2F21D60DDBCC2350883EDF246F9F60D2BED0887D8CD9629A2317C4342785BED83D75765C8CCB9770B156256AB3A3D2F1E5C213BF51DA215802212A3EB18ECA6AFB944D26382E2F314FFB6F83AA2BB057E17BF7BE2604BEFF9BEAF8C72594B5097B97CDE8B4724B7A41816481154886EA99878F7D1570F1365971A237B18B0C048B4E0B550F0E4B84B0AEA8111991ADDB782A2585632E85100B8E6B04048565A70F699A04DF650C7FD138F489FC7B6D5D306FD86902666DA4C2F5259795BAEFA426F391265C2840232DC57DF50093EB6A6BDA2884BE777701953FBA502A233D811A49EBF59241B5931C6DAFC1CF3CFD688E182D8D27FF2CE1CA1D5AB06A44ACF0BAC4A35BF5212955D52AF7FE1F0EED58205C7A08EDF536AB22727C670B56D3F134485D4D4CE3D93883EA07AB900E4B9E1B39B6C19FFACFF664224038B5DE4C97DF0008282E6DDFCB9CAD35CD18FD9EE5EDCE40800E741F5CACAC25CEA9EF53C3B43008FF371EF9453A4E4AB69112C5761E903B5656D7FCA40090A1B26C8670217A2DF0AB05F4E530E1445FB243F2E8376BA169BD46A57EED72BE4395D77F5C595E422DECC125DC9ED0445CC15BC977F7DB6F683A419D862606E8EEC600CB206CDB958E9C98DC23E42BCDAE6560561177C4A119E0C65B22121B127B0A3E4A1D92CB42C297E59EDCD056F32DF3B8BF436785A842414BE3203395703B08EA6DCE4ECFE7831A882C0E0644C9D8C259CBFB3DB05AFCA2669D61D2E12EC3EB7BB105BC96627D239F3009F246CB5312C35420251A92F49C0B4C4B51B5526919C6767A13068A220497DFDEA49FC181F4EA2BAF496AE58E2A924F27F658FE6F7F7E21E26F5BCE5D469ABD9F3F19766051E9352CDB1324388C5DB78A6BD1DC861BE588FFDE7DDFE87F1CB0E9F422BD2B95A1EDC2D65EF9A88F36465053F4611E2B425CA697D4ACB99B5DE6B699495EA2F8F840D4720E40F475D3761432EBAAB46C15F2C251AB9C0AA4FAADEFD40B3C707F7B2A6729E55342F8728D21D263FF204B95407976147B9482170466B718232AA3E82A0C53049C6B2C0884830CEDCD8B9992309BB49BFD6BFD4695B0060540DE388DABC39D732B392F38D433EE0C02BE154DD8A48DB89BD98CB161A63AEEB0D25CC9A29D7C6F3780EA87808E8463E9EE46CBD669FAD39B6BDF75B054951784154BAB9C0BC554F30258AC6B39D72A492C027EE8955704E44414DF957398A9A0DA9005E6B152930F7A71AEEAAF2543947A773545E7A8B0B9C13F0CCAD928617A82F3 20180607125328 2 6 100 8191 5 CEC852A0F60D2D7ECA1964A06B30C9C74CED6097183E357235647B360695981BFCA2F828FF22BB22CBCFF24E5B421D02E4169AA216FA977C33943011749067AF3996452BEB9C466CB71DB44802D60450F77A488C2C653115C2DD322D781CB1F4F70CBA60F5030B2569B1147F1EA25872E40326FF2F21D60DDBCC2350883EDF246F9F60D2BED0887D8CD9629A2317C4342785BED83D75765C8CCB9770B156256AB3A3D2F1E5C213BF51DA215802212A3EB18ECA6AFB944D26382E2F314FFB6F83AA2BB057E17BF7BE2604BEFF9BEAF8C72594B5097B97CDE8B4724B7A41816481154886EA99878F7D1570F1365971A237B18B0C048B4E0B550F0E4B84B0AEA8111991ADDB782A2585632E85100B8E6B04048565A70F699A04DF650C7FD138F489FC7B6D5D306FD86902666DA4C2F5259795BAEFA426F391265C2840232DC57DF50093EB6A6BDA2884BE777701953FBA502A233D811A49EBF59241B5931C6DAFC1CF3CFD688E182D8D27FF2CE1CA1D5AB06A44ACF0BAC4A35BF5212955D52AF7FE1F0EED58205C7A08EDF536AB22727C670B56D3F134485D4D4CE3D93883EA07AB900E4B9E1B39B6C19FFACFF664224038B5DE4C97DF0008282E6DDFCB9CAD35CD18FD9EE5EDCE40800E741F5CACAC25CEA9EF53C3B43008FF371EF9453A4E4AB69112C5761E903B5656D7FCA40090A1B26C8670217A2DF0AB05F4E530E1445FB243F2E8376BA169BD46A57EED72BE4395D77F5C595E422DECC125DC9ED0445CC15BC977F7DB6F683A419D862606E8EEC600CB206CDB958E9C98DC23E42BCDAE6560561177C4A119E0C65B22121B127B0A3E4A1D92CB42C297E59EDCD056F32DF3B8BF436785A842414BE3203395703B08EA6DCE4ECFE7831A882C0E0644C9D8C259CBFB3DB05AFCA2669D61D2E12EC3EB7BB105BC96627D239F3009F246CB5312C35420251A92F49C0B4C4B51B5526919C6767A13068A220497DFDEA49FC181F4EA2BAF496AE58E2A924F27F658FE6F7F7E21E26F5BCE5D469ABD9F3F19766051E9352CDB1324388C5DB78A6BD1DC861BE588FFDE7DDFE87F1CB0E9F422BD2B95A1EDC2D65EF9A88F36465053F4611E2B425CA697D4ACB99B5DE6B699495EA2F8F840D4720E40F475D3761432EBAAB46C15F2C251AB9C0AA4FAADEFD40B3C707F7B2A6729E55342F8728D21D263FF204B95407976147B9482170466B718232AA3E82A0C53049C6B2C0884830CEDCD8B9992309BB49BFD6BFD4695B0060540DE388DABC39D732B392F38D433EE0C02BE154DD8A48DB89BD98CB161A63AEEB0D25CC9A29D7C6F3780EA87808E8463E9EE46CBD669FAD39B6BDF75B054951784154BAB9C0BC554F30258AC6B39D72A492C027EE8955704E44414DF957398A9A0DA9005E6B152930F7A71AEEAAF2543947A773545E7A8B0B9C13F0CCAD928617CB5DF diff --git a/lib/public_key/src/pubkey_cert.erl b/lib/public_key/src/pubkey_cert.erl index c0d7b9be8e..61a1239d26 100644 --- a/lib/public_key/src/pubkey_cert.erl +++ b/lib/public_key/src/pubkey_cert.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2017. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/public_key/src/pubkey_crl.erl b/lib/public_key/src/pubkey_crl.erl index 3621e9c0da..50abd704de 100644 --- a/lib/public_key/src/pubkey_crl.erl +++ b/lib/public_key/src/pubkey_crl.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2016. All Rights Reserved. +%% Copyright Ericsson AB 2010-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/public_key/src/pubkey_moduli.hrl b/lib/public_key/src/pubkey_moduli.hrl index d3e2492cf2..768db624c7 100644 --- a/lib/public_key/src/pubkey_moduli.hrl +++ b/lib/public_key/src/pubkey_moduli.hrl @@ -352,75 +352,6 @@ 822475527894530229285616572113840248471252289865654730245673569989009517195150670386319918731764341482939973599599030444586627098726805325532602670776492782830517765627499389443537251148551632924278584658894256392558800578282770783952781444172123021909239761249266774011128112210797775865561188225245937456884457382356013600801136212313898917668235824265001348217562305609115401999529744566651634335823205650035802368807460021617740974817291658859354863897470423588038163002057414285871641295222833687523714538475514552855242579151518676338365048493158502050812316214163484509139051517638342109175849018785568945149403656833584359154601084850728291664155244903363511084332084337961858034913924653425899617872300974717518876133738228463402736897605365573279650828581423928774933880146466418294504197525108110672457161289146815032997391965470390378909910135534473119298720916725046763262261767220808960157446815529871730997078479389353851235418277521225090118250855917746026804319123727978865815215602336860198597431012309304151978584224046103750860062629864591763945976315990671793648388266723695721279940197843205282458713913995036945430630152450677414407385229033726976302905725696701985169880658807635234608315361840320908676438567}, {5, 822475527894530229285616572113840248471252289865654730245673569989009517195150670386319918731764341482939973599599030444586627098726805325532602670776492782830517765627499389443537251148551632924278584658894256392558800578282770783952781444172123021909239761249266774011128112210797775865561188225245937456884457382356013600801136212313898917668235824265001348217562305609115401999529744566651634335823205650035802368807460021617740974817291658859354863897470423588038163002057414285871641295222833687523714538475514552855242579151518676338365048493158502050812316214163484509139051517638342109175849018785568945149403656833584359154601084850728291664155244903363511084332084337961858034913924653425899617872300974717518876133738228463402736897605365573279650828581423928774933880146466418294504197525108110672457161289146815032997391965470390378909910135534473119298720916725046763262261767220808960157446815529871730997078479389353851235418277521225090118250855917746026804319123727978865815215602336860198597431012309304151978584224046103750860062629864591763945976315990671793648388266723695721279940197843205282458713913995036945430630152450677414407385229033726976302905725696701985169880658807635234608315361840320908697685503}]}, - {5121, - [{2, - 649437634112526011781048553400039087345326903154523759350403591156064026924461442249265424643851483793427933209842971350578174113891441825577284685965852170818096276564472575980385637569538579237402539408651576344680653270161629667112355235907244563250700929011643383515531122864487687325699022847401349887930148743573324935926928038313274118780408648738039221552094286634605859059178302001609063051473627616638916986695467354111409872240377448596882457893385604785675125747799925704521626444410537947809612042456889585358493384339367397319430466068497259487250009990226983476303290182605464299191144834590976900215207184217290547939862278004600042733098488459539365766704233072540349790910413963048455919274571091600102388120830520138960918238026360535071025369430480571742657175404099199715243175884516644982333939980185912023546244471169175154232817129035386017126612372249727029415960226233559719423091324216204450258205068541788852628274234145551296430267231614519548422573530219115334315814963859220772016127905259664901596292960624357867171314754794327851281187544474064975723856559261998464776997766060743177575177887669315503357551968629467385805021951612024977234727345596392623743650652821546588655008745166946053122284578514217986359571614564066926642313354080995691975795001216172305325876725224168312406194959039369235312034402577256851071978396414051789255283218481832719899346830805746190414783360826783268783647194536819312776246291560322670087150506500394384601386261997874464354519244142316219347283660495896740247097689699}, - {2, - 649437634112526011781048553400039087345326903154523759350403591156064026924461442249265424643851483793427933209842971350578174113891441825577284685965852170818096276564472575980385637569538579237402539408651576344680653270161629667112355235907244563250700929011643383515531122864487687325699022847401349887930148743573324935926928038313274118780408648738039221552094286634605859059178302001609063051473627616638916986695467354111409872240377448596882457893385604785675125747799925704521626444410537947809612042456889585358493384339367397319430466068497259487250009990226983476303290182605464299191144834590976900215207184217290547939862278004600042733098488459539365766704233072540349790910413963048455919274571091600102388120830520138960918238026360535071025369430480571742657175404099199715243175884516644982333939980185912023546244471169175154232817129035386017126612372249727029415960226233559719423091324216204450258205068541788852628274234145551296430267231614519548422573530219115334315814963859220772016127905259664901596292960624357867171314754794327851281187544474064975723856559261998464776997766060743177575177887669315503357551968629467385805021951612024977234727345596392623743650652821546588655008745166946053122284578514217986359571614564066926642313354080995691975795001216172305325876725224168312406194959039369235312034402577256851071978396414051789255283218481832719899346830805746190414783360826783268783647194536819312776246291560322670087150506500394384601386261997874464354519244142316219347283660495896740247187891059}, - {2, - 649437634112526011781048553400039087345326903154523759350403591156064026924461442249265424643851483793427933209842971350578174113891441825577284685965852170818096276564472575980385637569538579237402539408651576344680653270161629667112355235907244563250700929011643383515531122864487687325699022847401349887930148743573324935926928038313274118780408648738039221552094286634605859059178302001609063051473627616638916986695467354111409872240377448596882457893385604785675125747799925704521626444410537947809612042456889585358493384339367397319430466068497259487250009990226983476303290182605464299191144834590976900215207184217290547939862278004600042733098488459539365766704233072540349790910413963048455919274571091600102388120830520138960918238026360535071025369430480571742657175404099199715243175884516644982333939980185912023546244471169175154232817129035386017126612372249727029415960226233559719423091324216204450258205068541788852628274234145551296430267231614519548422573530219115334315814963859220772016127905259664901596292960624357867171314754794327851281187544474064975723856559261998464776997766060743177575177887669315503357551968629467385805021951612024977234727345596392623743650652821546588655008745166946053122284578514217986359571614564066926642313354080995691975795001216172305325876725224168312406194959039369235312034402577256851071978396414051789255283218481832719899346830805746190414783360826783268783647194536819312776246291560322670087150506500394384601386261997874464354519244142316219347283660495896740247233309283}, - {2, - 649437634112526011781048553400039087345326903154523759350403591156064026924461442249265424643851483793427933209842971350578174113891441825577284685965852170818096276564472575980385637569538579237402539408651576344680653270161629667112355235907244563250700929011643383515531122864487687325699022847401349887930148743573324935926928038313274118780408648738039221552094286634605859059178302001609063051473627616638916986695467354111409872240377448596882457893385604785675125747799925704521626444410537947809612042456889585358493384339367397319430466068497259487250009990226983476303290182605464299191144834590976900215207184217290547939862278004600042733098488459539365766704233072540349790910413963048455919274571091600102388120830520138960918238026360535071025369430480571742657175404099199715243175884516644982333939980185912023546244471169175154232817129035386017126612372249727029415960226233559719423091324216204450258205068541788852628274234145551296430267231614519548422573530219115334315814963859220772016127905259664901596292960624357867171314754794327851281187544474064975723856559261998464776997766060743177575177887669315503357551968629467385805021951612024977234727345596392623743650652821546588655008745166946053122284578514217986359571614564066926642313354080995691975795001216172305325876725224168312406194959039369235312034402577256851071978396414051789255283218481832719899346830805746190414783360826783268783647194536819312776246291560322670087150506500394384601386261997874464354519244142316219347283660495896740247329347539}, - {2, - 649437634112526011781048553400039087345326903154523759350403591156064026924461442249265424643851483793427933209842971350578174113891441825577284685965852170818096276564472575980385637569538579237402539408651576344680653270161629667112355235907244563250700929011643383515531122864487687325699022847401349887930148743573324935926928038313274118780408648738039221552094286634605859059178302001609063051473627616638916986695467354111409872240377448596882457893385604785675125747799925704521626444410537947809612042456889585358493384339367397319430466068497259487250009990226983476303290182605464299191144834590976900215207184217290547939862278004600042733098488459539365766704233072540349790910413963048455919274571091600102388120830520138960918238026360535071025369430480571742657175404099199715243175884516644982333939980185912023546244471169175154232817129035386017126612372249727029415960226233559719423091324216204450258205068541788852628274234145551296430267231614519548422573530219115334315814963859220772016127905259664901596292960624357867171314754794327851281187544474064975723856559261998464776997766060743177575177887669315503357551968629467385805021951612024977234727345596392623743650652821546588655008745166946053122284578514217986359571614564066926642313354080995691975795001216172305325876725224168312406194959039369235312034402577256851071978396414051789255283218481832719899346830805746190414783360826783268783647194536819312776246291560322670087150506500394384601386261997874464354519244142316219347283660495896740247374725179}, - {2, - 649437634112526011781048553400039087345326903154523759350403591156064026924461442249265424643851483793427933209842971350578174113891441825577284685965852170818096276564472575980385637569538579237402539408651576344680653270161629667112355235907244563250700929011643383515531122864487687325699022847401349887930148743573324935926928038313274118780408648738039221552094286634605859059178302001609063051473627616638916986695467354111409872240377448596882457893385604785675125747799925704521626444410537947809612042456889585358493384339367397319430466068497259487250009990226983476303290182605464299191144834590976900215207184217290547939862278004600042733098488459539365766704233072540349790910413963048455919274571091600102388120830520138960918238026360535071025369430480571742657175404099199715243175884516644982333939980185912023546244471169175154232817129035386017126612372249727029415960226233559719423091324216204450258205068541788852628274234145551296430267231614519548422573530219115334315814963859220772016127905259664901596292960624357867171314754794327851281187544474064975723856559261998464776997766060743177575177887669315503357551968629467385805021951612024977234727345596392623743650652821546588655008745166946053122284578514217986359571614564066926642313354080995691975795001216172305325876725224168312406194959039369235312034402577256851071978396414051789255283218481832719899346830805746190414783360826783268783647194536819312776246291560322670087150506500394384601386261997874464354519244142316219347283660495896740247374854083}, - {2, - 649437634112526011781048553400039087345326903154523759350403591156064026924461442249265424643851483793427933209842971350578174113891441825577284685965852170818096276564472575980385637569538579237402539408651576344680653270161629667112355235907244563250700929011643383515531122864487687325699022847401349887930148743573324935926928038313274118780408648738039221552094286634605859059178302001609063051473627616638916986695467354111409872240377448596882457893385604785675125747799925704521626444410537947809612042456889585358493384339367397319430466068497259487250009990226983476303290182605464299191144834590976900215207184217290547939862278004600042733098488459539365766704233072540349790910413963048455919274571091600102388120830520138960918238026360535071025369430480571742657175404099199715243175884516644982333939980185912023546244471169175154232817129035386017126612372249727029415960226233559719423091324216204450258205068541788852628274234145551296430267231614519548422573530219115334315814963859220772016127905259664901596292960624357867171314754794327851281187544474064975723856559261998464776997766060743177575177887669315503357551968629467385805021951612024977234727345596392623743650652821546588655008745166946053122284578514217986359571614564066926642313354080995691975795001216172305325876725224168312406194959039369235312034402577256851071978396414051789255283218481832719899346830805746190414783360826783268783647194536819312776246291560322670087150506500394384601386261997874464354519244142316219347283660495896740247416266563}, - {2, - 649437634112526011781048553400039087345326903154523759350403591156064026924461442249265424643851483793427933209842971350578174113891441825577284685965852170818096276564472575980385637569538579237402539408651576344680653270161629667112355235907244563250700929011643383515531122864487687325699022847401349887930148743573324935926928038313274118780408648738039221552094286634605859059178302001609063051473627616638916986695467354111409872240377448596882457893385604785675125747799925704521626444410537947809612042456889585358493384339367397319430466068497259487250009990226983476303290182605464299191144834590976900215207184217290547939862278004600042733098488459539365766704233072540349790910413963048455919274571091600102388120830520138960918238026360535071025369430480571742657175404099199715243175884516644982333939980185912023546244471169175154232817129035386017126612372249727029415960226233559719423091324216204450258205068541788852628274234145551296430267231614519548422573530219115334315814963859220772016127905259664901596292960624357867171314754794327851281187544474064975723856559261998464776997766060743177575177887669315503357551968629467385805021951612024977234727345596392623743650652821546588655008745166946053122284578514217986359571614564066926642313354080995691975795001216172305325876725224168312406194959039369235312034402577256851071978396414051789255283218481832719899346830805746190414783360826783268783647194536819312776246291560322670087150506500394384601386261997874464354519244142316219347283660495896740247449925747}, - {2, - 649437634112526011781048553400039087345326903154523759350403591156064026924461442249265424643851483793427933209842971350578174113891441825577284685965852170818096276564472575980385637569538579237402539408651576344680653270161629667112355235907244563250700929011643383515531122864487687325699022847401349887930148743573324935926928038313274118780408648738039221552094286634605859059178302001609063051473627616638916986695467354111409872240377448596882457893385604785675125747799925704521626444410537947809612042456889585358493384339367397319430466068497259487250009990226983476303290182605464299191144834590976900215207184217290547939862278004600042733098488459539365766704233072540349790910413963048455919274571091600102388120830520138960918238026360535071025369430480571742657175404099199715243175884516644982333939980185912023546244471169175154232817129035386017126612372249727029415960226233559719423091324216204450258205068541788852628274234145551296430267231614519548422573530219115334315814963859220772016127905259664901596292960624357867171314754794327851281187544474064975723856559261998464776997766060743177575177887669315503357551968629467385805021951612024977234727345596392623743650652821546588655008745166946053122284578514217986359571614564066926642313354080995691975795001216172305325876725224168312406194959039369235312034402577256851071978396414051789255283218481832719899346830805746190414783360826783268783647194536819312776246291560322670087150506500394384601386261997874464354519244142316219347283660495896740247491264043}, - {2, - 649437634112526011781048553400039087345326903154523759350403591156064026924461442249265424643851483793427933209842971350578174113891441825577284685965852170818096276564472575980385637569538579237402539408651576344680653270161629667112355235907244563250700929011643383515531122864487687325699022847401349887930148743573324935926928038313274118780408648738039221552094286634605859059178302001609063051473627616638916986695467354111409872240377448596882457893385604785675125747799925704521626444410537947809612042456889585358493384339367397319430466068497259487250009990226983476303290182605464299191144834590976900215207184217290547939862278004600042733098488459539365766704233072540349790910413963048455919274571091600102388120830520138960918238026360535071025369430480571742657175404099199715243175884516644982333939980185912023546244471169175154232817129035386017126612372249727029415960226233559719423091324216204450258205068541788852628274234145551296430267231614519548422573530219115334315814963859220772016127905259664901596292960624357867171314754794327851281187544474064975723856559261998464776997766060743177575177887669315503357551968629467385805021951612024977234727345596392623743650652821546588655008745166946053122284578514217986359571614564066926642313354080995691975795001216172305325876725224168312406194959039369235312034402577256851071978396414051789255283218481832719899346830805746190414783360826783268783647194536819312776246291560322670087150506500394384601386261997874464354519244142316219347283660495896740247565114563}, - {2, - 649437634112526011781048553400039087345326903154523759350403591156064026924461442249265424643851483793427933209842971350578174113891441825577284685965852170818096276564472575980385637569538579237402539408651576344680653270161629667112355235907244563250700929011643383515531122864487687325699022847401349887930148743573324935926928038313274118780408648738039221552094286634605859059178302001609063051473627616638916986695467354111409872240377448596882457893385604785675125747799925704521626444410537947809612042456889585358493384339367397319430466068497259487250009990226983476303290182605464299191144834590976900215207184217290547939862278004600042733098488459539365766704233072540349790910413963048455919274571091600102388120830520138960918238026360535071025369430480571742657175404099199715243175884516644982333939980185912023546244471169175154232817129035386017126612372249727029415960226233559719423091324216204450258205068541788852628274234145551296430267231614519548422573530219115334315814963859220772016127905259664901596292960624357867171314754794327851281187544474064975723856559261998464776997766060743177575177887669315503357551968629467385805021951612024977234727345596392623743650652821546588655008745166946053122284578514217986359571614564066926642313354080995691975795001216172305325876725224168312406194959039369235312034402577256851071978396414051789255283218481832719899346830805746190414783360826783268783647194536819312776246291560322670087150506500394384601386261997874464354519244142316219347283660495896740247602039643}, - {2, - 649437634112526011781048553400039087345326903154523759350403591156064026924461442249265424643851483793427933209842971350578174113891441825577284685965852170818096276564472575980385637569538579237402539408651576344680653270161629667112355235907244563250700929011643383515531122864487687325699022847401349887930148743573324935926928038313274118780408648738039221552094286634605859059178302001609063051473627616638916986695467354111409872240377448596882457893385604785675125747799925704521626444410537947809612042456889585358493384339367397319430466068497259487250009990226983476303290182605464299191144834590976900215207184217290547939862278004600042733098488459539365766704233072540349790910413963048455919274571091600102388120830520138960918238026360535071025369430480571742657175404099199715243175884516644982333939980185912023546244471169175154232817129035386017126612372249727029415960226233559719423091324216204450258205068541788852628274234145551296430267231614519548422573530219115334315814963859220772016127905259664901596292960624357867171314754794327851281187544474064975723856559261998464776997766060743177575177887669315503357551968629467385805021951612024977234727345596392623743650652821546588655008745166946053122284578514217986359571614564066926642313354080995691975795001216172305325876725224168312406194959039369235312034402577256851071978396414051789255283218481832719899346830805746190414783360826783268783647194536819312776246291560322670087150506500394384601386261997874464354519244142316219347283660495896740247648923259}, - {2, - 649437634112526011781048553400039087345326903154523759350403591156064026924461442249265424643851483793427933209842971350578174113891441825577284685965852170818096276564472575980385637569538579237402539408651576344680653270161629667112355235907244563250700929011643383515531122864487687325699022847401349887930148743573324935926928038313274118780408648738039221552094286634605859059178302001609063051473627616638916986695467354111409872240377448596882457893385604785675125747799925704521626444410537947809612042456889585358493384339367397319430466068497259487250009990226983476303290182605464299191144834590976900215207184217290547939862278004600042733098488459539365766704233072540349790910413963048455919274571091600102388120830520138960918238026360535071025369430480571742657175404099199715243175884516644982333939980185912023546244471169175154232817129035386017126612372249727029415960226233559719423091324216204450258205068541788852628274234145551296430267231614519548422573530219115334315814963859220772016127905259664901596292960624357867171314754794327851281187544474064975723856559261998464776997766060743177575177887669315503357551968629467385805021951612024977234727345596392623743650652821546588655008745166946053122284578514217986359571614564066926642313354080995691975795001216172305325876725224168312406194959039369235312034402577256851071978396414051789255283218481832719899346830805746190414783360826783268783647194536819312776246291560322670087150506500394384601386261997874464354519244142316219347283660495896740247676572003}, - {2, - 649437634112526011781048553400039087345326903154523759350403591156064026924461442249265424643851483793427933209842971350578174113891441825577284685965852170818096276564472575980385637569538579237402539408651576344680653270161629667112355235907244563250700929011643383515531122864487687325699022847401349887930148743573324935926928038313274118780408648738039221552094286634605859059178302001609063051473627616638916986695467354111409872240377448596882457893385604785675125747799925704521626444410537947809612042456889585358493384339367397319430466068497259487250009990226983476303290182605464299191144834590976900215207184217290547939862278004600042733098488459539365766704233072540349790910413963048455919274571091600102388120830520138960918238026360535071025369430480571742657175404099199715243175884516644982333939980185912023546244471169175154232817129035386017126612372249727029415960226233559719423091324216204450258205068541788852628274234145551296430267231614519548422573530219115334315814963859220772016127905259664901596292960624357867171314754794327851281187544474064975723856559261998464776997766060743177575177887669315503357551968629467385805021951612024977234727345596392623743650652821546588655008745166946053122284578514217986359571614564066926642313354080995691975795001216172305325876725224168312406194959039369235312034402577256851071978396414051789255283218481832719899346830805746190414783360826783268783647194536819312776246291560322670087150506500394384601386261997874464354519244142316219347283660495896740247749635347}, - {2, - 649437634112526011781048553400039087345326903154523759350403591156064026924461442249265424643851483793427933209842971350578174113891441825577284685965852170818096276564472575980385637569538579237402539408651576344680653270161629667112355235907244563250700929011643383515531122864487687325699022847401349887930148743573324935926928038313274118780408648738039221552094286634605859059178302001609063051473627616638916986695467354111409872240377448596882457893385604785675125747799925704521626444410537947809612042456889585358493384339367397319430466068497259487250009990226983476303290182605464299191144834590976900215207184217290547939862278004600042733098488459539365766704233072540349790910413963048455919274571091600102388120830520138960918238026360535071025369430480571742657175404099199715243175884516644982333939980185912023546244471169175154232817129035386017126612372249727029415960226233559719423091324216204450258205068541788852628274234145551296430267231614519548422573530219115334315814963859220772016127905259664901596292960624357867171314754794327851281187544474064975723856559261998464776997766060743177575177887669315503357551968629467385805021951612024977234727345596392623743650652821546588655008745166946053122284578514217986359571614564066926642313354080995691975795001216172305325876725224168312406194959039369235312034402577256851071978396414051789255283218481832719899346830805746190414783360826783268783647194536819312776246291560322670087150506500394384601386261997874464354519244142316219347283660495896740247795316107}, - {2, - 649437634112526011781048553400039087345326903154523759350403591156064026924461442249265424643851483793427933209842971350578174113891441825577284685965852170818096276564472575980385637569538579237402539408651576344680653270161629667112355235907244563250700929011643383515531122864487687325699022847401349887930148743573324935926928038313274118780408648738039221552094286634605859059178302001609063051473627616638916986695467354111409872240377448596882457893385604785675125747799925704521626444410537947809612042456889585358493384339367397319430466068497259487250009990226983476303290182605464299191144834590976900215207184217290547939862278004600042733098488459539365766704233072540349790910413963048455919274571091600102388120830520138960918238026360535071025369430480571742657175404099199715243175884516644982333939980185912023546244471169175154232817129035386017126612372249727029415960226233559719423091324216204450258205068541788852628274234145551296430267231614519548422573530219115334315814963859220772016127905259664901596292960624357867171314754794327851281187544474064975723856559261998464776997766060743177575177887669315503357551968629467385805021951612024977234727345596392623743650652821546588655008745166946053122284578514217986359571614564066926642313354080995691975795001216172305325876725224168312406194959039369235312034402577256851071978396414051789255283218481832719899346830805746190414783360826783268783647194536819312776246291560322670087150506500394384601386261997874464354519244142316219347283660495896740247845693523}, - {2, - 649437634112526011781048553400039087345326903154523759350403591156064026924461442249265424643851483793427933209842971350578174113891441825577284685965852170818096276564472575980385637569538579237402539408651576344680653270161629667112355235907244563250700929011643383515531122864487687325699022847401349887930148743573324935926928038313274118780408648738039221552094286634605859059178302001609063051473627616638916986695467354111409872240377448596882457893385604785675125747799925704521626444410537947809612042456889585358493384339367397319430466068497259487250009990226983476303290182605464299191144834590976900215207184217290547939862278004600042733098488459539365766704233072540349790910413963048455919274571091600102388120830520138960918238026360535071025369430480571742657175404099199715243175884516644982333939980185912023546244471169175154232817129035386017126612372249727029415960226233559719423091324216204450258205068541788852628274234145551296430267231614519548422573530219115334315814963859220772016127905259664901596292960624357867171314754794327851281187544474064975723856559261998464776997766060743177575177887669315503357551968629467385805021951612024977234727345596392623743650652821546588655008745166946053122284578514217986359571614564066926642313354080995691975795001216172305325876725224168312406194959039369235312034402577256851071978396414051789255283218481832719899346830805746190414783360826783268783647194536819312776246291560322670087150506500394384601386261997874464354519244142316219347283660495896740247846693459}, - {2, - 649437634112526011781048553400039087345326903154523759350403591156064026924461442249265424643851483793427933209842971350578174113891441825577284685965852170818096276564472575980385637569538579237402539408651576344680653270161629667112355235907244563250700929011643383515531122864487687325699022847401349887930148743573324935926928038313274118780408648738039221552094286634605859059178302001609063051473627616638916986695467354111409872240377448596882457893385604785675125747799925704521626444410537947809612042456889585358493384339367397319430466068497259487250009990226983476303290182605464299191144834590976900215207184217290547939862278004600042733098488459539365766704233072540349790910413963048455919274571091600102388120830520138960918238026360535071025369430480571742657175404099199715243175884516644982333939980185912023546244471169175154232817129035386017126612372249727029415960226233559719423091324216204450258205068541788852628274234145551296430267231614519548422573530219115334315814963859220772016127905259664901596292960624357867171314754794327851281187544474064975723856559261998464776997766060743177575177887669315503357551968629467385805021951612024977234727345596392623743650652821546588655008745166946053122284578514217986359571614564066926642313354080995691975795001216172305325876725224168312406194959039369235312034402577256851071978396414051789255283218481832719899346830805746190414783360826783268783647194536819312776246291560322670087150506500394384601386261997874464354519244142316219347283660495896740247903649683}, - {5, - 649437634112526011781048553400039087345326903154523759350403591156064026924461442249265424643851483793427933209842971350578174113891441825577284685965852170818096276564472575980385637569538579237402539408651576344680653270161629667112355235907244563250700929011643383515531122864487687325699022847401349887930148743573324935926928038313274118780408648738039221552094286634605859059178302001609063051473627616638916986695467354111409872240377448596882457893385604785675125747799925704521626444410537947809612042456889585358493384339367397319430466068497259487250009990226983476303290182605464299191144834590976900215207184217290547939862278004600042733098488459539365766704233072540349790910413963048455919274571091600102388120830520138960918238026360535071025369430480571742657175404099199715243175884516644982333939980185912023546244471169175154232817129035386017126612372249727029415960226233559719423091324216204450258205068541788852628274234145551296430267231614519548422573530219115334315814963859220772016127905259664901596292960624357867171314754794327851281187544474064975723856559261998464776997766060743177575177887669315503357551968629467385805021951612024977234727345596392623743650652821546588655008745166946053122284578514217986359571614564066926642313354080995691975795001216172305325876725224168312406194959039369235312034402577256851071978396414051789255283218481832719899346830805746190414783360826783268783647194536819312776246291560322670087150506500394384601386261997874464354519244142316219347283660495896740247105253887}, - {5, - 649437634112526011781048553400039087345326903154523759350403591156064026924461442249265424643851483793427933209842971350578174113891441825577284685965852170818096276564472575980385637569538579237402539408651576344680653270161629667112355235907244563250700929011643383515531122864487687325699022847401349887930148743573324935926928038313274118780408648738039221552094286634605859059178302001609063051473627616638916986695467354111409872240377448596882457893385604785675125747799925704521626444410537947809612042456889585358493384339367397319430466068497259487250009990226983476303290182605464299191144834590976900215207184217290547939862278004600042733098488459539365766704233072540349790910413963048455919274571091600102388120830520138960918238026360535071025369430480571742657175404099199715243175884516644982333939980185912023546244471169175154232817129035386017126612372249727029415960226233559719423091324216204450258205068541788852628274234145551296430267231614519548422573530219115334315814963859220772016127905259664901596292960624357867171314754794327851281187544474064975723856559261998464776997766060743177575177887669315503357551968629467385805021951612024977234727345596392623743650652821546588655008745166946053122284578514217986359571614564066926642313354080995691975795001216172305325876725224168312406194959039369235312034402577256851071978396414051789255283218481832719899346830805746190414783360826783268783647194536819312776246291560322670087150506500394384601386261997874464354519244142316219347283660495896740247123422007}, - {5, - 649437634112526011781048553400039087345326903154523759350403591156064026924461442249265424643851483793427933209842971350578174113891441825577284685965852170818096276564472575980385637569538579237402539408651576344680653270161629667112355235907244563250700929011643383515531122864487687325699022847401349887930148743573324935926928038313274118780408648738039221552094286634605859059178302001609063051473627616638916986695467354111409872240377448596882457893385604785675125747799925704521626444410537947809612042456889585358493384339367397319430466068497259487250009990226983476303290182605464299191144834590976900215207184217290547939862278004600042733098488459539365766704233072540349790910413963048455919274571091600102388120830520138960918238026360535071025369430480571742657175404099199715243175884516644982333939980185912023546244471169175154232817129035386017126612372249727029415960226233559719423091324216204450258205068541788852628274234145551296430267231614519548422573530219115334315814963859220772016127905259664901596292960624357867171314754794327851281187544474064975723856559261998464776997766060743177575177887669315503357551968629467385805021951612024977234727345596392623743650652821546588655008745166946053122284578514217986359571614564066926642313354080995691975795001216172305325876725224168312406194959039369235312034402577256851071978396414051789255283218481832719899346830805746190414783360826783268783647194536819312776246291560322670087150506500394384601386261997874464354519244142316219347283660495896740247138876567}, - {5, - 649437634112526011781048553400039087345326903154523759350403591156064026924461442249265424643851483793427933209842971350578174113891441825577284685965852170818096276564472575980385637569538579237402539408651576344680653270161629667112355235907244563250700929011643383515531122864487687325699022847401349887930148743573324935926928038313274118780408648738039221552094286634605859059178302001609063051473627616638916986695467354111409872240377448596882457893385604785675125747799925704521626444410537947809612042456889585358493384339367397319430466068497259487250009990226983476303290182605464299191144834590976900215207184217290547939862278004600042733098488459539365766704233072540349790910413963048455919274571091600102388120830520138960918238026360535071025369430480571742657175404099199715243175884516644982333939980185912023546244471169175154232817129035386017126612372249727029415960226233559719423091324216204450258205068541788852628274234145551296430267231614519548422573530219115334315814963859220772016127905259664901596292960624357867171314754794327851281187544474064975723856559261998464776997766060743177575177887669315503357551968629467385805021951612024977234727345596392623743650652821546588655008745166946053122284578514217986359571614564066926642313354080995691975795001216172305325876725224168312406194959039369235312034402577256851071978396414051789255283218481832719899346830805746190414783360826783268783647194536819312776246291560322670087150506500394384601386261997874464354519244142316219347283660495896740247150015567}, - {5, - 649437634112526011781048553400039087345326903154523759350403591156064026924461442249265424643851483793427933209842971350578174113891441825577284685965852170818096276564472575980385637569538579237402539408651576344680653270161629667112355235907244563250700929011643383515531122864487687325699022847401349887930148743573324935926928038313274118780408648738039221552094286634605859059178302001609063051473627616638916986695467354111409872240377448596882457893385604785675125747799925704521626444410537947809612042456889585358493384339367397319430466068497259487250009990226983476303290182605464299191144834590976900215207184217290547939862278004600042733098488459539365766704233072540349790910413963048455919274571091600102388120830520138960918238026360535071025369430480571742657175404099199715243175884516644982333939980185912023546244471169175154232817129035386017126612372249727029415960226233559719423091324216204450258205068541788852628274234145551296430267231614519548422573530219115334315814963859220772016127905259664901596292960624357867171314754794327851281187544474064975723856559261998464776997766060743177575177887669315503357551968629467385805021951612024977234727345596392623743650652821546588655008745166946053122284578514217986359571614564066926642313354080995691975795001216172305325876725224168312406194959039369235312034402577256851071978396414051789255283218481832719899346830805746190414783360826783268783647194536819312776246291560322670087150506500394384601386261997874464354519244142316219347283660495896740247182428143}, - {5, - 649437634112526011781048553400039087345326903154523759350403591156064026924461442249265424643851483793427933209842971350578174113891441825577284685965852170818096276564472575980385637569538579237402539408651576344680653270161629667112355235907244563250700929011643383515531122864487687325699022847401349887930148743573324935926928038313274118780408648738039221552094286634605859059178302001609063051473627616638916986695467354111409872240377448596882457893385604785675125747799925704521626444410537947809612042456889585358493384339367397319430466068497259487250009990226983476303290182605464299191144834590976900215207184217290547939862278004600042733098488459539365766704233072540349790910413963048455919274571091600102388120830520138960918238026360535071025369430480571742657175404099199715243175884516644982333939980185912023546244471169175154232817129035386017126612372249727029415960226233559719423091324216204450258205068541788852628274234145551296430267231614519548422573530219115334315814963859220772016127905259664901596292960624357867171314754794327851281187544474064975723856559261998464776997766060743177575177887669315503357551968629467385805021951612024977234727345596392623743650652821546588655008745166946053122284578514217986359571614564066926642313354080995691975795001216172305325876725224168312406194959039369235312034402577256851071978396414051789255283218481832719899346830805746190414783360826783268783647194536819312776246291560322670087150506500394384601386261997874464354519244142316219347283660495896740247202352967}, - {5, - 649437634112526011781048553400039087345326903154523759350403591156064026924461442249265424643851483793427933209842971350578174113891441825577284685965852170818096276564472575980385637569538579237402539408651576344680653270161629667112355235907244563250700929011643383515531122864487687325699022847401349887930148743573324935926928038313274118780408648738039221552094286634605859059178302001609063051473627616638916986695467354111409872240377448596882457893385604785675125747799925704521626444410537947809612042456889585358493384339367397319430466068497259487250009990226983476303290182605464299191144834590976900215207184217290547939862278004600042733098488459539365766704233072540349790910413963048455919274571091600102388120830520138960918238026360535071025369430480571742657175404099199715243175884516644982333939980185912023546244471169175154232817129035386017126612372249727029415960226233559719423091324216204450258205068541788852628274234145551296430267231614519548422573530219115334315814963859220772016127905259664901596292960624357867171314754794327851281187544474064975723856559261998464776997766060743177575177887669315503357551968629467385805021951612024977234727345596392623743650652821546588655008745166946053122284578514217986359571614564066926642313354080995691975795001216172305325876725224168312406194959039369235312034402577256851071978396414051789255283218481832719899346830805746190414783360826783268783647194536819312776246291560322670087150506500394384601386261997874464354519244142316219347283660495896740247251099967}, - {5, - 649437634112526011781048553400039087345326903154523759350403591156064026924461442249265424643851483793427933209842971350578174113891441825577284685965852170818096276564472575980385637569538579237402539408651576344680653270161629667112355235907244563250700929011643383515531122864487687325699022847401349887930148743573324935926928038313274118780408648738039221552094286634605859059178302001609063051473627616638916986695467354111409872240377448596882457893385604785675125747799925704521626444410537947809612042456889585358493384339367397319430466068497259487250009990226983476303290182605464299191144834590976900215207184217290547939862278004600042733098488459539365766704233072540349790910413963048455919274571091600102388120830520138960918238026360535071025369430480571742657175404099199715243175884516644982333939980185912023546244471169175154232817129035386017126612372249727029415960226233559719423091324216204450258205068541788852628274234145551296430267231614519548422573530219115334315814963859220772016127905259664901596292960624357867171314754794327851281187544474064975723856559261998464776997766060743177575177887669315503357551968629467385805021951612024977234727345596392623743650652821546588655008745166946053122284578514217986359571614564066926642313354080995691975795001216172305325876725224168312406194959039369235312034402577256851071978396414051789255283218481832719899346830805746190414783360826783268783647194536819312776246291560322670087150506500394384601386261997874464354519244142316219347283660495896740247277733823}, - {5, - 649437634112526011781048553400039087345326903154523759350403591156064026924461442249265424643851483793427933209842971350578174113891441825577284685965852170818096276564472575980385637569538579237402539408651576344680653270161629667112355235907244563250700929011643383515531122864487687325699022847401349887930148743573324935926928038313274118780408648738039221552094286634605859059178302001609063051473627616638916986695467354111409872240377448596882457893385604785675125747799925704521626444410537947809612042456889585358493384339367397319430466068497259487250009990226983476303290182605464299191144834590976900215207184217290547939862278004600042733098488459539365766704233072540349790910413963048455919274571091600102388120830520138960918238026360535071025369430480571742657175404099199715243175884516644982333939980185912023546244471169175154232817129035386017126612372249727029415960226233559719423091324216204450258205068541788852628274234145551296430267231614519548422573530219115334315814963859220772016127905259664901596292960624357867171314754794327851281187544474064975723856559261998464776997766060743177575177887669315503357551968629467385805021951612024977234727345596392623743650652821546588655008745166946053122284578514217986359571614564066926642313354080995691975795001216172305325876725224168312406194959039369235312034402577256851071978396414051789255283218481832719899346830805746190414783360826783268783647194536819312776246291560322670087150506500394384601386261997874464354519244142316219347283660495896740247373770207}, - {5, - 649437634112526011781048553400039087345326903154523759350403591156064026924461442249265424643851483793427933209842971350578174113891441825577284685965852170818096276564472575980385637569538579237402539408651576344680653270161629667112355235907244563250700929011643383515531122864487687325699022847401349887930148743573324935926928038313274118780408648738039221552094286634605859059178302001609063051473627616638916986695467354111409872240377448596882457893385604785675125747799925704521626444410537947809612042456889585358493384339367397319430466068497259487250009990226983476303290182605464299191144834590976900215207184217290547939862278004600042733098488459539365766704233072540349790910413963048455919274571091600102388120830520138960918238026360535071025369430480571742657175404099199715243175884516644982333939980185912023546244471169175154232817129035386017126612372249727029415960226233559719423091324216204450258205068541788852628274234145551296430267231614519548422573530219115334315814963859220772016127905259664901596292960624357867171314754794327851281187544474064975723856559261998464776997766060743177575177887669315503357551968629467385805021951612024977234727345596392623743650652821546588655008745166946053122284578514217986359571614564066926642313354080995691975795001216172305325876725224168312406194959039369235312034402577256851071978396414051789255283218481832719899346830805746190414783360826783268783647194536819312776246291560322670087150506500394384601386261997874464354519244142316219347283660495896740247486491463}, - {5, - 649437634112526011781048553400039087345326903154523759350403591156064026924461442249265424643851483793427933209842971350578174113891441825577284685965852170818096276564472575980385637569538579237402539408651576344680653270161629667112355235907244563250700929011643383515531122864487687325699022847401349887930148743573324935926928038313274118780408648738039221552094286634605859059178302001609063051473627616638916986695467354111409872240377448596882457893385604785675125747799925704521626444410537947809612042456889585358493384339367397319430466068497259487250009990226983476303290182605464299191144834590976900215207184217290547939862278004600042733098488459539365766704233072540349790910413963048455919274571091600102388120830520138960918238026360535071025369430480571742657175404099199715243175884516644982333939980185912023546244471169175154232817129035386017126612372249727029415960226233559719423091324216204450258205068541788852628274234145551296430267231614519548422573530219115334315814963859220772016127905259664901596292960624357867171314754794327851281187544474064975723856559261998464776997766060743177575177887669315503357551968629467385805021951612024977234727345596392623743650652821546588655008745166946053122284578514217986359571614564066926642313354080995691975795001216172305325876725224168312406194959039369235312034402577256851071978396414051789255283218481832719899346830805746190414783360826783268783647194536819312776246291560322670087150506500394384601386261997874464354519244142316219347283660495896740247595077447}, - {5, - 649437634112526011781048553400039087345326903154523759350403591156064026924461442249265424643851483793427933209842971350578174113891441825577284685965852170818096276564472575980385637569538579237402539408651576344680653270161629667112355235907244563250700929011643383515531122864487687325699022847401349887930148743573324935926928038313274118780408648738039221552094286634605859059178302001609063051473627616638916986695467354111409872240377448596882457893385604785675125747799925704521626444410537947809612042456889585358493384339367397319430466068497259487250009990226983476303290182605464299191144834590976900215207184217290547939862278004600042733098488459539365766704233072540349790910413963048455919274571091600102388120830520138960918238026360535071025369430480571742657175404099199715243175884516644982333939980185912023546244471169175154232817129035386017126612372249727029415960226233559719423091324216204450258205068541788852628274234145551296430267231614519548422573530219115334315814963859220772016127905259664901596292960624357867171314754794327851281187544474064975723856559261998464776997766060743177575177887669315503357551968629467385805021951612024977234727345596392623743650652821546588655008745166946053122284578514217986359571614564066926642313354080995691975795001216172305325876725224168312406194959039369235312034402577256851071978396414051789255283218481832719899346830805746190414783360826783268783647194536819312776246291560322670087150506500394384601386261997874464354519244142316219347283660495896740247618301767}, - {5, - 649437634112526011781048553400039087345326903154523759350403591156064026924461442249265424643851483793427933209842971350578174113891441825577284685965852170818096276564472575980385637569538579237402539408651576344680653270161629667112355235907244563250700929011643383515531122864487687325699022847401349887930148743573324935926928038313274118780408648738039221552094286634605859059178302001609063051473627616638916986695467354111409872240377448596882457893385604785675125747799925704521626444410537947809612042456889585358493384339367397319430466068497259487250009990226983476303290182605464299191144834590976900215207184217290547939862278004600042733098488459539365766704233072540349790910413963048455919274571091600102388120830520138960918238026360535071025369430480571742657175404099199715243175884516644982333939980185912023546244471169175154232817129035386017126612372249727029415960226233559719423091324216204450258205068541788852628274234145551296430267231614519548422573530219115334315814963859220772016127905259664901596292960624357867171314754794327851281187544474064975723856559261998464776997766060743177575177887669315503357551968629467385805021951612024977234727345596392623743650652821546588655008745166946053122284578514217986359571614564066926642313354080995691975795001216172305325876725224168312406194959039369235312034402577256851071978396414051789255283218481832719899346830805746190414783360826783268783647194536819312776246291560322670087150506500394384601386261997874464354519244142316219347283660495896740247645016047}, - {5, - 649437634112526011781048553400039087345326903154523759350403591156064026924461442249265424643851483793427933209842971350578174113891441825577284685965852170818096276564472575980385637569538579237402539408651576344680653270161629667112355235907244563250700929011643383515531122864487687325699022847401349887930148743573324935926928038313274118780408648738039221552094286634605859059178302001609063051473627616638916986695467354111409872240377448596882457893385604785675125747799925704521626444410537947809612042456889585358493384339367397319430466068497259487250009990226983476303290182605464299191144834590976900215207184217290547939862278004600042733098488459539365766704233072540349790910413963048455919274571091600102388120830520138960918238026360535071025369430480571742657175404099199715243175884516644982333939980185912023546244471169175154232817129035386017126612372249727029415960226233559719423091324216204450258205068541788852628274234145551296430267231614519548422573530219115334315814963859220772016127905259664901596292960624357867171314754794327851281187544474064975723856559261998464776997766060743177575177887669315503357551968629467385805021951612024977234727345596392623743650652821546588655008745166946053122284578514217986359571614564066926642313354080995691975795001216172305325876725224168312406194959039369235312034402577256851071978396414051789255283218481832719899346830805746190414783360826783268783647194536819312776246291560322670087150506500394384601386261997874464354519244142316219347283660495896740247783283863}, - {5, - 649437634112526011781048553400039087345326903154523759350403591156064026924461442249265424643851483793427933209842971350578174113891441825577284685965852170818096276564472575980385637569538579237402539408651576344680653270161629667112355235907244563250700929011643383515531122864487687325699022847401349887930148743573324935926928038313274118780408648738039221552094286634605859059178302001609063051473627616638916986695467354111409872240377448596882457893385604785675125747799925704521626444410537947809612042456889585358493384339367397319430466068497259487250009990226983476303290182605464299191144834590976900215207184217290547939862278004600042733098488459539365766704233072540349790910413963048455919274571091600102388120830520138960918238026360535071025369430480571742657175404099199715243175884516644982333939980185912023546244471169175154232817129035386017126612372249727029415960226233559719423091324216204450258205068541788852628274234145551296430267231614519548422573530219115334315814963859220772016127905259664901596292960624357867171314754794327851281187544474064975723856559261998464776997766060743177575177887669315503357551968629467385805021951612024977234727345596392623743650652821546588655008745166946053122284578514217986359571614564066926642313354080995691975795001216172305325876725224168312406194959039369235312034402577256851071978396414051789255283218481832719899346830805746190414783360826783268783647194536819312776246291560322670087150506500394384601386261997874464354519244142316219347283660495896740247816131343}, - {5, - 649437634112526011781048553400039087345326903154523759350403591156064026924461442249265424643851483793427933209842971350578174113891441825577284685965852170818096276564472575980385637569538579237402539408651576344680653270161629667112355235907244563250700929011643383515531122864487687325699022847401349887930148743573324935926928038313274118780408648738039221552094286634605859059178302001609063051473627616638916986695467354111409872240377448596882457893385604785675125747799925704521626444410537947809612042456889585358493384339367397319430466068497259487250009990226983476303290182605464299191144834590976900215207184217290547939862278004600042733098488459539365766704233072540349790910413963048455919274571091600102388120830520138960918238026360535071025369430480571742657175404099199715243175884516644982333939980185912023546244471169175154232817129035386017126612372249727029415960226233559719423091324216204450258205068541788852628274234145551296430267231614519548422573530219115334315814963859220772016127905259664901596292960624357867171314754794327851281187544474064975723856559261998464776997766060743177575177887669315503357551968629467385805021951612024977234727345596392623743650652821546588655008745166946053122284578514217986359571614564066926642313354080995691975795001216172305325876725224168312406194959039369235312034402577256851071978396414051789255283218481832719899346830805746190414783360826783268783647194536819312776246291560322670087150506500394384601386261997874464354519244142316219347283660495896740247874345167}]}, {6143, [{2, 30210369155779651888589832154992934074138507220822617750684652871127582962409899985645946445695043253195686248244998571402305199678770384280564307485035486218800068429843963439425551779021865465473175487387006425679517736862515996424867831461548685177440883895142510892080386024242790580351718476912346064665887342021801124592374644199240055614707801951400286738571835106941459565263538011604632591754698655670731747906958754306695890917250845082935314433013066321817795811216868330076422883334026288452966305998174700518168814979477732850177021088448137305837683404856702535774772361477172629825124721712806311678193131143271354379844514551970600366748790881625770009341628054007754553253798271412759265676226769394213651014204828629926952929904315923670432908575989917617637701451215022415020602741724294119817098286839062394376589141071497906909592069997336204239169602926838746847024777759842015748321319275357899423502183072151966760234482986263033779816032715802142058060420345923767082809675434739360245514122590144988299733345731573144371526046690587022732405227784184600940160738238518293781021765582962691631713434222022562950296250509633938674749281309291074381760671163336862668582157228513024763628253337784730104842959368278069041285218519766922739574404159050610285845118633942864745158438604557438873320358729699870593735296129291292133537433344339822168319576836729204869998368045864710009464349668393627057504172016907834801787571594303040666608785976162184230007930556331766254151014073812956419139773303370139752142739047952426344593274617190212675329544654619143794215577626176639637678757668518842586496846866347512471156638951909845841212416559374312458890749201523393740959899515314744619218832833815902335851353279396321673115498490049357132026679171719767095122045794337383943188601986364730890641872503895354104066212697067}, @@ -480,71 +411,6 @@ 30210369155779651888589832154992934074138507220822617750684652871127582962409899985645946445695043253195686248244998571402305199678770384280564307485035486218800068429843963439425551779021865465473175487387006425679517736862515996424867831461548685177440883895142510892080386024242790580351718476912346064665887342021801124592374644199240055614707801951400286738571835106941459565263538011604632591754698655670731747906958754306695890917250845082935314433013066321817795811216868330076422883334026288452966305998174700518168814979477732850177021088448137305837683404856702535774772361477172629825124721712806311678193131143271354379844514551970600366748790881625770009341628054007754553253798271412759265676226769394213651014204828629926952929904315923670432908575989917617637701451215022415020602741724294119817098286839062394376589141071497906909592069997336204239169602926838746847024777759842015748321319275357899423502183072151966760234482986263033779816032715802142058060420345923767082809675434739360245514122590144988299733345731573144371526046690587022732405227784184600940160738238518293781021765582962691631713434222022562950296250509633938674749281309291074381760671163336862668582157228513024763628253337784730104842959368278069041285218519766922739574404159050610285845118633942864745158438604557438873320358729699870593735296129291292133537433344339822168319576836729204869998368045864710009464349668393627057504172016907834801787571594303040666608785976162184230007930556331766254151014073812956419139773303370139752142739047952426344593274617190212675329544654619143794215577626176639637678757668518842586496846866347512471156638951909845841212416559374312458890749201523393740959899515314744619218832833815902335851353279396321673115498490049357132026679171719767095122045794337383943188601986364730890641872503895354104067042481887}, {5, 30210369155779651888589832154992934074138507220822617750684652871127582962409899985645946445695043253195686248244998571402305199678770384280564307485035486218800068429843963439425551779021865465473175487387006425679517736862515996424867831461548685177440883895142510892080386024242790580351718476912346064665887342021801124592374644199240055614707801951400286738571835106941459565263538011604632591754698655670731747906958754306695890917250845082935314433013066321817795811216868330076422883334026288452966305998174700518168814979477732850177021088448137305837683404856702535774772361477172629825124721712806311678193131143271354379844514551970600366748790881625770009341628054007754553253798271412759265676226769394213651014204828629926952929904315923670432908575989917617637701451215022415020602741724294119817098286839062394376589141071497906909592069997336204239169602926838746847024777759842015748321319275357899423502183072151966760234482986263033779816032715802142058060420345923767082809675434739360245514122590144988299733345731573144371526046690587022732405227784184600940160738238518293781021765582962691631713434222022562950296250509633938674749281309291074381760671163336862668582157228513024763628253337784730104842959368278069041285218519766922739574404159050610285845118633942864745158438604557438873320358729699870593735296129291292133537433344339822168319576836729204869998368045864710009464349668393627057504172016907834801787571594303040666608785976162184230007930556331766254151014073812956419139773303370139752142739047952426344593274617190212675329544654619143794215577626176639637678757668518842586496846866347512471156638951909845841212416559374312458890749201523393740959899515314744619218832833815902335851353279396321673115498490049357132026679171719767095122045794337383943188601986364730890641872503895354104067188249103}]}, - {7167, - [{2, - 5483947689873238016293916031421210417715226198450458695278359214939920808903888754748291585234470514731095293158971577871290333956546390471923967166891926796058037943347061391932048683243406115629059207198430761405517792845307599528545123275907240829958638010654676409407473464540147054022768326380533144139109013796005070619515692497303229683409723476100264437337159592967071649572068146217379004726298805073131642749312292639037974687912276595434011484665633472741970437405242931540792817150021104152269103538845906716294609703037973874968257574699425342614963850129729638055165011318321314629257017448744791130817504976352049315558175084707575517905220240735436472311376085977023193255767115772199401681050675227245864337841813577099951328506716260252484525951496988402157538736606893316558602219804539468211692272341101231081798647407433475330661334010047363328925076029072733284928506274503591566463356567449951050328939372527077820435552083641057965091365289077453354175453158803766717082697605135002698550392387018215808881309906132288850897899530025875371847565845964186403727130406970585738244224109459935326058926178367135154001854890789551481230848123840375019517586126229348925204229138925157417388983071508190365878773959469248844627811548238703202907823260627705881418371964608641916875906309702255332357870246656733850484860625288965209661089574538795660416296959228913342040037715375469749366782116594607018721130210724633134051682028201789300859780675503521182677498151404274736590115271652257881815395804591347270645487515405426176207583246761514846190105223591830994843572131787042752062568982393789415091692293114059135480055118138148767250658263869768994686391725791746769853583344796533031355717997717521032986544321281377626382079046164167636841143008587025918057095843339228816609264878115677918487854067887356202214595419884175437682477994901012403352228510424776160081628734502208471530373704829323453838286290450475142257346198410524038065099740082374843587568140564545670658917308004231061168599338344693117101685656038296427353228994834347618154742061252460559481017139006630352773110812670787299045754161442872626284680505796219}, - {2, - 5483947689873238016293916031421210417715226198450458695278359214939920808903888754748291585234470514731095293158971577871290333956546390471923967166891926796058037943347061391932048683243406115629059207198430761405517792845307599528545123275907240829958638010654676409407473464540147054022768326380533144139109013796005070619515692497303229683409723476100264437337159592967071649572068146217379004726298805073131642749312292639037974687912276595434011484665633472741970437405242931540792817150021104152269103538845906716294609703037973874968257574699425342614963850129729638055165011318321314629257017448744791130817504976352049315558175084707575517905220240735436472311376085977023193255767115772199401681050675227245864337841813577099951328506716260252484525951496988402157538736606893316558602219804539468211692272341101231081798647407433475330661334010047363328925076029072733284928506274503591566463356567449951050328939372527077820435552083641057965091365289077453354175453158803766717082697605135002698550392387018215808881309906132288850897899530025875371847565845964186403727130406970585738244224109459935326058926178367135154001854890789551481230848123840375019517586126229348925204229138925157417388983071508190365878773959469248844627811548238703202907823260627705881418371964608641916875906309702255332357870246656733850484860625288965209661089574538795660416296959228913342040037715375469749366782116594607018721130210724633134051682028201789300859780675503521182677498151404274736590115271652257881815395804591347270645487515405426176207583246761514846190105223591830994843572131787042752062568982393789415091692293114059135480055118138148767250658263869768994686391725791746769853583344796533031355717997717521032986544321281377626382079046164167636841143008587025918057095843339228816609264878115677918487854067887356202214595419884175437682477994901012403352228510424776160081628734502208471530373704829323453838286290450475142257346198410524038065099740082374843587568140564545670658917308004231061168599338344693117101685656038296427353228994834347618154742061252460559481017139006630352773110812670787299045754161442872626284680524856539}, - {2, - 5483947689873238016293916031421210417715226198450458695278359214939920808903888754748291585234470514731095293158971577871290333956546390471923967166891926796058037943347061391932048683243406115629059207198430761405517792845307599528545123275907240829958638010654676409407473464540147054022768326380533144139109013796005070619515692497303229683409723476100264437337159592967071649572068146217379004726298805073131642749312292639037974687912276595434011484665633472741970437405242931540792817150021104152269103538845906716294609703037973874968257574699425342614963850129729638055165011318321314629257017448744791130817504976352049315558175084707575517905220240735436472311376085977023193255767115772199401681050675227245864337841813577099951328506716260252484525951496988402157538736606893316558602219804539468211692272341101231081798647407433475330661334010047363328925076029072733284928506274503591566463356567449951050328939372527077820435552083641057965091365289077453354175453158803766717082697605135002698550392387018215808881309906132288850897899530025875371847565845964186403727130406970585738244224109459935326058926178367135154001854890789551481230848123840375019517586126229348925204229138925157417388983071508190365878773959469248844627811548238703202907823260627705881418371964608641916875906309702255332357870246656733850484860625288965209661089574538795660416296959228913342040037715375469749366782116594607018721130210724633134051682028201789300859780675503521182677498151404274736590115271652257881815395804591347270645487515405426176207583246761514846190105223591830994843572131787042752062568982393789415091692293114059135480055118138148767250658263869768994686391725791746769853583344796533031355717997717521032986544321281377626382079046164167636841143008587025918057095843339228816609264878115677918487854067887356202214595419884175437682477994901012403352228510424776160081628734502208471530373704829323453838286290450475142257346198410524038065099740082374843587568140564545670658917308004231061168599338344693117101685656038296427353228994834347618154742061252460559481017139006630352773110812670787299045754161442872626284680670831859}, - {2, - 5483947689873238016293916031421210417715226198450458695278359214939920808903888754748291585234470514731095293158971577871290333956546390471923967166891926796058037943347061391932048683243406115629059207198430761405517792845307599528545123275907240829958638010654676409407473464540147054022768326380533144139109013796005070619515692497303229683409723476100264437337159592967071649572068146217379004726298805073131642749312292639037974687912276595434011484665633472741970437405242931540792817150021104152269103538845906716294609703037973874968257574699425342614963850129729638055165011318321314629257017448744791130817504976352049315558175084707575517905220240735436472311376085977023193255767115772199401681050675227245864337841813577099951328506716260252484525951496988402157538736606893316558602219804539468211692272341101231081798647407433475330661334010047363328925076029072733284928506274503591566463356567449951050328939372527077820435552083641057965091365289077453354175453158803766717082697605135002698550392387018215808881309906132288850897899530025875371847565845964186403727130406970585738244224109459935326058926178367135154001854890789551481230848123840375019517586126229348925204229138925157417388983071508190365878773959469248844627811548238703202907823260627705881418371964608641916875906309702255332357870246656733850484860625288965209661089574538795660416296959228913342040037715375469749366782116594607018721130210724633134051682028201789300859780675503521182677498151404274736590115271652257881815395804591347270645487515405426176207583246761514846190105223591830994843572131787042752062568982393789415091692293114059135480055118138148767250658263869768994686391725791746769853583344796533031355717997717521032986544321281377626382079046164167636841143008587025918057095843339228816609264878115677918487854067887356202214595419884175437682477994901012403352228510424776160081628734502208471530373704829323453838286290450475142257346198410524038065099740082374843587568140564545670658917308004231061168599338344693117101685656038296427353228994834347618154742061252460559481017139006630352773110812670787299045754161442872626284680995675027}, - {2, - 5483947689873238016293916031421210417715226198450458695278359214939920808903888754748291585234470514731095293158971577871290333956546390471923967166891926796058037943347061391932048683243406115629059207198430761405517792845307599528545123275907240829958638010654676409407473464540147054022768326380533144139109013796005070619515692497303229683409723476100264437337159592967071649572068146217379004726298805073131642749312292639037974687912276595434011484665633472741970437405242931540792817150021104152269103538845906716294609703037973874968257574699425342614963850129729638055165011318321314629257017448744791130817504976352049315558175084707575517905220240735436472311376085977023193255767115772199401681050675227245864337841813577099951328506716260252484525951496988402157538736606893316558602219804539468211692272341101231081798647407433475330661334010047363328925076029072733284928506274503591566463356567449951050328939372527077820435552083641057965091365289077453354175453158803766717082697605135002698550392387018215808881309906132288850897899530025875371847565845964186403727130406970585738244224109459935326058926178367135154001854890789551481230848123840375019517586126229348925204229138925157417388983071508190365878773959469248844627811548238703202907823260627705881418371964608641916875906309702255332357870246656733850484860625288965209661089574538795660416296959228913342040037715375469749366782116594607018721130210724633134051682028201789300859780675503521182677498151404274736590115271652257881815395804591347270645487515405426176207583246761514846190105223591830994843572131787042752062568982393789415091692293114059135480055118138148767250658263869768994686391725791746769853583344796533031355717997717521032986544321281377626382079046164167636841143008587025918057095843339228816609264878115677918487854067887356202214595419884175437682477994901012403352228510424776160081628734502208471530373704829323453838286290450475142257346198410524038065099740082374843587568140564545670658917308004231061168599338344693117101685656038296427353228994834347618154742061252460559481017139006630352773110812670787299045754161442872626284681153034243}, - {2, - 5483947689873238016293916031421210417715226198450458695278359214939920808903888754748291585234470514731095293158971577871290333956546390471923967166891926796058037943347061391932048683243406115629059207198430761405517792845307599528545123275907240829958638010654676409407473464540147054022768326380533144139109013796005070619515692497303229683409723476100264437337159592967071649572068146217379004726298805073131642749312292639037974687912276595434011484665633472741970437405242931540792817150021104152269103538845906716294609703037973874968257574699425342614963850129729638055165011318321314629257017448744791130817504976352049315558175084707575517905220240735436472311376085977023193255767115772199401681050675227245864337841813577099951328506716260252484525951496988402157538736606893316558602219804539468211692272341101231081798647407433475330661334010047363328925076029072733284928506274503591566463356567449951050328939372527077820435552083641057965091365289077453354175453158803766717082697605135002698550392387018215808881309906132288850897899530025875371847565845964186403727130406970585738244224109459935326058926178367135154001854890789551481230848123840375019517586126229348925204229138925157417388983071508190365878773959469248844627811548238703202907823260627705881418371964608641916875906309702255332357870246656733850484860625288965209661089574538795660416296959228913342040037715375469749366782116594607018721130210724633134051682028201789300859780675503521182677498151404274736590115271652257881815395804591347270645487515405426176207583246761514846190105223591830994843572131787042752062568982393789415091692293114059135480055118138148767250658263869768994686391725791746769853583344796533031355717997717521032986544321281377626382079046164167636841143008587025918057095843339228816609264878115677918487854067887356202214595419884175437682477994901012403352228510424776160081628734502208471530373704829323453838286290450475142257346198410524038065099740082374843587568140564545670658917308004231061168599338344693117101685656038296427353228994834347618154742061252460559481017139006630352773110812670787299045754161442872626284681267142387}, - {2, - 5483947689873238016293916031421210417715226198450458695278359214939920808903888754748291585234470514731095293158971577871290333956546390471923967166891926796058037943347061391932048683243406115629059207198430761405517792845307599528545123275907240829958638010654676409407473464540147054022768326380533144139109013796005070619515692497303229683409723476100264437337159592967071649572068146217379004726298805073131642749312292639037974687912276595434011484665633472741970437405242931540792817150021104152269103538845906716294609703037973874968257574699425342614963850129729638055165011318321314629257017448744791130817504976352049315558175084707575517905220240735436472311376085977023193255767115772199401681050675227245864337841813577099951328506716260252484525951496988402157538736606893316558602219804539468211692272341101231081798647407433475330661334010047363328925076029072733284928506274503591566463356567449951050328939372527077820435552083641057965091365289077453354175453158803766717082697605135002698550392387018215808881309906132288850897899530025875371847565845964186403727130406970585738244224109459935326058926178367135154001854890789551481230848123840375019517586126229348925204229138925157417388983071508190365878773959469248844627811548238703202907823260627705881418371964608641916875906309702255332357870246656733850484860625288965209661089574538795660416296959228913342040037715375469749366782116594607018721130210724633134051682028201789300859780675503521182677498151404274736590115271652257881815395804591347270645487515405426176207583246761514846190105223591830994843572131787042752062568982393789415091692293114059135480055118138148767250658263869768994686391725791746769853583344796533031355717997717521032986544321281377626382079046164167636841143008587025918057095843339228816609264878115677918487854067887356202214595419884175437682477994901012403352228510424776160081628734502208471530373704829323453838286290450475142257346198410524038065099740082374843587568140564545670658917308004231061168599338344693117101685656038296427353228994834347618154742061252460559481017139006630352773110812670787299045754161442872626284681306144307}, - {2, - 5483947689873238016293916031421210417715226198450458695278359214939920808903888754748291585234470514731095293158971577871290333956546390471923967166891926796058037943347061391932048683243406115629059207198430761405517792845307599528545123275907240829958638010654676409407473464540147054022768326380533144139109013796005070619515692497303229683409723476100264437337159592967071649572068146217379004726298805073131642749312292639037974687912276595434011484665633472741970437405242931540792817150021104152269103538845906716294609703037973874968257574699425342614963850129729638055165011318321314629257017448744791130817504976352049315558175084707575517905220240735436472311376085977023193255767115772199401681050675227245864337841813577099951328506716260252484525951496988402157538736606893316558602219804539468211692272341101231081798647407433475330661334010047363328925076029072733284928506274503591566463356567449951050328939372527077820435552083641057965091365289077453354175453158803766717082697605135002698550392387018215808881309906132288850897899530025875371847565845964186403727130406970585738244224109459935326058926178367135154001854890789551481230848123840375019517586126229348925204229138925157417388983071508190365878773959469248844627811548238703202907823260627705881418371964608641916875906309702255332357870246656733850484860625288965209661089574538795660416296959228913342040037715375469749366782116594607018721130210724633134051682028201789300859780675503521182677498151404274736590115271652257881815395804591347270645487515405426176207583246761514846190105223591830994843572131787042752062568982393789415091692293114059135480055118138148767250658263869768994686391725791746769853583344796533031355717997717521032986544321281377626382079046164167636841143008587025918057095843339228816609264878115677918487854067887356202214595419884175437682477994901012403352228510424776160081628734502208471530373704829323453838286290450475142257346198410524038065099740082374843587568140564545670658917308004231061168599338344693117101685656038296427353228994834347618154742061252460559481017139006630352773110812670787299045754161442872626284681336711163}, - {2, - 5483947689873238016293916031421210417715226198450458695278359214939920808903888754748291585234470514731095293158971577871290333956546390471923967166891926796058037943347061391932048683243406115629059207198430761405517792845307599528545123275907240829958638010654676409407473464540147054022768326380533144139109013796005070619515692497303229683409723476100264437337159592967071649572068146217379004726298805073131642749312292639037974687912276595434011484665633472741970437405242931540792817150021104152269103538845906716294609703037973874968257574699425342614963850129729638055165011318321314629257017448744791130817504976352049315558175084707575517905220240735436472311376085977023193255767115772199401681050675227245864337841813577099951328506716260252484525951496988402157538736606893316558602219804539468211692272341101231081798647407433475330661334010047363328925076029072733284928506274503591566463356567449951050328939372527077820435552083641057965091365289077453354175453158803766717082697605135002698550392387018215808881309906132288850897899530025875371847565845964186403727130406970585738244224109459935326058926178367135154001854890789551481230848123840375019517586126229348925204229138925157417388983071508190365878773959469248844627811548238703202907823260627705881418371964608641916875906309702255332357870246656733850484860625288965209661089574538795660416296959228913342040037715375469749366782116594607018721130210724633134051682028201789300859780675503521182677498151404274736590115271652257881815395804591347270645487515405426176207583246761514846190105223591830994843572131787042752062568982393789415091692293114059135480055118138148767250658263869768994686391725791746769853583344796533031355717997717521032986544321281377626382079046164167636841143008587025918057095843339228816609264878115677918487854067887356202214595419884175437682477994901012403352228510424776160081628734502208471530373704829323453838286290450475142257346198410524038065099740082374843587568140564545670658917308004231061168599338344693117101685656038296427353228994834347618154742061252460559481017139006630352773110812670787299045754161442872626284681362638459}, - {2, - 5483947689873238016293916031421210417715226198450458695278359214939920808903888754748291585234470514731095293158971577871290333956546390471923967166891926796058037943347061391932048683243406115629059207198430761405517792845307599528545123275907240829958638010654676409407473464540147054022768326380533144139109013796005070619515692497303229683409723476100264437337159592967071649572068146217379004726298805073131642749312292639037974687912276595434011484665633472741970437405242931540792817150021104152269103538845906716294609703037973874968257574699425342614963850129729638055165011318321314629257017448744791130817504976352049315558175084707575517905220240735436472311376085977023193255767115772199401681050675227245864337841813577099951328506716260252484525951496988402157538736606893316558602219804539468211692272341101231081798647407433475330661334010047363328925076029072733284928506274503591566463356567449951050328939372527077820435552083641057965091365289077453354175453158803766717082697605135002698550392387018215808881309906132288850897899530025875371847565845964186403727130406970585738244224109459935326058926178367135154001854890789551481230848123840375019517586126229348925204229138925157417388983071508190365878773959469248844627811548238703202907823260627705881418371964608641916875906309702255332357870246656733850484860625288965209661089574538795660416296959228913342040037715375469749366782116594607018721130210724633134051682028201789300859780675503521182677498151404274736590115271652257881815395804591347270645487515405426176207583246761514846190105223591830994843572131787042752062568982393789415091692293114059135480055118138148767250658263869768994686391725791746769853583344796533031355717997717521032986544321281377626382079046164167636841143008587025918057095843339228816609264878115677918487854067887356202214595419884175437682477994901012403352228510424776160081628734502208471530373704829323453838286290450475142257346198410524038065099740082374843587568140564545670658917308004231061168599338344693117101685656038296427353228994834347618154742061252460559481017139006630352773110812670787299045754161442872626284681374503099}, - {2, - 5483947689873238016293916031421210417715226198450458695278359214939920808903888754748291585234470514731095293158971577871290333956546390471923967166891926796058037943347061391932048683243406115629059207198430761405517792845307599528545123275907240829958638010654676409407473464540147054022768326380533144139109013796005070619515692497303229683409723476100264437337159592967071649572068146217379004726298805073131642749312292639037974687912276595434011484665633472741970437405242931540792817150021104152269103538845906716294609703037973874968257574699425342614963850129729638055165011318321314629257017448744791130817504976352049315558175084707575517905220240735436472311376085977023193255767115772199401681050675227245864337841813577099951328506716260252484525951496988402157538736606893316558602219804539468211692272341101231081798647407433475330661334010047363328925076029072733284928506274503591566463356567449951050328939372527077820435552083641057965091365289077453354175453158803766717082697605135002698550392387018215808881309906132288850897899530025875371847565845964186403727130406970585738244224109459935326058926178367135154001854890789551481230848123840375019517586126229348925204229138925157417388983071508190365878773959469248844627811548238703202907823260627705881418371964608641916875906309702255332357870246656733850484860625288965209661089574538795660416296959228913342040037715375469749366782116594607018721130210724633134051682028201789300859780675503521182677498151404274736590115271652257881815395804591347270645487515405426176207583246761514846190105223591830994843572131787042752062568982393789415091692293114059135480055118138148767250658263869768994686391725791746769853583344796533031355717997717521032986544321281377626382079046164167636841143008587025918057095843339228816609264878115677918487854067887356202214595419884175437682477994901012403352228510424776160081628734502208471530373704829323453838286290450475142257346198410524038065099740082374843587568140564545670658917308004231061168599338344693117101685656038296427353228994834347618154742061252460559481017139006630352773110812670787299045754161442872626284681467737603}, - {2, - 5483947689873238016293916031421210417715226198450458695278359214939920808903888754748291585234470514731095293158971577871290333956546390471923967166891926796058037943347061391932048683243406115629059207198430761405517792845307599528545123275907240829958638010654676409407473464540147054022768326380533144139109013796005070619515692497303229683409723476100264437337159592967071649572068146217379004726298805073131642749312292639037974687912276595434011484665633472741970437405242931540792817150021104152269103538845906716294609703037973874968257574699425342614963850129729638055165011318321314629257017448744791130817504976352049315558175084707575517905220240735436472311376085977023193255767115772199401681050675227245864337841813577099951328506716260252484525951496988402157538736606893316558602219804539468211692272341101231081798647407433475330661334010047363328925076029072733284928506274503591566463356567449951050328939372527077820435552083641057965091365289077453354175453158803766717082697605135002698550392387018215808881309906132288850897899530025875371847565845964186403727130406970585738244224109459935326058926178367135154001854890789551481230848123840375019517586126229348925204229138925157417388983071508190365878773959469248844627811548238703202907823260627705881418371964608641916875906309702255332357870246656733850484860625288965209661089574538795660416296959228913342040037715375469749366782116594607018721130210724633134051682028201789300859780675503521182677498151404274736590115271652257881815395804591347270645487515405426176207583246761514846190105223591830994843572131787042752062568982393789415091692293114059135480055118138148767250658263869768994686391725791746769853583344796533031355717997717521032986544321281377626382079046164167636841143008587025918057095843339228816609264878115677918487854067887356202214595419884175437682477994901012403352228510424776160081628734502208471530373704829323453838286290450475142257346198410524038065099740082374843587568140564545670658917308004231061168599338344693117101685656038296427353228994834347618154742061252460559481017139006630352773110812670787299045754161442872626284681522006547}, - {2, - 5483947689873238016293916031421210417715226198450458695278359214939920808903888754748291585234470514731095293158971577871290333956546390471923967166891926796058037943347061391932048683243406115629059207198430761405517792845307599528545123275907240829958638010654676409407473464540147054022768326380533144139109013796005070619515692497303229683409723476100264437337159592967071649572068146217379004726298805073131642749312292639037974687912276595434011484665633472741970437405242931540792817150021104152269103538845906716294609703037973874968257574699425342614963850129729638055165011318321314629257017448744791130817504976352049315558175084707575517905220240735436472311376085977023193255767115772199401681050675227245864337841813577099951328506716260252484525951496988402157538736606893316558602219804539468211692272341101231081798647407433475330661334010047363328925076029072733284928506274503591566463356567449951050328939372527077820435552083641057965091365289077453354175453158803766717082697605135002698550392387018215808881309906132288850897899530025875371847565845964186403727130406970585738244224109459935326058926178367135154001854890789551481230848123840375019517586126229348925204229138925157417388983071508190365878773959469248844627811548238703202907823260627705881418371964608641916875906309702255332357870246656733850484860625288965209661089574538795660416296959228913342040037715375469749366782116594607018721130210724633134051682028201789300859780675503521182677498151404274736590115271652257881815395804591347270645487515405426176207583246761514846190105223591830994843572131787042752062568982393789415091692293114059135480055118138148767250658263869768994686391725791746769853583344796533031355717997717521032986544321281377626382079046164167636841143008587025918057095843339228816609264878115677918487854067887356202214595419884175437682477994901012403352228510424776160081628734502208471530373704829323453838286290450475142257346198410524038065099740082374843587568140564545670658917308004231061168599338344693117101685656038296427353228994834347618154742061252460559481017139006630352773110812670787299045754161442872626284681585814027}, - {2, - 5483947689873238016293916031421210417715226198450458695278359214939920808903888754748291585234470514731095293158971577871290333956546390471923967166891926796058037943347061391932048683243406115629059207198430761405517792845307599528545123275907240829958638010654676409407473464540147054022768326380533144139109013796005070619515692497303229683409723476100264437337159592967071649572068146217379004726298805073131642749312292639037974687912276595434011484665633472741970437405242931540792817150021104152269103538845906716294609703037973874968257574699425342614963850129729638055165011318321314629257017448744791130817504976352049315558175084707575517905220240735436472311376085977023193255767115772199401681050675227245864337841813577099951328506716260252484525951496988402157538736606893316558602219804539468211692272341101231081798647407433475330661334010047363328925076029072733284928506274503591566463356567449951050328939372527077820435552083641057965091365289077453354175453158803766717082697605135002698550392387018215808881309906132288850897899530025875371847565845964186403727130406970585738244224109459935326058926178367135154001854890789551481230848123840375019517586126229348925204229138925157417388983071508190365878773959469248844627811548238703202907823260627705881418371964608641916875906309702255332357870246656733850484860625288965209661089574538795660416296959228913342040037715375469749366782116594607018721130210724633134051682028201789300859780675503521182677498151404274736590115271652257881815395804591347270645487515405426176207583246761514846190105223591830994843572131787042752062568982393789415091692293114059135480055118138148767250658263869768994686391725791746769853583344796533031355717997717521032986544321281377626382079046164167636841143008587025918057095843339228816609264878115677918487854067887356202214595419884175437682477994901012403352228510424776160081628734502208471530373704829323453838286290450475142257346198410524038065099740082374843587568140564545670658917308004231061168599338344693117101685656038296427353228994834347618154742061252460559481017139006630352773110812670787299045754161442872626284681591978499}, - {2, - 5483947689873238016293916031421210417715226198450458695278359214939920808903888754748291585234470514731095293158971577871290333956546390471923967166891926796058037943347061391932048683243406115629059207198430761405517792845307599528545123275907240829958638010654676409407473464540147054022768326380533144139109013796005070619515692497303229683409723476100264437337159592967071649572068146217379004726298805073131642749312292639037974687912276595434011484665633472741970437405242931540792817150021104152269103538845906716294609703037973874968257574699425342614963850129729638055165011318321314629257017448744791130817504976352049315558175084707575517905220240735436472311376085977023193255767115772199401681050675227245864337841813577099951328506716260252484525951496988402157538736606893316558602219804539468211692272341101231081798647407433475330661334010047363328925076029072733284928506274503591566463356567449951050328939372527077820435552083641057965091365289077453354175453158803766717082697605135002698550392387018215808881309906132288850897899530025875371847565845964186403727130406970585738244224109459935326058926178367135154001854890789551481230848123840375019517586126229348925204229138925157417388983071508190365878773959469248844627811548238703202907823260627705881418371964608641916875906309702255332357870246656733850484860625288965209661089574538795660416296959228913342040037715375469749366782116594607018721130210724633134051682028201789300859780675503521182677498151404274736590115271652257881815395804591347270645487515405426176207583246761514846190105223591830994843572131787042752062568982393789415091692293114059135480055118138148767250658263869768994686391725791746769853583344796533031355717997717521032986544321281377626382079046164167636841143008587025918057095843339228816609264878115677918487854067887356202214595419884175437682477994901012403352228510424776160081628734502208471530373704829323453838286290450475142257346198410524038065099740082374843587568140564545670658917308004231061168599338344693117101685656038296427353228994834347618154742061252460559481017139006630352773110812670787299045754161442872626284681697407907}, - {2, - 5483947689873238016293916031421210417715226198450458695278359214939920808903888754748291585234470514731095293158971577871290333956546390471923967166891926796058037943347061391932048683243406115629059207198430761405517792845307599528545123275907240829958638010654676409407473464540147054022768326380533144139109013796005070619515692497303229683409723476100264437337159592967071649572068146217379004726298805073131642749312292639037974687912276595434011484665633472741970437405242931540792817150021104152269103538845906716294609703037973874968257574699425342614963850129729638055165011318321314629257017448744791130817504976352049315558175084707575517905220240735436472311376085977023193255767115772199401681050675227245864337841813577099951328506716260252484525951496988402157538736606893316558602219804539468211692272341101231081798647407433475330661334010047363328925076029072733284928506274503591566463356567449951050328939372527077820435552083641057965091365289077453354175453158803766717082697605135002698550392387018215808881309906132288850897899530025875371847565845964186403727130406970585738244224109459935326058926178367135154001854890789551481230848123840375019517586126229348925204229138925157417388983071508190365878773959469248844627811548238703202907823260627705881418371964608641916875906309702255332357870246656733850484860625288965209661089574538795660416296959228913342040037715375469749366782116594607018721130210724633134051682028201789300859780675503521182677498151404274736590115271652257881815395804591347270645487515405426176207583246761514846190105223591830994843572131787042752062568982393789415091692293114059135480055118138148767250658263869768994686391725791746769853583344796533031355717997717521032986544321281377626382079046164167636841143008587025918057095843339228816609264878115677918487854067887356202214595419884175437682477994901012403352228510424776160081628734502208471530373704829323453838286290450475142257346198410524038065099740082374843587568140564545670658917308004231061168599338344693117101685656038296427353228994834347618154742061252460559481017139006630352773110812670787299045754161442872626284681773510443}, - {2, - 5483947689873238016293916031421210417715226198450458695278359214939920808903888754748291585234470514731095293158971577871290333956546390471923967166891926796058037943347061391932048683243406115629059207198430761405517792845307599528545123275907240829958638010654676409407473464540147054022768326380533144139109013796005070619515692497303229683409723476100264437337159592967071649572068146217379004726298805073131642749312292639037974687912276595434011484665633472741970437405242931540792817150021104152269103538845906716294609703037973874968257574699425342614963850129729638055165011318321314629257017448744791130817504976352049315558175084707575517905220240735436472311376085977023193255767115772199401681050675227245864337841813577099951328506716260252484525951496988402157538736606893316558602219804539468211692272341101231081798647407433475330661334010047363328925076029072733284928506274503591566463356567449951050328939372527077820435552083641057965091365289077453354175453158803766717082697605135002698550392387018215808881309906132288850897899530025875371847565845964186403727130406970585738244224109459935326058926178367135154001854890789551481230848123840375019517586126229348925204229138925157417388983071508190365878773959469248844627811548238703202907823260627705881418371964608641916875906309702255332357870246656733850484860625288965209661089574538795660416296959228913342040037715375469749366782116594607018721130210724633134051682028201789300859780675503521182677498151404274736590115271652257881815395804591347270645487515405426176207583246761514846190105223591830994843572131787042752062568982393789415091692293114059135480055118138148767250658263869768994686391725791746769853583344796533031355717997717521032986544321281377626382079046164167636841143008587025918057095843339228816609264878115677918487854067887356202214595419884175437682477994901012403352228510424776160081628734502208471530373704829323453838286290450475142257346198410524038065099740082374843587568140564545670658917308004231061168599338344693117101685656038296427353228994834347618154742061252460559481017139006630352773110812670787299045754161442872626284681779980579}, - {2, - 5483947689873238016293916031421210417715226198450458695278359214939920808903888754748291585234470514731095293158971577871290333956546390471923967166891926796058037943347061391932048683243406115629059207198430761405517792845307599528545123275907240829958638010654676409407473464540147054022768326380533144139109013796005070619515692497303229683409723476100264437337159592967071649572068146217379004726298805073131642749312292639037974687912276595434011484665633472741970437405242931540792817150021104152269103538845906716294609703037973874968257574699425342614963850129729638055165011318321314629257017448744791130817504976352049315558175084707575517905220240735436472311376085977023193255767115772199401681050675227245864337841813577099951328506716260252484525951496988402157538736606893316558602219804539468211692272341101231081798647407433475330661334010047363328925076029072733284928506274503591566463356567449951050328939372527077820435552083641057965091365289077453354175453158803766717082697605135002698550392387018215808881309906132288850897899530025875371847565845964186403727130406970585738244224109459935326058926178367135154001854890789551481230848123840375019517586126229348925204229138925157417388983071508190365878773959469248844627811548238703202907823260627705881418371964608641916875906309702255332357870246656733850484860625288965209661089574538795660416296959228913342040037715375469749366782116594607018721130210724633134051682028201789300859780675503521182677498151404274736590115271652257881815395804591347270645487515405426176207583246761514846190105223591830994843572131787042752062568982393789415091692293114059135480055118138148767250658263869768994686391725791746769853583344796533031355717997717521032986544321281377626382079046164167636841143008587025918057095843339228816609264878115677918487854067887356202214595419884175437682477994901012403352228510424776160081628734502208471530373704829323453838286290450475142257346198410524038065099740082374843587568140564545670658917308004231061168599338344693117101685656038296427353228994834347618154742061252460559481017139006630352773110812670787299045754161442872626284681794840947}, - {2, - 5483947689873238016293916031421210417715226198450458695278359214939920808903888754748291585234470514731095293158971577871290333956546390471923967166891926796058037943347061391932048683243406115629059207198430761405517792845307599528545123275907240829958638010654676409407473464540147054022768326380533144139109013796005070619515692497303229683409723476100264437337159592967071649572068146217379004726298805073131642749312292639037974687912276595434011484665633472741970437405242931540792817150021104152269103538845906716294609703037973874968257574699425342614963850129729638055165011318321314629257017448744791130817504976352049315558175084707575517905220240735436472311376085977023193255767115772199401681050675227245864337841813577099951328506716260252484525951496988402157538736606893316558602219804539468211692272341101231081798647407433475330661334010047363328925076029072733284928506274503591566463356567449951050328939372527077820435552083641057965091365289077453354175453158803766717082697605135002698550392387018215808881309906132288850897899530025875371847565845964186403727130406970585738244224109459935326058926178367135154001854890789551481230848123840375019517586126229348925204229138925157417388983071508190365878773959469248844627811548238703202907823260627705881418371964608641916875906309702255332357870246656733850484860625288965209661089574538795660416296959228913342040037715375469749366782116594607018721130210724633134051682028201789300859780675503521182677498151404274736590115271652257881815395804591347270645487515405426176207583246761514846190105223591830994843572131787042752062568982393789415091692293114059135480055118138148767250658263869768994686391725791746769853583344796533031355717997717521032986544321281377626382079046164167636841143008587025918057095843339228816609264878115677918487854067887356202214595419884175437682477994901012403352228510424776160081628734502208471530373704829323453838286290450475142257346198410524038065099740082374843587568140564545670658917308004231061168599338344693117101685656038296427353228994834347618154742061252460559481017139006630352773110812670787299045754161442872626284681855986587}, - {2, - 5483947689873238016293916031421210417715226198450458695278359214939920808903888754748291585234470514731095293158971577871290333956546390471923967166891926796058037943347061391932048683243406115629059207198430761405517792845307599528545123275907240829958638010654676409407473464540147054022768326380533144139109013796005070619515692497303229683409723476100264437337159592967071649572068146217379004726298805073131642749312292639037974687912276595434011484665633472741970437405242931540792817150021104152269103538845906716294609703037973874968257574699425342614963850129729638055165011318321314629257017448744791130817504976352049315558175084707575517905220240735436472311376085977023193255767115772199401681050675227245864337841813577099951328506716260252484525951496988402157538736606893316558602219804539468211692272341101231081798647407433475330661334010047363328925076029072733284928506274503591566463356567449951050328939372527077820435552083641057965091365289077453354175453158803766717082697605135002698550392387018215808881309906132288850897899530025875371847565845964186403727130406970585738244224109459935326058926178367135154001854890789551481230848123840375019517586126229348925204229138925157417388983071508190365878773959469248844627811548238703202907823260627705881418371964608641916875906309702255332357870246656733850484860625288965209661089574538795660416296959228913342040037715375469749366782116594607018721130210724633134051682028201789300859780675503521182677498151404274736590115271652257881815395804591347270645487515405426176207583246761514846190105223591830994843572131787042752062568982393789415091692293114059135480055118138148767250658263869768994686391725791746769853583344796533031355717997717521032986544321281377626382079046164167636841143008587025918057095843339228816609264878115677918487854067887356202214595419884175437682477994901012403352228510424776160081628734502208471530373704829323453838286290450475142257346198410524038065099740082374843587568140564545670658917308004231061168599338344693117101685656038296427353228994834347618154742061252460559481017139006630352773110812670787299045754161442872626284682028742019}, - {2, - 5483947689873238016293916031421210417715226198450458695278359214939920808903888754748291585234470514731095293158971577871290333956546390471923967166891926796058037943347061391932048683243406115629059207198430761405517792845307599528545123275907240829958638010654676409407473464540147054022768326380533144139109013796005070619515692497303229683409723476100264437337159592967071649572068146217379004726298805073131642749312292639037974687912276595434011484665633472741970437405242931540792817150021104152269103538845906716294609703037973874968257574699425342614963850129729638055165011318321314629257017448744791130817504976352049315558175084707575517905220240735436472311376085977023193255767115772199401681050675227245864337841813577099951328506716260252484525951496988402157538736606893316558602219804539468211692272341101231081798647407433475330661334010047363328925076029072733284928506274503591566463356567449951050328939372527077820435552083641057965091365289077453354175453158803766717082697605135002698550392387018215808881309906132288850897899530025875371847565845964186403727130406970585738244224109459935326058926178367135154001854890789551481230848123840375019517586126229348925204229138925157417388983071508190365878773959469248844627811548238703202907823260627705881418371964608641916875906309702255332357870246656733850484860625288965209661089574538795660416296959228913342040037715375469749366782116594607018721130210724633134051682028201789300859780675503521182677498151404274736590115271652257881815395804591347270645487515405426176207583246761514846190105223591830994843572131787042752062568982393789415091692293114059135480055118138148767250658263869768994686391725791746769853583344796533031355717997717521032986544321281377626382079046164167636841143008587025918057095843339228816609264878115677918487854067887356202214595419884175437682477994901012403352228510424776160081628734502208471530373704829323453838286290450475142257346198410524038065099740082374843587568140564545670658917308004231061168599338344693117101685656038296427353228994834347618154742061252460559481017139006630352773110812670787299045754161442872626284682126322323}, - {5, - 5483947689873238016293916031421210417715226198450458695278359214939920808903888754748291585234470514731095293158971577871290333956546390471923967166891926796058037943347061391932048683243406115629059207198430761405517792845307599528545123275907240829958638010654676409407473464540147054022768326380533144139109013796005070619515692497303229683409723476100264437337159592967071649572068146217379004726298805073131642749312292639037974687912276595434011484665633472741970437405242931540792817150021104152269103538845906716294609703037973874968257574699425342614963850129729638055165011318321314629257017448744791130817504976352049315558175084707575517905220240735436472311376085977023193255767115772199401681050675227245864337841813577099951328506716260252484525951496988402157538736606893316558602219804539468211692272341101231081798647407433475330661334010047363328925076029072733284928506274503591566463356567449951050328939372527077820435552083641057965091365289077453354175453158803766717082697605135002698550392387018215808881309906132288850897899530025875371847565845964186403727130406970585738244224109459935326058926178367135154001854890789551481230848123840375019517586126229348925204229138925157417388983071508190365878773959469248844627811548238703202907823260627705881418371964608641916875906309702255332357870246656733850484860625288965209661089574538795660416296959228913342040037715375469749366782116594607018721130210724633134051682028201789300859780675503521182677498151404274736590115271652257881815395804591347270645487515405426176207583246761514846190105223591830994843572131787042752062568982393789415091692293114059135480055118138148767250658263869768994686391725791746769853583344796533031355717997717521032986544321281377626382079046164167636841143008587025918057095843339228816609264878115677918487854067887356202214595419884175437682477994901012403352228510424776160081628734502208471530373704829323453838286290450475142257346198410524038065099740082374843587568140564545670658917308004231061168599338344693117101685656038296427353228994834347618154742061252460559481017139006630352773110812670787299045754161442872626284680529504343}, - {5, - 5483947689873238016293916031421210417715226198450458695278359214939920808903888754748291585234470514731095293158971577871290333956546390471923967166891926796058037943347061391932048683243406115629059207198430761405517792845307599528545123275907240829958638010654676409407473464540147054022768326380533144139109013796005070619515692497303229683409723476100264437337159592967071649572068146217379004726298805073131642749312292639037974687912276595434011484665633472741970437405242931540792817150021104152269103538845906716294609703037973874968257574699425342614963850129729638055165011318321314629257017448744791130817504976352049315558175084707575517905220240735436472311376085977023193255767115772199401681050675227245864337841813577099951328506716260252484525951496988402157538736606893316558602219804539468211692272341101231081798647407433475330661334010047363328925076029072733284928506274503591566463356567449951050328939372527077820435552083641057965091365289077453354175453158803766717082697605135002698550392387018215808881309906132288850897899530025875371847565845964186403727130406970585738244224109459935326058926178367135154001854890789551481230848123840375019517586126229348925204229138925157417388983071508190365878773959469248844627811548238703202907823260627705881418371964608641916875906309702255332357870246656733850484860625288965209661089574538795660416296959228913342040037715375469749366782116594607018721130210724633134051682028201789300859780675503521182677498151404274736590115271652257881815395804591347270645487515405426176207583246761514846190105223591830994843572131787042752062568982393789415091692293114059135480055118138148767250658263869768994686391725791746769853583344796533031355717997717521032986544321281377626382079046164167636841143008587025918057095843339228816609264878115677918487854067887356202214595419884175437682477994901012403352228510424776160081628734502208471530373704829323453838286290450475142257346198410524038065099740082374843587568140564545670658917308004231061168599338344693117101685656038296427353228994834347618154742061252460559481017139006630352773110812670787299045754161442872626284680623896367}, - {5, - 5483947689873238016293916031421210417715226198450458695278359214939920808903888754748291585234470514731095293158971577871290333956546390471923967166891926796058037943347061391932048683243406115629059207198430761405517792845307599528545123275907240829958638010654676409407473464540147054022768326380533144139109013796005070619515692497303229683409723476100264437337159592967071649572068146217379004726298805073131642749312292639037974687912276595434011484665633472741970437405242931540792817150021104152269103538845906716294609703037973874968257574699425342614963850129729638055165011318321314629257017448744791130817504976352049315558175084707575517905220240735436472311376085977023193255767115772199401681050675227245864337841813577099951328506716260252484525951496988402157538736606893316558602219804539468211692272341101231081798647407433475330661334010047363328925076029072733284928506274503591566463356567449951050328939372527077820435552083641057965091365289077453354175453158803766717082697605135002698550392387018215808881309906132288850897899530025875371847565845964186403727130406970585738244224109459935326058926178367135154001854890789551481230848123840375019517586126229348925204229138925157417388983071508190365878773959469248844627811548238703202907823260627705881418371964608641916875906309702255332357870246656733850484860625288965209661089574538795660416296959228913342040037715375469749366782116594607018721130210724633134051682028201789300859780675503521182677498151404274736590115271652257881815395804591347270645487515405426176207583246761514846190105223591830994843572131787042752062568982393789415091692293114059135480055118138148767250658263869768994686391725791746769853583344796533031355717997717521032986544321281377626382079046164167636841143008587025918057095843339228816609264878115677918487854067887356202214595419884175437682477994901012403352228510424776160081628734502208471530373704829323453838286290450475142257346198410524038065099740082374843587568140564545670658917308004231061168599338344693117101685656038296427353228994834347618154742061252460559481017139006630352773110812670787299045754161442872626284680724430207}, - {5, - 5483947689873238016293916031421210417715226198450458695278359214939920808903888754748291585234470514731095293158971577871290333956546390471923967166891926796058037943347061391932048683243406115629059207198430761405517792845307599528545123275907240829958638010654676409407473464540147054022768326380533144139109013796005070619515692497303229683409723476100264437337159592967071649572068146217379004726298805073131642749312292639037974687912276595434011484665633472741970437405242931540792817150021104152269103538845906716294609703037973874968257574699425342614963850129729638055165011318321314629257017448744791130817504976352049315558175084707575517905220240735436472311376085977023193255767115772199401681050675227245864337841813577099951328506716260252484525951496988402157538736606893316558602219804539468211692272341101231081798647407433475330661334010047363328925076029072733284928506274503591566463356567449951050328939372527077820435552083641057965091365289077453354175453158803766717082697605135002698550392387018215808881309906132288850897899530025875371847565845964186403727130406970585738244224109459935326058926178367135154001854890789551481230848123840375019517586126229348925204229138925157417388983071508190365878773959469248844627811548238703202907823260627705881418371964608641916875906309702255332357870246656733850484860625288965209661089574538795660416296959228913342040037715375469749366782116594607018721130210724633134051682028201789300859780675503521182677498151404274736590115271652257881815395804591347270645487515405426176207583246761514846190105223591830994843572131787042752062568982393789415091692293114059135480055118138148767250658263869768994686391725791746769853583344796533031355717997717521032986544321281377626382079046164167636841143008587025918057095843339228816609264878115677918487854067887356202214595419884175437682477994901012403352228510424776160081628734502208471530373704829323453838286290450475142257346198410524038065099740082374843587568140564545670658917308004231061168599338344693117101685656038296427353228994834347618154742061252460559481017139006630352773110812670787299045754161442872626284680814262183}, - {5, - 5483947689873238016293916031421210417715226198450458695278359214939920808903888754748291585234470514731095293158971577871290333956546390471923967166891926796058037943347061391932048683243406115629059207198430761405517792845307599528545123275907240829958638010654676409407473464540147054022768326380533144139109013796005070619515692497303229683409723476100264437337159592967071649572068146217379004726298805073131642749312292639037974687912276595434011484665633472741970437405242931540792817150021104152269103538845906716294609703037973874968257574699425342614963850129729638055165011318321314629257017448744791130817504976352049315558175084707575517905220240735436472311376085977023193255767115772199401681050675227245864337841813577099951328506716260252484525951496988402157538736606893316558602219804539468211692272341101231081798647407433475330661334010047363328925076029072733284928506274503591566463356567449951050328939372527077820435552083641057965091365289077453354175453158803766717082697605135002698550392387018215808881309906132288850897899530025875371847565845964186403727130406970585738244224109459935326058926178367135154001854890789551481230848123840375019517586126229348925204229138925157417388983071508190365878773959469248844627811548238703202907823260627705881418371964608641916875906309702255332357870246656733850484860625288965209661089574538795660416296959228913342040037715375469749366782116594607018721130210724633134051682028201789300859780675503521182677498151404274736590115271652257881815395804591347270645487515405426176207583246761514846190105223591830994843572131787042752062568982393789415091692293114059135480055118138148767250658263869768994686391725791746769853583344796533031355717997717521032986544321281377626382079046164167636841143008587025918057095843339228816609264878115677918487854067887356202214595419884175437682477994901012403352228510424776160081628734502208471530373704829323453838286290450475142257346198410524038065099740082374843587568140564545670658917308004231061168599338344693117101685656038296427353228994834347618154742061252460559481017139006630352773110812670787299045754161442872626284680973785623}, - {5, - 5483947689873238016293916031421210417715226198450458695278359214939920808903888754748291585234470514731095293158971577871290333956546390471923967166891926796058037943347061391932048683243406115629059207198430761405517792845307599528545123275907240829958638010654676409407473464540147054022768326380533144139109013796005070619515692497303229683409723476100264437337159592967071649572068146217379004726298805073131642749312292639037974687912276595434011484665633472741970437405242931540792817150021104152269103538845906716294609703037973874968257574699425342614963850129729638055165011318321314629257017448744791130817504976352049315558175084707575517905220240735436472311376085977023193255767115772199401681050675227245864337841813577099951328506716260252484525951496988402157538736606893316558602219804539468211692272341101231081798647407433475330661334010047363328925076029072733284928506274503591566463356567449951050328939372527077820435552083641057965091365289077453354175453158803766717082697605135002698550392387018215808881309906132288850897899530025875371847565845964186403727130406970585738244224109459935326058926178367135154001854890789551481230848123840375019517586126229348925204229138925157417388983071508190365878773959469248844627811548238703202907823260627705881418371964608641916875906309702255332357870246656733850484860625288965209661089574538795660416296959228913342040037715375469749366782116594607018721130210724633134051682028201789300859780675503521182677498151404274736590115271652257881815395804591347270645487515405426176207583246761514846190105223591830994843572131787042752062568982393789415091692293114059135480055118138148767250658263869768994686391725791746769853583344796533031355717997717521032986544321281377626382079046164167636841143008587025918057095843339228816609264878115677918487854067887356202214595419884175437682477994901012403352228510424776160081628734502208471530373704829323453838286290450475142257346198410524038065099740082374843587568140564545670658917308004231061168599338344693117101685656038296427353228994834347618154742061252460559481017139006630352773110812670787299045754161442872626284680984913367}, - {5, - 5483947689873238016293916031421210417715226198450458695278359214939920808903888754748291585234470514731095293158971577871290333956546390471923967166891926796058037943347061391932048683243406115629059207198430761405517792845307599528545123275907240829958638010654676409407473464540147054022768326380533144139109013796005070619515692497303229683409723476100264437337159592967071649572068146217379004726298805073131642749312292639037974687912276595434011484665633472741970437405242931540792817150021104152269103538845906716294609703037973874968257574699425342614963850129729638055165011318321314629257017448744791130817504976352049315558175084707575517905220240735436472311376085977023193255767115772199401681050675227245864337841813577099951328506716260252484525951496988402157538736606893316558602219804539468211692272341101231081798647407433475330661334010047363328925076029072733284928506274503591566463356567449951050328939372527077820435552083641057965091365289077453354175453158803766717082697605135002698550392387018215808881309906132288850897899530025875371847565845964186403727130406970585738244224109459935326058926178367135154001854890789551481230848123840375019517586126229348925204229138925157417388983071508190365878773959469248844627811548238703202907823260627705881418371964608641916875906309702255332357870246656733850484860625288965209661089574538795660416296959228913342040037715375469749366782116594607018721130210724633134051682028201789300859780675503521182677498151404274736590115271652257881815395804591347270645487515405426176207583246761514846190105223591830994843572131787042752062568982393789415091692293114059135480055118138148767250658263869768994686391725791746769853583344796533031355717997717521032986544321281377626382079046164167636841143008587025918057095843339228816609264878115677918487854067887356202214595419884175437682477994901012403352228510424776160081628734502208471530373704829323453838286290450475142257346198410524038065099740082374843587568140564545670658917308004231061168599338344693117101685656038296427353228994834347618154742061252460559481017139006630352773110812670787299045754161442872626284681363215767}, - {5, - 5483947689873238016293916031421210417715226198450458695278359214939920808903888754748291585234470514731095293158971577871290333956546390471923967166891926796058037943347061391932048683243406115629059207198430761405517792845307599528545123275907240829958638010654676409407473464540147054022768326380533144139109013796005070619515692497303229683409723476100264437337159592967071649572068146217379004726298805073131642749312292639037974687912276595434011484665633472741970437405242931540792817150021104152269103538845906716294609703037973874968257574699425342614963850129729638055165011318321314629257017448744791130817504976352049315558175084707575517905220240735436472311376085977023193255767115772199401681050675227245864337841813577099951328506716260252484525951496988402157538736606893316558602219804539468211692272341101231081798647407433475330661334010047363328925076029072733284928506274503591566463356567449951050328939372527077820435552083641057965091365289077453354175453158803766717082697605135002698550392387018215808881309906132288850897899530025875371847565845964186403727130406970585738244224109459935326058926178367135154001854890789551481230848123840375019517586126229348925204229138925157417388983071508190365878773959469248844627811548238703202907823260627705881418371964608641916875906309702255332357870246656733850484860625288965209661089574538795660416296959228913342040037715375469749366782116594607018721130210724633134051682028201789300859780675503521182677498151404274736590115271652257881815395804591347270645487515405426176207583246761514846190105223591830994843572131787042752062568982393789415091692293114059135480055118138148767250658263869768994686391725791746769853583344796533031355717997717521032986544321281377626382079046164167636841143008587025918057095843339228816609264878115677918487854067887356202214595419884175437682477994901012403352228510424776160081628734502208471530373704829323453838286290450475142257346198410524038065099740082374843587568140564545670658917308004231061168599338344693117101685656038296427353228994834347618154742061252460559481017139006630352773110812670787299045754161442872626284681736147327}, - {5, - 5483947689873238016293916031421210417715226198450458695278359214939920808903888754748291585234470514731095293158971577871290333956546390471923967166891926796058037943347061391932048683243406115629059207198430761405517792845307599528545123275907240829958638010654676409407473464540147054022768326380533144139109013796005070619515692497303229683409723476100264437337159592967071649572068146217379004726298805073131642749312292639037974687912276595434011484665633472741970437405242931540792817150021104152269103538845906716294609703037973874968257574699425342614963850129729638055165011318321314629257017448744791130817504976352049315558175084707575517905220240735436472311376085977023193255767115772199401681050675227245864337841813577099951328506716260252484525951496988402157538736606893316558602219804539468211692272341101231081798647407433475330661334010047363328925076029072733284928506274503591566463356567449951050328939372527077820435552083641057965091365289077453354175453158803766717082697605135002698550392387018215808881309906132288850897899530025875371847565845964186403727130406970585738244224109459935326058926178367135154001854890789551481230848123840375019517586126229348925204229138925157417388983071508190365878773959469248844627811548238703202907823260627705881418371964608641916875906309702255332357870246656733850484860625288965209661089574538795660416296959228913342040037715375469749366782116594607018721130210724633134051682028201789300859780675503521182677498151404274736590115271652257881815395804591347270645487515405426176207583246761514846190105223591830994843572131787042752062568982393789415091692293114059135480055118138148767250658263869768994686391725791746769853583344796533031355717997717521032986544321281377626382079046164167636841143008587025918057095843339228816609264878115677918487854067887356202214595419884175437682477994901012403352228510424776160081628734502208471530373704829323453838286290450475142257346198410524038065099740082374843587568140564545670658917308004231061168599338344693117101685656038296427353228994834347618154742061252460559481017139006630352773110812670787299045754161442872626284681849549247}, - {5, - 5483947689873238016293916031421210417715226198450458695278359214939920808903888754748291585234470514731095293158971577871290333956546390471923967166891926796058037943347061391932048683243406115629059207198430761405517792845307599528545123275907240829958638010654676409407473464540147054022768326380533144139109013796005070619515692497303229683409723476100264437337159592967071649572068146217379004726298805073131642749312292639037974687912276595434011484665633472741970437405242931540792817150021104152269103538845906716294609703037973874968257574699425342614963850129729638055165011318321314629257017448744791130817504976352049315558175084707575517905220240735436472311376085977023193255767115772199401681050675227245864337841813577099951328506716260252484525951496988402157538736606893316558602219804539468211692272341101231081798647407433475330661334010047363328925076029072733284928506274503591566463356567449951050328939372527077820435552083641057965091365289077453354175453158803766717082697605135002698550392387018215808881309906132288850897899530025875371847565845964186403727130406970585738244224109459935326058926178367135154001854890789551481230848123840375019517586126229348925204229138925157417388983071508190365878773959469248844627811548238703202907823260627705881418371964608641916875906309702255332357870246656733850484860625288965209661089574538795660416296959228913342040037715375469749366782116594607018721130210724633134051682028201789300859780675503521182677498151404274736590115271652257881815395804591347270645487515405426176207583246761514846190105223591830994843572131787042752062568982393789415091692293114059135480055118138148767250658263869768994686391725791746769853583344796533031355717997717521032986544321281377626382079046164167636841143008587025918057095843339228816609264878115677918487854067887356202214595419884175437682477994901012403352228510424776160081628734502208471530373704829323453838286290450475142257346198410524038065099740082374843587568140564545670658917308004231061168599338344693117101685656038296427353228994834347618154742061252460559481017139006630352773110812670787299045754161442872626284681982651447}, - {5, - 5483947689873238016293916031421210417715226198450458695278359214939920808903888754748291585234470514731095293158971577871290333956546390471923967166891926796058037943347061391932048683243406115629059207198430761405517792845307599528545123275907240829958638010654676409407473464540147054022768326380533144139109013796005070619515692497303229683409723476100264437337159592967071649572068146217379004726298805073131642749312292639037974687912276595434011484665633472741970437405242931540792817150021104152269103538845906716294609703037973874968257574699425342614963850129729638055165011318321314629257017448744791130817504976352049315558175084707575517905220240735436472311376085977023193255767115772199401681050675227245864337841813577099951328506716260252484525951496988402157538736606893316558602219804539468211692272341101231081798647407433475330661334010047363328925076029072733284928506274503591566463356567449951050328939372527077820435552083641057965091365289077453354175453158803766717082697605135002698550392387018215808881309906132288850897899530025875371847565845964186403727130406970585738244224109459935326058926178367135154001854890789551481230848123840375019517586126229348925204229138925157417388983071508190365878773959469248844627811548238703202907823260627705881418371964608641916875906309702255332357870246656733850484860625288965209661089574538795660416296959228913342040037715375469749366782116594607018721130210724633134051682028201789300859780675503521182677498151404274736590115271652257881815395804591347270645487515405426176207583246761514846190105223591830994843572131787042752062568982393789415091692293114059135480055118138148767250658263869768994686391725791746769853583344796533031355717997717521032986544321281377626382079046164167636841143008587025918057095843339228816609264878115677918487854067887356202214595419884175437682477994901012403352228510424776160081628734502208471530373704829323453838286290450475142257346198410524038065099740082374843587568140564545670658917308004231061168599338344693117101685656038296427353228994834347618154742061252460559481017139006630352773110812670787299045754161442872626284682028345287}]}, {8191, [{2, 881045461522943321086778073300940478556218355297969911976909161486894133662493726399243167458009140047097403832005558428608022057031977203602474613009545091720237343240266129021712322298408144531017770778248716957128799297677903712279084392023819163463966227515269649827380536523405101157732109499388260769586116300288304982677621782718749987230168262305655004727816729815800533511307474309647349575475321354016095742812539133151602748038504759763440423033004809780519532557484323246180020130950200646945418165013898390018998574383185461948128020544881733962825915669056546421700831605194278745248581382984298520377716141327298492058956203175900776169162131066555283318461828456520694888875026876615176999234594065729624240200260702661257238500249077130700729314922467706084046564196655345340394880604121971434682608173172839952354536660137326680056370043971462309932399249681084807535691908014774608948539576573091871838175518964071111240660427603031489552613754601246540211425394120613055590993731825013169587667380928210308207859749014992608494977324762902471764267795852486660601596648189999837075454896316687482540387017067211974603191279230953515521397067907513757763367315888101693042775385804066821260256218210264761730017655260313514255350592287156466544141273327424165523630928036465070504326759160965905472140206420733620859718350162744176336519051572708874620380237875083788097274492094030771087416734847686560630664937136197575123272360898853159596587429099207143246305845544244799356813180338388773189837194780822656452007718856513775556746086854637309945375913748887588098858159436611056484858182222940508156607890079350636796413492320863755066068420264176273501354701451580424930853731245191488999731887724342512885115174853872824237863191925018415676193712562003253140276927443202251707682541813796844055771821904734125528936278921457950566975054216078541402261766645717620168828928676184585062086661069943438428009453737016388913674954661321656899115360462958864817727116179756063838264308057333138934007726602464976584961039426342748390610135215206761067742896313902468085136725214515755654144065167649442144794757021442299332980346481411405659939120613041046295543064197536090990615733218132400037369470292737483454477799095857545361137577823887518193742510875707781884043932899326953370891554930847941892191784528780453236476476667586752290919919524010419077090794678643057876460306613406236250774207318712135446125272108408598954645408345391859}, diff --git a/lib/public_key/src/pubkey_pbe.erl b/lib/public_key/src/pubkey_pbe.erl index e89e16f120..806f7c5b0f 100644 --- a/lib/public_key/src/pubkey_pbe.erl +++ b/lib/public_key/src/pubkey_pbe.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2011-2016. All Rights Reserved. +%% Copyright Ericsson AB 2011-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/public_key/src/pubkey_pem.erl b/lib/public_key/src/pubkey_pem.erl index bacc9ec600..b92790554f 100644 --- a/lib/public_key/src/pubkey_pem.erl +++ b/lib/public_key/src/pubkey_pem.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2016. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/public_key/src/public_key.erl b/lib/public_key/src/public_key.erl index 45dbd9bcc3..b34f905fc3 100644 --- a/lib/public_key/src/public_key.erl +++ b/lib/public_key/src/public_key.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2017. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/public_key/test/pbe_SUITE.erl b/lib/public_key/test/pbe_SUITE.erl index 8a5db4efec..523c9e2515 100644 --- a/lib/public_key/test/pbe_SUITE.erl +++ b/lib/public_key/test/pbe_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2011-2016. All Rights Reserved. +%% Copyright Ericsson AB 2011-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/public_key/test/public_key_SUITE.erl b/lib/public_key/test/public_key_SUITE.erl index 073bd6f4a2..cfd8e7a34b 100644 --- a/lib/public_key/test/public_key_SUITE.erl +++ b/lib/public_key/test/public_key_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2017. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/public_key/vsn.mk b/lib/public_key/vsn.mk index 99a0cc087e..5e9e463323 100644 --- a/lib/public_key/vsn.mk +++ b/lib/public_key/vsn.mk @@ -1 +1 @@ -PUBLIC_KEY_VSN = 1.5.2 +PUBLIC_KEY_VSN = 1.6.1 diff --git a/lib/reltool/doc/src/Makefile b/lib/reltool/doc/src/Makefile index e378cdf980..8daafb7a2b 100644 --- a/lib/reltool/doc/src/Makefile +++ b/lib/reltool/doc/src/Makefile @@ -88,6 +88,7 @@ clean clean_docs: fi \ done rm -rf $(HTMLDIR)/* + rm -rf $(XMLDIR) rm -f $(MAN3DIR)/* rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) rm -f errs core *~ diff --git a/lib/reltool/doc/src/notes.xml b/lib/reltool/doc/src/notes.xml index d7ad7c8dcc..52419259c6 100644 --- a/lib/reltool/doc/src/notes.xml +++ b/lib/reltool/doc/src/notes.xml @@ -5,7 +5,7 @@ <header> <copyright> <year>2009</year> - <year>2017</year> + <year>2018</year> <holder>Ericsson AB, All Rights Reserved</holder> </copyright> <legalnotice> @@ -38,7 +38,22 @@ thus constitutes one section in this document. The title of each section is the version number of Reltool.</p> - <section><title>Reltool 0.7.5</title> + <section><title>Reltool 0.7.6</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> Calls to <c>erlang:get_stacktrace()</c> are removed. + </p> + <p> + Own Id: OTP-14861</p> + </item> + </list> + </section> + +</section> + +<section><title>Reltool 0.7.5</title> <section><title>Improvements and New Features</title> <list> diff --git a/lib/reltool/src/reltool.app.src b/lib/reltool/src/reltool.app.src index dc21c1cfce..dc85464750 100644 --- a/lib/reltool/src/reltool.app.src +++ b/lib/reltool/src/reltool.app.src @@ -2,7 +2,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009-2016. All Rights Reserved. +%% Copyright Ericsson AB 2009-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/reltool/src/reltool.erl b/lib/reltool/src/reltool.erl index feb6925044..2e1dbfd713 100644 --- a/lib/reltool/src/reltool.erl +++ b/lib/reltool/src/reltool.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009-2016. All Rights Reserved. +%% Copyright Ericsson AB 2009-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/reltool/src/reltool_app_win.erl b/lib/reltool/src/reltool_app_win.erl index 335b40fa74..c84c1562ee 100644 --- a/lib/reltool/src/reltool_app_win.erl +++ b/lib/reltool/src/reltool_app_win.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009-2016. All Rights Reserved. +%% Copyright Ericsson AB 2009-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/reltool/src/reltool_fgraph_win.erl b/lib/reltool/src/reltool_fgraph_win.erl index a10a2281db..0875d5dd7d 100644 --- a/lib/reltool/src/reltool_fgraph_win.erl +++ b/lib/reltool/src/reltool_fgraph_win.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009-2016. All Rights Reserved. +%% Copyright Ericsson AB 2009-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/reltool/src/reltool_mod_win.erl b/lib/reltool/src/reltool_mod_win.erl index 8272d864c2..7df62b71be 100644 --- a/lib/reltool/src/reltool_mod_win.erl +++ b/lib/reltool/src/reltool_mod_win.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009-2017. All Rights Reserved. +%% Copyright Ericsson AB 2009-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/reltool/src/reltool_server.erl b/lib/reltool/src/reltool_server.erl index 66ee1a3742..47aba77835 100644 --- a/lib/reltool/src/reltool_server.erl +++ b/lib/reltool/src/reltool_server.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009-2016. All Rights Reserved. +%% Copyright Ericsson AB 2009-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -526,7 +526,7 @@ analyse(#state{sys=Sys} = S, Apps, Status) -> %% is included in a release (rel spec - see apps_in_rels above). %% Then initiate the same for each module, and check that there %% are no duplicated module names (in different applications) - %% where we can not decide which one to use. + %% where we cannot decide which one to use. %% Write all #app to app_tab and all #mod to mod_tab. Status2 = apps_init_is_included(S, Apps, RelApps, Status), diff --git a/lib/reltool/src/reltool_sys_win.erl b/lib/reltool/src/reltool_sys_win.erl index d87b1dc91f..e24f468f67 100644 --- a/lib/reltool/src/reltool_sys_win.erl +++ b/lib/reltool/src/reltool_sys_win.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009-2016. All Rights Reserved. +%% Copyright Ericsson AB 2009-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/reltool/src/reltool_target.erl b/lib/reltool/src/reltool_target.erl index 0eeeca4a61..64834ecc1d 100644 --- a/lib/reltool/src/reltool_target.erl +++ b/lib/reltool/src/reltool_target.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009-2017. All Rights Reserved. +%% Copyright Ericsson AB 2009-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/reltool/test/reltool_app_SUITE.erl b/lib/reltool/test/reltool_app_SUITE.erl index a51ee8875a..bd0ed5edca 100644 --- a/lib/reltool/test/reltool_app_SUITE.erl +++ b/lib/reltool/test/reltool_app_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2016. All Rights Reserved. +%% Copyright Ericsson AB 2010-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/reltool/test/reltool_manual_gui_SUITE.erl b/lib/reltool/test/reltool_manual_gui_SUITE.erl index 44da4ffd2c..3a4b012d2e 100644 --- a/lib/reltool/test/reltool_manual_gui_SUITE.erl +++ b/lib/reltool/test/reltool_manual_gui_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2012-2016. All Rights Reserved. +%% Copyright Ericsson AB 2012-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/reltool/test/reltool_server_SUITE.erl b/lib/reltool/test/reltool_server_SUITE.erl index 3622f6650c..fc976bfb94 100644 --- a/lib/reltool/test/reltool_server_SUITE.erl +++ b/lib/reltool/test/reltool_server_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009-2016. All Rights Reserved. +%% Copyright Ericsson AB 2009-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -1038,7 +1038,7 @@ create_standalone_app(Config) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% Generate standalone system with inlined archived application -%% Check that the inlined app can not be explicitly configured +%% Check that the inlined app cannot be explicitly configured create_standalone_app_clash(Config) -> %% Create archive @@ -1658,7 +1658,7 @@ set_apps_inlined(Config) -> ?m(true, Someapp1#app.is_included), ?m(true, Someapp1#app.is_pre_included), - %% Check that inlined app can not be configured + %% Check that inlined app cannot be configured Someapp2 = Someapp1#app{incl_cond=exclude}, ?msym({error, "Application someapp is inlined in '*escript* someapp'. " diff --git a/lib/reltool/test/reltool_test_lib.erl b/lib/reltool/test/reltool_test_lib.erl index 53aeb8c08c..be48ea4726 100644 --- a/lib/reltool/test/reltool_test_lib.erl +++ b/lib/reltool/test/reltool_test_lib.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009-2016. All Rights Reserved. +%% Copyright Ericsson AB 2009-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/reltool/test/reltool_wx_SUITE.erl b/lib/reltool/test/reltool_wx_SUITE.erl index 19707894ae..f6f7721762 100644 --- a/lib/reltool/test/reltool_wx_SUITE.erl +++ b/lib/reltool/test/reltool_wx_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009-2016. All Rights Reserved. +%% Copyright Ericsson AB 2009-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/reltool/test/rtt.erl b/lib/reltool/test/rtt.erl index d9e8e5520d..479241d86b 100644 --- a/lib/reltool/test/rtt.erl +++ b/lib/reltool/test/rtt.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2016. All Rights Reserved. +%% Copyright Ericsson AB 2010-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/reltool/vsn.mk b/lib/reltool/vsn.mk index 49997b1e52..c698790ede 100644 --- a/lib/reltool/vsn.mk +++ b/lib/reltool/vsn.mk @@ -1 +1 @@ -RELTOOL_VSN = 0.7.5 +RELTOOL_VSN = 0.7.6 diff --git a/lib/runtime_tools/doc/src/LTTng.xml b/lib/runtime_tools/doc/src/LTTng.xml index 4114542c74..89cbc805d8 100644 --- a/lib/runtime_tools/doc/src/LTTng.xml +++ b/lib/runtime_tools/doc/src/LTTng.xml @@ -3,7 +3,7 @@ <chapter> <header> <copyright> - <year>2016</year><year>2017</year> + <year>2016</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/runtime_tools/doc/src/Makefile b/lib/runtime_tools/doc/src/Makefile index 11583406b7..2399ed51e0 100644 --- a/lib/runtime_tools/doc/src/Makefile +++ b/lib/runtime_tools/doc/src/Makefile @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 1999-2016. All Rights Reserved. +# Copyright Ericsson AB 1999-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -120,6 +120,7 @@ debug opt: clean clean_docs: rm -rf $(HTMLDIR)/* + rm -rf $(XMLDIR) rm -f $(MAN3DIR)/* rm -f $(MAN6DIR)/* rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) diff --git a/lib/runtime_tools/doc/src/dbg.xml b/lib/runtime_tools/doc/src/dbg.xml index 276a41c415..ae60b610ed 100644 --- a/lib/runtime_tools/doc/src/dbg.xml +++ b/lib/runtime_tools/doc/src/dbg.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1996</year><year>2016</year> + <year>1996</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -113,7 +113,7 @@ <p>The imported variables will be replaced by match_spec <c>const</c> expressions, which is consistent with the static scoping for Erlang <c>fun()</c>s. Local or global - function calls can not be in the guard or body of the fun + function calls cannot be in the guard or body of the fun however. Calls to builtin match_spec functions of course is allowed:</p> <pre> @@ -756,7 +756,7 @@ Error: fun containing local erlang function calls ('is_atomm' called in guard)\ <c>cant_add_local_node</c> is returned. </p> <p>If a trace port (see <seealso marker="#trace_port-2"><c>trace_port/2</c></seealso>) is - running on the local node, remote nodes can not be traced with + running on the local node, remote nodes cannot be traced with a tracer process. The error reason <c>cant_trace_remote_pid_to_local_port</c> is returned. A trace port can however be started on the remote node with the diff --git a/lib/runtime_tools/doc/src/notes.xml b/lib/runtime_tools/doc/src/notes.xml index 355e3dd40d..5692df2ab4 100644 --- a/lib/runtime_tools/doc/src/notes.xml +++ b/lib/runtime_tools/doc/src/notes.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2004</year><year>2017</year> + <year>2004</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -32,6 +32,22 @@ <p>This document describes the changes made to the Runtime_Tools application.</p> +<section><title>Runtime_Tools 1.13</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + New utility module <c>scheduler</c> which makes it easier + to measure scheduler utilization.</p> + <p> + Own Id: OTP-14904</p> + </item> + </list> + </section> + +</section> + <section><title>Runtime_Tools 1.12.5</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/runtime_tools/doc/src/ref_man.xml b/lib/runtime_tools/doc/src/ref_man.xml index eb3a6f0f5c..fdca65422d 100644 --- a/lib/runtime_tools/doc/src/ref_man.xml +++ b/lib/runtime_tools/doc/src/ref_man.xml @@ -4,7 +4,7 @@ <application xmlns:xi="http://www.w3.org/2001/XInclude"> <header> <copyright> - <year>1999</year><year>2016</year> + <year>1999</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/runtime_tools/src/Makefile b/lib/runtime_tools/src/Makefile index 6faa9c2e35..76286c5499 100644 --- a/lib/runtime_tools/src/Makefile +++ b/lib/runtime_tools/src/Makefile @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 1999-2017. All Rights Reserved. +# Copyright Ericsson AB 1999-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. diff --git a/lib/runtime_tools/src/appmon_info.erl b/lib/runtime_tools/src/appmon_info.erl index 9c587475ca..d64206decf 100644 --- a/lib/runtime_tools/src/appmon_info.erl +++ b/lib/runtime_tools/src/appmon_info.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/runtime_tools/src/erts_alloc_config.erl b/lib/runtime_tools/src/erts_alloc_config.erl index 6c1a945484..845efaf9ef 100644 --- a/lib/runtime_tools/src/erts_alloc_config.erl +++ b/lib/runtime_tools/src/erts_alloc_config.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2016. All Rights Reserved. +%% Copyright Ericsson AB 2007-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/runtime_tools/src/msacc.erl b/lib/runtime_tools/src/msacc.erl index 37887cee72..7a3633be8e 100644 --- a/lib/runtime_tools/src/msacc.erl +++ b/lib/runtime_tools/src/msacc.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2014-2016. All Rights Reserved. +%% Copyright Ericsson AB 2014-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/runtime_tools/src/observer_backend.erl b/lib/runtime_tools/src/observer_backend.erl index 7ec3b38930..3a24986381 100644 --- a/lib/runtime_tools/src/observer_backend.erl +++ b/lib/runtime_tools/src/observer_backend.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2002-2017. All Rights Reserved. +%% Copyright Ericsson AB 2002-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -94,7 +94,7 @@ sys_info() -> {port_limit, erlang:system_info(port_limit)}, {port_count, erlang:system_info(port_count)}, {ets_limit, erlang:system_info(ets_limit)}, - {ets_count, length(ets:all())}, + {ets_count, erlang:system_info(ets_count)}, {dist_buf_busy_limit, erlang:system_info(dist_buf_busy_limit)} | MemInfo]. diff --git a/lib/runtime_tools/src/runtime_tools.app.src b/lib/runtime_tools/src/runtime_tools.app.src index 09a9b447c2..b026048b94 100644 --- a/lib/runtime_tools/src/runtime_tools.app.src +++ b/lib/runtime_tools/src/runtime_tools.app.src @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2017. All Rights Reserved. +%% Copyright Ericsson AB 1999-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/runtime_tools/src/system_information.erl b/lib/runtime_tools/src/system_information.erl index 136ee55b54..8f7bfa195b 100644 --- a/lib/runtime_tools/src/system_information.erl +++ b/lib/runtime_tools/src/system_information.erl @@ -400,7 +400,6 @@ os_getenv_erts_specific() -> "ERL_MALLOC_LIB", "ERL_MAX_PORTS", "ERL_MAX_ETS_TABLES", - "ERL_NO_VFORK", "ERL_NO_KERNEL_POLL", "ERL_THREAD_POOL_SIZE", "ERLC_EMULATOR", diff --git a/lib/runtime_tools/test/dbg_SUITE.erl b/lib/runtime_tools/test/dbg_SUITE.erl index c5dcccb887..98cbc0360f 100644 --- a/lib/runtime_tools/test/dbg_SUITE.erl +++ b/lib/runtime_tools/test/dbg_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2016. All Rights Reserved. +%% Copyright Ericsson AB 2010-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/runtime_tools/test/dyntrace_SUITE.erl b/lib/runtime_tools/test/dyntrace_SUITE.erl index 7ffbe54446..17bc104e9a 100644 --- a/lib/runtime_tools/test/dyntrace_SUITE.erl +++ b/lib/runtime_tools/test/dyntrace_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2012-2016. All Rights Reserved. +%% Copyright Ericsson AB 2012-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/runtime_tools/vsn.mk b/lib/runtime_tools/vsn.mk index 26869b9412..0a4ad61537 100644 --- a/lib/runtime_tools/vsn.mk +++ b/lib/runtime_tools/vsn.mk @@ -1 +1 @@ -RUNTIME_TOOLS_VSN = 1.12.5 +RUNTIME_TOOLS_VSN = 1.13 diff --git a/lib/sasl/doc/src/Makefile b/lib/sasl/doc/src/Makefile index baf563ca62..8e1e8b502c 100644 --- a/lib/sasl/doc/src/Makefile +++ b/lib/sasl/doc/src/Makefile @@ -101,6 +101,7 @@ debug opt: clean clean_docs: rm -rf $(HTMLDIR)/* + rm -rf $(XMLDIR) rm -f $(MAN3DIR)/* rm -f $(MAN4DIR)/* rm -f $(MAN6DIR)/* diff --git a/lib/sasl/doc/src/appup.xml b/lib/sasl/doc/src/appup.xml index d9d339884f..102ea9e5d7 100644 --- a/lib/sasl/doc/src/appup.xml +++ b/lib/sasl/doc/src/appup.xml @@ -4,7 +4,7 @@ <fileref> <header> <copyright> - <year>1997</year><year>2016</year> + <year>1997</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/sasl/doc/src/error_logging.xml b/lib/sasl/doc/src/error_logging.xml index 8731b73599..e6c244c1b9 100644 --- a/lib/sasl/doc/src/error_logging.xml +++ b/lib/sasl/doc/src/error_logging.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>1997</year><year>2016</year> + <year>1997</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -33,8 +33,8 @@ <file>error_logging.xml</file> </header> <note> - <p>The SASL error logging concept desribed in this section is - deprecated since OTP-21, when the + <p>The SASL error logging concept described in this section is + deprecated since Erlang/OTP 21.0, when the new <seealso marker="kernel:logger_chapter">logging API</seealso> was introduced.</p> <p>The new default behaviour is that the SASL application no @@ -45,9 +45,10 @@ which is setup by Kernel. <seealso marker="#progress_report">Progress reports</seealso> are by default not logged, but can be enabled - by setting the Kernel configuration - parameter <seealso marker="kernel:kernel_app#logger_progress_reports"> - <c>logger_progress_reports</c></seealso> to <c>log</c>.</p> + by setting the primary log level to <c>info</c>, for example by + using the Kernel configuration + parameter <seealso marker="kernel:kernel_app#logger_level"> + <c>logger_level</c></seealso>.</p> <p>The old SASL error logging behaviour can be re-enabled by setting the Kernel configuration parameter <seealso marker="kernel:kernel_app#logger_sasl_compatible"> @@ -70,10 +71,10 @@ <item>Progress report</item> <item>Crash report</item> </list> - <p>When the SASL application is started, it adds a logger handler + <p>When the SASL application is started, it adds a Logger handler that formats and writes these reports, as specified in the <seealso marker="sasl_app#deprecated_error_logger_config">configuration - parameters for SASL</seealso></p> + parameters for SASL</seealso>.</p> <section> <marker id="supervisor_report"/> diff --git a/lib/sasl/doc/src/notes.xml b/lib/sasl/doc/src/notes.xml index 791e9c063a..3dda5b06e8 100644 --- a/lib/sasl/doc/src/notes.xml +++ b/lib/sasl/doc/src/notes.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2004</year><year>2017</year> + <year>2004</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -31,6 +31,88 @@ </header> <p>This document describes the changes made to the SASL application.</p> +<section><title>SASL 3.2</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p>A new logging API is added to Erlang/OTP, see the + <seealso + marker="kernel:logger"><c>logger(3)</c></seealso> manual + page, and section <seealso + marker="kernel:logger_chapter">Logging</seealso> in the + Kernel User's Guide.</p> + <p>Calls to <c>error_logger</c> are automatically + redirected to the new API, and legacy error logger event + handlers can still be used. It is, however, recommended + to use the Logger API directly when writing new code.</p> + <p>Notice the following potential incompatibilities:</p> + <list> <item><p>Kernel configuration parameters + <c>error_logger</c> still works, but is overruled if the + default handler's output destination is configured with + Kernel configuration parameter <c>logger</c>.</p> <p>In + general, parameters for configuring error logger are + overwritten by new parameters for configuring + Logger.</p></item> <item><p>The concept of SASL error + logging is deprecated, meaning that by default the SASL + application does not affect which log events are + logged.</p> <p>By default, supervisor reports and crash + reports are logged by the default Logger handler started + by Kernel, and end up at the same destination (terminal + or file) as other standard log event from Erlang/OTP.</p> + <p>Progress reports are not logged by default, but can be + enabled by setting the primary log level to info, for + example with the Kernel configuration parameter + <c>logger_level</c>.</p> <p>To obtain backwards + compatibility with the SASL error logging functionality + from earlier releases, set Kernel configuration parameter + <c>logger_sasl_compatible</c> to <c>true</c>. This + prevents the default Logger handler from logging any + supervisor-, crash-, or progress reports. Instead, SASL + adds a separate Logger handler during application start, + which takes care of these log events. The SASL + configuration parameters <c>sasl_error_logger</c> and + <c>sasl_errlog_type</c> specify the destination (terminal + or file) and severity level to log for these + events.</p></item></list> + <p> + Since Logger is new in Erlang/OTP 21.0, we do reserve the + right to introduce changes to the Logger API and + functionality in patches following this release. These + changes might or might not be backwards compatible with + the initial version.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-13295</p> + </item> + <item> + <p> + The old and outdated "Status Inspection" tool (modules + <c>si</c> and <c>si_sasl_sup</c>) is removed.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-14469</p> + </item> + <item> + <p> + When creating the release tar file, systools now includes + sys.config.src if it exists in the + $ROOT/releases/<vsn>/ directory. This is to allow + adjustments, e.g. resolving environment variables, after + unpacking the release, but before installing it. This + functionality requires a custom tool which uses + sys.config.src as input and creates a correct sys.config + file.</p> + <p> + Own Id: OTP-14950 Aux Id: PR-1560 </p> + </item> + </list> + </section> + +</section> + <section><title>SASL 3.1.2</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/sasl/doc/src/release_handler.xml b/lib/sasl/doc/src/release_handler.xml index 975188f489..9ba276aeac 100644 --- a/lib/sasl/doc/src/release_handler.xml +++ b/lib/sasl/doc/src/release_handler.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1996</year><year>2016</year> + <year>1996</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/sasl/doc/src/sasl_app.xml b/lib/sasl/doc/src/sasl_app.xml index be275879ee..fc83f63fe6 100644 --- a/lib/sasl/doc/src/sasl_app.xml +++ b/lib/sasl/doc/src/sasl_app.xml @@ -4,7 +4,7 @@ <appref> <header> <copyright> - <year>1996</year><year>2016</year> + <year>1996</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -102,10 +102,11 @@ reports</em>, <em>crash reports</em>, and <em>progress reports</em>. These reports are now also printed by the default logger handler started by the Kernel application. Progress - reports are by default stopped by a filter, but can be - added by setting the Kernel configuration - parameter <seealso marker="kernel:kernel_app#logger_progress_reports"> - <c>logger_progress_reports</c></seealso> to <c>log</c>.</p> + reports are by default stopped by the primary log level, but can + be enabled by setting this level to <c>info</c>, for example by + using the Kernel configuration + parameter <seealso marker="kernel:kernel_app#logger_level"> + <c>logger_level</c></seealso>.</p> <p>If the old error logger event handlers are still desired, they must be added by calling <c>error_logger:add_report_handler/1,2</c>.</p> @@ -131,14 +132,13 @@ </taglist> <p>A similar behaviour, but still using the new logger API, can be obtained by setting the Kernel application environment - variable <seealso marker="kernel:kernel_app#logger_sasl_compatible"><c>logger_sasl_compatible=true</c></seealso>. This will add a - second instance of the standard logger handler - named <c>sasl</c>, which will only print the SASL reports. No - SASL reports will then be printed by the Kernel logger - handler.</p> - <p>The <c>sasl</c> handler will be configured according to the - values of the following SASL application environment - variables.</p> + variable <seealso marker="kernel:kernel_app#logger_sasl_compatible"> + <c>logger_sasl_compatible</c></seealso> to <c>true</c>. This + adds a second instance of the standard Logger handler, + named <c>sasl</c>, which only prints the SASL reports. No SASL + reports are then printed by the Kernel logger handler.</p> + <p>The <c>sasl</c> handler is configured according to the values + of the following SASL application environment variables.</p> <taglist> <tag><c><![CDATA[sasl_error_logger = Value ]]></c></tag> <item> diff --git a/lib/sasl/src/Makefile b/lib/sasl/src/Makefile index 45cd814bf8..7338bdf016 100644 --- a/lib/sasl/src/Makefile +++ b/lib/sasl/src/Makefile @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 1996-2016. All Rights Reserved. +# Copyright Ericsson AB 1996-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. diff --git a/lib/sasl/src/rb.erl b/lib/sasl/src/rb.erl index 28829132a1..bef4268d3a 100644 --- a/lib/sasl/src/rb.erl +++ b/lib/sasl/src/rb.erl @@ -890,7 +890,7 @@ read_rep(Fd, FilePosition, Device, Abort, Log) -> handle_bad_form(Date, Msg, Device, Abort, Log) -> io:format("rb: ERROR! A report on bad form was encountered. " ++ - "It can not be printed to the log.~n~n"), + "It cannot be printed to the log.~n~n"), io:format("Details:~n~p ~tp~n~n", [Date,Msg]), case {Abort,Device,open_log_file(Log)} of {true,standard_io,standard_io} -> diff --git a/lib/sasl/src/release_handler.erl b/lib/sasl/src/release_handler.erl index 4fd3fc0d36..48feac1a21 100644 --- a/lib/sasl/src/release_handler.erl +++ b/lib/sasl/src/release_handler.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2017. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -1120,7 +1120,7 @@ new_emulator_make_hybrid_config(CurrentVsn,ToVsn,TmpVsn,RelDir,Masters) -> {ok,[FC]} -> FC; {error,Error1} -> - io:format("Warning: ~w can not read ~tp: ~tp~n", + io:format("Warning: ~w cannot read ~tp: ~tp~n", [?MODULE,FromFile,Error1]), [] end, @@ -1130,7 +1130,7 @@ new_emulator_make_hybrid_config(CurrentVsn,ToVsn,TmpVsn,RelDir,Masters) -> {ok,[ToConfig]} -> [lists:keyfind(App,1,ToConfig) || App <- [kernel,stdlib,sasl]]; {error,Error2} -> - io:format("Warning: ~w can not read ~tp: ~tp~n", + io:format("Warning: ~w cannot read ~tp: ~tp~n", [?MODULE,ToFile,Error2]), [false,false,false] end, diff --git a/lib/sasl/src/release_handler_1.erl b/lib/sasl/src/release_handler_1.erl index ca97515299..bf18691687 100644 --- a/lib/sasl/src/release_handler_1.erl +++ b/lib/sasl/src/release_handler_1.erl @@ -147,7 +147,7 @@ split_instructions([], Before) -> %% If PrePurgeMethod == soft_purge, the function will succeed %% only if there is no process running old code of any of the %% modules. Else it will throw {error,Mod}, where Mod is the -%% first module found that can not be soft_purged. +%% first module found that cannot be soft_purged. %% %% If PrePurgeMethod == brutal_purge, the function will %% always succeed and return a list of all modules that are diff --git a/lib/sasl/src/sasl.app.src b/lib/sasl/src/sasl.app.src index 60d08ffa54..688aff16f1 100644 --- a/lib/sasl/src/sasl.app.src +++ b/lib/sasl/src/sasl.app.src @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/sasl/src/sasl.appup.src b/lib/sasl/src/sasl.appup.src index 221427874c..83ee328af2 100644 --- a/lib/sasl/src/sasl.appup.src +++ b/lib/sasl/src/sasl.appup.src @@ -1,7 +1,7 @@ %% -*- erlang -*- %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2016. All Rights Reserved. +%% Copyright Ericsson AB 1999-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/sasl/src/sasl.erl b/lib/sasl/src/sasl.erl index 92b80694d7..0c68c93dc6 100644 --- a/lib/sasl/src/sasl.erl +++ b/lib/sasl/src/sasl.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -131,6 +131,10 @@ add_sasl_logger(std, undefined) -> ok; add_sasl_logger(Dest, Level) -> FC = #{legacy_header=>true, single_line=>false}, + case Level of + info -> allow_progress(); + _ -> ok + end, ok = logger:add_handler(sasl,logger_std_h, #{level=>Level, filter_default=>stop, @@ -139,8 +143,8 @@ add_sasl_logger(Dest, Level) -> {fun logger_filters:remote_gl/2,stop}}, {sasl_domain, {fun logger_filters:domain/2, - {log,equal,[beam,erlang,otp,sasl]}}}], - logger_std_h=>#{type=>Dest}, + {log,equal,[otp,sasl]}}}], + config=>#{type=>Dest}, formatter=>{logger_formatter,FC}}). delete_sasl_logger(undefined) -> ok; @@ -151,6 +155,7 @@ delete_sasl_logger(_Type) -> add_error_logger_mf(undefined) -> ok; add_error_logger_mf({Dir, MaxB, MaxF}) -> + allow_progress(), error_logger:add_report_handler( log_mf_h, log_mf_h:init(Dir, MaxB, MaxF, fun pred/1)). @@ -161,6 +166,13 @@ delete_error_logger_mf(_) -> pred({_Type, GL, _Msg}) when node(GL) =/= node() -> false; pred(_) -> true. +allow_progress() -> + #{level:=PL} = logger:get_primary_config(), + case logger:compare_levels(info,PL) of + lt -> ok = logger:set_primary_config(level,info); + _ -> ok + end. + %%%----------------------------------------------------------------- %%% supervisor functionality %%%----------------------------------------------------------------- diff --git a/lib/sasl/src/sasl_report_file_h.erl b/lib/sasl/src/sasl_report_file_h.erl index d3b5c7dc0d..05d6acd076 100644 --- a/lib/sasl/src/sasl_report_file_h.erl +++ b/lib/sasl/src/sasl_report_file_h.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/sasl/test/installer.erl b/lib/sasl/test/installer.erl index e38d0cfa7b..5429008a5f 100644 --- a/lib/sasl/test/installer.erl +++ b/lib/sasl/test/installer.erl @@ -905,7 +905,7 @@ start_client(TestNode,Client,Sname) -> wait_started(TestNode,Node) after 30000 -> ?print([{start_client,failed,Node},net_adm:ping(Node)]), - ?fail({"can not start", Node}) + ?fail({"cannot start", Node}) end. start_client_unix(TestNode,Sname,Node) -> diff --git a/lib/sasl/test/rb_SUITE.erl b/lib/sasl/test/rb_SUITE.erl index 4ba2540545..e5ca1775d5 100644 --- a/lib/sasl/test/rb_SUITE.erl +++ b/lib/sasl/test/rb_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2011-2016. All Rights Reserved. +%% Copyright Ericsson AB 2011-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -423,7 +423,7 @@ start_stop_log(Config) -> StdioResult2 = capture(fun() -> rb:log_list() end), {ok,<<>>} = file:read_file(OutFile), - %% Test that standard_io is used if log file can not be opened + %% Test that standard_io is used if log file cannot be opened ok = rb:start_log(filename:join(nonexistingdir,"newfile.txt")), StdioResult = capture(fun() -> rb:show(1) end), {ok,<<>>} = file:read_file(OutFile), diff --git a/lib/sasl/test/release_handler_SUITE.erl b/lib/sasl/test/release_handler_SUITE.erl index edd2efdf05..2ff2f29591 100644 --- a/lib/sasl/test/release_handler_SUITE.erl +++ b/lib/sasl/test/release_handler_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2011-2016. All Rights Reserved. +%% Copyright Ericsson AB 2011-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/sasl/test/sasl_SUITE.erl b/lib/sasl/test/sasl_SUITE.erl index 7b63684c53..fc80e37210 100644 --- a/lib/sasl/test/sasl_SUITE.erl +++ b/lib/sasl/test/sasl_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2011-2016. All Rights Reserved. +%% Copyright Ericsson AB 2011-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/sasl/test/sasl_report_SUITE.erl b/lib/sasl/test/sasl_report_SUITE.erl index 72ee2f0a10..a03932133e 100644 --- a/lib/sasl/test/sasl_report_SUITE.erl +++ b/lib/sasl/test/sasl_report_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2015-2017. All Rights Reserved. +%% Copyright Ericsson AB 2015-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -21,6 +21,7 @@ -export([all/0, suite/0,groups/0,init_per_suite/1, end_per_suite/1, init_per_group/2,end_per_group/2]). -export([gen_server_crash/1, gen_server_crash_unicode/1]). +-export([legacy_gen_server_crash/1, legacy_gen_server_crash_unicode/1]). -export([crash_me/0,start_link/0,init/1,handle_cast/2,terminate/2]). @@ -29,7 +30,10 @@ suite() -> [{ct_hooks,[ts_install_cth]}]. all() -> - [gen_server_crash, gen_server_crash_unicode]. + [gen_server_crash, + gen_server_crash_unicode, + legacy_gen_server_crash, + legacy_gen_server_crash_unicode]. groups() -> []. @@ -52,17 +56,77 @@ gen_server_crash(Config) -> gen_server_crash_unicode(Config) -> gen_server_crash(Config, unicode). +legacy_gen_server_crash(Config) -> + legacy_gen_server_crash(Config,latin1). + +legacy_gen_server_crash_unicode(Config) -> + legacy_gen_server_crash(Config,unicode). + gen_server_crash(Config, Encoding) -> + TC = list_to_atom(lists:concat([?FUNCTION_NAME,"_",Encoding])), + PrivDir = filename:join(?config(priv_dir,Config),?MODULE), + ConfigFileName = filename:join(PrivDir,TC), + KernelLog = filename:join(PrivDir,lists:concat([TC,"_kernel.log"])), + SaslLog = filename:join(PrivDir,lists:concat([TC,"_sasl.log"])), + KernelConfig = [{logger,[{handler,default,logger_std_h, + #{config=>#{type=>{file,KernelLog}}}}]}, + {logger_sasl_compatible,true}], + Modes = [write, {encoding, Encoding}], + SaslConfig = [{sasl_error_logger,{file,SaslLog,Modes}}], + ok = filelib:ensure_dir(SaslLog), + + ok = file:write_file(ConfigFileName ++ ".config", + io_lib:format("[{kernel, ~p},~n{sasl, ~p}].", + [KernelConfig,SaslConfig])), + {ok,Node} = + test_server:start_node( + TC, peer, + [{args, ["-pa ",filename:dirname(code:which(?MODULE)), + " -boot start_sasl -kernel start_timer true " + "-config ",ConfigFileName]}]), + + %% Set depth + ok = rpc:call(Node,logger,update_formatter_config,[default,depth,30]), + ok = rpc:call(Node,logger,update_formatter_config,[sasl,depth,30]), + + %% Make sure remote node logs it's own logs, and current node does + %% not log them. + ok = rpc:call(Node,logger,remove_handler_filter,[default,remote_gl]), + ok = rpc:call(Node,logger,remove_handler_filter,[sasl,remote_gl]), + ok = logger:add_primary_filter(no_remote,{fun(#{meta:=#{pid:=Pid}},_) + when node(Pid)=/=node() -> stop; + (_,_) -> ignore + end,[]}), + ct:log("Local node Logger config:~n~p", + [rpc:call(Node,logger,get_config,[])]), + ct:log("Remote node Logger config:~n~p", + [rpc:call(Node,logger,get_config,[])]), + ct:log("Remote node error_logger handlers: ~p", + [rpc:call(Node,error_logger,which_report_handlers,[])]), + + ok = rpc:call(Node,?MODULE,crash_me,[]), + + test_server:stop_node(Node), + ok = logger:remove_primary_filter(no_remote), + + check_file(KernelLog, utf8, 70000, 150000), + check_file(SaslLog, Encoding, 70000, 150000), + + ok = file:delete(KernelLog), + ok = file:delete(SaslLog), + + ok. + +legacy_gen_server_crash(Config, Encoding) -> StopFilter = {fun(_,_) -> stop end, ok}, logger:add_handler_filter(default,stop_all,StopFilter), + logger:add_handler_filter(sasl,stop_all,StopFilter), logger:add_handler_filter(cth_log_redirect,stop_all,StopFilter), try do_gen_server_crash(Config, Encoding) after - ok = application:unset_env(kernel, logger_sasl_compatible), - ok = application:unset_env(sasl, sasl_error_logger), - ok = application:unset_env(kernel, error_logger_format_depth), logger:remove_handler_filter(default,stop_all), + logger:remove_handler_filter(sasl,stop_all), logger:remove_handler_filter(cth_log_redirect,stop_all) end, ok. @@ -74,22 +138,17 @@ do_gen_server_crash(Config, Encoding) -> SaslLog = filename:join(LogDir, "sasl.log"), ok = filelib:ensure_dir(SaslLog), - application:stop(sasl), Modes = [write, {encoding, Encoding}], - ok = application:set_env(kernel, logger_sasl_compatible, true), - ok = application:set_env(sasl, sasl_error_logger, {file,SaslLog,Modes}, - [{persistent,true}]), application:set_env(kernel, error_logger_format_depth, 30), error_logger:logfile({open,KernelLog}), - application:start(sasl), - logger:i(print), + error_logger:add_report_handler(sasl_report_file_h,{SaslLog,Modes,all}), + ct:log("Logger config:~n~p",[logger:get_config()]), ct:log("error_logger handlers: ~p",[error_logger:which_report_handlers()]), crash_me(), - error_logger:logfile(close), - application:stop(sasl), + error_logger:delete_report_handler(sasl_report_file_h), check_file(KernelLog, utf8, 70000, 150000), check_file(SaslLog, Encoding, 70000, 150000), diff --git a/lib/sasl/test/systools_SUITE.erl b/lib/sasl/test/systools_SUITE.erl index ad61186921..6c913850b9 100644 --- a/lib/sasl/test/systools_SUITE.erl +++ b/lib/sasl/test/systools_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2012-2017. All Rights Reserved. +%% Copyright Ericsson AB 2012-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -1654,7 +1654,7 @@ abnormal_relup(Config) when is_list(Config) -> ok. -%% make_relup: Check relup can not be created is sasl is not in rel file. +%% make_relup: Check relup cannot be created is sasl is not in rel file. no_sasl_relup(Config) when is_list(Config) -> {ok, OldDir} = file:get_cwd(), {Dir1,Name1} = create_script(latest1_no_sasl,Config), diff --git a/lib/sasl/vsn.mk b/lib/sasl/vsn.mk index 52b168598a..0972ae20ba 100644 --- a/lib/sasl/vsn.mk +++ b/lib/sasl/vsn.mk @@ -1 +1 @@ -SASL_VSN = 3.1.2 +SASL_VSN = 3.2 diff --git a/lib/snmp/doc/src/Makefile b/lib/snmp/doc/src/Makefile index 3ebee792f9..a6a9477b05 100644 --- a/lib/snmp/doc/src/Makefile +++ b/lib/snmp/doc/src/Makefile @@ -140,6 +140,7 @@ clean_man: clean_html: @echo "cleaning html:" rm -rf $(HTMLDIR)/* + rm -rf $(XMLDIR) $(MAN7DIR)/%.7: $(MIBSDIR)/%.mib @echo "processing $*" diff --git a/lib/snmp/doc/src/notes.xml b/lib/snmp/doc/src/notes.xml index 8d48cb911d..f64e0cca97 100644 --- a/lib/snmp/doc/src/notes.xml +++ b/lib/snmp/doc/src/notes.xml @@ -34,7 +34,23 @@ </header> - <section><title>SNMP 5.2.10</title> + <section><title>SNMP 5.2.11</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + The Snmp MIB compiler now allows using a + TEXTUAL-CONVENTION type before defining it.</p> + <p> + Own Id: OTP-14196 Aux Id: ERIERL-161 </p> + </item> + </list> + </section> + +</section> + +<section><title>SNMP 5.2.10</title> <section><title>Fixed Bugs and Malfunctions</title> <list> diff --git a/lib/snmp/src/agent/snmpa_set_lib.erl b/lib/snmp/src/agent/snmpa_set_lib.erl index 57507a36e8..97b8ddf7c4 100644 --- a/lib/snmp/src/agent/snmpa_set_lib.erl +++ b/lib/snmp/src/agent/snmpa_set_lib.erl @@ -46,9 +46,9 @@ %%* 6) IF value is outside the acceptable range THEN wrongValue. %% 7) IF variable does not exist and could not ever be created %% THEN noCreation. -%% 8) IF variable can not be created now THEN inconsistentName. -%% 9) IF value can not be set now THEN inconsistentValue. -%%* 9) IF instances of the variable can not be modified THEN notWritable. +%% 8) IF variable cannot be created now THEN inconsistentName. +%% 9) IF value cannot be set now THEN inconsistentValue. +%%* 9) IF instances of the variable cannot be modified THEN notWritable. %% 10) IF an unavailable resource is needed THEN resourceUnavailable. %% 11) IF any other error THEN genErr. %% 12) Otherwise ok! diff --git a/lib/snmp/src/agent/snmpa_trap.erl b/lib/snmp/src/agent/snmpa_trap.erl index e75016f7ec..293d1f3ccf 100644 --- a/lib/snmp/src/agent/snmpa_trap.erl +++ b/lib/snmp/src/agent/snmpa_trap.erl @@ -830,11 +830,11 @@ do_send_v1_trap(Enter, Spec, V1Res, NVbs, ExtraInfo, NetIf, SysUpTime) -> case lists:keyfind(transportDomainUdpIpv4, 1, Transports) of false -> ?vtrace( - "snmpa_trap: can not send v1 trap " + "snmpa_trap: cannot send v1 trap " "without IPv4 domain: ~p", [Transports]), user_err( - "snmpa_trap: can not send v1 trap " + "snmpa_trap: cannot send v1 trap " "without IPv4 domain: ~p", [Transports]); DomainAddr -> diff --git a/lib/snmp/src/compile/snmpc.erl b/lib/snmp/src/compile/snmpc.erl index 4416626a4c..7f627d66d9 100644 --- a/lib/snmp/src/compile/snmpc.erl +++ b/lib/snmp/src/compile/snmpc.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2017. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -456,6 +456,7 @@ compile_parsed_data(#pdata{mib_name = MibName, RelChk = get_relaxed_row_name_assign_check(Opts), Data = #dldata{deprecated = Deprecated, relaxed_row_name_assign_check = RelChk}, + mc_new_type_loop(Definitions), put(augmentations, false), definitions_loop(Definitions, Data), MibName. @@ -481,7 +482,40 @@ do_update_imports([{{Mib, ImportsFromMib0},_Line}|Imports], Acc) -> update_status(Name, Status) -> #cdata{status_ets = Ets} = get(cdata), ets:insert(Ets, {Name, Status}). - + + +mc_new_type_loop( + [{#mc_new_type{ + name = NewTypeName, + macro = Macro, + syntax = OldType, + display_hint = DisplayHint},Line}|T]) -> + ?vlog2("typeloop -> new_type:" + "~n Macro: ~p" + "~n NewTypeName: ~p" + "~n OldType: ~p" + "~n DisplayHint: ~p", + [Macro, NewTypeName, OldType, DisplayHint], Line), + ensure_macro_imported(Macro,Line), + Types = (get(cdata))#cdata.asn1_types, + case lists:keysearch(NewTypeName, #asn1_type.aliasname, Types) of + {value,_} -> + snmpc_lib:print_error("Type ~w already defined.", + [NewTypeName],Line); + false -> + %% NameOfOldType = element(2,OldType), + ASN1 = snmpc_lib:make_ASN1type(OldType), + snmpc_lib:add_cdata(#cdata.asn1_types, + [ASN1#asn1_type{aliasname = NewTypeName, + imported = false, + display_hint = DisplayHint}]) + end, + mc_new_type_loop(T); +mc_new_type_loop([_|T]) -> + mc_new_type_loop(T); +mc_new_type_loop([]) -> + ok. + %% A deprecated object definitions_loop([{#mc_object_type{name = ObjName, status = deprecated}, @@ -745,32 +779,8 @@ definitions_loop([{#mc_object_type{name = NameOfTable, ColMEs]), definitions_loop(RestObjs, Data); -definitions_loop([{#mc_new_type{name = NewTypeName, - macro = Macro, - syntax = OldType, - display_hint = DisplayHint},Line}|T], - Data) -> - ?vlog2("defloop -> new_type:" - "~n Macro: ~p" - "~n NewTypeName: ~p" - "~n OldType: ~p" - "~n DisplayHint: ~p", - [Macro, NewTypeName, OldType, DisplayHint], Line), - ensure_macro_imported(Macro,Line), - Types = (get(cdata))#cdata.asn1_types, - case lists:keysearch(NewTypeName, #asn1_type.aliasname, Types) of - {value,_} -> - snmpc_lib:print_error("Type ~w already defined.", - [NewTypeName],Line); - false -> - %% NameOfOldType = element(2,OldType), - ASN1 = snmpc_lib:make_ASN1type(OldType), - snmpc_lib:add_cdata(#cdata.asn1_types, - [ASN1#asn1_type{aliasname = NewTypeName, - imported = false, - display_hint = DisplayHint}]) - end, - definitions_loop(T, Data); +definitions_loop([{#mc_new_type{},_}|T], Data) -> + definitions_loop(T, Data); %% Plain variable definitions_loop([{#mc_object_type{name = NewVarName, diff --git a/lib/snmp/test/snmp_compiler_test.erl b/lib/snmp/test/snmp_compiler_test.erl index 2b6bba4ee6..0a7b729d1f 100644 --- a/lib/snmp/test/snmp_compiler_test.erl +++ b/lib/snmp/test/snmp_compiler_test.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2003-2017. All Rights Reserved. +%% Copyright Ericsson AB 2003-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -58,7 +58,8 @@ otp_10799/1, otp_10808/1, otp_14145/1, - otp_13014/1 + otp_13014/1, + otp_14196/1 ]). %%---------------------------------------------------------------------- @@ -138,7 +139,7 @@ all() -> groups() -> [{tickets, [], [otp_6150, otp_8574, otp_8595, otp_10799, otp_10808, otp_14145, - otp_13014]}]. + otp_13014, otp_14196]}]. init_per_group(_GroupName, Config) -> Config. @@ -489,6 +490,22 @@ otp_13014(Config) when is_list(Config) -> TableInfo, ok. +%%====================================================================== + +otp_14196(suite) -> + []; +otp_14196(Config) when is_list(Config) -> + put(tname, otp14196), + p("starting with Config: ~p~n", [Config]), + + Dir = ?config(case_top_dir, Config), + MibDir = ?config(mib_dir, Config), + MibFile = join(MibDir, "OTP14196-MIB.mib"), + ?line {ok, Mib} = + snmpc:compile(MibFile, [{outdir, Dir}, {verbosity, trace}]), + p("Mib: ~n~p~n", [Mib]), + ok. + %%====================================================================== diff --git a/lib/snmp/test/snmp_test_data/OTP14196-MIB.mib b/lib/snmp/test/snmp_test_data/OTP14196-MIB.mib new file mode 100644 index 0000000000..0b3c718a02 --- /dev/null +++ b/lib/snmp/test/snmp_test_data/OTP14196-MIB.mib @@ -0,0 +1,47 @@ +OTP14196-MIB DEFINITIONS ::= BEGIN + +IMPORTS + MODULE-IDENTITY, OBJECT-TYPE, snmpModules, mib-2 + FROM SNMPv2-SMI + TEXTUAL-CONVENTION + FROM SNMPv2-TC + OBJECT-GROUP + FROM SNMPv2-CONF + ; + +otp14196MIB MODULE-IDENTITY + LAST-UPDATED "1004210000Z" + ORGANIZATION "" + CONTACT-INFO + "" + DESCRIPTION + "Test mib for OTP-14196" + ::= { snmpModules 1 } + +testCompliances OBJECT IDENTIFIER ::= { otp14196MIB 1 } +test OBJECT IDENTIFIER ::= { mib-2 15 } + +typeA OBJECT-TYPE + SYNTAX TypeAType + MAX-ACCESS accessible-for-notify + STATUS current + DESCRIPTION + "Test type for OTP-14196" + ::= { test 4711 } + +TypeAType ::= TEXTUAL-CONVENTION + STATUS current + DESCRIPTION + "" + SYNTAX INTEGER + +testGroups OBJECT IDENTIFIER ::= { testCompliances 1 } + +testGroupA OBJECT-GROUP + OBJECTS { typeA } + STATUS current + DESCRIPTION + "" + ::= { testGroups 17 } + +END diff --git a/lib/snmp/test/test-mibs/ALARM-MIB.mib b/lib/snmp/test/test-mibs/ALARM-MIB.mib index 18e43d4b4b..10d2adbff9 100644 --- a/lib/snmp/test/test-mibs/ALARM-MIB.mib +++ b/lib/snmp/test/test-mibs/ALARM-MIB.mib @@ -330,7 +330,7 @@ alarmModelRowStatus OBJECT-TYPE cannot be used as an alarm suppression mechanism. Entries that are notInService will disappear as described in RFC2579. - This row can not be modified while it is being + This row cannot be modified while it is being referenced by a value of alarmActiveModelPointer. In these cases, an error of `inconsistentValue' will be returned to the manager. diff --git a/lib/snmp/test/test-mibs/SNMPv2-TC.mib b/lib/snmp/test/test-mibs/SNMPv2-TC.mib index fd6a728ab5..1d75c4bbd8 100644 --- a/lib/snmp/test/test-mibs/SNMPv2-TC.mib +++ b/lib/snmp/test/test-mibs/SNMPv2-TC.mib @@ -454,7 +454,7 @@ value | | see 1| ->C| ->D associated with this column or that there is no conceptual row for which this column would be accessible in the MIB view used by the retrieval. As - such, the management station can not issue any + such, the management station cannot issue any management protocol set operations to create an instance of this column. @@ -576,7 +576,7 @@ value | | see 1| ->C| ->D associated with this column or that there is no conceptual row for which this column would be accessible in the MIB view used by the retrieval. As - such, the management station can not issue any + such, the management station cannot issue any management protocol set operations to create an instance of this column. diff --git a/lib/snmp/vsn.mk b/lib/snmp/vsn.mk index 2c97683625..96123f02f5 100644 --- a/lib/snmp/vsn.mk +++ b/lib/snmp/vsn.mk @@ -19,6 +19,6 @@ # %CopyrightEnd% APPLICATION = snmp -SNMP_VSN = 5.2.10 +SNMP_VSN = 5.2.11 PRE_VSN = APP_VSN = "$(APPLICATION)-$(SNMP_VSN)$(PRE_VSN)" diff --git a/lib/ssh/doc/src/Makefile b/lib/ssh/doc/src/Makefile index 7c4dbd7af8..77fa356092 100644 --- a/lib/ssh/doc/src/Makefile +++ b/lib/ssh/doc/src/Makefile @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 2004-2017. All Rights Reserved. +# Copyright Ericsson AB 2004-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -119,6 +119,7 @@ html: images $(HTML_REF_MAN_FILE) clean clean_docs: rm -rf $(HTMLDIR)/* + rm -rf $(XMLDIR) rm -f $(MAN3DIR)/* rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) rm -f $(SPECS_FILES) diff --git a/lib/ssh/doc/src/configure_algos.xml b/lib/ssh/doc/src/configure_algos.xml index 15aece8968..fa45b1cb4c 100644 --- a/lib/ssh/doc/src/configure_algos.xml +++ b/lib/ssh/doc/src/configure_algos.xml @@ -5,7 +5,7 @@ <header> <copyright> <year>2017</year> - <year>2017</year> + <year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/ssh/doc/src/introduction.xml b/lib/ssh/doc/src/introduction.xml index 6fd8425adf..8444daf0cc 100644 --- a/lib/ssh/doc/src/introduction.xml +++ b/lib/ssh/doc/src/introduction.xml @@ -5,7 +5,7 @@ <header> <copyright> <year>2012</year> - <year>2016</year> + <year>2018</year> <holder>Ericsson AB, All Rights Reserved</holder> </copyright> <legalnotice> diff --git a/lib/ssh/doc/src/notes.xml b/lib/ssh/doc/src/notes.xml index d0ed674eee..58e8c9ab5b 100644 --- a/lib/ssh/doc/src/notes.xml +++ b/lib/ssh/doc/src/notes.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2004</year><year>2017</year> + <year>2004</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -30,6 +30,195 @@ <file>notes.xml</file> </header> +<section><title>Ssh 4.7</title> + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + If the daemon port listener is restarted, it could + potentially fail with <c>eaddrinuse</c> if the timing is + unlucky. It will now retry and exponentially back off the + listener restart a few times before failing.</p> + <p> + Own Id: OTP-14955</p> + </item> + <item> + <p> + A channel callback module always got the module name as + reason in a call to terminate. Now it will get the proper + Reason, usually 'normal'.</p> + <p> + Own Id: OTP-15084</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + The option <c>exec</c> has new option values defined to + make it much more easy to implement an own <c>exec</c> + server.</p> + <p> + An option called <c>exec</c> for daemons implementing the + handling of 'exec' requests has existed a long time but + has been undocumented. The old undocumented value - as + well as its behavior - is kept for compatibility EXCEPT + that error messages are changed and are sent as + "stderror" text.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-14851</p> + </item> + <item> + <p> + Updated ssh_connection:shell/2 documentation.</p> + <p> + Own Id: OTP-14880</p> + </item> + <item> + <p> + The experimental <c>ssh_dbg</c> module is completely + re-written. Its purpose is to make tracing and debugging + easier on deployed systems.</p> + <p> + Own Id: OTP-14896</p> + </item> + <item> + <p> + The SSH supervisor structure has been slightly changed. + This makes stopping the ssh application considerably + faster if there are open connections. This is important + in for example restarts.</p> + <p> + Own Id: OTP-14988</p> + </item> + <item> + <p> + The type specifications in SSH are completly reworked and + the following types are renamed:</p> + <p> + <c>ssh:ssh_connection_ref()</c> is changed to + <c>ssh:connection_ref()</c>, </p> + <p> + <c>ssh:ssh_daemon_ref()</c> is changed to + <c>ssh:daemon_ref()</c>,</p> + <p> + <c>ssh:ssh_channel_id()</c> is changed to + <c>ssh:channel_id()</c>.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-15002 Aux Id: OTP-15030 </p> + </item> + <item> + <p> + The internal timer handling in SSH is now based on the + gen_statem timers.</p> + <p> + Own Id: OTP-15019</p> + </item> + <item> + <p> + Removed the undocumented and unused modules + <c>ssh_client_key.erl</c> and <c>ssh_server_key.erl</c>.</p> + <p> + Own Id: OTP-15028</p> + </item> + <item> + <p> + The Reference Manual pages are partly updated.</p> + <p> + The ssh page is now generated from specs and types, is + restructured and is partly rephrased.</p> + <p> + The ssh_channel, ssh_connection, ssh_client_key_api, + ssh_server_key_api and ssh_sftp pages are updated with + links, correct type names and some minor changes.</p> + <p> + Own Id: OTP-15030 Aux Id: OTP-15002 </p> + </item> + <item> + <p> + The behaviors <c>ssh_channel</c> and + <c>ssh_daemon_channel</c> are renamed to + <c>ssh_client_channel</c> and <c>ssh_server_channel</c> + respectively.</p> + <p> + The old modules are kept for compatibility but should + preferably be replaced when updating callback modules + referring them.</p> + <p> + Own Id: OTP-15041</p> + </item> + <item> + <p> + New test suite for channels.</p> + <p> + Own Id: OTP-15051</p> + </item> + <item> + <p> + The <c>rekey_limit</c> option could now set the max time + as well as the previously max data amount.</p> + <p> + Own Id: OTP-15069 Aux Id: ERL-617 </p> + </item> + <item> + <p> + Changed process exit supervision from links to monitors.</p> + <p> + Own Id: OTP-15082</p> + </item> + <item> + <p> + Better handling of misbehaving channel callback modules.</p> + <p> + Own Id: OTP-15083</p> + </item> + <item> + <p> + A new moduli file is generated. This file is used for the + recommended <c>diffie-hellman-group-exchange-sha256</c> + key exchange algorithm in SSH.</p> + <p> + Own Id: OTP-15113</p> + </item> + </list> + </section> +</section> + +<section><title>Ssh 4.6.9.1</title> + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + SFTP clients reported the error reason <c>""</c> if a + non-OTP sftp server was killed during a long file + transmission.</p> + <p> + Now the signal name (for example <c>"KILL"</c>) will be + the error reason if the server's reason is empty.</p> + <p> + The documentation also lacked type information about this + class of errors.</p> + <p> + Own Id: OTP-15148 Aux Id: ERIERL-194 </p> + </item> + <item> + <p> + Fix ssh_sftp decode error for sftp protocol version 4</p> + <p> + Own Id: OTP-15149 Aux Id: ERIERL-199 </p> + </item> + </list> + </section> + +</section> + <section><title>Ssh 4.6.9</title> <section><title>Fixed Bugs and Malfunctions</title> @@ -2576,7 +2765,7 @@ </item> <item> <p> - Fixed internal error on when client and server can not + Fixed internal error on when client and server cannot agree o which authmethod to use.</p> <p> Own Id: OTP-10731 Aux Id: seq12237 </p> diff --git a/lib/ssh/doc/src/ref_man.xml b/lib/ssh/doc/src/ref_man.xml index 76e6520f94..df37b0244f 100644 --- a/lib/ssh/doc/src/ref_man.xml +++ b/lib/ssh/doc/src/ref_man.xml @@ -4,7 +4,7 @@ <application xmlns:xi="http://www.w3.org/2001/XInclude"> <header> <copyright> - <year>2004</year><year>2016</year> + <year>2004</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/ssh/doc/src/ssh.xml b/lib/ssh/doc/src/ssh.xml index 407956cc6f..f238bf2ca8 100644 --- a/lib/ssh/doc/src/ssh.xml +++ b/lib/ssh/doc/src/ssh.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>2004</year><year>2017</year> + <year>2004</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/ssh/doc/src/ssh_app.xml b/lib/ssh/doc/src/ssh_app.xml index 2ebd176e12..9ec909d733 100644 --- a/lib/ssh/doc/src/ssh_app.xml +++ b/lib/ssh/doc/src/ssh_app.xml @@ -4,7 +4,7 @@ <appref> <header> <copyright> - <year>2012</year><year>2017</year> + <year>2012</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/ssh/doc/src/ssh_client_channel.xml b/lib/ssh/doc/src/ssh_client_channel.xml index eed49beffa..9be4007c68 100644 --- a/lib/ssh/doc/src/ssh_client_channel.xml +++ b/lib/ssh/doc/src/ssh_client_channel.xml @@ -5,7 +5,7 @@ <header> <copyright> <year>2009</year> - <year>2016</year> + <year>2018</year> <holder>Ericsson AB, All Rights Reserved</holder> </copyright> <legalnotice> diff --git a/lib/ssh/doc/src/ssh_client_key_api.xml b/lib/ssh/doc/src/ssh_client_key_api.xml index 9fc54341ed..bc77756147 100644 --- a/lib/ssh/doc/src/ssh_client_key_api.xml +++ b/lib/ssh/doc/src/ssh_client_key_api.xml @@ -5,7 +5,7 @@ <header> <copyright> <year>2012</year> - <year>2016</year> + <year>2018</year> <holder>Ericsson AB, All Rights Reserved</holder> </copyright> <legalnotice> diff --git a/lib/ssh/doc/src/ssh_connection.xml b/lib/ssh/doc/src/ssh_connection.xml index 821dfef93d..8e1cf156a8 100644 --- a/lib/ssh/doc/src/ssh_connection.xml +++ b/lib/ssh/doc/src/ssh_connection.xml @@ -5,7 +5,7 @@ <header> <copyright> <year>2008</year> - <year>2015</year> + <year>2018</year> <holder>Ericsson AB, All Rights Reserved</holder> </copyright> <legalnotice> diff --git a/lib/ssh/doc/src/ssh_protocol.xml b/lib/ssh/doc/src/ssh_protocol.xml index 53f0524b97..0d99a96997 100644 --- a/lib/ssh/doc/src/ssh_protocol.xml +++ b/lib/ssh/doc/src/ssh_protocol.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2013</year><year>2016</year> + <year>2013</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/ssh/doc/src/ssh_server_channel.xml b/lib/ssh/doc/src/ssh_server_channel.xml index af51ec470b..31ba9a3231 100644 --- a/lib/ssh/doc/src/ssh_server_channel.xml +++ b/lib/ssh/doc/src/ssh_server_channel.xml @@ -5,7 +5,7 @@ <header> <copyright> <year>2009</year> - <year>2016</year> + <year>2018</year> <holder>Ericsson AB, All Rights Reserved</holder> </copyright> <legalnotice> diff --git a/lib/ssh/doc/src/ssh_server_key_api.xml b/lib/ssh/doc/src/ssh_server_key_api.xml index cf3b1d319f..e2a31bd5f5 100644 --- a/lib/ssh/doc/src/ssh_server_key_api.xml +++ b/lib/ssh/doc/src/ssh_server_key_api.xml @@ -5,7 +5,7 @@ <header> <copyright> <year>2012</year> - <year>2015</year> + <year>2018</year> <holder>Ericsson AB, All Rights Reserved</holder> </copyright> <legalnotice> diff --git a/lib/ssh/doc/src/ssh_sftp.xml b/lib/ssh/doc/src/ssh_sftp.xml index 60f643d052..ea55126cb3 100644 --- a/lib/ssh/doc/src/ssh_sftp.xml +++ b/lib/ssh/doc/src/ssh_sftp.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>2005</year><year>2017</year> + <year>2005</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -46,9 +46,9 @@ <taglist> <tag><c>reason()</c></tag> <item> - <p>= <c>atom()</c> A description of the reason why an operation failed.</p> + <p>= <c>atom() | string() | tuple() </c>A description of the reason why an operation failed.</p> <p> - The value is formed from the sftp error codes in the protocol-level responses as defined in + The <c>atom()</c> value is formed from the sftp error codes in the protocol-level responses as defined in <url href="https://tools.ietf.org/id/draft-ietf-secsh-filexfer-13.txt">draft-ietf-secsh-filexfer-13.txt</url> section 9.1. </p> @@ -57,6 +57,10 @@ E.g. the error code <c>SSH_FX_NO_SUCH_FILE</c> will cause the <c>reason()</c> to be <c>no_such_file</c>. </p> + <p>The <c>string()</c> reason is the error information from the server in case of an exit-signal. If that information is empty, the reason is the exit signal name. + </p> + <p>The <c>tuple()</c> reason are other errors like the <c>{exit_status,integer()}</c> if the exit status is not 0. + </p> </item> <tag><c>connection_ref() =</c></tag> diff --git a/lib/ssh/doc/src/ssh_sftpd.xml b/lib/ssh/doc/src/ssh_sftpd.xml index a25ce123b3..3b34150e98 100644 --- a/lib/ssh/doc/src/ssh_sftpd.xml +++ b/lib/ssh/doc/src/ssh_sftpd.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>2005</year><year>2016</year> + <year>2005</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/ssh/doc/src/usersguide.xml b/lib/ssh/doc/src/usersguide.xml index d902df6848..38ffa48cde 100644 --- a/lib/ssh/doc/src/usersguide.xml +++ b/lib/ssh/doc/src/usersguide.xml @@ -4,7 +4,7 @@ <part xmlns:xi="http://www.w3.org/2001/XInclude"> <header> <copyright> - <year>2012</year><year>2016</year> + <year>2012</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/ssh/doc/src/using_ssh.xml b/lib/ssh/doc/src/using_ssh.xml index efd2a997f5..80662e9a70 100644 --- a/lib/ssh/doc/src/using_ssh.xml +++ b/lib/ssh/doc/src/using_ssh.xml @@ -5,7 +5,7 @@ <header> <copyright> <year>2012</year> - <year>2017</year> + <year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/ssh/src/Makefile b/lib/ssh/src/Makefile index 5e4efb6b99..6d64a45112 100644 --- a/lib/ssh/src/Makefile +++ b/lib/ssh/src/Makefile @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 2004-2017. All Rights Reserved. +# Copyright Ericsson AB 2004-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. diff --git a/lib/ssh/src/ssh.erl b/lib/ssh/src/ssh.erl index 7ddb1ca5be..086fa6e5f8 100644 --- a/lib/ssh/src/ssh.erl +++ b/lib/ssh/src/ssh.erl @@ -1,7 +1,7 @@ % %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2017. All Rights Reserved. +%% Copyright Ericsson AB 2004-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssh/src/ssh.hrl b/lib/ssh/src/ssh.hrl index 2efd239aae..01c44cb371 100644 --- a/lib/ssh/src/ssh.hrl +++ b/lib/ssh/src/ssh.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2017. All Rights Reserved. +%% Copyright Ericsson AB 2004-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -492,4 +492,29 @@ -define(wr_record(N), ?wr_record(N, [])). +%% Circular trace buffer macros + +-record(circ_buf_entry, + { + module, + line, + function, + pid = self(), + value + }). + +-define(CIRC_BUF_IN(VALUE), + ssh_dbg:cbuf_in( + #circ_buf_entry{module = ?MODULE, + line = ?LINE, + function = {?FUNCTION_NAME,?FUNCTION_ARITY}, + pid = self(), + value = (VALUE) + }) + ). + +-define(CIRC_BUF_IN_ONCE(VALUE), + ((fun(V) -> ?CIRC_BUF_IN(V), V end)(VALUE)) + ). + -endif. % SSH_HRL defined diff --git a/lib/ssh/src/ssh_acceptor.erl b/lib/ssh/src/ssh_acceptor.erl index 516a9febaa..11ce80354e 100644 --- a/lib/ssh/src/ssh_acceptor.erl +++ b/lib/ssh/src/ssh_acceptor.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2017. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssh/src/ssh_acceptor_sup.erl b/lib/ssh/src/ssh_acceptor_sup.erl index 10fd4452bf..15a2238dd3 100644 --- a/lib/ssh/src/ssh_acceptor_sup.erl +++ b/lib/ssh/src/ssh_acceptor_sup.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2017. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssh/src/ssh_auth.erl b/lib/ssh/src/ssh_auth.erl index bf3f5a68e4..4e4aa440de 100644 --- a/lib/ssh/src/ssh_auth.erl +++ b/lib/ssh/src/ssh_auth.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2017. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssh/src/ssh_channel.erl b/lib/ssh/src/ssh_channel.erl index 81c495a815..443bd05086 100644 --- a/lib/ssh/src/ssh_channel.erl +++ b/lib/ssh/src/ssh_channel.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2016. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssh/src/ssh_cli.erl b/lib/ssh/src/ssh_cli.erl index fcc1d3d59f..af51356355 100644 --- a/lib/ssh/src/ssh_cli.erl +++ b/lib/ssh/src/ssh_cli.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2017. All Rights Reserved. +%% Copyright Ericsson AB 2005-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssh/src/ssh_client_channel.erl b/lib/ssh/src/ssh_client_channel.erl index 8b5e196412..f985d8e273 100644 --- a/lib/ssh/src/ssh_client_channel.erl +++ b/lib/ssh/src/ssh_client_channel.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2016. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssh/src/ssh_client_key_api.erl b/lib/ssh/src/ssh_client_key_api.erl index d0d8ab25d6..384740b786 100644 --- a/lib/ssh/src/ssh_client_key_api.erl +++ b/lib/ssh/src/ssh_client_key_api.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2011-2016. All Rights Reserved. +%% Copyright Ericsson AB 2011-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssh/src/ssh_connect.hrl b/lib/ssh/src/ssh_connect.hrl index 3c61638285..9a060b8304 100644 --- a/lib/ssh/src/ssh_connect.hrl +++ b/lib/ssh/src/ssh_connect.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2017. All Rights Reserved. +%% Copyright Ericsson AB 2005-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssh/src/ssh_connection.erl b/lib/ssh/src/ssh_connection.erl index dad7636e3f..83f85b1d8e 100644 --- a/lib/ssh/src/ssh_connection.erl +++ b/lib/ssh/src/ssh_connection.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2017. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssh/src/ssh_connection_handler.erl b/lib/ssh/src/ssh_connection_handler.erl index f16aff947b..8e4831a601 100644 --- a/lib/ssh/src/ssh_connection_handler.erl +++ b/lib/ssh/src/ssh_connection_handler.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2017. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssh/src/ssh_connection_sup.erl b/lib/ssh/src/ssh_connection_sup.erl index 2e8450090a..79804b8630 100644 --- a/lib/ssh/src/ssh_connection_sup.erl +++ b/lib/ssh/src/ssh_connection_sup.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2017. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssh/src/ssh_daemon_channel.erl b/lib/ssh/src/ssh_daemon_channel.erl index fdb6c10971..30c4773a7a 100644 --- a/lib/ssh/src/ssh_daemon_channel.erl +++ b/lib/ssh/src/ssh_daemon_channel.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2013-2016. All Rights Reserved. +%% Copyright Ericsson AB 2013-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssh/src/ssh_dbg.erl b/lib/ssh/src/ssh_dbg.erl index 2ee4237e05..3681e03cc2 100644 --- a/lib/ssh/src/ssh_dbg.erl +++ b/lib/ssh/src/ssh_dbg.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2017. All Rights Reserved. +%% Copyright Ericsson AB 2004-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -54,7 +54,13 @@ start_tracer/0, start_tracer/1, on/1, on/0, off/1, off/0, - go_on/0 + go_on/0, + %% Circular buffer + cbuf_start/0, cbuf_start/1, + cbuf_stop_clear/0, + cbuf_in/1, + cbuf_list/0, + fmt_cbuf_items/0, fmt_cbuf_item/1 ]). -export([shrink_bin/1, @@ -331,3 +337,103 @@ ts({_,_,Usec}=Now) when is_integer(Usec) -> io_lib:format("~.2.0w:~.2.0w:~.2.0w.~.6.0w",[HH,MM,SS,Usec]); ts(_) -> "-". + +%%%================================================================ +-define(CIRC_BUF, circ_buf). + +cbuf_start() -> + cbuf_start(20). + +cbuf_start(CbufMaxLen) -> + put(?CIRC_BUF, {CbufMaxLen,queue:new()}), + ok. + + +cbuf_stop_clear() -> + case erase(?CIRC_BUF) of + undefined -> + []; + {_CbufMaxLen,Queue} -> + queue:to_list(Queue) + end. + + +cbuf_in(Value) -> + case get(?CIRC_BUF) of + undefined -> + disabled; + {CbufMaxLen,Queue} -> + UpdatedQueue = + try queue:head(Queue) of + {Value, TS0, Cnt0} -> + %% Same Value as last saved in the queue + queue:in_r({Value, TS0, Cnt0+1}, + queue:drop(Queue) + ); + _ -> + queue:in_r({Value, erlang:timestamp(), 1}, + truncate_cbuf(Queue, CbufMaxLen) + ) + catch + error:empty -> + queue:in_r({Value, erlang:timestamp(), 1}, Queue) + end, + put(?CIRC_BUF, {CbufMaxLen,UpdatedQueue}), + ok + end. + + +cbuf_list() -> + case get(?CIRC_BUF) of + undefined -> + []; + {_CbufMaxLen,Queue} -> + queue:to_list(Queue) + end. + + +truncate_cbuf(Q, CbufMaxLen) -> + case queue:len(Q) of + N when N>=CbufMaxLen -> + truncate_cbuf(element(2,queue:out_r(Q)), CbufMaxLen); + _ -> + Q + end. + +fmt_cbuf_items() -> + lists:flatten( + io_lib:format("Circular trace buffer. Latest item first.~n~s~n", + [case get(?CIRC_BUF) of + {Max,_} -> + L = cbuf_list(), + [io_lib:format("==== ~.*w: ~s~n",[num_digits(Max),N,fmt_cbuf_item(X)]) || + {N,X} <- lists:zip(lists:seq(1,length(L)), L) + ]; + _ -> + io_lib:format("Not started.~n",[]) + end])). + + +num_digits(0) -> 1; +num_digits(N) when N>0 -> 1+trunc(math:log10(N)). + + +fmt_cbuf_item({Value, TimeStamp, N}) -> + io_lib:format("~s~s~n~s~n", + [fmt_ts(TimeStamp), + [io_lib:format(" (Repeated ~p times)",[N]) || N>1], + fmt_value(Value)]). + + +fmt_ts(TS = {_,_,Us}) -> + {{YY,MM,DD},{H,M,S}} = calendar:now_to_universal_time(TS), + io_lib:format("~w-~.2.0w-~.2.0w ~.2.0w:~.2.0w:~.2.0w.~.6.0w UTC",[YY,MM,DD,H,M,S,Us]). + +fmt_value(#circ_buf_entry{module = M, + line = L, + function = {F,A}, + pid = Pid, + value = V}) -> + io_lib:format("~p:~p ~p/~p ~p~n~s",[M,L,F,A,Pid,fmt_value(V)]); +fmt_value(Value) -> + io_lib:format("~p",[Value]). diff --git a/lib/ssh/src/ssh_file.erl b/lib/ssh/src/ssh_file.erl index 9cab2fe0bd..832952ed52 100644 --- a/lib/ssh/src/ssh_file.erl +++ b/lib/ssh/src/ssh_file.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2017. All Rights Reserved. +%% Copyright Ericsson AB 2005-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssh/src/ssh_info.erl b/lib/ssh/src/ssh_info.erl index ee244f1432..79cd95e422 100644 --- a/lib/ssh/src/ssh_info.erl +++ b/lib/ssh/src/ssh_info.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2016. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssh/src/ssh_message.erl b/lib/ssh/src/ssh_message.erl index a2251eab97..55c0548c9b 100644 --- a/lib/ssh/src/ssh_message.erl +++ b/lib/ssh/src/ssh_message.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2013-2017. All Rights Reserved. +%% Copyright Ericsson AB 2013-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssh/src/ssh_no_io.erl b/lib/ssh/src/ssh_no_io.erl index 25be0023e9..31963118cb 100644 --- a/lib/ssh/src/ssh_no_io.erl +++ b/lib/ssh/src/ssh_no_io.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2016. All Rights Reserved. +%% Copyright Ericsson AB 2005-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssh/src/ssh_options.erl b/lib/ssh/src/ssh_options.erl index fe95d2ac54..bc9f2156bc 100644 --- a/lib/ssh/src/ssh_options.erl +++ b/lib/ssh/src/ssh_options.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2017. All Rights Reserved. +%% Copyright Ericsson AB 2004-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssh/src/ssh_server_channel.erl b/lib/ssh/src/ssh_server_channel.erl index f1c9a85639..555080e9ee 100644 --- a/lib/ssh/src/ssh_server_channel.erl +++ b/lib/ssh/src/ssh_server_channel.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2013-2016. All Rights Reserved. +%% Copyright Ericsson AB 2013-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssh/src/ssh_server_channel_sup.erl b/lib/ssh/src/ssh_server_channel_sup.erl index 70799db714..ff74061bb3 100644 --- a/lib/ssh/src/ssh_server_channel_sup.erl +++ b/lib/ssh/src/ssh_server_channel_sup.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2016. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssh/src/ssh_server_key_api.erl b/lib/ssh/src/ssh_server_key_api.erl index a285bf9475..ef1fe7d69c 100644 --- a/lib/ssh/src/ssh_server_key_api.erl +++ b/lib/ssh/src/ssh_server_key_api.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2011-2016. All Rights Reserved. +%% Copyright Ericsson AB 2011-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssh/src/ssh_sftp.erl b/lib/ssh/src/ssh_sftp.erl index 9c391abc43..1b2ba5a50b 100644 --- a/lib/ssh/src/ssh_sftp.erl +++ b/lib/ssh/src/ssh_sftp.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2017. All Rights Reserved. +%% Copyright Ericsson AB 2005-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -798,13 +798,22 @@ handle_ssh_msg({ssh_cm, _, {signal, _, _}}, State) -> %% Ignore signals according to RFC 4254 section 6.9. {ok, State}; -handle_ssh_msg({ssh_cm, _, {exit_signal, ChannelId, _, Error, _}}, +handle_ssh_msg({ssh_cm, _, {exit_signal, ChannelId, Signal, Error0, _}}, State0) -> + Error = + case Error0 of + "" -> Signal; + _ -> Error0 + end, State = reply_all(State0, {error, Error}), {stop, ChannelId, State}; handle_ssh_msg({ssh_cm, _, {exit_status, ChannelId, Status}}, State0) -> - State = reply_all(State0, {error, {exit_status, Status}}), + State = + case State0 of + 0 -> State0; + _ -> reply_all(State0, {error, {exit_status, Status}}) + end, {stop, ChannelId, State}. %%-------------------------------------------------------------------- diff --git a/lib/ssh/src/ssh_sftpd.erl b/lib/ssh/src/ssh_sftpd.erl index cb2eab1fec..278f6a9780 100644 --- a/lib/ssh/src/ssh_sftpd.erl +++ b/lib/ssh/src/ssh_sftpd.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2017. All Rights Reserved. +%% Copyright Ericsson AB 2005-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -125,9 +125,9 @@ handle_ssh_msg({ssh_cm, _, {signal, _, _}}, State) -> %% Ignore signals according to RFC 4254 section 6.9. {ok, State}; -handle_ssh_msg({ssh_cm, _, {exit_signal, ChannelId, _, Error, _}}, State) -> - Report = io_lib:format("Connection closed by peer ~n Error ~p~n", - [Error]), +handle_ssh_msg({ssh_cm, _, {exit_signal, ChannelId, Signal, Error, _}}, State) -> + Report = io_lib:format("Connection closed by peer signal ~p~n Error ~p~n", + [Signal,Error]), error_logger:error_report(Report), {stop, ChannelId, State}; diff --git a/lib/ssh/src/ssh_shell.erl b/lib/ssh/src/ssh_shell.erl index 084daa6821..cdc9a6df5b 100644 --- a/lib/ssh/src/ssh_shell.erl +++ b/lib/ssh/src/ssh_shell.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009-2016. All Rights Reserved. +%% Copyright Ericsson AB 2009-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssh/src/ssh_subsystem_sup.erl b/lib/ssh/src/ssh_subsystem_sup.erl index f80be7f20b..5fc8f7e764 100644 --- a/lib/ssh/src/ssh_subsystem_sup.erl +++ b/lib/ssh/src/ssh_subsystem_sup.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2017. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssh/src/ssh_sup.erl b/lib/ssh/src/ssh_sup.erl index 8183016ba5..61afbcd2ed 100644 --- a/lib/ssh/src/ssh_sup.erl +++ b/lib/ssh/src/ssh_sup.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2017. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssh/src/ssh_system_sup.erl b/lib/ssh/src/ssh_system_sup.erl index 469f9560e9..ed7c0c2bd5 100644 --- a/lib/ssh/src/ssh_system_sup.erl +++ b/lib/ssh/src/ssh_system_sup.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2017. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssh/src/ssh_transport.erl b/lib/ssh/src/ssh_transport.erl index 235f09a178..9ec16b420d 100644 --- a/lib/ssh/src/ssh_transport.erl +++ b/lib/ssh/src/ssh_transport.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2017. All Rights Reserved. +%% Copyright Ericsson AB 2004-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssh/src/ssh_transport.hrl b/lib/ssh/src/ssh_transport.hrl index 7d5a4c153e..f424a4ff63 100644 --- a/lib/ssh/src/ssh_transport.hrl +++ b/lib/ssh/src/ssh_transport.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2017. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssh/src/ssh_xfer.erl b/lib/ssh/src/ssh_xfer.erl index e1680c120e..7bb9c2d101 100644 --- a/lib/ssh/src/ssh_xfer.erl +++ b/lib/ssh/src/ssh_xfer.erl @@ -734,7 +734,7 @@ decode_ATTR(Vsn, <<?UINT32(Flags), Tail/binary>>) -> {Type,Tail2} = if Vsn =< 3 -> {?SSH_FILEXFER_TYPE_UNKNOWN, Tail}; - Vsn >= 5 -> + true -> <<?BYTE(T), TL/binary>> = Tail, {T, TL} end, diff --git a/lib/ssh/src/sshc_sup.erl b/lib/ssh/src/sshc_sup.erl index f4b39dbbdc..869de244ac 100644 --- a/lib/ssh/src/sshc_sup.erl +++ b/lib/ssh/src/sshc_sup.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2017. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssh/src/sshd_sup.erl b/lib/ssh/src/sshd_sup.erl index 779a861a54..b5361abba5 100644 --- a/lib/ssh/src/sshd_sup.erl +++ b/lib/ssh/src/sshd_sup.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2017. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssh/test/Makefile b/lib/ssh/test/Makefile index 64282cf3b9..e221e94075 100644 --- a/lib/ssh/test/Makefile +++ b/lib/ssh/test/Makefile @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 2004-2017. All Rights Reserved. +# Copyright Ericsson AB 2004-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. diff --git a/lib/ssh/test/property_test/ssh_eqc_client_info_timing.erl b/lib/ssh/test/property_test/ssh_eqc_client_info_timing.erl index 19e2754eba..6d0d8f5d99 100644 --- a/lib/ssh/test/property_test/ssh_eqc_client_info_timing.erl +++ b/lib/ssh/test/property_test/ssh_eqc_client_info_timing.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2016. All Rights Reserved. +%% Copyright Ericsson AB 2004-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssh/test/property_test/ssh_eqc_subsys.erl b/lib/ssh/test/property_test/ssh_eqc_subsys.erl index e7de3ea068..087b3ebfa7 100644 --- a/lib/ssh/test/property_test/ssh_eqc_subsys.erl +++ b/lib/ssh/test/property_test/ssh_eqc_subsys.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2016. All Rights Reserved. +%% Copyright Ericsson AB 2004-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssh/test/ssh_algorithms_SUITE.erl b/lib/ssh/test/ssh_algorithms_SUITE.erl index 3a5478f847..5e589e585f 100644 --- a/lib/ssh/test/ssh_algorithms_SUITE.erl +++ b/lib/ssh/test/ssh_algorithms_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2017. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssh/test/ssh_basic_SUITE.erl b/lib/ssh/test/ssh_basic_SUITE.erl index 807e23ff01..778ae1e7b6 100644 --- a/lib/ssh/test/ssh_basic_SUITE.erl +++ b/lib/ssh/test/ssh_basic_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2017. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssh/test/ssh_bench_dev_null.erl b/lib/ssh/test/ssh_bench_dev_null.erl index f9da80b6d7..868eea5643 100644 --- a/lib/ssh/test/ssh_bench_dev_null.erl +++ b/lib/ssh/test/ssh_bench_dev_null.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2017. All Rights Reserved. +%% Copyright Ericsson AB 2005-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssh/test/ssh_chan_behaviours_client.erl b/lib/ssh/test/ssh_chan_behaviours_client.erl index 07ac21ba97..15f17733d6 100644 --- a/lib/ssh/test/ssh_chan_behaviours_client.erl +++ b/lib/ssh/test/ssh_chan_behaviours_client.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2016. All Rights Reserved. +%% Copyright Ericsson AB 2005-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssh/test/ssh_chan_behaviours_server.erl b/lib/ssh/test/ssh_chan_behaviours_server.erl index a5ec19e0cf..1408675a6e 100644 --- a/lib/ssh/test/ssh_chan_behaviours_server.erl +++ b/lib/ssh/test/ssh_chan_behaviours_server.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2016. All Rights Reserved. +%% Copyright Ericsson AB 2005-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssh/test/ssh_compat_SUITE.erl b/lib/ssh/test/ssh_compat_SUITE.erl index 6c0e010bf5..1c607bebe8 100644 --- a/lib/ssh/test/ssh_compat_SUITE.erl +++ b/lib/ssh/test/ssh_compat_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2017. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssh/test/ssh_connection_SUITE.erl b/lib/ssh/test/ssh_connection_SUITE.erl index 257f2f70d7..778e4a5fc8 100644 --- a/lib/ssh/test/ssh_connection_SUITE.erl +++ b/lib/ssh/test/ssh_connection_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2017. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssh/test/ssh_dbg_SUITE.erl b/lib/ssh/test/ssh_dbg_SUITE.erl index 5439817d10..ab7918fa90 100644 --- a/lib/ssh/test/ssh_dbg_SUITE.erl +++ b/lib/ssh/test/ssh_dbg_SUITE.erl @@ -38,11 +38,20 @@ suite() -> {timetrap,{seconds,60}}]. all() -> - [basic, - dbg_alg_terminate, - dbg_ssh_messages, - dbg_connections, - dbg_channels + [{group, dbg}, + {group, circ_buf} + ]. + +groups() -> + [{dbg, [], [dbg_basic, + dbg_alg_terminate, + dbg_ssh_messages, + dbg_connections, + dbg_channels]}, + {circ_buf, [], [cb_basic, + cb_print, + cb_macros_print + ]} ]. %%-------------------------------------------------------------------- @@ -82,7 +91,7 @@ end_per_testcase(_TC, Config) -> %% Test Cases -------------------------------------------------------- %%-------------------------------------------------------------------- -basic(_Config) -> +dbg_basic(_Config) -> L0 = ssh_dbg:start(), true = is_pid(whereis(ssh_dbg)), true = is_list(L0), @@ -342,6 +351,53 @@ dbg_channels(Config) -> stop_and_fail_if_unhandled_dbg_msgs(Ref, [C,D], Pid). %%-------------------------------------------------------------------- +cb_basic(_Config) -> + %% Check that the circular buffer is disabled at start: + [] = ssh_dbg:cbuf_list(), + disabled = ssh_dbg:cbuf_in(anything), + [] = ssh_dbg:cbuf_list(), + %% Start it and enter three values, first is duplicated; + ok = ssh_dbg:cbuf_start(3), + ok = ssh_dbg:cbuf_in(v1), + ok = ssh_dbg:cbuf_in(v1), + ok = ssh_dbg:cbuf_in(v2), + ok = ssh_dbg:cbuf_in(v3), + [{v3,_,1}, {v2,_,1}, {v1,_,2}] = ssh_dbg:cbuf_list(), + %% Check that a fourth value erase the first entered: + ok = ssh_dbg:cbuf_in(v4), + [{v4,_,1}, {v3,_,1}, {v2,_,1}] = ssh_dbg:cbuf_list(), + %% Check that entering a value that is in the tail but not in the head is treated as a new value: + ok = ssh_dbg:cbuf_in(v2), + [{v2,_,1}, {v4,_,1}, {v3,_,1}] = ssh_dbg:cbuf_list(), + %% Stop and check that the buffer is returned: + [{v2,_,1}, {v4,_,1}, {v3,_,1}] = ssh_dbg:cbuf_stop_clear(), + %% Stopping a stopped buffer returns empty: + [] = ssh_dbg:cbuf_stop_clear(), + %% Check that a value can't be entered in a stopped buffer: + disabled = ssh_dbg:cbuf_in(v2). + +%%-------------------------------------------------------------------- +cb_print(_Config) -> + ssh_dbg:cbuf_start(), + [begin + ssh_dbg:cbuf_in(V), + ct:log("Enter ~p",[V]) + end || V <- lists:seq(1,10)], + ct:log("~s",[ssh_dbg:fmt_cbuf_items()]), + ssh_dbg:cbuf_stop_clear(). + +%%-------------------------------------------------------------------- +cb_macros_print(_Config) -> + ssh_dbg:cbuf_start(), + [begin + V = {test,V0}, + ?CIRC_BUF_IN(V), + ct:log("Enter ~p",[V]) + end || V0 <- lists:seq(1,5)], + ct:log("~s",[ssh_dbg:fmt_cbuf_items()]), + ssh_dbg:cbuf_stop_clear(). + +%%-------------------------------------------------------------------- %%-------------------------------------------------------------------- %%-------------------------------------------------------------------- diff --git a/lib/ssh/test/ssh_echo_server.erl b/lib/ssh/test/ssh_echo_server.erl index d03fe9543e..e039439f87 100644 --- a/lib/ssh/test/ssh_echo_server.erl +++ b/lib/ssh/test/ssh_echo_server.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2016. All Rights Reserved. +%% Copyright Ericsson AB 2005-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssh/test/ssh_engine_SUITE.erl b/lib/ssh/test/ssh_engine_SUITE.erl index c131a70973..c2e6ac1fee 100644 --- a/lib/ssh/test/ssh_engine_SUITE.erl +++ b/lib/ssh/test/ssh_engine_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2017. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssh/test/ssh_options_SUITE.erl b/lib/ssh/test/ssh_options_SUITE.erl index 86a8ac5aa8..daf62483cd 100644 --- a/lib/ssh/test/ssh_options_SUITE.erl +++ b/lib/ssh/test/ssh_options_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2017. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssh/test/ssh_peername_sockname_server.erl b/lib/ssh/test/ssh_peername_sockname_server.erl index 5e35fd6612..1cc53edf6d 100644 --- a/lib/ssh/test/ssh_peername_sockname_server.erl +++ b/lib/ssh/test/ssh_peername_sockname_server.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2016. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssh/test/ssh_sup_SUITE.erl b/lib/ssh/test/ssh_sup_SUITE.erl index b81f66948d..a0e3d809be 100644 --- a/lib/ssh/test/ssh_sup_SUITE.erl +++ b/lib/ssh/test/ssh_sup_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2015-2017. All Rights Reserved. +%% Copyright Ericsson AB 2015-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssh/test/ssh_test_lib.erl b/lib/ssh/test/ssh_test_lib.erl index 65970535f4..416cc301db 100644 --- a/lib/ssh/test/ssh_test_lib.erl +++ b/lib/ssh/test/ssh_test_lib.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2017. All Rights Reserved. +%% Copyright Ericsson AB 2004-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssh/test/ssh_to_openssh_SUITE.erl b/lib/ssh/test/ssh_to_openssh_SUITE.erl index 9df404d7ed..334281f53b 100644 --- a/lib/ssh/test/ssh_to_openssh_SUITE.erl +++ b/lib/ssh/test/ssh_to_openssh_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2017. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssh/vsn.mk b/lib/ssh/vsn.mk index f10e7aa96a..c3572b5b70 100644 --- a/lib/ssh/vsn.mk +++ b/lib/ssh/vsn.mk @@ -1,5 +1,4 @@ #-*-makefile-*- ; force emacs to enter makefile-mode -SSH_VSN = 4.6.9 - +SSH_VSN = 4.7 APP_VSN = "ssh-$(SSH_VSN)" diff --git a/lib/ssl/doc/src/Makefile b/lib/ssl/doc/src/Makefile index f9128e8e45..d459463322 100644 --- a/lib/ssl/doc/src/Makefile +++ b/lib/ssl/doc/src/Makefile @@ -102,6 +102,7 @@ html: gifs $(HTML_REF_MAN_FILE) clean clean_docs: rm -rf $(HTMLDIR)/* + rm -rf $(XMLDIR) rm -f $(MAN3DIR)/* rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) rm -f errs core *~ diff --git a/lib/ssl/doc/src/notes.xml b/lib/ssl/doc/src/notes.xml index 34fe352d08..87bf25452f 100644 --- a/lib/ssl/doc/src/notes.xml +++ b/lib/ssl/doc/src/notes.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>1999</year><year>2017</year> + <year>1999</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -27,6 +27,201 @@ </header> <p>This document describes the changes made to the SSL application.</p> +<section><title>SSL 9.0</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Correct handling of ECDH suites.</p> + <p> + Own Id: OTP-14974</p> + </item> + <item> + <p> + Proper handling of clients that choose to send an empty + answer to a certificate request</p> + <p> + Own Id: OTP-15050</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Distribution over SSL (inet_tls) has, to improve + performance, been rewritten to not use intermediate + processes and ports.</p> + <p> + Own Id: OTP-14465</p> + </item> + <item> + <p> + Add suport for ECDHE_PSK cipher suites</p> + <p> + Own Id: OTP-14547</p> + </item> + <item> + <p> + For security reasons no longer support 3-DES cipher + suites by default</p> + <p> + *** INCOMPATIBILITY with possibly ***</p> + <p> + Own Id: OTP-14768</p> + </item> + <item> + <p> + For security reasons RSA-key exchange cipher suites are + no longer supported by default</p> + <p> + *** INCOMPATIBILITY with possible ***</p> + <p> + Own Id: OTP-14769</p> + </item> + <item> + <p> + The interoperability option to fallback to insecure + renegotiation now has to be explicitly turned on.</p> + <p> + *** INCOMPATIBILITY with possibly ***</p> + <p> + Own Id: OTP-14789</p> + </item> + <item> + <p> + Drop support for SSLv2 enabled clients. SSLv2 has been + broken for decades and never supported by the Erlang + SSL/TLS implementation. This option was by default + disabled and enabling it has proved to sometimes break + connections not using SSLv2 enabled clients.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-14824</p> + </item> + <item> + <p> + Remove CHACHA20_POLY1305 ciphers form default for now. We + have discovered interoperability problems, ERL-538, that + we believe needs to be solved in crypto.</p> + <p> + *** INCOMPATIBILITY with possibly ***</p> + <p> + Own Id: OTP-14882</p> + </item> + <item> + <p> + Generalize DTLS packet multiplexing to make it easier to + add future DTLS features and uses.</p> + <p> + Own Id: OTP-14888</p> + </item> + <item> + <p> + Use uri_string module instead of http_uri.</p> + <p> + Own Id: OTP-14902</p> + </item> + <item> + <p> + The SSL distribution protocol <c>-proto inet_tls</c> has + stopped setting the SSL option + <c>server_name_indication</c>. New verify funs for client + and server in <c>inet_tls_dist</c> has been added, not + documented yet, that checks node name if present in peer + certificate. Usage is still also yet to be documented.</p> + <p> + Own Id: OTP-14969 Aux Id: OTP-14465, ERL-598 </p> + </item> + <item> + <p> + Deprecate ssl:ssl_accept/[1,2,3] in favour of + ssl:handshake/[1,2,3]</p> + <p> + Own Id: OTP-15056</p> + </item> + <item> + <p> + Customizes the hostname verification of the peer + certificate, as different protocols that use TLS such as + HTTP or LDAP may want to do it differently</p> + <p> + Own Id: OTP-15102 Aux Id: ERL-542, OTP-14962 </p> + </item> + <item> + <p> + Add utility function for converting erlang cipher suites + to a string represenation (ERL-600).</p> + <p> + Own Id: OTP-15106</p> + </item> + <item> + <p> + First version with support for DTLS</p> + <p> + Own Id: OTP-15142</p> + </item> + </list> + </section> + +</section> + +<section><title>SSL 8.2.6.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Correct handling of empty server SNI extension</p> + <p> + Own Id: OTP-15168</p> + </item> + <item> + <p> + Correct cipher suite handling for ECDHE_*, the incorrect + handling could cause an incorrrect suite to be selected + and most likly fail the handshake.</p> + <p> + Own Id: OTP-15203</p> + </item> + </list> + </section> + +</section> + +<section><title>SSL 8.2.6.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Improve cipher suite handling correcting ECC and TLS-1.2 + requierments. Backport of solution for ERL-641</p> + <p> + Own Id: OTP-15178</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Option keyfile defaults to certfile and should be trumped + with key. This failed for engine keys.</p> + <p> + Own Id: OTP-15193</p> + </item> + </list> + </section> + +</section> + <section><title>SSL 8.2.6</title> <section><title>Fixed Bugs and Malfunctions</title> @@ -196,7 +391,7 @@ <p> TLS sessions must be registered with SNI if provided, so that sessions where client hostname verification would - fail can not connect reusing a session created when the + fail cannot connect reusing a session created when the server name verification succeeded.</p> <p> Own Id: OTP-14632</p> @@ -374,7 +569,7 @@ public_key:pkix_verify_hostname/2 to verify the hostname of the connection with the server certificates specified hostname during certificate path validation. The user may - explicitly disables it. Also if the hostname can not be + explicitly disables it. Also if the hostname cannot be derived from the first argument to connect or is not supplied by the server name indication option, the check will not be performed.</p> diff --git a/lib/ssl/doc/src/ssl.xml b/lib/ssl/doc/src/ssl.xml index 825bf46459..6efa022a79 100644 --- a/lib/ssl/doc/src/ssl.xml +++ b/lib/ssl/doc/src/ssl.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1999</year><year>2017</year> + <year>1999</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -88,6 +88,7 @@ <p><c>| {client_preferred_next_protocols, {client | server, [binary()]} | {client | server, [binary()], binary()}}</c></p> <p><c>| {log_alert, boolean()}</c></p> + <p><c>| {log_level, atom()}</c></p> <p><c>| {server_name_indication, hostname() | disable}</c></p> <p><c>| {customize_hostname_check, list()}</c></p> <p><c>| {sni_hosts, [{hostname(), [ssl_option()]}]}</c></p> @@ -199,14 +200,14 @@ | sect163r1 | sect163r2 | secp160k1 | secp160r1 | secp160r2</c></p></item> <tag><c>hello_extensions() =</c></tag> - <item><p><c>#{renegotiation_info => + <item><p><c>#{renegotiation_info => binary() | undefined, signature_algs => [{hash(), ecsda| rsa| dsa}] | undefined alpn => binary() | undefined, - next_protocol_negotiation, + next_protocol_negotiation => binary() | undefined, srp => string() | undefined, - ec_point_formats , - elliptic_curves = [oid] | undefined - sni = string()} + ec_point_formats => list() | undefined, + elliptic_curves => [oid] | undefined, + sni => string() | undefined} }</c></p></item> @@ -409,7 +410,7 @@ marker="public_key:public_key#pkix_path_validation-3">public_key:pkix_path_valid <item>check is only performed on the peer certificate.</item> <tag><c>best_effort</c></tag> - <item>if certificate revocation status can not be determined + <item>if certificate revocation status cannot be determined it will be accepted as valid.</item> </taglist> @@ -796,7 +797,17 @@ fun(srp, Username :: string(), UserState :: term()) -> the client.</p></item> <tag><c>{log_alert, boolean()}</c></tag> - <item><p>If set to <c>false</c>, error reports are not displayed.</p></item> + <item><p>If set to <c>false</c>, error reports are not displayed.</p> + <p>Deprecated in OTP 22, use <seealso marker="#log_level">log_level</seealso> instead.</p> + </item> + + <tag><marker id="log_level"/><c>{log_level, atom()}</c></tag> + <item><p>Specifies the log level for TLS/DTLS. It can take the following + values (ordered by increasing verbosity level): <c>emergency, alert, critical, error, + warning, notice, info, debug.</c></p> + <p>At verbosity level <c>notice</c> and above error reports are + displayed in TLS. The level <c>debug</c> triggers verbose logging of TLS protocol + messages and logging of ignored alerts in DTLS.</p></item> <tag><c>{honor_cipher_order, boolean()}</c></tag> <item><p>If set to <c>true</c>, use the server preference for cipher @@ -896,16 +907,7 @@ fun(srp, Username :: string(), UserState :: term()) -> <v>Type = erlang | openssl | all</v> </type> <desc> - <p>Returns a list of supported cipher suites. - This function will become deprecated in OTP 21, and replaced - by <seealso marker="#cipher_suites-2">ssl:cipher-suites/2</seealso> - <c>cipher_suites()</c> is equivalent to <c>cipher_suites(erlang).</c> - Type <c>openssl</c> is provided for backwards compatibility with the - old SSL, which used OpenSSL. <c>cipher_suites(all)</c> returns - all available cipher suites. The cipher suites not present - in <c>cipher_suites(erlang)</c> but included in - <c>cipher_suites(all)</c> are not used unless explicitly configured - by the user.</p> + <p>Deprecated in OTP 21, use <seealso marker="#cipher_suites-2">cipher_suites/2</seealso> instead.</p> </desc> </func> @@ -948,7 +950,7 @@ fun(srp, Username :: string(), UserState :: term()) -> <func> <name>connect(Socket, SslOptions) -> </name> - <name>connect(Socket, SslOptions, Timeout) -> {ok, TLSSocket} | {ok, TLSSocket, Ext} + <name>connect(Socket, SslOptions, Timeout) -> {ok, SslSocket} | {ok, SslSocket, Ext} | {error, Reason}</name> <fsummary>Upgrades a <c>gen_tcp</c>, or equivalent, connected socket to an TLS socket.</fsummary> @@ -956,7 +958,7 @@ fun(srp, Username :: string(), UserState :: term()) -> <v>Socket = socket()</v> <v>SslOptions = [{handshake, hello| full} | ssl_option()]</v> <v>Timeout = integer() | infinity</v> - <v>TLSSocket = sslsocket()</v> + <v>SslSocket = sslsocket()</v> <v>Ext = hello_extensions()</v> <v>Reason = term()</v> </type> @@ -973,8 +975,8 @@ fun(srp, Username :: string(), UserState :: term()) -> <p> If the option <c>{handshake, hello}</c> is used the handshake is paused after receiving the server hello message - and the success response is <c>{ok, TLSSocket, Ext}</c> - instead of <c>{ok, TLSSocket}</c>. Thereafter the handshake is continued or + and the success response is <c>{ok, SslSocket, Ext}</c> + instead of <c>{ok, SslSocket}</c>. Thereafter the handshake is continued or canceled by calling <seealso marker="#handshake_continue-3"> <c>handshake_continue/3</c></seealso> or <seealso marker="#handshake_cancel-1"><c>handshake_cancel/1</c></seealso>. @@ -986,7 +988,7 @@ fun(srp, Username :: string(), UserState :: term()) -> <func> <name>connect(Host, Port, Options) -></name> <name>connect(Host, Port, Options, Timeout) -> - {ok, SslSocket}| {ok, TLSSocket, Ext} | {error, Reason}</name> + {ok, SslSocket}| {ok, SslSocket, Ext} | {error, Reason}</name> <fsummary>Opens an TLS/DTLS connection to <c>Host</c>, <c>Port</c>.</fsummary> <type> <v>Host = host()</v> @@ -1017,8 +1019,8 @@ fun(srp, Username :: string(), UserState :: term()) -> <p> If the option <c>{handshake, hello}</c> is used the handshake is paused after receiving the server hello message - and the success response is <c>{ok, TLSSocket, Ext}</c> - instead of <c>{ok, TLSSocket}</c>. Thereafter the handshake is continued or + and the success response is <c>{ok, SslSocket, Ext}</c> + instead of <c>{ok, SslSocket}</c>. Thereafter the handshake is continued or canceled by calling <seealso marker="#handshake_continue-3"> <c>handshake_continue/3</c></seealso> or <seealso marker="#handshake_cancel-1"><c>handshake_cancel/1</c></seealso>. @@ -1074,7 +1076,8 @@ fun(srp, Username :: string(), UserState :: term()) -> <fsummary>Returns all the connection information. </fsummary> <type> - <v>Item = protocol | cipher_suite | sni_hostname | ecc | session_id | atom()</v> + <v>SslSocket = sslsocket()</v> + <v>Item = protocol | selected_cipher_suite | sni_hostname | ecc | session_id | atom()</v> <d>Meaningful atoms, not specified above, are the ssl option names.</d> <v>Result = [{Item::atom(), Value::term()}]</v> <v>Reason = term()</v> @@ -1082,6 +1085,9 @@ fun(srp, Username :: string(), UserState :: term()) -> <desc><p>Returns the most relevant information about the connection, ssl options that are undefined will be filtered out. Note that values that affect the security of the connection will only be returned if explicitly requested by connection_information/2.</p> + <note><p>The legacy <c>Item = cipher_suite</c> is still supported + and returns the cipher suite on its (undocumented) legacy format. + It should be replaced by <c>selected_cipher_suite</c>.</p></note> </desc> </func> @@ -1091,6 +1097,7 @@ fun(srp, Username :: string(), UserState :: term()) -> <fsummary>Returns the requested connection information. </fsummary> <type> + <v>SslSocket = sslsocket()</v> <v>Items = [Item]</v> <v>Item = protocol | cipher_suite | sni_hostname | ecc | session_id | client_random | server_random | master_secret | atom()</v> @@ -1133,7 +1140,7 @@ fun(srp, Username :: string(), UserState :: term()) -> </func> <func> - <name>getopts(Socket, OptionNames) -> + <name>getopts(SslSocket, OptionNames) -> {ok, [socketoption()]} | {error, Reason}</name> <fsummary>Gets the values of the specified options.</fsummary> <type> @@ -1147,13 +1154,13 @@ fun(srp, Username :: string(), UserState :: term()) -> </func> <func> - <name>getstat(Socket) -> + <name>getstat(SslSocket) -> {ok, OptionValues} | {error, inet:posix()}</name> - <name>getstat(Socket, OptionNames) -> + <name>getstat(SslSocket, OptionNames) -> {ok, OptionValues} | {error, inet:posix()}</name> <fsummary>Get one or more statistic options for a socket</fsummary> <type> - <v>Socket = sslsocket()</v> + <v>SslSocket = sslsocket()</v> <v>OptionNames = [atom()]</v> <v>OptionValues = [{inet:stat_option(), integer()}]</v> </type> @@ -1164,28 +1171,27 @@ fun(srp, Username :: string(), UserState :: term()) -> </func> <func> - <name>handshake(Socket) -> </name> - <name>handshake(Socket, Timeout) -> {ok, Socket} | {error, Reason}</name> + <name>handshake(HsSocket) -> </name> + <name>handshake(HsSocket, Timeout) -> {ok, SslSocket} | {error, Reason}</name> <fsummary>Performs server-side SSL/TLS handshake.</fsummary> <type> - <v>Socket = sslsocket()</v> + <v>HsSocket = SslSocket = sslsocket()</v> <v>Timeout = integer()</v> <v>Reason = term()</v> </type> <desc> <p>Performs the SSL/TLS/DTLS server-side handshake.</p> - <p><c>Socket</c> is a socket as returned by - <seealso marker="#transport_accept-2">ssl:transport_accept/[1,2]</seealso>. - </p> + <p>Returns a new TLS/DTLS socket if the handshake is successful.</p> </desc> </func> <func> <name>handshake(Socket, SslOptions) -> </name> - <name>handshake(Socket, SslOptions, Timeout) -> {ok, Socket} | {ok, Socket, Ext} | {error, Reason}</name> + <name>handshake(Socket, SslOptions, Timeout) -> {ok, SslSocket} | {ok, SslSocket, Ext} | {error, Reason}</name> <fsummary>Performs server-side SSL/TLS/DTLS handshake.</fsummary> <type> <v>Socket = socket() | sslsocket() </v> + <v>SslSocket = sslsocket() </v> <v>Ext = hello_extensions()</v> <v>SslOptions = [{handshake, hello| full} | ssl_option()]</v> <v>Timeout = integer()</v> @@ -1194,22 +1200,23 @@ fun(srp, Username :: string(), UserState :: term()) -> <desc> <p>If <c>Socket</c> is a ordinary <c>socket()</c>: upgrades a <c>gen_tcp</c>, or equivalent, socket to an SSL socket, that is, performs - the SSL/TLS server-side handshake and returns the SSL socket.</p> + the SSL/TLS server-side handshake and returns a TLS socket.</p> - <warning><p>The Socket shall be in passive mode ({active, - false}) before calling this function or the handshake can fail - due to a race condition.</p></warning> + <warning><p>The <c>Socket</c> shall be in passive mode ({active, + false}) before calling this function or else the behavior of this function + is undefined. + </p></warning> <p>If <c>Socket</c> is an <c>sslsocket()</c>: provides extra SSL/TLS/DTLS options to those specified in - <seealso marker="#listen-2">ssl:listen/2 </seealso> and then performs - the SSL/TLS/DTLS handshake.</p> - + <seealso marker="#listen-2">listen/2 </seealso> and then performs + the SSL/TLS/DTLS handshake. Returns a new TLS/DTLS socket if the handshake is successful.</p> + <p> If option <c>{handshake, hello}</c> is specified the handshake is paused after receiving the client hello message and the - sucess response is <c>{ok, TLSSocket, Ext}</c> instead of <c>{ok, - TLSSocket}</c>. Thereafter the handshake is continued or + success response is <c>{ok, SslSocket, Ext}</c> instead of <c>{ok, + SslSocket}</c>. Thereafter the handshake is continued or canceled by calling <seealso marker="#handshake_continue-3"> <c>handshake_continue/3</c></seealso> or <seealso marker="#handshake_cancel-1"><c>handshake_cancel/1</c></seealso>. @@ -1218,10 +1225,10 @@ fun(srp, Username :: string(), UserState :: term()) -> </func> <func> - <name>handshake_cancel(Socket) -> ok </name> + <name>handshake_cancel(SslSocket) -> ok </name> <fsummary>Cancel handshake with a fatal alert</fsummary> <type> - <v>Socket = sslsocket()</v> + <v>SslSocket = sslsocket()</v> </type> <desc> <p>Cancel the handshake with a fatal <c>USER_CANCELED</c> alert.</p> @@ -1229,10 +1236,11 @@ fun(srp, Username :: string(), UserState :: term()) -> </func> <func> - <name>handshake_continue(Socket, SSLOptions, Timeout) -> {ok, Socket} | {error, Reason}</name> + <name>handshake_continue(HsSocket, SSLOptions) -> {ok, SslSocket} | {error, Reason}</name> + <name>handshake_continue(HsSocket, SSLOptions, Timeout) -> {ok, SslSocket} | {error, Reason}</name> <fsummary>Continue the SSL/TLS handshake.</fsummary> <type> - <v>Socket = sslsocket()</v> + <v>HsSocket = SslSocket = sslsocket()</v> <v>SslOptions = [ssl_option()]</v> <v>Timeout = integer()</v> <v>Reason = term()</v> @@ -1257,10 +1265,10 @@ fun(srp, Username :: string(), UserState :: term()) -> </func> <func> - <name>negotiated_protocol(Socket) -> {ok, Protocol} | {error, protocol_not_negotiated}</name> + <name>negotiated_protocol(SslSocket) -> {ok, Protocol} | {error, protocol_not_negotiated}</name> <fsummary>Returns the protocol negotiated through ALPN or NPN extensions.</fsummary> <type> - <v>Socket = sslsocket()</v> + <v>SslSocket = sslsocket()</v> <v>Protocol = binary()</v> </type> <desc> @@ -1271,25 +1279,26 @@ fun(srp, Username :: string(), UserState :: term()) -> </func> <func> - <name>peercert(Socket) -> {ok, Cert} | {error, Reason}</name> + <name>peercert(SslSocket) -> {ok, Cert} | {error, Reason}</name> <fsummary>Returns the peer certificate.</fsummary> <type> - <v>Socket = sslsocket()</v> + <v>SslSocket = sslsocket()</v> <v>Cert = binary()</v> </type> <desc> <p>The peer certificate is returned as a DER-encoded binary. The certificate can be decoded with - <c>public_key:pkix_decode_cert/2</c>.</p> + <seealso marker="public_key:public_key#pkix_decode_cert-2">public_key:pkix_decode_cert/2</seealso> + </p> </desc> </func> <func> - <name>peername(Socket) -> {ok, {Address, Port}} | + <name>peername(SslSocket) -> {ok, {Address, Port}} | {error, Reason}</name> <fsummary>Returns the peer address and port.</fsummary> <type> - <v>Socket = sslsocket()</v> + <v>SslSocket = sslsocket()</v> <v>Address = ipaddress()</v> <v>Port = integer()</v> </type> @@ -1335,12 +1344,12 @@ fun(srp, Username :: string(), UserState :: term()) -> </func> <func> - <name>recv(Socket, Length) -> </name> - <name>recv(Socket, Length, Timeout) -> {ok, Data} | {error, + <name>recv(SslSocket, Length) -> </name> + <name>recv(SslSocket, Length, Timeout) -> {ok, Data} | {error, Reason}</name> <fsummary>Receives data on a socket.</fsummary> <type> - <v>Socket = sslsocket()</v> + <v>SslSocket = sslsocket()</v> <v>Length = integer()</v> <v>Timeout = integer()</v> <v>Data = [char()] | binary()</v> @@ -1362,10 +1371,10 @@ fun(srp, Username :: string(), UserState :: term()) -> </func> <func> - <name>renegotiate(Socket) -> ok | {error, Reason}</name> + <name>renegotiate(SslSocket) -> ok | {error, Reason}</name> <fsummary>Initiates a new handshake.</fsummary> <type> - <v>Socket = sslsocket()</v> + <v>SslSocket = sslsocket()</v> </type> <desc><p>Initiates a new handshake. A notable return value is <c>{error, renegotiation_rejected}</c> indicating that the peer @@ -1375,10 +1384,10 @@ fun(srp, Username :: string(), UserState :: term()) -> </func> <func> - <name>send(Socket, Data) -> ok | {error, Reason}</name> + <name>send(SslSocket, Data) -> ok | {error, Reason}</name> <fsummary>Writes data to a socket.</fsummary> <type> - <v>Socket = sslsocket()</v> + <v>SslSocket = sslsocket()</v> <v>Data = iodata()</v> </type> <desc> @@ -1389,10 +1398,10 @@ fun(srp, Username :: string(), UserState :: term()) -> </func> <func> - <name>setopts(Socket, Options) -> ok | {error, Reason}</name> + <name>setopts(SslSocket, Options) -> ok | {error, Reason}</name> <fsummary>Sets socket options.</fsummary> <type> - <v>Socket = sslsocket()</v> + <v>SslSocket = sslsocket()</v> <v>Options = [socketoption]()</v> </type> <desc> @@ -1402,10 +1411,21 @@ fun(srp, Username :: string(), UserState :: term()) -> </func> <func> - <name>shutdown(Socket, How) -> ok | {error, Reason}</name> + <name>set_log_level(Level) -> ok | {error, Reason}</name> + <fsummary>Sets log level for the SSL application.</fsummary> + <type> + <v>Level = atom()</v> + </type> + <desc> + <p>Sets log level for the SSL application.</p> + </desc> + </func> + + <func> + <name>shutdown(SslSocket, How) -> ok | {error, Reason}</name> <fsummary>Immediately closes a socket.</fsummary> <type> - <v>Socket = sslsocket()</v> + <v>SslSocket = sslsocket()</v> <v>How = read | write | read_write</v> <v>Reason = reason()</v> </type> @@ -1420,19 +1440,17 @@ fun(srp, Username :: string(), UserState :: term()) -> </func> <func> - <name>ssl_accept(Socket) -> </name> - <name>ssl_accept(Socket, Timeout) -> ok | {error, Reason}</name> + <name>ssl_accept(SslSocket) -> </name> + <name>ssl_accept(SslSocket, Timeout) -> ok | {error, Reason}</name> <fsummary>Performs server-side SSL/TLS handshake.</fsummary> <type> - <v>Socket = sslsocket()</v> + <v>SslSocket = sslsocket()</v> <v>Timeout = integer()</v> <v>Reason = term()</v> </type> <desc> - <p>Performs the SSL/TLS/DTLS server-side handshake.</p> - <p><c>Socket</c> is a socket as returned by - <seealso marker="#transport_accept-2">ssl:transport_accept/[1,2]</seealso> - </p> + <p>Deprecated in OTP 21, use <seealso marker="#handshake-1">handshake/[1,2]</seealso> instead.</p> + <note><p>handshake/[1,2] always returns a new socket.</p></note> </desc> </func> @@ -1447,29 +1465,17 @@ fun(srp, Username :: string(), UserState :: term()) -> <v>Reason = term()</v> </type> <desc> - <p>If <c>Socket</c> is a <c>socket()</c>: upgrades a <c>gen_tcp</c>, - or equivalent, socket to an SSL socket, that is, performs - the SSL/TLS server-side handshake and returns the SSL socket.</p> - - <warning><p>The listen socket is to be in mode <c>{active, false}</c> - before telling the client that the server is ready to upgrade - by calling this function, else the upgrade succeeds or does not - succeed depending on timing.</p></warning> - - <p>If <c>Socket</c> is an <c>sslsocket()</c>: provides extra SSL/TLS/DTLS - options to those specified in - <seealso marker="#listen-2">ssl:listen/2 </seealso> and then performs - the SSL/TLS/DTLS handshake. - </p> + <p>Deprecated in OTP 21, use <seealso marker="#handshake-3">handshake/[2,3]</seealso> instead.</p> + <note><p>handshake/[2,3] always returns a new socket.</p></note> </desc> </func> <func> - <name>sockname(Socket) -> {ok, {Address, Port}} | + <name>sockname(SslSocket) -> {ok, {Address, Port}} | {error, Reason}</name> <fsummary>Returns the local address and port.</fsummary> <type> - <v>Socket = sslsocket()</v> + <v>SslSocket = sslsocket()</v> <v>Address = ipaddress()</v> <v>Port = integer()</v> </type> @@ -1501,32 +1507,44 @@ fun(srp, Username :: string(), UserState :: term()) -> </func> <func> + <name>suite_to_str(CipherSuite) -> String</name> + <fsummary>Returns the string representation of a cipher suite.</fsummary> + <type> + <v>CipherSuite = erl_cipher_suite()</v> + <v>String = string()</v> + </type> + <desc> + <p>Returns the string representation of a cipher suite.</p> + </desc> + </func> + + <func> <name>transport_accept(ListenSocket) -></name> <name>transport_accept(ListenSocket, Timeout) -> - {ok, NewSocket} | {error, Reason}</name> + {ok, SslSocket} | {error, Reason}</name> <fsummary>Accepts an incoming connection and prepares for <c>ssl_accept</c>.</fsummary> <type> - <v>ListenSocket = NewSocket = sslsocket()</v> + <v>ListenSocket = SslSocket = sslsocket()</v> <v>Timeout = integer()</v> <v>Reason = reason()</v> </type> <desc> <p>Accepts an incoming connection request on a listen socket. <c>ListenSocket</c> must be a socket returned from - <seealso marker="#listen-2"> ssl:listen/2</seealso>. + <seealso marker="#listen-2"> listen/2</seealso>. The socket returned is to be passed to - <seealso marker="#ssl_accept-2"> ssl:ssl_accept[2,3]</seealso> + <seealso marker="#handshake-2"> handshake/[2,3]</seealso> to complete handshaking, that is, establishing the SSL/TLS/DTLS connection.</p> <warning> - <p>The socket returned can only be used with - <seealso marker="#ssl_accept-2"> ssl:ssl_accept[2,3]</seealso>. - No traffic can be sent or received before that call.</p> + <p>Most API functions require that the TLS/DTLS + connection is established to work as expected. + </p> </warning> <p>The accepted socket inherits the options set for <c>ListenSocket</c> in - <seealso marker="#listen-2"> ssl:listen/2</seealso>.</p> + <seealso marker="#listen-2"> listen/2</seealso>.</p> <p>The default value for <c>Timeout</c> is <c>infinity</c>. If <c>Timeout</c> is specified and no connection is accepted @@ -1557,7 +1575,7 @@ fun(srp, Username :: string(), UserState :: term()) -> <seealso marker="#listen-2"> listen/2</seealso>, and <seealso marker="#ssl_accept-2">ssl_accept/[1,2,3]</seealso>. For the negotiated SSL/TLS version, see <seealso - marker="#connection_information-1">ssl:connection_information/1 + marker="#connection_information-1">connection_information/1 </seealso>.</item> <tag><c>supported_dtls</c></tag> @@ -1567,7 +1585,7 @@ fun(srp, Username :: string(), UserState :: term()) -> <seealso marker="#listen-2"> listen/2</seealso>, and <seealso marker="#ssl_accept-2">ssl_accept/[1,2,3]</seealso>. For the negotiated DTLS version, see <seealso - marker="#connection_information-1">ssl:connection_information/1 + marker="#connection_information-1">connection_information/1 </seealso>.</item> <tag><c>available</c></tag> diff --git a/lib/ssl/doc/src/ssl_app.xml b/lib/ssl/doc/src/ssl_app.xml index e22d43db0e..f6d9021d4a 100644 --- a/lib/ssl/doc/src/ssl_app.xml +++ b/lib/ssl/doc/src/ssl_app.xml @@ -4,7 +4,7 @@ <appref> <header> <copyright> - <year>1999</year><year>2016</year> + <year>1999</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/ssl/doc/src/ssl_crl_cache.xml b/lib/ssl/doc/src/ssl_crl_cache.xml index 738487759a..71c6d5e49e 100644 --- a/lib/ssl/doc/src/ssl_crl_cache.xml +++ b/lib/ssl/doc/src/ssl_crl_cache.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>2015</year><year>2015</year> + <year>2015</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/ssl/doc/src/ssl_distribution.xml b/lib/ssl/doc/src/ssl_distribution.xml index e14f3f90dc..1774bd8f77 100644 --- a/lib/ssl/doc/src/ssl_distribution.xml +++ b/lib/ssl/doc/src/ssl_distribution.xml @@ -191,7 +191,7 @@ Eshell V5.0 (abort with ^G) Any available SSL/TLS option can be specified in an options file, but note that options that take a <c>fun()</c> has to use the syntax <c>fun Mod:Func/Arity</c> since a function - body can not be compiled when consulting a file. + body cannot be compiled when consulting a file. </p> <p> Do not tamper with the socket options diff --git a/lib/ssl/doc/src/ssl_introduction.xml b/lib/ssl/doc/src/ssl_introduction.xml index a416924eb1..adcfb091b7 100644 --- a/lib/ssl/doc/src/ssl_introduction.xml +++ b/lib/ssl/doc/src/ssl_introduction.xml @@ -5,7 +5,7 @@ <header> <copyright> <year>2015</year> - <year>2017</year> + <year>2018</year> <holder>Ericsson AB, All Rights Reserved</holder> </copyright> <legalnotice> diff --git a/lib/ssl/doc/src/ssl_protocol.xml b/lib/ssl/doc/src/ssl_protocol.xml index 0b12dc7dc5..3ab836443f 100644 --- a/lib/ssl/doc/src/ssl_protocol.xml +++ b/lib/ssl/doc/src/ssl_protocol.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2003</year><year>2015</year> + <year>2003</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/ssl/doc/src/using_ssl.xml b/lib/ssl/doc/src/using_ssl.xml index 3ef33df719..f2f9b66a31 100644 --- a/lib/ssl/doc/src/using_ssl.xml +++ b/lib/ssl/doc/src/using_ssl.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2003</year><year>2016</year> + <year>2003</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -66,7 +66,7 @@ ssl:listen(9999, [{certfile, "cert.pem"}, {keyfile, "key.pem"},{reuseaddr, true} {ok,{sslsocket, [...]}}</code> <p><em>Step 3:</em> Do a transport accept on the TLS listen socket:</p> - <code type="erl">3 server> {ok, Socket} = ssl:transport_accept(ListenSocket). + <code type="erl">3 server> {ok, TLSTransportSocket} = ssl:transport_accept(ListenSocket). {ok,{sslsocket, [...]}}</code> <p><em>Step 4:</em> Start the client side: </p> @@ -77,7 +77,7 @@ ok</code> {ok,{sslsocket, [...]}}</code> <p><em>Step 5:</em> Do the TLS handshake:</p> - <code type="erl">4 server> ok = ssl:ssl_accept(Socket). + <code type="erl">4 server> {ok, Socket} = ssl:handshake(TLSTransportSocket). ok</code> <p><em>Step 6:</em> Send a message over TLS:</p> @@ -126,13 +126,13 @@ ok</code> ok</code> <p><em>Step 6:</em> Do the TLS handshake:</p> - <code type="erl">5 server> {ok, TLSSocket} = ssl:ssl_accept(Socket, [{cacertfile, "cacerts.pem"}, + <code type="erl">5 server> {ok, TLSSocket} = ssl:handshake(Socket, [{cacertfile, "cacerts.pem"}, {certfile, "cert.pem"}, {keyfile, "key.pem"}]). {ok,{sslsocket,[...]}}</code> <p><em>Step 7:</em> Upgrade to an TLS connection. The client and server must agree upon the upgrade. The server must call - <c>ssl:accept/2</c> before the client calls <c>ssl:connect/3.</c></p> + <c>ssl:handshake/2</c> before the client calls <c>ssl:connect/3.</c></p> <code type="erl">3 client>{ok, TLSSocket} = ssl:connect(Socket, [{cacertfile, "cacerts.pem"}, {certfile, "cert.pem"}, {keyfile, "key.pem"}], infinity). {ok,{sslsocket,[...]}}</code> diff --git a/lib/ssl/src/Makefile b/lib/ssl/src/Makefile index c389aa8cfe..1db18d4e5a 100644 --- a/lib/ssl/src/Makefile +++ b/lib/ssl/src/Makefile @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 1999-2017. All Rights Reserved. +# Copyright Ericsson AB 1999-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -86,7 +86,8 @@ MODULES= \ ssl_record \ ssl_v3 \ tls_v1 \ - dtls_v1 + dtls_v1 \ + ssl_logger INTERNAL_HRL_FILES = \ ssl_alert.hrl ssl_cipher.hrl \ @@ -118,7 +119,7 @@ EXTRA_ERLC_FLAGS = +warn_unused_vars ERL_COMPILE_FLAGS += -I$(ERL_TOP)/lib/kernel/src \ -pz $(EBIN) \ -pz $(ERL_TOP)/lib/public_key/ebin \ - $(EXTRA_ERLC_FLAGS) -DVSN=\"$(VSN)\" + $(EXTRA_ERLC_FLAGS) # ---------------------------------------------------- diff --git a/lib/ssl/src/dtls_connection.erl b/lib/ssl/src/dtls_connection.erl index 4e3f65d9c6..c0e81d6a28 100644 --- a/lib/ssl/src/dtls_connection.erl +++ b/lib/ssl/src/dtls_connection.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2013-2017. All Rights Reserved. +%% Copyright Ericsson AB 2013-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -32,6 +32,7 @@ -include("ssl_internal.hrl"). -include("ssl_srp.hrl"). -include_lib("public_key/include/public_key.hrl"). +-include_lib("kernel/include/logger.hrl"). %% Internal application API @@ -91,13 +92,14 @@ start_link(Role, Host, Port, Socket, Options, User, CbInfo) -> init([Role, Host, Port, Socket, Options, User, CbInfo]) -> process_flag(trap_exit, true), - State0 = initial_state(Role, Host, Port, Socket, Options, User, CbInfo), + State0 = #state{protocol_specific = Map} = initial_state(Role, Host, Port, Socket, Options, User, CbInfo), try State = ssl_connection:ssl_config(State0#state.ssl_options, Role, State0), gen_statem:enter_loop(?MODULE, [], init, State) catch throw:Error -> - gen_statem:enter_loop(?MODULE, [], error, {Error,State0}) + EState = State0#state{protocol_specific = Map#{error => Error}}, + gen_statem:enter_loop(?MODULE, [], error, EState) end. %%==================================================================== %% State transition handling @@ -470,7 +472,8 @@ init(Type, Event, State) -> %%-------------------------------------------------------------------- error(enter, _, State) -> {keep_state, State}; -error({call, From}, {start, _Timeout}, {Error, State}) -> +error({call, From}, {start, _Timeout}, + #state{protocol_specific = #{error := Error}} = State) -> ssl_connection:stop_and_reply( normal, {reply, From, {error, Error}}, State); error({call, _} = Call, Msg, State) -> @@ -926,7 +929,7 @@ handle_own_alert(Alert, Version, StateName, #state{data_tag = udp, ssl_options = Options} = State0) -> case ignore_alert(Alert, State0) of {true, State} -> - log_ignore_alert(Options#ssl_options.log_alert, StateName, Alert, Role), + log_ignore_alert(Options#ssl_options.log_level, StateName, Alert, Role), {next_state, StateName, State}; {false, State} -> ssl_connection:handle_own_alert(Alert, Version, StateName, State) @@ -1123,9 +1126,9 @@ is_ignore_alert(#alert{description = ?ILLEGAL_PARAMETER}) -> is_ignore_alert(_) -> false. -log_ignore_alert(true, StateName, Alert, Role) -> +log_ignore_alert(debug, StateName, Alert, Role) -> Txt = ssl_alert:alert_txt(Alert), - error_logger:format("DTLS over UDP ~p: In state ~p ignored to send ALERT ~s as DoS-attack mitigation \n", - [Role, StateName, Txt]); -log_ignore_alert(false, _, _,_) -> + ?LOG_ERROR("DTLS over UDP ~p: In state ~p ignored to send ALERT ~s as DoS-attack mitigation \n", + [Role, StateName, Txt]); +log_ignore_alert(_, _, _, _) -> ok. diff --git a/lib/ssl/src/dtls_handshake.erl b/lib/ssl/src/dtls_handshake.erl index 1a415a5f76..35c213a182 100644 --- a/lib/ssl/src/dtls_handshake.erl +++ b/lib/ssl/src/dtls_handshake.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2013-2017. All Rights Reserved. +%% Copyright Ericsson AB 2013-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssl/src/dtls_listener_sup.erl b/lib/ssl/src/dtls_listener_sup.erl index 6939f1ef3b..dc30696a2c 100644 --- a/lib/ssl/src/dtls_listener_sup.erl +++ b/lib/ssl/src/dtls_listener_sup.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2016-2016. All Rights Reserved. +%% Copyright Ericsson AB 2016-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssl/src/dtls_packet_demux.erl b/lib/ssl/src/dtls_packet_demux.erl index 1672626165..e03a4e9cb9 100644 --- a/lib/ssl/src/dtls_packet_demux.erl +++ b/lib/ssl/src/dtls_packet_demux.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2016-2017. All Rights Reserved. +%% Copyright Ericsson AB 2016-2018. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -24,6 +24,7 @@ -behaviour(gen_server). -include("ssl_internal.hrl"). +-include_lib("kernel/include/logger.hrl"). %% API -export([start_link/5, active_once/3, accept/2, sockname/1, close/1, @@ -146,11 +147,11 @@ handle_info({Transport, Socket, IP, InPortNo, _} = Msg, #state{listener = Socket %% appears to make things work as expected! handle_info({Error, Socket, econnreset = Error}, #state{listener = Socket, transport = {_,_,_, udp_error}} = State) -> Report = io_lib:format("Ignore SSL UDP Listener: Socket error: ~p ~n", [Error]), - error_logger:info_report(Report), + ?LOG_NOTICE(Report), {noreply, State}; handle_info({Error, Socket, Error}, #state{listener = Socket, transport = {_,_,_, Error}} = State) -> Report = io_lib:format("SSL Packet muliplxer shutdown: Socket error: ~p ~n", [Error]), - error_logger:info_report(Report), + ?LOG_NOTICE(Report), {noreply, State#state{close=true}}; handle_info({'DOWN', _, process, Pid, _}, #state{clients = Clients, diff --git a/lib/ssl/src/dtls_record.erl b/lib/ssl/src/dtls_record.erl index 2938fd460f..9eb0d8e2d7 100644 --- a/lib/ssl/src/dtls_record.erl +++ b/lib/ssl/src/dtls_record.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2013-2017. All Rights Reserved. +%% Copyright Ericsson AB 2013-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssl/src/dtls_socket.erl b/lib/ssl/src/dtls_socket.erl index 8dd62bc352..b26d3ae41a 100644 --- a/lib/ssl/src/dtls_socket.erl +++ b/lib/ssl/src/dtls_socket.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2016-2017. All Rights Reserved. +%% Copyright Ericsson AB 2016-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssl/src/dtls_v1.erl b/lib/ssl/src/dtls_v1.erl index 0f6344b6f7..df687f579b 100644 --- a/lib/ssl/src/dtls_v1.erl +++ b/lib/ssl/src/dtls_v1.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2013-2017. All Rights Reserved. +%% Copyright Ericsson AB 2013-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssl/src/inet_tls_dist.erl b/lib/ssl/src/inet_tls_dist.erl index aa3d7e3f72..1194f4fc72 100644 --- a/lib/ssl/src/inet_tls_dist.erl +++ b/lib/ssl/src/inet_tls_dist.erl @@ -41,6 +41,7 @@ -include_lib("public_key/include/public_key.hrl"). -include("ssl_api.hrl"). +-include_lib("kernel/include/logger.hrl"). %% ------------------------------------------------------------------------- @@ -226,7 +227,7 @@ accept_loop(Driver, Listen, Kernel) -> true -> accept_loop(Driver, Listen, Kernel, Socket); {false,IP} -> - error_logger:error_msg( + ?LOG_ERROR( "** Connection attempt from " "disallowed IP ~w ** ~n", [IP]), ?shutdown2(no_node, trace({disallowed, IP})) @@ -261,7 +262,7 @@ accept_loop(Driver, Listen, Kernel, Socket) -> {error, {options, _}} = Error -> %% Bad options: that's probably our fault. %% Let's log that. - error_logger:error_msg( + ?LOG_ERROR( "Cannot accept TLS distribution connection: ~s~n", [ssl:format_error(Error)]), gen_tcp:close(Socket), @@ -437,7 +438,7 @@ allowed_nodes(SslSocket, Allowed) -> PeerCert, allowed_hosts(Allowed), PeerIP) of [] -> - error_logger:error_msg( + ?LOG_ERROR( "** Connection attempt from " "disallowed node(s) ~p ** ~n", [PeerIP]), ?shutdown2( @@ -691,12 +692,12 @@ split_node(Driver, Node, LongOrShortNames) -> {node, Name, Host} -> check_node(Driver, Node, Name, Host, LongOrShortNames); {host, _} -> - error_logger:error_msg( + ?LOG_ERROR( "** Nodename ~p illegal, no '@' character **~n", [Node]), ?shutdown2(Node, trace({illegal_node_n@me, Node})); _ -> - error_logger:error_msg( + ?LOG_ERROR( "** Nodename ~p illegal **~n", [Node]), ?shutdown2(Node, trace({illegal_node_name, Node})) end. @@ -708,7 +709,7 @@ check_node(Driver, Node, Name, Host, LongOrShortNames) -> {ok, _} -> {Name, Host}; _ -> - error_logger:error_msg( + ?LOG_ERROR( "** System running to use " "fully qualified hostnames **~n" "** Hostname ~s is illegal **~n", @@ -716,7 +717,7 @@ check_node(Driver, Node, Name, Host, LongOrShortNames) -> ?shutdown2(Node, trace({not_longnames, Host})) end; [_,_|_] when LongOrShortNames =:= shortnames -> - error_logger:error_msg( + ?LOG_ERROR( "** System NOT running to use " "fully qualified hostnames **~n" "** Hostname ~s is illegal **~n", @@ -846,13 +847,13 @@ monitor_pid(Pid) -> %% MRef = erlang:monitor(process, Pid), %% receive %% {'DOWN', MRef, _, _, normal} -> - %% error_logger:error_report( - %% [dist_proc_died, + %% ?LOG_ERROR( + %% [{slogan, dist_proc_died}, %% {reason, normal}, %% {pid, Pid}]); %% {'DOWN', MRef, _, _, Reason} -> - %% error_logger:info_report( - %% [dist_proc_died, + %% ?LOG_NOTICE( + %% [{slogan, dist_proc_died}, %% {reason, Reason}, %% {pid, Pid}]) %% end diff --git a/lib/ssl/src/ssl.app.src b/lib/ssl/src/ssl.app.src index da281829cb..9679ea4687 100644 --- a/lib/ssl/src/ssl.app.src +++ b/lib/ssl/src/ssl.app.src @@ -51,6 +51,8 @@ ssl_crl_cache, ssl_crl_cache_api, ssl_crl_hash_dir, + %% Logging + ssl_logger, %% App structure ssl_app, ssl_sup, diff --git a/lib/ssl/src/ssl.erl b/lib/ssl/src/ssl.erl index 565cc9e1bc..09953908ce 100644 --- a/lib/ssl/src/ssl.erl +++ b/lib/ssl/src/ssl.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2017. All Rights Reserved. +%% Copyright Ericsson AB 1999-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -40,7 +40,7 @@ %% Socket handling -export([connect/3, connect/2, connect/4, listen/2, transport_accept/1, transport_accept/2, - handshake/1, handshake/2, handshake/3, + handshake/1, handshake/2, handshake/3, handshake_continue/2, handshake_continue/3, handshake_cancel/1, ssl_accept/1, ssl_accept/2, ssl_accept/3, controlling_process/2, peername/1, peercert/1, sockname/1, @@ -55,7 +55,8 @@ format_error/1, renegotiate/1, prf/5, negotiated_protocol/1, connection_information/1, connection_information/2]). %% Misc --export([handle_options/2, tls_version/1, new_ssl_options/3]). +-export([handle_options/2, tls_version/1, new_ssl_options/3, suite_to_str/1, + set_log_level/1]). -deprecated({ssl_accept, 1, eventually}). -deprecated({ssl_accept, 2, eventually}). @@ -87,6 +88,7 @@ stop() -> application:stop(ssl). %%-------------------------------------------------------------------- + -spec connect(host() | port(), [connect_option()]) -> {ok, #sslsocket{}} | {error, reason()}. -spec connect(host() | port(), [connect_option()] | inet:port_number(), @@ -209,6 +211,8 @@ ssl_accept(Socket, SslOptions, Timeout) -> %% Description: Performs accept on an ssl listen socket. e.i. performs %% ssl handshake. %%-------------------------------------------------------------------- + +%% Performs the SSL/TLS/DTLS server-side handshake. handshake(ListenSocket) -> handshake(ListenSocket, infinity). @@ -216,6 +220,12 @@ handshake(#sslsocket{} = Socket, Timeout) when (is_integer(Timeout) andalso Tim (Timeout == infinity) -> ssl_connection:handshake(Socket, Timeout); +%% If Socket is a ordinary socket(): upgrades a gen_tcp, or equivalent, socket to +%% an SSL socket, that is, performs the SSL/TLS server-side handshake and returns +%% the SSL socket. +%% +%% If Socket is an sslsocket(): provides extra SSL/TLS/DTLS options to those +%% specified in ssl:listen/2 and then performs the SSL/TLS/DTLS handshake. handshake(ListenSocket, SslOptions) when is_port(ListenSocket) -> handshake(ListenSocket, SslOptions, infinity). @@ -259,6 +269,16 @@ handshake(Socket, SslOptions, Timeout) when is_port(Socket), Error = {error, _Reason} -> Error end. + +%%-------------------------------------------------------------------- +-spec handshake_continue(#sslsocket{}, [ssl_option()]) -> + {ok, #sslsocket{}} | {error, reason()}. +%% +%% +%% Description: Continues the handshke possible with newly supplied options. +%%-------------------------------------------------------------------- +handshake_continue(Socket, SSLOptions) -> + handshake_continue(Socket, SSLOptions, infinity). %%-------------------------------------------------------------------- -spec handshake_continue(#sslsocket{}, [ssl_option()], timeout()) -> {ok, #sslsocket{}} | {error, reason()}. @@ -772,6 +792,42 @@ tls_version({3, _} = Version) -> tls_version({254, _} = Version) -> dtls_v1:corresponding_tls_version(Version). + +%%-------------------------------------------------------------------- +-spec suite_to_str(ssl_cipher:erl_cipher_suite()) -> string(). +%% +%% Description: Return the string representation of a cipher suite. +%%-------------------------------------------------------------------- +suite_to_str(Cipher) -> + ssl_cipher:suite_to_str(Cipher). + + +%%-------------------------------------------------------------------- +-spec set_log_level(atom()) -> ok | {error, term()}. +%% +%% Description: Set log level for the SSL application +%%-------------------------------------------------------------------- +set_log_level(Level) -> + case application:get_all_key(ssl) of + {ok, PropList} -> + Modules = proplists:get_value(modules, PropList), + set_module_level(Modules, Level); + undefined -> + {error, ssl_not_started} + end. + +set_module_level(Modules, Level) -> + Fun = fun (Module) -> + ok = logger:set_module_level(Module, Level) + end, + try lists:map(Fun, Modules) of + _ -> + ok + catch + error:{badmatch, Error} -> + Error + end. + %%%-------------------------------------------------------------- %%% Internal functions %%%-------------------------------------------------------------------- @@ -831,9 +887,10 @@ handle_options(Opts0, #ssl_options{protocol = Protocol, cacerts = CaCerts0, [] -> new_ssl_options(SslOpts1, NewVerifyOpts, RecordCB); Value -> - Versions = [RecordCB:protocol_version(Vsn) || Vsn <- Value], + Versions0 = [RecordCB:protocol_version(Vsn) || Vsn <- Value], + Versions1 = lists:sort(fun RecordCB:is_higher/2, Versions0), new_ssl_options(proplists:delete(versions, SslOpts1), - NewVerifyOpts#ssl_options{versions = Versions}, record_cb(Protocol)) + NewVerifyOpts#ssl_options{versions = Versions1}, record_cb(Protocol)) end; %% Handle all options in listen and connect @@ -856,7 +913,8 @@ handle_options(Opts0, Role, Host) -> [] -> RecordCb:supported_protocol_versions(); Vsns -> - [RecordCb:protocol_version(Vsn) || Vsn <- Vsns] + Versions0 = [RecordCb:protocol_version(Vsn) || Vsn <- Vsns], + lists:sort(fun RecordCb:is_higher/2, Versions0) end, Protocol = handle_option(protocol, Opts, tls), @@ -868,7 +926,7 @@ handle_options(Opts0, Role, Host) -> ok end, - SSLOptions = #ssl_options{ + SSLOptions0 = #ssl_options{ versions = Versions, verify = validate_option(verify, Verify), verify_fun = VerifyFun, @@ -915,7 +973,6 @@ handle_options(Opts0, Role, Host) -> next_protocol_selector = make_next_protocol_selector( handle_option(client_preferred_next_protocols, Opts, undefined)), - log_alert = handle_option(log_alert, Opts, true), server_name_indication = handle_option(server_name_indication, Opts, default_option_role(client, server_name_indication_default(Host), Role)), @@ -941,6 +998,10 @@ handle_options(Opts0, Role, Host) -> handshake = handle_option(handshake, Opts, full), customize_hostname_check = handle_option(customize_hostname_check, Opts, []) }, + LogLevel = handle_option(log_alert, Opts, true), + SSLOptions = SSLOptions0#ssl_options{ + log_level = handle_option(log_level, Opts, LogLevel) + }, CbInfo = proplists:get_value(cb_info, Opts, default_cb_info(Protocol)), SslOptions = [protocol, versions, verify, verify_fun, partial_chain, @@ -952,7 +1013,7 @@ handle_options(Opts0, Role, Host) -> cb_info, renegotiate_at, secure_renegotiate, hibernate_after, erl_dist, alpn_advertised_protocols, sni_hosts, sni_fun, alpn_preferred_protocols, next_protocols_advertised, - client_preferred_next_protocols, log_alert, + client_preferred_next_protocols, log_alert, log_level, server_name_indication, honor_cipher_order, padding_check, crl_check, crl_cache, fallback, signature_algs, eccs, honor_ecc_order, beast_mitigation, max_handshake_size, handshake, customize_hostname_check], @@ -1131,7 +1192,20 @@ validate_option(client_preferred_next_protocols, {Precedence, PreferredProtocols Value; validate_option(client_preferred_next_protocols, undefined) -> undefined; -validate_option(log_alert, Value) when is_boolean(Value) -> +validate_option(log_alert, true) -> + notice; +validate_option(log_alert, false) -> + warning; +validate_option(log_level, Value) when + is_atom(Value) andalso + (Value =:= emergency orelse + Value =:= alert orelse + Value =:= critical orelse + Value =:= error orelse + Value =:= warning orelse + Value =:= notice orelse + Value =:= info orelse + Value =:= debug) -> Value; validate_option(next_protocols_advertised, Value) when is_list(Value) -> validate_binary_list(next_protocols_advertised, Value), @@ -1239,7 +1313,8 @@ validate_binary_list(Opt, List) -> end, List). validate_versions([], Versions) -> Versions; -validate_versions([Version | Rest], Versions) when Version == 'tlsv1.2'; +validate_versions([Version | Rest], Versions) when Version == 'tlsv1.3'; + Version == 'tlsv1.2'; Version == 'tlsv1.1'; Version == tlsv1; Version == sslv3 -> @@ -1252,10 +1327,11 @@ validate_versions([Ver| _], Versions) -> tls_validate_versions([], Versions) -> Versions; -tls_validate_versions([Version | Rest], Versions) when Version == 'tlsv1.2'; - Version == 'tlsv1.1'; - Version == tlsv1; - Version == sslv3 -> +tls_validate_versions([Version | Rest], Versions) when Version == 'tlsv1.3'; + Version == 'tlsv1.2'; + Version == 'tlsv1.1'; + Version == tlsv1; + Version == sslv3 -> tls_validate_versions(Rest, Versions); tls_validate_versions([Ver| _], Versions) -> throw({error, {options, {Ver, {versions, Versions}}}}). @@ -1506,8 +1582,10 @@ new_ssl_options([{next_protocols_advertised, Value} | Rest], #ssl_options{} = Op new_ssl_options([{client_preferred_next_protocols, Value} | Rest], #ssl_options{} = Opts, RecordCB) -> new_ssl_options(Rest, Opts#ssl_options{next_protocol_selector = make_next_protocol_selector(validate_option(client_preferred_next_protocols, Value))}, RecordCB); -new_ssl_options([{log_alert, Value} | Rest], #ssl_options{} = Opts, RecordCB) -> - new_ssl_options(Rest, Opts#ssl_options{log_alert = validate_option(log_alert, Value)}, RecordCB); +new_ssl_options([{log_alert, Value} | Rest], #ssl_options{} = Opts, RecordCB) -> + new_ssl_options(Rest, Opts#ssl_options{log_level = validate_option(log_alert, Value)}, RecordCB); +new_ssl_options([{log_level, Value} | Rest], #ssl_options{} = Opts, RecordCB) -> + new_ssl_options(Rest, Opts#ssl_options{log_level = validate_option(log_level, Value)}, RecordCB); new_ssl_options([{server_name_indication, Value} | Rest], #ssl_options{} = Opts, RecordCB) -> new_ssl_options(Rest, Opts#ssl_options{server_name_indication = validate_option(server_name_indication, Value)}, RecordCB); new_ssl_options([{honor_cipher_order, Value} | Rest], #ssl_options{} = Opts, RecordCB) -> diff --git a/lib/ssl/src/ssl_alert.erl b/lib/ssl/src/ssl_alert.erl index fc7b1e6d1c..34e9797f1f 100644 --- a/lib/ssl/src/ssl_alert.erl +++ b/lib/ssl/src/ssl_alert.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2016. All Rights Reserved. +%% Copyright Ericsson AB 2007-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssl/src/ssl_alert.hrl b/lib/ssl/src/ssl_alert.hrl index 35670edea5..b23123905e 100644 --- a/lib/ssl/src/ssl_alert.hrl +++ b/lib/ssl/src/ssl_alert.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2016. All Rights Reserved. +%% Copyright Ericsson AB 2007-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssl/src/ssl_app.erl b/lib/ssl/src/ssl_app.erl index 62e8765d4a..2a5047c75c 100644 --- a/lib/ssl/src/ssl_app.erl +++ b/lib/ssl/src/ssl_app.erl @@ -29,9 +29,26 @@ -export([start/2, stop/1]). start(_Type, _StartArgs) -> + start_logger(), ssl_sup:start_link(). stop(_State) -> + stop_logger(), ok. +%% +%% Description: Start SSL logger +start_logger() -> + Config = #{level => debug, + filter_default => stop, + formatter => {ssl_logger, #{}}}, + Filter = {fun logger_filters:domain/2,{log,sub,[otp,ssl]}}, + logger:add_handler(ssl_handler, logger_std_h, Config), + logger:add_handler_filter(ssl_handler, filter_non_ssl, Filter). + +%% +%% Description: Stop SSL logger +stop_logger() -> + logger:remove_handler(ssl_handler). + diff --git a/lib/ssl/src/ssl_certificate.erl b/lib/ssl/src/ssl_certificate.erl index dbd2ebf539..017d18ee2c 100644 --- a/lib/ssl/src/ssl_certificate.erl +++ b/lib/ssl/src/ssl_certificate.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2017 All Rights Reserved. +%% Copyright Ericsson AB 2007-2018 All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -69,7 +69,7 @@ trusted_cert_and_path(CertChain, CertDbHandle, CertDbRef, PartialChainHandler) - case SignedAndIssuerID of {error, issuer_not_found} -> - %% The root CA was not sent and can not be found. + %% The root CA was not sent and cannot be found. handle_incomplete_chain(Path, PartialChainHandler); {self, _} when length(Path) == 1 -> {selfsigned_peer, Path}; diff --git a/lib/ssl/src/ssl_cipher.erl b/lib/ssl/src/ssl_cipher.erl index 3f8b9a8a9b..07ec823829 100644 --- a/lib/ssl/src/ssl_cipher.erl +++ b/lib/ssl/src/ssl_cipher.erl @@ -1,7 +1,7 @@ % %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2017. All Rights Reserved. +%% Copyright Ericsson AB 2007-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -43,7 +43,7 @@ filter/3, filter_suites/1, filter_suites/2, hash_algorithm/1, sign_algorithm/1, is_acceptable_hash/2, is_fallback/1, random_bytes/1, calc_mac_hash/4, - is_stream_ciphersuite/1]). + is_stream_ciphersuite/1, suite_to_str/1]). -export_type([cipher_suite/0, erl_cipher_suite/0, old_erl_cipher_suite/0, openssl_cipher_suite/0, @@ -187,7 +187,7 @@ block_cipher(Fun, BlockSz, #cipher_state{key=Key, iv=IV} = CS0, block_cipher(Fun, BlockSz, #cipher_state{key=Key, iv=IV} = CS0, Mac, Fragment, {3, N}) - when N == 2; N == 3 -> + when N == 2; N == 3; N == 4 -> NextIV = random_iv(IV), L0 = build_cipher_block(BlockSz, Mac, Fragment), L = [NextIV|L0], @@ -320,6 +320,8 @@ suites({3, Minor}) -> suites({_, Minor}) -> dtls_v1:suites(Minor). +all_suites({3, 4}) -> + all_suites({3, 3}); all_suites({3, _} = Version) -> suites(Version) ++ chacha_suites(Version) @@ -478,11 +480,12 @@ rc4_suites({3, Minor}) -> rc4_suites(0) -> [?TLS_RSA_WITH_RC4_128_SHA, ?TLS_RSA_WITH_RC4_128_MD5]; -rc4_suites(N) when N =< 3 -> +rc4_suites(N) when N =< 4 -> [?TLS_ECDHE_ECDSA_WITH_RC4_128_SHA, ?TLS_ECDHE_RSA_WITH_RC4_128_SHA, ?TLS_ECDH_ECDSA_WITH_RC4_128_SHA, ?TLS_ECDH_RSA_WITH_RC4_128_SHA]. + %%-------------------------------------------------------------------- -spec des_suites(Version::ssl_record:ssl_version()) -> [cipher_suite()]. %% @@ -517,13 +520,14 @@ rsa_suites(0) -> ?TLS_RSA_WITH_AES_128_CBC_SHA, ?TLS_RSA_WITH_3DES_EDE_CBC_SHA ]; -rsa_suites(N) when N =< 3 -> +rsa_suites(N) when N =< 4 -> [ ?TLS_RSA_WITH_AES_256_GCM_SHA384, ?TLS_RSA_WITH_AES_256_CBC_SHA256, ?TLS_RSA_WITH_AES_128_GCM_SHA256, ?TLS_RSA_WITH_AES_128_CBC_SHA256 ]. + %%-------------------------------------------------------------------- -spec suite_definition(cipher_suite()) -> erl_cipher_suite(). %% @@ -1877,6 +1881,32 @@ suite(#{key_exchange := dhe_rsa, prf := sha256}) -> ?TLS_DHE_RSA_WITH_CHACHA20_POLY1305_SHA256. + +%%-------------------------------------------------------------------- +-spec suite_to_str(erl_cipher_suite()) -> string(). +%% +%% Description: Return the string representation of a cipher suite. +%%-------------------------------------------------------------------- +suite_to_str(#{key_exchange := null, + cipher := null, + mac := null, + prf := null}) -> + "TLS_EMPTY_RENEGOTIATION_INFO_SCSV"; +suite_to_str(#{key_exchange := Kex, + cipher := Cipher, + mac := aead, + prf := PRF}) -> + "TLS_" ++ string:to_upper(atom_to_list(Kex)) ++ + "_WITH_" ++ string:to_upper(atom_to_list(Cipher)) ++ + "_" ++ string:to_upper(atom_to_list(PRF)); +suite_to_str(#{key_exchange := Kex, + cipher := Cipher, + mac := Mac}) -> + "TLS_" ++ string:to_upper(atom_to_list(Kex)) ++ + "_WITH_" ++ string:to_upper(atom_to_list(Cipher)) ++ + "_" ++ string:to_upper(atom_to_list(Mac)). + + %%-------------------------------------------------------------------- -spec openssl_suite(openssl_cipher_suite()) -> cipher_suite(). %% @@ -2404,7 +2434,7 @@ mac_hash({_,_}, ?NULL, _MacSecret, _SeqNo, _Type, mac_hash({3, 0}, MacAlg, MacSecret, SeqNo, Type, Length, Fragment) -> ssl_v3:mac_hash(MacAlg, MacSecret, SeqNo, Type, Length, Fragment); mac_hash({3, N} = Version, MacAlg, MacSecret, SeqNo, Type, Length, Fragment) - when N =:= 1; N =:= 2; N =:= 3 -> + when N =:= 1; N =:= 2; N =:= 3; N =:= 4 -> tls_v1:mac_hash(MacAlg, MacSecret, SeqNo, Type, Version, Length, Fragment). @@ -2609,7 +2639,7 @@ generic_block_cipher_from_bin({3, N}, T, IV, HashSize) next_iv = IV}; generic_block_cipher_from_bin({3, N}, T, IV, HashSize) - when N == 2; N == 3 -> + when N == 2; N == 3; N == 4 -> Sz1 = byte_size(T) - 1, <<_:Sz1/binary, ?BYTE(PadLength)>> = T, IVLength = byte_size(IV), @@ -2683,6 +2713,8 @@ filter_suites_pubkey(ec, Ciphers, _, OtpCert) -> ec_ecdhe_suites(Ciphers)), filter_keyuse_suites(keyAgreement, Uses, CiphersSuites, ec_ecdh_suites(Ciphers)). +filter_suites_signature(rsa, Ciphers, {3, N}) when N >= 3 -> + Ciphers; filter_suites_signature(rsa, Ciphers, Version) -> (Ciphers -- ecdsa_signed_suites(Ciphers, Version)) -- dsa_signed_suites(Ciphers, Version); filter_suites_signature(dsa, Ciphers, Version) -> @@ -2749,6 +2781,8 @@ ecdsa_signed_suites(Ciphers, Version) -> rsa_keyed(dhe_rsa) -> true; +rsa_keyed(ecdhe_rsa) -> + true; rsa_keyed(rsa) -> true; rsa_keyed(rsa_psk) -> @@ -2812,6 +2846,8 @@ ec_keyed(ecdh_ecdsa) -> true; ec_keyed(ecdh_rsa) -> true; +ec_keyed(ecdhe_ecdsa) -> + true; ec_keyed(_) -> false. diff --git a/lib/ssl/src/ssl_cipher.hrl b/lib/ssl/src/ssl_cipher.hrl index e5462d8402..ba6a98b92a 100644 --- a/lib/ssl/src/ssl_cipher.hrl +++ b/lib/ssl/src/ssl_cipher.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2015. All Rights Reserved. +%% Copyright Ericsson AB 2007-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssl/src/ssl_config.erl b/lib/ssl/src/ssl_config.erl index 452a98e683..1e6dab9276 100644 --- a/lib/ssl/src/ssl_config.erl +++ b/lib/ssl/src/ssl_config.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2017. All Rights Reserved. +%% Copyright Ericsson AB 2007-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -91,9 +91,9 @@ init_certificates(undefined, #{pem_cache := PemCache} = Config, CertFile, server end; init_certificates(Cert, Config, _, _) -> {ok, Config#{own_certificate => Cert}}. -init_private_key(_, #{algorithm := Alg} = Key, <<>>, _Password, _Client) when Alg == ecdsa; - Alg == rsa; - Alg == dss -> +init_private_key(_, #{algorithm := Alg} = Key, _, _Password, _Client) when Alg == ecdsa; + Alg == rsa; + Alg == dss -> case maps:is_key(engine, Key) andalso maps:is_key(key_id, Key) of true -> Key; diff --git a/lib/ssl/src/ssl_connection.erl b/lib/ssl/src/ssl_connection.erl index ec034af44c..6e98baf984 100644 --- a/lib/ssl/src/ssl_connection.erl +++ b/lib/ssl/src/ssl_connection.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2013-2017. All Rights Reserved. +%% Copyright Ericsson AB 2013-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -35,6 +35,7 @@ -include("ssl_internal.hrl"). -include("ssl_srp.hrl"). -include_lib("public_key/include/public_key.hrl"). +-include_lib("kernel/include/logger.hrl"). %% Setup @@ -334,14 +335,20 @@ handle_own_alert(Alert, Version, StateName, connection_states = ConnectionStates, ssl_options = SslOpts} = State) -> try %% Try to tell the other side - {BinMsg, _} = - Connection:encode_alert(Alert, Version, ConnectionStates), - Connection:send(Transport, Socket, BinMsg) + {BinMsg, _} = + Connection:encode_alert(Alert, Version, ConnectionStates), + Connection:send(Transport, Socket, BinMsg), + Report = #{direction => outbound, + protocol => 'tls_record', + message => BinMsg}, + ssl_logger:debug(SslOpts#ssl_options.log_level, Report, #{domain => [otp,ssl,tls_record]}) catch _:_ -> %% Can crash if we are in a uninitialized state ignore end, try %% Try to tell the local user - log_alert(SslOpts#ssl_options.log_alert, Role, Connection:protocol_name(), StateName, Alert#alert{role = Role}), + log_alert(SslOpts#ssl_options.log_level, Role, + Connection:protocol_name(), StateName, + Alert#alert{role = Role}), handle_normal_shutdown(Alert,StateName, State) catch _:_ -> ok @@ -372,8 +379,9 @@ handle_alert(#alert{level = ?FATAL} = Alert, StateName, port = Port, session = Session, user_application = {_Mon, Pid}, role = Role, socket_options = Opts, tracker = Tracker} = State) -> invalidate_session(Role, Host, Port, Session), - log_alert(SslOpts#ssl_options.log_alert, Role, Connection:protocol_name(), - StateName, Alert#alert{role = opposite_role(Role)}), + log_alert(SslOpts#ssl_options.log_level, Role, + Connection:protocol_name(), StateName, + Alert#alert{role = opposite_role(Role)}), alert_user(Transport, Tracker, Socket, StateName, Opts, Pid, From, Alert, Role, Connection), stop(normal, State); @@ -384,8 +392,9 @@ handle_alert(#alert{level = ?WARNING, description = ?CLOSE_NOTIFY} = Alert, handle_alert(#alert{level = ?WARNING, description = ?NO_RENEGOTIATION} = Alert, StateName, #state{role = Role, ssl_options = SslOpts, protocol_cb = Connection, renegotiation = {true, internal}} = State) -> - log_alert(SslOpts#ssl_options.log_alert, Role, - Connection:protocol_name(), StateName, Alert#alert{role = opposite_role(Role)}), + log_alert(SslOpts#ssl_options.log_level, Role, + Connection:protocol_name(), StateName, + Alert#alert{role = opposite_role(Role)}), handle_normal_shutdown(Alert, StateName, State), stop({shutdown, peer_close}, State); @@ -393,8 +402,9 @@ handle_alert(#alert{level = ?WARNING, description = ?NO_RENEGOTIATION} = Alert, #state{role = Role, ssl_options = SslOpts, renegotiation = {true, From}, protocol_cb = Connection} = State0) -> - log_alert(SslOpts#ssl_options.log_alert, Role, - Connection:protocol_name(), StateName, Alert#alert{role = opposite_role(Role)}), + log_alert(SslOpts#ssl_options.log_level, Role, + Connection:protocol_name(), StateName, + Alert#alert{role = opposite_role(Role)}), gen_statem:reply(From, {error, renegotiation_rejected}), {Record, State1} = Connection:next_record(State0), %% Go back to connection! @@ -404,8 +414,9 @@ handle_alert(#alert{level = ?WARNING, description = ?NO_RENEGOTIATION} = Alert, %% Gracefully log and ignore all other warning alerts handle_alert(#alert{level = ?WARNING} = Alert, StateName, #state{ssl_options = SslOpts, protocol_cb = Connection, role = Role} = State0) -> - log_alert(SslOpts#ssl_options.log_alert, Role, - Connection:protocol_name(), StateName, Alert#alert{role = opposite_role(Role)}), + log_alert(SslOpts#ssl_options.log_level, Role, + Connection:protocol_name(), StateName, + Alert#alert{role = opposite_role(Role)}), {Record, State} = Connection:next_record(State0), Connection:next_event(StateName, Record, State). @@ -419,7 +430,7 @@ write_application_data(Data0, {FromPid, _} = From, transport_cb = Transport, connection_states = ConnectionStates0, socket_options = SockOpts, - ssl_options = #ssl_options{renegotiate_at = RenegotiateAt}} = State) -> + ssl_options = #ssl_options{renegotiate_at = RenegotiateAt} = SslOpts} = State) -> Data = encode_packet(Data0, SockOpts), case time_to_renegotiate(Data, ConnectionStates0, RenegotiateAt) of @@ -430,16 +441,21 @@ write_application_data(Data0, {FromPid, _} = From, {Msgs, ConnectionStates} = Connection:encode_data(Data, Version, ConnectionStates0), NewState = State#state{connection_states = ConnectionStates}, - case Connection:send(Transport, Socket, Msgs) of - ok when FromPid =:= self() -> - hibernate_after(connection, NewState, []); - Error when FromPid =:= self() -> - stop({shutdown, Error}, NewState); - ok -> - hibernate_after(connection, NewState, [{reply, From, ok}]); - Result -> - hibernate_after(connection, NewState, [{reply, From, Result}]) - end + RetVal = case Connection:send(Transport, Socket, Msgs) of + ok when FromPid =:= self() -> + hibernate_after(connection, NewState, []); + Error when FromPid =:= self() -> + stop({shutdown, Error}, NewState); + ok -> + hibernate_after(connection, NewState, [{reply, From, ok}]); + Result -> + hibernate_after(connection, NewState, [{reply, From, Result}]) + end, + Report = #{direction => outbound, + protocol => 'tls_record', + message => Msgs}, + ssl_logger:debug(SslOpts#ssl_options.log_level, Report, #{domain => [otp,ssl,tls_record]}), + RetVal end. read_application_data(Data, #state{user_application = {_Mon, Pid}, @@ -634,8 +650,10 @@ init(_Type, _Event, _State, _Connection) -> tls_connection | dtls_connection) -> gen_statem:state_function_result(). %%-------------------------------------------------------------------- -error({call, From}, Msg, State, Connection) -> - handle_call(Msg, From, ?FUNCTION_NAME, State, Connection). +error({call, From}, {close, _}, State, _Connection) -> + stop_and_reply(normal, {reply, From, ok}, State); +error({call, From}, _Msg, State, _Connection) -> + {next_state, ?FUNCTION_NAME, State, [{reply, From, {error, closed}}]}. %%-------------------------------------------------------------------- -spec hello(gen_statem:event_type(), @@ -791,6 +809,7 @@ certify(internal, #server_key_exchange{exchange_keys = Keys}, #state{role = client, negotiated_version = Version, key_algorithm = Alg, public_key_info = PubKeyInfo, + session = Session, connection_states = ConnectionStates} = State, Connection) when Alg == dhe_dss; Alg == dhe_rsa; Alg == ecdhe_rsa; Alg == ecdhe_ecdsa; @@ -812,7 +831,8 @@ certify(internal, #server_key_exchange{exchange_keys = Keys}, ConnectionStates, ssl:tls_version(Version), PubKeyInfo) of true -> calculate_secret(Params#server_key_params.params, - State#state{hashsign_algorithm = HashSign}, + State#state{hashsign_algorithm = HashSign, + session = session_handle_params(Params#server_key_params.params, Session)}, Connection); false -> handle_own_alert(?ALERT_REC(?FATAL, ?DECRYPT_ERROR), @@ -1339,7 +1359,7 @@ handle_info({ErrorTag, Socket, econnaborted}, StateName, handle_info({ErrorTag, Socket, Reason}, StateName, #state{socket = Socket, error_tag = ErrorTag} = State) -> Report = io_lib:format("SSL: Socket error: ~p ~n", [Reason]), - error_logger:error_report(Report), + ?LOG_ERROR(Report), handle_normal_shutdown(?ALERT_REC(?FATAL, ?CLOSE_NOTIFY), StateName, State), stop(normal, State); @@ -1388,7 +1408,7 @@ handle_info({cancel_start_or_recv, _RecvFrom}, StateName, State) -> handle_info(Msg, StateName, #state{socket = Socket, error_tag = Tag} = State) -> Report = io_lib:format("SSL: Got unexpected info: ~p ~n", [{Msg, Tag, Socket}]), - error_logger:info_report(Report), + ?LOG_NOTICE(Report), {next_state, StateName, State}. %%==================================================================== @@ -1472,7 +1492,7 @@ connection_info(#state{sni_hostname = SNIHostname, RecordCB = record_cb(Connection), CipherSuiteDef = #{key_exchange := KexAlg} = ssl_cipher:suite_definition(CipherSuite), IsNamedCurveSuite = lists:member(KexAlg, - [ecdh_ecdsa, ecdhe_ecdsa, ecdh_rsa, ecdh_anon]), + [ecdh_ecdsa, ecdhe_ecdsa, ecdh_rsa, ecdhe_rsa, ecdh_anon]), CurveInfo = case ECCCurve of {namedCurve, Curve} when IsNamedCurveSuite -> [{ecc, {named_curve, pubkey_cert_records:namedCurves(Curve)}}]; @@ -1482,6 +1502,7 @@ connection_info(#state{sni_hostname = SNIHostname, [{protocol, RecordCB:protocol_version(Version)}, {session_id, SessionId}, {cipher_suite, ssl_cipher:erl_suite_definition(CipherSuiteDef)}, + {selected_cipher_suite, CipherSuiteDef}, {sni_hostname, SNIHostname} | CurveInfo] ++ ssl_options_list(Opts). security_info(#state{connection_states = ConnectionStates}) -> @@ -1496,13 +1517,18 @@ do_server_hello(Type, #hello_extensions{next_protocol_negotiation = NextProtocol ServerHelloExt, #state{negotiated_version = Version, session = #session{session_id = SessId}, - connection_states = ConnectionStates0} + connection_states = ConnectionStates0, + ssl_options = #ssl_options{versions = [HighestVersion|_]}} = State0, Connection) when is_atom(Type) -> - + %% TLS 1.3 - Section 4.1.3 + %% Override server random values for TLS 1.3 downgrade protection mechanism. + ConnectionStates1 = update_server_random(ConnectionStates0, Version, HighestVersion), + State1 = State0#state{connection_states = ConnectionStates1}, ServerHello = - ssl_handshake:server_hello(SessId, ssl:tls_version(Version), ConnectionStates0, ServerHelloExt), + ssl_handshake:server_hello(SessId, ssl:tls_version(Version), + ConnectionStates1, ServerHelloExt), State = server_hello(ServerHello, - State0#state{expecting_next_protocol_negotiation = + State1#state{expecting_next_protocol_negotiation = NextProtocols =/= undefined}, Connection), case Type of new -> @@ -1511,6 +1537,60 @@ do_server_hello(Type, #hello_extensions{next_protocol_negotiation = NextProtocol resumed_server_hello(State, Connection) end. +update_server_random(#{pending_read := #{security_parameters := ReadSecParams0} = + ReadState0, + pending_write := #{security_parameters := WriteSecParams0} = + WriteState0} = ConnectionStates, + Version, HighestVersion) -> + ReadRandom = override_server_random( + ReadSecParams0#security_parameters.server_random, + Version, + HighestVersion), + WriteRandom = override_server_random( + WriteSecParams0#security_parameters.server_random, + Version, + HighestVersion), + ReadSecParams = ReadSecParams0#security_parameters{server_random = ReadRandom}, + WriteSecParams = WriteSecParams0#security_parameters{server_random = WriteRandom}, + ReadState = ReadState0#{security_parameters => ReadSecParams}, + WriteState = WriteState0#{security_parameters => WriteSecParams}, + + ConnectionStates#{pending_read => ReadState, pending_write => WriteState}. + +%% TLS 1.3 - Section 4.1.3 +%% +%% If negotiating TLS 1.2, TLS 1.3 servers MUST set the last eight bytes +%% of their Random value to the bytes: +%% +%% 44 4F 57 4E 47 52 44 01 +%% +%% If negotiating TLS 1.1 or below, TLS 1.3 servers MUST and TLS 1.2 +%% servers SHOULD set the last eight bytes of their Random value to the +%% bytes: +%% +%% 44 4F 57 4E 47 52 44 00 +override_server_random(<<Random0:24/binary,_:8/binary>> = Random, {M,N}, {Major,Minor}) + when Major > 3 orelse Major =:= 3 andalso Minor >= 4 -> %% TLS 1.3 or above + if M =:= 3 andalso N =:= 3 -> %% Negotating TLS 1.2 + Down = ?RANDOM_OVERRIDE_TLS12, + <<Random0/binary,Down/binary>>; + M =:= 3 andalso N < 3 -> %% Negotating TLS 1.1 or prior + Down = ?RANDOM_OVERRIDE_TLS11, + <<Random0/binary,Down/binary>>; + true -> + Random + end; +override_server_random(<<Random0:24/binary,_:8/binary>> = Random, {M,N}, {Major,Minor}) + when Major =:= 3 andalso Minor =:= 3 -> %% TLS 1.2 + if M =:= 3 andalso N < 3 -> %% Negotating TLS 1.1 or prior + Down = ?RANDOM_OVERRIDE_TLS11, + <<Random0/binary,Down/binary>>; + true -> + Random + end; +override_server_random(Random, _, _) -> + Random. + new_server_hello(#server_hello{cipher_suite = CipherSuite, compression_method = Compression, session_id = SessionId}, @@ -1575,11 +1655,9 @@ handle_peer_cert_key(client, _, KeyAlg, #state{session = Session} = State) when KeyAlg == ecdh_rsa; KeyAlg == ecdh_ecdsa -> ECDHKey = public_key:generate_key(PublicKeyParams), - {namedCurve, Oid} = PublicKeyParams, - Curve = pubkey_cert_records:namedCurves(Oid), %% Need API function PremasterSecret = ssl_handshake:premaster_secret(PublicKey, ECDHKey), master_secret(PremasterSecret, State#state{diffie_hellman_keys = ECDHKey, - session = Session#session{ecc = {named_curve, Curve}}}); + session = Session#session{ecc = PublicKeyParams}}); %% We do currently not support cipher suites that use fixed DH. %% If we want to implement that the following clause can be used %% to extract DH parameters form cert. @@ -1756,9 +1834,11 @@ key_exchange(#state{role = server, key_algorithm = Algo, PrivateKey}), State = Connection:queue_handshake(Msg, State0), State#state{diffie_hellman_keys = DHKeys}; -key_exchange(#state{role = server, private_key = Key, key_algorithm = Algo} = State, _) +key_exchange(#state{role = server, private_key = #'ECPrivateKey'{parameters = ECCurve} = Key, key_algorithm = Algo, + session = Session} = State, _) when Algo == ecdh_ecdsa; Algo == ecdh_rsa -> - State#state{diffie_hellman_keys = Key}; + State#state{diffie_hellman_keys = Key, + session = Session#session{ecc = ECCurve}}; key_exchange(#state{role = server, key_algorithm = Algo, hashsign_algorithm = HashSignAlgo, private_key = PrivateKey, @@ -1914,12 +1994,13 @@ key_exchange(#state{role = client, key_exchange(#state{role = client, key_algorithm = Algorithm, negotiated_version = Version, - diffie_hellman_keys = Keys} = State0, Connection) + session = Session, + diffie_hellman_keys = #'ECPrivateKey'{parameters = ECCurve} = Key} = State0, Connection) when Algorithm == ecdhe_ecdsa; Algorithm == ecdhe_rsa; Algorithm == ecdh_ecdsa; Algorithm == ecdh_rsa; Algorithm == ecdh_anon -> - Msg = ssl_handshake:key_exchange(client, ssl:tls_version(Version), {ecdh, Keys}), - Connection:queue_handshake(Msg, State0); + Msg = ssl_handshake:key_exchange(client, ssl:tls_version(Version), {ecdh, Key}), + Connection:queue_handshake(Msg, State0#state{session = Session#session{ecc = ECCurve}}); key_exchange(#state{role = client, ssl_options = SslOpts, key_algorithm = psk, @@ -2391,7 +2472,7 @@ handle_trusted_certs_db(#state{ssl_options = handle_trusted_certs_db(#state{cert_db_ref = Ref, cert_db = CertDb, ssl_options = #ssl_options{cacertfile = <<>>}}) when CertDb =/= undefined -> - %% Certs provided as DER directly can not be shared + %% Certs provided as DER directly cannot be shared %% with other connections and it is safe to delete them when the connection ends. ssl_pkix_db:remove_trusted_certs(Ref, CertDb); handle_trusted_certs_db(#state{file_ref_db = undefined}) -> @@ -2442,6 +2523,11 @@ cancel_timer(Timer) -> erlang:cancel_timer(Timer), ok. +session_handle_params(#server_ecdh_params{curve = ECCurve}, Session) -> + Session#session{ecc = ECCurve}; +session_handle_params(_, Session) -> + Session. + register_session(client, Host, Port, #session{is_resumable = new} = Session0) -> Session = Session0#session{is_resumable = true}, ssl_manager:register_session(Host, Port, Session), @@ -2522,7 +2608,7 @@ ssl_options_list([ciphers = Key | Keys], [Value | Values], Acc) -> ssl_options_list(Keys, Values, [{Key, lists:map( fun(Suite) -> - ssl_cipher:erl_suite_definition(Suite) + ssl_cipher:suite_definition(Suite) end, Value)} | Acc]); ssl_options_list([Key | Keys], [Value | Values], Acc) -> @@ -2739,14 +2825,14 @@ alert_user(Transport, Tracker, Socket, Active, Pid, From, Alert, Role, Connectio Transport, Socket, Connection, Tracker), ReasonCode}) end. -log_alert(true, Role, ProtocolName, StateName, #alert{role = Role} = Alert) -> +log_alert(Level, Role, ProtocolName, StateName, #alert{role = Role} = Alert) -> Txt = ssl_alert:own_alert_txt(Alert), - error_logger:info_report(io_lib:format("~s ~p: In state ~p ~s\n", [ProtocolName, Role, StateName, Txt])); -log_alert(true, Role, ProtocolName, StateName, Alert) -> + Report = io_lib:format("~s ~p: In state ~p ~s\n", [ProtocolName, Role, StateName, Txt]), + ssl_logger:notice(Level, Report); +log_alert(Level, Role, ProtocolName, StateName, Alert) -> Txt = ssl_alert:alert_txt(Alert), - error_logger:info_report(io_lib:format("~s ~p: In state ~p ~s\n", [ProtocolName, Role, StateName, Txt])); -log_alert(false, _, _, _, _) -> - ok. + Report = io_lib:format("~s ~p: In state ~p ~s\n", [ProtocolName, Role, StateName, Txt]), + ssl_logger:notice(Level, Report). invalidate_session(client, Host, Port, Session) -> ssl_manager:invalidate_session(Host, Port, Session); @@ -2836,7 +2922,7 @@ erl_dist_stop_reason( Reason, #state{ssl_options = #ssl_options{erl_dist = true}}) -> case Reason of normal -> - %% We can not exit with normal since that will not bring + %% We cannot exit with normal since that will not bring %% down the rest of the distribution processes {shutdown, normal}; _ -> Reason diff --git a/lib/ssl/src/ssl_connection.hrl b/lib/ssl/src/ssl_connection.hrl index a61ff747ae..811aa779d5 100644 --- a/lib/ssl/src/ssl_connection.hrl +++ b/lib/ssl/src/ssl_connection.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2013-2017. All Rights Reserved. +%% Copyright Ericsson AB 2013-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssl/src/ssl_connection_sup.erl b/lib/ssl/src/ssl_connection_sup.erl index 1aa7c5844f..934dd39df5 100644 --- a/lib/ssl/src/ssl_connection_sup.erl +++ b/lib/ssl/src/ssl_connection_sup.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1998-2016. All Rights Reserved. +%% Copyright Ericsson AB 1998-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssl/src/ssl_crl_cache.erl b/lib/ssl/src/ssl_crl_cache.erl index 66f46da75f..9c1af86eeb 100644 --- a/lib/ssl/src/ssl_crl_cache.erl +++ b/lib/ssl/src/ssl_crl_cache.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2015-2016. All Rights Reserved. +%% Copyright Ericsson AB 2015-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssl/src/ssl_crl_hash_dir.erl b/lib/ssl/src/ssl_crl_hash_dir.erl index bb62737232..9478ff9b78 100644 --- a/lib/ssl/src/ssl_crl_hash_dir.erl +++ b/lib/ssl/src/ssl_crl_hash_dir.erl @@ -20,6 +20,7 @@ -module(ssl_crl_hash_dir). -include_lib("public_key/include/public_key.hrl"). +-include_lib("kernel/include/logger.hrl"). -behaviour(ssl_crl_cache_api). @@ -55,7 +56,7 @@ select(Issuer, {_DbHandle, [{dir, Dir}]}) -> %% is happy with that, but if it's true, this is an error. []; {error, Error} -> - error_logger:error_report( + ?LOG_ERROR( [{cannot_find_crl, Error}, {dir, Dir}, {module, ?MODULE}, @@ -86,7 +87,7 @@ find_crls(Issuer, Hash, Dir, N, Acc) -> error:Error -> %% Something is wrong with the file. Report %% it, and try the next one. - error_logger:error_report( + ?LOG_ERROR( [{crl_parse_error, Error}, {filename, Filename}, {module, ?MODULE}, diff --git a/lib/ssl/src/ssl_handshake.erl b/lib/ssl/src/ssl_handshake.erl index 71eeb00183..73757e6b65 100644 --- a/lib/ssl/src/ssl_handshake.erl +++ b/lib/ssl/src/ssl_handshake.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2013-2017. All Rights Reserved. +%% Copyright Ericsson AB 2013-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -53,7 +53,7 @@ -export([certify/7, certificate_verify/6, verify_signature/5, master_secret/4, server_key_exchange_hash/2, verify_connection/6, init_handshake_history/0, update_handshake_history/2, verify_server_key/5, - select_version/3, extension_value/1 + select_version/3, select_supported_version/2, extension_value/1 ]). %% Encode @@ -505,6 +505,21 @@ verify_server_key(#server_key_params{params_bin = EncParams, select_version(RecordCB, ClientVersion, Versions) -> do_select_version(RecordCB, ClientVersion, Versions). + +%% Called by TLS 1.2/1.3 Server when "supported_versions" is present +%% in ClientHello. +%% Input lists are ordered (highest first) +select_supported_version([], _ServerVersions) -> + undefined; +select_supported_version([ClientVersion|T], ServerVersions) -> + case lists:member(ClientVersion, ServerVersions) of + true -> + ClientVersion; + false -> + select_supported_version(T, ServerVersions) + end. + + %%==================================================================== %% Encode handshake %%==================================================================== @@ -632,7 +647,20 @@ encode_hello_extensions([#sni{hostname = Hostname} | Rest], Acc) -> ?UINT16(ServerNameLength), ?BYTE(?SNI_NAMETYPE_HOST_NAME), ?UINT16(HostLen), HostnameBin/binary, - Acc/binary>>). + Acc/binary>>); +encode_hello_extensions([#client_hello_versions{versions = Versions0} | Rest], Acc) -> + Versions = encode_versions(Versions0), + VerLen = byte_size(Versions), + Len = VerLen + 2, + encode_hello_extensions(Rest, <<?UINT16(?SUPPORTED_VERSIONS_EXT), + ?UINT16(Len), ?UINT16(VerLen), Versions/binary, Acc/binary>>); +encode_hello_extensions([#server_hello_selected_version{selected_version = Version0} | Rest], Acc) -> + Version = encode_versions(Version0), + Len = byte_size(Version), %% 2 + encode_hello_extensions(Rest, <<?UINT16(?SUPPORTED_VERSIONS_EXT), + ?UINT16(Len), Version/binary, Acc/binary>>). + + encode_client_protocol_negotiation(undefined, _) -> undefined; @@ -930,7 +958,8 @@ premaster_secret(EncSecret, #'RSAPrivateKey'{} = RSAPrivateKey) -> %%==================================================================== client_hello_extensions(Version, CipherSuites, #ssl_options{signature_algs = SupportedHashSigns, - eccs = SupportedECCs} = SslOpts, ConnectionStates, Renegotiation) -> + eccs = SupportedECCs, + versions = Versions} = SslOpts, ConnectionStates, Renegotiation) -> {EcPointFormats, EllipticCurves} = case advertises_ec_ciphers(lists:map(fun ssl_cipher:suite_definition/1, CipherSuites)) of true -> @@ -940,18 +969,29 @@ client_hello_extensions(Version, CipherSuites, end, SRP = srp_user(SslOpts), - #hello_extensions{ - renegotiation_info = renegotiation_info(tls_record, client, - ConnectionStates, Renegotiation), - srp = SRP, - signature_algs = available_signature_algs(SupportedHashSigns, Version), - ec_point_formats = EcPointFormats, - elliptic_curves = EllipticCurves, - alpn = encode_alpn(SslOpts#ssl_options.alpn_advertised_protocols, Renegotiation), - next_protocol_negotiation = - encode_client_protocol_negotiation(SslOpts#ssl_options.next_protocol_selector, - Renegotiation), - sni = sni(SslOpts#ssl_options.server_name_indication)}. + HelloExtensions = + #hello_extensions{ + renegotiation_info = renegotiation_info(tls_record, client, + ConnectionStates, Renegotiation), + srp = SRP, + signature_algs = available_signature_algs(SupportedHashSigns, Version), + ec_point_formats = EcPointFormats, + elliptic_curves = EllipticCurves, + alpn = encode_alpn(SslOpts#ssl_options.alpn_advertised_protocols, Renegotiation), + next_protocol_negotiation = + encode_client_protocol_negotiation(SslOpts#ssl_options.next_protocol_selector, + Renegotiation), + sni = sni(SslOpts#ssl_options.server_name_indication)}, + + %% Add "supported_versions" extension if TLS 1.3 + case Version of + {3,4} -> + HelloExtensions#hello_extensions{ + client_hello_versions = #client_hello_versions{ + versions = Versions}}; + _Else -> + HelloExtensions + end. handle_client_hello_extensions(RecordCB, Random, ClientCipherSuites, #hello_extensions{renegotiation_info = Info, @@ -1055,7 +1095,8 @@ select_curve(undefined, _, _) -> %%-------------------------------------------------------------------- select_hashsign(_, _, KeyExAlgo, _, _Version) when KeyExAlgo == dh_anon; KeyExAlgo == ecdh_anon; - KeyExAlgo == srp_anon -> + KeyExAlgo == srp_anon; + KeyExAlgo == psk -> {null, anon}; %% The signature_algorithms extension was introduced with TLS 1.2. Ignore it if we have %% negotiated a lower version. @@ -1064,17 +1105,14 @@ select_hashsign(HashSigns, Cert, KeyExAlgo, select_hashsign(HashSigns, Cert, KeyExAlgo, tls_v1:default_signature_algs(Version), Version); select_hashsign(#hash_sign_algos{hash_sign_algos = HashSigns}, Cert, KeyExAlgo, SupportedHashSigns, {Major, Minor}) when Major >= 3 andalso Minor >= 3 -> - #'OTPCertificate'{tbsCertificate = TBSCert, - signatureAlgorithm = {_,SignAlgo, _}} = public_key:pkix_decode_cert(Cert, otp), + #'OTPCertificate'{tbsCertificate = TBSCert} = public_key:pkix_decode_cert(Cert, otp), #'OTPSubjectPublicKeyInfo'{algorithm = {_, SubjAlgo, _}} = TBSCert#'OTPTBSCertificate'.subjectPublicKeyInfo, - Sign = sign_algo(SignAlgo), SubSign = sign_algo(SubjAlgo), case lists:filter(fun({_, S} = Algos) when S == SubSign -> - is_acceptable_hash_sign(Algos, Sign, - SubSign, KeyExAlgo, SupportedHashSigns); + is_acceptable_hash_sign(Algos, KeyExAlgo, SupportedHashSigns); (_) -> false end, HashSigns) of @@ -1724,6 +1762,16 @@ encode_alpn(undefined, _) -> encode_alpn(Protocols, _) -> #alpn{extension_data = lists:foldl(fun encode_protocol/2, <<>>, Protocols)}. + +encode_versions(Versions) -> + encode_versions(lists:reverse(Versions), <<>>). +%% +encode_versions([], Acc) -> + Acc; +encode_versions([{M,N}|T], Acc) -> + encode_versions(T, <<?BYTE(M),?BYTE(N),Acc/binary>>). + + hello_extensions_list(#hello_extensions{renegotiation_info = RenegotiationInfo, srp = SRP, signature_algs = HashSigns, @@ -1731,9 +1779,13 @@ hello_extensions_list(#hello_extensions{renegotiation_info = RenegotiationInfo, elliptic_curves = EllipticCurves, alpn = ALPN, next_protocol_negotiation = NextProtocolNegotiation, - sni = Sni}) -> + sni = Sni, + client_hello_versions = Versions, + server_hello_selected_version = Version}) -> [Ext || Ext <- [RenegotiationInfo, SRP, HashSigns, - EcPointFormats, EllipticCurves, ALPN, NextProtocolNegotiation, Sni], Ext =/= undefined]. + EcPointFormats, EllipticCurves, ALPN, + NextProtocolNegotiation, Sni, + Versions, Version], Ext =/= undefined]. %%-------------Decode handshakes--------------------------------- dec_server_key(<<?UINT16(PLen), P:PLen/binary, @@ -1933,15 +1985,28 @@ dec_hello_extensions(<<?UINT16(?EC_POINT_FORMATS_EXT), ?UINT16(Len), ECPointFormats}}); dec_hello_extensions(<<?UINT16(?SNI_EXT), ?UINT16(Len), Rest/binary>>, Acc) when Len == 0 -> - dec_hello_extensions(Rest, Acc#hello_extensions{sni = ""}); %% Server may send an empy SNI + dec_hello_extensions(Rest, Acc#hello_extensions{sni = #sni{hostname = ""}}); %% Server may send an empy SNI dec_hello_extensions(<<?UINT16(?SNI_EXT), ?UINT16(Len), ExtData:Len/binary, Rest/binary>>, Acc) -> <<?UINT16(_), NameList/binary>> = ExtData, dec_hello_extensions(Rest, Acc#hello_extensions{sni = dec_sni(NameList)}); + +dec_hello_extensions(<<?UINT16(?SUPPORTED_VERSIONS_EXT), ?UINT16(Len), + ExtData:Len/binary, Rest/binary>>, Acc) when Len > 2 -> + <<?UINT16(_),Versions/binary>> = ExtData, + dec_hello_extensions(Rest, Acc#hello_extensions{ + client_hello_versions = + #client_hello_versions{versions = decode_versions(Versions)}}); + +dec_hello_extensions(<<?UINT16(?SUPPORTED_VERSIONS_EXT), ?UINT16(Len), + ?UINT16(Version), Rest/binary>>, Acc) when Len =:= 2, Version =:= 16#0304 -> + dec_hello_extensions(Rest, Acc#hello_extensions{ + server_hello_selected_version = + #server_hello_selected_version{selected_version = [{3,4}]}}); + %% Ignore data following the ClientHello (i.e., %% extensions) if not understood. - dec_hello_extensions(<<?UINT16(_), ?UINT16(Len), _Unknown:Len/binary, Rest/binary>>, Acc) -> dec_hello_extensions(Rest, Acc); %% This theoretically should not happen if the protocol is followed, but if it does it is ignored. @@ -1963,6 +2028,15 @@ decode_alpn(undefined) -> decode_alpn(#alpn{extension_data=Data}) -> decode_protocols(Data, []). +decode_versions(Versions) -> + decode_versions(Versions, []). +%% +decode_versions(<<>>, Acc) -> + lists:reverse(Acc); +decode_versions(<<?BYTE(M),?BYTE(N),Rest/binary>>, Acc) -> + decode_versions(Rest, [{M,N}|Acc]). + + decode_next_protocols({next_protocol_negotiation, Protocols}) -> decode_protocols(Protocols, []). @@ -2231,37 +2305,7 @@ sign_algo(Alg) -> {_, Sign} =public_key:pkix_sign_types(Alg), Sign. -is_acceptable_hash_sign(Algos, _, _, KeyExAlgo, SupportedHashSigns) when - KeyExAlgo == dh_dss; - KeyExAlgo == dh_rsa; - KeyExAlgo == ecdh_rsa; - KeyExAlgo == ecdh_ecdsa - -> - %% *dh_* could be called only *dh in TLS-1.2 - is_acceptable_hash_sign(Algos, SupportedHashSigns); -is_acceptable_hash_sign(Algos, rsa, ecdsa, ecdhe_rsa, SupportedHashSigns) -> - is_acceptable_hash_sign(Algos, SupportedHashSigns); -is_acceptable_hash_sign({_, rsa} = Algos, rsa, _, dhe_rsa, SupportedHashSigns) -> - is_acceptable_hash_sign(Algos, SupportedHashSigns); -is_acceptable_hash_sign({_, rsa} = Algos, rsa, rsa, ecdhe_rsa, SupportedHashSigns) -> - is_acceptable_hash_sign(Algos, SupportedHashSigns); -is_acceptable_hash_sign({_, rsa} = Algos, rsa, rsa, rsa, SupportedHashSigns) -> - is_acceptable_hash_sign(Algos, SupportedHashSigns); -is_acceptable_hash_sign({_, rsa} = Algos, rsa, _, srp_rsa, SupportedHashSigns) -> - is_acceptable_hash_sign(Algos, SupportedHashSigns); -is_acceptable_hash_sign({_, rsa} = Algos, rsa, _, rsa_psk, SupportedHashSigns) -> - is_acceptable_hash_sign(Algos, SupportedHashSigns); -is_acceptable_hash_sign({_, dsa} = Algos, dsa, _, dhe_dss, SupportedHashSigns) -> - is_acceptable_hash_sign(Algos, SupportedHashSigns); -is_acceptable_hash_sign({_, dsa} = Algos, dsa, _, srp_dss, SupportedHashSigns) -> - is_acceptable_hash_sign(Algos, SupportedHashSigns); -is_acceptable_hash_sign({_, ecdsa} = Algos, ecdsa, _, dhe_ecdsa, SupportedHashSigns) -> - is_acceptable_hash_sign(Algos, SupportedHashSigns); -is_acceptable_hash_sign({_, ecdsa} = Algos, ecdsa, ecdsa, ecdh_ecdsa, SupportedHashSigns) -> - is_acceptable_hash_sign(Algos, SupportedHashSigns); -is_acceptable_hash_sign({_, ecdsa} = Algos, ecdsa, ecdsa, ecdhe_ecdsa, SupportedHashSigns) -> - is_acceptable_hash_sign(Algos, SupportedHashSigns); -is_acceptable_hash_sign(_, _, _, KeyExAlgo, _) when +is_acceptable_hash_sign( _, KeyExAlgo, _) when KeyExAlgo == psk; KeyExAlgo == dhe_psk; KeyExAlgo == ecdhe_psk; @@ -2270,8 +2314,9 @@ is_acceptable_hash_sign(_, _, _, KeyExAlgo, _) when KeyExAlgo == ecdhe_anon -> true; -is_acceptable_hash_sign(_,_,_,_,_) -> - false. +is_acceptable_hash_sign(Algos,_, SupportedHashSigns) -> + is_acceptable_hash_sign(Algos, SupportedHashSigns). + is_acceptable_hash_sign(Algos, SupportedHashSigns) -> lists:member(Algos, SupportedHashSigns). @@ -2464,13 +2509,7 @@ cert_curve(Cert, ECCCurve0, CipherSuite) -> #'OTPSubjectPublicKeyInfo'{algorithm = AlgInfo} = TBSCert#'OTPTBSCertificate'.subjectPublicKeyInfo, {namedCurve, Oid} = AlgInfo#'PublicKeyAlgorithm'.parameters, - try pubkey_cert_records:namedCurves(Oid) of - Curve -> - {{named_curve, Curve}, CipherSuite} - catch - _:_ -> - {no_curve, no_suite} - end; + {{namedCurve, Oid}, CipherSuite}; _ -> {ECCCurve0, CipherSuite} end. diff --git a/lib/ssl/src/ssl_handshake.hrl b/lib/ssl/src/ssl_handshake.hrl index a191fcf766..9cc6f570fc 100644 --- a/lib/ssl/src/ssl_handshake.hrl +++ b/lib/ssl/src/ssl_handshake.hrl @@ -105,7 +105,9 @@ srp, ec_point_formats, elliptic_curves, - sni + sni, + client_hello_versions, + server_hello_selected_version }). -record(server_hello, { @@ -377,4 +379,13 @@ hostname = undefined }). +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%% Supported Versions TLS 1.3 section 4.2.1 +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +-define(SUPPORTED_VERSIONS_EXT, 43). + +-record(client_hello_versions, {versions}). +-record(server_hello_selected_version, {selected_version}). + -endif. % -ifdef(ssl_handshake). diff --git a/lib/ssl/src/ssl_internal.hrl b/lib/ssl/src/ssl_internal.hrl index b736047678..0d3093c1ae 100644 --- a/lib/ssl/src/ssl_internal.hrl +++ b/lib/ssl/src/ssl_internal.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2017. All Rights Reserved. +%% Copyright Ericsson AB 2007-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -25,6 +25,7 @@ -include_lib("public_key/include/public_key.hrl"). +-define(VSN, "8.2.6"). -define(SECRET_PRINTOUT, "***"). -type reason() :: term(). @@ -71,14 +72,39 @@ -define(FALSE, 1). %% sslv3 is considered insecure due to lack of padding check (Poodle attack) -%% Keep as interop with legacy software but do not support as default --define(ALL_AVAILABLE_VERSIONS, ['tlsv1.2', 'tlsv1.1', tlsv1, sslv3]). +%% Keep as interop with legacy software but do not support as default +%% tlsv1.3 is under development (experimental). +-define(ALL_AVAILABLE_VERSIONS, ['tlsv1.3', 'tlsv1.2', 'tlsv1.1', tlsv1, sslv3]). -define(ALL_AVAILABLE_DATAGRAM_VERSIONS, ['dtlsv1.2', dtlsv1]). +%% Defines the default versions when not specified by an ssl option. -define(ALL_SUPPORTED_VERSIONS, ['tlsv1.2', 'tlsv1.1', tlsv1]). -define(MIN_SUPPORTED_VERSIONS, ['tlsv1.1', tlsv1]). + +%% Versions allowed in TLSCiphertext.version (TLS 1.2 and prior) and +%% TLSCiphertext.legacy_record_version (TLS 1.3). +%% TLS 1.3 sets TLSCiphertext.legacy_record_version to 0x0303 for all records +%% generated other than an than an initial ClientHello, where it MAY also be 0x0301. +%% Thus, the allowed range is limited to 0x0300 - 0x0303. +-define(ALL_TLS_RECORD_VERSIONS, ['tlsv1.2', 'tlsv1.1', tlsv1, sslv3]). + -define(ALL_DATAGRAM_SUPPORTED_VERSIONS, ['dtlsv1.2', dtlsv1]). -define(MIN_DATAGRAM_SUPPORTED_VERSIONS, [dtlsv1]). +%% TLS 1.3 - Section 4.1.3 +%% +%% If negotiating TLS 1.2, TLS 1.3 servers MUST set the last eight bytes +%% of their Random value to the bytes: +%% +%% 44 4F 57 4E 47 52 44 01 +%% +%% If negotiating TLS 1.1 or below, TLS 1.3 servers MUST and TLS 1.2 +%% servers SHOULD set the last eight bytes of their Random value to the +%% bytes: +%% +%% 44 4F 57 4E 47 52 44 00 +-define(RANDOM_OVERRIDE_TLS12, <<16#44,16#4F,16#57,16#4E,16#47,16#52,16#44,16#01>>). +-define(RANDOM_OVERRIDE_TLS11, <<16#44,16#4F,16#57,16#4E,16#47,16#52,16#44,16#00>>). + -define('24H_in_msec', 86400000). -define('24H_in_sec', 86400). @@ -127,7 +153,7 @@ alpn_preferred_protocols = undefined :: [binary()] | undefined, next_protocols_advertised = undefined :: [binary()] | undefined, next_protocol_selector = undefined, %% fun([binary()]) -> binary()) - log_alert :: boolean(), + log_level = notice :: atom(), server_name_indication = undefined, sni_hosts :: [{inet:hostname(), [tuple()]}], sni_fun :: function() | undefined, @@ -180,6 +206,8 @@ -type gen_fsm_state_return() :: {next_state, state_name(), term()} | {next_state, state_name(), term(), timeout()} | {stop, term(), term()}. +-type ssl_options() :: #ssl_options{}. + -endif. % -ifdef(ssl_internal). diff --git a/lib/ssl/src/ssl_logger.erl b/lib/ssl/src/ssl_logger.erl new file mode 100644 index 0000000000..35c8dcfd48 --- /dev/null +++ b/lib/ssl/src/ssl_logger.erl @@ -0,0 +1,349 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 1999-2018. All Rights Reserved. +%% +%% Licensed under the Apache License, Version 2.0 (the "License"); +%% you may not use this file except in compliance with the License. +%% You may obtain a copy of the License at +%% +%% http://www.apache.org/licenses/LICENSE-2.0 +%% +%% Unless required by applicable law or agreed to in writing, software +%% distributed under the License is distributed on an "AS IS" BASIS, +%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +%% See the License for the specific language governing permissions and +%% limitations under the License. +%% +%% %CopyrightEnd% +%% + +-module(ssl_logger). + +-export([debug/3, + format/2, + notice/2]). + +-define(DEC2HEX(X), + if ((X) >= 0) andalso ((X) =< 9) -> (X) + $0; + ((X) >= 10) andalso ((X) =< 15) -> (X) + $a - 10 + end). + +-define(rec_info(T,R),lists:zip(record_info(fields,T),tl(tuple_to_list(R)))). + +-include("tls_record.hrl"). +-include("ssl_internal.hrl"). +-include("tls_handshake.hrl"). +-include_lib("kernel/include/logger.hrl"). + +%%------------------------------------------------------------------------- +%% External API +%%------------------------------------------------------------------------- + +%% SSL log formatter +format(#{level:= _Level, msg:= {report, Msg}, meta:= _Meta}, _Config0) -> + #{direction := Direction, + protocol := Protocol, + message := BinMsg0} = Msg, + case Protocol of + 'tls_record' -> + BinMsg = lists:flatten(BinMsg0), + format_tls_record(Direction, BinMsg); + 'handshake' -> + format_handshake(Direction, BinMsg0); + _Other -> + [] + end. + +%% Stateful logging +debug(Level, Report, Meta) -> + case logger:compare_levels(Level, debug) of + lt -> + ?LOG_DEBUG(Report, Meta); + eq -> + ?LOG_DEBUG(Report, Meta); + _ -> + ok + end. + +%% Stateful logging +notice(Level, Report) -> + case logger:compare_levels(Level, notice) of + lt -> + ?LOG_NOTICE(Report); + eq -> + ?LOG_NOTICE(Report); + _ -> + ok + end. + + +%%------------------------------------------------------------------------- +%% Handshake Protocol +%%------------------------------------------------------------------------- +format_handshake(Direction, BinMsg) -> + {Header, Message} = parse_handshake(Direction, BinMsg), + io_lib:format("~s~n~s~n", [Header, Message]). + + +parse_handshake(Direction, #client_hello{ + client_version = Version + } = ClientHello) -> + Header = io_lib:format("~s ~s Handshake, ClientHello", + [header_prefix(Direction), + version(Version)]), + Message = io_lib:format("~p", [?rec_info(client_hello, ClientHello)]), + {Header, Message}; +parse_handshake(Direction, #server_hello{ + server_version = Version + } = ServerHello) -> + Header = io_lib:format("~s ~s Handshake, ServerHello", + [header_prefix(Direction), + version(Version)]), + Message = io_lib:format("~p", [?rec_info(server_hello, ServerHello)]), + {Header, Message}; +parse_handshake(Direction, #certificate{} = Certificate) -> + Header = io_lib:format("~s Handshake, Certificate", + [header_prefix(Direction)]), + Message = io_lib:format("~p", [?rec_info(certificate, Certificate)]), + {Header, Message}; +parse_handshake(Direction, #server_key_exchange{} = ServerKeyExchange) -> + Header = io_lib:format("~s Handshake, ServerKeyExchange", + [header_prefix(Direction)]), + Message = io_lib:format("~p", [?rec_info(server_key_exchange, ServerKeyExchange)]), + {Header, Message}; +parse_handshake(Direction, #server_key_params{} = ServerKeyExchange) -> + Header = io_lib:format("~s Handshake, ServerKeyExchange", + [header_prefix(Direction)]), + Message = io_lib:format("~p", [?rec_info(server_key_params, ServerKeyExchange)]), + {Header, Message}; +parse_handshake(Direction, #certificate_request{} = CertificateRequest) -> + Header = io_lib:format("~s Handshake, CertificateRequest", + [header_prefix(Direction)]), + Message = io_lib:format("~p", [?rec_info(certificate_request, CertificateRequest)]), + {Header, Message}; +parse_handshake(Direction, #server_hello_done{} = ServerHelloDone) -> + Header = io_lib:format("~s Handshake, ServerHelloDone", + [header_prefix(Direction)]), + Message = io_lib:format("~p", [?rec_info(server_hello_done, ServerHelloDone)]), + {Header, Message}; +parse_handshake(Direction, #client_key_exchange{} = ClientKeyExchange) -> + Header = io_lib:format("~s Handshake, ClientKeyExchange", + [header_prefix(Direction)]), + Message = io_lib:format("~p", [?rec_info(client_key_exchange, ClientKeyExchange)]), + {Header, Message}; +parse_handshake(Direction, #certificate_verify{} = CertificateVerify) -> + Header = io_lib:format("~s Handshake, CertificateVerify", + [header_prefix(Direction)]), + Message = io_lib:format("~p", [?rec_info(certificate_verify, CertificateVerify)]), + {Header, Message}; +parse_handshake(Direction, #finished{} = Finished) -> + Header = io_lib:format("~s Handshake, Finished", + [header_prefix(Direction)]), + Message = io_lib:format("~p", [?rec_info(finished, Finished)]), + {Header, Message}; +parse_handshake(Direction, #hello_request{} = HelloRequest) -> + Header = io_lib:format("~s Handshake, HelloRequest", + [header_prefix(Direction)]), + Message = io_lib:format("~p", [?rec_info(hello_request, HelloRequest)]), + {Header, Message}. + + +version({3,3}) -> + "TLS 1.2"; +version({3,2}) -> + "TLS 1.1"; +version({3,1}) -> + "TLS 1.0"; +version({3,0}) -> + "SSL 3.0"; +version({M,N}) -> + io_lib:format("TLS [0x0~B0~B]", [M,N]). + + +header_prefix(inbound) -> + "<<<"; +header_prefix(outbound) -> + ">>>". + + +%%------------------------------------------------------------------------- +%% TLS Record Protocol +%%------------------------------------------------------------------------- +format_tls_record(Direction, BinMsg) -> + {Message, Size} = convert_to_hex('tls_record', BinMsg), + Header = io_lib:format("~s (~B bytes) ~s~n", + [header_prefix_tls_record(Direction), + Size, + tls_record_version(BinMsg)]), + Header ++ Message. + + +header_prefix_tls_record(inbound) -> + "reading"; +header_prefix_tls_record(outbound) -> + "writing". + + +tls_record_version([<<?BYTE(B),?BYTE(3),?BYTE(3),_/binary>>|_]) -> + io_lib:format("TLS 1.2 Record Protocol, ~s", [msg_type(B)]); +tls_record_version([<<?BYTE(B),?BYTE(3),?BYTE(2),_/binary>>|_]) -> + io_lib:format("TLS 1.1 Record Protocol, ~s", [msg_type(B)]); +tls_record_version([<<?BYTE(B),?BYTE(3),?BYTE(1),_/binary>>|_]) -> + io_lib:format("TLS 1.0 Record Protocol, ~s", [msg_type(B)]); +tls_record_version([<<?BYTE(B),?BYTE(3),?BYTE(0),_/binary>>|_]) -> + io_lib:format("SSL 3.0 Record Protocol, ~s", [msg_type(B)]); +tls_record_version([<<?BYTE(B),?BYTE(M),?BYTE(N),_/binary>>|_]) -> + io_lib:format("TLS [0x0~B0~B] Record Protocol, ~s", [M, N, msg_type(B)]). + + +msg_type(20) -> "change_cipher_spec"; +msg_type(21) -> "alert"; +msg_type(22) -> "handshake"; +msg_type(23) -> "application_data"; +msg_type(_) -> unknown. + + +%%------------------------------------------------------------------------- +%% Hex encoding functions +%%------------------------------------------------------------------------- +convert_to_hex(Protocol, BinMsg) -> + convert_to_hex(Protocol, BinMsg, [], [], 0). +%% +convert_to_hex(P, [], Row0, Acc, C) when C rem 16 =:= 0 -> + Row = lists:reverse(end_row(P, Row0)), + {lists:reverse(Acc) ++ Row ++ io_lib:nl(), C}; +convert_to_hex(P, [], Row0, Acc, C) -> + Row = lists:reverse(end_row(P, Row0)), + Padding = calculate_padding(Row0, Acc), + PaddedRow = string:pad(Row, Padding, leading, $ ), + {lists:reverse(Acc) ++ PaddedRow ++ io_lib:nl(), C}; +convert_to_hex(P, [H|T], Row, Acc, C) when is_list(H) -> + convert_to_hex(P, H ++ T, Row, Acc, C); +convert_to_hex(P, [<<>>|T], Row, Acc, C) -> + convert_to_hex(P, T, Row, Acc, C); + +%% First line +convert_to_hex(P, [<<A:4,B:4,R/binary>>|T], Row, Acc, C) when C =:= 0 -> + convert_to_hex(P, [<<R/binary>>|T], + update_row(<<A:4,B:4>>, Row), + prepend_first_row(P, A, B, Acc, C), + C + 1); +%% New line +convert_to_hex(P, [<<A:4,B:4,R/binary>>|T], Row, Acc, C) when C rem 16 =:= 0 -> + convert_to_hex(P, [<<R/binary>>|T], + update_row(<<A:4,B:4>>, []), + prepend_row(P, A, B, Row, Acc, C), + C + 1); +%% Add 8th hex with extra whitespace +%% 0000 - 16 03 02 00 bd 01 00 00 b9 ... +%% ^^^^ +convert_to_hex(P, [<<A:4,B:4,R/binary>>|T], Row, Acc, C) when C rem 8 =:= 7 -> + convert_to_hex(P, [<<R/binary>>|T], + update_row(<<A:4,B:4>>, Row), + prepend_eighths_hex(A, B, Acc), + C + 1); +convert_to_hex(P, [<<A:4,B:4,R/binary>>|T], Row, Acc, C) -> + convert_to_hex(P, [<<R/binary>>|T], + update_row(<<A:4,B:4>>, Row), + prepend_hex(A, B, Acc), + C + 1); +%% First line +convert_to_hex(P, [H|T], Row, Acc, C) when is_integer(H), C =:= 0 -> + convert_to_hex(P, T, + update_row(H, Row), + prepend_first_row(P, H, Acc, C), + C + 1); +%% New line +convert_to_hex(P, [H|T], Row, Acc, C) when is_integer(H), C rem 16 =:= 0 -> + convert_to_hex(P, T, + update_row(H, []), + prepend_row(P, H, Row, Acc, C), + C + 1); +%% Add 8th hex with extra whitespace +%% 0000 - 16 03 02 00 bd 01 00 00 b9 ... +%% ^^^^ +convert_to_hex(P, [H|T], Row, Acc, C) when is_integer(H), C rem 8 =:= 7 -> + convert_to_hex(P, T, + update_row(H, Row), + prepend_eighths_hex(H, Acc), + C + 1); +convert_to_hex(P, [H|T], Row, Acc, C) when is_integer(H) -> + convert_to_hex(P, T, + update_row(H, Row), + prepend_hex(H, Acc), + C + 1). + + +row_prefix(tls_record, N) -> + S = string:pad(string:to_lower(erlang:integer_to_list(N, 16)),4,leading,$0), + lists:reverse(lists:flatten(S ++ " - ")). + + +end_row(tls_record, Row) -> + Row ++ " ". + + +%% Calculate padding of the "printable character" lines in order to be +%% visually aligned. +calculate_padding(Row, Acc) -> + %% Number of new line characters + NNL = (length(Acc) div 75) * length(io_lib:nl()), + %% Length of the last printed line + Length = (length(Acc) - NNL) rem 75, + %% Adjusted length of the last printed line + PaddedLength = 75 - (16 - length(Row)), %% Length + %% Padding + PaddedLength - Length. + + +%%------------------------------------------------------------------------- +%% Functions operating on reversed lists +%%------------------------------------------------------------------------- +update_row(B, Row) when is_binary(B) -> + case binary_to_list(B) of + [C] when 32 =< C, C =< 126 -> + [C|Row]; + _Else -> + [$.|Row] + end; +update_row(C, Row) when 32 =< C, C =< 126 -> + [C|Row]; +update_row(_, Row) -> + [$.|Row]. + + +prepend_first_row(P, A, B, Acc, C) -> + prepend_hex(A, B,row_prefix(P, C) ++ Acc). +%% +prepend_first_row(P, N, Acc, C) -> + prepend_hex(N,row_prefix(P, C) ++ Acc). + +prepend_row(P, A, B, Row, Acc, C) -> + prepend_hex(A, B,row_prefix(P, C) ++ io_lib:nl() ++ end_row(P, Row) ++ Acc). +%% +prepend_row(P, N, Row, Acc, C) -> + prepend_hex(N,row_prefix(P, C) ++ io_lib:nl() ++ end_row(P, Row) ++ Acc). + + + +prepend_hex(A, B, Acc) -> + [$ ,?DEC2HEX(B),?DEC2HEX(A)|Acc]. +%% +prepend_hex(N, Acc) -> + " " ++ number_to_hex(N) ++ Acc. + + +prepend_eighths_hex(A, B, Acc) -> + [$ ,$ ,?DEC2HEX(B),?DEC2HEX(A)|Acc]. +%% +prepend_eighths_hex(N, Acc) -> + " " ++ number_to_hex(N) ++ Acc. + +number_to_hex(N) -> + case string:to_lower(erlang:integer_to_list(N, 16)) of + H when length(H) < 2 -> + lists:append(H, "0"); + H -> + lists:reverse(H) + end. diff --git a/lib/ssl/src/ssl_manager.erl b/lib/ssl/src/ssl_manager.erl index f44fe6a2bf..6ec65023e4 100644 --- a/lib/ssl/src/ssl_manager.erl +++ b/lib/ssl/src/ssl_manager.erl @@ -499,10 +499,10 @@ last_delay_timer({{_,_},_}, TRef, {LastServer, _}) -> last_delay_timer({_,_}, TRef, {_, LastClient}) -> {TRef, LastClient}. -%% If we can not generate a not allready in use session ID in +%% If we cannot generate a not allready in use session ID in %% ?GEN_UNIQUE_ID_MAX_TRIES we make the new session uncacheable The %% value of ?GEN_UNIQUE_ID_MAX_TRIES is stolen from open SSL which -%% states : "If we can not find a session id in +%% states : "If we cannot find a session id in %% ?GEN_UNIQUE_ID_MAX_TRIES either the RAND code is broken or someone %% is trying to open roughly very close to 2^128 (or 2^256) SSL %% sessions to our server" @@ -513,7 +513,7 @@ new_id(Port, Tries, Cache, CacheCb) -> case CacheCb:lookup(Cache, {Port, Id}) of undefined -> Now = erlang:monotonic_time(), - %% New sessions can not be set to resumable + %% New sessions cannot be set to resumable %% until handshake is compleate and the %% other session values are set. CacheCb:update(Cache, {Port, Id}, #session{session_id = Id, diff --git a/lib/ssl/src/ssl_pkix_db.erl b/lib/ssl/src/ssl_pkix_db.erl index 8828c3a0d8..4132733ae0 100644 --- a/lib/ssl/src/ssl_pkix_db.erl +++ b/lib/ssl/src/ssl_pkix_db.erl @@ -27,6 +27,7 @@ -include("ssl_internal.hrl"). -include_lib("public_key/include/public_key.hrl"). -include_lib("kernel/include/file.hrl"). +-include_lib("kernel/include/logger.hrl"). -export([create/1, create_pem_cache/1, add_crls/3, remove_crls/2, remove/1, add_trusted_certs/3, @@ -311,7 +312,7 @@ decode_certs(Ref, Cert) -> error:_ -> Report = io_lib:format("SSL WARNING: Ignoring a CA cert as " "it could not be correctly decoded.~n", []), - error_logger:info_report(Report), + ?LOG_NOTICE(Report), undefined end. diff --git a/lib/ssl/src/ssl_record.erl b/lib/ssl/src/ssl_record.erl index c0eee466ae..659e1485ac 100644 --- a/lib/ssl/src/ssl_record.erl +++ b/lib/ssl/src/ssl_record.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2013-2017. All Rights Reserved. +%% Copyright Ericsson AB 2013-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssl/src/ssl_record.hrl b/lib/ssl/src/ssl_record.hrl index ed007f58d7..e8ce50040f 100644 --- a/lib/ssl/src/ssl_record.hrl +++ b/lib/ssl/src/ssl_record.hrl @@ -74,7 +74,7 @@ -define(INITIAL_BYTES, 5). -define(MAX_SEQENCE_NUMBER, 18446744073709551615). %% (1 bsl 64) - 1 = 18446744073709551615 -%% Sequence numbers can not wrap so when max is about to be reached we should renegotiate. +%% Sequence numbers cannot wrap so when max is about to be reached we should renegotiate. %% We will renegotiate a little before so that there will be sequence numbers left %% for the rehandshake and a little data. Currently we decided to renegotiate a little more %% often as we can have a cheaper test to check if it is time to renegotiate. It will still diff --git a/lib/ssl/src/tls_connection.erl b/lib/ssl/src/tls_connection.erl index ef84c5320e..8320d3f7f3 100644 --- a/lib/ssl/src/tls_connection.erl +++ b/lib/ssl/src/tls_connection.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2017. All Rights Reserved. +%% Copyright Ericsson AB 2007-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -38,7 +38,8 @@ -include("ssl_api.hrl"). -include("ssl_internal.hrl"). -include("ssl_srp.hrl"). --include_lib("public_key/include/public_key.hrl"). +-include_lib("public_key/include/public_key.hrl"). +-include_lib("kernel/include/logger.hrl"). %% Internal application API @@ -111,12 +112,13 @@ start_link(Role, Host, Port, Socket, Options, User, CbInfo) -> init([Role, Host, Port, Socket, Options, User, CbInfo]) -> process_flag(trap_exit, true), - State0 = initial_state(Role, Host, Port, Socket, Options, User, CbInfo), + State0 = #state{protocol_specific = Map} = initial_state(Role, Host, Port, Socket, Options, User, CbInfo), try State = ssl_connection:ssl_config(State0#state.ssl_options, Role, State0), gen_statem:enter_loop(?MODULE, [], init, State) catch throw:Error -> - gen_statem:enter_loop(?MODULE, [], error, {Error, State0}) + EState = State0#state{protocol_specific = Map#{error => Error}}, + gen_statem:enter_loop(?MODULE, [], error, EState) end. %%==================================================================== %% State transition handling @@ -129,6 +131,7 @@ next_record(#state{protocol_buffers = = Buffers, connection_states = ConnStates0, ssl_options = #ssl_options{padding_check = Check}} = State) -> + case tls_record:decode_cipher_text(CT, ConnStates0, Check) of {Plain, ConnStates} -> {Plain, State#state{protocol_buffers = @@ -264,9 +267,19 @@ send_handshake(Handshake, State) -> queue_handshake(Handshake, #state{negotiated_version = Version, tls_handshake_history = Hist0, flight_buffer = Flight0, - connection_states = ConnectionStates0} = State0) -> + connection_states = ConnectionStates0, + ssl_options = SslOpts} = State0) -> {BinHandshake, ConnectionStates, Hist} = encode_handshake(Handshake, Version, ConnectionStates0, Hist0), + Report = #{direction => outbound, + protocol => 'tls_record', + message => BinHandshake}, + HandshakeMsg = #{direction => outbound, + protocol => 'handshake', + message => Handshake}, + ssl_logger:debug(SslOpts#ssl_options.log_level, HandshakeMsg, #{domain => [otp,ssl,handshake]}), + ssl_logger:debug(SslOpts#ssl_options.log_level, Report, #{domain => [otp,ssl,tls_record]}), + State0#state{connection_states = ConnectionStates, tls_handshake_history = Hist, flight_buffer = Flight0 ++ [BinHandshake]}. @@ -278,10 +291,15 @@ send_handshake_flight(#state{socket = Socket, {State0#state{flight_buffer = []}, []}. queue_change_cipher(Msg, #state{negotiated_version = Version, - flight_buffer = Flight0, - connection_states = ConnectionStates0} = State0) -> + flight_buffer = Flight0, + connection_states = ConnectionStates0, + ssl_options = SslOpts} = State0) -> {BinChangeCipher, ConnectionStates} = encode_change_cipher(Msg, Version, ConnectionStates0), + Report = #{direction => outbound, + protocol => 'tls_record', + message => BinChangeCipher}, + ssl_logger:debug(SslOpts#ssl_options.log_level, Report, #{domain => [otp,ssl,tls_record]}), State0#state{connection_states = ConnectionStates, flight_buffer = Flight0 ++ [BinChangeCipher]}. @@ -309,10 +327,16 @@ empty_connection_state(ConnectionEnd, BeastMitigation) -> send_alert(Alert, #state{negotiated_version = Version, socket = Socket, transport_cb = Transport, - connection_states = ConnectionStates0} = State0) -> + connection_states = ConnectionStates0, + ssl_options = SslOpts} = State0) -> {BinMsg, ConnectionStates} = encode_alert(Alert, Version, ConnectionStates0), + send(Transport, Socket, BinMsg), + Report = #{direction => outbound, + protocol => 'tls_record', + message => BinMsg}, + ssl_logger:debug(SslOpts#ssl_options.log_level, Report, #{domain => [otp,ssl,tls_record]}), State0#state{connection_states = ConnectionStates}. %%-------------------------------------------------------------------- @@ -415,6 +439,14 @@ init({call, From}, {start, Timeout}, {BinMsg, ConnectionStates, Handshake} = encode_handshake(Hello, HelloVersion, ConnectionStates0, Handshake0), send(Transport, Socket, BinMsg), + Report = #{direction => outbound, + protocol => 'tls_record', + message => BinMsg}, + HelloMsg = #{direction => outbound, + protocol => 'handshake', + message => Hello}, + ssl_logger:debug(SslOpts#ssl_options.log_level, HelloMsg, #{domain => [otp,ssl,handshake]}), + ssl_logger:debug(SslOpts#ssl_options.log_level, Report, #{domain => [otp,ssl,tls_record]}), State1 = State0#state{connection_states = ConnectionStates, negotiated_version = Version, %% Requested version session = @@ -432,17 +464,12 @@ init(Type, Event, State) -> {start, timeout()} | term(), #state{}) -> gen_statem:state_function_result(). %%-------------------------------------------------------------------- - -error({call, From}, {start, _Timeout}, {Error, State}) -> - ssl_connection:stop_and_reply( - normal, {reply, From, {error, Error}}, State); error({call, From}, {start, _Timeout}, #state{protocol_specific = #{error := Error}} = State) -> ssl_connection:stop_and_reply( normal, {reply, From, {error, Error}}, State); -error({call, _} = Call, Msg, {Error, #state{protocol_specific = Map} = State}) -> - gen_handshake(?FUNCTION_NAME, Call, Msg, - State#state{protocol_specific = Map#{error => Error}}); +error({call, _} = Call, Msg, State) -> + gen_handshake(?FUNCTION_NAME, Call, Msg, State); error(_, _, _) -> {keep_state_and_data, [postpone]}. @@ -651,11 +678,11 @@ initial_state(Role, Host, Port, Socket, {SSLOptions, SocketOptions, Tracker}, Us next_tls_record(Data, StateName, #state{protocol_buffers = #protocol_buffers{tls_record_buffer = Buf0, - tls_cipher_texts = CT0} = Buffers} - = State0) -> - case tls_record:get_tls_records(Data, + tls_cipher_texts = CT0} = Buffers, + ssl_options = SslOpts} = State0) -> + case tls_record:get_tls_records(Data, acceptable_record_versions(StateName, State0), - Buf0) of + Buf0, SslOpts) of {Records, Buf1} -> CT1 = CT0 ++ Records, next_record(State0#state{protocol_buffers = @@ -666,10 +693,18 @@ next_tls_record(Data, StateName, #state{protocol_buffers = end. +%% TLS 1.3 Client/Server +%% - Ignore TLSPlaintext.legacy_record_version +%% - Verify that TLSCiphertext.legacy_record_version is set to 0x0303 for all records +%% other than an initial ClientHello, where it MAY also be 0x0301. acceptable_record_versions(hello, _) -> - [tls_record:protocol_version(Vsn) || Vsn <- ?ALL_AVAILABLE_VERSIONS]; + [tls_record:protocol_version(Vsn) || Vsn <- ?ALL_TLS_RECORD_VERSIONS]; +acceptable_record_versions(_, #state{negotiated_version = {Major, Minor}}) + when Major > 3; Major =:= 3, Minor >= 4 -> + [{3, 3}]; acceptable_record_versions(_, #state{negotiated_version = Version}) -> [Version]. + handle_record_alert(Alert, _) -> Alert. diff --git a/lib/ssl/src/tls_handshake.erl b/lib/ssl/src/tls_handshake.erl index 0058b9c8ae..7004c7a2f4 100644 --- a/lib/ssl/src/tls_handshake.erl +++ b/lib/ssl/src/tls_handshake.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2017. All Rights Reserved. +%% Copyright Ericsson AB 2007-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -31,6 +31,7 @@ -include("ssl_internal.hrl"). -include("ssl_cipher.hrl"). -include_lib("public_key/include/public_key.hrl"). +-include_lib("kernel/include/logger.hrl"). %% Handshake handling -export([client_hello/8, hello/4]). @@ -60,6 +61,18 @@ client_hello(Host, Port, ConnectionStates, } = SslOpts, Cache, CacheCb, Renegotiation, OwnCert) -> Version = tls_record:highest_protocol_version(Versions), + + %% In TLS 1.3, the client indicates its version preferences in the + %% "supported_versions" extension (Section 4.2.1) and the + %% legacy_version field MUST be set to 0x0303, which is the version + %% number for TLS 1.2. + LegacyVersion = + case tls_record:is_higher(Version, {3,2}) of + true -> + {3,3}; + false -> + Version + end, #{security_parameters := SecParams} = ssl_record:pending_connection_state(ConnectionStates, read), AvailableCipherSuites = ssl_handshake:available_suites(UserSuites, Version), @@ -70,7 +83,7 @@ client_hello(Host, Port, ConnectionStates, CipherSuites = ssl_handshake:cipher_suites(AvailableCipherSuites, Renegotiation, Fallback), Id = ssl_session:client_id({Host, Port, SslOpts}, Cache, CacheCb, OwnCert), #client_hello{session_id = Id, - client_version = Version, + client_version = LegacyVersion, cipher_suites = CipherSuites, compression_methods = ssl_record:compressions(), random = SecParams#security_parameters.client_random, @@ -92,6 +105,65 @@ client_hello(Host, Port, ConnectionStates, %% %% Description: Handles a received hello message %%-------------------------------------------------------------------- + + +%% TLS 1.3 - Section 4.1.3 +%% TLS 1.3 clients receiving a ServerHello indicating TLS 1.2 or below +%% MUST check that the last eight bytes are not equal to either of these +%% values. +hello(#server_hello{server_version = {Major, Minor}, + random = <<_:24/binary,Down:8/binary>>}, + #ssl_options{versions = [{M,N}|_]}, _, _) + when (M > 3 orelse M =:= 3 andalso N >= 4) andalso %% TLS 1.3 client + (Major =:= 3 andalso Minor =:= 3 andalso %% Negotiating TLS 1.2 + Down =:= ?RANDOM_OVERRIDE_TLS12) orelse + + (M > 3 orelse M =:= 3 andalso N >= 4) andalso %% TLS 1.3 client + (Major =:= 3 andalso Minor < 3 andalso %% Negotiating TLS 1.1 or prior + Down =:= ?RANDOM_OVERRIDE_TLS11) -> + ?ALERT_REC(?FATAL, ?ILLEGAL_PARAMETER); + +%% TLS 1.2 clients SHOULD also check that the last eight bytes are not +%% equal to the second value if the ServerHello indicates TLS 1.1 or below. +hello(#server_hello{server_version = {Major, Minor}, + random = <<_:24/binary,Down:8/binary>>}, + #ssl_options{versions = [{M,N}|_]}, _, _) + when (M =:= 3 andalso N =:= 3) andalso %% TLS 1.2 client + (Major =:= 3 andalso Minor < 3 andalso %% Negotiating TLS 1.1 or prior + Down =:= ?RANDOM_OVERRIDE_TLS11) -> + ?ALERT_REC(?FATAL, ?ILLEGAL_PARAMETER); + + +%% TLS 1.3 - 4.2.1. Supported Versions +%% If the "supported_versions" extension in the ServerHello contains a +%% version not offered by the client or contains a version prior to TLS +%% 1.3, the client MUST abort the handshake with an "illegal_parameter" +%% alert. +%%-------------------------------------------------------------------- +%% TLS 1.2 Client +%% +%% - If "supported_version" is present (ServerHello): +%% - Abort handshake with an "illegal_parameter" alert +hello(#server_hello{server_version = Version, + extensions = #hello_extensions{ + server_hello_selected_version = + #server_hello_selected_version{selected_version = Version} + }}, + #ssl_options{versions = SupportedVersions}, + _ConnectionStates0, _Renegotiation) -> + case tls_record:is_higher({3,4}, Version) of + true -> + ?ALERT_REC(?FATAL, ?ILLEGAL_PARAMETER); + false -> + case tls_record:is_acceptable_version(Version, SupportedVersions) of + true -> + %% Implement TLS 1.3 statem ??? + ?ALERT_REC(?FATAL, ?PROTOCOL_VERSION); + false -> + ?ALERT_REC(?FATAL, ?ILLEGAL_PARAMETER) + end + end; + hello(#server_hello{server_version = Version, random = Random, cipher_suite = CipherSuite, compression_method = Compression, @@ -106,6 +178,37 @@ hello(#server_hello{server_version = Version, random = Random, false -> ?ALERT_REC(?FATAL, ?PROTOCOL_VERSION) end; + + +%% TLS 1.2 Server +%% - If "supported_versions" is present (ClientHello): +%% - Select version from "supported_versions" (ignore ClientHello.legacy_version) +%% - If server only supports versions greater than "supported_versions": +%% - Abort handshake with a "protocol_version" alert (*) +%% - If "supported_versions" is absent (ClientHello): +%% - Negotiate the minimum of ClientHello.legacy_version and TLS 1.2 (**) +%% - If server only supports versions greater than ClientHello.legacy_version: +%% - Abort handshake with a "protocol_version" alert +%% +%% (*) Sends alert even if there is a gap in supported versions +%% e.g. Server 1.0,1.2 Client 1.1,1.3 +%% (**) Current implementation can negotiate a version not supported by the client +%% e.g. Server 1.0,1.2 Client 1.1 -> ServerHello 1.0 +hello(#client_hello{client_version = _ClientVersion, + cipher_suites = CipherSuites, + extensions = #hello_extensions{ + client_hello_versions = + #client_hello_versions{versions = ClientVersions} + }} = Hello, + #ssl_options{versions = Versions} = SslOpts, + Info, Renegotiation) -> + try + Version = ssl_handshake:select_supported_version(ClientVersions, Versions), + do_hello(Version, Versions, CipherSuites, Hello, SslOpts, Info, Renegotiation) + catch + _:_ -> + ?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE, malformed_handshake_data) + end; hello(#client_hello{client_version = ClientVersion, cipher_suites = CipherSuites} = Hello, @@ -113,18 +216,7 @@ hello(#client_hello{client_version = ClientVersion, Info, Renegotiation) -> try Version = ssl_handshake:select_version(tls_record, ClientVersion, Versions), - case ssl_cipher:is_fallback(CipherSuites) of - true -> - Highest = tls_record:highest_protocol_version(Versions), - case tls_record:is_higher(Highest, Version) of - true -> - ?ALERT_REC(?FATAL, ?INAPPROPRIATE_FALLBACK); - false -> - handle_client_hello(Version, Hello, SslOpts, Info, Renegotiation) - end; - false -> - handle_client_hello(Version, Hello, SslOpts, Info, Renegotiation) - end + do_hello(Version, Versions, CipherSuites, Hello, SslOpts, Info, Renegotiation) catch _:_ -> ?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE, malformed_handshake_data) @@ -241,6 +333,31 @@ handle_server_hello_extensions(Version, SessionId, Random, CipherSuite, {ConnectionStates, ProtoExt, Protocol} -> {Version, SessionId, ConnectionStates, ProtoExt, Protocol} end. + + +do_hello(undefined, _Versions, _CipherSuites, _Hello, _SslOpts, _Info, _Renegotiation) -> + ?ALERT_REC(?FATAL, ?PROTOCOL_VERSION); +do_hello(Version, Versions, CipherSuites, Hello, SslOpts, Info, Renegotiation) -> + case tls_record:is_higher({3,4}, Version) of + true -> %% TLS 1.2 and older + case ssl_cipher:is_fallback(CipherSuites) of + true -> + Highest = tls_record:highest_protocol_version(Versions), + case tls_record:is_higher(Highest, Version) of + true -> + ?ALERT_REC(?FATAL, ?INAPPROPRIATE_FALLBACK); + false -> + handle_client_hello(Version, Hello, SslOpts, Info, Renegotiation) + end; + false -> + handle_client_hello(Version, Hello, SslOpts, Info, Renegotiation) + end; + false -> + %% Implement TLS 1.3 statem ??? + ?ALERT_REC(?FATAL, ?PROTOCOL_VERSION) + end. + + %%-------------------------------------------------------------------- enc_handshake(#hello_request{}, _Version) -> {?HELLO_REQUEST, <<>>}; @@ -272,6 +389,10 @@ get_tls_handshake_aux(Version, <<?BYTE(Type), ?UINT24(Length), Raw = <<?BYTE(Type), ?UINT24(Length), Body/binary>>, try decode_handshake(Version, Type, Body) of Handshake -> + Report = #{direction => inbound, + protocol => 'handshake', + message => Handshake}, + ssl_logger:debug(Opts#ssl_options.log_level, Report, #{domain => [otp,ssl,handshake]}), get_tls_handshake_aux(Version, Rest, Opts, [{Handshake,Raw} | Acc]) catch _:_ -> diff --git a/lib/ssl/src/tls_record.erl b/lib/ssl/src/tls_record.erl index aa70508f1e..444759aafa 100644 --- a/lib/ssl/src/tls_record.erl +++ b/lib/ssl/src/tls_record.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2017. All Rights Reserved. +%% Copyright Ericsson AB 2007-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -30,9 +30,10 @@ -include("ssl_alert.hrl"). -include("tls_handshake.hrl"). -include("ssl_cipher.hrl"). +-include_lib("kernel/include/logger.hrl"). %% Handling of incoming data --export([get_tls_records/3, init_connection_states/2]). +-export([get_tls_records/4, init_connection_states/2]). %% Encoding TLS records -export([encode_handshake/3, encode_alert_record/3, @@ -75,24 +76,24 @@ init_connection_states(Role, BeastMitigation) -> pending_write => Pending}. %%-------------------------------------------------------------------- --spec get_tls_records(binary(), [tls_version()], binary()) -> {[binary()], binary()} | #alert{}. +-spec get_tls_records(binary(), [tls_version()], binary(), ssl_options()) -> {[binary()], binary()} | #alert{}. %% %% and returns it as a list of tls_compressed binaries also returns leftover %% Description: Given old buffer and new data from TCP, packs up a records %% data %%-------------------------------------------------------------------- -get_tls_records(Data, Versions, Buffer) -> +get_tls_records(Data, Versions, Buffer, SslOpts) -> BinData = list_to_binary([Buffer, Data]), case erlang:byte_size(BinData) of N when N >= 3 -> case assert_version(BinData, Versions) of true -> - get_tls_records_aux(BinData, []); + get_tls_records_aux(BinData, [], SslOpts); false -> ?ALERT_REC(?FATAL, ?BAD_RECORD_MAC) end; _ -> - get_tls_records_aux(BinData, []) + get_tls_records_aux(BinData, [], SslOpts) end. %%==================================================================== @@ -229,6 +230,8 @@ decode_cipher_text(#ssl_tls{type = Type, version = Version, %% Description: Creates a protocol version record from a version atom %% or vice versa. %%-------------------------------------------------------------------- +protocol_version('tlsv1.3') -> + {3, 4}; protocol_version('tlsv1.2') -> {3, 3}; protocol_version('tlsv1.1') -> @@ -239,6 +242,8 @@ protocol_version(sslv3) -> {3, 0}; protocol_version(sslv2) -> %% Backwards compatibility {2, 0}; +protocol_version({3, 4}) -> + 'tlsv1.3'; protocol_version({3, 3}) -> 'tlsv1.2'; protocol_version({3, 2}) -> @@ -399,33 +404,61 @@ assert_version(<<?BYTE(_), ?BYTE(MajVer), ?BYTE(MinVer), _/binary>>, Versions) - get_tls_records_aux(<<?BYTE(?APPLICATION_DATA),?BYTE(MajVer),?BYTE(MinVer), ?UINT16(Length), Data:Length/binary, Rest/binary>>, - Acc) -> + Acc, SslOpts) -> + RawTLSRecord = <<?BYTE(?APPLICATION_DATA),?BYTE(MajVer),?BYTE(MinVer), + ?UINT16(Length), Data:Length/binary>>, + Report = #{direction => inbound, + protocol => 'tls_record', + message => [RawTLSRecord]}, + ssl_logger:debug(SslOpts#ssl_options.log_level, Report, #{domain => [otp,ssl,tls_record]}), get_tls_records_aux(Rest, [#ssl_tls{type = ?APPLICATION_DATA, version = {MajVer, MinVer}, - fragment = Data} | Acc]); + fragment = Data} | Acc], + SslOpts); get_tls_records_aux(<<?BYTE(?HANDSHAKE),?BYTE(MajVer),?BYTE(MinVer), - ?UINT16(Length), - Data:Length/binary, Rest/binary>>, Acc) -> + ?UINT16(Length), + Data:Length/binary, Rest/binary>>, Acc, SslOpts) -> + RawTLSRecord = <<?BYTE(?HANDSHAKE),?BYTE(MajVer),?BYTE(MinVer), + ?UINT16(Length), Data:Length/binary>>, + Report = #{direction => inbound, + protocol => 'tls_record', + message => [RawTLSRecord]}, + ssl_logger:debug(SslOpts#ssl_options.log_level, Report, #{domain => [otp,ssl,tls_record]}), get_tls_records_aux(Rest, [#ssl_tls{type = ?HANDSHAKE, version = {MajVer, MinVer}, - fragment = Data} | Acc]); + fragment = Data} | Acc], + SslOpts); get_tls_records_aux(<<?BYTE(?ALERT),?BYTE(MajVer),?BYTE(MinVer), ?UINT16(Length), Data:Length/binary, - Rest/binary>>, Acc) -> + Rest/binary>>, Acc, SslOpts) -> + RawTLSRecord = <<?BYTE(?ALERT),?BYTE(MajVer),?BYTE(MinVer), + ?UINT16(Length), Data:Length/binary>>, + Report = #{direction => inbound, + protocol => 'tls_record', + message => [RawTLSRecord]}, + ssl_logger:debug(SslOpts#ssl_options.log_level, Report, #{domain => [otp,ssl,tls_record]}), get_tls_records_aux(Rest, [#ssl_tls{type = ?ALERT, version = {MajVer, MinVer}, - fragment = Data} | Acc]); + fragment = Data} | Acc], + SslOpts); get_tls_records_aux(<<?BYTE(?CHANGE_CIPHER_SPEC),?BYTE(MajVer),?BYTE(MinVer), ?UINT16(Length), Data:Length/binary, Rest/binary>>, - Acc) -> + Acc, SslOpts) -> + RawTLSRecord = <<?BYTE(?CHANGE_CIPHER_SPEC),?BYTE(MajVer),?BYTE(MinVer), + ?UINT16(Length), Data:Length/binary>>, + Report = #{direction => inbound, + protocol => 'tls_record', + message => [RawTLSRecord]}, + ssl_logger:debug(SslOpts#ssl_options.log_level, Report, #{domain => [otp,ssl,tls_record]}), get_tls_records_aux(Rest, [#ssl_tls{type = ?CHANGE_CIPHER_SPEC, version = {MajVer, MinVer}, - fragment = Data} | Acc]); + fragment = Data} | Acc], + SslOpts); get_tls_records_aux(<<0:1, _CT:7, ?BYTE(_MajVer), ?BYTE(_MinVer), ?UINT16(Length), _/binary>>, - _Acc) when Length > ?MAX_CIPHER_TEXT_LENGTH -> + _Acc, _SslOpts) when Length > ?MAX_CIPHER_TEXT_LENGTH -> ?ALERT_REC(?FATAL, ?RECORD_OVERFLOW); -get_tls_records_aux(Data, Acc) -> +get_tls_records_aux(Data, Acc, _SslOpts) -> case size(Data) =< ?MAX_CIPHER_TEXT_LENGTH + ?INITIAL_BYTES of true -> {lists:reverse(Acc), Data}; diff --git a/lib/ssl/src/tls_socket.erl b/lib/ssl/src/tls_socket.erl index 453a908401..154281f1c2 100644 --- a/lib/ssl/src/tls_socket.erl +++ b/lib/ssl/src/tls_socket.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1998-2016. All Rights Reserved. +%% Copyright Ericsson AB 1998-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssl/src/tls_v1.erl b/lib/ssl/src/tls_v1.erl index a31ab8d044..6ef6040761 100644 --- a/lib/ssl/src/tls_v1.erl +++ b/lib/ssl/src/tls_v1.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2017. All Rights Reserved. +%% Copyright Ericsson AB 2007-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -74,7 +74,7 @@ finished(Role, Version, PrfAlgo, MasterSecret, Handshake) prf(?MD5SHA, MasterSecret, finished_label(Role), [MD5, SHA], 12); finished(Role, Version, PrfAlgo, MasterSecret, Handshake) - when Version == 3 -> + when Version == 3; Version == 4 -> %% RFC 5246 - 7.4.9. Finished %% struct { %% opaque verify_data[12]; @@ -85,6 +85,7 @@ finished(Role, Version, PrfAlgo, MasterSecret, Handshake) Hash = crypto:hash(mac_algo(PrfAlgo), Handshake), prf(PrfAlgo, MasterSecret, finished_label(Role), Hash, 12). + -spec certificate_verify(md5sha | sha, integer(), [binary()]) -> binary(). certificate_verify(md5sha, _Version, Handshake) -> @@ -154,7 +155,7 @@ setup_keys(Version, _PrfAlgo, MasterSecret, ServerRandom, ClientRandom, HashSize %% TLS v1.2 setup_keys(Version, PrfAlgo, MasterSecret, ServerRandom, ClientRandom, HashSize, KeyMatLen, IVSize) - when Version == 3 -> + when Version == 3; Version == 4 -> %% RFC 5246 - 6.3. Key calculation %% key_block = PRF(SecurityParameters.master_secret, %% "key expansion", @@ -192,7 +193,7 @@ mac_hash(Method, Mac_write_secret, Seq_num, Type, {Major, Minor}, Fragment]), Mac. --spec suites(1|2|3) -> [ssl_cipher:cipher_suite()]. +-spec suites(1|2|3|4) -> [ssl_cipher:cipher_suite()]. suites(Minor) when Minor == 1; Minor == 2 -> [ @@ -244,8 +245,15 @@ suites(3) -> %% ?TLS_DH_DSS_WITH_AES_256_GCM_SHA384, %% ?TLS_DH_RSA_WITH_AES_128_GCM_SHA256, %% ?TLS_DH_DSS_WITH_AES_128_GCM_SHA256 - ] ++ suites(2). + ] ++ suites(2); + + +suites(4) -> + suites(3). + +signature_algs({3, 4}, HashSigns) -> + signature_algs({3, 3}, HashSigns); signature_algs({3, 3}, HashSigns) -> CryptoSupports = crypto:supports(), Hashes = proplists:get_value(hashs, CryptoSupports), @@ -273,6 +281,8 @@ signature_algs({3, 3}, HashSigns) -> end, [], HashSigns), lists:reverse(Supported). +default_signature_algs({3, 4}) -> + default_signature_algs({3, 3}); default_signature_algs({3, 3} = Version) -> Default = [%% SHA2 {sha512, ecdsa}, diff --git a/lib/ssl/test/Makefile b/lib/ssl/test/Makefile index 845f5bee2e..9dfb2eba53 100644 --- a/lib/ssl/test/Makefile +++ b/lib/ssl/test/Makefile @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 1999-2017. All Rights Reserved. +# Copyright Ericsson AB 1999-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. diff --git a/lib/ssl/test/ssl_ECC_SUITE.erl b/lib/ssl/test/ssl_ECC_SUITE.erl index 6e2d86571a..c93f066825 100644 --- a/lib/ssl/test/ssl_ECC_SUITE.erl +++ b/lib/ssl/test/ssl_ECC_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2017. All Rights Reserved. +%% Copyright Ericsson AB 2007-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -43,10 +43,10 @@ all() -> groups() -> [ - {'tlsv1.2', [], test_cases()}, + {'tlsv1.2', [], [mix_sign | test_cases()]}, {'tlsv1.1', [], test_cases()}, {'tlsv1', [], test_cases()}, - {'dtlsv1.2', [], test_cases()}, + {'dtlsv1.2', [], [mix_sign | test_cases()]}, {'dtlsv1', [], test_cases()} ]. @@ -288,22 +288,22 @@ client_ecdh_rsa_server_ecdhe_ecdsa_server_custom(Config) -> client_ecdh_rsa_server_ecdhe_rsa_server_custom(Config) -> Default = ssl_test_lib:default_cert_chain_conf(), {COpts0, SOpts0} = ssl_test_lib:make_ec_cert_chains([{server_chain, Default}, - {client_chain, Default}], - ecdh_rsa, ecdhe_rsa, Config), + {client_chain, Default}], + ecdh_rsa, ecdhe_rsa, Config), COpts = ssl_test_lib:ssl_options(COpts0, Config), SOpts = ssl_test_lib:ssl_options(SOpts0, Config), ECCOpts = [{honor_ecc_order, true}, {eccs, [secp256r1, sect571r1]}], case ssl_test_lib:supported_eccs(ECCOpts) of - true -> ssl_test_lib:ecc_test(undefined, COpts, SOpts, [], ECCOpts, Config); + true -> ssl_test_lib:ecc_test(secp256r1, COpts, SOpts, [], ECCOpts, Config); false -> {skip, "unsupported named curves"} end. client_ecdhe_rsa_server_ecdhe_ecdsa_server_custom(Config) -> Default = ssl_test_lib:default_cert_chain_conf(), - {COpts0, SOpts0} = ssl_test_lib:make_ec_cert_chains([{server_chain, Default}, - {client_chain, Default}], - ecdhe_rsa, ecdhe_ecdsa, Config), + {COpts0, SOpts0} = ssl_test_lib:make_ec_cert_chains([{server_chain, Default}, + {client_chain, Default}], + ecdhe_rsa, ecdhe_ecdsa, Config), COpts = ssl_test_lib:ssl_options(COpts0, Config), SOpts = ssl_test_lib:ssl_options(SOpts0, Config), ECCOpts = [{honor_ecc_order, true}, {eccs, [secp256r1, sect571r1]}], @@ -315,29 +315,30 @@ client_ecdhe_rsa_server_ecdhe_ecdsa_server_custom(Config) -> client_ecdhe_rsa_server_ecdhe_rsa_server_custom(Config) -> Default = ssl_test_lib:default_cert_chain_conf(), {COpts0, SOpts0} = ssl_test_lib:make_ec_cert_chains([{server_chain, Default}, - {client_chain, Default}], - ecdhe_rsa, ecdhe_rsa, Config), + {client_chain, Default}], + ecdhe_rsa, ecdhe_rsa, Config), COpts = ssl_test_lib:ssl_options(COpts0, Config), SOpts = ssl_test_lib:ssl_options(SOpts0, Config), ECCOpts = [{honor_ecc_order, true}, {eccs, [secp256r1, sect571r1]}], case ssl_test_lib:supported_eccs(ECCOpts) of - true -> ssl_test_lib:ecc_test(undefined, COpts, SOpts, [], ECCOpts, Config); + true -> ssl_test_lib:ecc_test(secp256r1, COpts, SOpts, [], ECCOpts, Config); false -> {skip, "unsupported named curves"} end. client_ecdhe_rsa_server_ecdh_rsa_server_custom(Config) -> Default = ssl_test_lib:default_cert_chain_conf(), Ext = x509_test:extensions([{key_usage, [keyEncipherment]}]), {COpts0, SOpts0} = ssl_test_lib:make_ec_cert_chains([{server_chain, [[], [], [{extensions, Ext}]]}, - {client_chain, Default}], - ecdhe_rsa, ecdh_rsa, Config), + {client_chain, Default}], + ecdhe_rsa, ecdh_rsa, Config), COpts = ssl_test_lib:ssl_options(COpts0, Config), SOpts = ssl_test_lib:ssl_options(SOpts0, Config), ECCOpts = [{honor_ecc_order, true}, {eccs, [secp256r1, sect571r1]}], - + Expected = pubkey_cert_records:namedCurves(hd(tls_v1:ecc_curves(0))), %% The certificate curve + case ssl_test_lib:supported_eccs(ECCOpts) of - true -> ssl_test_lib:ecc_test(undefined, COpts, SOpts, [], ECCOpts, Config); + true -> ssl_test_lib:ecc_test(Expected, COpts, SOpts, [], ECCOpts, Config); false -> {skip, "unsupported named curves"} end. @@ -345,7 +346,7 @@ client_ecdhe_ecdsa_server_ecdhe_ecdsa_server_custom(Config) -> Default = ssl_test_lib:default_cert_chain_conf(), {COpts0, SOpts0} = ssl_test_lib:make_ec_cert_chains([{server_chain, Default}, {client_chain, Default}], - ecdhe_ecdsa, ecdhe_ecdsa, Config), + ecdhe_ecdsa, ecdhe_ecdsa, Config), COpts = ssl_test_lib:ssl_options(COpts0, Config), SOpts = ssl_test_lib:ssl_options(SOpts0, Config), ECCOpts = [{honor_ecc_order, true}, {eccs, [secp256r1, sect571r1]}], @@ -357,13 +358,13 @@ client_ecdhe_ecdsa_server_ecdhe_ecdsa_server_custom(Config) -> client_ecdhe_ecdsa_server_ecdhe_rsa_server_custom(Config) -> Default = ssl_test_lib:default_cert_chain_conf(), {COpts0, SOpts0} = ssl_test_lib:make_ec_cert_chains([{server_chain, Default}, - {client_chain, Default}], - ecdhe_ecdsa, ecdhe_rsa, Config), + {client_chain, Default}], + ecdhe_ecdsa, ecdhe_rsa, Config), COpts = ssl_test_lib:ssl_options(COpts0, Config), SOpts = ssl_test_lib:ssl_options(SOpts0, Config), ECCOpts = [{honor_ecc_order, true}, {eccs, [secp256r1, sect571r1]}], case ssl_test_lib:supported_eccs(ECCOpts) of - true -> ssl_test_lib:ecc_test(undefined, COpts, SOpts, [], ECCOpts, Config); + true -> ssl_test_lib:ecc_test(secp256r1, COpts, SOpts, [], ECCOpts, Config); false -> {skip, "unsupported named curves"} end. @@ -371,7 +372,7 @@ client_ecdhe_ecdsa_server_ecdhe_ecdsa_client_custom(Config) -> Default = ssl_test_lib:default_cert_chain_conf(), {COpts0, SOpts0} = ssl_test_lib:make_ec_cert_chains([{server_chain, Default}, {client_chain, Default}], - ecdhe_ecdsa, ecdhe_ecdsa, Config), + ecdhe_ecdsa, ecdhe_ecdsa, Config), COpts = ssl_test_lib:ssl_options(COpts0, Config), SOpts = ssl_test_lib:ssl_options(SOpts0, Config), ECCOpts = [{eccs, [secp256r1, sect571r1]}], @@ -383,8 +384,8 @@ client_ecdhe_ecdsa_server_ecdhe_ecdsa_client_custom(Config) -> client_ecdhe_rsa_server_ecdhe_ecdsa_client_custom(Config) -> Default = ssl_test_lib:default_cert_chain_conf(), {COpts0, SOpts0} = ssl_test_lib:make_ec_cert_chains([{server_chain, Default}, - {client_chain, Default}], - ecdhe_rsa, ecdhe_ecdsa, Config), + {client_chain, Default}], + ecdhe_rsa, ecdhe_ecdsa, Config), COpts = ssl_test_lib:ssl_options(COpts0, Config), SOpts = ssl_test_lib:ssl_options(SOpts0, Config), ECCOpts = [{eccs, [secp256r1, sect571r1]}], @@ -392,3 +393,12 @@ client_ecdhe_rsa_server_ecdhe_ecdsa_client_custom(Config) -> true -> ssl_test_lib:ecc_test(secp256r1, COpts, SOpts, ECCOpts, [], Config); false -> {skip, "unsupported named curves"} end. + +mix_sign(Config) -> + {COpts0, SOpts0} = ssl_test_lib:make_mix_cert(Config), + COpts = ssl_test_lib:ssl_options(COpts0, Config), + SOpts = ssl_test_lib:ssl_options(SOpts0, Config), + ECDHE_ECDSA = + ssl:filter_cipher_suites(ssl:cipher_suites(default, 'tlsv1.2'), + [{key_exchange, fun(ecdhe_ecdsa) -> true; (_) -> false end}]), + ssl_test_lib:basic_test(COpts, [{ciphers, ECDHE_ECDSA} | SOpts], Config). diff --git a/lib/ssl/test/ssl_ECC_openssl_SUITE.erl b/lib/ssl/test/ssl_ECC_openssl_SUITE.erl index 5a08b152a6..81a7dfd2da 100644 --- a/lib/ssl/test/ssl_ECC_openssl_SUITE.erl +++ b/lib/ssl/test/ssl_ECC_openssl_SUITE.erl @@ -57,13 +57,13 @@ all_groups() -> groups() -> case ssl_test_lib:openssl_sane_dtls() of true -> - [{'tlsv1.2', [], test_cases()}, + [{'tlsv1.2', [], [mix_sign | test_cases()]}, {'tlsv1.1', [], test_cases()}, {'tlsv1', [], test_cases()}, - {'dtlsv1.2', [], test_cases()}, + {'dtlsv1.2', [], [mix_sign | test_cases()]}, {'dtlsv1', [], test_cases()}]; false -> - [{'tlsv1.2', [], test_cases()}, + [{'tlsv1.2', [], [mix_sign | test_cases()]}, {'tlsv1.1', [], test_cases()}, {'tlsv1', [], test_cases()}] end. @@ -202,6 +202,17 @@ client_ecdh_ecdsa_server_ecdhe_ecdsa(Config) when is_list(Config) -> ssl_ECC:client_ecdh_ecdsa_server_ecdhe_ecdsa(Config). client_ecdhe_ecdsa_server_ecdhe_ecdsa(Config) when is_list(Config) -> ssl_ECC:client_ecdhe_ecdsa_server_ecdhe_ecdsa(Config). + +mix_sign(Config) -> + {COpts0, SOpts0} = ssl_test_lib:make_mix_cert(Config), + COpts = ssl_test_lib:ssl_options(COpts0, Config), + SOpts = ssl_test_lib:ssl_options(SOpts0, Config), + ECDHE_ECDSA = + ssl:filter_cipher_suites(ssl:cipher_suites(default, 'tlsv1.2'), + [{key_exchange, fun(ecdhe_ecdsa) -> true; (_) -> false end}]), + ssl_test_lib:basic_test(COpts, [{ciphers, ECDHE_ECDSA} | SOpts], [{client_type, erlang}, + {server_type, openssl} | Config]). + %%-------------------------------------------------------------------- %% Internal functions ------------------------------------------------ %%-------------------------------------------------------------------- diff --git a/lib/ssl/test/ssl_alpn_handshake_SUITE.erl b/lib/ssl/test/ssl_alpn_handshake_SUITE.erl index 055f05a900..27062d4801 100644 --- a/lib/ssl/test/ssl_alpn_handshake_SUITE.erl +++ b/lib/ssl/test/ssl_alpn_handshake_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2016. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssl/test/ssl_basic_SUITE.erl b/lib/ssl/test/ssl_basic_SUITE.erl index 162c63850f..480988b6e4 100644 --- a/lib/ssl/test/ssl_basic_SUITE.erl +++ b/lib/ssl/test/ssl_basic_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2017. All Rights Reserved. +%% Copyright Ericsson AB 2007-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -108,7 +108,8 @@ basic_tests() -> clear_pem_cache, defaults, fallback, - cipher_format + cipher_format, + suite_to_str ]. basic_tests_tls() -> @@ -146,8 +147,7 @@ options_tests_tls() -> tls_tcp_reuseaddr]. api_tests() -> - [connection_info, - secret_connection_info, + [secret_connection_info, connection_information, peercert, peercert_with_client_cert, @@ -242,7 +242,9 @@ error_handling_tests()-> [close_transport_accept, recv_active, recv_active_once, - recv_error_handling + recv_error_handling, + call_in_error_state, + close_in_error_state ]. error_handling_tests_tls()-> @@ -475,6 +477,8 @@ init_per_testcase(TestCase, Config) when TestCase == tls_ssl_accept_timeout; TestCase == tls_client_closes_socket; TestCase == tls_closed_in_active_once; TestCase == tls_downgrade -> + ssl:stop(), + ssl:start(), ssl_test_lib:ct_log_supported_protocol_versions(Config), ct:timetrap({seconds, 15}), Config; @@ -609,7 +613,16 @@ new_options_in_accept(Config) when is_list(Config) -> [_ , _ | ServerSslOpts] = ssl_test_lib:ssl_options(server_opts, Config), %% Remove non ssl opts {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), Version = ssl_test_lib:protocol_options(Config, [{tls, sslv3}, {dtls, dtlsv1}]), - Cipher = ssl_test_lib:protocol_options(Config, [{tls, {rsa,rc4_128,sha}}, {dtls, {rsa,aes_128_cbc,sha}}]), + Cipher = ssl_test_lib:protocol_options(Config, [{tls, #{key_exchange =>rsa, + cipher => rc4_128, + mac => sha, + prf => default_prf + }}, + {dtls, #{key_exchange =>rsa, + cipher => aes_128_cbc, + mac => sha, + prf => default_prf + }}]), Server = ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, {from, self()}, {ssl_extra_opts, [{versions, [Version]}, @@ -738,41 +751,6 @@ prf(Config) when is_list(Config) -> %%-------------------------------------------------------------------- -connection_info() -> - [{doc,"Test the API function ssl:connection_information/2"}]. -connection_info(Config) when is_list(Config) -> - ClientOpts = ssl_test_lib:ssl_options(client_verification_opts, Config), - ServerOpts = ssl_test_lib:ssl_options(server_verification_opts, Config), - {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), - - Server = ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, - {from, self()}, - {mfa, {?MODULE, connection_info_result, []}}, - {options, ServerOpts}]), - - Port = ssl_test_lib:inet_port(Server), - Client = ssl_test_lib:start_client([{node, ClientNode}, {port, Port}, - {host, Hostname}, - {from, self()}, - {mfa, {?MODULE, connection_info_result, []}}, - {options, - [{ciphers,[{dhe_rsa, aes_128_cbc, sha}]} | - ClientOpts]}]), - - ct:log("Testcase ~p, Client ~p Server ~p ~n", - [self(), Client, Server]), - - Version = ssl_test_lib:protocol_version(Config), - - ServerMsg = ClientMsg = {ok, {Version, {dhe_rsa, aes_128_cbc, sha}}}, - - ssl_test_lib:check_result(Server, ServerMsg, Client, ClientMsg), - - ssl_test_lib:close(Server), - ssl_test_lib:close(Client). - -%%-------------------------------------------------------------------- - secret_connection_info() -> [{doc,"Test the API function ssl:connection_information/2"}]. secret_connection_info(Config) when is_list(Config) -> @@ -1227,7 +1205,27 @@ cipher_format(Config) when is_list(Config) -> ssl:close(Socket1), {ok, Socket2} = ssl:listen(0, [{ciphers, ssl:cipher_suites(openssl)}]), ssl:close(Socket2). - + +%%-------------------------------------------------------------------- +suite_to_str() -> + [{doc, "Test that the suite_to_str API works"}]. +suite_to_str(Config) when is_list(Config) -> + "TLS_EMPTY_RENEGOTIATION_INFO_SCSV" = + ssl:suite_to_str(#{key_exchange => null, + cipher => null, + mac => null, + prf => null}), + "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256" = + ssl:suite_to_str(#{key_exchange => ecdhe_ecdsa, + cipher => aes_128_gcm, + mac => aead, + prf => sha256}), + "TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256" = + ssl:suite_to_str(#{key_exchange => ecdh_rsa, + cipher => aes_128_cbc, + mac => sha256, + prf => sha256}). + %%-------------------------------------------------------------------- peername() -> @@ -3454,16 +3452,50 @@ tls_tcp_reuseaddr(Config) when is_list(Config) -> honor_server_cipher_order() -> [{doc,"Test API honor server cipher order."}]. honor_server_cipher_order(Config) when is_list(Config) -> - ClientCiphers = [{dhe_rsa, aes_128_cbc, sha}, {dhe_rsa, aes_256_cbc, sha}], - ServerCiphers = [{dhe_rsa, aes_256_cbc, sha}, {dhe_rsa, aes_128_cbc, sha}], -honor_cipher_order(Config, true, ServerCiphers, ClientCiphers, {dhe_rsa, aes_256_cbc, sha}). + ClientCiphers = [#{key_exchange => dhe_rsa, + cipher => aes_128_cbc, + mac => sha, + prf => default_prf}, + #{key_exchange => dhe_rsa, + cipher => aes_256_cbc, + mac => sha, + prf => default_prf}], + ServerCiphers = [#{key_exchange => dhe_rsa, + cipher => aes_256_cbc, + mac =>sha, + prf => default_prf}, + #{key_exchange => dhe_rsa, + cipher => aes_128_cbc, + mac => sha, + prf => default_prf}], + honor_cipher_order(Config, true, ServerCiphers, ClientCiphers, #{key_exchange => dhe_rsa, + cipher => aes_256_cbc, + mac => sha, + prf => default_prf}). honor_client_cipher_order() -> [{doc,"Test API honor server cipher order."}]. honor_client_cipher_order(Config) when is_list(Config) -> - ClientCiphers = [{dhe_rsa, aes_128_cbc, sha}, {dhe_rsa, aes_256_cbc, sha}], - ServerCiphers = [{dhe_rsa, aes_256_cbc, sha}, {dhe_rsa, aes_128_cbc, sha}], -honor_cipher_order(Config, false, ServerCiphers, ClientCiphers, {dhe_rsa, aes_128_cbc, sha}). + ClientCiphers = [#{key_exchange => dhe_rsa, + cipher => aes_128_cbc, + mac => sha, + prf => default_prf}, + #{key_exchange => dhe_rsa, + cipher => aes_256_cbc, + mac => sha, + prf => default_prf}], + ServerCiphers = [#{key_exchange => dhe_rsa, + cipher => aes_256_cbc, + mac =>sha, + prf => default_prf}, + #{key_exchange => dhe_rsa, + cipher => aes_128_cbc, + mac => sha, + prf => default_prf}], +honor_cipher_order(Config, false, ServerCiphers, ClientCiphers, #{key_exchange => dhe_rsa, + cipher => aes_128_cbc, + mac => sha, + prf => default_prf}). honor_cipher_order(Config, Honor, ServerCiphers, ClientCiphers, Expected) -> ClientOpts = ssl_test_lib:ssl_options(client_opts, Config), @@ -3495,7 +3527,7 @@ honor_cipher_order(Config, Honor, ServerCiphers, ClientCiphers, Expected) -> %%-------------------------------------------------------------------- tls_ciphersuite_vs_version() -> - [{doc,"Test a SSLv3 client can not negotiate a TLSv* cipher suite."}]. + [{doc,"Test a SSLv3 client cannot negotiate a TLSv* cipher suite."}]. tls_ciphersuite_vs_version(Config) when is_list(Config) -> {_ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), @@ -3972,6 +4004,64 @@ recv_error_handling(Config) when is_list(Config) -> ssl:close(SslSocket), ssl_test_lib:check_result(Server, ok). + + +%%-------------------------------------------------------------------- +call_in_error_state() -> + [{doc,"Special case of call error handling"}]. +call_in_error_state(Config) when is_list(Config) -> + ServerOpts0 = ssl_test_lib:ssl_options(server_opts, Config), + ClientOpts = ssl_test_lib:ssl_options(client_opts, Config), + ServerOpts = [{cacertfile, "foo.pem"} | proplists:delete(cacertfile, ServerOpts0)], + Pid = spawn_link(?MODULE, run_error_server, [[self() | ServerOpts]]), + receive + {Pid, Port} -> + spawn_link(?MODULE, run_client_error, [[Port, ClientOpts]]) + end, + receive + {error, closed} -> + ok; + Other -> + ct:fail(Other) + end. + +run_client_error([Port, Opts]) -> + ssl:connect("localhost", Port, Opts). + +run_error_server([ Pid | Opts]) -> + {ok, Listen} = ssl:listen(0, Opts), + {ok,{_, Port}} = ssl:sockname(Listen), + Pid ! {self(), Port}, + {ok, Socket} = ssl:transport_accept(Listen), + Pid ! ssl:controlling_process(Socket, self()). + +%%-------------------------------------------------------------------- + +close_in_error_state() -> + [{doc,"Special case of closing socket in error state"}]. +close_in_error_state(Config) when is_list(Config) -> + ServerOpts0 = ssl_test_lib:ssl_options(server_opts, Config), + ServerOpts = [{cacertfile, "foo.pem"} | proplists:delete(cacertfile, ServerOpts0)], + ClientOpts = ssl_test_lib:ssl_options(client_opts, Config), + _ = spawn_link(?MODULE, run_error_server_close, [[self() | ServerOpts]]), + receive + {_Pid, Port} -> + spawn_link(?MODULE, run_client_error, [[Port, ClientOpts]]) + end, + receive + ok -> + ok; + Other -> + ct:fail(Other) + end. + +run_error_server_close([Pid | Opts]) -> + {ok, Listen} = ssl:listen(0, Opts), + {ok,{_, Port}} = ssl:sockname(Listen), + Pid ! {self(), Port}, + {ok, Socket} = ssl:transport_accept(Listen), + Pid ! ssl:close(Socket). + %%-------------------------------------------------------------------- rizzo() -> @@ -4191,17 +4281,17 @@ unordered_protocol_versions_server(Config) when is_list(Config) -> {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), Server = ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, {from, self()}, - {mfa, {?MODULE, connection_info_result, []}}, + {mfa, {?MODULE, protocol_info_result, []}}, {options, [{versions, ['tlsv1.1', 'tlsv1.2']} | ServerOpts]}]), Port = ssl_test_lib:inet_port(Server), Client = ssl_test_lib:start_client([{node, ClientNode}, {port, Port}, {host, Hostname}, {from, self()}, - {mfa, {?MODULE, connection_info_result, []}}, + {mfa, {?MODULE, protocol_info_result, []}}, {options, ClientOpts}]), - CipherSuite = first_rsa_suite(ssl:cipher_suites()), - ServerMsg = ClientMsg = {ok, {'tlsv1.2', CipherSuite}}, + + ServerMsg = ClientMsg = {ok,'tlsv1.2'}, ssl_test_lib:check_result(Server, ServerMsg, Client, ClientMsg). %%-------------------------------------------------------------------- @@ -4216,18 +4306,17 @@ unordered_protocol_versions_client(Config) when is_list(Config) -> {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), Server = ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, {from, self()}, - {mfa, {?MODULE, connection_info_result, []}}, + {mfa, {?MODULE, protocol_info_result, []}}, {options, ServerOpts }]), Port = ssl_test_lib:inet_port(Server), Client = ssl_test_lib:start_client([{node, ClientNode}, {port, Port}, {host, Hostname}, {from, self()}, - {mfa, {?MODULE, connection_info_result, []}}, + {mfa, {?MODULE, protocol_info_result, []}}, {options, [{versions, ['tlsv1.1', 'tlsv1.2']} | ClientOpts]}]), - - CipherSuite = first_rsa_suite(ssl:cipher_suites()), - ServerMsg = ClientMsg = {ok, {'tlsv1.2', CipherSuite}}, + + ServerMsg = ClientMsg = {ok, 'tlsv1.2'}, ssl_test_lib:check_result(Server, ServerMsg, Client, ClientMsg). %%-------------------------------------------------------------------- @@ -4943,6 +5032,7 @@ run_suites(Ciphers, Config, Type) -> [{ciphers, Ciphers} | ssl_test_lib:ssl_options(server_ecdsa_opts, Config)]} end, + ct:pal("ssl_test_lib:filter_suites(~p ~p) -> ~p ", [Ciphers, Version, ssl_test_lib:filter_suites(Ciphers, Version)]), Result = lists:map(fun(Cipher) -> cipher(Cipher, Version, Config, ClientOpts, ServerOpts) end, ssl_test_lib:filter_suites(Ciphers, Version)), @@ -4955,7 +5045,7 @@ run_suites(Ciphers, Config, Type) -> end. erlang_cipher_suite(Suite) when is_list(Suite)-> - ssl_cipher:erl_suite_definition(ssl_cipher:openssl_suite(Suite)); + ssl_cipher:suite_definition(ssl_cipher:openssl_suite(Suite)); erlang_cipher_suite(Suite) -> Suite. @@ -5007,8 +5097,13 @@ connection_information_result(Socket) -> end. connection_info_result(Socket) -> - {ok, Info} = ssl:connection_information(Socket, [protocol, cipher_suite]), - {ok, {proplists:get_value(protocol, Info), proplists:get_value(cipher_suite, Info)}}. + {ok, Info} = ssl:connection_information(Socket, [protocol, selected_cipher_suite]), + {ok, {proplists:get_value(protocol, Info), proplists:get_value(selected_cipher_suite, Info)}}. + +protocol_info_result(Socket) -> + {ok, [{protocol, PVersion}]} = ssl:connection_information(Socket, [protocol]), + {ok, PVersion}. + version_info_result(Socket) -> {ok, [{version, Version}]} = ssl:connection_information(Socket, [version]), {ok, Version}. @@ -5137,20 +5232,6 @@ try_recv_active_once(Socket) -> {error, einval} = ssl:recv(Socket, 11), ok. -first_rsa_suite([{ecdhe_rsa, _, _} = Suite | _]) -> - Suite; -first_rsa_suite([{dhe_rsa, _, _} = Suite| _]) -> - Suite; -first_rsa_suite([{rsa, _, _} = Suite| _]) -> - Suite; -first_rsa_suite([{ecdhe_rsa, _, _, _} = Suite | _]) -> - Suite; -first_rsa_suite([{dhe_rsa, _, _, _} = Suite| _]) -> - Suite; -first_rsa_suite([{rsa, _, _, _} = Suite| _]) -> - Suite; -first_rsa_suite([_ | Rest]) -> - first_rsa_suite(Rest). wait_for_send(Socket) -> %% Make sure TLS process processed send message event diff --git a/lib/ssl/test/ssl_certificate_verify_SUITE.erl b/lib/ssl/test/ssl_certificate_verify_SUITE.erl index dca25b774b..c0981a9eaf 100644 --- a/lib/ssl/test/ssl_certificate_verify_SUITE.erl +++ b/lib/ssl/test/ssl_certificate_verify_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2012-2017. All Rights Reserved. +%% Copyright Ericsson AB 2012-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -513,7 +513,7 @@ verify_fun_always_run_client(Config) when is_list(Config) -> Port = ssl_test_lib:inet_port(Server), %% If user verify fun is called correctly we fail the connection. - %% otherwise we can not tell this case apart form where we miss + %% otherwise we cannot tell this case apart form where we miss %% to call users verify fun FunAndState = {fun(_,{extension, _}, UserState) -> {unknown, UserState}; @@ -552,7 +552,7 @@ verify_fun_always_run_server(Config) when is_list(Config) -> {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), %% If user verify fun is called correctly we fail the connection. - %% otherwise we can not tell this case apart form where we miss + %% otherwise we cannot tell this case apart form where we miss %% to call users verify fun FunAndState = {fun(_,{extension, _}, UserState) -> {unknown, UserState}; diff --git a/lib/ssl/test/ssl_crl_SUITE.erl b/lib/ssl/test/ssl_crl_SUITE.erl index 668c76e38d..23c5eaf84d 100644 --- a/lib/ssl/test/ssl_crl_SUITE.erl +++ b/lib/ssl/test/ssl_crl_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2016. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssl/test/ssl_engine_SUITE.erl b/lib/ssl/test/ssl_engine_SUITE.erl index 71891356e8..1423c99dc2 100644 --- a/lib/ssl/test/ssl_engine_SUITE.erl +++ b/lib/ssl/test/ssl_engine_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2017-2017. All Rights Reserved. +%% Copyright Ericsson AB 2017-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -117,8 +117,23 @@ private_key(Config) when is_list(Config) -> EngineServerConf = [{key, #{algorithm => rsa, engine => Engine, key_id => ServerKey}} | proplists:delete(key, ServerConf)], + + EngineFileClientConf = [{key, #{algorithm => rsa, + engine => Engine, + key_id => ClientKey}} | + proplists:delete(keyfile, FileClientConf)], + + EngineFileServerConf = [{key, #{algorithm => rsa, + engine => Engine, + key_id => ServerKey}} | + proplists:delete(keyfile, FileServerConf)], + %% Test with engine test_tls_connection(EngineServerConf, EngineClientConf, Config), + + %% Test with engine and present file arugments + test_tls_connection(EngineFileServerConf, EngineFileClientConf, Config), + %% Test that sofware fallback is available test_tls_connection(ServerConf, [{reuse_sessions, false} |ClientConf], Config). diff --git a/lib/ssl/test/ssl_handshake_SUITE.erl b/lib/ssl/test/ssl_handshake_SUITE.erl index cd3d972c07..9ae04184e2 100644 --- a/lib/ssl/test/ssl_handshake_SUITE.erl +++ b/lib/ssl/test/ssl_handshake_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2017. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -149,7 +149,7 @@ decode_single_hello_sni_extension_correctly(_Config) -> Exts = Decoded. decode_empty_server_sni_correctly(_Config) -> - Exts = #hello_extensions{sni = ""}, + Exts = #hello_extensions{sni = #sni{hostname = ""}}, SNI = <<?UINT16(?SNI_EXT),?UINT16(0)>>, Decoded = ssl_handshake:decode_hello_extensions(SNI), Exts = Decoded. diff --git a/lib/ssl/test/ssl_npn_handshake_SUITE.erl b/lib/ssl/test/ssl_npn_handshake_SUITE.erl index 6bf2aa2786..1c7d6b5f9f 100644 --- a/lib/ssl/test/ssl_npn_handshake_SUITE.erl +++ b/lib/ssl/test/ssl_npn_handshake_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2016. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssl/test/ssl_payload_SUITE.erl b/lib/ssl/test/ssl_payload_SUITE.erl index ef05241759..5939800001 100644 --- a/lib/ssl/test/ssl_payload_SUITE.erl +++ b/lib/ssl/test/ssl_payload_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2016. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssl/test/ssl_sni_SUITE.erl b/lib/ssl/test/ssl_sni_SUITE.erl index 7e78c41444..251b6a2639 100644 --- a/lib/ssl/test/ssl_sni_SUITE.erl +++ b/lib/ssl/test/ssl_sni_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2015-2016. All Rights Reserved. +%% Copyright Ericsson AB 2015-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssl/test/ssl_test_lib.erl b/lib/ssl/test/ssl_test_lib.erl index 1e88ca15de..7202e3662c 100644 --- a/lib/ssl/test/ssl_test_lib.erl +++ b/lib/ssl/test/ssl_test_lib.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2017. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -309,7 +309,7 @@ client_cont_loop(Node, Host, Port, Pid, Transport, Options, ContOpts, Opts) -> case rpc:call(Node, Transport, connect, [Host, Port, Options]) of {ok, Socket0, _} -> ct:log("~p:~p~nClient: handshake_continue(~p, ~p, infinity) ~n", [?MODULE, ?LINE, Socket0, ContOpts]), - case rpc:call(Node, Transport, handshake_continue, [Socket0, ContOpts, infinity]) of + case rpc:call(Node, Transport, handshake_continue, [Socket0, ContOpts]) of {ok, Socket} -> Pid ! {connected, Socket}, {Module, Function, Args} = proplists:get_value(mfa, Opts), @@ -585,6 +585,17 @@ default_cert_chain_conf() -> %% Use only default options [[],[],[]]. +gen_conf(mix, mix, UserClient, UserServer) -> + ClientTag = conf_tag("client"), + ServerTag = conf_tag("server"), + + DefaultClient = default_cert_chain_conf(), + DefaultServer = default_cert_chain_conf(), + + ClientConf = merge_chain_spec(UserClient, DefaultClient, []), + ServerConf = merge_chain_spec(UserServer, DefaultServer, []), + + new_format([{ClientTag, ClientConf}, {ServerTag, ServerConf}]); gen_conf(ClientChainType, ServerChainType, UserClient, UserServer) -> ClientTag = conf_tag("client"), ServerTag = conf_tag("server"), @@ -678,6 +689,32 @@ merge_spec(User, Default, [Conf | Rest], Acc) -> merge_spec(User, Default, Rest, [{Conf, Value} | Acc]) end. +make_mix_cert(Config) -> + Ext = x509_test:extensions([{key_usage, [digitalSignature]}]), + Digest = {digest, appropriate_sha(crypto:supports())}, + CurveOid = hd(tls_v1:ecc_curves(0)), + ClientFileBase = filename:join([proplists:get_value(priv_dir, Config), "mix"]), + ServerFileBase = filename:join([proplists:get_value(priv_dir, Config), "mix"]), + ClientChain = [[Digest, {key, {namedCurve, CurveOid}}], + [Digest, {key, hardcode_rsa_key(1)}], + [Digest, {key, {namedCurve, CurveOid}}, {extensions, Ext}] + ], + ServerChain = [[Digest, {key, {namedCurve, CurveOid}}], + [Digest, {key, hardcode_rsa_key(2)}], + [Digest, {key, {namedCurve, CurveOid}},{extensions, Ext}] + ], + ClientChainType =ServerChainType = mix, + CertChainConf = gen_conf(ClientChainType, ServerChainType, ClientChain, ServerChain), + ClientFileBase = filename:join([proplists:get_value(priv_dir, Config), atom_to_list(ClientChainType)]), + ServerFileBase = filename:join([proplists:get_value(priv_dir, Config), atom_to_list(ServerChainType)]), + GenCertData = public_key:pkix_test_data(CertChainConf), + [{server_config, ServerConf}, + {client_config, ClientConf}] = + x509_test:gen_pem_config_files(GenCertData, ClientFileBase, ServerFileBase), + {[{verify, verify_peer} | ClientConf], + [{reuseaddr, true}, {verify, verify_peer} | ServerConf] + }. + make_ecdsa_cert(Config) -> CryptoSupport = crypto:supports(), case proplists:get_bool(ecdsa, proplists:get_value(public_keys, CryptoSupport)) of @@ -1097,8 +1134,6 @@ check_ecc(SSL, Role, Expect) -> {ok, Data} = ssl:connection_information(SSL), case lists:keyfind(ecc, 1, Data) of {ecc, {named_curve, Expect}} -> ok; - false when Expect == undefined -> ok; - false when Expect == secp256r1 andalso Role == client_no_ecc -> ok; Other -> {error, Role, Expect, Other} end. @@ -1170,13 +1205,13 @@ rsa_suites(CounterPart) -> lists:member(cipher_atom(Cipher), Ciphers); ({ecdhe_rsa, Cipher, _}) when ECC == true -> lists:member(cipher_atom(Cipher), Ciphers); + ({ecdhe_rsa, Cipher, _,_}) when ECC == true -> + lists:member(cipher_atom(Cipher), Ciphers); ({rsa, Cipher, _, _}) -> lists:member(cipher_atom(Cipher), Ciphers); ({dhe_rsa, Cipher, _,_}) -> lists:member(cipher_atom(Cipher), Ciphers); - ({ecdhe_rsa, Cipher, _,_}) when ECC == true -> - lists:member(cipher_atom(Cipher), Ciphers); - (_) -> + (_) -> false end, common_ciphers(CounterPart)). @@ -1186,13 +1221,13 @@ common_ciphers(crypto) -> common_ciphers(openssl) -> OpenSslSuites = string:tokens(string:strip(os:cmd("openssl ciphers"), right, $\n), ":"), - [ssl_cipher:erl_suite_definition(S) + [ssl_cipher:suite_definition(S) || S <- ssl_cipher:suites(tls_record:highest_protocol_version([])), lists:member(ssl_cipher:openssl_suite_name(S), OpenSslSuites) ]. available_suites(Version) -> - [ssl_cipher:erl_suite_definition(Suite) || + [ssl_cipher:suite_definition(Suite) || Suite <- ssl_cipher:filter_suites(ssl_cipher:suites(Version))]. @@ -1274,10 +1309,18 @@ ecdh_dh_anonymous_suites(Version) -> (_) -> false end}]). +psk_suites({3,_} = Version) -> + ssl:filter_cipher_suites([ssl_cipher:suite_definition(S) || S <- ssl_cipher:psk_suites(Version)], []); psk_suites(Version) -> - ssl:filter_cipher_suites([ssl_cipher:suite_definition(S) || S <- ssl_cipher:psk_suites(Version)], []). + ssl:filter_cipher_suites(psk_suites(dtls_v1:corresponding_tls_version(Version)), + [{cipher, + fun(rc4_128) -> + false; + (_) -> + true + end}]). -psk_anon_suites(Version) -> +psk_anon_suites({3,_} = Version) -> ssl:filter_cipher_suites([ssl_cipher:suite_definition(S) || S <- ssl_cipher:psk_suites_anon(Version)], [{key_exchange, fun(psk) -> @@ -1286,8 +1329,18 @@ psk_anon_suites(Version) -> true; (_) -> false + end}]); + +psk_anon_suites(Version) -> + ssl:filter_cipher_suites(psk_anon_suites(dtls_v1:corresponding_tls_version(Version)), + [{cipher, + fun(rc4_128) -> + false; + (_) -> + true end}]). + srp_suites() -> ssl:filter_cipher_suites([ssl_cipher:suite_definition(S) || S <- ssl_cipher:srp_suites()], [{key_exchange, @@ -1308,7 +1361,7 @@ srp_dss_suites() -> false end}]). chacha_suites(Version) -> - [ssl_cipher:erl_suite_definition(S) || S <- ssl_cipher:filter_suites(ssl_cipher:chacha_suites(Version))]. + [ssl_cipher:suite_definition(S) || S <- ssl_cipher:filter_suites(ssl_cipher:chacha_suites(Version))]. rc4_suites(Version) -> @@ -1338,7 +1391,7 @@ der_to_pem(File, Entries) -> cipher_result(Socket, Result) -> {ok, Info} = ssl:connection_information(Socket), - Result = {ok, {proplists:get_value(protocol, Info), proplists:get_value(cipher_suite, Info)}}, + Result = {ok, {proplists:get_value(protocol, Info), proplists:get_value(selected_cipher_suite, Info)}}, ct:log("~p:~p~nSuccessfull connect: ~p~n", [?MODULE,?LINE, Result]), %% Importante to send two packets here %% to properly test "cipher state" handling @@ -1450,10 +1503,13 @@ check_key_exchange_send_active(Socket, KeyEx) -> send_recv_result_active(Socket). check_key_exchange({KeyEx,_, _}, KeyEx, _) -> + ct:pal("Kex: ~p", [KeyEx]), true; check_key_exchange({KeyEx,_,_,_}, KeyEx, _) -> + ct:pal("Kex: ~p", [KeyEx]), true; check_key_exchange(KeyEx1, KeyEx2, Version) -> + ct:pal("Kex: ~p ~p", [KeyEx1, KeyEx2]), case Version of 'tlsv1.2' -> v_1_2_check(element(1, KeyEx1), KeyEx2); @@ -1530,7 +1586,7 @@ is_sane_ecc(crypto) -> true end; is_sane_ecc(_) -> - true. + sufficient_crypto_support(cipher_ec). is_fips(openssl) -> VersionStr = os:cmd("openssl version"), @@ -1601,11 +1657,7 @@ openssl_sane_dtls() -> false; "OpenSSL 1.0.2k-freebsd" ++ _ -> false; - "OpenSSL 1.0.2d" ++ _ -> - false; - "OpenSSL 1.0.2n" ++ _ -> - false; - "OpenSSL 1.0.2m" ++ _ -> + "OpenSSL 1.0.2" ++ _ -> false; "OpenSSL 1.0.0" ++ _ -> false; @@ -1713,7 +1765,7 @@ filter_suites([Cipher | _] = Ciphers, AtomVersion) when is_list(Cipher)-> filter_suites([ssl_cipher:openssl_suite(S) || S <- Ciphers], AtomVersion); filter_suites([Cipher | _] = Ciphers, AtomVersion) when is_binary(Cipher)-> - filter_suites([ssl_cipher:erl_suite_definition(S) || S <- Ciphers], + filter_suites([ssl_cipher:suite_definition(S) || S <- Ciphers], AtomVersion); filter_suites(Ciphers0, AtomVersion) -> Version = tls_version(AtomVersion), @@ -1725,7 +1777,7 @@ filter_suites(Ciphers0, AtomVersion) -> ++ ssl_cipher:srp_suites_anon() ++ ssl_cipher:rc4_suites(Version), Supported1 = ssl_cipher:filter_suites(Supported0), - Supported2 = [ssl_cipher:erl_suite_definition(S) || S <- Supported1], + Supported2 = [ssl_cipher:suite_definition(S) || S <- Supported1], [Cipher || Cipher <- Ciphers0, lists:member(Cipher, Supported2)]. -define(OPENSSL_QUIT, "Q\n"). diff --git a/lib/ssl/test/ssl_to_openssl_SUITE.erl b/lib/ssl/test/ssl_to_openssl_SUITE.erl index 4f02d8d15d..7fc5e13400 100644 --- a/lib/ssl/test/ssl_to_openssl_SUITE.erl +++ b/lib/ssl/test/ssl_to_openssl_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2017. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/ssl/vsn.mk b/lib/ssl/vsn.mk index eb85a55717..75d959accf 100644 --- a/lib/ssl/vsn.mk +++ b/lib/ssl/vsn.mk @@ -1 +1 @@ -SSL_VSN = 8.2.6 +SSL_VSN = 9.1 diff --git a/lib/stdlib/doc/src/Makefile b/lib/stdlib/doc/src/Makefile index 5c6b714f80..4541b4a463 100644 --- a/lib/stdlib/doc/src/Makefile +++ b/lib/stdlib/doc/src/Makefile @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 1997-2017. All Rights Reserved. +# Copyright Ericsson AB 1997-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -155,6 +155,7 @@ debug opt: clean clean_docs: rm -rf $(HTMLDIR)/* + rm -rf $(XMLDIR) rm -f $(MAN3DIR)/* rm -f $(MAN6DIR)/* rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) diff --git a/lib/stdlib/doc/src/assert_hrl.xml b/lib/stdlib/doc/src/assert_hrl.xml index 33f29f38da..4dc7299609 100644 --- a/lib/stdlib/doc/src/assert_hrl.xml +++ b/lib/stdlib/doc/src/assert_hrl.xml @@ -4,7 +4,7 @@ <fileref> <header> <copyright> - <year>2012</year><year>2017</year> + <year>2012</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/stdlib/doc/src/c.xml b/lib/stdlib/doc/src/c.xml index 697e1715e7..b6cb6f5aae 100644 --- a/lib/stdlib/doc/src/c.xml +++ b/lib/stdlib/doc/src/c.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1996</year><year>2016</year> + <year>1996</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/stdlib/doc/src/digraph.xml b/lib/stdlib/doc/src/digraph.xml index db96beed6c..a5252b443b 100644 --- a/lib/stdlib/doc/src/digraph.xml +++ b/lib/stdlib/doc/src/digraph.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1996</year><year>2016</year> + <year>1996</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/stdlib/doc/src/ets.xml b/lib/stdlib/doc/src/ets.xml index 1995262145..f8c54fb79a 100644 --- a/lib/stdlib/doc/src/ets.xml +++ b/lib/stdlib/doc/src/ets.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1996</year><year>2017</year> + <year>1996</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/stdlib/doc/src/gb_sets.xml b/lib/stdlib/doc/src/gb_sets.xml index 2a3785dc27..03397b4503 100644 --- a/lib/stdlib/doc/src/gb_sets.xml +++ b/lib/stdlib/doc/src/gb_sets.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>2001</year><year>2016</year> + <year>2001</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/stdlib/doc/src/gen_event.xml b/lib/stdlib/doc/src/gen_event.xml index 6170801e87..f793ec7fdf 100644 --- a/lib/stdlib/doc/src/gen_event.xml +++ b/lib/stdlib/doc/src/gen_event.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1996</year><year>2017</year> + <year>1996</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/stdlib/doc/src/gen_server.xml b/lib/stdlib/doc/src/gen_server.xml index 27edbc8de7..106bda85f5 100644 --- a/lib/stdlib/doc/src/gen_server.xml +++ b/lib/stdlib/doc/src/gen_server.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1996</year><year>2017</year> + <year>1996</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/stdlib/doc/src/gen_statem.xml b/lib/stdlib/doc/src/gen_statem.xml index a808d3af55..eb0f7d24f0 100644 --- a/lib/stdlib/doc/src/gen_statem.xml +++ b/lib/stdlib/doc/src/gen_statem.xml @@ -722,7 +722,7 @@ handle_event(_, _, State, Data) -> right before entering the initial state even though this formally is not a state change. In this case <c>OldState</c> will be the same as <c>State</c>, - which can not happen for a subsequent state change, + which cannot happen for a subsequent state change, but will happen when repeating the state enter call. </p> </desc> @@ -1256,7 +1256,7 @@ handle_event(_, _, State, Data) -> <desc> <p> <c><anno>State</anno></c> is the current state - and it can not be changed since the state callback + and it cannot be changed since the state callback was called with a <seealso marker="#type-state_enter"><em>state enter call</em></seealso>. </p> @@ -1922,7 +1922,7 @@ handle_event(_, _, State, Data) -> <p> If the function returns a failure <c>Reason</c>, the ongoing upgrade fails and rolls back to the old release. - Note that <c>Reason</c> can not be an <c>{ok,_,_}</c> tuple + Note that <c>Reason</c> cannot be an <c>{ok,_,_}</c> tuple since that will be regarded as a <c>{ok,NewState,NewData}</c> tuple, and that a tuple matching <c>{ok,_}</c> @@ -2208,7 +2208,7 @@ init(Args) -> erlang:error(not_implemented, [Args]).</pre> <seealso marker="erts:erlang#throw/1"><c>throw</c></seealso> to return the result, which can be useful. For example to bail out with <c>throw(keep_state_and_data)</c> - from deep within complex code that can not + from deep within complex code that cannot return <c>{next_state,State,Data}</c> because <c>State</c> or <c>Data</c> is no longer in scope. </p> diff --git a/lib/stdlib/doc/src/io.xml b/lib/stdlib/doc/src/io.xml index f1037ec76b..d4a2713840 100644 --- a/lib/stdlib/doc/src/io.xml +++ b/lib/stdlib/doc/src/io.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1996</year><year>2017</year> + <year>1996</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -332,11 +332,22 @@ Here T = [{attributes,[[{id,age,1.5}, {tag,{'PRIVATE',3}}, {mode,implicit}] ok</pre> + + <p>As from Erlang/OTP 21.0, a field width of value + <c>0</c> can be used for specifying that a line is + infinitely long, which means that no line breaks + are inserted. For example:</p> + + <pre> +5> <input>io:fwrite("~0p~n", [lists:seq(1, 30)]).</input> +[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30] +ok</pre> + <p>When the modifier <c>l</c> is specified, no detection of printable character lists takes place, for example:</p> <pre> -5> <input>S = [{a,"a"}, {b, "b"}].</input> -6> <input>io:fwrite("~15p~n", [S]).</input> +6> <input>S = [{a,"a"}, {b, "b"}], + io:fwrite("~15p~n", [S]).</input> [{a,"a"}, {b,"b"}] ok diff --git a/lib/stdlib/doc/src/maps.xml b/lib/stdlib/doc/src/maps.xml index 987d92989d..4c5199ca2b 100644 --- a/lib/stdlib/doc/src/maps.xml +++ b/lib/stdlib/doc/src/maps.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>2013</year><year>2016</year> + <year>2013</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -35,9 +35,10 @@ <datatypes> <datatype> - <name name="iterator"/> + <name name="iterator" n_vars="2"/> <desc> - <p>An iterator representing the key value associations in a map.</p> + <p>An iterator representing the associations in a map with keys of type + <c><anno>Key</anno></c> and values of type <c><anno>Value</anno></c>.</p> <p>Created using <seealso marker="#iterator-1"><c>maps:iterator/1</c></seealso>.</p> <p>Consumed by <seealso marker="#next-1"><c>maps:next/1</c></seealso>, <seealso marker="#filter-2"><c>maps:filter/2</c></seealso>, @@ -45,6 +46,10 @@ <seealso marker="#map-2"><c>maps:map/2</c></seealso>.</p> </desc> </datatype> + + <datatype> + <name name="iterator" n_vars="0"/> + </datatype> </datatypes> <funcs> @@ -90,13 +95,13 @@ <name name="fold" arity="3"/> <fsummary></fsummary> <desc> - <p>Calls <c>F(K, V, AccIn)</c> for every <c><anno>K</anno></c> to value - <c><anno>V</anno></c> association in <c><anno>MapOrIter</anno></c> in - any order. Function <c>fun F/3</c> must return a new - accumulator, which is passed to the next successive call. - This function returns the final value of the accumulator. The initial - accumulator value <c><anno>Init</anno></c> is returned if the map is - empty.</p> + <p>Calls <c>F(Key, Value, AccIn)</c> for every <c><anno>Key</anno></c> + to value <c><anno>Value</anno></c> association in + <c><anno>MapOrIter</anno></c> in any order. Function <c>fun F/3</c> + must return a new accumulator, which is passed to the next successive + call. This function returns the final value of the accumulator. + The initial accumulator value <c><anno>Init</anno></c> is returned + if the map is empty.</p> <p>The call fails with a <c>{badmap,Map}</c> exception if <c><anno>MapOrIter</anno></c> is not a map or valid iterator, or with <c>badarg</c> if <c><anno>Fun</anno></c> is not a @@ -234,11 +239,12 @@ none</code> <fsummary></fsummary> <desc> <p>Produces a new map <c><anno>Map</anno></c> by calling function - <c>fun F(K, V1)</c> for every <c><anno>K</anno></c> to value - <c><anno>V1</anno></c> association in <c><anno>MapOrIter</anno></c> in - any order. Function <c>fun F/2</c> must return value - <c><anno>V2</anno></c> to be associated with key <c><anno>K</anno></c> - for the new map <c><anno>Map</anno></c>.</p> + <c>fun F(Key, Value1)</c> for every <c><anno>Key</anno></c> to value + <c><anno>Value1</anno></c> association in + <c><anno>MapOrIter</anno></c> in any order. Function <c>fun Fun/2</c> + must return value <c><anno>Value2</anno></c> to be associated with + key <c><anno>Key</anno></c> for the new map + <c><anno>Map</anno></c>.</p> <p>The call fails with a <c>{badmap,Map}</c> exception if <c><anno>MapOrIter</anno></c> is not a map or valid iterator, or with <c>badarg</c> if <c><anno>Fun</anno></c> is not a diff --git a/lib/stdlib/doc/src/notes.xml b/lib/stdlib/doc/src/notes.xml index e26c4aba74..8b6de03f5f 100644 --- a/lib/stdlib/doc/src/notes.xml +++ b/lib/stdlib/doc/src/notes.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2004</year><year>2017</year> + <year>2004</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -31,6 +31,444 @@ </header> <p>This document describes the changes made to the STDLIB application.</p> +<section><title>STDLIB 3.5.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> Fix a bug that could cause a crash when formatting a + list of non-characters using the control sequences + <c>p</c> or <c>P</c> and limiting the output with the + option <c>chars_limit</c>. </p> + <p> + Own Id: OTP-15159</p> + </item> + </list> + </section> + +</section> + +<section><title>STDLIB 3.5</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p><c>gen_statem</c> improvements.</p> <p> When using an + exception that is valid but not allowed in a state enter + call, the reason has been changed from + <c>{bad_action_from_state_function,Action}</c> to + <c>{bad_state_enter_action_from_state_function,Action}</c>. + </p><p> Timer parsing has been improved. Many erroneous + timeout tuples was not handled correctly. </p><p> The + documentation has been updated, in particular the User's + Guide and the pointer to it from the Reference Manual is + much more obvious. </p> + <p> + Own Id: OTP-14015</p> + </item> + <item> + <p> + The type specifications for <c>file:posix/0</c> and + <c>inet:posix/0</c> have been updated according to which + errors file and socket operations should be able to + return.</p> + <p> + Own Id: OTP-14019 Aux Id: ERL-550 </p> + </item> + <item> + <p> File operations used to accept <seealso + marker="kernel:file#type-name_all">filenames</seealso> + containing null characters (integer value zero). This + caused the name to be truncated and in some cases + arguments to primitive operations to be mixed up. + Filenames containing null characters inside the filename + are now <em>rejected</em> and will cause primitive file + operations to fail. </p> <p> Also environment variable + operations used to accept <seealso + marker="kernel:os#type-env_var_name">names</seealso> and + <seealso + marker="kernel:os#type-env_var_value">values</seealso> of + environment variables containing null characters (integer + value zero). This caused operations to silently produce + erroneous results. Environment variable names and values + containing null characters inside the name or value are + now <em>rejected</em> and will cause environment variable + operations to fail. </p> <p>Primitive environment + variable operations also used to accept the <c>$=</c> + character in environment variable names causing various + problems. <c>$=</c> characters in environment variable + names are now also <em>rejected</em>. </p> <p>Also + <seealso + marker="kernel:os#cmd/1"><c>os:cmd/1</c></seealso> now + reject null characters inside its <seealso + marker="kernel:os#type-os_command">command</seealso>. + </p> <p><seealso + marker="erts:erlang#open_port/2"><c>erlang:open_port/2</c></seealso> + will also reject null characters inside the port name + from now on.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-14543 Aux Id: ERL-370 </p> + </item> + <item> + <p> Make <c>io_lib:unscan_format/1</c> work with pad char + and default precision. </p> + <p> + Own Id: OTP-14958 Aux Id: PR-1735 </p> + </item> + <item> + <p> The control sequence modifiers <c>t</c> and <c>l</c> + can be used together in the same control sequence which + makes it possible to have Unicode atoms and no detection + of printable character lists at the same time. </p> + <p> + Own Id: OTP-14971 Aux Id: PR-1743 </p> + </item> + <item> + <p> Fix a bug in the Erlang code linter: the check of + guard expressions no longer returns <c>false</c> if the + map syntax is used. The bug affected the Erlang shell, + the Debugger, and other modules evaluating abstract code. + </p> + <p> + Own Id: OTP-15035 Aux Id: ERL-613 </p> + </item> + <item> + <p> + A sys debug fun of type {Fun,State} should not be + possible to install twice. This was, however, possible if + the current State was 'undefined', which was mistaken for + non-existing fun. This has been corrected.</p> + <p> + Own Id: OTP-15049</p> + </item> + <item> + <p> + Fix <c>io:putchars/2</c> stacktrace rewriting at errors + to point to a valid function.</p> + <p> + Own Id: OTP-15101</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + The <c>gen_server</c> has gotten a new callback + <c>handle_continue/2</c> for check pointing the state. + This is useful at least when implementing behaviours on + top of <c>gen_server</c> and for some start up scenarios.</p> + <p> + Own Id: OTP-13019 Aux Id: PR-1490 </p> + </item> + <item> + <p> The semantics of timeout parameter + <c>{clean_timeout,infinity}</c> to + <c>gen_statem:call/3</c> has been changed to use a proxy + process for the call. With this change + <c>clean_timeout</c> implicates a proxy process with no + exceptions. This may be a hard to observe + incompatibility: in the presence of network problems a + late reply could arrive in the caller's message queue + when catching errors. That will not happen after this + correction. </p><p> The semantics of timeout parameter + <c>infinity</c> has not been changed. </p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-13073 Aux Id: PR-1595 </p> + </item> + <item> + <p>A new logging API is added to Erlang/OTP, see the + <seealso + marker="kernel:logger"><c>logger(3)</c></seealso> manual + page, and section <seealso + marker="kernel:logger_chapter">Logging</seealso> in the + Kernel User's Guide.</p> + <p>Calls to <c>error_logger</c> are automatically + redirected to the new API, and legacy error logger event + handlers can still be used. It is, however, recommended + to use the Logger API directly when writing new code.</p> + <p>Notice the following potential incompatibilities:</p> + <list> <item><p>Kernel configuration parameters + <c>error_logger</c> still works, but is overruled if the + default handler's output destination is configured with + Kernel configuration parameter <c>logger</c>.</p> <p>In + general, parameters for configuring error logger are + overwritten by new parameters for configuring + Logger.</p></item> <item><p>The concept of SASL error + logging is deprecated, meaning that by default the SASL + application does not affect which log events are + logged.</p> <p>By default, supervisor reports and crash + reports are logged by the default Logger handler started + by Kernel, and end up at the same destination (terminal + or file) as other standard log event from Erlang/OTP.</p> + <p>Progress reports are not logged by default, but can be + enabled by setting the primary log level to info, for + example with the Kernel configuration parameter + <c>logger_level</c>.</p> <p>To obtain backwards + compatibility with the SASL error logging functionality + from earlier releases, set Kernel configuration parameter + <c>logger_sasl_compatible</c> to <c>true</c>. This + prevents the default Logger handler from logging any + supervisor-, crash-, or progress reports. Instead, SASL + adds a separate Logger handler during application start, + which takes care of these log events. The SASL + configuration parameters <c>sasl_error_logger</c> and + <c>sasl_errlog_type</c> specify the destination (terminal + or file) and severity level to log for these + events.</p></item></list> + <p> + Since Logger is new in Erlang/OTP 21.0, we do reserve the + right to introduce changes to the Logger API and + functionality in patches following this release. These + changes might or might not be backwards compatible with + the initial version.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-13295</p> + </item> + <item> + <p> Add functions + <c>calendar:system_time_to_local_time/2</c> and + <c>calendar:system_time_to_universal_time/2</c>. </p> + <p> + Own Id: OTP-13413</p> + </item> + <item> + <p> Functions <c>rand:uniform_real/0</c> and + <c>rand:uniform_real_s/1</c> have been added. They + produce uniformly distributed numbers in the range <c>0.0 + =< X < 1.0</c> that are as close to random real + numbers as Normalized IEEE 754 Double Precision allows. + Because the random real number exactly <c>0.0</c> is + infinitely improbable they will never return exactly + <c>0.0</c>. </p><p> These properties are useful when you + need to call for example <c>math:log(X)</c> or <c>1 / + X</c> on a random value <c>X</c>, since that will never + fail with a number from these new functions. </p> + <p> + Own Id: OTP-13764 Aux Id: PR-1574 </p> + </item> + <item> + <p> + Added maps:iterator/0 and maps:next/1 to be used for + iterating over the key-value associations in a map.</p> + <p> + Own Id: OTP-14012</p> + </item> + <item> + <p>Changed the default behaviour of <c>.erlang</c> + loading: <c>.erlang</c> is no longer loaded from the + current directory. <c>c:erlangrc(PathList)</c> can be + used to search and load an <c>.erlang</c> file from user + specified directories.</p> <p><c>escript</c>, + <c>erlc</c>, <c>dialyzer</c> and <c>typer</c> no longer + load an <c>.erlang</c> at all.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-14439</p> + </item> + <item> + <p> + Added new uri_string module to stdlib for handling URIs + (RFC 3986).</p> + <p> + Own Id: OTP-14496</p> + </item> + <item> + <p> + Update Unicode specification to version 10.0.</p> + <p> + Own Id: OTP-14503</p> + </item> + <item> + <p><c>filelib:wildcard()</c> now allows characters with a + special meaning to be escaped using backslashes.</p> + <p>This is an incompatible change, but note that the use + of backslashes in wildcards would already work + differently on Windows and Unix. Existing calls to + <c>filelib:wildcard()</c> needs to be updated. On + Windows, directory separators must always be written as a + slash.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-14577</p> + </item> + <item> + <p> + The supervisor now stores its child specifications in a + map instead of a list. This causes a significant + improvement when starting many children under a + non-simple_one_for_one supervisor.</p> + <p> + Own Id: OTP-14586</p> + </item> + <item> + <p> The <c>base64</c> module is optimized. </p> <p> Note + that the functions <c>encode/1</c>, <c>decode/1</c>, and + <c>mime_decode/1</c> fail unless called with an argument + of the documented type. They used to accept any + <c>iodata()</c>. </p> + <p> + Own Id: OTP-14624 Aux Id: PR-1565 </p> + </item> + <item> + <p> Add function <c>lists:search/2</c>. </p> + <p> + Own Id: OTP-14675 Aux Id: PR-102 </p> + </item> + <item> + <p> + uri_string module extended with functions for handling + application/x-www-form-urlencoded query strings based on + the HTML5 specification.</p> + <p> + Own Id: OTP-14747</p> + </item> + <item> + <p> Add functions + <c>calendar:rfc3339_to_system_time/1,2</c> and + <c>calendar:system_time_to_rfc3339/1,2</c>. </p> + <p> + Own Id: OTP-14764</p> + </item> + <item> + <p> The stack traces returned by the functions of the + <c>erl_eval</c> module more accurately reflect where the + exception occurred. </p> + <p> + Own Id: OTP-14826 Aux Id: PR 1540 </p> + </item> + <item> + <p> Add options <c>atime</c>, <c>mtime</c>, <c>ctime</c>, + <c>uid</c>, and <c>gid</c> to the <c>erl_tar:add/3,4</c> + functions. </p> + <p> + Own Id: OTP-14834 Aux Id: PR 1608 </p> + </item> + <item> + <p>Added <c>ets:whereis/1</c> for retrieving the table + identifier of a named table.</p> + <p> + Own Id: OTP-14884</p> + </item> + <item> + <p> + Improved URI normalization functions in the uri_string + module.</p> + <p> + Own Id: OTP-14910</p> + </item> + <item> + <p> The new functions <c>io_lib:fwrite/3</c> and + <c>io_lib:format/3</c> take a third argument, an option + list. The only option is <c>chars_limit</c>, which is + used for limiting the number of returned characters. The + limit is soft, which means that the number of returned + characters exceeds the limit with at most a smallish + amount. If the limit is set, the functions + <c>format/3</c> and <c>fwrite/3</c> try to distribute the + number of characters evenly over the control sequences + <c>pPswW</c>. Furthermore, the control sequences + <c>pPwP</c> try to distribute the number of characters + evenly over substructures. </p> <p> A modification of the + control sequences <c>pPwW</c> is that even if there is no + limit on the number of returned characters, all + associations of a map are printed to the same depth. The + aim is to give a more consistent output as the order of + map keys is not defined. As before, if the depth is less + than the number of associations of a map, the selection + of associations to print is arbitrary. </p> + <p> + Own Id: OTP-14983</p> + </item> + <item> + <p> Add functions <c>ordsets:is_empty/1</c> and + <c>sets:is_empty/1</c>. </p> + <p> + Own Id: OTP-14996 Aux Id: ERL-557, PR-1703 </p> + </item> + <item> + <p> + Improve performance of <c>string:uppercase/1</c>, + <c>string:lowercase/1</c> and <c>string:casefold/1</c> + when handling ASCII characters.</p> + <p> + Own Id: OTP-14998</p> + </item> + <item> + <p>External funs with literal values for module, name, + and arity (e.g. <c>erlang:abs/1</c>) are now treated as + literals. That means more efficient code that produces + less garbage on the heap.</p> + <p> + Own Id: OTP-15003</p> + </item> + <item> + <p> + sys:statistics(Pid,get) did not report 'out' messages + from gen_server. This is now corrected.</p> + <p> + Own Id: OTP-15047</p> + </item> + <item> + <p> + A sys debug function can now have the format + {Id,Fun,State} in addition to the old {Fun,State}. This + allows installing multiple instances of a debug fun.</p> + <p> + Own Id: OTP-15048</p> + </item> + <item> + <p> The <c>lib</c> module is removed:</p> <list + type="bulleted"> <item><c>lib:error_message/2</c> is + removed.</item> <item><c>lib:flush_receive/0</c> is + removed.</item> <item><c>lib:nonl/1</c> is + removed.</item> <item><c>lib:progname/0</c> is replaced + by <c>ct:get_progname/0</c>.</item> + <item><c>lib:send/2</c> is removed.</item> + <item><c>lib:sendw/2</c> is removed.</item> </list> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-15072 Aux Id: PR 1786, OTP-15114 </p> + </item> + <item> + <p> + Function <c>ets:delete_all_objects/1</c> now yields the + scheduler thread for large tables that take significant + time to clear. This to improve real time characteristics + of other runnable processes.</p> + <p> + Own Id: OTP-15078</p> + </item> + <item> + <p> In control sequences of the functions + <c>io:fwrite/2,3</c> and <c>io_lib:fwrite/2,3</c> + containing <c>p</c> or <c>P</c>, a field width of value + <c>0</c> means that no line breaks are inserted. This is + in contrast to the old behaviour, where <c>0</c> used to + insert line breaks after every subterm. To insert line + breaks after every subterm, a field width of value + <c>1</c> can be used. </p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-15103 Aux Id: ERL-607 </p> + </item> + </list> + </section> + +</section> + <section><title>STDLIB 3.4.5</title> <section><title>Fixed Bugs and Malfunctions</title> @@ -3221,7 +3659,7 @@ you use erlang:halt/2 with an integer first argument and an option list containing {flush,false} as the second argument. Note that now is flushing not dependant of the - exit code, and you can not only flush async threads + exit code, and you cannot only flush async threads operations which we deemed as a strange behaviour anyway. </p> <p>Also, erlang:halt/1,2 has gotten a new feature: If the @@ -3735,9 +4173,9 @@ Supervisors should not save child-specs for temporary processes when they terminate as they should not be restarted. Saving the temporary child spec will result in - that you can not start a new temporary process with the + that you cannot start a new temporary process with the same child spec as an already terminated temporary - process. Since R14B02 you can not restart a temporary + process. Since R14B02 you cannot restart a temporary temporary process as arguments are no longer saved, it has however always been semantically incorrect to restart a temporary process. Thanks to Filipe David Manana for diff --git a/lib/stdlib/doc/src/ordsets.xml b/lib/stdlib/doc/src/ordsets.xml index 2d891d7a5a..11f98c8fb7 100644 --- a/lib/stdlib/doc/src/ordsets.xml +++ b/lib/stdlib/doc/src/ordsets.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1996</year><year>2016</year> + <year>1996</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/stdlib/doc/src/proc_lib.xml b/lib/stdlib/doc/src/proc_lib.xml index 51380ae51c..b85fab67d5 100644 --- a/lib/stdlib/doc/src/proc_lib.xml +++ b/lib/stdlib/doc/src/proc_lib.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1996</year><year>2017</year> + <year>1996</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/stdlib/doc/src/ref_man.xml b/lib/stdlib/doc/src/ref_man.xml index c6f30d272d..8d61833d1f 100644 --- a/lib/stdlib/doc/src/ref_man.xml +++ b/lib/stdlib/doc/src/ref_man.xml @@ -4,7 +4,7 @@ <application xmlns:xi="http://www.w3.org/2001/XInclude"> <header> <copyright> - <year>1996</year><year>2016</year> + <year>1996</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/stdlib/doc/src/sets.xml b/lib/stdlib/doc/src/sets.xml index 1ed96ddc3f..8db3e1e623 100644 --- a/lib/stdlib/doc/src/sets.xml +++ b/lib/stdlib/doc/src/sets.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>2000</year><year>2017</year> + <year>2000</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/stdlib/doc/src/string.xml b/lib/stdlib/doc/src/string.xml index 4a3d37dcb6..3348464eba 100644 --- a/lib/stdlib/doc/src/string.xml +++ b/lib/stdlib/doc/src/string.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1996</year><year>2017</year> + <year>1996</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/stdlib/doc/src/sys.xml b/lib/stdlib/doc/src/sys.xml index 59e5bb6cb5..9fe816e33a 100644 --- a/lib/stdlib/doc/src/sys.xml +++ b/lib/stdlib/doc/src/sys.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1996</year><year>2017</year> + <year>1996</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/stdlib/doc/src/timer.xml b/lib/stdlib/doc/src/timer.xml index 53107ade2c..e913e33589 100644 --- a/lib/stdlib/doc/src/timer.xml +++ b/lib/stdlib/doc/src/timer.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1996</year><year>2016</year> + <year>1996</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/stdlib/include/assert.hrl b/lib/stdlib/include/assert.hrl index 2ec89e7d8a..28d25c6589 100644 --- a/lib/stdlib/include/assert.hrl +++ b/lib/stdlib/include/assert.hrl @@ -140,7 +140,7 @@ -endif. %% This is mostly a convenience which gives more detailed reports. -%% Note: Guard is a guarded pattern, and can not be used for value. +%% Note: Guard is a guarded pattern, and cannot be used for value. -ifdef(NOASSERT). -define(assertMatch(Guard, Expr), ok). -define(assertMatch(Guard, Expr, Comment), ok). @@ -289,7 +289,7 @@ end). -endif. -%% Note: Class and Term are patterns, and can not be used for value. +%% Note: Class and Term are patterns, and cannot be used for value. %% Term can be a guarded pattern, but Class cannot. -ifdef(NOASSERT). -define(assertException(Class, Term, Expr), ok). @@ -364,7 +364,7 @@ ?assertException(throw, Term, Expr, Comment)). %% This is the inverse case of assertException, for convenience. -%% Note: Class and Term are patterns, and can not be used for value. +%% Note: Class and Term are patterns, and cannot be used for value. %% Both Class and Term can be guarded patterns. -ifdef(NOASSERT). -define(assertNotException(Class, Term, Expr), ok). diff --git a/lib/stdlib/src/Makefile b/lib/stdlib/src/Makefile index dfe6bf3e68..c95f7637f7 100644 --- a/lib/stdlib/src/Makefile +++ b/lib/stdlib/src/Makefile @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 1996-2017. All Rights Reserved. +# Copyright Ericsson AB 1996-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. diff --git a/lib/stdlib/src/beam_lib.erl b/lib/stdlib/src/beam_lib.erl index 24349c74e8..01181b1097 100644 --- a/lib/stdlib/src/beam_lib.erl +++ b/lib/stdlib/src/beam_lib.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2000-2017. All Rights Reserved. +%% Copyright Ericsson AB 2000-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -690,30 +690,31 @@ chunk_to_data(debug_info=Id, Chunk, File, _Cs, AtomTable, Mod) -> <<0:8,N:8,Mode0:N/binary,Rest/binary>> -> Mode = binary_to_atom(Mode0, utf8), Term = decrypt_chunk(Mode, Mod, File, Id, Rest), - {AtomTable, {Id, Term}}; + {AtomTable, {Id, anno_from_term(Term)}}; _ -> case catch binary_to_term(Chunk) of {'EXIT', _} -> error({invalid_chunk, File, chunk_name_to_id(Id, File)}); Term -> - {AtomTable, {Id, Term}} + {AtomTable, {Id, anno_from_term(Term)}} end end; chunk_to_data(abstract_code=Id, Chunk, File, _Cs, AtomTable, Mod) -> + %% Before Erlang/OTP 20.0. case Chunk of <<>> -> {AtomTable, {Id, no_abstract_code}}; <<0:8,N:8,Mode0:N/binary,Rest/binary>> -> Mode = binary_to_atom(Mode0, utf8), Term = decrypt_chunk(Mode, Mod, File, Id, Rest), - {AtomTable, {Id, anno_from_term(Term)}}; + {AtomTable, {Id, old_anno_from_term(Term)}}; _ -> case catch binary_to_term(Chunk) of {'EXIT', _} -> error({invalid_chunk, File, chunk_name_to_id(Id, File)}); Term -> try - {AtomTable, {Id, anno_from_term(Term)}} + {AtomTable, {Id, old_anno_from_term(Term)}} catch _:_ -> error({invalid_chunk, File, @@ -947,14 +948,24 @@ decrypt_chunk(Type, Module, File, Id, Bin) -> error({key_missing_or_invalid, File, Id}) end. -anno_from_term({raw_abstract_v1, Forms}) -> +old_anno_from_term({raw_abstract_v1, Forms}) -> {raw_abstract_v1, anno_from_forms(Forms)}; -anno_from_term({Tag, Forms}) when Tag =:= abstract_v1; Tag =:= abstract_v2 -> +old_anno_from_term({Tag, Forms}) when Tag =:= abstract_v1; + Tag =:= abstract_v2 -> try {Tag, anno_from_forms(Forms)} catch _:_ -> {Tag, Forms} end; +old_anno_from_term(T) -> + T. + +anno_from_term({debug_info_v1=Tag1, erl_abstract_code=Tag2, {Forms, Opts}}) -> + try {Tag1, Tag2, {anno_from_forms(Forms), Opts}} + catch + _:_ -> + {Tag1, Tag2, {Forms, Opts}} + end; anno_from_term(T) -> T. diff --git a/lib/stdlib/src/binary.erl b/lib/stdlib/src/binary.erl index 7d0e42489e..52b9fedc9c 100644 --- a/lib/stdlib/src/binary.erl +++ b/lib/stdlib/src/binary.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2017. All Rights Reserved. +%% Copyright Ericsson AB 2010-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/stdlib/src/c.erl b/lib/stdlib/src/c.erl index 13f78841aa..0362b72536 100644 --- a/lib/stdlib/src/c.erl +++ b/lib/stdlib/src/c.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2017. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/stdlib/src/dets.erl b/lib/stdlib/src/dets.erl index e1a36abc70..e016d5a80e 100644 --- a/lib/stdlib/src/dets.erl +++ b/lib/stdlib/src/dets.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2017. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/stdlib/src/dets_utils.erl b/lib/stdlib/src/dets_utils.erl index 4c8ea9e82b..12394bd1ad 100644 --- a/lib/stdlib/src/dets_utils.erl +++ b/lib/stdlib/src/dets_utils.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2001-2017. All Rights Reserved. +%% Copyright Ericsson AB 2001-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/stdlib/src/edlin.erl b/lib/stdlib/src/edlin.erl index 5df9c504f9..f027d05f55 100644 --- a/lib/stdlib/src/edlin.erl +++ b/lib/stdlib/src/edlin.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/stdlib/src/erl_eval.erl b/lib/stdlib/src/erl_eval.erl index 31c0e60fe1..2066b2f60f 100644 --- a/lib/stdlib/src/erl_eval.erl +++ b/lib/stdlib/src/erl_eval.erl @@ -329,7 +329,8 @@ expr({'fun',Line,{clauses,Cs}} = Ex, Bs, Lf, Ef, RBs) -> 20 -> fun (A,B,C,D,E,F,G,H,I,J,K,L,M,N,O,P,Q,R,S,T) -> eval_fun([A,B,C,D,E,F,G,H,I,J,K,L,M,N,O,P,Q,R,S,T], Info) end; _Other -> - erlang:raise(error, {'argument_limit',{'fun',Line,Cs}}, + L = erl_anno:location(Line), + erlang:raise(error, {'argument_limit',{'fun',L,to_terms(Cs)}}, ?STACKTRACE) end, ret_expr(F, Bs, RBs); @@ -381,7 +382,9 @@ expr({named_fun,Line,Name,Cs} = Ex, Bs, Lf, Ef, RBs) -> eval_named_fun([A,B,C,D,E,F,G,H,I,J,K,L,M,N,O,P,Q,R,S,T], RF, Info) end; _Other -> - erlang:raise(error, {'argument_limit',{named_fun,Line,Name,Cs}}, + L = erl_anno:location(Line), + erlang:raise(error, {'argument_limit', + {named_fun,L,Name,to_terms(Cs)}}, ?STACKTRACE) end, ret_expr(F, Bs, RBs); @@ -1092,7 +1095,7 @@ match(Pat, Term, Bs) -> match(Pat, Term, Bs, BBs) -> case catch match1(Pat, Term, Bs, BBs) of invalid -> - erlang:raise(error, {illegal_pattern,Pat}, ?STACKTRACE); + erlang:raise(error, {illegal_pattern,to_term(Pat)}, ?STACKTRACE); Other -> Other end. @@ -1288,6 +1291,12 @@ merge_bindings(Bs1, Bs2) -> %% end %% end, Bs2, Bs1). +to_terms(Abstrs) -> + [to_term(Abstr) || Abstr <- Abstrs]. + +to_term(Abstr) -> + erl_parse:anno_to_term(Abstr). + %% Substitute {value, A, Item} for {var, A, Var}, preserving A. %% {value, A, Item} is a shell/erl_eval convention, and for example %% the linter cannot handle it. diff --git a/lib/stdlib/src/erl_internal.erl b/lib/stdlib/src/erl_internal.erl index dd509191ef..939abaff00 100644 --- a/lib/stdlib/src/erl_internal.erl +++ b/lib/stdlib/src/erl_internal.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1998-2017. All Rights Reserved. +%% Copyright Ericsson AB 1998-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -74,6 +74,7 @@ guard_bif(element, 2) -> true; guard_bif(float, 1) -> true; guard_bif(floor, 1) -> true; guard_bif(hd, 1) -> true; +guard_bif(is_map_key, 2) -> true; guard_bif(length, 1) -> true; guard_bif(map_size, 1) -> true; guard_bif(map_get, 2) -> true; @@ -109,7 +110,6 @@ new_type_test(is_function, 2) -> true; new_type_test(is_integer, 1) -> true; new_type_test(is_list, 1) -> true; new_type_test(is_map, 1) -> true; -new_type_test(is_map_key, 2) -> true; new_type_test(is_number, 1) -> true; new_type_test(is_pid, 1) -> true; new_type_test(is_port, 1) -> true; diff --git a/lib/stdlib/src/erl_parse.yrl b/lib/stdlib/src/erl_parse.yrl index 0c338b5952..9602f0bcd9 100644 --- a/lib/stdlib/src/erl_parse.yrl +++ b/lib/stdlib/src/erl_parse.yrl @@ -2,7 +2,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2017. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -980,7 +980,7 @@ Erlang code. -type af_unary_op(T) :: {'op', anno(), unary_op(), T}. --type unary_op() :: '+' | '*' | 'bnot' | 'not'. +-type unary_op() :: '+' | '-' | 'bnot' | 'not'. %% See also lib/stdlib/{src/erl_bits.erl,include/erl_bits.hrl}. -type type_specifier_list() :: 'default' | [type_specifier(), ...]. diff --git a/lib/stdlib/src/erl_posix_msg.erl b/lib/stdlib/src/erl_posix_msg.erl index 8959fea498..b9ed4a3a9d 100644 --- a/lib/stdlib/src/erl_posix_msg.erl +++ b/lib/stdlib/src/erl_posix_msg.erl @@ -81,9 +81,9 @@ message_1(el2hlt) -> <<"level 2 halted">>; message_1(el2nsync) -> <<"level 2 not synchronized">>; message_1(el3hlt) -> <<"level 3 halted">>; message_1(el3rst) -> <<"level 3 reset">>; -message_1(elibacc) -> <<"can not access a needed shared library">>; +message_1(elibacc) -> <<"cannot access a needed shared library">>; message_1(elibbad) -> <<"accessing a corrupted shared library">>; -message_1(elibexec) -> <<"can not exec a shared library directly">>; +message_1(elibexec) -> <<"cannot exec a shared library directly">>; message_1(elibmax) -> <<"attempting to link in more shared libraries than system limit">>; message_1(elibscn) -> <<".lib section in a.out corrupted">>; diff --git a/lib/stdlib/src/erl_pp.erl b/lib/stdlib/src/erl_pp.erl index 367dbefb82..dd302a2880 100644 --- a/lib/stdlib/src/erl_pp.erl +++ b/lib/stdlib/src/erl_pp.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2017. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -59,7 +59,7 @@ _ -> ?TEST(T) end). -define(EXPRS_TEST(L), - [?TEST(E) || E <- L]). + _ = [?TEST(E) || E <- L]). -define(TEST(T), %% Assumes that erl_anno has been compiled with DEBUG=true. %% erl_pp does not use the annoations, but test it anyway. diff --git a/lib/stdlib/src/escript.erl b/lib/stdlib/src/escript.erl index 89a81684f5..3f14894b55 100644 --- a/lib/stdlib/src/escript.erl +++ b/lib/stdlib/src/escript.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2017. All Rights Reserved. +%% Copyright Ericsson AB 2007-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/stdlib/src/ets.erl b/lib/stdlib/src/ets.erl index a35f79c0d9..29f907ad73 100644 --- a/lib/stdlib/src/ets.erl +++ b/lib/stdlib/src/ets.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2017. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/stdlib/src/file_sorter.erl b/lib/stdlib/src/file_sorter.erl index 7f74e71136..191e050538 100644 --- a/lib/stdlib/src/file_sorter.erl +++ b/lib/stdlib/src/file_sorter.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2001-2016. All Rights Reserved. +%% Copyright Ericsson AB 2001-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/stdlib/src/gen.erl b/lib/stdlib/src/gen.erl index 2e6223d2bb..a7f743bd4c 100644 --- a/lib/stdlib/src/gen.erl +++ b/lib/stdlib/src/gen.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2017. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/stdlib/src/gen_event.erl b/lib/stdlib/src/gen_event.erl index 3ee2031d02..8213282867 100644 --- a/lib/stdlib/src/gen_event.erl +++ b/lib/stdlib/src/gen_event.erl @@ -589,7 +589,7 @@ server_update(Handler1, Func, Event, SName) -> ?LOG_WARNING(#{label=>{gen_event,no_handle_info}, module=>Mod1, message=>Event}, - #{domain=>[beam,erlang,otp], + #{domain=>[otp], report_cb=>fun gen_event:format_log/1, error_logger=>#{tag=>warning_msg}}), % warningmap?? {ok, Handler1}; @@ -751,7 +751,7 @@ report_error(Handler, Reason, State, LastIn, SName) -> state=>format_status(terminate,Handler#handler.module, get(),State), reason=>Reason}, - #{domain=>[beam,erlang,otp], + #{domain=>[otp], report_cb=>fun gen_event:format_log/1, error_logger=>#{tag=>error}}). diff --git a/lib/stdlib/src/gen_fsm.erl b/lib/stdlib/src/gen_fsm.erl index 1646186761..caaaf8fa2e 100644 --- a/lib/stdlib/src/gen_fsm.erl +++ b/lib/stdlib/src/gen_fsm.erl @@ -505,7 +505,7 @@ handle_msg(Msg, Parent, Name, StateName, StateData, Mod, _Time, HibernateAfterTi ?LOG_WARNING(#{label=>{gen_fsm,no_handle_info}, module=>Mod, message=>Msg}, - #{domain=>[beam,erlang,otp], + #{domain=>[otp], report_cb=>fun gen_fsm:format_log/1, error_logger=>#{tag=>warning_msg}}), loop(Parent, Name, StateName, StateData, Mod, infinity, HibernateAfterTimeout, []); @@ -616,7 +616,7 @@ error_info(Reason, Name, Msg, StateName, StateData, Debug) -> state_name=>StateName, state_data=>StateData, reason=>Reason}, - #{domain=>[beam,erlang,otp], + #{domain=>[otp], report_cb=>fun gen_fsm:format_log/1, error_logger=>#{tag=>error}}), sys:print_log(Debug), diff --git a/lib/stdlib/src/gen_server.erl b/lib/stdlib/src/gen_server.erl index 09f77c0810..44e9231ebe 100644 --- a/lib/stdlib/src/gen_server.erl +++ b/lib/stdlib/src/gen_server.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2017. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -645,7 +645,7 @@ try_dispatch(Mod, Func, Msg, State) -> #{label=>{gen_server,no_handle_info}, module=>Mod, message=>Msg}, - #{domain=>[beam,erlang,otp], + #{domain=>[otp], report_cb=>fun gen_server:format_log/1, error_logger=>#{tag=>warning_msg}}), {ok, {noreply, State}}; @@ -891,7 +891,7 @@ error_info(Reason, Name, From, Msg, Mod, State, Debug) -> state=>format_status(terminate, Mod, get(), State), reason=>Reason, client_info=>client_stacktrace(From)}, - #{domain=>[beam,erlang,otp], + #{domain=>[otp], report_cb=>fun gen_server:format_log/1, error_logger=>#{tag=>error}}), sys:print_log(Debug), diff --git a/lib/stdlib/src/gen_statem.erl b/lib/stdlib/src/gen_statem.erl index b36b8cd5a5..faa43fbc1e 100644 --- a/lib/stdlib/src/gen_statem.erl +++ b/lib/stdlib/src/gen_statem.erl @@ -1448,13 +1448,13 @@ loop_event_done( [?sys_debug( Debug_0, {S#state.name,State}, - {postpone,Event_0,State}), + {postpone,Event_0,NextState}), Event_0|P_0]; false -> [?sys_debug( Debug_0, {S#state.name,State}, - {consume,Event_0,State})|P_0] + {consume,Event_0,NextState})|P_0] end, {Events_2,P_2,Timers_2} = %% Move all postponed events to queue, @@ -1900,7 +1900,7 @@ error_info( state_enter=>StateEnter, state=>format_status(terminate, get(), S), reason=>{Class,Reason,Stacktrace}}, - #{domain=>[beam,erlang,otp], + #{domain=>[otp], report_cb=>fun gen_statem:format_log/1, error_logger=>#{tag=>error}}). diff --git a/lib/stdlib/src/io.erl b/lib/stdlib/src/io.erl index 5d5773c80c..63c9a6bddf 100644 --- a/lib/stdlib/src/io.erl +++ b/lib/stdlib/src/io.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2015. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/stdlib/src/io_lib_pretty.erl b/lib/stdlib/src/io_lib_pretty.erl index 3d5a979b3e..ba9d9e8434 100644 --- a/lib/stdlib/src/io_lib_pretty.erl +++ b/lib/stdlib/src/io_lib_pretty.erl @@ -131,6 +131,8 @@ print(Term, Col, Ll, D, M0, T, RecDefFun, Enc, Str) when is_tuple(Term); %% use Len as CHAR_MAX if M0 = -1 M = max_cs(M0, Len), if + Ll =:= 0 -> + write(If); Len < Ll - Col, Len =< M -> %% write the whole thing on a single line when there is room write(If); @@ -720,7 +722,7 @@ printable_list(L, _D, T, latin1) when T < 0 -> io_lib:printable_latin1_list(L); printable_list(L, _D, T, Enc) when T >= 0 -> case slice(L, tsub(T, 2)) of - {prefix, ""} -> + false -> false; {prefix, Prefix} when Enc =:= latin1 -> io_lib:printable_latin1_list(Prefix) andalso {true, Prefix}; @@ -736,11 +738,17 @@ printable_list(L, _D, T, _Uni) when T < 0-> io_lib:printable_list(L). slice(L, N) -> - case string:length(L) =< N of + try string:length(L) =< N of true -> all; false -> - {prefix, string:slice(L, 0, N)} + case string:slice(L, 0, N) of + "" -> + false; + Prefix -> + {prefix, Prefix} + end + catch _:_ -> false end. printable_bin0(Bin, D, T, Enc) -> diff --git a/lib/stdlib/src/maps.erl b/lib/stdlib/src/maps.erl index a13f340709..51965ddb57 100644 --- a/lib/stdlib/src/maps.erl +++ b/lib/stdlib/src/maps.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2013-2016. All Rights Reserved. +%% Copyright Ericsson AB 2013-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -21,7 +21,7 @@ -module(maps). -export([get/3, filter/2,fold/3, - map/2, size/1, + map/2, size/1, new/0, update_with/3, update_with/4, without/2, with/2, iterator/1, next/1]). @@ -29,13 +29,15 @@ %% BIFs -export([get/2, find/2, from_list/1, is_key/2, keys/1, merge/2, - new/0, put/3, remove/2, take/2, + put/3, remove/2, take/2, to_list/1, update/3, values/1]). --opaque iterator() :: {term(), term(), iterator()} - | none | nonempty_improper_list(integer(),map()). +-opaque iterator(Key, Value) :: {Key, Value, iterator(Key, Value)} | none + | nonempty_improper_list(integer(), #{Key => Value}). --export_type([iterator/0]). +-type iterator() :: iterator(term(), term()). + +-export_type([iterator/2, iterator/0]). -dialyzer({no_improper_lists, iterator/1}). @@ -50,9 +52,7 @@ get(_,_) -> erlang:nif_error(undef). -spec find(Key,Map) -> {ok, Value} | error when - Key :: term(), - Map :: map(), - Value :: term(). + Map :: #{Key => Value, _ => _}. find(_,_) -> erlang:nif_error(undef). @@ -75,9 +75,8 @@ is_key(_,_) -> erlang:nif_error(undef). -spec keys(Map) -> Keys when - Map :: map(), - Keys :: [Key], - Key :: term(). + Map :: #{Key => _}, + Keys :: [Key]. keys(_) -> erlang:nif_error(undef). @@ -91,13 +90,6 @@ keys(_) -> erlang:nif_error(undef). merge(_,_) -> erlang:nif_error(undef). - --spec new() -> Map when - Map :: map(). - -new() -> erlang:nif_error(undef). - - %% Shadowed by erl_bif_types: maps:put/3 -spec put(Key,Value,Map1) -> Map2 when Key :: term(), @@ -116,17 +108,13 @@ put(_,_,_) -> erlang:nif_error(undef). remove(_,_) -> erlang:nif_error(undef). -spec take(Key,Map1) -> {Value,Map2} | error when - Key :: term(), - Map1 :: map(), - Value :: term(), - Map2 :: map(). + Map1 :: #{Key => Value, _ => _}, + Map2 :: #{_ => _}. take(_,_) -> erlang:nif_error(undef). -spec to_list(Map) -> [{Key,Value}] when - Map :: map(), - Key :: term(), - Value :: term(). + Map :: #{Key => Value}. to_list(Map) when is_map(Map) -> to_list_internal(erts_internal:map_next(0, Map, [])); @@ -140,79 +128,69 @@ to_list_internal(Acc) -> %% Shadowed by erl_bif_types: maps:update/3 -spec update(Key,Value,Map1) -> Map2 when - Key :: term(), - Value :: term(), - Map1 :: map(), - Map2 :: map(). + Map1 :: #{Key := _, _ => _}, + Map2 :: #{Key := Value, _ => _}. update(_,_,_) -> erlang:nif_error(undef). -spec values(Map) -> Values when - Map :: map(), - Values :: [Value], - Value :: term(). + Map :: #{_ => Value}, + Values :: [Value]. values(_) -> erlang:nif_error(undef). %% End of BIFs +-spec new() -> Map when + Map :: #{}. + +new() -> #{}. + -spec update_with(Key,Fun,Map1) -> Map2 when - Key :: term(), - Map1 :: map(), - Map2 :: map(), - Fun :: fun((Value1 :: term()) -> Value2 :: term()). + Map1 :: #{Key := Value1, _ => _}, + Map2 :: #{Key := Value2, _ => _}, + Fun :: fun((Value1) -> Value2). update_with(Key,Fun,Map) when is_function(Fun,1), is_map(Map) -> - try maps:get(Key,Map) of - Val -> maps:update(Key,Fun(Val),Map) - catch - error:{badkey,_} -> - erlang:error({badkey,Key},[Key,Fun,Map]) + case Map of + #{Key := Value} -> Map#{Key := Fun(Value)}; + #{} -> erlang:error({badkey,Key},[Key,Fun,Map]) end; update_with(Key,Fun,Map) -> erlang:error(error_type(Map),[Key,Fun,Map]). -spec update_with(Key,Fun,Init,Map1) -> Map2 when - Key :: term(), - Map1 :: Map1, - Map2 :: Map2, - Fun :: fun((Value1 :: term()) -> Value2 :: term()), - Init :: term(). + Map1 :: #{Key => Value1, _ => _}, + Map2 :: #{Key := Value2 | Init, _ => _}, + Fun :: fun((Value1) -> Value2). update_with(Key,Fun,Init,Map) when is_function(Fun,1), is_map(Map) -> - case maps:find(Key,Map) of - {ok,Val} -> maps:update(Key,Fun(Val),Map); - error -> maps:put(Key,Init,Map) + case Map of + #{Key := Value} -> Map#{Key := Fun(Value)}; + #{} -> Map#{Key => Init} end; update_with(Key,Fun,Init,Map) -> erlang:error(error_type(Map),[Key,Fun,Init,Map]). -spec get(Key, Map, Default) -> Value | Default when - Key :: term(), - Map :: map(), - Value :: term(), - Default :: term(). + Map :: #{Key => Value, _ => _}. get(Key,Map,Default) when is_map(Map) -> - case maps:find(Key, Map) of - {ok, Value} -> - Value; - error -> - Default + case Map of + #{Key := Value} -> Value; + #{} -> Default end; get(Key,Map,Default) -> erlang:error({badmap,Map},[Key,Map,Default]). --spec filter(Pred,MapOrIter) -> Map when +-spec filter(Pred, MapOrIter) -> Map when Pred :: fun((Key, Value) -> boolean()), - Key :: term(), - Value :: term(), - MapOrIter :: map() | iterator(), - Map :: map(). + MapOrIter :: #{Key => Value} | iterator(Key, Value), + Map :: #{Key => Value}. filter(Pred,Map) when is_function(Pred,2), is_map(Map) -> maps:from_list(filter_1(Pred, iterator(Map))); @@ -235,21 +213,18 @@ filter_1(Pred, Iter) -> end. -spec fold(Fun,Init,MapOrIter) -> Acc when - Fun :: fun((K, V, AccIn) -> AccOut), + Fun :: fun((Key, Value, AccIn) -> AccOut), Init :: term(), - Acc :: term(), - AccIn :: term(), - AccOut :: term(), - MapOrIter :: map() | iterator(), - K :: term(), - V :: term(). + Acc :: AccOut, + AccIn :: Init | AccOut, + MapOrIter :: #{Key => Value} | iterator(Key, Value). fold(Fun,Init,Map) when is_function(Fun,3), is_map(Map) -> fold_1(Fun,Init,iterator(Map)); fold(Fun,Init,Iterator) when is_function(Fun,3), ?IS_ITERATOR(Iterator) -> fold_1(Fun,Init,Iterator); fold(Fun,Init,Map) -> - erlang:error(error_type(Map),[Fun,Init,Map]). + erlang:error(error_type_iter(Map),[Fun,Init,Map]). fold_1(Fun, Acc, Iter) -> case next(Iter) of @@ -260,19 +235,16 @@ fold_1(Fun, Acc, Iter) -> end. -spec map(Fun,MapOrIter) -> Map when - Fun :: fun((K, V1) -> V2), - MapOrIter :: map() | iterator(), - Map :: map(), - K :: term(), - V1 :: term(), - V2 :: term(). + Fun :: fun((Key, Value1) -> Value2), + MapOrIter :: #{Key => Value1} | iterator(Key, Value1), + Map :: #{Key => Value2}. map(Fun,Map) when is_function(Fun, 2), is_map(Map) -> maps:from_list(map_1(Fun, iterator(Map))); map(Fun,Iterator) when is_function(Fun, 2), ?IS_ITERATOR(Iterator) -> maps:from_list(map_1(Fun, Iterator)); map(Fun,Map) -> - erlang:error(error_type(Map),[Fun,Map]). + erlang:error(error_type_iter(Map),[Fun,Map]). map_1(Fun, Iter) -> case next(Iter) of @@ -291,17 +263,15 @@ size(Val) -> erlang:error({badmap,Val},[Val]). -spec iterator(Map) -> Iterator when - Map :: map(), - Iterator :: iterator(). + Map :: #{Key => Value}, + Iterator :: iterator(Key, Value). iterator(M) when is_map(M) -> [0 | M]; iterator(M) -> erlang:error({badmap, M}, [M]). -spec next(Iterator) -> {Key, Value, NextIterator} | 'none' when - Iterator :: iterator(), - Key :: term(), - Value :: term(), - NextIterator :: iterator(). + Iterator :: iterator(Key, Value), + NextIterator :: iterator(Key, Value). next({K, V, I}) -> {K, V, I}; next([Path | Map]) when is_integer(Path), is_map(Map) -> @@ -318,29 +288,29 @@ next(Iter) -> K :: term(). without(Ks,M) when is_list(Ks), is_map(M) -> - lists:foldl(fun(K, M1) -> maps:remove(K, M1) end, M, Ks); + lists:foldl(fun maps:remove/2, M, Ks); without(Ks,M) -> erlang:error(error_type(M),[Ks,M]). -spec with(Ks, Map1) -> Map2 when Ks :: [K], - Map1 :: map(), - Map2 :: map(), - K :: term(). + Map1 :: #{K => V, _ => _}, + Map2 :: #{K => V}. with(Ks,Map1) when is_list(Ks), is_map(Map1) -> - Fun = fun(K, List) -> - case maps:find(K, Map1) of - {ok, V} -> - [{K, V} | List]; - error -> - List - end - end, - maps:from_list(lists:foldl(Fun, [], Ks)); + maps:from_list(with_1(Ks, Map1)); with(Ks,M) -> erlang:error(error_type(M),[Ks,M]). +with_1([K|Ks], Map) -> + case Map of + #{K := V} -> [{K,V}|with_1(Ks, Map)]; + #{} -> with_1(Ks, Map) + end; +with_1([], _Map) -> []. -error_type(M) when is_map(M); ?IS_ITERATOR(M) -> badarg; +error_type(M) when is_map(M) -> badarg; error_type(V) -> {badmap, V}. + +error_type_iter(M) when is_map(M); ?IS_ITERATOR(M) -> badarg; +error_type_iter(V) -> {badmap, V}. diff --git a/lib/stdlib/src/ms_transform.erl b/lib/stdlib/src/ms_transform.erl index 428c23524b..d117481d2e 100644 --- a/lib/stdlib/src/ms_transform.erl +++ b/lib/stdlib/src/ms_transform.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2002-2017. All Rights Reserved. +%% Copyright Ericsson AB 2002-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/stdlib/src/ordsets.erl b/lib/stdlib/src/ordsets.erl index 939e147ad8..176047079b 100644 --- a/lib/stdlib/src/ordsets.erl +++ b/lib/stdlib/src/ordsets.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/stdlib/src/otp_internal.erl b/lib/stdlib/src/otp_internal.erl index ceec3079a1..aaed13ba3a 100644 --- a/lib/stdlib/src/otp_internal.erl +++ b/lib/stdlib/src/otp_internal.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2017. All Rights Reserved. +%% Copyright Ericsson AB 1999-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -620,6 +620,8 @@ obsolete_1(ssl, ssl_accept, 2) -> {deprecated, "deprecated; use ssl:handshake/2 instead"}; obsolete_1(ssl, ssl_accept, 3) -> {deprecated, "deprecated; use ssl:handshake/3 instead"}; +obsolete_1(otp_mib, F, _) when F =:= load; F =:= unload -> + {deprecated, "deprecated; functionality will be removed in a future release"}; %% not obsolete diff --git a/lib/stdlib/src/pool.erl b/lib/stdlib/src/pool.erl index b12ff205b1..599be55607 100644 --- a/lib/stdlib/src/pool.erl +++ b/lib/stdlib/src/pool.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/stdlib/src/proc_lib.erl b/lib/stdlib/src/proc_lib.erl index 5f14e78f91..d07c62500b 100644 --- a/lib/stdlib/src/proc_lib.erl +++ b/lib/stdlib/src/proc_lib.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2017. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -30,7 +30,7 @@ start/3, start/4, start/5, start_link/3, start_link/4, start_link/5, hibernate/3, init_ack/1, init_ack/2, - init_p/3,init_p/5,format/1,format/2,format/3,report_cb/1, + init_p/3,init_p/5,format/1,format/2,format/3,report_cb/2, initial_call/1, translate_initial_call/1, stop/1, stop/3]). @@ -508,8 +508,8 @@ crash_report(Class, Reason, StartF, Stacktrace) -> ?LOG_ERROR(#{label=>{proc_lib,crash}, report=>[my_info(Class, Reason, StartF, Stacktrace), linked_info(self())]}, - #{domain=>[beam,erlang,otp,sasl], - report_cb=>fun proc_lib:report_cb/1, + #{domain=>[otp,sasl], + report_cb=>fun proc_lib:report_cb/2, logger_formatter=>#{title=>"CRASH REPORT"}, error_logger=>#{tag=>error_report,type=>crash_report}}). @@ -750,14 +750,15 @@ check(Res) -> Res. %%% Format a generated crash info structure. %%% ----------------------------------------------------------- --spec report_cb(CrashReport) -> {Format,Args} when - CrashReport :: #{label=>{proc_lib,crash},report=>[term()]}, - Format :: io:format(), - Args :: [term()]. -report_cb(#{label:={proc_lib,crash}, - report:=CrashReport}) -> - Depth = error_logger:get_format_depth(), - get_format_and_args(CrashReport, utf8, Depth). +-spec report_cb(CrashReport,FormatOpts) -> unicode:chardata() when + CrashReport :: #{label => {proc_lib,crash}, + report => [term()]}, + FormatOpts :: logger:report_cb_config(). +report_cb(#{label:={proc_lib,crash}, report:=CrashReport}, Extra) -> + Default = #{chars_limit => unlimited, + depth => unlimited, + encoding => latin1}, + do_format(CrashReport, maps:merge(Default,Extra)). -spec format(CrashReport) -> string() when CrashReport :: [term()]. @@ -777,66 +778,47 @@ format(CrashReport, Encoding) -> Depth :: unlimited | pos_integer(). format(CrashReport, Encoding, Depth) -> - {F,A} = get_format_and_args(CrashReport, Encoding, Depth), - lists:flatten(io_lib:format(F,A)). + do_format(CrashReport, #{chars_limit => unlimited, + depth => Depth, + encoding => Encoding}). -get_format_and_args([OwnReport,LinkReport], Encoding, Depth) -> - Extra = {Encoding,Depth}, +do_format([OwnReport,LinkReport], Extra) -> MyIndent = " ", - {OwnFormat,OwnArgs} = format_report(OwnReport, MyIndent, Extra, [], []), - {LinkFormat,LinkArgs} = format_link_report(LinkReport, MyIndent, Extra, [], []), - {" crasher:~n"++OwnFormat++" neighbours:~n"++LinkFormat,OwnArgs++LinkArgs}. + OwnFormat = format_report(OwnReport, MyIndent, Extra), + LinkFormat = format_link_report(LinkReport, MyIndent, Extra), + Str = io_lib:format(" crasher:~n~ts neighbours:~n~ts", + [OwnFormat, LinkFormat]), + lists:flatten(Str). -format_link_report([], _Indent, _Extra, Format, Args) -> - {lists:flatten(lists:reverse(Format)),lists:append(lists:reverse(Args))}; -format_link_report([Link|Reps], Indent, Extra, Format, Args) -> +format_link_report([Link|Reps], Indent, Extra) -> Rep = case Link of {neighbour,Rep0} -> Rep0; _ -> Link end, LinkIndent = [" ",Indent], - {LinkFormat,LinkArgs} = format_report(Rep, LinkIndent, Extra, [], []), - F = "~sneighbour:\n"++LinkFormat, - A = [Indent|LinkArgs], - format_link_report(Reps, Indent, Extra, [F|Format], [A|Args]); -format_link_report(Rep, Indent, Extra, Format, Args) -> - {F,A} = format_report(Rep, Indent, Extra, [], []), - format_link_report([], Indent, Extra, [F|Format],[A|Args]). - -format_report([], _Indent, _Extra, Format, Args) -> - {lists:flatten(lists:reverse(Format)),lists:append(lists:reverse(Args))}; -format_report([Rep|Reps], Indent, Extra, Format, Args) -> - {F,A} = format_rep(Rep, Indent, Extra), - format_report(Reps, Indent, Extra, [F|Format], [A|Args]); -format_report(Rep, Indent, {Enc,unlimited}=Extra, Format, Args) -> - {F,A} = {"~s~"++modifier(Enc)++"p~n", [Indent, Rep]}, - format_report([], Indent, Extra, [F|Format], [A|Args]); -format_report(Rep, Indent, {Enc,Depth}=Extra, Format, Args) -> - {F,A} = {"~s~"++modifier(Enc)++"P~n", [Indent, Rep, Depth]}, - format_report([], Indent, Extra, [F|Format], [A|Args]). - -format_rep({initial_call,InitialCall}, Indent, Extra) -> - format_mfa(Indent, InitialCall, Extra); -format_rep({error_info,{Class,Reason,StackTrace}}, _Indent, Extra) -> - {lists:flatten(format_exception(Class, Reason, StackTrace, Extra)),[]}; -format_rep({Tag,Data}, Indent, Extra) -> - format_tag(Indent, Tag, Data, Extra). - -format_mfa(Indent, {M,F,Args}=StartF, {Enc,_}=Extra) -> - try - A = length(Args), - {lists:flatten([Indent,"initial call: ",atom_to_list(M), - $:,to_string(F, Enc),$/,integer_to_list(A),"\n"]),[]} - catch - error:_ -> - format_tag(Indent, initial_call, StartF, Extra) - end. - -format_tag(Indent, Tag, Data, {Enc,Depth}) -> - {P,Tl} = p(Enc, Depth), - {"~s~p: ~80.18" ++ P ++ "\n", [Indent, Tag, Data|Tl]}. + [Indent,"neighbour:\n",format_report(Rep, LinkIndent, Extra)| + format_link_report(Reps, Indent, Extra)]; +format_link_report(Rep, Indent, Extra) -> + format_report(Rep, Indent, Extra). + +format_report(Rep, Indent, Extra) when is_list(Rep) -> + format_rep(Rep, Indent, Extra); +format_report(Rep, Indent, #{encoding:=Enc,depth:=unlimited}) -> + io_lib:format("~s~"++modifier(Enc)++"p~n", [Indent, Rep]); +format_report(Rep, Indent, #{encoding:=Enc,depth:=Depth}) -> + io_lib:format("~s~"++modifier(Enc)++"P~n", [Indent, Rep, Depth]). + +format_rep([{initial_call,InitialCall}|Rep], Indent, Extra) -> + [format_mfa(Indent, InitialCall, Extra)|format_rep(Rep, Indent, Extra)]; +format_rep([{error_info,{Class,Reason,StackTrace}}|Rep], Indent, Extra) -> + [format_exception(Class, Reason, StackTrace, Extra)| + format_rep(Rep, Indent, Extra)]; +format_rep([{Tag,Data}|Rep], Indent, Extra) -> + [format_tag(Indent, Tag, Data, Extra)|format_rep(Rep, Indent, Extra)]; +format_rep(_, _, _Extra) -> + []. -format_exception(Class, Reason, StackTrace, {Enc,_}=Extra) -> +format_exception(Class, Reason, StackTrace, #{encoding:=Enc}=Extra) -> PF = pp_fun(Extra), StackFun = fun(M, _F, _A) -> (M =:= erl_eval) or (M =:= ?MODULE) end, %% EI = " exception: ", @@ -844,17 +826,37 @@ format_exception(Class, Reason, StackTrace, {Enc,_}=Extra) -> [EI, erl_error:format_exception(1+length(EI), Class, Reason, StackTrace, StackFun, PF, Enc), "\n"]. +format_mfa(Indent, {M,F,Args}=StartF, #{encoding:=Enc}=Extra) -> + try + A = length(Args), + [Indent,"initial call: ",atom_to_list(M),$:,to_string(F, Enc),$/, + integer_to_list(A),"\n"] + catch + error:_ -> + format_tag(Indent, initial_call, StartF, Extra) + end. + to_string(A, latin1) -> io_lib:write_atom_as_latin1(A); to_string(A, _) -> io_lib:write_atom(A). -pp_fun({Enc,Depth}) -> +pp_fun(#{encoding:=Enc,depth:=Depth,chars_limit:=Limit}) -> {P,Tl} = p(Enc, Depth), + Opts = if is_integer(Limit) -> [{chars_limit,Limit}]; + true -> [] + end, fun(Term, I) -> - io_lib:format("~." ++ integer_to_list(I) ++ P, [Term|Tl]) + io_lib:format("~." ++ integer_to_list(I) ++ P, [Term|Tl], Opts) end. +format_tag(Indent, Tag, Data, #{encoding:=Enc,depth:=Depth,chars_limit:=Limit}) -> + {P,Tl} = p(Enc, Depth), + Opts = if is_integer(Limit) -> [{chars_limit,Limit}]; + true -> [] + end, + io_lib:format("~s~p: ~80.18" ++ P ++ "\n", [Indent, Tag, Data|Tl], Opts). + p(Encoding, Depth) -> {Letter, Tl} = case Depth of unlimited -> {"p", []}; diff --git a/lib/stdlib/src/qlc.erl b/lib/stdlib/src/qlc.erl index 4a0e976ba4..a1c1117e31 100644 --- a/lib/stdlib/src/qlc.erl +++ b/lib/stdlib/src/qlc.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2017. All Rights Reserved. +%% Copyright Ericsson AB 2004-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/stdlib/src/rand.erl b/lib/stdlib/src/rand.erl index 362e98006e..4951dc727b 100644 --- a/lib/stdlib/src/rand.erl +++ b/lib/stdlib/src/rand.erl @@ -486,7 +486,7 @@ uniform_real_s(Alg, Next, M0, BitNo, R1, V1, Bits) -> {M1 * math:pow(2.0, BitNo - 56), {Alg, R1}}; BitNo =:= -1008 -> %% Endgame - %% For the last round we can not have 14 zeros or more + %% For the last round we cannot have 14 zeros or more %% at the top of M1 because then we will underflow, %% so we need at least 43 bits if diff --git a/lib/stdlib/src/sets.erl b/lib/stdlib/src/sets.erl index ac0fc80526..8adb9016e2 100644 --- a/lib/stdlib/src/sets.erl +++ b/lib/stdlib/src/sets.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2000-2016. All Rights Reserved. +%% Copyright Ericsson AB 2000-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/stdlib/src/slave.erl b/lib/stdlib/src/slave.erl index 37c1f6bfd9..5e8c1a43ea 100644 --- a/lib/stdlib/src/slave.erl +++ b/lib/stdlib/src/slave.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2017. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/stdlib/src/string.erl b/lib/stdlib/src/string.erl index cf48b882e4..2939e78d9d 100644 --- a/lib/stdlib/src/string.erl +++ b/lib/stdlib/src/string.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2017. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/stdlib/src/supervisor.erl b/lib/stdlib/src/supervisor.erl index eb46ac611a..1ac7334830 100644 --- a/lib/stdlib/src/supervisor.erl +++ b/lib/stdlib/src/supervisor.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2017. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -43,7 +43,7 @@ {errorContext,Error}, {reason,Reason}, {offender,extract_child(Child)}]}, - #{domain=>[beam,erlang,otp,sasl], + #{domain=>[otp,sasl], report_cb=>fun logger:format_otp_report/1, logger_formatter=>#{title=>"SUPERVISOR REPORT"}, error_logger=>#{tag=>error_report, @@ -580,7 +580,7 @@ handle_info({'EXIT', Pid, Reason}, State) -> handle_info(Msg, State) -> ?LOG_ERROR("Supervisor received unexpected message: ~tp~n",[Msg], - #{domain=>[beam,erlang,otp], + #{domain=>[otp], error_logger=>#{tag=>error}}), {noreply, State}. @@ -1419,7 +1419,7 @@ report_progress(Child, SupName) -> ?LOG_INFO(#{label=>{supervisor,progress}, report=>[{supervisor,SupName}, {started,extract_child(Child)}]}, - #{domain=>[beam,erlang,otp,sasl], + #{domain=>[otp,sasl], report_cb=>fun logger:format_otp_report/1, logger_formatter=>#{title=>"PROGRESS REPORT"}, error_logger=>#{tag=>info_report,type=>progress}}). diff --git a/lib/stdlib/src/supervisor_bridge.erl b/lib/stdlib/src/supervisor_bridge.erl index 39372935fa..21ba6f53af 100644 --- a/lib/stdlib/src/supervisor_bridge.erl +++ b/lib/stdlib/src/supervisor_bridge.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -135,7 +135,7 @@ report_progress(Pid, Mod, StartArgs, SupName) -> report=>[{supervisor, SupName}, {started, [{pid, Pid}, {mfa, {Mod, init, [StartArgs]}}]}]}, - #{domain=>[beam,erlang,otp,sasl], + #{domain=>[otp,sasl], report_cb=>fun logger:format_otp_report/1, logger_formatter=>#{title=>"PROGRESS REPORT"}, error_logger=>#{tag=>info_report,type=>progress}}). @@ -146,7 +146,7 @@ report_error(Error, Reason, #state{name = Name, pid = Pid, mod = Mod}) -> {errorContext, Error}, {reason, Reason}, {offender, [{pid, Pid}, {mod, Mod}]}]}, - #{domain=>[beam,erlang,otp,sasl], + #{domain=>[otp,sasl], report_cb=>fun logger:format_otp_report/1, logger_formatter=>#{title=>"SUPERVISOR REPORT"}, error_logger=>#{tag=>error_report,type=>supervisor_report}}). diff --git a/lib/stdlib/src/uri_string.erl b/lib/stdlib/src/uri_string.erl index 28d36ea229..d33dc89af8 100644 --- a/lib/stdlib/src/uri_string.erl +++ b/lib/stdlib/src/uri_string.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2017. All Rights Reserved. +%% Copyright Ericsson AB 2017-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -297,7 +297,10 @@ NormalizedURI :: uri_string() | error(). normalize(URIMap) -> - normalize(URIMap, []). + try normalize(URIMap, []) + catch + throw:{error, Atom, RestData} -> {error, Atom, RestData} + end. -spec normalize(URI, Options) -> NormalizedURI when @@ -412,7 +415,7 @@ transcode(URIString, Options) when is_list(URIString) -> %% (application/x-www-form-urlencoded encoding algorithm) %%------------------------------------------------------------------------- -spec compose_query(QueryList) -> QueryString when - QueryList :: [{unicode:chardata(), unicode:chardata()}], + QueryList :: [{unicode:chardata(), unicode:chardata() | true}], QueryString :: uri_string() | error(). compose_query(List) -> @@ -420,7 +423,7 @@ compose_query(List) -> -spec compose_query(QueryList, Options) -> QueryString when - QueryList :: [{unicode:chardata(), unicode:chardata()}], + QueryList :: [{unicode:chardata(), unicode:chardata() | true}], Options :: [{encoding, atom()}], QueryString :: uri_string() | error(). @@ -432,6 +435,11 @@ compose_query(List, Options) -> throw:{error, Atom, RestData} -> {error, Atom, RestData} end. %% +compose_query([{Key,true}|Rest], Options, IsList, Acc) -> + Separator = get_separator(Rest), + K = form_urlencode(Key, Options), + IsListNew = IsList orelse is_list(Key), + compose_query(Rest, Options, IsListNew, <<Acc/binary,K/binary,Separator/binary>>); compose_query([{Key,Value}|Rest], Options, IsList, Acc) -> Separator = get_separator(Rest), K = form_urlencode(Key, Options), @@ -451,7 +459,7 @@ compose_query([], _Options, IsList, Acc) -> %%------------------------------------------------------------------------- -spec dissect_query(QueryString) -> QueryList when QueryString :: uri_string(), - QueryList :: [{unicode:chardata(), unicode:chardata()}] + QueryList :: [{unicode:chardata(), unicode:chardata() | true}] | error(). dissect_query(<<>>) -> []; @@ -523,34 +531,34 @@ parse_relative_part(?STRING_REST("//", Rest), URI) -> {T, URI1} -> Userinfo = calculate_parsed_userinfo(Rest, T), URI2 = maybe_add_path(URI1), - URI2#{userinfo => decode_userinfo(Userinfo)} + URI2#{userinfo => Userinfo} catch throw:{_,_,_} -> {T, URI1} = parse_host(Rest, URI), Host = calculate_parsed_host_port(Rest, T), URI2 = maybe_add_path(URI1), - URI2#{host => decode_host(remove_brackets(Host))} + URI2#{host => remove_brackets(Host)} end; parse_relative_part(?STRING_REST($/, Rest), URI) -> {T, URI1} = parse_segment(Rest, URI), % path-absolute Path = calculate_parsed_part(Rest, T), - URI1#{path => decode_path(?STRING_REST($/, Path))}; + URI1#{path => ?STRING_REST($/, Path)}; parse_relative_part(?STRING_REST($?, Rest), URI) -> {T, URI1} = parse_query(Rest, URI), % path-empty ?query Query = calculate_parsed_query_fragment(Rest, T), URI2 = maybe_add_path(URI1), - URI2#{query => decode_query(Query)}; + URI2#{query => Query}; parse_relative_part(?STRING_REST($#, Rest), URI) -> {T, URI1} = parse_fragment(Rest, URI), % path-empty Fragment = calculate_parsed_query_fragment(Rest, T), URI2 = maybe_add_path(URI1), - URI2#{fragment => decode_fragment(Fragment)}; + URI2#{fragment => Fragment}; parse_relative_part(?STRING_REST(Char, Rest), URI) -> case is_segment_nz_nc(Char) of true -> {T, URI1} = parse_segment_nz_nc(Rest, URI), % path-noscheme Path = calculate_parsed_part(Rest, T), - URI1#{path => decode_path(?STRING_REST(Char, Path))}; + URI1#{path => ?STRING_REST(Char, Path)}; false -> throw({error,invalid_uri,[Char]}) end. @@ -593,11 +601,11 @@ parse_segment(?STRING_REST($/, Rest), URI) -> parse_segment(?STRING_REST($?, Rest), URI) -> {T, URI1} = parse_query(Rest, URI), % ?query Query = calculate_parsed_query_fragment(Rest, T), - {Rest, URI1#{query => decode_query(Query)}}; + {Rest, URI1#{query => Query}}; parse_segment(?STRING_REST($#, Rest), URI) -> {T, URI1} = parse_fragment(Rest, URI), Fragment = calculate_parsed_query_fragment(Rest, T), - {Rest, URI1#{fragment => decode_fragment(Fragment)}}; + {Rest, URI1#{fragment => Fragment}}; parse_segment(?STRING_REST(Char, Rest), URI) -> case is_pchar(Char) of true -> parse_segment(Rest, URI); @@ -616,11 +624,11 @@ parse_segment_nz_nc(?STRING_REST($/, Rest), URI) -> parse_segment_nz_nc(?STRING_REST($?, Rest), URI) -> {T, URI1} = parse_query(Rest, URI), % ?query Query = calculate_parsed_query_fragment(Rest, T), - {Rest, URI1#{query => decode_query(Query)}}; + {Rest, URI1#{query => Query}}; parse_segment_nz_nc(?STRING_REST($#, Rest), URI) -> {T, URI1} = parse_fragment(Rest, URI), Fragment = calculate_parsed_query_fragment(Rest, T), - {Rest, URI1#{fragment => decode_fragment(Fragment)}}; + {Rest, URI1#{fragment => Fragment}}; parse_segment_nz_nc(?STRING_REST(Char, Rest), URI) -> case is_segment_nz_nc(Char) of true -> parse_segment_nz_nc(Rest, URI); @@ -709,31 +717,31 @@ parse_hier(?STRING_REST("//", Rest), URI) -> try parse_userinfo(Rest, URI) of {T, URI1} -> Userinfo = calculate_parsed_userinfo(Rest, T), - {Rest, URI1#{userinfo => decode_userinfo(Userinfo)}} + {Rest, URI1#{userinfo => Userinfo}} catch throw:{_,_,_} -> {T, URI1} = parse_host(Rest, URI), Host = calculate_parsed_host_port(Rest, T), - {Rest, URI1#{host => decode_host(remove_brackets(Host))}} + {Rest, URI1#{host => remove_brackets(Host)}} end; parse_hier(?STRING_REST($/, Rest), URI) -> {T, URI1} = parse_segment(Rest, URI), % path-absolute Path = calculate_parsed_part(Rest, T), - {Rest, URI1#{path => decode_path(?STRING_REST($/, Path))}}; + {Rest, URI1#{path => ?STRING_REST($/, Path)}}; parse_hier(?STRING_REST($?, Rest), URI) -> {T, URI1} = parse_query(Rest, URI), % path-empty ?query Query = calculate_parsed_query_fragment(Rest, T), - {Rest, URI1#{query => decode_query(Query)}}; + {Rest, URI1#{query => Query}}; parse_hier(?STRING_REST($#, Rest), URI) -> {T, URI1} = parse_fragment(Rest, URI), % path-empty Fragment = calculate_parsed_query_fragment(Rest, T), - {Rest, URI1#{fragment => decode_fragment(Fragment)}}; + {Rest, URI1#{fragment => Fragment}}; parse_hier(?STRING_REST(Char, Rest), URI) -> % path-rootless case is_pchar(Char) of true -> % segment_nz {T, URI1} = parse_segment(Rest, URI), Path = calculate_parsed_part(Rest, T), - {Rest, URI1#{path => decode_path(?STRING_REST(Char, Path))}}; + {Rest, URI1#{path => ?STRING_REST(Char, Path)}}; false -> throw({error,invalid_uri,[Char]}) end; parse_hier(?STRING_EMPTY, URI) -> @@ -770,7 +778,7 @@ parse_userinfo(?CHAR($@), URI) -> parse_userinfo(?STRING_REST($@, Rest), URI) -> {T, URI1} = parse_host(Rest, URI), Host = calculate_parsed_host_port(Rest, T), - {Rest, URI1#{host => decode_host(remove_brackets(Host))}}; + {Rest, URI1#{host => remove_brackets(Host)}}; parse_userinfo(?STRING_REST(Char, Rest), URI) -> case is_userinfo(Char) of true -> parse_userinfo(Rest, URI); @@ -836,20 +844,25 @@ parse_host(?STRING_REST($:, Rest), URI) -> parse_host(?STRING_REST($/, Rest), URI) -> {T, URI1} = parse_segment(Rest, URI), % path-abempty Path = calculate_parsed_part(Rest, T), - {Rest, URI1#{path => decode_path(?STRING_REST($/, Path))}}; + {Rest, URI1#{path => ?STRING_REST($/, Path)}}; parse_host(?STRING_REST($?, Rest), URI) -> {T, URI1} = parse_query(Rest, URI), % path-empty ?query Query = calculate_parsed_query_fragment(Rest, T), - {Rest, URI1#{query => decode_query(Query)}}; + {Rest, URI1#{query => Query}}; parse_host(?STRING_REST($[, Rest), URI) -> parse_ipv6_bin(Rest, [], URI); parse_host(?STRING_REST($#, Rest), URI) -> {T, URI1} = parse_fragment(Rest, URI), % path-empty Fragment = calculate_parsed_query_fragment(Rest, T), - {Rest, URI1#{fragment => decode_fragment(Fragment)}}; + {Rest, URI1#{fragment => Fragment}}; parse_host(?STRING_REST(Char, Rest), URI) -> case is_digit(Char) of - true -> parse_ipv4_bin(Rest, [Char], URI); + true -> + try parse_ipv4_bin(Rest, [Char], URI) + catch + throw:{_,_,_} -> + parse_reg_name(?STRING_REST(Char, Rest), URI) + end; false -> parse_reg_name(?STRING_REST(Char, Rest), URI) end; parse_host(?STRING_EMPTY, URI) -> @@ -865,15 +878,15 @@ parse_reg_name(?STRING_REST($:, Rest), URI) -> parse_reg_name(?STRING_REST($/, Rest), URI) -> {T, URI1} = parse_segment(Rest, URI), % path-abempty Path = calculate_parsed_part(Rest, T), - {Rest, URI1#{path => decode_path(?STRING_REST($/, Path))}}; + {Rest, URI1#{path => ?STRING_REST($/, Path)}}; parse_reg_name(?STRING_REST($?, Rest), URI) -> {T, URI1} = parse_query(Rest, URI), % path-empty ?query Query = calculate_parsed_query_fragment(Rest, T), - {Rest, URI1#{query => decode_query(Query)}}; + {Rest, URI1#{query => Query}}; parse_reg_name(?STRING_REST($#, Rest), URI) -> {T, URI1} = parse_fragment(Rest, URI), % path-empty Fragment = calculate_parsed_query_fragment(Rest, T), - {Rest, URI1#{fragment => decode_fragment(Fragment)}}; + {Rest, URI1#{fragment => Fragment}}; parse_reg_name(?STRING_REST(Char, Rest), URI) -> case is_reg_name(Char) of true -> parse_reg_name(Rest, URI); @@ -899,17 +912,17 @@ parse_ipv4_bin(?STRING_REST($/, Rest), Acc, URI) -> _ = validate_ipv4_address(lists:reverse(Acc)), {T, URI1} = parse_segment(Rest, URI), % path-abempty Path = calculate_parsed_part(Rest, T), - {Rest, URI1#{path => decode_path(?STRING_REST($/, Path))}}; + {Rest, URI1#{path => ?STRING_REST($/, Path)}}; parse_ipv4_bin(?STRING_REST($?, Rest), Acc, URI) -> _ = validate_ipv4_address(lists:reverse(Acc)), {T, URI1} = parse_query(Rest, URI), % path-empty ?query Query = calculate_parsed_query_fragment(Rest, T), - {Rest, URI1#{query => decode_query(Query)}}; + {Rest, URI1#{query => Query}}; parse_ipv4_bin(?STRING_REST($#, Rest), Acc, URI) -> _ = validate_ipv4_address(lists:reverse(Acc)), {T, URI1} = parse_fragment(Rest, URI), % path-empty Fragment = calculate_parsed_query_fragment(Rest, T), - {Rest, URI1#{fragment => decode_fragment(Fragment)}}; + {Rest, URI1#{fragment => Fragment}}; parse_ipv4_bin(?STRING_REST(Char, Rest), Acc, URI) -> case is_ipv4(Char) of true -> parse_ipv4_bin(Rest, [Char|Acc], URI); @@ -961,15 +974,15 @@ parse_ipv6_bin_end(?STRING_REST($:, Rest), URI) -> parse_ipv6_bin_end(?STRING_REST($/, Rest), URI) -> {T, URI1} = parse_segment(Rest, URI), % path-abempty Path = calculate_parsed_part(Rest, T), - {Rest, URI1#{path => decode_path(?STRING_REST($/, Path))}}; + {Rest, URI1#{path => ?STRING_REST($/, Path)}}; parse_ipv6_bin_end(?STRING_REST($?, Rest), URI) -> {T, URI1} = parse_query(Rest, URI), % path-empty ?query Query = calculate_parsed_query_fragment(Rest, T), - {Rest, URI1#{query => decode_query(Query)}}; + {Rest, URI1#{query => Query}}; parse_ipv6_bin_end(?STRING_REST($#, Rest), URI) -> {T, URI1} = parse_fragment(Rest, URI), % path-empty Fragment = calculate_parsed_query_fragment(Rest, T), - {Rest, URI1#{fragment => decode_fragment(Fragment)}}; + {Rest, URI1#{fragment => Fragment}}; parse_ipv6_bin_end(?STRING_REST(Char, Rest), URI) -> case is_ipv6(Char) of true -> parse_ipv6_bin_end(Rest, URI); @@ -999,15 +1012,15 @@ validate_ipv6_address(Addr) -> parse_port(?STRING_REST($/, Rest), URI) -> {T, URI1} = parse_segment(Rest, URI), % path-abempty Path = calculate_parsed_part(Rest, T), - {Rest, URI1#{path => decode_path(?STRING_REST($/, Path))}}; + {Rest, URI1#{path => ?STRING_REST($/, Path)}}; parse_port(?STRING_REST($?, Rest), URI) -> {T, URI1} = parse_query(Rest, URI), % path-empty ?query Query = calculate_parsed_query_fragment(Rest, T), - {Rest, URI1#{query => decode_query(Query)}}; + {Rest, URI1#{query => Query}}; parse_port(?STRING_REST($#, Rest), URI) -> {T, URI1} = parse_fragment(Rest, URI), % path-empty Fragment = calculate_parsed_query_fragment(Rest, T), - {Rest, URI1#{fragment => decode_fragment(Fragment)}}; + {Rest, URI1#{fragment => Fragment}}; parse_port(?STRING_REST(Char, Rest), URI) -> case is_digit(Char) of true -> parse_port(Rest, URI); @@ -1033,7 +1046,7 @@ parse_port(?STRING_EMPTY, URI) -> parse_query(?STRING_REST($#, Rest), URI) -> {T, URI1} = parse_fragment(Rest, URI), Fragment = calculate_parsed_query_fragment(Rest, T), - {Rest, URI1#{fragment => decode_fragment(Fragment)}}; + {Rest, URI1#{fragment => Fragment}}; parse_query(?STRING_REST(Char, Rest), URI) -> case is_query(Char) of true -> parse_query(Rest, URI); @@ -1088,6 +1101,31 @@ is_fragment(Char) -> is_pchar(Char). %% %%------------------------------------------------------------------------- +%% Return true if input char is reserved. +-spec is_reserved(char()) -> boolean(). +is_reserved($:) -> true; +is_reserved($/) -> true; +is_reserved($?) -> true; +is_reserved($#) -> true; +is_reserved($[) -> true; +is_reserved($]) -> true; +is_reserved($@) -> true; + +is_reserved($!) -> true; +is_reserved($$) -> true; +is_reserved($&) -> true; +is_reserved($') -> true; +is_reserved($() -> true; +is_reserved($)) -> true; + +is_reserved($*) -> true; +is_reserved($+) -> true; +is_reserved($,) -> true; +is_reserved($;) -> true; +is_reserved($=) -> true; +is_reserved(_) -> false. + + %% Check if char is sub-delim. -spec is_sub_delim(char()) -> boolean(). is_sub_delim($!) -> true; @@ -1276,36 +1314,6 @@ byte_size_exl_head(Binary) -> byte_size(Binary) + 1. %% %% pct-encoded = "%" HEXDIG HEXDIG %%------------------------------------------------------------------------- --spec decode_userinfo(binary()) -> binary(). -decode_userinfo(Cs) -> - check_utf8(decode(Cs, fun is_userinfo/1, <<>>)). - --spec decode_host(binary()) -> binary(). -decode_host(Cs) -> - check_utf8(decode(Cs, fun is_host/1, <<>>)). - --spec decode_path(binary()) -> binary(). -decode_path(Cs) -> - check_utf8(decode(Cs, fun is_path/1, <<>>)). - --spec decode_query(binary()) -> binary(). -decode_query(Cs) -> - check_utf8(decode(Cs, fun is_query/1, <<>>)). - --spec decode_fragment(binary()) -> binary(). -decode_fragment(Cs) -> - check_utf8(decode(Cs, fun is_fragment/1, <<>>)). - - -%% Returns Cs if it is utf8 encoded. -check_utf8(Cs) -> - case unicode:characters_to_list(Cs) of - {incomplete,_,_} -> - throw({error,invalid_utf8,Cs}); - {error,_,_} -> - throw({error,invalid_utf8,Cs}); - _ -> Cs - end. %%------------------------------------------------------------------------- %% Percent-encode @@ -1351,20 +1359,56 @@ encode_fragment(Cs) -> %%------------------------------------------------------------------------- %% Helper funtions for percent-decode %%------------------------------------------------------------------------- -decode(<<$%,C0,C1,Cs/binary>>, Fun, Acc) -> + +-spec decode(list()|binary()) -> list() | binary(). +decode(Cs) -> + decode(Cs, <<>>). +%% +decode(L, Acc) when is_list(L) -> + B0 = unicode:characters_to_binary(L), + B1 = decode(B0, Acc), + unicode:characters_to_list(B1); +decode(<<$%,C0,C1,Cs/binary>>, Acc) -> case is_hex_digit(C0) andalso is_hex_digit(C1) of true -> B = ?HEX2DEC(C0)*16+?HEX2DEC(C1), - decode(Cs, Fun, <<Acc/binary, B>>); + case is_reserved(B) of + true -> + %% [2.2] Characters in the reserved set are protected from + %% normalization. + %% [2.1] For consistency, URI producers and normalizers should + %% use uppercase hexadecimal digits for all percent- + %% encodings. + H0 = hex_to_upper(C0), + H1 = hex_to_upper(C1), + decode(Cs, <<Acc/binary,$%,H0,H1>>); + false -> + decode(Cs, <<Acc/binary, B>>) + end; false -> throw({error,invalid_percent_encoding,<<$%,C0,C1>>}) end; -decode(<<C,Cs/binary>>, Fun, Acc) -> - case Fun(C) of - true -> decode(Cs, Fun, <<Acc/binary, C>>); - false -> throw({error,invalid_percent_encoding,<<C,Cs/binary>>}) - end; -decode(<<>>, _Fun, Acc) -> - Acc. +decode(<<C,Cs/binary>>, Acc) -> + decode(Cs, <<Acc/binary, C>>); +decode(<<>>, Acc) -> + check_utf8(Acc). + +%% Returns Cs if it is utf8 encoded. +check_utf8(Cs) -> + case unicode:characters_to_list(Cs) of + {incomplete,_,_} -> + throw({error,invalid_utf8,Cs}); + {error,_,_} -> + throw({error,invalid_utf8,Cs}); + _ -> Cs + end. + +%% Convert hex digit to uppercase form +hex_to_upper(H) when $a =< H, H =< $f -> + H - 32; +hex_to_upper(H) when $0 =< H, H =< $9;$A =< H, H =< $F-> + H; +hex_to_upper(H) -> + throw({error,invalid_input, H}). %% Check if char is allowed in host -spec is_host(char()) -> boolean(). @@ -1850,13 +1894,12 @@ dissect_query_key(<<$=,T/binary>>, IsList, Acc, Key, Value) -> dissect_query_value(T, IsList, Acc, Key, Value); dissect_query_key(<<"&#",T/binary>>, IsList, Acc, Key, Value) -> dissect_query_key(T, IsList, Acc, <<Key/binary,"&#">>, Value); -dissect_query_key(<<$&,_T/binary>>, _IsList, _Acc, _Key, _Value) -> - throw({error, missing_value, "&"}); +dissect_query_key(T = <<$&,_/binary>>, IsList, Acc, Key, <<>>) -> + dissect_query_value(T, IsList, Acc, Key, true); dissect_query_key(<<H,T/binary>>, IsList, Acc, Key, Value) -> dissect_query_key(T, IsList, Acc, <<Key/binary,H>>, Value); -dissect_query_key(B, _, _, _, _) -> - throw({error, missing_value, B}). - +dissect_query_key(T = <<>>, IsList, Acc, Key, <<>>) -> + dissect_query_value(T, IsList, Acc, Key, true). dissect_query_value(<<$&,T/binary>>, IsList, Acc, Key, Value) -> K = form_urldecode(IsList, Key), @@ -1869,9 +1912,10 @@ dissect_query_value(<<>>, IsList, Acc, Key, Value) -> V = form_urldecode(IsList, Value), lists:reverse([{K,V}|Acc]). - %% HTML 5.2 - 4.10.21.6 URL-encoded form data - WHATWG URL (10 Jan 2018) - UTF-8 %% HTML 5.0 - 4.10.22.6 URL-encoded form data - decoding (non UTF-8) +form_urldecode(_, true) -> + true; form_urldecode(true, B) -> Result = base10_decode(form_urldecode(B, <<>>)), convert_to_list(Result, utf8); @@ -1925,9 +1969,10 @@ base10_decode_unicode(<<H,_/binary>>, _, _) -> %%------------------------------------------------------------------------- normalize_map(URIMap) -> - normalize_path_segment( - normalize_scheme_based( - normalize_case(URIMap))). + normalize_path_segment( + normalize_scheme_based( + normalize_percent_encoding( + normalize_case(URIMap)))). %% 6.2.2.1. Case Normalization @@ -1942,6 +1987,18 @@ normalize_case(#{} = Map) -> Map. +%% 6.2.2.2. Percent-Encoding Normalization +normalize_percent_encoding(Map) -> + Fun = fun (K,V) when K =:= userinfo; K =:= host; K =:= path; + K =:= query; K =:= fragment -> + decode(V); + %% Handle port and scheme + (_,V) -> + V + end, + maps:map(Fun, Map). + + to_lower(Cs) when is_list(Cs) -> B = convert_to_binary(Cs, utf8, utf8), convert_to_list(to_lower(B), utf8); diff --git a/lib/stdlib/src/zip.erl b/lib/stdlib/src/zip.erl index 39be2abff6..a922bf3fbe 100644 --- a/lib/stdlib/src/zip.erl +++ b/lib/stdlib/src/zip.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2006-2017. All Rights Reserved. +%% Copyright Ericsson AB 2006-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/stdlib/test/Makefile b/lib/stdlib/test/Makefile index ae2e3d0e2b..bbe3cefa42 100644 --- a/lib/stdlib/test/Makefile +++ b/lib/stdlib/test/Makefile @@ -88,6 +88,7 @@ MODULES= \ unicode_SUITE \ unicode_util_SUITE \ uri_string_SUITE \ + uri_string_property_test_SUITE \ win32reg_SUITE \ y2k_SUITE \ select_SUITE \ @@ -152,6 +153,6 @@ release_tests_spec: make_emakefile $(INSTALL_DATA) stdlib.spec stdlib_bench.spec $(EMAKEFILE) \ $(ERL_FILES) $(COVERFILE) "$(RELSYSDIR)" chmod -R u+w "$(RELSYSDIR)" - @tar cf - *_SUITE_data | (cd "$(RELSYSDIR)"; tar xf -) + @tar cf - *_SUITE_data property_test | (cd "$(RELSYSDIR)"; tar xf -) release_docs_spec: diff --git a/lib/stdlib/test/array_SUITE.erl b/lib/stdlib/test/array_SUITE.erl index 956582c4fd..df520ebb54 100644 --- a/lib/stdlib/test/array_SUITE.erl +++ b/lib/stdlib/test/array_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2016. All Rights Reserved. +%% Copyright Ericsson AB 2007-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/stdlib/test/beam_lib_SUITE.erl b/lib/stdlib/test/beam_lib_SUITE.erl index 73219f8fd8..3597d6d94b 100644 --- a/lib/stdlib/test/beam_lib_SUITE.erl +++ b/lib/stdlib/test/beam_lib_SUITE.erl @@ -78,7 +78,7 @@ normal(Conf) when is_list(Conf) -> BeamFile = Simple ++ ".beam", simple_file(Source), - NoOfTables = length(ets:all()), + NoOfTables = erlang:system_info(ets_count), P0 = pps(), do_normal(Source, PrivDir, BeamFile, []), @@ -95,7 +95,7 @@ normal(Conf) when is_list(Conf) -> file:delete(BeamFile), file:delete(Source), - NoOfTables = length(ets:all()), + NoOfTables = erlang:system_info(ets_count), true = (P0 == pps()), ok. @@ -173,7 +173,7 @@ error(Conf) when is_list(Conf) -> WrongFile = Simple ++ "foo.beam", simple_file(Source), - NoOfTables = length(ets:all()), + NoOfTables = erlang:system_info(ets_count), P0 = pps(), {ok,_} = compile:file(Source, [{outdir,PrivDir},debug_info]), ACopy = filename:join(PrivDir, "a_copy.beam"), @@ -213,7 +213,7 @@ error(Conf) when is_list(Conf) -> %% we have eliminated them. ok = file:write_file(BeamFile, <<"FOR1",5:32,"BEAMfel">>), - NoOfTables = length(ets:all()), + NoOfTables = erlang:system_info(ets_count), true = (P0 == pps()), file:delete(Source), file:delete(WrongFile), @@ -273,7 +273,7 @@ cmp(Conf) when is_list(Conf) -> {Source2D1, BeamFile2D1} = make_beam(Dir1, simple2, concat), {SourceD2, BeamFileD2} = make_beam(Dir2, simple, concat), - NoOfTables = length(ets:all()), + NoOfTables = erlang:system_info(ets_count), P0 = pps(), %% cmp @@ -300,7 +300,7 @@ cmp(Conf) when is_list(Conf) -> ver(not_a_directory, beam_lib:diff_dirs(foo, bar)), true = (P0 == pps()), - NoOfTables = length(ets:all()), + NoOfTables = erlang:system_info(ets_count), delete_files([SourceD1, BeamFileD1, Source2D1, BeamFile2D1, SourceD2, BeamFileD2]), @@ -321,7 +321,7 @@ cmp_literals(Conf) when is_list(Conf) -> {SourceD1, BeamFileD1} = make_beam(Dir1, simple, constant), {SourceD2, BeamFileD2} = make_beam(Dir2, simple, constant2), - NoOfTables = length(ets:all()), + NoOfTables = erlang:system_info(ets_count), P0 = pps(), %% cmp @@ -334,7 +334,7 @@ cmp_literals(Conf) when is_list(Conf) -> ver(chunks_different, beam_lib:cmp(B1, B2)), true = (P0 == pps()), - NoOfTables = length(ets:all()), + NoOfTables = erlang:system_info(ets_count), delete_files([SourceD1, BeamFileD1, SourceD2, BeamFileD2]), @@ -351,7 +351,7 @@ strip(Conf) when is_list(Conf) -> {Source4D1, BeamFile4D1} = make_beam(PrivDir, constant, constant), {Source5D1, BeamFile5D1} = make_beam(PrivDir, lines, lines), - NoOfTables = length(ets:all()), + NoOfTables = erlang:system_info(ets_count), P0 = pps(), %% strip binary @@ -392,7 +392,7 @@ strip(Conf) when is_list(Conf) -> (catch lines:t(atom)), true = (P0 == pps()), - NoOfTables = length(ets:all()), + NoOfTables = erlang:system_info(ets_count), delete_files([SourceD1, BeamFileD1, Source2D1, BeamFile2D1, @@ -457,7 +457,7 @@ building(Conf) when is_list(Conf) -> {SourceD1, BeamFileD1} = make_beam(Dir1, building, member), - NoOfTables = length(ets:all()), + NoOfTables = erlang:system_info(ets_count), P0 = pps(), %% read all chunks @@ -487,7 +487,7 @@ building(Conf) when is_list(Conf) -> end, ChunkIds), true = (P0 == pps()), - NoOfTables = length(ets:all()), + NoOfTables = erlang:system_info(ets_count), delete_files([SourceD1, BeamFileD1, BeamFileD2]), file:del_dir(Dir1), @@ -535,7 +535,7 @@ encrypted_abstr_1(Conf) -> %% Avoid getting an extra port when crypto starts erl_ddll. erl_ddll:start(), - NoOfTables = length(ets:all()), + NoOfTables = erlang:system_info(ets_count), P0 = pps(), Key = "#a_crypto_key", @@ -549,7 +549,7 @@ encrypted_abstr_1(Conf) -> ok = crypto:stop(), %To get rid of extra ets tables. file:delete(BeamFile), file:delete(Source), - NoOfTables = length(ets:all()), + NoOfTables = erlang:system_info(ets_count), true = (P0 == pps()), ok. @@ -658,7 +658,7 @@ encrypted_abstr_file_1(Conf) -> %% Avoid getting an extra port when crypto starts erl_ddll. erl_ddll:start(), - NoOfTables = length(ets:all()), + NoOfTables = erlang:system_info(ets_count), P0 = pps(), Key = "Long And niCe 99Krypto Key", @@ -676,7 +676,7 @@ encrypted_abstr_file_1(Conf) -> file:delete(filename:join(PrivDir, ".erlang.crypt")), file:delete(BeamFile), file:delete(Source), - NoOfTables = length(ets:all()), + NoOfTables = erlang:system_info(ets_count), true = (P0 == pps()), ok. diff --git a/lib/stdlib/test/c_SUITE.erl b/lib/stdlib/test/c_SUITE.erl index f01988478c..bd84cdd228 100644 --- a/lib/stdlib/test/c_SUITE.erl +++ b/lib/stdlib/test/c_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1998-2016. All Rights Reserved. +%% Copyright Ericsson AB 1998-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/stdlib/test/dets_SUITE.erl b/lib/stdlib/test/dets_SUITE.erl index 7d82790b82..fe324391af 100644 --- a/lib/stdlib/test/dets_SUITE.erl +++ b/lib/stdlib/test/dets_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2017. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/stdlib/test/epp_SUITE.erl b/lib/stdlib/test/epp_SUITE.erl index a3e294ffea..10e1b75e0f 100644 --- a/lib/stdlib/test/epp_SUITE.erl +++ b/lib/stdlib/test/epp_SUITE.erl @@ -1372,7 +1372,7 @@ otp_8562(Config) when is_list(Config) -> otp_8911(Config) when is_list(Config) -> case test_server:is_cover() of true -> - {skip, "Testing cover, so can not run when cover is already running"}; + {skip, "Testing cover, so cannot run when cover is already running"}; false -> do_otp_8911(Config) end. diff --git a/lib/stdlib/test/erl_lint_SUITE.erl b/lib/stdlib/test/erl_lint_SUITE.erl index f9ab83a120..c1613a7273 100644 --- a/lib/stdlib/test/erl_lint_SUITE.erl +++ b/lib/stdlib/test/erl_lint_SUITE.erl @@ -2730,7 +2730,7 @@ bif_clash(Config) when is_list(Config) -> [], {errors,[{2,erl_lint,{call_to_redefined_old_bif,{size,1}}}],[]}}, - %% Verify that warnings can not be turned off in the old way. + %% Verify that warnings cannot be turned off in the old way. {clash2, <<"-export([t/1,size/1]). t(X) -> diff --git a/lib/stdlib/test/error_logger_h_SUITE.erl b/lib/stdlib/test/error_logger_h_SUITE.erl index d533305939..bf9b6d9ad6 100644 --- a/lib/stdlib/test/error_logger_h_SUITE.erl +++ b/lib/stdlib/test/error_logger_h_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2015-2016. All Rights Reserved. +%% Copyright Ericsson AB 2015-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/stdlib/test/ets_SUITE.erl b/lib/stdlib/test/ets_SUITE.erl index a97fe4a5d9..fee8b204f4 100644 --- a/lib/stdlib/test/ets_SUITE.erl +++ b/lib/stdlib/test/ets_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2017. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -6112,40 +6112,11 @@ etsmem() -> ets:info(T,memory),ets:info(T,type)} end, ets:all()), - EtsAllocInfo = erlang:system_info({allocator,ets_alloc}), + EtsAllocSize = erts_debug:alloc_blocks_size(ets_alloc), ErlangMemoryEts = try erlang:memory(ets) catch error:notsup -> notsup end, - Mem = - {ErlangMemoryEts, - case EtsAllocInfo of - false -> undefined; - MemInfo -> - CS = lists:foldl( - fun ({instance, _, L}, Acc) -> - {value,{mbcs,MBCS}} = lists:keysearch(mbcs, 1, L), - {value,{sbcs,SBCS}} = lists:keysearch(sbcs, 1, L), - NewAcc = [MBCS, SBCS | Acc], - case lists:keysearch(mbcs_pool, 1, L) of - {value,{mbcs_pool, MBCS_POOL}} -> - [MBCS_POOL|NewAcc]; - _ -> NewAcc - end - end, - [], - MemInfo), - lists:foldl( - fun(L, {Bl0,BlSz0}) -> - {value,BlTup} = lists:keysearch(blocks, 1, L), - blocks = element(1, BlTup), - Bl = element(2, BlTup), - {value,BlSzTup} = lists:keysearch(blocks_size, 1, L), - blocks_size = element(1, BlSzTup), - BlSz = element(2, BlSzTup), - {Bl0+Bl,BlSz0+BlSz} - end, {0,0}, CS) - end}, - {Mem,AllTabs}. - + Mem = {ErlangMemoryEts, EtsAllocSize}, + {Mem, AllTabs}. verify_etsmem(MI) -> wait_for_test_procs(), diff --git a/lib/stdlib/test/id_transform_SUITE.erl b/lib/stdlib/test/id_transform_SUITE.erl index 186df41d3f..0addf09461 100644 --- a/lib/stdlib/test/id_transform_SUITE.erl +++ b/lib/stdlib/test/id_transform_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2003-2016. All Rights Reserved. +%% Copyright Ericsson AB 2003-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/stdlib/test/io_SUITE.erl b/lib/stdlib/test/io_SUITE.erl index 13f2cbd27b..79cee54335 100644 --- a/lib/stdlib/test/io_SUITE.erl +++ b/lib/stdlib/test/io_SUITE.erl @@ -31,7 +31,8 @@ otp_10836/1, io_lib_width_too_small/1, io_with_huge_message_queue/1, format_string/1, maps/1, coverage/1, otp_14178_unicode_atoms/1, otp_14175/1, - otp_14285/1, limit_term/1, otp_14983/1]). + otp_14285/1, limit_term/1, otp_14983/1, otp_15103/1, + otp_15159/1]). -export([pretty/2, trf/3]). @@ -63,7 +64,7 @@ all() -> io_lib_print_binary_depth_one, otp_10302, otp_10755, otp_10836, io_lib_width_too_small, io_with_huge_message_queue, format_string, maps, coverage, otp_14178_unicode_atoms, otp_14175, - otp_14285, limit_term, otp_14983]. + otp_14285, limit_term, otp_14983, otp_15103, otp_15159]. %% Error cases for output. error_1(Config) when is_list(Config) -> @@ -2615,3 +2616,26 @@ trf(Format, Args, T) -> trf(Format, Args, T, Opts) -> lists:flatten(io_lib:format(Format, Args, [{chars_limit, T}|Opts])). + +otp_15103(_Config) -> + T = lists:duplicate(5, {a,b,c}), + + S1 = io_lib:format("~0p", [T]), + "[{a,b,c},{a,b,c},{a,b,c},{a,b,c},{a,b,c}]" = lists:flatten(S1), + S2 = io_lib:format("~-0p", [T]), + "[{a,b,c},{a,b,c},{a,b,c},{a,b,c},{a,b,c}]" = lists:flatten(S2), + S3 = io_lib:format("~1p", [T]), + "[{a,\n b,\n c},\n {a,\n b,\n c},\n {a,\n b,\n c},\n {a,\n b,\n" + " c},\n {a,\n b,\n c}]" = lists:flatten(S3), + + S4 = io_lib:format("~0P", [T, 5]), + "[{a,b,c},{a,b,...},{a,...},{...}|...]" = lists:flatten(S4), + S5 = io_lib:format("~1P", [T, 5]), + "[{a,\n b,\n c},\n {a,\n b,...},\n {a,...},\n {...}|...]" = + lists:flatten(S5), + ok. + +otp_15159(_Config) -> + "[atom]" = + lists:flatten(io_lib:format("~p", [[atom]], [{chars_limit,5}])), + ok. diff --git a/lib/stdlib/test/lists_SUITE.erl b/lib/stdlib/test/lists_SUITE.erl index 837ab4e97e..af94fc79bc 100644 --- a/lib/stdlib/test/lists_SUITE.erl +++ b/lib/stdlib/test/lists_SUITE.erl @@ -1679,7 +1679,7 @@ make_fun() -> receive {Pid, Fun} -> Fun end. make_fun(Pid) -> - Pid ! {self(), fun make_fun/1}. + Pid ! {self(), fun (X) -> {X, Pid} end}. fun_pid(Fun) -> erlang:fun_info(Fun, pid). diff --git a/lib/stdlib/test/maps_SUITE.erl b/lib/stdlib/test/maps_SUITE.erl index a75751b31d..6f3cd8bf1b 100644 --- a/lib/stdlib/test/maps_SUITE.erl +++ b/lib/stdlib/test/maps_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2016. All Rights Reserved. +%% Copyright Ericsson AB 1997-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -108,6 +108,8 @@ t_without_2(_Config) -> %% error case ?badmap(a,without,[[a,b],a]) = (catch maps:without([a,b],id(a))), ?badmap(a,without,[{a,b},a]) = (catch maps:without({a,b},id(a))), + ?badmap({0,<<>>,97},without,[[],{0,<<>>,97}]) = (catch maps:without([], {0,<<>>,97})), + ?badmap({0,<<>>,97},without,[[false, -20, -8],{0,<<>>,97}]) = (catch maps:without([false, -20, -8], {0, <<>>, 97})), ?badarg(without,[a,#{}]) = (catch maps:without(a,#{})), ok. @@ -120,6 +122,8 @@ t_with_2(_Config) -> %% error case ?badmap(a,with,[[a,b],a]) = (catch maps:with([a,b],id(a))), ?badmap(a,with,[{a,b},a]) = (catch maps:with({a,b},id(a))), + ?badmap({0,<<>>,97},with,[[],{0,<<>>,97}]) = (catch maps:with([], {0,<<>>,97})), + ?badmap({0,<<>>,97},with,[[false, -20, -8],{0,<<>>,97}]) = (catch maps:with([false, -20, -8], {0, <<>>, 97})), ?badarg(with,[a,#{}]) = (catch maps:with(a,#{})), ok. diff --git a/lib/stdlib/test/proc_lib_SUITE.erl b/lib/stdlib/test/proc_lib_SUITE.erl index 81bf9020b8..127b1317e4 100644 --- a/lib/stdlib/test/proc_lib_SUITE.erl +++ b/lib/stdlib/test/proc_lib_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2017. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -542,13 +542,14 @@ system_terminate(Reason,_Parent,_Deb,_State) -> t_format(_Config) -> - logger:add_handler_filter(logger_std_h,stop_all,{fun(_,_) -> stop end,ok}), + {ok,#{level:=Level}} = logger:get_handler_config(default), + logger:set_handler_config(default,level,none), error_logger:add_report_handler(?MODULE, self()), try t_format() after error_logger:delete_report_handler(?MODULE), - logger:remove_handler_filter(logger_std_h,stop_all) + logger:set_handler_config(default,level,Level) end, ok. @@ -585,11 +586,12 @@ t_format() -> ok. t_format_arbitrary(_Config) -> - logger:add_handler_filter(logger_std_h,stop_all,{fun(_,_) -> stop end,ok}), + {ok,#{level:=Level}} = logger:get_handler_config(default), + logger:set_handler_config(default,level,none), try t_format_arbitrary() after - logger:remove_handler_filter(logger_std_h,stop_all) + logger:set_handler_config(default,level,Level) end, ok. diff --git a/lib/stdlib/test/property_test/uri_string_recompose.erl b/lib/stdlib/test/property_test/uri_string_recompose.erl index e51a671172..39fadf23c2 100644 --- a/lib/stdlib/test/property_test/uri_string_recompose.erl +++ b/lib/stdlib/test/property_test/uri_string_recompose.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2017. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -65,15 +65,29 @@ -define(QUERY, {query, query_map()}). -define(FRAGMENT, {fragment, fragment_map()}). +%% Non-unicode +-define(USER_NU, {userinfo, non_unicode()}). +-define(HOST_NU, {host, host_map_nu()}). +-define(PATH_ABE_NU, {path, path_abempty_map_nu()}). +-define(PATH_ABS_NU, {path, path_absolute_map_nu()}). +-define(PATH_NOS_NU, {path, path_noscheme_map_nu()}). +-define(PATH_ROO_NU, {path, path_rootless_map_nu()}). +-define(QUERY_NU, {query, query_map_nu()}). +-define(FRAGMENT_NU, {fragment, fragment_map_nu()}). %%%======================================================================== %%% Properties %%%======================================================================== prop_recompose() -> + ?FORALL(Map, map_no_unicode(), + Map =:= uri_string:parse(uri_string:recompose(Map))). + +prop_normalize() -> ?FORALL(Map, map(), - Map =:= uri_string:parse(uri_string:recompose(Map)) - ). + uri_string:normalize(Map, [return_map]) =:= + uri_string:normalize(uri_string:parse(uri_string:recompose(Map)), + [return_map])). %% Stats prop_map_key_length_collect() -> @@ -96,6 +110,9 @@ prop_scheme_collect() -> map() -> ?LET(Gen, comp_proplist(), proplist_to_map(Gen)). +map_no_unicode() -> + ?LET(Gen, comp_proplist_nu(), proplist_to_map(Gen)). + comp_proplist() -> frequency([ {2, [?SCHEME,?PATH_ABS]}, @@ -166,6 +183,76 @@ comp_proplist() -> {2, [?USER,?HOST,?PORT,?PATH_ABE,?QUERY,?FRAGMENT]} ]). +comp_proplist_nu() -> + frequency([ + {2, [?SCHEME,?PATH_ABS_NU]}, + {2, [?SCHEME,?PATH_ROO_NU]}, + {2, [?SCHEME,?PATH_EMP]}, + {2, [?SCHEME,?HOST_NU,?PATH_ABE_NU]}, + {2, [?SCHEME,?USER_NU,?HOST_NU,?PATH_ABE_NU]}, + {2, [?SCHEME,?HOST_NU,?PORT,?PATH_ABE_NU]}, + {2, [?SCHEME,?USER_NU,?HOST_NU,?PORT,?PATH_ABE_NU]}, + + {2, [?PATH_ABS_NU]}, + {2, [?PATH_NOS_NU]}, + {2, [?PATH_EMP]}, + {2, [?HOST_NU,?PATH_ABE_NU]}, + {2, [?USER_NU,?HOST_NU,?PATH_ABE_NU]}, + {2, [?HOST_NU,?PORT,?PATH_ABE_NU]}, + {2, [?USER_NU,?HOST_NU,?PORT,?PATH_ABE_NU]}, + + + {2, [?SCHEME,?PATH_ABS_NU,?QUERY_NU]}, + {2, [?SCHEME,?PATH_ROO_NU,?QUERY_NU]}, + {2, [?SCHEME,?PATH_EMP,?QUERY_NU]}, + {2, [?SCHEME,?HOST_NU,?PATH_ABE_NU,?QUERY_NU]}, + {2, [?SCHEME,?USER_NU,?HOST_NU,?PATH_ABE_NU,?QUERY_NU]}, + {2, [?SCHEME,?HOST_NU,?PORT,?PATH_ABE_NU,?QUERY_NU]}, + {2, [?SCHEME,?USER_NU,?HOST_NU,?PORT,?PATH_ABE_NU,?QUERY_NU]}, + + {2, [?PATH_ABS_NU,?QUERY_NU]}, + {2, [?PATH_NOS_NU,?QUERY_NU]}, + {2, [?PATH_EMP,?QUERY_NU]}, + {2, [?HOST_NU,?PATH_ABE_NU,?QUERY_NU]}, + {2, [?USER_NU,?HOST_NU,?PATH_ABE_NU,?QUERY_NU]}, + {2, [?HOST_NU,?PORT,?PATH_ABE_NU,?QUERY_NU]}, + {2, [?USER_NU,?HOST_NU,?PORT,?PATH_ABE_NU,?QUERY_NU]}, + + + {2, [?SCHEME,?PATH_ABS_NU,?FRAGMENT_NU]}, + {2, [?SCHEME,?PATH_ROO_NU,?FRAGMENT_NU]}, + {2, [?SCHEME,?PATH_EMP,?FRAGMENT_NU]}, + {2, [?SCHEME,?HOST_NU,?PATH_ABE_NU,?FRAGMENT_NU]}, + {2, [?SCHEME,?USER_NU,?HOST_NU,?PATH_ABE_NU,?FRAGMENT_NU]}, + {2, [?SCHEME,?HOST_NU,?PORT,?PATH_ABE_NU,?FRAGMENT_NU]}, + {2, [?SCHEME,?USER_NU,?HOST_NU,?PORT,?PATH_ABE_NU,?FRAGMENT_NU]}, + + {2, [?PATH_ABS_NU,?FRAGMENT_NU]}, + {2, [?PATH_NOS_NU,?FRAGMENT_NU]}, + {2, [?PATH_EMP,?FRAGMENT_NU]}, + {2, [?HOST_NU,?PATH_ABE_NU,?FRAGMENT_NU]}, + {2, [?USER_NU,?HOST_NU,?PATH_ABE_NU,?FRAGMENT_NU]}, + {2, [?HOST_NU,?PORT,?PATH_ABE_NU,?FRAGMENT_NU]}, + {2, [?USER_NU,?HOST_NU,?PORT,?PATH_ABE_NU,?FRAGMENT_NU]}, + + + {2, [?SCHEME,?PATH_ABS_NU,?QUERY_NU,?FRAGMENT_NU]}, + {2, [?SCHEME,?PATH_ROO_NU,?QUERY_NU,?FRAGMENT_NU]}, + {2, [?SCHEME,?PATH_EMP,?QUERY_NU,?FRAGMENT_NU]}, + {2, [?SCHEME,?HOST_NU,?PATH_ABE_NU,?QUERY_NU,?FRAGMENT_NU]}, + {2, [?SCHEME,?USER_NU,?HOST_NU,?PATH_ABE_NU,?QUERY_NU,?FRAGMENT_NU]}, + {2, [?SCHEME,?HOST_NU,?PORT,?PATH_ABE_NU,?QUERY_NU,?FRAGMENT_NU]}, + {2, [?SCHEME,?USER_NU,?HOST_NU,?PORT,?PATH_ABE_NU,?QUERY_NU,?FRAGMENT_NU]}, + + {2, [?PATH_ABS_NU,?QUERY_NU,?FRAGMENT_NU]}, + {2, [?PATH_NOS_NU,?QUERY_NU,?FRAGMENT_NU]}, + {2, [?PATH_EMP,?QUERY_NU,?FRAGMENT_NU]}, + {2, [?HOST_NU,?PATH_ABE_NU,?QUERY_NU,?FRAGMENT_NU]}, + {2, [?USER_NU,?HOST_NU,?PATH_ABE_NU,?QUERY_NU,?FRAGMENT_NU]}, + {2, [?HOST_NU,?PORT,?PATH_ABE_NU,?QUERY_NU,?FRAGMENT_NU]}, + {2, [?USER_NU,?HOST_NU,?PORT,?PATH_ABE_NU,?QUERY_NU,?FRAGMENT_NU]} + ]). + %%------------------------------------------------------------------------- %% Path @@ -174,6 +261,11 @@ path_abempty_map() -> frequency([{90, path_abe_map()}, {10, path_empty_map()}]). +path_abempty_map_nu() -> + frequency([{90, path_abe_map_nu()}, + {10, path_empty_map()}]). + + path_abe_map() -> ?SIZED(Length, path_abe_map(Length, [])). %% @@ -182,6 +274,14 @@ path_abe_map(0, Segments) -> path_abe_map(N, Segments) -> path_abe_map(N-1, [slash(),segment()|Segments]). +path_abe_map_nu() -> + ?SIZED(Length, path_abe_map_nu(Length, [])). +%% +path_abe_map_nu(0, Segments) -> + ?LET(Gen, Segments, lists:append(Gen)); +path_abe_map_nu(N, Segments) -> + path_abe_map_nu(N-1, [slash(),segment_nu()|Segments]). + path_absolute_map() -> ?SIZED(Length, path_absolute_map(Length, [])). @@ -191,6 +291,14 @@ path_absolute_map(0, Segments) -> path_absolute_map(N, Segments) -> path_absolute_map(N-1, [slash(),segment()|Segments]). +path_absolute_map_nu() -> + ?SIZED(Length, path_absolute_map_nu(Length, [])). +%% +path_absolute_map_nu(0, Segments) -> + ?LET(Gen, [slash(),segment_nz_nu()|Segments], lists:append(Gen)); +path_absolute_map_nu(N, Segments) -> + path_absolute_map_nu(N-1, [slash(),segment_nu()|Segments]). + path_noscheme_map() -> ?SIZED(Length, path_noscheme_map(Length, [])). @@ -200,6 +308,15 @@ path_noscheme_map(0, Segments) -> path_noscheme_map(N, Segments) -> path_noscheme_map(N-1, [slash(),segment()|Segments]). +path_noscheme_map_nu() -> + ?SIZED(Length, path_noscheme_map_nu(Length, [])). +%% +path_noscheme_map_nu(0, Segments) -> + ?LET(Gen, [segment_nz_nc_nu()|Segments], lists:append(Gen)); +path_noscheme_map_nu(N, Segments) -> + path_noscheme_map_nu(N-1, [slash(),segment_nu()|Segments]). + + path_rootless_map() -> ?SIZED(Length, path_rootless_map(Length, [])). %% @@ -208,24 +325,59 @@ path_rootless_map(0, Segments) -> path_rootless_map(N, Segments) -> path_rootless_map(N-1, [slash(),segment()|Segments]). +path_rootless_map_nu() -> + ?SIZED(Length, path_rootless_map_nu(Length, [])). +%% +path_rootless_map_nu(0, Segments) -> + ?LET(Gen, [segment_nz_nu()|Segments], lists:append(Gen)); +path_rootless_map_nu(N, Segments) -> + path_rootless_map_nu(N-1, [slash(),segment_nu()|Segments]). + segment_nz() -> non_empty(segment()). -segment_nz_nc() -> - non_empty(list(frequency([{30, unreserved()}, - {10, sub_delims()}, - {10, unicode_char()}, - {5, oneof([$@])} - ]))). +segment_nz_nu() -> + non_empty(segment_nu()). +segment_nz_nc() -> + ?LET(Gen, + non_empty(list(frequency([{30, unreserved()}, + {10, ptc_encoded_reserved()}, + {10, sub_delims()}, + {10, unicode_char()}, + {5, oneof([$@])} + ]))), + lists:flatten(Gen)). + +segment_nz_nc_nu() -> + ?LET(Gen, + non_empty(list(frequency([{30, unreserved()}, + {10, ptc_encoded_reserved()}, + {10, sub_delims()}, + {5, oneof([$@])} + ]))), + lists:flatten(Gen)). + segment() -> - list(frequency([{30, unreserved()}, - {10, sub_delims()}, - {10, unicode_char()}, - {5, oneof([$:, $@])} - ])). + ?LET(Gen, + list(frequency([{30, unreserved()}, + {10, ptc_encoded_reserved()}, + {10, sub_delims()}, + {10, unicode_char()}, + {5, oneof([$:, $@])} + ])), + lists:flatten(Gen)). + +segment_nu() -> + ?LET(Gen, + list(frequency([{30, unreserved()}, + {10, ptc_encoded_reserved()}, + {10, sub_delims()}, + {5, oneof([$:, $@])} + ])), + lists:flatten(Gen)). slash() -> "/". @@ -235,19 +387,35 @@ path_empty_map() -> %%------------------------------------------------------------------------- -%% Path +%% Host %%------------------------------------------------------------------------- host_map() -> frequency([{30, reg_name()}, {30, ip_address()} ]). +host_map_nu() -> + frequency([{30, reg_name_nu()}, + {30, ip_address()} + ]). reg_name() -> - list(frequency([{30, alpha()}, - {10, sub_delims()}, - {10, unicode_char()} - ])). + ?LET(Gen, + list(frequency([{30, alpha()}, + {10, sub_delims()}, + {10, ptc_encoded_reserved()}, + {10, unicode_char()} + ])), + lists:flatten(Gen)). + +reg_name_nu() -> + ?LET(Gen, + list(frequency([{30, alpha()}, + {10, sub_delims()}, + {10, ptc_encoded_reserved()} + ])), + lists:flatten(Gen)). + ip_address() -> oneof(["127.0.0.1", "::127.0.0.1", @@ -258,10 +426,13 @@ ip_address() -> %% Generating only reg-names host_uri() -> - non_empty(list(frequency([{30, unreserved()}, - {10, sub_delims()}, - {10, pct_encoded()} - ]))). + ?LET(Gen, + non_empty(list(frequency([{30, unreserved()}, + {10, sub_delims()}, + {10, ptc_encoded_reserved()}, + {10, pct_encoded()} + ]))), + lists:flatten(Gen)). %%------------------------------------------------------------------------- %% Port, Query, Fragment @@ -274,6 +445,9 @@ port() -> query_map() -> unicode(). +query_map_nu() -> + non_unicode(). + query_uri() -> [$?| non_empty(list(frequency([{20, pchar()}, @@ -283,6 +457,10 @@ query_uri() -> fragment_map() -> unicode(). +fragment_map_nu() -> + non_unicode(). + + fragment_uri() -> [$?| non_empty(list(frequency([{20, pchar()}, {5, oneof([$/, $?])} % punctuation @@ -311,9 +489,14 @@ scheme(N, L) -> %%------------------------------------------------------------------------- unicode() -> list(frequency([{20, alpha()}, % alpha - {10, digit()}, % digit - {10, unicode_char()} % unicode - ])). + {10, digit()}, % digit + {10, unicode_char()} % unicode + ])). + +non_unicode() -> + list(frequency([{20, alpha()}, % alpha + {10, digit()} % digit + ])). scheme_char() -> frequency([{20, alpha()}, % alpha @@ -327,6 +510,7 @@ sub_delims() -> pchar() -> frequency([{20, unreserved()}, + {5, ptc_encoded_reserved()}, {5, pct_encoded()}, {5, sub_delims()}, {1, oneof([$:, $@])} % punctuation @@ -351,6 +535,22 @@ digit() -> pct_encoded() -> oneof(["%C3%A4", "%C3%A5", "%C3%B6"]). +%%------------------------------------------------------------------------- +%% [RFC 3986, Chapter 2.2. Reserved Characters] +%% +%% reserved = gen-delims / sub-delims +%% +%% gen-delims = ":" / "/" / "?" / "#" / "[" / "]" / "@" +%% 3A 2F 3F 23 5B 5D 40 +%% sub-delims = "!" / "$" / "&" / "'" / "(" / ")" +%% 21 24 26 27 28 29 +%% / "*" / "+" / "," / ";" / "=" +%% 2A 2B 2C 3B 3D +%%------------------------------------------------------------------------- +ptc_encoded_reserved() -> + oneof(["%3A","%2F","%3F","%23","%5B","%5D","%40", + "%21","%24","%26","%27","%28","%29", + "%2A","%2B","%2C","%3B","3D"]). %%%======================================================================== %%% Helpers @@ -359,3 +559,13 @@ proplist_to_map(L) -> lists:foldl(fun({K,V},M) -> M#{K => V}; (_,M) -> M end, #{}, L). + +map_scheme_host_to_lower(Map) -> + Fun = fun (scheme,V) -> + string:to_lower(V); + (host,V) -> + string:to_lower(V); + (_,V) -> + V + end, + maps:map(Fun, Map). diff --git a/lib/stdlib/test/qlc_SUITE.erl b/lib/stdlib/test/qlc_SUITE.erl index 5c189a6c73..d7354438f9 100644 --- a/lib/stdlib/test/qlc_SUITE.erl +++ b/lib/stdlib/test/qlc_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2017. All Rights Reserved. +%% Copyright Ericsson AB 2004-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/stdlib/test/re_SUITE.erl b/lib/stdlib/test/re_SUITE.erl index 7b82647416..c9ef9da990 100644 --- a/lib/stdlib/test/re_SUITE.erl +++ b/lib/stdlib/test/re_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2017. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/stdlib/test/sets_SUITE.erl b/lib/stdlib/test/sets_SUITE.erl index 7066d07e19..2c1b388d52 100644 --- a/lib/stdlib/test/sets_SUITE.erl +++ b/lib/stdlib/test/sets_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2016. All Rights Reserved. +%% Copyright Ericsson AB 2004-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/stdlib/test/sets_test_lib.erl b/lib/stdlib/test/sets_test_lib.erl index 93d027704b..e4d476ba54 100644 --- a/lib/stdlib/test/sets_test_lib.erl +++ b/lib/stdlib/test/sets_test_lib.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2015. All Rights Reserved. +%% Copyright Ericsson AB 2004-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/stdlib/test/string_SUITE.erl b/lib/stdlib/test/string_SUITE.erl index ab412dcc70..251e09121c 100644 --- a/lib/stdlib/test/string_SUITE.erl +++ b/lib/stdlib/test/string_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2017. All Rights Reserved. +%% Copyright Ericsson AB 2004-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/stdlib/test/supervisor_1.erl b/lib/stdlib/test/supervisor_1.erl index c3ccacc587..0d3dc67d3b 100644 --- a/lib/stdlib/test/supervisor_1.erl +++ b/lib/stdlib/test/supervisor_1.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/stdlib/test/supervisor_SUITE.erl b/lib/stdlib/test/supervisor_SUITE.erl index 761df8eb40..ed7dd04171 100644 --- a/lib/stdlib/test/supervisor_SUITE.erl +++ b/lib/stdlib/test/supervisor_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/stdlib/test/sys_SUITE.erl b/lib/stdlib/test/sys_SUITE.erl index 439a23d82d..3278eb0eb0 100644 --- a/lib/stdlib/test/sys_SUITE.erl +++ b/lib/stdlib/test/sys_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/stdlib/test/unicode_util_SUITE.erl b/lib/stdlib/test/unicode_util_SUITE.erl index 40b1c260a5..962b307b07 100644 --- a/lib/stdlib/test/unicode_util_SUITE.erl +++ b/lib/stdlib/test/unicode_util_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2017. All Rights Reserved. +%% Copyright Ericsson AB 2017-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/stdlib/test/uri_string_SUITE.erl b/lib/stdlib/test/uri_string_SUITE.erl index 92f8bb3292..ddaead9c7c 100644 --- a/lib/stdlib/test/uri_string_SUITE.erl +++ b/lib/stdlib/test/uri_string_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2017. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -23,6 +23,12 @@ -export([all/0, suite/0,groups/0, normalize/1, normalize_map/1, normalize_return_map/1, normalize_negative/1, + normalize_binary_pct_encoded_userinfo/1, + normalize_binary_pct_encoded_query/1, + normalize_binary_pct_encoded_fragment/1, + normalize_pct_encoded_userinfo/1, + normalize_pct_encoded_query/1, + normalize_pct_encoded_fragment/1, parse_binary_fragment/1, parse_binary_host/1, parse_binary_host_ipv4/1, parse_binary_host_ipv6/1, parse_binary_path/1, parse_binary_pct_encoded_fragment/1, parse_binary_pct_encoded_query/1, @@ -41,7 +47,8 @@ transcode_basic/1, transcode_options/1, transcode_mixed/1, transcode_negative/1, compose_query/1, compose_query_latin1/1, compose_query_negative/1, dissect_query/1, dissect_query_negative/1, - interop_query_latin1/1, interop_query_utf8/1 + interop_query_latin1/1, interop_query_utf8/1, + regression_parse/1, regression_recompose/1, regression_normalize/1 ]). @@ -71,6 +78,12 @@ all() -> normalize_map, normalize_return_map, normalize_negative, + normalize_binary_pct_encoded_userinfo, + normalize_binary_pct_encoded_query, + normalize_binary_pct_encoded_fragment, + normalize_pct_encoded_userinfo, + normalize_pct_encoded_query, + normalize_pct_encoded_fragment, parse_binary_scheme, parse_binary_userinfo, parse_binary_pct_encoded_userinfo, @@ -120,7 +133,10 @@ all() -> dissect_query, dissect_query_negative, interop_query_latin1, - interop_query_utf8 + interop_query_utf8, + regression_parse, + regression_recompose, + regression_normalize ]. groups() -> @@ -338,20 +354,23 @@ parse_binary_userinfo(_Config) -> uri_string:parse(<<"foo://user:password@localhost">>). parse_binary_pct_encoded_userinfo(_Config) -> - #{scheme := <<"user">>, path := <<"合@気道"/utf8>>} = + #{scheme := <<"user">>, path := <<"%E5%90%88@%E6%B0%97%E9%81%93">>} = uri_string:parse(<<"user:%E5%90%88@%E6%B0%97%E9%81%93">>), - #{path := <<"合気道@"/utf8>>} = uri_string:parse(<<"%E5%90%88%E6%B0%97%E9%81%93@">>), - #{path := <<"/合気道@"/utf8>>} = uri_string:parse(<<"/%E5%90%88%E6%B0%97%E9%81%93@">>), - #{path := <<"合@気道"/utf8>>} = uri_string:parse(<<"%E5%90%88@%E6%B0%97%E9%81%93">>), - #{userinfo := <<"合"/utf8>>, host := <<"気道"/utf8>>} = + #{path := <<"%E5%90%88%E6%B0%97%E9%81%93@">>} = + uri_string:parse(<<"%E5%90%88%E6%B0%97%E9%81%93@">>), + #{path := <<"/%E5%90%88%E6%B0%97%E9%81%93@">>} = + uri_string:parse(<<"/%E5%90%88%E6%B0%97%E9%81%93@">>), + #{path := <<"%E5%90%88@%E6%B0%97%E9%81%93">>} = + uri_string:parse(<<"%E5%90%88@%E6%B0%97%E9%81%93">>), + #{userinfo := <<"%E5%90%88">>, host := <<"%E6%B0%97%E9%81%93">>} = uri_string:parse(<<"//%E5%90%88@%E6%B0%97%E9%81%93">>), - #{userinfo := <<"合:気"/utf8>>, host := <<"道"/utf8>>} = + #{userinfo := <<"%E5%90%88:%E6%B0%97">>, host := <<"%E9%81%93">>} = uri_string:parse(<<"//%E5%90%88:%E6%B0%97@%E9%81%93">>), - #{scheme := <<"foo">>, path := <<"/合気道@"/utf8>>} = + #{scheme := <<"foo">>, path := <<"/%E5%90%88%E6%B0%97%E9%81%93@">>} = uri_string:parse(<<"foo:/%E5%90%88%E6%B0%97%E9%81%93@">>), - #{scheme := <<"foo">>, userinfo := <<"合"/utf8>>, host := <<"気道"/utf8>>} = + #{scheme := <<"foo">>, userinfo := <<"%E5%90%88">>, host := <<"%E6%B0%97%E9%81%93">>} = uri_string:parse(<<"foo://%E5%90%88@%E6%B0%97%E9%81%93">>), - #{scheme := <<"foo">>, userinfo := <<"合:気"/utf8>>, host := <<"道"/utf8>>} = + #{scheme := <<"foo">>, userinfo := <<"%E5%90%88:%E6%B0%97">>, host := <<"%E9%81%93">>} = uri_string:parse(<<"foo://%E5%90%88:%E6%B0%97@%E9%81%93">>), {error,invalid_uri,"@"} = uri_string:parse(<<"//%E5%90%88@%E6%B0%97%E9%81%93@">>), {error,invalid_uri,":"} = uri_string:parse(<<"foo://%E5%90%88@%E6%B0%97%E9%81%93@">>). @@ -369,8 +388,8 @@ parse_binary_host_ipv4(_Config) -> #{host := <<"127.0.0.1">>, query := <<"name=ferret">>} = uri_string:parse(<<"//127.0.0.1?name=ferret">>), #{host := <<"127.0.0.1">>, fragment := <<"nose">>} = uri_string:parse(<<"//127.0.0.1#nose">>), - {error,invalid_uri,"x"} = uri_string:parse(<<"//127.0.0.x">>), - {error,invalid_uri,"1227.0.0.1"} = uri_string:parse(<<"//1227.0.0.1">>). + #{host := <<"127.0.0.x">>,path := <<>>} = uri_string:parse(<<"//127.0.0.x">>), + #{host := <<"1227.0.0.1">>,path := <<>>} = uri_string:parse(<<"//1227.0.0.1">>). parse_binary_host_ipv6(_Config) -> #{host := <<"::127.0.0.1">>} = uri_string:parse(<<"//[::127.0.0.1]">>), @@ -439,9 +458,9 @@ parse_binary_query(_Config) -> parse_binary_pct_encoded_query(_Config) -> #{scheme := <<"foo">>, host := <<"example.com">>, path := <<"/">>, - query := <<"name=合気道"/utf8>>} = + query := <<"name=%E5%90%88%E6%B0%97%E9%81%93">>} = uri_string:parse(<<"foo://example.com/?name=%E5%90%88%E6%B0%97%E9%81%93">>), - #{host := <<"example.com">>, path := <<"/">>, query := <<"name=合気道"/utf8>>} = + #{host := <<"example.com">>, path := <<"/">>, query := <<"name=%E5%90%88%E6%B0%97%E9%81%93">>} = uri_string:parse(<<"//example.com/?name=%E5%90%88%E6%B0%97%E9%81%93">>). parse_binary_fragment(_Config) -> @@ -472,9 +491,11 @@ parse_binary_fragment(_Config) -> uri_string:parse(<<"//example.com/#nose">>). parse_binary_pct_encoded_fragment(_Config) -> - #{scheme := <<"foo">>, host := <<"example.com">>, fragment := <<"合気道"/utf8>>} = + #{scheme := <<"foo">>, host := <<"example.com">>, + fragment := <<"%E5%90%88%E6%B0%97%E9%81%93">>} = uri_string:parse(<<"foo://example.com#%E5%90%88%E6%B0%97%E9%81%93">>), - #{host := <<"example.com">>, path := <<"/">>, fragment := <<"合気道"/utf8>>} = + #{host := <<"example.com">>, path := <<"/">>, + fragment := <<"%E5%90%88%E6%B0%97%E9%81%93">>} = uri_string:parse(<<"//example.com/#%E5%90%88%E6%B0%97%E9%81%93">>). parse_scheme(_Config) -> @@ -506,25 +527,27 @@ parse_userinfo(_Config) -> uri_string:parse("foo://user:password@localhost"). parse_pct_encoded_userinfo(_Config) -> - #{scheme := "user", path := "合@気道"} = + #{scheme := "user", path := "%E5%90%88@%E6%B0%97%E9%81%93"} = uri_string:parse("user:%E5%90%88@%E6%B0%97%E9%81%93"), - #{path := "合気道@"} = uri_string:parse("%E5%90%88%E6%B0%97%E9%81%93@"), - #{path := "/合気道@"} = uri_string:parse("/%E5%90%88%E6%B0%97%E9%81%93@"), - #{path := "合@気道"} = uri_string:parse("%E5%90%88@%E6%B0%97%E9%81%93"), - #{userinfo := "合", host := "気道"} = + #{path := "%E5%90%88%E6%B0%97%E9%81%93@"} = + uri_string:parse("%E5%90%88%E6%B0%97%E9%81%93@"), + #{path := "/%E5%90%88%E6%B0%97%E9%81%93@"} = + uri_string:parse("/%E5%90%88%E6%B0%97%E9%81%93@"), + #{path := "%E5%90%88@%E6%B0%97%E9%81%93"} = + uri_string:parse("%E5%90%88@%E6%B0%97%E9%81%93"), + #{userinfo := "%E5%90%88", host := "%E6%B0%97%E9%81%93"} = uri_string:parse("//%E5%90%88@%E6%B0%97%E9%81%93"), - #{userinfo := "合:気", host := "道"} = + #{userinfo := "%E5%90%88:%E6%B0%97", host := "%E9%81%93"} = uri_string:parse("//%E5%90%88:%E6%B0%97@%E9%81%93"), - #{scheme := "foo", path := "/合気道@"} = + #{scheme := "foo", path := "/%E5%90%88%E6%B0%97%E9%81%93@"} = uri_string:parse("foo:/%E5%90%88%E6%B0%97%E9%81%93@"), - #{scheme := "foo", userinfo := "合", host := "気道"} = + #{scheme := "foo", userinfo := "%E5%90%88", host := "%E6%B0%97%E9%81%93"} = uri_string:parse("foo://%E5%90%88@%E6%B0%97%E9%81%93"), - #{scheme := "foo", userinfo := "合:気", host := "道"} = + #{scheme := "foo", userinfo := "%E5%90%88:%E6%B0%97", host := "%E9%81%93"} = uri_string:parse("foo://%E5%90%88:%E6%B0%97@%E9%81%93"), {error,invalid_uri,"@"} = uri_string:parse("//%E5%90%88@%E6%B0%97%E9%81%93@"), {error,invalid_uri,":"} = uri_string:parse("foo://%E5%90%88@%E6%B0%97%E9%81%93@"). - parse_host(_Config) -> #{host := "hostname"} = uri_string:parse("//hostname"), #{host := "hostname",scheme := "foo"} = uri_string:parse("foo://hostname"), @@ -538,8 +561,8 @@ parse_host_ipv4(_Config) -> #{host := "127.0.0.1", path := "/over/there"} = uri_string:parse("//127.0.0.1/over/there"), #{host := "127.0.0.1", query := "name=ferret"} = uri_string:parse("//127.0.0.1?name=ferret"), #{host := "127.0.0.1", fragment := "nose"} = uri_string:parse("//127.0.0.1#nose"), - {error,invalid_uri,"x"} = uri_string:parse("//127.0.0.x"), - {error,invalid_uri,"1227.0.0.1"} = uri_string:parse("//1227.0.0.1"). + #{host := "127.0.0.x",path := []} = uri_string:parse("//127.0.0.x"), + #{host := "1227.0.0.1",path := []} = uri_string:parse("//1227.0.0.1"). parse_host_ipv6(_Config) -> #{host := "::127.0.0.1"} = uri_string:parse("//[::127.0.0.1]"), @@ -602,9 +625,9 @@ parse_query(_Config) -> parse_pct_encoded_query(_Config) -> #{scheme := "foo", host := "example.com", path := "/", - query := "name=合気道"} = + query := "name=%E5%90%88%E6%B0%97%E9%81%93"} = uri_string:parse("foo://example.com/?name=%E5%90%88%E6%B0%97%E9%81%93"), - #{host := "example.com", path := "/", query := "name=合気道"} = + #{host := "example.com", path := "/", query := "name=%E5%90%88%E6%B0%97%E9%81%93"} = uri_string:parse("//example.com/?name=%E5%90%88%E6%B0%97%E9%81%93"). parse_fragment(_Config) -> @@ -635,9 +658,11 @@ parse_fragment(_Config) -> uri_string:parse("//example.com/#nose"). parse_pct_encoded_fragment(_Config) -> - #{scheme := "foo", host := "example.com", fragment := "合気道"} = + #{scheme := "foo", host := "example.com", + fragment := "%E5%90%88%E6%B0%97%E9%81%93"} = uri_string:parse("foo://example.com#%E5%90%88%E6%B0%97%E9%81%93"), - #{host := "example.com", path := "/", fragment := "合気道"} = + #{host := "example.com", path := "/", + fragment := "%E5%90%88%E6%B0%97%E9%81%93"} = uri_string:parse("//example.com/#%E5%90%88%E6%B0%97%E9%81%93"). parse_list(_Config) -> @@ -711,9 +736,7 @@ parse_negative(_Config) -> {error,invalid_uri,":"} = uri_string:parse("foo://usär@host"), {error,invalid_uri,"ö"} = uri_string:parse("//host/path?foö=bar"), {error,invalid_uri,"ö"} = uri_string:parse("//host/path#foö"), - {error,invalid_uri,"127.256.0.1"} = uri_string:parse("//127.256.0.1"), {error,invalid_uri,":::127.0.0.1"} = uri_string:parse("//[:::127.0.0.1]"), - {error,invalid_utf8,<<0,0,0,246>>} = uri_string:parse("//%00%00%00%F6"), {error,invalid_uri,"A"} = uri_string:parse("//localhost:A8"). @@ -839,9 +862,11 @@ transcode_negative(_Config) -> compose_query(_Config) -> [] = uri_string:compose_query([]), "foo=1&bar=2" = uri_string:compose_query([{<<"foo">>,"1"}, {"bar", "2"}]), + "foo=1&bar" = uri_string:compose_query([{<<"foo">>,"1"}, {"bar", true}]), "foo=1&b%C3%A4r=2" = uri_string:compose_query([{"foo","1"}, {"bär", "2"}],[{encoding,utf8}]), "foo=1&b%C3%A4r=2" = uri_string:compose_query([{"foo","1"}, {"bär", "2"}],[{encoding,unicode}]), "foo=1&b%E4r=2" = uri_string:compose_query([{"foo","1"}, {"bär", "2"}],[{encoding,latin1}]), + "foo&b%E4r=2" = uri_string:compose_query([{"foo",true}, {"bär", "2"}],[{encoding,latin1}]), "foo+bar=1&%E5%90%88=2" = uri_string:compose_query([{"foo bar","1"}, {"合", "2"}]), "foo+bar=1&%26%2321512%3B=2" = uri_string:compose_query([{"foo bar","1"}, {"合", "2"}],[{encoding,latin1}]), @@ -883,11 +908,13 @@ dissect_query(_Config) -> [{"föo bar","1"},{"ö","2"}] = uri_string:dissect_query("föo+bar=1&%C3%B6=2"), [{<<"föo bar"/utf8>>,<<"1">>},{<<"ö"/utf8>>,<<"2">>}] = - uri_string:dissect_query(<<"föo+bar=1&%C3%B6=2"/utf8>>). + uri_string:dissect_query(<<"föo+bar=1&%C3%B6=2"/utf8>>), + [{"foo1",true},{"bar","2"}] = + uri_string:dissect_query("foo1&bar=2"), + [{<<"foo1">>,<<"1">>},{<<"bar">>,true}] = + uri_string:dissect_query(<<"foo1=1&bar">>). dissect_query_negative(_Config) -> - {error,missing_value,"&"} = - uri_string:dissect_query("foo1&bar=2"), {error,invalid_percent_encoding,"%XX%B6"} = uri_string:dissect_query("foo=%XX%B6&bar=2"), {error,invalid_input,[153]} = uri_string:dissect_query("foo=%99%B6&bar=2"), @@ -913,7 +940,9 @@ normalize(_Config) -> <<"sftp://localhost">> = uri_string:normalize(<<"sftp://localhost:22">>), <<"tftp://localhost">> = - uri_string:normalize(<<"tftp://localhost:69">>). + uri_string:normalize(<<"tftp://localhost:69">>), + <<"/foo/%2F/bar">> = + uri_string:normalize(<<"/foo/%2f/%62ar">>). normalize_map(_Config) -> "/a/g" = uri_string:normalize(#{path => "/a/b/c/./../../g"}), @@ -942,7 +971,9 @@ normalize_map(_Config) -> host => <<"localhost">>}), <<"tftp://localhost">> = uri_string:normalize(#{scheme => <<"tftp">>,port => 69,path => <<>>, - host => <<"localhost">>}). + host => <<"localhost">>}), + "/foo/%2F/bar" = + uri_string:normalize(#{path => "/foo/%2f/%62ar"}). normalize_return_map(_Config) -> #{scheme := "http",path := "/a/g",host := "localhost-örebro"} = @@ -963,7 +994,82 @@ normalize_negative(_Config) -> {error,invalid_uri,":"} = uri_string:normalize("http://[192.168.0.1]", [return_map]), {error,invalid_uri,":"} = - uri_string:normalize(<<"http://[192.168.0.1]">>, [return_map]). + uri_string:normalize(<<"http://[192.168.0.1]">>, [return_map]), + {error,invalid_utf8,<<0,0,0,246>>} = uri_string:normalize("//%00%00%00%F6"). + +normalize_binary_pct_encoded_userinfo(_Config) -> + #{scheme := <<"user">>, path := <<"合@気道"/utf8>>} = + uri_string:normalize(<<"user:%E5%90%88@%E6%B0%97%E9%81%93">>, [return_map]), + #{path := <<"合気道@"/utf8>>} = + uri_string:normalize(<<"%E5%90%88%E6%B0%97%E9%81%93@">>, [return_map]), + #{path := <<"/合気道@"/utf8>>} = + uri_string:normalize(<<"/%E5%90%88%E6%B0%97%E9%81%93@">>, [return_map]), + #{path := <<"合@気道"/utf8>>} = + uri_string:normalize(<<"%E5%90%88@%E6%B0%97%E9%81%93">>, [return_map]), + #{userinfo := <<"合"/utf8>>, host := <<"気道"/utf8>>} = + uri_string:normalize(<<"//%E5%90%88@%E6%B0%97%E9%81%93">>, [return_map]), + #{userinfo := <<"合:気"/utf8>>, host := <<"道"/utf8>>} = + uri_string:normalize(<<"//%E5%90%88:%E6%B0%97@%E9%81%93">>, [return_map]), + #{scheme := <<"foo">>, path := <<"/合気道@"/utf8>>} = + uri_string:normalize(<<"foo:/%E5%90%88%E6%B0%97%E9%81%93@">>, [return_map]), + #{scheme := <<"foo">>, userinfo := <<"合"/utf8>>, host := <<"気道"/utf8>>} = + uri_string:normalize(<<"foo://%E5%90%88@%E6%B0%97%E9%81%93">>, [return_map]), + #{scheme := <<"foo">>, userinfo := <<"合:気"/utf8>>, host := <<"道"/utf8>>} = + uri_string:normalize(<<"foo://%E5%90%88:%E6%B0%97@%E9%81%93">>, [return_map]), + {error,invalid_uri,"@"} = + uri_string:normalize(<<"//%E5%90%88@%E6%B0%97%E9%81%93@">>, [return_map]), + {error,invalid_uri,":"} = + uri_string:normalize(<<"foo://%E5%90%88@%E6%B0%97%E9%81%93@">>, [return_map]). + +normalize_binary_pct_encoded_query(_Config) -> + #{scheme := <<"foo">>, host := <<"example.com">>, path := <<"/">>, + query := <<"name=合気道"/utf8>>} = + uri_string:normalize(<<"foo://example.com/?name=%E5%90%88%E6%B0%97%E9%81%93">>, [return_map]), + #{host := <<"example.com">>, path := <<"/">>, query := <<"name=合気道"/utf8>>} = + uri_string:normalize(<<"//example.com/?name=%E5%90%88%E6%B0%97%E9%81%93">>, [return_map]). + +normalize_binary_pct_encoded_fragment(_Config) -> + #{scheme := <<"foo">>, host := <<"example.com">>, fragment := <<"合気道"/utf8>>} = + uri_string:normalize(<<"foo://example.com#%E5%90%88%E6%B0%97%E9%81%93">>, [return_map]), + #{host := <<"example.com">>, path := <<"/">>, fragment := <<"合気道"/utf8>>} = + uri_string:normalize(<<"//example.com/#%E5%90%88%E6%B0%97%E9%81%93">>, [return_map]). + +normalize_pct_encoded_userinfo(_Config) -> + #{scheme := "user", path := "合@気道"} = + uri_string:normalize("user:%E5%90%88@%E6%B0%97%E9%81%93", [return_map]), + #{path := "合気道@"} = + uri_string:normalize("%E5%90%88%E6%B0%97%E9%81%93@", [return_map]), + #{path := "/合気道@"} = + uri_string:normalize("/%E5%90%88%E6%B0%97%E9%81%93@", [return_map]), + #{path := "合@気道"} = + uri_string:normalize("%E5%90%88@%E6%B0%97%E9%81%93", [return_map]), + #{userinfo := "合", host := "気道"} = + uri_string:normalize("//%E5%90%88@%E6%B0%97%E9%81%93", [return_map]), + #{userinfo := "合:気", host := "道"} = + uri_string:normalize("//%E5%90%88:%E6%B0%97@%E9%81%93", [return_map]), + #{scheme := "foo", path := "/合気道@"} = + uri_string:normalize("foo:/%E5%90%88%E6%B0%97%E9%81%93@", [return_map]), + #{scheme := "foo", userinfo := "合", host := "気道"} = + uri_string:normalize("foo://%E5%90%88@%E6%B0%97%E9%81%93", [return_map]), + #{scheme := "foo", userinfo := "合:気", host := "道"} = + uri_string:normalize("foo://%E5%90%88:%E6%B0%97@%E9%81%93", [return_map]), + {error,invalid_uri,"@"} = + uri_string:normalize("//%E5%90%88@%E6%B0%97%E9%81%93@", [return_map]), + {error,invalid_uri,":"} = + uri_string:normalize("foo://%E5%90%88@%E6%B0%97%E9%81%93@", [return_map]). + +normalize_pct_encoded_query(_Config) -> + #{scheme := "foo", host := "example.com", path := "/", + query := "name=合気道"} = + uri_string:normalize("foo://example.com/?name=%E5%90%88%E6%B0%97%E9%81%93", [return_map]), + #{host := "example.com", path := "/", query := "name=合気道"} = + uri_string:normalize("//example.com/?name=%E5%90%88%E6%B0%97%E9%81%93", [return_map]). + +normalize_pct_encoded_fragment(_Config) -> + #{scheme := "foo", host := "example.com", fragment := "合気道"} = + uri_string:normalize("foo://example.com#%E5%90%88%E6%B0%97%E9%81%93", [return_map]), + #{host := "example.com", path := "/", fragment := "合気道"} = + uri_string:normalize("//example.com/#%E5%90%88%E6%B0%97%E9%81%93", [return_map]). interop_query_utf8(_Config) -> Q = uri_string:compose_query([{"foo bar","1"}, {"合", "2"}]), @@ -977,3 +1083,86 @@ interop_query_latin1(_Config) -> Uri1 = uri_string:transcode(Uri, [{in_encoding, latin1}]), #{query := Q1} = uri_string:parse(Uri1), [{"foo bar","1"}, {"合", "2"}] = uri_string:dissect_query(Q1). + +regression_parse(_Config) -> + #{host := "Bar",path := [],scheme := "FOo"} = + uri_string:parse("FOo://Bar"), + #{host := "bar",path := [],scheme := "foo"} = + uri_string:parse("foo://bar"), + #{host := "A%2f",path := "/%62ar",scheme := "foo"} = + uri_string:parse("foo://A%2f/%62ar"), + #{host := "a%2F",path := "/bar",scheme := "foo"} = + uri_string:parse("foo://a%2F/bar"), + #{host := "%C3%B6",path := [],scheme := "FOo"} = + uri_string:parse("FOo://%C3%B6"). + +regression_recompose(_Config) -> + "FOo://Bar" = + uri_string:recompose(#{host => "Bar",path => [],scheme => "FOo"}), + "foo://bar" = + uri_string:recompose(#{host => "bar",path => [],scheme => "foo"}), + "foo://A%2f/%62ar" = + uri_string:recompose(#{host => "A%2f",path => "/%62ar",scheme => "foo"}), + "foo://a%2F/bar" = + uri_string:recompose(#{host => "a%2F",path => "/bar",scheme => "foo"}), + "FOo://%C3%B6" = + uri_string:recompose(#{host => "%C3%B6",path => [],scheme => "FOo"}), + "FOo://%C3%B6" = + uri_string:recompose(#{host => "ö",path => [],scheme => "FOo"}). + +regression_normalize(_Config) -> + "foo://bar" = + uri_string:normalize("FOo://Bar"), + #{host := "bar",path := [],scheme := "foo"} = + uri_string:normalize("FOo://Bar", [return_map]), + + "foo://bar" = + uri_string:normalize("foo://bar"), + #{host := "bar",path := [],scheme := "foo"} = + uri_string:normalize("foo://bar", [return_map]), + + "foo://a%2F/bar" = + uri_string:normalize("foo://A%2f/%62ar"), + #{host := "a%2F",path := "/bar",scheme := "foo"} = + uri_string:normalize("foo://A%2f/%62ar", [return_map]), + + "foo://a%2F/bar" = + uri_string:normalize("foo://a%2F/bar"), + #{host := "a%2F",path := "/bar",scheme := "foo"} = + uri_string:normalize("foo://a%2F/bar", [return_map]), + + "foo://%C3%B6" = + uri_string:normalize("FOo://%C3%B6"), + #{host := "ö",path := [],scheme := "foo"} = + uri_string:normalize("FOo://%C3%B6", [return_map]), + + + "foo://bar" = + uri_string:normalize(#{host => "Bar",path => [],scheme => "FOo"}), + #{host := "bar",path := [],scheme := "foo"} = + uri_string:normalize(#{host => "Bar",path => [],scheme => "FOo"}, [return_map]), + + "foo://bar" = + uri_string:normalize(#{host => "bar",path => [],scheme => "foo"}), + #{host := "bar",path := [],scheme := "foo"} = + uri_string:normalize(#{host => "bar",path => [],scheme => "foo"}, [return_map]), + + "foo://a%2F/bar" = + uri_string:normalize(#{host => "A%2f",path => "/%62ar",scheme => "foo"}), + #{host := "a%2F",path := "/bar",scheme := "foo"} = + uri_string:normalize(#{host => "A%2f",path => "/%62ar",scheme => "foo"}, [return_map]), + + "foo://a%2F/bar" = + uri_string:normalize(#{host => "a%2F",path => "/bar",scheme => "foo"}), + #{host := "a%2F",path := "/bar",scheme := "foo"} = + uri_string:normalize(#{host => "a%2F",path => "/bar",scheme => "foo"}, [return_map]), + + "foo://%C3%B6" = + uri_string:normalize(#{host => "%C3%B6",path => [],scheme => "FOo"}), + #{host := "ö",path := [],scheme := "foo"} = + uri_string:normalize(#{host => "%C3%B6",path => [],scheme => "FOo"}, [return_map]), + + "foo://%C3%B6" = + uri_string:normalize(#{host => "ö",path => [],scheme => "FOo"}), + #{host := "ö",path := [],scheme := "foo"} = + uri_string:normalize(#{host => "ö",path => [],scheme => "FOo"}, [return_map]). diff --git a/lib/stdlib/test/uri_string_property_test_SUITE.erl b/lib/stdlib/test/uri_string_property_test_SUITE.erl index ae2c61c7aa..f1d27924db 100644 --- a/lib/stdlib/test/uri_string_property_test_SUITE.erl +++ b/lib/stdlib/test/uri_string_property_test_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2017. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -22,7 +22,7 @@ -include_lib("common_test/include/ct.hrl"). -compile(export_all). -all() -> [recompose]. +all() -> [recompose, normalize]. init_per_suite(Config) -> ct_property_test:init_per_suite(Config). @@ -37,3 +37,8 @@ recompose(Config) -> ct_property_test:quickcheck( uri_string_recompose:prop_recompose(), Config). + +normalize(Config) -> + ct_property_test:quickcheck( + uri_string_recompose:prop_normalize(), + Config). diff --git a/lib/stdlib/test/zip_SUITE.erl b/lib/stdlib/test/zip_SUITE.erl index e5ba629c55..081bffa7cb 100644 --- a/lib/stdlib/test/zip_SUITE.erl +++ b/lib/stdlib/test/zip_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2006-2017. All Rights Reserved. +%% Copyright Ericsson AB 2006-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/stdlib/vsn.mk b/lib/stdlib/vsn.mk index 09a4d6fb50..1d833430f1 100644 --- a/lib/stdlib/vsn.mk +++ b/lib/stdlib/vsn.mk @@ -1 +1 @@ -STDLIB_VSN = 3.4.5 +STDLIB_VSN = 3.5.1 diff --git a/lib/syntax_tools/doc/src/Makefile b/lib/syntax_tools/doc/src/Makefile index a346b9a0bd..d953287bad 100644 --- a/lib/syntax_tools/doc/src/Makefile +++ b/lib/syntax_tools/doc/src/Makefile @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 2006-2017. All Rights Reserved. +# Copyright Ericsson AB 2006-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -125,6 +125,7 @@ debug opt: clean clean_docs: rm -rf $(HTMLDIR)/* + rm -rf $(XMLDIR) rm -f $(MAN3DIR)/* rm -f $(XML_REF3_FILES) $(XML_CHAPTER_FILES) *.html rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) diff --git a/lib/syntax_tools/doc/src/notes.xml b/lib/syntax_tools/doc/src/notes.xml index bd2bcde2c2..44944e57c3 100644 --- a/lib/syntax_tools/doc/src/notes.xml +++ b/lib/syntax_tools/doc/src/notes.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2007</year><year>2017</year> + <year>2007</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -32,6 +32,31 @@ <p>This document describes the changes made to the Syntax_Tools application.</p> +<section><title>Syntax_Tools 2.1.5</title> + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Update to use the new string api instead of the old.</p> + <p> + Own Id: OTP-15036</p> + </item> + </list> + </section> +</section> + +<section><title>Syntax_Tools 2.1.4.1</title> + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> Fix a bug regarding reverting map types. </p> + <p> + Own Id: OTP-15098 Aux Id: ERIERL-177 </p> + </item> + </list> + </section> +</section> + <section><title>Syntax_Tools 2.1.4</title> <section><title>Fixed Bugs and Malfunctions</title> @@ -83,6 +108,20 @@ </section> +<section><title>Syntax_Tools 2.1.1.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> Fix a bug regarding reverting map types. </p> + <p> + Own Id: OTP-15098 Aux Id: ERIERL-177 </p> + </item> + </list> + </section> + +</section> + <section><title>Syntax_Tools 2.1.1</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/syntax_tools/src/erl_syntax.erl b/lib/syntax_tools/src/erl_syntax.erl index 029f1e88ac..758aff32fd 100644 --- a/lib/syntax_tools/src/erl_syntax.erl +++ b/lib/syntax_tools/src/erl_syntax.erl @@ -7223,7 +7223,7 @@ macro_arguments(Node) -> %% @doc Returns the syntax tree corresponding to an Erlang term. %% `Term' must be a literal term, i.e., one that can be %% represented as a source code literal. Thus, it may not contain a -%% process identifier, port, reference, binary or function value as a +%% process identifier, port, reference or function value as a %% subterm. The function recognises printable strings, in order to get a %% compact and readable representation. Evaluation fails with reason %% `badarg' if `Term' is not a literal term. @@ -7257,6 +7257,13 @@ abstract(T) when is_map(T) -> || {Key,Value} <- maps:to_list(T)]); abstract(T) when is_binary(T) -> binary([binary_field(integer(B)) || B <- binary_to_list(T)]); +abstract(T) when is_bitstring(T) -> + S = bit_size(T), + ByteS = S div 8, + BitS = S rem 8, + <<Bin:ByteS/binary, I:BitS>> = T, + binary([binary_field(integer(B)) || B <- binary_to_list(Bin)] + ++ [binary_field(integer(I), integer(BitS), [])]); abstract(T) -> erlang:error({badarg, T}). @@ -7332,15 +7339,20 @@ concrete(Node) -> Node0 -> maps:merge(concrete(Node0),M0) end; binary -> - Fs = [revert_binary_field( - binary_field(binary_field_body(F), - case binary_field_size(F) of - none -> none; - S -> - revert(S) - end, - binary_field_types(F))) - || F <- binary_fields(Node)], + Fs = [begin + B = binary_field_body(F), + {Body, Size} = + case type(B) of + size_qualifier -> + {size_qualifier_body(B), + size_qualifier_argument(B)}; + _ -> + {B, none} + end, + revert_binary_field( + binary_field(Body, Size, binary_field_types(F))) + end + || F <- binary_fields(Node)], {value, B, _} = eval_bits:expr_grp(Fs, [], fun(F, _) -> @@ -7413,7 +7425,14 @@ is_literal(T) -> is_literal_binary_field(F) -> case binary_field_types(F) of - [] -> is_literal(binary_field_body(F)); + [] -> B = binary_field_body(F), + case type(B) of + size_qualifier -> + is_literal(size_qualifier_body(B)) andalso + is_literal(size_qualifier_argument(B)); + _ -> + is_literal(B) + end; _ -> false end. diff --git a/lib/syntax_tools/src/erl_syntax_lib.erl b/lib/syntax_tools/src/erl_syntax_lib.erl index ced0dba3e2..352165893f 100644 --- a/lib/syntax_tools/src/erl_syntax_lib.erl +++ b/lib/syntax_tools/src/erl_syntax_lib.erl @@ -1981,7 +1981,7 @@ analyze_application(Node) -> %% %% @see analyze_type_name/1 --type typeName() :: atom() | {module(), atom(), arity()} | {atom(), arity()}. +-type typeName() :: atom() | {module(), {atom(), arity()}} | {atom(), arity()}. -spec analyze_type_application(erl_syntax:syntaxTree()) -> typeName(). diff --git a/lib/syntax_tools/src/igor.erl b/lib/syntax_tools/src/igor.erl index 16e3511734..b712b77e9f 100644 --- a/lib/syntax_tools/src/igor.erl +++ b/lib/syntax_tools/src/igor.erl @@ -660,7 +660,7 @@ merge_files1(Files, Opts) -> %% transitions), code replacement is expected to be detected. Then, if %% we in the merged code do not check at these points if the %% <em>target</em> module (the result of the merge) has been replaced, -%% we can not be sure in general that we will be able to do code +%% we cannot be sure in general that we will be able to do code %% replacement of the merged state machine - it could run forever %% without detecting the code change. Therefore, all such calls must %% remain remote-calls (detecting code changes), but may call the target diff --git a/lib/syntax_tools/test/syntax_tools_SUITE.erl b/lib/syntax_tools/test/syntax_tools_SUITE.erl index c8e6448d37..4cddf8f0c3 100644 --- a/lib/syntax_tools/test/syntax_tools_SUITE.erl +++ b/lib/syntax_tools/test/syntax_tools_SUITE.erl @@ -157,6 +157,7 @@ t_abstract_type(Config) when is_list(Config) -> {[$a,$b,$c],string}, {"hello world",string}, {<<1,2,3>>,binary}, + {<<1,2,3:4>>,binary}, {#{a=>1,"b"=>2},map_expr}, {#{#{i=>1}=>1,"b"=>#{v=>2}},map_expr}, {{a,b,c},tuple}]), diff --git a/lib/syntax_tools/vsn.mk b/lib/syntax_tools/vsn.mk index 8d37c40742..4d13267f16 100644 --- a/lib/syntax_tools/vsn.mk +++ b/lib/syntax_tools/vsn.mk @@ -1 +1 @@ -SYNTAX_TOOLS_VSN = 2.1.4 +SYNTAX_TOOLS_VSN = 2.1.5 diff --git a/lib/tftp/Makefile b/lib/tftp/Makefile index 5c3ed52b28..a4559fbc2e 100644 --- a/lib/tftp/Makefile +++ b/lib/tftp/Makefile @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 1996-2016. All Rights Reserved. +# Copyright Ericsson AB 1996-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. diff --git a/lib/tftp/doc/src/Makefile b/lib/tftp/doc/src/Makefile index a2fdcf6325..5d76799e41 100644 --- a/lib/tftp/doc/src/Makefile +++ b/lib/tftp/doc/src/Makefile @@ -103,6 +103,7 @@ pdf: $(TOP_PDF_FILE) html: gifs $(HTML_REF_MAN_FILE) clean clean_docs: clean_html clean_man clean_pdf + rm -rf $(XMLDIR) rm -f errs core *~ man: $(MAN3_FILES) diff --git a/lib/tftp/src/Makefile b/lib/tftp/src/Makefile index ed1551ba04..602ae845a4 100644 --- a/lib/tftp/src/Makefile +++ b/lib/tftp/src/Makefile @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 2005-2016. All Rights Reserved. +# Copyright Ericsson AB 2005-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. diff --git a/lib/tftp/src/tftp.erl b/lib/tftp/src/tftp.erl index 27ed13694b..31e4c651e8 100644 --- a/lib/tftp/src/tftp.erl +++ b/lib/tftp/src/tftp.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2016. All Rights Reserved. +%% Copyright Ericsson AB 2005-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/tftp/src/tftp.hrl b/lib/tftp/src/tftp.hrl index 25543e0b9e..a26aeee389 100644 --- a/lib/tftp/src/tftp.hrl +++ b/lib/tftp/src/tftp.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2016. All Rights Reserved. +%% Copyright Ericsson AB 2005-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/tftp/src/tftp_binary.erl b/lib/tftp/src/tftp_binary.erl index 09adcfc41f..3438ba235b 100644 --- a/lib/tftp/src/tftp_binary.erl +++ b/lib/tftp/src/tftp_binary.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2016. All Rights Reserved. +%% Copyright Ericsson AB 2005-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/tftp/src/tftp_engine.erl b/lib/tftp/src/tftp_engine.erl index f14354ad6a..78a105bcc8 100644 --- a/lib/tftp/src/tftp_engine.erl +++ b/lib/tftp/src/tftp_engine.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2016. All Rights Reserved. +%% Copyright Ericsson AB 2005-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/tftp/src/tftp_file.erl b/lib/tftp/src/tftp_file.erl index 43b588f71a..5922fc9418 100644 --- a/lib/tftp/src/tftp_file.erl +++ b/lib/tftp/src/tftp_file.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2016. All Rights Reserved. +%% Copyright Ericsson AB 2005-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/tftp/src/tftp_lib.erl b/lib/tftp/src/tftp_lib.erl index 454754f0a3..407a273f58 100644 --- a/lib/tftp/src/tftp_lib.erl +++ b/lib/tftp/src/tftp_lib.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2016. All Rights Reserved. +%% Copyright Ericsson AB 2005-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/tftp/src/tftp_logger.erl b/lib/tftp/src/tftp_logger.erl index a869958484..548ed509dc 100644 --- a/lib/tftp/src/tftp_logger.erl +++ b/lib/tftp/src/tftp_logger.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2016. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/tftp/src/tftp_sup.erl b/lib/tftp/src/tftp_sup.erl index 0475e53e42..13b753612b 100644 --- a/lib/tftp/src/tftp_sup.erl +++ b/lib/tftp/src/tftp_sup.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2016. All Rights Reserved. +%% Copyright Ericsson AB 2005-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/tftp/test/tftp_SUITE.erl b/lib/tftp/test/tftp_SUITE.erl index fd1d209c25..a0f6cb1ca4 100644 --- a/lib/tftp/test/tftp_SUITE.erl +++ b/lib/tftp/test/tftp_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2006-2016. All Rights Reserved. +%% Copyright Ericsson AB 2006-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/tftp/test/tftp_test_lib.erl b/lib/tftp/test/tftp_test_lib.erl index 45386389cb..04534228c2 100644 --- a/lib/tftp/test/tftp_test_lib.erl +++ b/lib/tftp/test/tftp_test_lib.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2016. All Rights Reserved. +%% Copyright Ericsson AB 2007-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/tftp/test/tftp_test_lib.hrl b/lib/tftp/test/tftp_test_lib.hrl index e7a5a37d2c..eb8ed77fc1 100644 --- a/lib/tftp/test/tftp_test_lib.hrl +++ b/lib/tftp/test/tftp_test_lib.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2016. All Rights Reserved. +%% Copyright Ericsson AB 2007-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/tools/doc/src/Makefile b/lib/tools/doc/src/Makefile index 4b663106a0..5ff4fe3113 100644 --- a/lib/tools/doc/src/Makefile +++ b/lib/tools/doc/src/Makefile @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 1997-2017. All Rights Reserved. +# Copyright Ericsson AB 1997-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -120,6 +120,7 @@ debug opt: clean clean_docs: rm -rf $(HTMLDIR)/* + rm -rf $(XMLDIR) rm -f $(MAN3DIR)/* rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) rm -f $(SPECDIR)/* diff --git a/lib/tools/doc/src/cover.xml b/lib/tools/doc/src/cover.xml index 15cd784253..6c6b20aad8 100644 --- a/lib/tools/doc/src/cover.xml +++ b/lib/tools/doc/src/cover.xml @@ -234,7 +234,7 @@ <c>{already_cover_compiled,no_beam_found,Module}</c> is returned.</p> <p><c>{error,BeamFile}</c> is returned if the compiled code - can not be loaded on the node.</p> + cannot be loaded on the node.</p> <p>If a list of <c>ModFiles</c> is given as input, a list of <c>Result</c> will be returned. The order of the returned list is undefined.</p> @@ -470,7 +470,7 @@ <p>Exports the current coverage data for <c>Module</c> to the file <c>ExportFile</c>. It is recommended to name the <c>ExportFile</c> with the extension <c>.coverdata</c>, since - other filenames can not be read by the web based interface to + other filenames cannot be read by the web based interface to cover.</p> <p>If <c>Module</c> is not given, data for all Cover compiled or earlier imported modules is exported.</p> @@ -496,7 +496,7 @@ <p>Coverage data from several export files can be imported into one system. The coverage data is then added up when analysing.</p> - <p>Coverage data for a module can not be imported from the + <p>Coverage data for a module cannot be imported from the same file twice unless the module is first reset or compiled. The check is based on the filename, so you can easily fool the system by renaming your export file.</p> diff --git a/lib/tools/doc/src/fprof.xml b/lib/tools/doc/src/fprof.xml index 72624bd33b..1fd828d127 100644 --- a/lib/tools/doc/src/fprof.xml +++ b/lib/tools/doc/src/fprof.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>2001</year><year>2016</year> + <year>2001</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/lib/tools/doc/src/instrument.xml b/lib/tools/doc/src/instrument.xml index 9fd9332373..79bacb2927 100644 --- a/lib/tools/doc/src/instrument.xml +++ b/lib/tools/doc/src/instrument.xml @@ -111,15 +111,18 @@ default, but this can be configured an a per-allocator basis with the <seealso marker="erts:erts_alloc#M_atags"><c>+M<S>atags</c> </seealso> emulator option.</p> - <p>If tagged allocations are not enabled on any of the specified - allocator types, the call will fail with - <c>{error, not_enabled}</c>.</p> + <p>If the specified allocator types are not enabled, the call will fail + with <c>{error, not_enabled}</c>.</p> <p>The following options can be used:</p> <taglist> <tag><c>allocator_types</c></tag> <item> - <p>The allocator types that will be searched. Defaults to all - <c>alloc_util</c> allocators.</p> + <p>The allocator types that will be searched. Note that blocks can + move freely between allocator types, so restricting the search to + certain allocators may return unexpected types (e.g. process + heaps when searching binary_alloc), or hide blocks that were + migrated out.</p> + <p>Defaults to all <c>alloc_util</c> allocators.</p> </item> <tag><c>scheduler_ids</c></tag> <item> diff --git a/lib/tools/doc/src/lcnt.xml b/lib/tools/doc/src/lcnt.xml index 0c24375b91..d2595cdb60 100644 --- a/lib/tools/doc/src/lcnt.xml +++ b/lib/tools/doc/src/lcnt.xml @@ -5,7 +5,7 @@ <header> <copyright> <year>2009</year> - <year>2017</year> + <year>2018</year> <holder>Ericsson AB, All Rights Reserved</holder> </copyright> <legalnotice> diff --git a/lib/tools/doc/src/notes.xml b/lib/tools/doc/src/notes.xml index 45f276c09e..8b0a2ca283 100644 --- a/lib/tools/doc/src/notes.xml +++ b/lib/tools/doc/src/notes.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2004</year><year>2017</year> + <year>2004</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -31,6 +31,27 @@ </header> <p>This document describes the changes made to the Tools application.</p> +<section><title>Tools 3.0</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p>Added <c>instrument:allocations</c> and + <c>instrument:carriers</c> for retrieving information + about memory utilization and fragmentation.</p> + <p>The old <c>instrument</c> interface has been removed, + as have the related options <c>+Mim</c> and + <c>+Mis</c>.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-14961</p> + </item> + </list> + </section> + +</section> + <section><title>Tools 2.11.2</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/tools/emacs/erlang.el b/lib/tools/emacs/erlang.el index e08db0ea79..3211132254 100644 --- a/lib/tools/emacs/erlang.el +++ b/lib/tools/emacs/erlang.el @@ -9,7 +9,7 @@ ;; %CopyrightBegin% ;; -;; Copyright Ericsson AB 1996-2017. All Rights Reserved. +;; Copyright Ericsson AB 1996-2018. All Rights Reserved. ;; ;; Licensed under the Apache License, Version 2.0 (the "License"); ;; you may not use this file except in compliance with the License. diff --git a/lib/tools/priv/styles.css b/lib/tools/priv/styles.css new file mode 100644 index 0000000000..e10e94e3ad --- /dev/null +++ b/lib/tools/priv/styles.css @@ -0,0 +1,91 @@ +body { + font: 14px/1.6 "Helvetica Neue", Helvetica, Arial, sans-serif; + margin: 0; + padding: 0; + color: #000; + border-top: 2px solid #ddd; + background-color: #fff; + + min-height: 100%; + display: flex; + flex-direction: column; +} + +h1 { + width: 100%; + border-bottom: 1px solid #eee; + margin-bottom: 0; + font-weight: 100; + font-size: 1.1em; + letter-spacing: 1px; +} + +h1 code { + font-size: 0.96em; +} + +code { + font: 12px monospace; +} + +footer { + background: #eee; + width: 100%; + padding: 10px 0; + text-align: right; + border-top: 1px solid #ddd; + display: flex; + flex: 1; + order: 2; + justify-content: center; +} + +table { + width: 100%; + margin-top: 10px; + border-collapse: collapse; + border: 1px solid #cbcbcb; + color: #000; + -webkit-border-radius: 3px; + -moz-border-radius: 3px; +} +table thead { + display: none; +} +table td.line, +table td.hits { + width: 20px; + background: #eaeaea; + text-align: center; + font-size: 11px; + padding: 0 10px; + color: #949494; +} +table td.hits { + width: 10px; + padding: 2px 5px; + color: rgba(0, 0, 0, 0.6); + background-color: #f0f0f0; +} +tr.miss td.line, +tr.miss td.hits { + background-color: #ffdce0; + border-color: #fdaeb7; +} +tr.miss td { + background-color: #ffeef0; +} +tr.hit td.line, +tr.hit td.hits { + background-color: #cdffd8; + border-color: #bef5cb; +} +tr.hit td { + background-color: #e6ffed; +} +td.source { + padding-left: 15px; + line-height: 15px; + white-space: pre; + font: 12px monospace; +} diff --git a/lib/tools/src/Makefile b/lib/tools/src/Makefile index 032bd612db..a869ae6a00 100644 --- a/lib/tools/src/Makefile +++ b/lib/tools/src/Makefile @@ -72,6 +72,9 @@ APP_TARGET = $(EBIN)/$(APP_FILE) APPUP_SRC = $(APPUP_FILE).src APPUP_TARGET = $(EBIN)/$(APPUP_FILE) +PRIVDIR = ../priv +CSS = $(PRIVDIR)/styles.css + # ---------------------------------------------------- # FLAGS # ---------------------------------------------------- @@ -110,5 +113,7 @@ release_spec: opt $(INSTALL_DIR) "$(RELSYSDIR)/ebin" $(INSTALL_DATA) $(TARGET_FILES) $(APP_TARGET) $(APPUP_TARGET) \ "$(RELSYSDIR)/ebin" + $(INSTALL_DIR) "$(RELSYSDIR)/priv" + $(INSTALL_DATA) $(CSS) "$(RELSYSDIR)/priv" release_docs_spec: diff --git a/lib/tools/src/cover.erl b/lib/tools/src/cover.erl index bf5faa165d..337d9d637a 100644 --- a/lib/tools/src/cover.erl +++ b/lib/tools/src/cover.erl @@ -144,6 +144,8 @@ end). -define(SPAWN_DBG(Tag,Value),put(Tag,Value)). +-define(STYLESHEET, "styles.css"). +-define(TOOLS_APP, tools). -include_lib("stdlib/include/ms_transform.hrl"). @@ -2415,20 +2417,8 @@ do_analyse_to_file1(Module, OutFile, ErlFile, HTML) -> case file:open(OutFile, [write,raw,delayed_write]) of {ok, OutFd} -> Enc = encoding(ErlFile), - if HTML -> - Header = - ["<!DOCTYPE HTML PUBLIC " - "\"-//W3C//DTD HTML 3.2 Final//EN\">\n" - "<html>\n" - "<head>\n" - "<meta http-equiv=\"Content-Type\"" - " content=\"text/html; charset=", - html_encoding(Enc),"\"/>\n" - "<title>",OutFile,"</title>\n" - "</head>" - "<body style='background-color: white;" - " color: black'>\n" - "<pre>\n"], + if HTML -> + Header = create_header(OutFile, Enc), H1Bin = unicode:characters_to_binary(Header,Enc,Enc), ok = file:write(OutFd,H1Bin); true -> ok @@ -2445,14 +2435,19 @@ do_analyse_to_file1(Module, OutFile, ErlFile, HTML) -> string:pad(integer_to_list(Mi), 2, leading, $0), string:pad(integer_to_list(S), 2, leading, $0)]), - H2Bin = unicode:characters_to_binary( - ["File generated from ",ErlFile," by COVER ", - Timestamp,"\n\n" - "**************************************" - "**************************************" - "\n\n"], - Enc, Enc), - ok = file:write(OutFd, H2Bin), + OutFileInfo = + if HTML -> + create_footer(ErlFile, Timestamp); + true -> + ["File generated from ",ErlFile," by COVER ", + Timestamp, "\n\n", + "**************************************" + "**************************************" + "\n\n"] + end, + + H2Bin = unicode:characters_to_binary(OutFileInfo,Enc,Enc), + ok = file:write(OutFd, H2Bin), Pattern = {#bump{module=Module,line='$1',_='_'},'$2'}, MS = [{Pattern,[{is_integer,'$1'},{'>','$1',0}],[{{'$1','$2'}}]}], @@ -2462,7 +2457,7 @@ do_analyse_to_file1(Module, OutFile, ErlFile, HTML) -> print_lines(Module, CovLines, InFd, OutFd, 1, HTML), if HTML -> - ok = file:write(OutFd, "</pre>\n</body>\n</html>\n"); + ok = file:write(OutFd, close_html()); true -> ok end, @@ -2497,12 +2492,11 @@ print_lines(Module, CovLines, InFd, OutFd, L, HTML) -> case CovLines of [{L,N}|CovLines1] -> if N=:=0, HTML=:=true -> - LineNoNL = Line -- "\n", - Str = " 0", - %%Str = string:pad("0", 6, leading, $\s), - RedLine = ["<font color=red>",Str,fill1(), - LineNoNL,"</font>\n"], - ok = file:write(OutFd, RedLine); + MissedLine = table_row("miss", Line, L, N), + ok = file:write(OutFd, MissedLine); + HTML=:=true -> + HitLine = table_row("hit", Line, L, N), + ok = file:write(OutFd, HitLine); N < 1000000 -> Str = string:pad(integer_to_list(N), 6, leading, $\s), ok = file:write(OutFd, [Str,fill1(),Line]); @@ -2515,7 +2509,11 @@ print_lines(Module, CovLines, InFd, OutFd, L, HTML) -> end, print_lines(Module, CovLines1, InFd, OutFd, L+1, HTML); _ -> %Including comment lines - ok = file:write(OutFd, [tab(),Line]), + NonCoveredContent = + if HTML -> table_row(Line, L); + true -> [tab(),Line] + end, + ok = file:write(OutFd, NonCoveredContent), print_lines(Module, CovLines, InFd, OutFd, L+1, HTML) end end. @@ -2525,6 +2523,59 @@ fill1() -> "..| ". fill2() -> ".| ". fill3() -> "| ". +%% HTML sections +create_header(OutFile, Enc) -> + ["<!doctype html>\n" + "<html>\n" + "<head>\n" + "<meta charset=\"",html_encoding(Enc),"\">\n" + "<title>",OutFile,"</title>\n" + "<style>"] ++ + read_stylesheet() ++ + ["</style>\n", + "</head>\n" + "<body>\n" + "<h1><code>",OutFile,"</code></h1>\n"]. + +create_footer(ErlFile, Timestamp) -> + ["<footer><p>File generated from <code>",ErlFile, + "</code> by <a href=\"http://erlang.org/doc/man/cover.html\">cover</a> at ", + Timestamp,"</p></footer>\n<table>\n<tbody>\n"]. + +close_html() -> + ["</tbody>\n", + "<thead>\n", + "<tr>\n", + "<th>Line</th>\n", + "<th>Hits</th>\n", + "<th>Source</th>\n", + "</tr>\n", + "</thead>\n", + "</table>\n", + "</body>\n" + "</html>\n"]. + +table_row(CssClass, Line, L, N) -> + ["<tr class=\"",CssClass,"\">\n", table_data(Line, L, N)]. +table_row(Line, L) -> + ["<tr>\n", table_data(Line, L, "")]. + +table_data(Line, L, N) -> + LineNoNL = Line -- "\n", + ["<td class=\"line\" id=\"L",integer_to_list(L),"\">", + integer_to_list(L), + "</td>\n", + "<td class=\"hits\">",maybe_integer_to_list(N),"</td>\n", + "<td class=\"source\"><code>",LineNoNL,"</code></td>\n</tr>\n"]. + +maybe_integer_to_list(N) when is_integer(N) -> integer_to_list(N); +maybe_integer_to_list(_) -> "". + +read_stylesheet() -> + PrivDir = code:priv_dir(?TOOLS_APP), + {ok, Css} = file:read_file(filename:join(PrivDir, ?STYLESHEET)), + [Css]. + %%%--Export-------------------------------------------------------------- do_export(Module, OutFile, From, State) -> case file:open(OutFile,[write,binary,raw,delayed_write]) of diff --git a/lib/tools/src/fprof.erl b/lib/tools/src/fprof.erl index a3b4bfdddf..36d4828861 100644 --- a/lib/tools/src/fprof.erl +++ b/lib/tools/src/fprof.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2001-2017. All Rights Reserved. +%% Copyright Ericsson AB 2001-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/tools/src/lcnt.erl b/lib/tools/src/lcnt.erl index 1db90c1d86..ee6057e4f5 100644 --- a/lib/tools/src/lcnt.erl +++ b/lib/tools/src/lcnt.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2017. All Rights Reserved. +%% Copyright Ericsson AB 2010-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/tools/test/emacs_SUITE.erl b/lib/tools/test/emacs_SUITE.erl index f4e78da667..5839f9ce5b 100644 --- a/lib/tools/test/emacs_SUITE.erl +++ b/lib/tools/test/emacs_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2016. All Rights Reserved. +%% Copyright Ericsson AB 2005-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/tools/test/instrument_SUITE.erl b/lib/tools/test/instrument_SUITE.erl index 8c521b2e1a..33259df58f 100644 --- a/lib/tools/test/instrument_SUITE.erl +++ b/lib/tools/test/instrument_SUITE.erl @@ -77,6 +77,8 @@ allocations_ramv(Config) when is_list(Config) -> verify_allocations_disabled(_AllocType, Result) -> verify_allocations_disabled(Result). +verify_allocations_disabled({ok, {_HistStart, _UnscannedBytes, Allocs}}) -> + true = Allocs =:= #{}; verify_allocations_disabled({error, not_enabled}) -> ok. @@ -91,6 +93,13 @@ verify_allocations_enabled(_AllocType, Result) -> verify_allocations_enabled({ok, {_HistStart, _UnscannedBytes, Allocs}}) -> true = Allocs =/= #{}. +verify_allocations_output(#{}, {ok, {_, _, Allocs}}) when Allocs =:= #{} -> + %% This happens when the allocator is enabled but tagging is disabled. If + %% there's an error that causes Allocs to always be empty when enabled it + %% will be caught by verify_allocations_enabled. + ok; +verify_allocations_output(#{}, {error, not_enabled}) -> + ok; verify_allocations_output(#{ histogram_start := HistStart, histogram_width := HistWidth }, {ok, {HistStart, _UnscannedBytes, ByOrigin}}) -> @@ -124,8 +133,6 @@ verify_allocations_output(#{ histogram_start := HistStart, [BlockCount, GenTotalBlockCount]) end, - ok; -verify_allocations_output(#{}, {error, not_enabled}) -> ok. %% %% %% %% %% %% @@ -214,7 +221,8 @@ verify_carriers_output(#{ histogram_start := HistStart, ct:fail("Carrier count is ~p, expected at least ~p (SBC).", [CarrierCount, GenSBCCount]); CarrierCount >= GenSBCCount -> - ok + ct:pal("Found ~p carriers, required at least ~p (SBC)." , + [CarrierCount, GenSBCCount]) end, ok; @@ -292,9 +300,19 @@ start_slave(Args) -> MicroSecs = erlang:monotonic_time(), Name = "instr" ++ integer_to_list(MicroSecs), Pa = filename:dirname(code:which(?MODULE)), - {ok, Node} = test_server:start_node(list_to_atom(Name), - slave, - [{args, "-pa " ++ Pa ++ " " ++ Args}]), + + %% We pass arguments through ZFLAGS as the nightly tests rotate + %% +Meamax/+Meamin which breaks the _enabled and _disabled tests unless + %% overridden. + ZFlags = os:getenv("ERL_ZFLAGS", ""), + {ok, Node} = try + os:putenv("ERL_ZFLAGS", ZFlags ++ [" " | Args]), + test_server:start_node(list_to_atom(Name), + slave, + [{args, "-pa " ++ Pa}]) + after + os:putenv("ERL_ZFLAGS", ZFlags) + end, Node. generate_test_blocks() -> @@ -309,8 +327,9 @@ generate_test_blocks() -> MBCs = [<<I, 0:64/unit:8>> || I <- lists:seq(1, ?GENERATED_MBC_BLOCK_COUNT)], Runner ! Ref, - receive after infinity -> ok end, - unreachable ! {SBCs, MBCs} + receive + gurka -> gaffel ! {SBCs, MBCs} + end end), receive Ref -> ok diff --git a/lib/tools/test/lcnt_SUITE.erl b/lib/tools/test/lcnt_SUITE.erl index a79572a742..8a2f6bfc89 100644 --- a/lib/tools/test/lcnt_SUITE.erl +++ b/lib/tools/test/lcnt_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2017. All Rights Reserved. +%% Copyright Ericsson AB 2010-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/tools/test/xref_SUITE.erl b/lib/tools/test/xref_SUITE.erl index 018f632948..4ec75f7962 100644 --- a/lib/tools/test/xref_SUITE.erl +++ b/lib/tools/test/xref_SUITE.erl @@ -2233,18 +2233,18 @@ variables(Conf) when is_list(Conf) -> {{error, _, _}, _} = xref_base:variables(S108, [{verbose,false}]), {ok, S109} = xref_base:set_library_path(S108, [], [{verbose,false}]), - Tabs = length(ets:all()), + NoOfTables = erlang:system_info(ets_count), {ok, S110} = eval("Eplus := closure E, TT := Eplus", 'closure()', S109), {{ok, [{user, ['Eplus','TT']}]}, S111} = xref_base:variables(S110), {ok, S112} = xref_base:forget(S111, ['TT','Eplus']), - true = Tabs =:= length(ets:all()), + true = NoOfTables =:= erlang:system_info(ets_count), {ok, NS0} = eval("Eplus := closure E", 'closure()', S112), {{ok, [{user, ['Eplus']}]}, NS} = xref_base:variables(NS0), ok = xref_base:delete(NS), - true = Tabs =:= length(ets:all()), + true = NoOfTables =:= erlang:system_info(ets_count), ok = file:delete(Beam), ok. diff --git a/lib/tools/vsn.mk b/lib/tools/vsn.mk index f9723c0f9b..bb3f4c66c0 100644 --- a/lib/tools/vsn.mk +++ b/lib/tools/vsn.mk @@ -1 +1 @@ -TOOLS_VSN = 2.11.2 +TOOLS_VSN = 3.0 diff --git a/lib/wx/api_gen/gen_util.erl b/lib/wx/api_gen/gen_util.erl index 3068a2f4ea..d27a9ae548 100644 --- a/lib/wx/api_gen/gen_util.erl +++ b/lib/wx/api_gen/gen_util.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2017. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/wx/api_gen/gl_gen.erl b/lib/wx/api_gen/gl_gen.erl index 6f68517c16..c0509ed802 100644 --- a/lib/wx/api_gen/gl_gen.erl +++ b/lib/wx/api_gen/gl_gen.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2016. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/wx/api_gen/gl_gen_erl.erl b/lib/wx/api_gen/gl_gen_erl.erl index 7e673c2987..e62d182a37 100644 --- a/lib/wx/api_gen/gl_gen_erl.erl +++ b/lib/wx/api_gen/gl_gen_erl.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2016. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/wx/api_gen/gl_scan_doc.erl b/lib/wx/api_gen/gl_scan_doc.erl index 6ed5438608..c793d4940e 100644 --- a/lib/wx/api_gen/gl_scan_doc.erl +++ b/lib/wx/api_gen/gl_scan_doc.erl @@ -2,7 +2,7 @@ %%-------------------------------------------------------------------- %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2012-2016. All Rights Reserved. +%% Copyright Ericsson AB 2012-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/wx/api_gen/wx_gen.erl b/lib/wx/api_gen/wx_gen.erl index 4ba57501a5..cec6ac9ccf 100644 --- a/lib/wx/api_gen/wx_gen.erl +++ b/lib/wx/api_gen/wx_gen.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2016. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/wx/api_gen/wx_gen_cpp.erl b/lib/wx/api_gen/wx_gen_cpp.erl index cc4e1b5301..70a3530526 100644 --- a/lib/wx/api_gen/wx_gen_cpp.erl +++ b/lib/wx/api_gen/wx_gen_cpp.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2017. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/wx/api_gen/wx_gen_erl.erl b/lib/wx/api_gen/wx_gen_erl.erl index dfee7270b4..797533309b 100644 --- a/lib/wx/api_gen/wx_gen_erl.erl +++ b/lib/wx/api_gen/wx_gen_erl.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2016. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/wx/api_gen/wxapi.conf b/lib/wx/api_gen/wxapi.conf index e2ef2d890a..c1b55b6875 100644 --- a/lib/wx/api_gen/wxapi.conf +++ b/lib/wx/api_gen/wxapi.conf @@ -2,7 +2,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2016. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/wx/c_src/wxe_driver.c b/lib/wx/c_src/wxe_driver.c index 26ae3564e7..8b8c625971 100644 --- a/lib/wx/c_src/wxe_driver.c +++ b/lib/wx/c_src/wxe_driver.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2008-2016. All Rights Reserved. + * Copyright Ericsson AB 2008-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/lib/wx/c_src/wxe_ps_init.c b/lib/wx/c_src/wxe_ps_init.c index 277f3ac25f..4b3b47a80b 100644 --- a/lib/wx/c_src/wxe_ps_init.c +++ b/lib/wx/c_src/wxe_ps_init.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2008-2016. All Rights Reserved. + * Copyright Ericsson AB 2008-2018. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/lib/wx/doc/overview.edoc b/lib/wx/doc/overview.edoc index 054016f515..843a9c1320 100644 --- a/lib/wx/doc/overview.edoc +++ b/lib/wx/doc/overview.edoc @@ -218,7 +218,7 @@ the <em>fun</em> returns. The callbacks are always invoked by another process and have exclusive usage of the GUI when invoked. This means that a callback <em>fun</em> -can not use the process dictionary and should not make calls to other +cannot use the process dictionary and should not make calls to other processes. Calls to another process inside a callback <em>fun</em> may cause a deadlock if the other process is waiting on completion of his call to the GUI. diff --git a/lib/wx/doc/src/Makefile b/lib/wx/doc/src/Makefile index c132c628f7..f66d63f63b 100644 --- a/lib/wx/doc/src/Makefile +++ b/lib/wx/doc/src/Makefile @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 2008-2017. All Rights Reserved. +# Copyright Ericsson AB 2008-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -121,7 +121,8 @@ $(XML_CHAPTER_FILES:%=$(XMLDIR)/%): ../overview.edoc debug opt: clean clean_docs: - rm -rf $(HTMLDIR)/* + rm -rf $(HTMLDIR)/* + rm -rf $(XMLDIR) rm -f $(MAN3DIR)/* rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) rm -f $(SPECDIR)/* diff --git a/lib/wx/doc/src/notes.xml b/lib/wx/doc/src/notes.xml index 69ea906ec0..45638dff35 100644 --- a/lib/wx/doc/src/notes.xml +++ b/lib/wx/doc/src/notes.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2009</year><year>2017</year> + <year>2009</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -32,6 +32,22 @@ <p>This document describes the changes made to the wxErlang application.</p> +<section><title>Wx 1.8.4</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Changed implementation so wx can now be built towards + wxWidgets-3.1.1.</p> + <p> + Own Id: OTP-15027</p> + </item> + </list> + </section> + +</section> + <section><title>Wx 1.8.3</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/wx/examples/demo/ex_aui.erl b/lib/wx/examples/demo/ex_aui.erl index 97805621ed..49fa86f630 100644 --- a/lib/wx/examples/demo/ex_aui.erl +++ b/lib/wx/examples/demo/ex_aui.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009-2016. All Rights Reserved. +%% Copyright Ericsson AB 2009-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/wx/examples/demo/ex_graphicsContext.erl b/lib/wx/examples/demo/ex_graphicsContext.erl index 1193578037..1e6ffce18d 100644 --- a/lib/wx/examples/demo/ex_graphicsContext.erl +++ b/lib/wx/examples/demo/ex_graphicsContext.erl @@ -74,7 +74,7 @@ do_init(Config) -> pen = Pen, brush = Brush, font = Font}}. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% Sync events i.e. from callbacks must return ok, it can not return a new state. +%% Sync events i.e. from callbacks must return ok, it cannot return a new state. %% Do the redrawing here. handle_sync_event(#wx{event = #wxPaint{}},_, #state{win=Win, pen = Pen, brush = Brush, font = Font}) -> diff --git a/lib/wx/examples/simple/hello.erl b/lib/wx/examples/simple/hello.erl index bf870c6f3e..8491a20af5 100644 --- a/lib/wx/examples/simple/hello.erl +++ b/lib/wx/examples/simple/hello.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009-2016. All Rights Reserved. +%% Copyright Ericsson AB 2009-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/wx/examples/simple/hello2.erl b/lib/wx/examples/simple/hello2.erl index b9da622b6b..656c056d9a 100644 --- a/lib/wx/examples/simple/hello2.erl +++ b/lib/wx/examples/simple/hello2.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009-2016. All Rights Reserved. +%% Copyright Ericsson AB 2009-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/wx/examples/simple/menu.erl b/lib/wx/examples/simple/menu.erl index 93573fb97d..7c0400bd1e 100644 --- a/lib/wx/examples/simple/menu.erl +++ b/lib/wx/examples/simple/menu.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009-2016. All Rights Reserved. +%% Copyright Ericsson AB 2009-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/wx/examples/simple/minimal.erl b/lib/wx/examples/simple/minimal.erl index 346f86433a..45efc06462 100644 --- a/lib/wx/examples/simple/minimal.erl +++ b/lib/wx/examples/simple/minimal.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009-2016. All Rights Reserved. +%% Copyright Ericsson AB 2009-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/wx/examples/sudoku/sudoku.erl b/lib/wx/examples/sudoku/sudoku.erl index 353f90d86f..a6a8d07e57 100644 --- a/lib/wx/examples/sudoku/sudoku.erl +++ b/lib/wx/examples/sudoku/sudoku.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009-2016. All Rights Reserved. +%% Copyright Ericsson AB 2009-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/wx/examples/sudoku/sudoku_game.erl b/lib/wx/examples/sudoku/sudoku_game.erl index aa15c05653..6463583e0f 100644 --- a/lib/wx/examples/sudoku/sudoku_game.erl +++ b/lib/wx/examples/sudoku/sudoku_game.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009-2016. All Rights Reserved. +%% Copyright Ericsson AB 2009-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/wx/examples/sudoku/sudoku_gui.erl b/lib/wx/examples/sudoku/sudoku_gui.erl index 4c4ad83cd7..e436bf2909 100644 --- a/lib/wx/examples/sudoku/sudoku_gui.erl +++ b/lib/wx/examples/sudoku/sudoku_gui.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009-2016. All Rights Reserved. +%% Copyright Ericsson AB 2009-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/wx/examples/xrc/xrc.erl b/lib/wx/examples/xrc/xrc.erl index 7e967777d2..a64e1f33ef 100644 --- a/lib/wx/examples/xrc/xrc.erl +++ b/lib/wx/examples/xrc/xrc.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009-2016. All Rights Reserved. +%% Copyright Ericsson AB 2009-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/wx/src/wx.erl b/lib/wx/src/wx.erl index 6dd3c148db..a20758c826 100644 --- a/lib/wx/src/wx.erl +++ b/lib/wx/src/wx.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2016. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/wx/src/wxe_master.erl b/lib/wx/src/wxe_master.erl index a55917f0aa..edef6b3613 100644 --- a/lib/wx/src/wxe_master.erl +++ b/lib/wx/src/wxe_master.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2017. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/wx/src/wxe_server.erl b/lib/wx/src/wxe_server.erl index d6d9dbd629..e241a11e8b 100644 --- a/lib/wx/src/wxe_server.erl +++ b/lib/wx/src/wxe_server.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2016. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/wx/test/wx_app_SUITE.erl b/lib/wx/test/wx_app_SUITE.erl index a5202d8448..99c8ef0416 100644 --- a/lib/wx/test/wx_app_SUITE.erl +++ b/lib/wx/test/wx_app_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2010-2016. All Rights Reserved. +%% Copyright Ericsson AB 2010-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/wx/test/wx_class_SUITE.erl b/lib/wx/test/wx_class_SUITE.erl index c610b9c4f4..56749ba72b 100644 --- a/lib/wx/test/wx_class_SUITE.erl +++ b/lib/wx/test/wx_class_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2016. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/wx/test/wx_event_SUITE.erl b/lib/wx/test/wx_event_SUITE.erl index a564f89e58..1cc194d569 100644 --- a/lib/wx/test/wx_event_SUITE.erl +++ b/lib/wx/test/wx_event_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2016. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -347,9 +347,9 @@ connect_in_callback(Config) -> %% such that new events are not fired until the previous %% callback have returned. - %% That means that a callback can not wait for other events + %% That means that a callback cannot wait for other events %% in receive since they will not come. - %% It also means that you can not attach a new callback directly from + %% It also means that you cannot attach a new callback directly from %% the callback since that callback will be removed when the temporary %% process that executes the outer callback (may) die(s) before the callback %% is invoked diff --git a/lib/wx/test/wx_opengl_SUITE.erl b/lib/wx/test/wx_opengl_SUITE.erl index 19ea731dfb..053740ed8b 100644 --- a/lib/wx/test/wx_opengl_SUITE.erl +++ b/lib/wx/test/wx_opengl_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2016. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/wx/test/wx_test_lib.erl b/lib/wx/test/wx_test_lib.erl index af508ff490..7bf2bb11bb 100644 --- a/lib/wx/test/wx_test_lib.erl +++ b/lib/wx/test/wx_test_lib.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2016. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/wx/test/wx_xtra_SUITE.erl b/lib/wx/test/wx_xtra_SUITE.erl index 486843ec63..8142d9566f 100644 --- a/lib/wx/test/wx_xtra_SUITE.erl +++ b/lib/wx/test/wx_xtra_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009-2016. All Rights Reserved. +%% Copyright Ericsson AB 2009-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/wx/test/wxt.erl b/lib/wx/test/wxt.erl index 2b380606d5..280443404f 100644 --- a/lib/wx/test/wxt.erl +++ b/lib/wx/test/wxt.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2016. All Rights Reserved. +%% Copyright Ericsson AB 2008-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/wx/vsn.mk b/lib/wx/vsn.mk index 7da4529c98..37a5477631 100644 --- a/lib/wx/vsn.mk +++ b/lib/wx/vsn.mk @@ -1 +1 @@ -WX_VSN = 1.8.3 +WX_VSN = 1.8.4 diff --git a/lib/xmerl/doc/src/Makefile b/lib/xmerl/doc/src/Makefile index 94100910ef..0def492246 100644 --- a/lib/xmerl/doc/src/Makefile +++ b/lib/xmerl/doc/src/Makefile @@ -1,7 +1,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 2004-2017. All Rights Reserved. +# Copyright Ericsson AB 2004-2018. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -140,6 +140,7 @@ debug opt: clean clean_docs: rm -rf $(HTMLDIR)/* + rm -rf $(XMLDIR) rm -f $(MAN3DIR)/* rm -f $(MAN6DIR)/* rm -f $(XMERL_XML_FILES) diff --git a/lib/xmerl/doc/src/notes.xml b/lib/xmerl/doc/src/notes.xml index f62a8dc53d..b7d1db7dfc 100644 --- a/lib/xmerl/doc/src/notes.xml +++ b/lib/xmerl/doc/src/notes.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2004</year><year>2017</year> + <year>2004</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -32,6 +32,21 @@ <p>This document describes the changes made to the Xmerl application.</p> +<section><title>Xmerl 1.3.17</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fix typos in documentation.</p> + <p> + Own Id: OTP-15039</p> + </item> + </list> + </section> + +</section> + <section><title>Xmerl 1.3.16</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/xmerl/src/xmerl_scan.erl b/lib/xmerl/src/xmerl_scan.erl index a1f6ad4e2c..a7538180e6 100644 --- a/lib/xmerl/src/xmerl_scan.erl +++ b/lib/xmerl/src/xmerl_scan.erl @@ -279,7 +279,7 @@ int_file_decl(F, Options,_ExtCharset) -> %% @spec string(Text::list()) -> {xmlElement(),Rest} %% Rest = list() -%% @equiv string(Test, []) +%% @equiv string(Text, []) string(Str) -> string(Str, []). diff --git a/lib/xmerl/src/xmerl_xsd.erl b/lib/xmerl/src/xmerl_xsd.erl index d727084175..2836bb0e5b 100644 --- a/lib/xmerl/src/xmerl_xsd.erl +++ b/lib/xmerl/src/xmerl_xsd.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2006-2016. All Rights Reserved. +%% Copyright Ericsson AB 2006-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/lib/xmerl/test/xmerl_SUITE_data/eventp/CMOM.xml b/lib/xmerl/test/xmerl_SUITE_data/eventp/CMOM.xml index c2533248d1..0379c18214 100644 --- a/lib/xmerl/test/xmerl_SUITE_data/eventp/CMOM.xml +++ b/lib/xmerl/test/xmerl_SUITE_data/eventp/CMOM.xml @@ -276,7 +276,7 @@ </exception> <exception name="UpgradeNotPossibleException"> - <description>Before an upgrade is started it was found that the upgrade can not take place. A possible reason is that the upgrade package that is running in the node is not in the upgrade window of this upgrade package. </description> + <description>Before an upgrade is started it was found that the upgrade cannot take place. A possible reason is that the upgrade package that is running in the node is not in the upgrade window of this upgrade package. </description> <exceptionParameter name="message"> <dataType> <string/> @@ -1703,7 +1703,7 @@ Example: <exception name="NoSuchAttributeException"> <description>Exception thrown when an MO attribute is requested to be accessed but the access -method for the is not defined (the attribute can not be accessed)</description> +method for the is not defined (the attribute cannot be accessed)</description> </exception> <exception name="MoCardinalityViolationException"> @@ -3125,7 +3125,7 @@ active = Synchronization is used in system clock generation. <description>RefState can have the following values: failed = synchronization reference is not capable to perform its required tasks. degraded = capability of synchronization reference to perform its required tasks is degraded e.g. because of signal level degradation. This value is only applicable for traffic carrying (ET physical path termination) synchronization references. Note: attribute degradationIsFault controls whether synchronization reference degradation is interpreted as a synchronization reference fault or not. -lossOfTracking = system clock regulation algorithm on TU board can not follow the 8kHz synchronization reference signal either because of the poor quality of the signal or because of a HW fault at TU board. If all synchronization references repeatedly end up to state lossOfTracking, fault is likely in TU HW. +lossOfTracking = system clock regulation algorithm on TU board cannot follow the 8kHz synchronization reference signal either because of the poor quality of the signal or because of a HW fault at TU board. If all synchronization references repeatedly end up to state lossOfTracking, fault is likely in TU HW. ok = synchronization reference is capable of performing its required tasks. </description> <enumMember name="failed"> @@ -11270,7 +11270,7 @@ Note! This action requires a transaction.</description> The changing of the IP address with operation assignIpAddress might cause interruption of the communication if the network management tool is connected via the ethernet link. -Note: The EthernetLink MO can not be deleted! +Note: The EthernetLink MO cannot be deleted! Note: The performance monitoring counters in the EthernetLink MO has a "Wrap-around time" of approximately 2 hours. </description> @@ -11532,8 +11532,8 @@ Note! This action requires a transaction.</description> <description>This MO holds the IP routing table. The IpRoutingTable MO is automatically created when the Ip MO is created. -The IpRoutingTable MO can not be created manually. -The IpRoutingTable MO can not be deleted. +The IpRoutingTable MO cannot be created manually. +The IpRoutingTable MO cannot be deleted. </description> <systemCreated/> <attribute name="userLabel"> @@ -12967,7 +12967,7 @@ Note! When using fractional atm, timeslot 1 must be a part of the fraction. Note! ETM1 does not support EPD and PPD -Note! ETM1 does not have a proper buffer management. Thus fairness of UBR+ traffic can not be guaranteed and shaping on UBR+ traffic is not possible. +Note! ETM1 does not have a proper buffer management. Thus fairness of UBR+ traffic cannot be guaranteed and shaping on UBR+ traffic is not possible. Note! The number of VCC TP+VPC TP with performance monitoring enabled (i.e. PM mode <> off) is restricted to 1 per port. </description> @@ -13371,7 +13371,7 @@ Struct element description : -Date is in string format, max length is 40. Format for date is: weekday month date hour:min:seconds year. -Status is in string format, max length is 40. -Note! The identity can not be the same as name. Identity should contain the product identity. +Note! The identity cannot be the same as name. Identity should contain the product identity. </description> @@ -13562,7 +13562,7 @@ Note! For comment and operatorName spaces (' ') are also allowed within the stri This action does not require a transaction. -Note! The configurationVersionName and identity can not be the same. The identity should be the product identity. +Note! The configurationVersionName and identity cannot be the same. The identity should be the product identity. </description> <returnType> <void/> @@ -17367,7 +17367,7 @@ NOTE: There is a restriction of a maximum of 32 Mtp2Tp's per MP.</description> <attribute name="mtp2ProfileItuId"> <description>Reference to a Mtp2ProfileItu MO. -Note: The bitRate can not be changed.</description> +Note: The bitRate cannot be changed.</description> <mandatory/> <noNotification/> <dataType> @@ -17566,7 +17566,7 @@ NOTE: There is a restriction of a maximum of 32 Mtp2Tp's per MP.</description> <attribute name="mtp2ProfileAnsiId"> <description>Reference to a Mtp2ProfileAnsi MO. -Note: The bitRate can not be changed.</description> +Note: The bitRate cannot be changed.</description> <mandatory/> <noNotification/> <dataType> @@ -21777,7 +21777,7 @@ Each E1/DS1/J1 channel can suport up to 2 VP connections. The ET-MC41 board support one biderectional F4/F5 PM flow per E1 channel. -Note! The ETMC41 supports IMA. However the E1 ports being part of the same IMA group can not be selected randomly. +Note! The ETMC41 supports IMA. However the E1 ports being part of the same IMA group cannot be selected randomly. Note! The number of VCC TP+VPC TP with performance monitoring enabled (i.e. PM mode <> off) is restricted to 1 / port @@ -21883,7 +21883,7 @@ NOTE: There is a restriction of a maximum of 32 Mtp2Tp's per MP. <attribute name="mtp2ProfileChinaId"> <description>Reference to a Mtp2ProfileChina MO. -Note: The bitRate can not be changed.</description> +Note: The bitRate cannot be changed.</description> <mandatory/> <noNotification/> <dataType> diff --git a/lib/xmerl/test/xmerl_SUITE_data/eventp/CelloMOM.xml b/lib/xmerl/test/xmerl_SUITE_data/eventp/CelloMOM.xml index 3b5d8ae2ad..3b9ccac0f4 100644 --- a/lib/xmerl/test/xmerl_SUITE_data/eventp/CelloMOM.xml +++ b/lib/xmerl/test/xmerl_SUITE_data/eventp/CelloMOM.xml @@ -276,7 +276,7 @@ </exception>
<exception name="UpgradeNotPossibleException">
- <description>Before an upgrade is started it was found that the upgrade can not take place. A possible reason is that the upgrade package that is running in the node is not in the upgrade window of this upgrade package. </description>
+ <description>Before an upgrade is started it was found that the upgrade cannot take place. A possible reason is that the upgrade package that is running in the node is not in the upgrade window of this upgrade package. </description>
<exceptionParameter name="message">
<dataType>
<string/>
@@ -1703,7 +1703,7 @@ Example: <exception name="NoSuchAttributeException">
<description>Exception thrown when an MO attribute is requested to be accessed but the access
-method for the is not defined (the attribute can not be accessed)</description>
+method for the is not defined (the attribute cannot be accessed)</description>
</exception>
<exception name="MoCardinalityViolationException">
@@ -3125,7 +3125,7 @@ active = Synchronization is used in system clock generation. <description>RefState can have the following values:
failed = synchronization reference is not capable to perform its required tasks.
degraded = capability of synchronization reference to perform its required tasks is degraded e.g. because of signal level degradation. This value is only applicable for traffic carrying (ET physical path termination) synchronization references. Note: attribute degradationIsFault controls whether synchronization reference degradation is interpreted as a synchronization reference fault or not.
-lossOfTracking = system clock regulation algorithm on TU board can not follow the 8kHz synchronization reference signal either because of the poor quality of the signal or because of a HW fault at TU board. If all synchronization references repeatedly end up to state lossOfTracking, fault is likely in TU HW.
+lossOfTracking = system clock regulation algorithm on TU board cannot follow the 8kHz synchronization reference signal either because of the poor quality of the signal or because of a HW fault at TU board. If all synchronization references repeatedly end up to state lossOfTracking, fault is likely in TU HW.
ok = synchronization reference is capable of performing its required tasks.
</description>
<enumMember name="failed">
@@ -11270,7 +11270,7 @@ Note! This action requires a transaction.</description> The changing of the IP address with operation assignIpAddress might cause interruption of the communication if the network management tool is connected via the ethernet link.
-Note: The EthernetLink MO can not be deleted!
+Note: The EthernetLink MO cannot be deleted!
Note: The performance monitoring counters in the EthernetLink MO has a "Wrap-around time" of approximately 2 hours.
</description>
@@ -11532,8 +11532,8 @@ Note! This action requires a transaction.</description> <description>This MO holds the IP routing table.
The IpRoutingTable MO is automatically created when the Ip MO is created.
-The IpRoutingTable MO can not be created manually.
-The IpRoutingTable MO can not be deleted.
+The IpRoutingTable MO cannot be created manually.
+The IpRoutingTable MO cannot be deleted.
</description>
<systemCreated/>
<attribute name="userLabel">
@@ -12967,7 +12967,7 @@ Note! When using fractional atm, timeslot 1 must be a part of the fraction. Note! ETM1 does not support EPD and PPD
-Note! ETM1 does not have a proper buffer management. Thus fairness of UBR+ traffic can not be guaranteed and shaping on UBR+ traffic is not possible.
+Note! ETM1 does not have a proper buffer management. Thus fairness of UBR+ traffic cannot be guaranteed and shaping on UBR+ traffic is not possible.
Note! The number of VCC TP+VPC TP with performance monitoring enabled (i.e. PM mode <> off) is restricted to 1 per port.
</description>
@@ -13371,7 +13371,7 @@ Struct element description : -Date is in string format, max length is 40. Format for date is: weekday month date hour:min:seconds year.
-Status is in string format, max length is 40.
-Note! The identity can not be the same as name. Identity should contain the product identity.
+Note! The identity cannot be the same as name. Identity should contain the product identity.
</description>
@@ -13562,7 +13562,7 @@ Note! For comment and operatorName spaces (' ') are also allowed within the stri This action does not require a transaction.
-Note! The configurationVersionName and identity can not be the same. The identity should be the product identity.
+Note! The configurationVersionName and identity cannot be the same. The identity should be the product identity.
</description>
<returnType>
<void/>
@@ -17367,7 +17367,7 @@ NOTE: There is a restriction of a maximum of 32 Mtp2Tp's per MP.</description> <attribute name="mtp2ProfileItuId">
<description>Reference to a Mtp2ProfileItu MO.
-Note: The bitRate can not be changed.</description>
+Note: The bitRate cannot be changed.</description>
<mandatory/>
<noNotification/>
<dataType>
@@ -17566,7 +17566,7 @@ NOTE: There is a restriction of a maximum of 32 Mtp2Tp's per MP.</description> <attribute name="mtp2ProfileAnsiId">
<description>Reference to a Mtp2ProfileAnsi MO.
-Note: The bitRate can not be changed.</description>
+Note: The bitRate cannot be changed.</description>
<mandatory/>
<noNotification/>
<dataType>
@@ -21777,7 +21777,7 @@ Each E1/DS1/J1 channel can suport up to 2 VP connections. The ET-MC41 board support one biderectional F4/F5 PM flow per E1 channel.
-Note! The ETMC41 supports IMA. However the E1 ports being part of the same IMA group can not be selected randomly.
+Note! The ETMC41 supports IMA. However the E1 ports being part of the same IMA group cannot be selected randomly.
Note! The number of VCC TP+VPC TP with performance monitoring enabled (i.e. PM mode <> off) is restricted to 1 / port
@@ -21883,7 +21883,7 @@ NOTE: There is a restriction of a maximum of 32 Mtp2Tp's per MP. <attribute name="mtp2ProfileChinaId">
<description>Reference to a Mtp2ProfileChina MO.
-Note: The bitRate can not be changed.</description>
+Note: The bitRate cannot be changed.</description>
<mandatory/>
<noNotification/>
<dataType>
diff --git a/lib/xmerl/vsn.mk b/lib/xmerl/vsn.mk index ddff0c8894..be11935f2f 100644 --- a/lib/xmerl/vsn.mk +++ b/lib/xmerl/vsn.mk @@ -1 +1 @@ -XMERL_VSN = 1.3.16 +XMERL_VSN = 1.3.17 diff --git a/make/cross_check_erl b/make/cross_check_erl index f8ba73023a..524514a63f 100755 --- a/make/cross_check_erl +++ b/make/cross_check_erl @@ -90,7 +90,7 @@ start() -> EOF erlc cross_check_erl.erl 2>/dev/null \ - && used_otp=`erl -noshell -noinput -pa . -run cross_check_erl 2>/dev/null` + && used_otp=`erl -noshell -noinput -boot start_clean -pa . -run cross_check_erl 2>/dev/null` res=$? diff --git a/make/make_emakefile.in b/make/make_emakefile.in index fbca77887a..e0740d1013 100755 --- a/make/make_emakefile.in +++ b/make/make_emakefile.in @@ -1,4 +1,4 @@ -#!@PERL@ +#!/usr/bin/env @PERL@ # -*- cperl -*- use strict; diff --git a/make/otp.mk.in b/make/otp.mk.in index c514a150ca..df29d26833 100644 --- a/make/otp.mk.in +++ b/make/otp.mk.in @@ -87,6 +87,13 @@ AR = @AR@ PERL = @PERL@ LLVM_PROFDATA = @LLVM_PROFDATA@ +MIXED_CYGWIN_VC = @MIXED_CYGWIN_VC@ +MIXED_MSYS_VC = @MIXED_MSYS_VC@ +MIXED_VC = @MIXED_VC@ +MIXED_CYGWIN_MINGW = @MIXED_CYGWIN_MINGW@ +MIXED_CYGWIN = @MIXED_CYGWIN@ +MIXED_MSYS = @MIXED_MSYS@ + BITS64 = @BITS64@ OTP_RELEASE = @OTP_RELEASE@ diff --git a/make/otp_release_targets.mk b/make/otp_release_targets.mk index f57116569c..d1fbf6c58f 100644 --- a/make/otp_release_targets.mk +++ b/make/otp_release_targets.mk @@ -41,6 +41,8 @@ _create_xml_dirs := $(shell mkdir -p $(XMLDIR)) endif XML_GEN_FILES+=$(patsubst %.xml,$(XMLDIR)/%.xml,$(XML_FILES)) + +ifeq ($(strip $(NO_GITHUB_DOC_LINKS)),) $(XMLDIR)/%.xml: %.xml $(gen_verbose)escript $(DOCGEN)/priv/bin/github_link.escript $< \ "$(subst $(ERL_TOP)/,,$(CURDIR)/$^)" "NA" $@ @@ -48,6 +50,13 @@ $(XMLDIR)/%.xml: %.xml $(XMLDIR)/%.xmlsrc: %.xmlsrc $(gen_verbose)escript $(DOCGEN)/priv/bin/github_link.escript $< \ "$(subst $(ERL_TOP)/,,$(CURDIR)/$^)" "NA" $@ +else +## Just copy the files if the application does not want github edit links +$(XMLDIR)/%.xml: %.xml + $(gen_verbose)$(CP) $< $@ +$(XMLDIR)/%.xmlsrc: %.xmlsrc + $(gen_verbose)$(CP) $< $@ +endif ifeq ($(TOPDOC),) diff --git a/otp_versions.table b/otp_versions.table index 22762f0691..609b98ff6b 100644 --- a/otp_versions.table +++ b/otp_versions.table @@ -1,3 +1,15 @@ +OTP-21.0.5 : compiler-7.2.3 crypto-4.3.1 erts-10.0.5 # asn1-5.0.6 common_test-1.16 debugger-4.2.5 dialyzer-3.3 diameter-2.1.5 edoc-0.9.3 eldap-1.2.4 erl_docgen-0.8 erl_interface-3.10.3 et-1.6.2 eunit-2.3.6 ftp-1.0 hipe-3.18 inets-7.0 jinterface-1.9 kernel-6.0 megaco-3.18.3 mnesia-4.15.4 observer-2.8 odbc-2.12.1 os_mon-2.4.5 otp_mibs-1.2 parsetools-2.1.7 public_key-1.6.1 reltool-0.7.6 runtime_tools-1.13 sasl-3.2 snmp-5.2.11 ssh-4.7 ssl-9.0 stdlib-3.5.1 syntax_tools-2.1.5 tftp-1.0 tools-3.0 wx-1.8.4 xmerl-1.3.17 : +OTP-21.0.4 : erts-10.0.4 # asn1-5.0.6 common_test-1.16 compiler-7.2.2 crypto-4.3 debugger-4.2.5 dialyzer-3.3 diameter-2.1.5 edoc-0.9.3 eldap-1.2.4 erl_docgen-0.8 erl_interface-3.10.3 et-1.6.2 eunit-2.3.6 ftp-1.0 hipe-3.18 inets-7.0 jinterface-1.9 kernel-6.0 megaco-3.18.3 mnesia-4.15.4 observer-2.8 odbc-2.12.1 os_mon-2.4.5 otp_mibs-1.2 parsetools-2.1.7 public_key-1.6.1 reltool-0.7.6 runtime_tools-1.13 sasl-3.2 snmp-5.2.11 ssh-4.7 ssl-9.0 stdlib-3.5.1 syntax_tools-2.1.5 tftp-1.0 tools-3.0 wx-1.8.4 xmerl-1.3.17 : +OTP-21.0.3 : erts-10.0.3 # asn1-5.0.6 common_test-1.16 compiler-7.2.2 crypto-4.3 debugger-4.2.5 dialyzer-3.3 diameter-2.1.5 edoc-0.9.3 eldap-1.2.4 erl_docgen-0.8 erl_interface-3.10.3 et-1.6.2 eunit-2.3.6 ftp-1.0 hipe-3.18 inets-7.0 jinterface-1.9 kernel-6.0 megaco-3.18.3 mnesia-4.15.4 observer-2.8 odbc-2.12.1 os_mon-2.4.5 otp_mibs-1.2 parsetools-2.1.7 public_key-1.6.1 reltool-0.7.6 runtime_tools-1.13 sasl-3.2 snmp-5.2.11 ssh-4.7 ssl-9.0 stdlib-3.5.1 syntax_tools-2.1.5 tftp-1.0 tools-3.0 wx-1.8.4 xmerl-1.3.17 : +OTP-21.0.2 : compiler-7.2.2 erts-10.0.2 public_key-1.6.1 stdlib-3.5.1 # asn1-5.0.6 common_test-1.16 crypto-4.3 debugger-4.2.5 dialyzer-3.3 diameter-2.1.5 edoc-0.9.3 eldap-1.2.4 erl_docgen-0.8 erl_interface-3.10.3 et-1.6.2 eunit-2.3.6 ftp-1.0 hipe-3.18 inets-7.0 jinterface-1.9 kernel-6.0 megaco-3.18.3 mnesia-4.15.4 observer-2.8 odbc-2.12.1 os_mon-2.4.5 otp_mibs-1.2 parsetools-2.1.7 reltool-0.7.6 runtime_tools-1.13 sasl-3.2 snmp-5.2.11 ssh-4.7 ssl-9.0 syntax_tools-2.1.5 tftp-1.0 tools-3.0 wx-1.8.4 xmerl-1.3.17 : +OTP-21.0.1 : compiler-7.2.1 erts-10.0.1 # asn1-5.0.6 common_test-1.16 crypto-4.3 debugger-4.2.5 dialyzer-3.3 diameter-2.1.5 edoc-0.9.3 eldap-1.2.4 erl_docgen-0.8 erl_interface-3.10.3 et-1.6.2 eunit-2.3.6 ftp-1.0 hipe-3.18 inets-7.0 jinterface-1.9 kernel-6.0 megaco-3.18.3 mnesia-4.15.4 observer-2.8 odbc-2.12.1 os_mon-2.4.5 otp_mibs-1.2 parsetools-2.1.7 public_key-1.6 reltool-0.7.6 runtime_tools-1.13 sasl-3.2 snmp-5.2.11 ssh-4.7 ssl-9.0 stdlib-3.5 syntax_tools-2.1.5 tftp-1.0 tools-3.0 wx-1.8.4 xmerl-1.3.17 : +OTP-21.0 : asn1-5.0.6 common_test-1.16 compiler-7.2 crypto-4.3 debugger-4.2.5 dialyzer-3.3 diameter-2.1.5 edoc-0.9.3 eldap-1.2.4 erl_docgen-0.8 erl_interface-3.10.3 erts-10.0 et-1.6.2 eunit-2.3.6 ftp-1.0 hipe-3.18 inets-7.0 jinterface-1.9 kernel-6.0 mnesia-4.15.4 observer-2.8 os_mon-2.4.5 otp_mibs-1.2 parsetools-2.1.7 public_key-1.6 reltool-0.7.6 runtime_tools-1.13 sasl-3.2 ssh-4.7 ssl-9.0 stdlib-3.5 syntax_tools-2.1.5 tftp-1.0 tools-3.0 wx-1.8.4 xmerl-1.3.17 # megaco-3.18.3 odbc-2.12.1 snmp-5.2.11 : +OTP-20.3.8.5 : compiler-7.1.5.1 crypto-4.2.2.1 erts-9.3.3.3 mnesia-4.15.3.1 ssl-8.2.6.2 # asn1-5.0.5.1 common_test-1.15.4 cosEvent-2.2.2 cosEventDomain-1.2.2 cosFileTransfer-1.2.2 cosNotification-1.2.3 cosProperty-1.2.3 cosTime-1.2.3 cosTransactions-1.3.3 debugger-4.2.4 dialyzer-3.2.4 diameter-2.1.4 edoc-0.9.2 eldap-1.2.3 erl_docgen-0.7.3 erl_interface-3.10.2.1 et-1.6.1 eunit-2.3.5 hipe-3.17.1 ic-4.4.4.2 inets-6.5.2.2 jinterface-1.8.1 kernel-5.4.3.2 megaco-3.18.3 observer-2.7 odbc-2.12.1 orber-3.8.4 os_mon-2.4.4 otp_mibs-1.1.2 parsetools-2.1.6 public_key-1.5.2 reltool-0.7.5 runtime_tools-1.12.5 sasl-3.1.2 snmp-5.2.11 ssh-4.6.9.1 stdlib-3.4.5 syntax_tools-2.1.4.1 tools-2.11.2 wx-1.8.3 xmerl-1.3.16 : +OTP-20.3.8.4 : asn1-5.0.5.1 # common_test-1.15.4 compiler-7.1.5 cosEvent-2.2.2 cosEventDomain-1.2.2 cosFileTransfer-1.2.2 cosNotification-1.2.3 cosProperty-1.2.3 cosTime-1.2.3 cosTransactions-1.3.3 crypto-4.2.2 debugger-4.2.4 dialyzer-3.2.4 diameter-2.1.4 edoc-0.9.2 eldap-1.2.3 erl_docgen-0.7.3 erl_interface-3.10.2.1 erts-9.3.3.2 et-1.6.1 eunit-2.3.5 hipe-3.17.1 ic-4.4.4.2 inets-6.5.2.2 jinterface-1.8.1 kernel-5.4.3.2 megaco-3.18.3 mnesia-4.15.3 observer-2.7 odbc-2.12.1 orber-3.8.4 os_mon-2.4.4 otp_mibs-1.1.2 parsetools-2.1.6 public_key-1.5.2 reltool-0.7.5 runtime_tools-1.12.5 sasl-3.1.2 snmp-5.2.11 ssh-4.6.9.1 ssl-8.2.6.1 stdlib-3.4.5 syntax_tools-2.1.4.1 tools-2.11.2 wx-1.8.3 xmerl-1.3.16 : +OTP-20.3.8.3 : erts-9.3.3.2 ic-4.4.4.2 inets-6.5.2.2 kernel-5.4.3.2 ssl-8.2.6.1 # asn1-5.0.5 common_test-1.15.4 compiler-7.1.5 cosEvent-2.2.2 cosEventDomain-1.2.2 cosFileTransfer-1.2.2 cosNotification-1.2.3 cosProperty-1.2.3 cosTime-1.2.3 cosTransactions-1.3.3 crypto-4.2.2 debugger-4.2.4 dialyzer-3.2.4 diameter-2.1.4 edoc-0.9.2 eldap-1.2.3 erl_docgen-0.7.3 erl_interface-3.10.2.1 et-1.6.1 eunit-2.3.5 hipe-3.17.1 jinterface-1.8.1 megaco-3.18.3 mnesia-4.15.3 observer-2.7 odbc-2.12.1 orber-3.8.4 os_mon-2.4.4 otp_mibs-1.1.2 parsetools-2.1.6 public_key-1.5.2 reltool-0.7.5 runtime_tools-1.12.5 sasl-3.1.2 snmp-5.2.11 ssh-4.6.9.1 stdlib-3.4.5 syntax_tools-2.1.4.1 tools-2.11.2 wx-1.8.3 xmerl-1.3.16 : +OTP-20.3.8.2 : erl_interface-3.10.2.1 erts-9.3.3.1 ic-4.4.4.1 kernel-5.4.3.1 # asn1-5.0.5 common_test-1.15.4 compiler-7.1.5 cosEvent-2.2.2 cosEventDomain-1.2.2 cosFileTransfer-1.2.2 cosNotification-1.2.3 cosProperty-1.2.3 cosTime-1.2.3 cosTransactions-1.3.3 crypto-4.2.2 debugger-4.2.4 dialyzer-3.2.4 diameter-2.1.4 edoc-0.9.2 eldap-1.2.3 erl_docgen-0.7.3 et-1.6.1 eunit-2.3.5 hipe-3.17.1 inets-6.5.2.1 jinterface-1.8.1 megaco-3.18.3 mnesia-4.15.3 observer-2.7 odbc-2.12.1 orber-3.8.4 os_mon-2.4.4 otp_mibs-1.1.2 parsetools-2.1.6 public_key-1.5.2 reltool-0.7.5 runtime_tools-1.12.5 sasl-3.1.2 snmp-5.2.11 ssh-4.6.9.1 ssl-8.2.6 stdlib-3.4.5 syntax_tools-2.1.4.1 tools-2.11.2 wx-1.8.3 xmerl-1.3.16 : +OTP-20.3.8.1 : inets-6.5.2.1 ssh-4.6.9.1 syntax_tools-2.1.4.1 # asn1-5.0.5 common_test-1.15.4 compiler-7.1.5 cosEvent-2.2.2 cosEventDomain-1.2.2 cosFileTransfer-1.2.2 cosNotification-1.2.3 cosProperty-1.2.3 cosTime-1.2.3 cosTransactions-1.3.3 crypto-4.2.2 debugger-4.2.4 dialyzer-3.2.4 diameter-2.1.4 edoc-0.9.2 eldap-1.2.3 erl_docgen-0.7.3 erl_interface-3.10.2 erts-9.3.3 et-1.6.1 eunit-2.3.5 hipe-3.17.1 ic-4.4.4 jinterface-1.8.1 kernel-5.4.3 megaco-3.18.3 mnesia-4.15.3 observer-2.7 odbc-2.12.1 orber-3.8.4 os_mon-2.4.4 otp_mibs-1.1.2 parsetools-2.1.6 public_key-1.5.2 reltool-0.7.5 runtime_tools-1.12.5 sasl-3.1.2 snmp-5.2.11 ssl-8.2.6 stdlib-3.4.5 tools-2.11.2 wx-1.8.3 xmerl-1.3.16 : +OTP-20.3.8 : erts-9.3.3 snmp-5.2.11 # asn1-5.0.5 common_test-1.15.4 compiler-7.1.5 cosEvent-2.2.2 cosEventDomain-1.2.2 cosFileTransfer-1.2.2 cosNotification-1.2.3 cosProperty-1.2.3 cosTime-1.2.3 cosTransactions-1.3.3 crypto-4.2.2 debugger-4.2.4 dialyzer-3.2.4 diameter-2.1.4 edoc-0.9.2 eldap-1.2.3 erl_docgen-0.7.3 erl_interface-3.10.2 et-1.6.1 eunit-2.3.5 hipe-3.17.1 ic-4.4.4 inets-6.5.2 jinterface-1.8.1 kernel-5.4.3 megaco-3.18.3 mnesia-4.15.3 observer-2.7 odbc-2.12.1 orber-3.8.4 os_mon-2.4.4 otp_mibs-1.1.2 parsetools-2.1.6 public_key-1.5.2 reltool-0.7.5 runtime_tools-1.12.5 sasl-3.1.2 ssh-4.6.9 ssl-8.2.6 stdlib-3.4.5 syntax_tools-2.1.4 tools-2.11.2 wx-1.8.3 xmerl-1.3.16 : OTP-20.3.7 : erl_docgen-0.7.3 erts-9.3.2 inets-6.5.2 # asn1-5.0.5 common_test-1.15.4 compiler-7.1.5 cosEvent-2.2.2 cosEventDomain-1.2.2 cosFileTransfer-1.2.2 cosNotification-1.2.3 cosProperty-1.2.3 cosTime-1.2.3 cosTransactions-1.3.3 crypto-4.2.2 debugger-4.2.4 dialyzer-3.2.4 diameter-2.1.4 edoc-0.9.2 eldap-1.2.3 erl_interface-3.10.2 et-1.6.1 eunit-2.3.5 hipe-3.17.1 ic-4.4.4 jinterface-1.8.1 kernel-5.4.3 megaco-3.18.3 mnesia-4.15.3 observer-2.7 odbc-2.12.1 orber-3.8.4 os_mon-2.4.4 otp_mibs-1.1.2 parsetools-2.1.6 public_key-1.5.2 reltool-0.7.5 runtime_tools-1.12.5 sasl-3.1.2 snmp-5.2.10 ssh-4.6.9 ssl-8.2.6 stdlib-3.4.5 syntax_tools-2.1.4 tools-2.11.2 wx-1.8.3 xmerl-1.3.16 : OTP-20.3.6 : crypto-4.2.2 ssh-4.6.9 # asn1-5.0.5 common_test-1.15.4 compiler-7.1.5 cosEvent-2.2.2 cosEventDomain-1.2.2 cosFileTransfer-1.2.2 cosNotification-1.2.3 cosProperty-1.2.3 cosTime-1.2.3 cosTransactions-1.3.3 debugger-4.2.4 dialyzer-3.2.4 diameter-2.1.4 edoc-0.9.2 eldap-1.2.3 erl_docgen-0.7.2 erl_interface-3.10.2 erts-9.3.1 et-1.6.1 eunit-2.3.5 hipe-3.17.1 ic-4.4.4 inets-6.5.1 jinterface-1.8.1 kernel-5.4.3 megaco-3.18.3 mnesia-4.15.3 observer-2.7 odbc-2.12.1 orber-3.8.4 os_mon-2.4.4 otp_mibs-1.1.2 parsetools-2.1.6 public_key-1.5.2 reltool-0.7.5 runtime_tools-1.12.5 sasl-3.1.2 snmp-5.2.10 ssl-8.2.6 stdlib-3.4.5 syntax_tools-2.1.4 tools-2.11.2 wx-1.8.3 xmerl-1.3.16 : OTP-20.3.5 : erts-9.3.1 ssl-8.2.6 # asn1-5.0.5 common_test-1.15.4 compiler-7.1.5 cosEvent-2.2.2 cosEventDomain-1.2.2 cosFileTransfer-1.2.2 cosNotification-1.2.3 cosProperty-1.2.3 cosTime-1.2.3 cosTransactions-1.3.3 crypto-4.2.1 debugger-4.2.4 dialyzer-3.2.4 diameter-2.1.4 edoc-0.9.2 eldap-1.2.3 erl_docgen-0.7.2 erl_interface-3.10.2 et-1.6.1 eunit-2.3.5 hipe-3.17.1 ic-4.4.4 inets-6.5.1 jinterface-1.8.1 kernel-5.4.3 megaco-3.18.3 mnesia-4.15.3 observer-2.7 odbc-2.12.1 orber-3.8.4 os_mon-2.4.4 otp_mibs-1.1.2 parsetools-2.1.6 public_key-1.5.2 reltool-0.7.5 runtime_tools-1.12.5 sasl-3.1.2 snmp-5.2.10 ssh-4.6.8 stdlib-3.4.5 syntax_tools-2.1.4 tools-2.11.2 wx-1.8.3 xmerl-1.3.16 : @@ -26,6 +38,7 @@ OTP-20.0.3 : asn1-5.0.2 compiler-7.1.1 erts-9.0.3 ssh-4.5.1 # common_test-1.15.1 OTP-20.0.2 : asn1-5.0.1 erts-9.0.2 kernel-5.3.1 # common_test-1.15.1 compiler-7.1 cosEvent-2.2.1 cosEventDomain-1.2.1 cosFileTransfer-1.2.1 cosNotification-1.2.2 cosProperty-1.2.2 cosTime-1.2.2 cosTransactions-1.3.2 crypto-4.0 debugger-4.2.2 dialyzer-3.2 diameter-2.0 edoc-0.9 eldap-1.2.2 erl_docgen-0.7 erl_interface-3.10 et-1.6 eunit-2.3.3 hipe-3.16 ic-4.4.2 inets-6.4 jinterface-1.8 megaco-3.18.2 mnesia-4.15 observer-2.4 odbc-2.12 orber-3.8.3 os_mon-2.4.2 otp_mibs-1.1.1 parsetools-2.1.5 public_key-1.4.1 reltool-0.7.4 runtime_tools-1.12.1 sasl-3.0.4 snmp-5.2.6 ssh-4.5 ssl-8.2 stdlib-3.4.1 syntax_tools-2.1.2 tools-2.10.1 wx-1.8.1 xmerl-1.3.15 : OTP-20.0.1 : common_test-1.15.1 erts-9.0.1 runtime_tools-1.12.1 stdlib-3.4.1 tools-2.10.1 # asn1-5.0 compiler-7.1 cosEvent-2.2.1 cosEventDomain-1.2.1 cosFileTransfer-1.2.1 cosNotification-1.2.2 cosProperty-1.2.2 cosTime-1.2.2 cosTransactions-1.3.2 crypto-4.0 debugger-4.2.2 dialyzer-3.2 diameter-2.0 edoc-0.9 eldap-1.2.2 erl_docgen-0.7 erl_interface-3.10 et-1.6 eunit-2.3.3 hipe-3.16 ic-4.4.2 inets-6.4 jinterface-1.8 kernel-5.3 megaco-3.18.2 mnesia-4.15 observer-2.4 odbc-2.12 orber-3.8.3 os_mon-2.4.2 otp_mibs-1.1.1 parsetools-2.1.5 public_key-1.4.1 reltool-0.7.4 sasl-3.0.4 snmp-5.2.6 ssh-4.5 ssl-8.2 syntax_tools-2.1.2 wx-1.8.1 xmerl-1.3.15 : OTP-20.0 : asn1-5.0 common_test-1.15 compiler-7.1 cosProperty-1.2.2 crypto-4.0 debugger-4.2.2 dialyzer-3.2 diameter-2.0 edoc-0.9 erl_docgen-0.7 erl_interface-3.10 erts-9.0 eunit-2.3.3 hipe-3.16 inets-6.4 jinterface-1.8 kernel-5.3 megaco-3.18.2 mnesia-4.15 observer-2.4 orber-3.8.3 parsetools-2.1.5 public_key-1.4.1 reltool-0.7.4 runtime_tools-1.12 sasl-3.0.4 snmp-5.2.6 ssh-4.5 ssl-8.2 stdlib-3.4 syntax_tools-2.1.2 tools-2.10 wx-1.8.1 xmerl-1.3.15 # cosEvent-2.2.1 cosEventDomain-1.2.1 cosFileTransfer-1.2.1 cosNotification-1.2.2 cosTime-1.2.2 cosTransactions-1.3.2 eldap-1.2.2 et-1.6 ic-4.4.2 odbc-2.12 os_mon-2.4.2 otp_mibs-1.1.1 : +OTP-19.3.6.10 : erts-8.3.5.5 syntax_tools-2.1.1.1 # asn1-4.0.4 common_test-1.14 compiler-7.0.4.1 cosEvent-2.2.1 cosEventDomain-1.2.1 cosFileTransfer-1.2.1 cosNotification-1.2.2 cosProperty-1.2.1 cosTime-1.2.2 cosTransactions-1.3.2 crypto-3.7.4 debugger-4.2.1 dialyzer-3.1.1 diameter-1.12.2 edoc-0.8.1 eldap-1.2.2 erl_docgen-0.6.1 erl_interface-3.9.3 et-1.6 eunit-2.3.2 gs-1.6.2 hipe-3.15.4 ic-4.4.2 inets-6.3.9 jinterface-1.7.1 kernel-5.2.0.1 megaco-3.18.1 mnesia-4.14.3.1 observer-2.3.1 odbc-2.12 orber-3.8.2 os_mon-2.4.2 otp_mibs-1.1.1 parsetools-2.1.4 percept-0.9 public_key-1.4 reltool-0.7.3 runtime_tools-1.11.1 sasl-3.0.3 snmp-5.2.5 ssh-4.4.2.4 ssl-8.1.3.1.1 stdlib-3.3 tools-2.9.1 typer-0.9.12 wx-1.8 xmerl-1.3.14 : OTP-19.3.6.9 : ssh-4.4.2.4 # asn1-4.0.4 common_test-1.14 compiler-7.0.4.1 cosEvent-2.2.1 cosEventDomain-1.2.1 cosFileTransfer-1.2.1 cosNotification-1.2.2 cosProperty-1.2.1 cosTime-1.2.2 cosTransactions-1.3.2 crypto-3.7.4 debugger-4.2.1 dialyzer-3.1.1 diameter-1.12.2 edoc-0.8.1 eldap-1.2.2 erl_docgen-0.6.1 erl_interface-3.9.3 erts-8.3.5.4 et-1.6 eunit-2.3.2 gs-1.6.2 hipe-3.15.4 ic-4.4.2 inets-6.3.9 jinterface-1.7.1 kernel-5.2.0.1 megaco-3.18.1 mnesia-4.14.3.1 observer-2.3.1 odbc-2.12 orber-3.8.2 os_mon-2.4.2 otp_mibs-1.1.1 parsetools-2.1.4 percept-0.9 public_key-1.4 reltool-0.7.3 runtime_tools-1.11.1 sasl-3.0.3 snmp-5.2.5 ssl-8.1.3.1.1 stdlib-3.3 syntax_tools-2.1.1 tools-2.9.1 typer-0.9.12 wx-1.8 xmerl-1.3.14 : OTP-19.3.6.8 : ssh-4.4.2.3 # asn1-4.0.4 common_test-1.14 compiler-7.0.4.1 cosEvent-2.2.1 cosEventDomain-1.2.1 cosFileTransfer-1.2.1 cosNotification-1.2.2 cosProperty-1.2.1 cosTime-1.2.2 cosTransactions-1.3.2 crypto-3.7.4 debugger-4.2.1 dialyzer-3.1.1 diameter-1.12.2 edoc-0.8.1 eldap-1.2.2 erl_docgen-0.6.1 erl_interface-3.9.3 erts-8.3.5.4 et-1.6 eunit-2.3.2 gs-1.6.2 hipe-3.15.4 ic-4.4.2 inets-6.3.9 jinterface-1.7.1 kernel-5.2.0.1 megaco-3.18.1 mnesia-4.14.3.1 observer-2.3.1 odbc-2.12 orber-3.8.2 os_mon-2.4.2 otp_mibs-1.1.1 parsetools-2.1.4 percept-0.9 public_key-1.4 reltool-0.7.3 runtime_tools-1.11.1 sasl-3.0.3 snmp-5.2.5 ssl-8.1.3.1.1 stdlib-3.3 syntax_tools-2.1.1 tools-2.9.1 typer-0.9.12 wx-1.8 xmerl-1.3.14 : OTP-19.3.6.7 : kernel-5.2.0.1 # asn1-4.0.4 common_test-1.14 compiler-7.0.4.1 cosEvent-2.2.1 cosEventDomain-1.2.1 cosFileTransfer-1.2.1 cosNotification-1.2.2 cosProperty-1.2.1 cosTime-1.2.2 cosTransactions-1.3.2 crypto-3.7.4 debugger-4.2.1 dialyzer-3.1.1 diameter-1.12.2 edoc-0.8.1 eldap-1.2.2 erl_docgen-0.6.1 erl_interface-3.9.3 erts-8.3.5.4 et-1.6 eunit-2.3.2 gs-1.6.2 hipe-3.15.4 ic-4.4.2 inets-6.3.9 jinterface-1.7.1 megaco-3.18.1 mnesia-4.14.3.1 observer-2.3.1 odbc-2.12 orber-3.8.2 os_mon-2.4.2 otp_mibs-1.1.1 parsetools-2.1.4 percept-0.9 public_key-1.4 reltool-0.7.3 runtime_tools-1.11.1 sasl-3.0.3 snmp-5.2.5 ssh-4.4.2.2 ssl-8.1.3.1.1 stdlib-3.3 syntax_tools-2.1.1 tools-2.9.1 typer-0.9.12 wx-1.8 xmerl-1.3.14 : diff --git a/scripts/build-otp b/scripts/build-otp index ad0eb07359..abf8d5d67f 100755 --- a/scripts/build-otp +++ b/scripts/build-otp @@ -1,5 +1,7 @@ #!/bin/bash +set -e + function progress { local file=$1 ls=$(ls -l $file) @@ -14,15 +16,19 @@ function progress { } function do_and_log { + start_time=`date +%s` log="logs/latest-log.$$" echo "" >$log echo -n "$1..." (progress $log) & pid=$! disown - if ./otp_build $2 $3 >$log 2>&1; then + shift + if $* >$log 2>&1; then kill $pid >/dev/null 2>&1 - echo " done." + stop_time=`date +%s` + diff_time=$((stop_time-start_time)) + echo " done, took $diff_time seconds" else kill $pid >/dev/null 2>&1 echo " failed." @@ -36,12 +42,38 @@ if [ ! -d "logs" ]; then mkdir logs fi -do_and_log "Autoconfing" autoconf -do_and_log "Configuring" configure -do_and_log "Building OTP" boot -a +do_and_log "Autoconfing" ./otp_build autoconf +do_and_log "Configuring" ./otp_build configure +do_and_log "Building OTP" ./otp_build boot -a + +if [ "$1" = "release" ]; then + do_and_log "Releasing OTP" ./otp_build release -a +fi -if [ $1 = "release" ]; then - do_and_log "Releasing OTP" release -a +if [ "$1" = "docs" ]; then + DOC_TARGET=${TRAVIS_BRANCH:-release/`erts/autoconf/config.guess`} + DOC_TARGET=${TRAVIS_TAG:-$DOC_TARGET} + TESTROOT=$PWD/$DOC_TARGET do_and_log "Building documentation" make release_docs + do_and_log "Linting documentation" make xmllint + # The code below prepares this build to be used as a deploy to + # github pages for documentation. + if [ "$TRAVIS_PULL_REQUEST" = "false" -a "$TRAVIS_TAG" = "" -a "$TRAVIS_REPO_SLUG" = "erlang/otp" ]; then + set -x + rm -rf logs + SHA=`git rev-parse --verify HEAD` + DATE=`git show -s --format=%ci` + git clean -xfdq -e $DOC_TARGET + git fetch https://github.com/erlang/cd master + git checkout -f FETCH_HEAD + rm -rf _docs/$DOC_TARGET + mv $DOC_TARGET _docs/$DOC_TARGET + echo "---" > _docs/$DOC_TARGET.md + echo "title: $DOC_TARGET" >> _docs/$DOC_TARGET.md + echo "sha: $SHA" >> _docs/$DOC_TARGET.md + echo "generated: $DATE" >> _docs/$DOC_TARGET.md + echo "---" >> _docs/$DOC_TARGET.md + set +x + fi fi exit 0 diff --git a/scripts/bundle-otp b/scripts/bundle-otp new file mode 100755 index 0000000000..0ffe7d8c77 --- /dev/null +++ b/scripts/bundle-otp @@ -0,0 +1,35 @@ +#!/bin/bash + +set -e + +if [ "$TRAVIS_PULL_REQUEST" = "false" -a "$TRAVIS_REPO_SLUG" != "erlang/otp" ]; then + exit 0 +fi + +OTP_META_FILE=$ERL_TOP/${TRAVIS_TAG}-bundle.txt +OTP_FILE=$ERL_TOP/${TRAVIS_TAG}-bundle.tar.gz + +REPOSITORIES="otp,$TRAVIS_TAG corba,.*" + +mkdir bundle + +for repo in $REPOSITORIES; do + OLD_IFS=$IFS + IFS=',' + set -- $repo + IFS=$OLD_IFS + cd $ERL_TOP/bundle/ + git clone https://github.com/erlang/$1 $1 + cd $1 + echo $1 $2 + TAG=`git tag -l | grep -P "$2" | sort -V | tail -1` + git checkout $TAG + SHA=`git rev-parse --verify HEAD` + rm -rf .git + echo "$1 $TAG $SHA" >> $OTP_META_FILE +done + +cd $ERL_TOP/bundle/ +tar czf $OTP_FILE * + +exit 0 diff --git a/scripts/diffable b/scripts/diffable index 08d2d5cb35..6a9792e857 100755 --- a/scripts/diffable +++ b/scripts/diffable @@ -117,16 +117,25 @@ get_files(Apps, #{}=Opts) -> {Files,Opts}. add_opts([F|Fs], Opts0) -> - Opts = case filename:basename(F) of - "group_history.erl" -> + Opts = case vsn_is_harmful(F) of + true -> Opts0 -- [{d,'VSN',1}]; - _ -> + false -> Opts0 end, [{F,Opts}|add_opts(Fs, Opts0)]; add_opts([], _Opts) -> []. +vsn_is_harmful(F) -> + case filename:basename(F) of + "group_history.erl" -> + true; + _ -> + App = filename:basename(filename:dirname(filename:dirname(F))), + App =:= "ssl" + end. + get_src(["preloaded"|Apps]) -> WC = filename:join(code:root_dir(), "erts/preloaded/src/*.erl"), filelib:wildcard(WC) ++ get_src(Apps); diff --git a/scripts/pre-push b/scripts/pre-push index 0349378056..71e9fd1e75 100755 --- a/scripts/pre-push +++ b/scripts/pre-push @@ -22,15 +22,30 @@ # <local ref> <local sha1> <remote ref> <remote sha1> # -RELEASES="20 19 18 17 r16 r15 r14 r13" +NEW_RELEASES="21 20 19 18 17" +OLD_RELEASES="r16 r15 r14 r13" +RELEASES="$NEW_RELEASES $OLD_RELEASES" # First commit on master, not allowed in other branches -MASTER_ONLY=f52748254f17ba42e344798e8c787a1e3361fa33 +MASTER_ONLY=aea2a053e28a11497796879715be29ab0c3cd1a0 # Number of commits and files allowed in one push by this script NCOMMITS_MAX=100 NFILES_MAX=100 + +# Example testing this script for "git push upstream OTP-20.3.8.2": +# +#> null=0000000000000000000000000000000000000000 +#> echo "refs/tags/OTP-20.3.8.2 dummysha refs/tags/OTP-20.3.8.2 $null" | scripts/pre-push upstream https://github.com/erlang/otp.git + +# Example to test "git push upstream master" +# +#> local_sha=`git rev-parse master` +#> remote_sha=`git rev-parse upstream/master` +#> echo "refs/heads/master $local_sha refs/heads/master $remote_sha" | scripts/pre-push upstream https://github.com/erlang/otp.git + + remote="$1" url="$2" @@ -158,8 +173,24 @@ then exit 1 fi ;; - refs/tags/OTP-20.* | refs/tags/OTP-19.* | refs/tags/OTP-18.* | refs/tags/OTP-17.*) + refs/tags/OTP-*) tag=${remote_ref#refs/tags/} + REL="UNKNOWN" + for x in $NEW_RELEASES; do + if [ ${tag#OTP-$x.} != $tag ] + then + REL=$x + break + fi + done + if [ $REL = "UNKNOWN" ] + then + echo "$0 says:" + echo "***" + echo "*** Unknown OTP release number in tag '$tag'" + echo "***" + exit 1 + fi if [ "$remote_sha" != $null ] then echo "$0 says:" diff --git a/scripts/run-dialyzer b/scripts/run-dialyzer index c9da647952..621de3fa65 100755 --- a/scripts/run-dialyzer +++ b/scripts/run-dialyzer @@ -1,16 +1,17 @@ #!/bin/bash set -e +set -x $ERL_TOP/bin/dialyzer --build_plt --apps asn1 compiler crypto dialyzer edoc erts et ftp hipe inets kernel mnesia observer public_key runtime_tools snmp ssh ssl stdlib syntax_tools tftp wx xmerl --statistics $ERL_TOP/bin/dialyzer -n -Wunknown -Wunmatched_returns --apps compiler erts ftp tftp kernel stdlib asn1 crypto dialyzer hipe parsetools public_key runtime_tools sasl tools --statistics $ERL_TOP/bin/dialyzer -n --apps common_test debugger edoc ftp inets mnesia observer ssh ssl syntax_tools tftp wx xmerl --statistics # In travis we don't dialyze everything as it takes too much time -if [ "X$DIALYZE_ALL_APPLICATIONS" = "Xtrue" ]; then +if [ "X$TRAVIS" != "Xtrue" ]; then $ERL_TOP/bin/dialyzer -n -Wunknown -Wunmatched_returns --apps eldap erl_docgen et odbc --statistics $ERL_TOP/bin/dialyzer -n --apps eunit reltool os_mon --statistics # These application are not run always as the currently have dialyzer warnings - # $ERL_TOP/bin/dialyzer -n --apps cosEvent cosEventDomain cosFileTransfer cosNotification cosProperty cosTime cosTransactions diameter megaco orber snmp --statistics + # $ERL_TOP/bin/dialyzer -n --apps diameter megaco snmp --statistics fi diff --git a/system/doc/design_principles/Makefile b/system/doc/design_principles/Makefile index 41d2d1208f..242bf1c9a4 100644 --- a/system/doc/design_principles/Makefile +++ b/system/doc/design_principles/Makefile @@ -108,6 +108,7 @@ debug opt: clean clean_docs: rm -rf $(HTMLDIR) + rm -rf $(XMLDIR) rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) rm -f errs core *~ diff --git a/system/doc/design_principles/applications.xml b/system/doc/design_principles/applications.xml index c673fde07e..3b7b8fdaee 100644 --- a/system/doc/design_principles/applications.xml +++ b/system/doc/design_principles/applications.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>1997</year><year>2016</year> + <year>1997</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -363,9 +363,13 @@ ok application are running.</p> <marker id="application_master"></marker> <p>The application controller then creates an - <em>application master</em> for the application. The application master - is the group leader of all the processes in the application. - The application master starts the application by calling + <em>application master</em> for the application. The application + master becomes the group leader of all the processes in the + application. I/O is forwarded to the previous group leader, + though, this is just a way to identify processes that belong to + the application. Used for example to find itself from any process, + or, reciprocally, to kill them all when it terminates.</p> + <p>The application master starts the application by calling the application callback function <c>start/2</c> in the module, and with the start argument, defined by the <c>mod</c> key in the <c>.app</c> file.</p> diff --git a/system/doc/design_principles/statem.xml b/system/doc/design_principles/statem.xml index 80ee9c992f..98fd1fd69d 100644 --- a/system/doc/design_principles/statem.xml +++ b/system/doc/design_principles/statem.xml @@ -956,14 +956,14 @@ callback_mode() -> <seealso marker="stdlib:gen_statem#cast/2"><c>gen_statem:cast/2</c></seealso>: </p> <code type="erl"><![CDATA[ -button(Digit) -> - gen_statem:cast(?NAME, {button,Digit}). +button(Button) -> + gen_statem:cast(?NAME, {button,Button}). ]]></code> <p> The first argument is the name of the <c>gen_statem</c> and must agree with the name used to start it. So, we use the same macro <c>?NAME</c> as when starting. - <c>{button,Digit}</c> is the event content. + <c>{button,Button}</c> is the event content. </p> <p> The event is sent to the <c>gen_statem</c>. @@ -1171,7 +1171,7 @@ open(...) -> ... ; callback_mode() -> handle_event_function. -handle_event(cast, {button,Digit}, State, #{code := Code} = Data) -> +handle_event(cast, {button,Button}, State, #{code := Code} = Data) -> case State of locked -> #{length := Length, buttons := Buttons} = Data, @@ -1305,7 +1305,7 @@ stop() -> locked(timeout, _, Data) -> {next_state, locked, Data#{buttons := []}}; locked( - cast, {button,Digit}, + cast, {button,Button}, #{code := Code, length := Length, buttons := Buttons} = Data) -> ... true -> % Incomplete | Incorrect @@ -1364,7 +1364,7 @@ locked( <code type="erl"><![CDATA[ ... locked( - cast, {button,Digit}, + cast, {button,Button}, #{code := Code, length := Length, buttons := Buttons} = Data) -> ... if @@ -1410,7 +1410,7 @@ open(cast, {button,_}, Data) -> <seealso marker="erts:erlang#start_timer/4"><c>erlang:start_timer/3,4</c></seealso>. Most time-out tasks can be performed with the time-out features in <c>gen_statem</c>, - but an example of one that can not is if you should need + but an example of one that cannot is if you should need the return value from <seealso marker="erts:erlang#cancel_timer/2"><c>erlang:cancel_timer(Tref)</c></seealso>, that is; the remaining time of the timer. </p> @@ -1421,7 +1421,7 @@ open(cast, {button,_}, Data) -> <code type="erl"><![CDATA[ ... locked( - cast, {button,Digit}, + cast, {button,Button}, #{code := Code, length := Length, buttons := Buttons} = Data) -> ... if @@ -1662,7 +1662,7 @@ locked(enter, _OldState, Data) -> do_lock(), {keep_state,Data#{buttons => []}}; locked( - cast, {button,Digit}, + cast, {button,Button}, #{code := Code, length := Length, buttons := Buttons} = Data) -> ... if @@ -1747,7 +1747,7 @@ open(state_timeout, lock, Data) -> </p> <code type="erl"><![CDATA[ ... --export(down/1, up/1). +-export([down/1, up/1]). ... down(Button) -> gen_statem:cast(?NAME, {down,Button}). @@ -1759,15 +1759,15 @@ up(Button) -> locked(enter, _OldState, Data) -> do_lock(), - {keep_state,Data#{remaining => Code, buf => []}}; + {keep_state,Data#{buttons => []}}; locked( - internal, {button,Digit}, + internal, {button,Button}, #{code := Code, length := Length, buttons := Buttons} = Data) -> ... ]]></code> <code type="erl"><![CDATA[ handle_common(cast, {down,Button}, Data) -> - {keep_state, Data#{button := Button}}; + {keep_state, Data#{button => Button}}; handle_common(cast, {up,Button}, Data) -> case Data of #{button := Button} -> @@ -1833,10 +1833,10 @@ start_link(Code) -> stop() -> gen_statem:stop(?NAME). -down(Digit) -> - gen_statem:cast(?NAME, {down,Digit}). -up(Digit) -> - gen_statem:cast(?NAME, {up,Digit}). +down(Button) -> + gen_statem:cast(?NAME, {down,Button}). +up(Button) -> + gen_statem:cast(?NAME, {up,Button}). code_length() -> gen_statem:call(?NAME, code_length). ]]></code> @@ -1873,7 +1873,7 @@ locked(enter, _OldState, Data) -> locked(state_timeout, button, Data) -> {keep_state, Data#{buttons := []}}; locked( - internal, {button,Digit}, + internal, {button,Button}, #{code := Code, length := Length, buttons := Buttons} = Data) -> NewButtons = if @@ -1884,7 +1884,6 @@ locked( end ++ [Button], if NewButtons =:= Code -> % Correct - do_unlock(), {next_state, open, Data}; true -> % Incomplete | Incorrect {keep_state, Data#{buttons := NewButtons}, @@ -1940,7 +1939,7 @@ handle_event(enter, _OldState, locked, Data) -> handle_event(state_timeout, button, locked, Data) -> {keep_state, Data#{buttons := []}}; handle_event( - internal, {button,Digit}, locked, + internal, {button,Button}, locked, #{code := Code, length := Length, buttons := Buttons} = Data) -> NewButtons = if @@ -1951,7 +1950,6 @@ handle_event( end ++ [Button], if NewButtons =:= Code -> % Correct - do_unlock(), {next_state, open, Data}; true -> % Incomplete | Incorrect {keep_state, Data#{buttons := NewButtons}, @@ -2152,7 +2150,7 @@ handle_event(enter, _OldState, {locked,_}, Data) -> handle_event(state_timeout, button, {locked,_}, Data) -> {keep_state, Data#{buttons := []}}; handle_event( - cast, {button,Digit}, {locked,LockButton}, + cast, {button,Button}, {locked,LockButton}, #{code := Code, length := Length, buttons := Buttons} = Data) -> NewButtons = if @@ -2163,7 +2161,6 @@ handle_event( end ++ [Button], if NewButtons =:= Code -> % Correct - do_unlock(), {next_state, {open,LockButton}, Data}; true -> % Incomplete | Incorrect {keep_state, Data#{buttons := NewButtons}, @@ -2177,11 +2174,11 @@ handle_event(enter, _OldState, {open,_}, _Data) -> do_unlock(), {keep_state_and_data, [{state_timeout,10000,lock}]}; % Time in milliseconds -handle_event(state_timeout, lock, {open,_}, Data) -> - {next_state, locked, Data}; +handle_event(state_timeout, lock, {open,LockButton}, Data) -> + {next_state, {locked,LockButton}, Data}; handle_event(cast, {button,LockButton}, {open,LockButton}, Data) -> {next_state, {locked,LockButton}, Data}; -handle_event(cast, {button,_}, {open,_}, Data) -> +handle_event(cast, {button,_}, {open,_}, _Data) -> {keep_state_and_data,[postpone]}; ]]></code> <code type="erl"><![CDATA[ diff --git a/system/doc/efficiency_guide/Makefile b/system/doc/efficiency_guide/Makefile index f6ad638853..72bcd2ee73 100644 --- a/system/doc/efficiency_guide/Makefile +++ b/system/doc/efficiency_guide/Makefile @@ -99,6 +99,7 @@ debug opt: clean clean_docs: rm -rf $(HTMLDIR) + rm -rf $(XMLDIR) rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) rm -f errs core *~ diff --git a/system/doc/embedded/Makefile b/system/doc/embedded/Makefile index 2b09c5b852..396aef276b 100644 --- a/system/doc/embedded/Makefile +++ b/system/doc/embedded/Makefile @@ -87,6 +87,7 @@ debug opt: clean clean_docs: rm -rf $(HTMLDIR) + rm -rf $(XMLDIR) rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) rm -f errs core *~ diff --git a/system/doc/embedded/starting.xml b/system/doc/embedded/starting.xml index 11bf9b412a..6888f9c959 100644 --- a/system/doc/embedded/starting.xml +++ b/system/doc/embedded/starting.xml @@ -231,7 +231,7 @@ exec $BINDIR/erlexec -boot $RELDIR/$VSN/start -config $RELDIR/$VSN/sys $* < <p>If a diskless and/or read-only client node with the SASL configuration parameter <c>static_emulator</c> set to <c>true</c> is about to start the <c>-boot</c> and <c>-config</c> flags must be - changed. As such a client can not read a new <c>start_erl.data</c> + changed. As such a client cannot read a new <c>start_erl.data</c> file (the file is not possible to change dynamically) the boot and config files is always fetched from the same place (but with a new contents if a new release has been installed). The diff --git a/system/doc/getting_started/Makefile b/system/doc/getting_started/Makefile index 7b90fe1337..cdf1e121c2 100644 --- a/system/doc/getting_started/Makefile +++ b/system/doc/getting_started/Makefile @@ -86,6 +86,7 @@ debug opt: clean clean_docs: rm -rf $(HTMLDIR) + rm -rf $(XMLDIR) rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) rm -f errs core *~ diff --git a/system/doc/installation_guide/Makefile b/system/doc/installation_guide/Makefile index 91e7cb2772..4a1335cf31 100644 --- a/system/doc/installation_guide/Makefile +++ b/system/doc/installation_guide/Makefile @@ -113,6 +113,7 @@ debug opt: clean clean_docs: rm -f $(GENERATED_XML_FILES) rm -rf $(HTMLDIR) + rm -rf $(XMLDIR) rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) rm -f errs core *~ diff --git a/system/doc/oam/Makefile b/system/doc/oam/Makefile index b09ae1aed2..147f56f885 100644 --- a/system/doc/oam/Makefile +++ b/system/doc/oam/Makefile @@ -88,6 +88,7 @@ debug opt: clean clean_docs: rm -rf $(HTMLDIR) + rm -rf $(XMLDIR) rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) rm -f errs core *~ diff --git a/system/doc/programming_examples/Makefile b/system/doc/programming_examples/Makefile index 2d04e8b5e2..e4737ba069 100644 --- a/system/doc/programming_examples/Makefile +++ b/system/doc/programming_examples/Makefile @@ -85,6 +85,7 @@ debug opt: clean clean_docs: rm -rf $(HTMLDIR) + rm -rf $(XMLDIR) rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) rm -f errs core *~ diff --git a/system/doc/reference_manual/Makefile b/system/doc/reference_manual/Makefile index 02a7f002ed..d034ad2ff8 100644 --- a/system/doc/reference_manual/Makefile +++ b/system/doc/reference_manual/Makefile @@ -96,6 +96,7 @@ debug opt: clean clean_docs: rm -rf $(HTMLDIR) + rm -rf $(XMLDIR) rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) rm -f errs core *~ diff --git a/system/doc/reference_manual/expressions.xml b/system/doc/reference_manual/expressions.xml index 94e40dd077..1f48233c39 100644 --- a/system/doc/reference_manual/expressions.xml +++ b/system/doc/reference_manual/expressions.xml @@ -567,6 +567,10 @@ Expr1 <input>op</input> Expr2</pre> order is defined:</p> <pre> number < atom < reference < fun < port < pid < tuple < map < nil < list < bit string</pre> + <p><c>nil</c> in the previous expression represents the empty list + (<c>[]</c>), which is regarded as a separate type from + <c>list/0</c>. That is why <c>nil < list</c>. + </p> <p>Lists are compared element by element. Tuples are ordered by size, two tuples with the same size are compared element by element.</p> @@ -574,6 +578,7 @@ number < atom < reference < fun < port < pid < tuple < map ascending term order and then by values in key order. In maps key order integers types are considered less than floats types. </p> + <p>Atoms are compared using their string value, codepoint by codepoint.</p> <p>When comparing an integer to a float, the term with the lesser precision is converted into the type of the other term, unless the operator is one of <c>=:=</c> or <c>=/=</c>. A float is more precise than diff --git a/system/doc/reference_manual/introduction.xml b/system/doc/reference_manual/introduction.xml index c9f5fcb2db..69e52e0b37 100644 --- a/system/doc/reference_manual/introduction.xml +++ b/system/doc/reference_manual/introduction.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2003</year><year>2017</year> + <year>2003</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/system/doc/system_architecture_intro/Makefile b/system/doc/system_architecture_intro/Makefile index ebfcc3a1c8..eb885a744d 100644 --- a/system/doc/system_architecture_intro/Makefile +++ b/system/doc/system_architecture_intro/Makefile @@ -81,6 +81,7 @@ debug opt: clean clean_docs: rm -rf $(HTMLDIR) + rm -rf $(XMLDIR) rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) rm -f errs core *~ diff --git a/system/doc/system_principles/Makefile b/system/doc/system_principles/Makefile index bb74125f3a..1979deda4c 100644 --- a/system/doc/system_principles/Makefile +++ b/system/doc/system_principles/Makefile @@ -82,6 +82,7 @@ debug opt: clean clean_docs: rm -rf $(HTMLDIR) + rm -rf $(XMLDIR) rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) rm -f errs core *~ diff --git a/system/doc/system_principles/error_logging.xml b/system/doc/system_principles/error_logging.xml index 9d95ce8f3b..9cbf7a2e94 100644 --- a/system/doc/system_principles/error_logging.xml +++ b/system/doc/system_principles/error_logging.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2003</year><year>2016</year> + <year>2003</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -62,46 +62,48 @@ Error in process <0.27.0> with exit value: {{badmatch,[1,2,3]},[{m,f,1},{shell,e <p>The standard behaviours (<c>supervisor</c>, <c>gen_server</c>, and so on) send progress and error information to Logger. Progress reports are by default not logged, but can be - enabled by setting the Kernel configuration - parameter <c>logger_progress_reports</c> to <c>log</c>. Supervisor - reports, crash reports and other error and information reports - are by default logged through the log handler which is - set up when the Kernel application is started.</p> + enabled by setting the primary log level to <c>info</c>, for + example by using the Kernel configuration + parameter <c>logger_level</c>. Supervisor reports, crash reports + and other error and information reports are by default logged + through the log handler which is set up when the Kernel + application is started.</p> <p>Prior to Erlang/OTP 21.0, supervisor, crash, and progress reports were only logged when the SASL application was running. This behaviour can, for backwards compatibility, be enabled by setting the Kernel configuration - parameter <c>logger_sasl_compatible</c> to <c>true</c>. For more - information, see + parameter <seealso marker="kernel:kernel_app#logger_sasl_compatible"> + <c>logger_sasl_compatible</c></seealso> + to <c>true</c>. For more information, see <seealso marker="sasl:error_logging">SASL Error Logging</seealso> in the SASL User's Guide.</p> <pre> -% <input>erl -kernel logger_progress_reports log</input> -Erlang/OTP 21 [erts-10.0] [source-76388a1] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe] +% <input>erl -kernel logger_level info</input> +Erlang/OTP 21 [erts-10.0] [source-13c50db] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe] -=PROGRESS REPORT==== 18-May-2018::21:33:41.705292 === +=PROGRESS REPORT==== 8-Jun-2018::16:54:19.916404 === application: kernel started_at: nonode@nohost -=PROGRESS REPORT==== 18-May-2018::21:33:41.708900 === +=PROGRESS REPORT==== 8-Jun-2018::16:54:19.922908 === application: stdlib started_at: nonode@nohost -=PROGRESS REPORT==== 18-May-2018::21:33:41.726003 === +=PROGRESS REPORT==== 8-Jun-2018::16:54:19.925755 === supervisor: {local,kernel_safe_sup} - started: [{pid,<0.75.0>}, + started: [{pid,<0.74.0>}, {id,disk_log_sup}, {mfargs,{disk_log_sup,start_link,[]}}, {restart_type,permanent}, {shutdown,1000}, {child_type,supervisor}] -=PROGRESS REPORT==== 18-May-2018::21:33:41.726348 === +=PROGRESS REPORT==== 8-Jun-2018::16:54:19.926056 === supervisor: {local,kernel_safe_sup} - started: [{pid,<0.76.0>}, + started: [{pid,<0.75.0>}, {id,disk_log_server}, {mfargs,{disk_log_server,start_link,[]}}, {restart_type,permanent}, {shutdown,2000}, {child_type,worker}] -Eshell V9.3.1 (abort with ^G) +Eshell V10.0 (abort with ^G) 1> </pre> </section> </chapter> diff --git a/system/doc/system_principles/misc.xml b/system/doc/system_principles/misc.xml new file mode 100644 index 0000000000..dd6c2a1336 --- /dev/null +++ b/system/doc/system_principles/misc.xml @@ -0,0 +1,198 @@ +<?xml version="1.0" encoding="utf-8" ?> +<!DOCTYPE chapter SYSTEM "chapter.dtd"> + +<chapter> + <header> + <copyright> + <year>2018</year> + <holder>Ericsson AB. All Rights Reserved.</holder> + </copyright> + <legalnotice> + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + + </legalnotice> + + <title>Support, Compatibility, Deprecations, and Removal</title> + <prepared></prepared> + <responsible></responsible> + <docno></docno> + <approved></approved> + <checked></checked> + <date>2018-05-21</date> + <rev></rev> + <file>misc.xml</file> + </header> + + <section> + <marker id="supported_releases"/> + <title>Supported Releases</title> + <p> + In general, bugs are only fixed on the latest + <seealso marker="versions#releases_and_patches">release</seealso>, + and new features are introduced in the upcoming release that is + under development. However, when we, due to internal reasons, fix + bugs on older releases, these will be available and announced as well. + </p> + <p> + Due to the above, pull requests are only accepted on the + <c>maint</c> and the <c>master</c> branches in our + <url href="https://github.com/erlang/otp">git repository</url>. + The <c>maint</c> branch contains changes planned for the next + <seealso marker="versions#releases_and_patches">maintenance patch package</seealso> + on the latest OTP release and the <c>master</c> branch contain + changes planned for the upcoming OTP release. + </p> + </section> + + <section> + <marker id="compatibility"/> + <title>Compatibility</title> + <p> + We always strive to remain as compatible as possible + even in the cases where we give no compatibility guarantees. + </p> + <p> + Different parts of the system will be handled differently + regarding compatibility. The following items describe how + different parts of the system are handled. + </p> + <taglist> + <tag>Erlang Distribution</tag> + <item> + <p> + Erlang nodes can communicate across at least + two preceding and two subsequent releases. + </p> + </item> + <tag>Compiled BEAM Code, NIF Libraries and Drivers</tag> + <item> + <p> + Compiled code can be loaded on at least two + subsequent releases. + </p> + <p> + Loading on previous releases is <em>not</em> supported. + </p> + </item> + <tag>Compiled HiPE Code</tag> + <item> + <p> + Compiled HiPE code can be loaded on the exact same build + of ERTS that was used when compiling the code. It might + however work on other builds, the emulator verifies + checksums in order to determine if it can load the code + or not. Note that HiPE has some limitations. For more + information see the documentation of the + <seealso marker="hipe:HiPE_app">HiPE</seealso> application. + </p> + </item> + <tag>APIs</tag> + <item> + <p>Compatible between releases.</p> + </item> + <tag>Compiler Warnings</tag> + <item> + <p>New warnings may be issued between releases.</p> + </item> + <tag>Command Line Arguments</tag> + <item> + <p>Incompatible changes may occur between releases.</p> + </item> + <tag>OTP Build Procedures</tag> + <item><p>Incompatible changes may occur between releases.</p></item> + </taglist> + <p> + Under certain circumstances incompatible changes might be + introduced even in parts of the system that should be compatible + between releases. Things that might trigger incompatible changes + like this are: + </p> + <taglist> + <tag>Security Issues</tag> + <item> + <p> + It might be necessary to introduce incompatible changes + in order to solve a security issue. This kind of + incompatibility might occur in a patch. + </p> + </item> + <tag>Bug Fixes</tag> + <item> + <p> + We will not be bug-compatible. A bug fix might introduce + incompatible changes. This kind of incompatibility + might occur in a patch. + </p> + </item> + <tag>Severe Previous Design Issues</tag> + <item> + <p> + Some parts of OTP were designed a very long time ago and + did not necessarily take today's computing environments into + account. In some cases the consequences of those design + decisions are too severe. This may be performance wise, + scalability wise, etc. If we deem the consequences too + severe, we might introduce incompatible changes. This kind + of incompatibility will not be introduced in a patch, but + instead in the next release. + </p> + </item> + </taglist> + <p> + Peripheral, trace, and debug functionality is at greater + risk of being changed in an incompatible way than functionality + in the language itself and core libraries used during operation. + </p> + </section> + + <section> + <marker id="deprecation"/> + <title>Deprecation</title> + <p> + Functionality is deprecated when new functionality is + introduced that is preferred to be used instead of the + old functionality that is being deprecated. The deprecation + does <em>not</em> imply removal of the functionality unless + an upcoming removal is explicitly stated in the deprecation. + </p> + <p> + Deprecated functionality will be documented as deprecated, and + compiler warnings will be issued, when appropriate, as + early as possible. That is, the new preferred functionality + will appear at the same time as the deprecation is issued. + A new deprecation will at least be announced in a release + note and the documentation. + </p> + </section> + + <section> + <marker id="removal"/> + <title>Removal</title> + <p> + Legacy solutions may eventually need to be removed. In such + cases, they will be phased out on a long enough time period + to give users the time to adapt. Before removal of + functionality it will be deprecated at least during one + release with an explicit announcement about + the upcoming removal. A new deprecation will at least be + announced in a release note and the documentation. + </p> + <p> + Peripheral, trace, and debug functionality is at greater + risk of removal than functionality in the language itself + and core libraries used during operation. + </p> + </section> + +</chapter> + diff --git a/system/doc/system_principles/part.xml b/system/doc/system_principles/part.xml index 1b87ecd350..6699389eec 100644 --- a/system/doc/system_principles/part.xml +++ b/system/doc/system_principles/part.xml @@ -4,7 +4,7 @@ <part xmlns:xi="http://www.w3.org/2001/XInclude"> <header> <copyright> - <year>1997</year><year>2016</year> + <year>1997</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -34,4 +34,5 @@ <xi:include href="create_target.xml"/> <xi:include href="upgrade.xml"/> <xi:include href="versions.xml"/> + <xi:include href="misc.xml"/> </part> diff --git a/system/doc/system_principles/system_principles.xml b/system/doc/system_principles/system_principles.xml index b3981f0735..500522d778 100644 --- a/system/doc/system_principles/system_principles.xml +++ b/system/doc/system_principles/system_principles.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>1996</year><year>2015</year> + <year>1996</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> diff --git a/system/doc/system_principles/versions.xml b/system/doc/system_principles/versions.xml index b9f7fa4bf6..11dc9d5637 100644 --- a/system/doc/system_principles/versions.xml +++ b/system/doc/system_principles/versions.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2014</year><year>2016</year> + <year>2014</year><year>2018</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -32,9 +32,9 @@ <rev></rev> <file>versions.xml</file> </header> - <marker id="versions section"></marker> <section> + <marker id="versions section"></marker> <title>OTP Version</title> <p>As of OTP release 17, the OTP release number corresponds to the major part of the OTP version. The OTP version as a concept was @@ -136,8 +136,8 @@ </section> <section> - <title>Version Scheme</title> <marker id="version_scheme"/> + <title>Version Scheme</title> <note><p>The version scheme was changed as of OTP 17.0. This implies that application versions used prior to OTP 17.0 do not adhere to this version scheme. <seealso marker="#otp_17_0_app_versions">A list of @@ -207,8 +207,41 @@ </section> <section> - <title>OTP 17.0 Application Versions</title> + <marker id="releases_and_patches"/> + <title>Releases and Patches</title> + <p> + When a new OTP release is released it will have an OTP + version on the form <c><Major>.0</c> where the + major OTP version number equals the release number. + The major version number is increased one step since the + last major version. All other OTP versions with the same + major OTP version number are patches on that OTP release. + </p> + <p> + Patches are either released as maintenance patch packages + or emergency patch packages. The only difference is that + maintenance patch packages are planned and usually contain + more changes than emergency patch packages. Emergency patch + packages are released to solve one or more specific issues + when such are discovered. + </p> + <p> + The release of a maintenance patch package usually imply + an increase of the OTP <c><Minor></c> version while + the release of an emergency patch package usually imply an + increase of the OTP <c><Patch></c> version. This is + however not necessarily always the case since changes of + OTP versions are based on the actual changes in the code + and not based on whether the patch was planned or not. + For more information see the + <seealso marker="#version_scheme">Version Scheme</seealso> + section above. + </p> + </section> + + <section> <marker id="otp_17_0_app_versions"/> + <title>OTP 17.0 Application Versions</title> <p>The following list details the application versions that were part of OTP 17.0. If the normal part of an application version number compares diff --git a/system/doc/system_principles/xmlfiles.mk b/system/doc/system_principles/xmlfiles.mk index 77d6747414..353c2c7f7f 100644 --- a/system/doc/system_principles/xmlfiles.mk +++ b/system/doc/system_principles/xmlfiles.mk @@ -21,7 +21,8 @@ SYSTEM_PRINCIPLES_CHAPTER_FILES = \ system_principles.xml \ error_logging.xml \ upgrade.xml \ - versions.xml + versions.xml \ + misc.xml SYSTEM_PRINCIPLES_CHAPTER_GEN_FILES = \ create_target.xml diff --git a/system/doc/tutorial/Makefile b/system/doc/tutorial/Makefile index 70aba663b5..5867096fc8 100644 --- a/system/doc/tutorial/Makefile +++ b/system/doc/tutorial/Makefile @@ -110,6 +110,7 @@ debug opt: clean clean_docs: rm -rf $(HTMLDIR) + rm -rf $(XMLDIR) rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) rm -f errs core *~ diff --git a/xcomp/erl-xcomp-armv8-rpi3-linux-gnueabihf.conf b/xcomp/erl-xcomp-armv8-rpi3-linux-gnueabihf.conf new file mode 100644 index 0000000000..dffc4a4b56 --- /dev/null +++ b/xcomp/erl-xcomp-armv8-rpi3-linux-gnueabihf.conf @@ -0,0 +1,276 @@ +## -*-shell-script-*- +## +## %CopyrightBegin% +## +## Copyright Ericsson AB 2009-2018. All Rights Reserved. +## +## Licensed under the Apache License, Version 2.0 (the "License"); +## you may not use this file except in compliance with the License. +## You may obtain a copy of the License at +## +## http://www.apache.org/licenses/LICENSE-2.0 +## +## Unless required by applicable law or agreed to in writing, software +## distributed under the License is distributed on an "AS IS" BASIS, +## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +## See the License for the specific language governing permissions and +## limitations under the License. +## +## %CopyrightEnd% +## +## File: erl-xcomp-armv8-rpi3-linux-gnueabihf.conf +## Author: Péter Dimitrov +## Tested: macOS High Sierra 10.13.2 / +## Raspberry Pi 3 Model B Rev 1.2 +## +## ----------------------------------------------------------------------------- +## When cross compiling Erlang/OTP using `otp_build', copy this file and set +## the variables needed below. Then pass the path to the copy of this file as +## an argument to `otp_build' in the configure stage: +## `otp_build configure --xcomp-conf=<FILE>' +## ----------------------------------------------------------------------------- + +## Note that you cannot define arbitrary variables in a cross compilation +## configuration file. Only the ones listed below will be guaranteed to be +## visible throughout the whole execution of all `configure' scripts. Other +## variables needs to be defined as arguments to `configure' or exported in +## the environment. + +## -- Variables for `otp_build' Only ------------------------------------------- + +## Variables in this section are only used, when configuring Erlang/OTP for +## cross compilation using `$ERL_TOP/otp_build configure'. + +## *NOTE*! These variables currently have *no* effect if you configure using +## the `configure' script directly. + +# * `erl_xcomp_build' - The build system used. This value will be passed as +# `--build=$erl_xcomp_build' argument to the `configure' script. It does +# not have to be a full `CPU-VENDOR-OS' triplet, but can be. The full +# `CPU-VENDOR-OS' triplet will be created by +# `$ERL_TOP/erts/autoconf/config.sub $erl_xcomp_build'. If set to `guess', +# the build system will be guessed using +# `$ERL_TOP/erts/autoconf/config.guess'. +erl_xcomp_build=guess + +# * `erl_xcomp_host' - Cross host/target system to build for. This value will +# be passed as `--host=$erl_xcomp_host' argument to the `configure' script. +# It does not have to be a full `CPU-VENDOR-OS' triplet, but can be. The +# full `CPU-VENDOR-OS' triplet will be created by +# `$ERL_TOP/erts/autoconf/config.sub $erl_xcomp_host'. +erl_xcomp_host=armv8-rpi3-linux-gnueabihf + +# * `erl_xcomp_configure_flags' - Extra configure flags to pass to the +# `configure' script. +#erl_xcomp_configure_flags= + +## -- Cross Compiler and Other Tools ------------------------------------------- + +## If the cross compilation tools are prefixed by `<HOST>-' you probably do +## not need to set these variables (where `<HOST>' is what has been passed as +## `--host=<HOST>' argument to `configure'). + +## All variables in this section can also be used when native compiling. + +# * `CC' - C compiler. +CC=armv8-rpi3-linux-gnueabihf-gcc + +# * `CFLAGS' - C compiler flags. +#CFLAGS= + +# * `STATIC_CFLAGS' - Static C compiler flags. +#STATIC_CFLAGS= + +# * `CFLAG_RUNTIME_LIBRARY_PATH' - This flag should set runtime library +# search path for the shared libraries. Note that this actually is a +# linker flag, but it needs to be passed via the compiler. +#CFLAG_RUNTIME_LIBRARY_PATH= + +# * `CPP' - C pre-processor. +CPP=armv8-rpi3-linux-gnueabihf-cpp + +# * `CPPFLAGS' - C pre-processor flags. +#CPPFLAGS= + +# * `CXX' - C++ compiler. +CXX=armv8-rpi3-linux-gnueabihf-g++ + +# * `CXXFLAGS' - C++ compiler flags. +#CXXFLAGS= + +# * `LD' - Linker. +LD=armv8-rpi3-linux-gnueabihf-ld + +# * `LDFLAGS' - Linker flags. +LDFLAGS="-L$RPI_SYSROOT/lib" + +# * `LIBS' - Libraries. +#LIBS= + +## -- *D*ynamic *E*rlang *D*river Linking -- + +## *NOTE*! Either set all or none of the `DED_LD*' variables. + +# * `DED_LD' - Linker for Dynamically loaded Erlang Drivers. +#DED_LD= + +# * `DED_LDFLAGS' - Linker flags to use with `DED_LD'. +#DED_LDFLAGS= + +# * `DED_LD_FLAG_RUNTIME_LIBRARY_PATH' - This flag should set runtime library +# search path for shared libraries when linking with `DED_LD'. +#DED_LD_FLAG_RUNTIME_LIBRARY_PATH= + +## -- Large File Support -- + +## *NOTE*! Either set all or none of the `LFS_*' variables. + +# * `LFS_CFLAGS' - Large file support C compiler flags. +#LFS_CFLAGS= + +# * `LFS_LDFLAGS' - Large file support linker flags. +#LFS_LDFLAGS= + +# * `LFS_LIBS' - Large file support libraries. +#LFS_LIBS= + +## -- Other Tools -- + +# * `RANLIB' - `ranlib' archive index tool. +RANLIB=armv8-rpi3-linux-gnueabihf-ranlib + +# * `AR' - `ar' archiving tool. +AR=armv8-rpi3-linux-gnueabihf-ar + +# * `GETCONF' - `getconf' system configuration inspection tool. `getconf' is +# currently used for finding out large file support flags to use, and +# on Linux systems for finding out if we have an NPTL thread library or +# not. +#GETCONF= + +## -- Cross System Root Locations ---------------------------------------------- + +# * `erl_xcomp_sysroot' - The absolute path to the system root of the cross +# compilation environment. Currently, the `crypto', `odbc', `ssh' and +# `ssl' applications need the system root. These applications will be +# skipped if the system root has not been set. The system root might be +# needed for other things too. If this is the case and the system root +# has not been set, `configure' will fail and request you to set it. +erl_xcomp_sysroot="$RPI_SYSROOT" + +# * `erl_xcomp_isysroot' - The absolute path to the system root for includes +# of the cross compilation environment. If not set, this value defaults +# to `$erl_xcomp_sysroot', i.e., only set this value if the include system +# root path is not the same as the system root path. +#erl_xcomp_isysroot= + +## -- Optional Feature, and Bug Tests ------------------------------------------ + +## These tests cannot (always) be done automatically when cross compiling. You +## usually do not need to set these variables. Only set these if you really +## know what you are doing. + +## Note that some of these values will override results of tests performed +## by `configure', and some will not be used until `configure' is sure that +## it cannot figure the result out. + +## The `configure' script will issue a warning when a default value is used. +## When a variable has been set, no warning will be issued. + +# * `erl_xcomp_after_morecore_hook' - `yes|no'. Defaults to `no'. If `yes', +# the target system must have a working `__after_morecore_hook' that can be +# used for tracking used `malloc()' implementations core memory usage. +# This is currently only used by unsupported features. +#erl_xcomp_after_morecore_hook= + +# * `erl_xcomp_bigendian' - `yes|no'. No default. If `yes', the target system +# must be big endian. If `no', little endian. This can often be +# automatically detected, but not always. If not automatically detected, +# `configure' will fail unless this variable is set. Since no default +# value is used, `configure' will try to figure this out automatically. +#erl_xcomp_bigendian= + +# * `erl_xcomp_double_middle` - `yes|no`. No default. If `yes`, the +# target system must have doubles in "middle-endian" format. If +# `no`, it has "regular" endianness. This can often be automatically +# detected, but not always. If not automatically detected, +# `configure` will fail unless this variable is set. Since no +# default value is used, `configure` will try to figure this out +# automatically. +#erl_xcomp_double_middle_endian + +# * `erl_xcomp_clock_gettime_cpu_time' - `yes|no'. Defaults to `no'. If `yes', +# the target system must have a working `clock_gettime()' implementation +# that can be used for retrieving process CPU time. +#erl_xcomp_clock_gettime_cpu_time= + +# * `erl_xcomp_getaddrinfo' - `yes|no'. Defaults to `no'. If `yes', the target +# system must have a working `getaddrinfo()' implementation that can +# handle both IPv4 and IPv6. +#erl_xcomp_getaddrinfo= + +# * `erl_xcomp_gethrvtime_procfs_ioctl' - `yes|no'. Defaults to `no'. If `yes', +# the target system must have a working `gethrvtime()' implementation and +# is used with procfs `ioctl()'. +#erl_xcomp_gethrvtime_procfs_ioctl= + +# * `erl_xcomp_dlsym_brk_wrappers' - `yes|no'. Defaults to `no'. If `yes', the +# target system must have a working `dlsym(RTLD_NEXT, <S>)' implementation +# that can be used on `brk' and `sbrk' symbols used by the `malloc()' +# implementation in use, and by this track the `malloc()' implementations +# core memory usage. This is currently only used by unsupported features. +#erl_xcomp_dlsym_brk_wrappers= + +# * `erl_xcomp_kqueue' - `yes|no'. Defaults to `no'. If `yes', the target +# system must have a working `kqueue()' implementation that returns a file +# descriptor which can be used by `poll()' and/or `select()'. If `no' and +# the target system has not got `epoll()' or `/dev/poll', the kernel-poll +# feature will be disabled. +#erl_xcomp_kqueue= + +# * `erl_xcomp_linux_clock_gettime_correction' - `yes|no'. Defaults to `yes' on +# Linux; otherwise, `no'. If `yes', `clock_gettime(CLOCK_MONOTONIC, _)' on +# the target system must work. This variable is recommended to be set to +# `no' on Linux systems with kernel versions less than 2.6. +#erl_xcomp_linux_clock_gettime_correction= + +# * `erl_xcomp_linux_nptl' - `yes|no'. Defaults to `yes' on Linux; otherwise, +# `no'. If `yes', the target system must have NPTL (Native POSIX Thread +# Library). Older Linux systems have LinuxThreads instead of NPTL (Linux +# kernel versions typically less than 2.6). +#erl_xcomp_linux_nptl= + +# * `erl_xcomp_linux_usable_sigaltstack' - `yes|no'. Defaults to `yes' on Linux; +# otherwise, `no'. If `yes', `sigaltstack()' must be usable on the target +# system. `sigaltstack()' on Linux kernel versions less than 2.4 are +# broken. +#erl_xcomp_linux_usable_sigaltstack= + +# * `erl_xcomp_linux_usable_sigusrx' - `yes|no'. Defaults to `yes'. If `yes', +# the `SIGUSR1' and `SIGUSR2' signals must be usable by the ERTS. Old +# LinuxThreads thread libraries (Linux kernel versions typically less than +# 2.2) used these signals and made them unusable by the ERTS. +#erl_xcomp_linux_usable_sigusrx= + +# * `erl_xcomp_poll' - `yes|no'. Defaults to `no' on Darwin/MacOSX; otherwise, +# `yes'. If `yes', the target system must have a working `poll()' +# implementation that also can handle devices. If `no', `select()' will be +# used instead of `poll()'. +#erl_xcomp_poll= + +# * `erl_xcomp_putenv_copy' - `yes|no'. Defaults to `no'. If `yes', the target +# system must have a `putenv()' implementation that stores a copy of the +# key/value pair. +#erl_xcomp_putenv_copy= + +# * `erl_xcomp_reliable_fpe' - `yes|no'. Defaults to `no'. If `yes', the target +# system must have reliable floating point exceptions. +#erl_xcomp_reliable_fpe= + +# * `erl_xcomp_posix_memalign' - `yes|no'. Defaults to `yes' if `posix_memalign' +# system call exists; otherwise `no'. If `yes', the target system must have a +# `posix_memalign' implementation that accepts larger than page size +# alignment. +#erl_xcomp_posix_memalign= + +## ----------------------------------------------------------------------------- |
